[R] Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost

Background

Token-based billing is causing my company to reevaluate small language models. I came across this paper that shows SLM supervised fine-tuning on traces from orchestration of frontier models can be nearly as performant and much cheaper.

Key Findings

The paper demonstrates that compiling agentic workflows into LLM weights through supervised fine-tuning on orchestration traces yields near-frontier quality at approximately two orders of magnitude less cost.

Practical Considerations

Has anyone tried this in the real world? The approach involves:

• Using traces from orchestration of frontier models
• Applying supervised fine-tuning to small language models (SLMs)
• Achieving comparable performance at significantly reduced operational costs

Technical Details

The method essentially distills the reasoning and decision-making patterns of larger, more expensive models into smaller, more efficient ones through careful training on execution traces.