The conventional wisdom in AI cost management: use smaller, cheaper models for routine tasks and save money. New research published on arXiv challenges this directly - the paper's core claim is that frontier models (the most capable, most expensive options like GPT-4o and Claude 3 Opus) are actually the most cost-efficient once you measure properly.
The problem with the "cheaper models save money" logic is that it measures cost per token (the chunks of text the model processes) rather than cost per useful completed task. A model that fails 30% of the time, or produces outputs requiring significant correction, has an effective cost per successful completion that multiplies fast. Add in the developer hours spent on retry logic, prompt engineering to coax acceptable results from weaker models, and human review of borderline outputs, and the economics of "go cheap" often don't hold.
This doesn't mean every task should go to the most expensive model available. For simple, high-volume, well-defined work - classifying support tickets, extracting structured data from clean inputs, yes/no decisions on clear criteria - capable mid-tier models perform without meaningful quality loss. The research points to complex reasoning tasks as the inflection point: multi-step problems, ambiguous instructions, nuanced judgment calls. That's where frontier models justify the price gap.
The practical implication for practitioners managing API costs: stop optimizing on token price and start measuring cost per successful completion. The task that takes 4 attempts on a cheaper model and 1 on a frontier model costs more with the "cheaper" option. Run the comparison on your actual workload before committing to a routing strategy.