Choosing the right model
Picking a model is balancing four axes - quality, cost, latency, context size - for your specific task, not finding 'the best model'.
In plain terms
You don't hire a Michelin chef to butter toast. Summarizing tickets? A small cheap model is fine. Writing legal analysis? Pay for the frontier model. Most real products mix models: cheap ones for easy steps, expensive ones where quality is felt.
Why it matters
Model prices differ by 100x or more. Grabbing the biggest model for everything makes apps slow and unaffordable; grabbing the cheapest makes them dumb. Deliberate routing is where AI engineers earn their salary.
How it works
1) Write 20-50 test cases from your real task (-> evals). 2) Run the candidates on them. 3) Compare quality first, then check price/latency budgets. Families roughly tier as: frontier (Claude Opus/Sonnet, GPT-4-class, Gemini Pro) -> mid (Sonnet/GPT-4o-mini-class, Gemini Flash) -> small/fast (Haiku, open 8B models). Re-test quarterly; the market moves fast.
When you use it
At project start, whenever bills or latency hurt, and whenever a new model generation ships.
Common mistakes
- Choosing by vibes or Twitter hype instead of your own eval set.
- Optimizing cost before the product works - get it good, then get it cheap.
- Hardcoding one model everywhere so upgrades require a codebase-wide hunt.
Best practices
- Put model name, temperature and prompt version in config, not code.
- Route: classify request difficulty and send easy ones to a cheap model.
- Track cost per feature, not just total spend.
Try it yourself
Take one task (e.g. 'summarize this email thread in 2 lines'), run it on a frontier, mid and small model, and honestly judge: which is the cheapest one that's good enough?
Resources
- Artificial Analysis Independent quality/price/speed comparisons across all major models.