Frontier or open model? How to choose without wasting money
A practical decision guide for teams: when a paid frontier API is the right call, when open weights win, and why most production systems should use both.
The honest answer for almost every production system in 2026 is both — a cheap model handling the bulk of traffic and an expensive one handling the hard cases. But you still need to decide which is which.
Use a frontier API when
- The task needs the best available reasoning and a wrong answer is expensive — pricing, contracts, medical or legal text.
- Volume is low enough that per-token cost is not your dominant expense.
- You need multimodal input, long context and tool use to all work reliably together, today, with no engineering effort.
- You have no GPU budget and no appetite for running infrastructure.
Use open weights when
- Volume is high and the task is repetitive. Classification, extraction, tagging, routing. This is where API bills quietly become the largest line item.
- The data cannot leave your building. Client records, financials, anything under a confidentiality clause.
- You need it to keep working regardless. Availability can change for policy reasons — as it did in June 2026. Weights on your own disk cannot be revoked.
- You are shipping a product someone else runs. On-premise delivery, offline apps, embedded devices.
The pattern that actually works
- Route everything to the cheapest model that passes your evaluation set.
- Detect low confidence — short answers, hedging, tool-call failures, explicit "I'm not sure".
- Escalate only those requests to the frontier model.
- Log both, and re-check the split monthly. Models get cheaper; your routing should follow.
Teams that do this typically find the frontier model handles well under a fifth of traffic while carrying most of the perceived quality.
What people get wrong
Choosing on benchmarks. Public leaderboards measure someone else's workload. Twenty real examples from your own business, scored by you, are worth more than any leaderboard position.
Ignoring context cost. A model with a one-million-token window will happily let you send a million tokens. Every one of them is billable, on every call.
Forgetting the hardware bill. "Free" open weights still need a GPU, power and someone to keep it running. Below a certain volume, an API is genuinely cheaper — do the arithmetic before the ideology.
Need the hardware side priced? Harare Tech Supply stocks laptops, GPUs and storage in Harare.
