When NOT to Use a Model Router (Yes, Really)

Skip routing if every request needs the best model

Some workloads have zero routable slack. A legal-contract reviewer who compares clauses needs adversarial reasoning on every call. A research assistant generating Python for drug-interaction simulations needs frontier quality every time. If the answer to "can a cheaper model do this?" is "no, the cost of a wrong answer is too high," pin the request to the best model and skip the classifier.

You'll still benefit from a gateway (failover, pricing consolidation, observability), but the "route to cheaper model" value prop doesn't apply to you. That's fine. Use model="claude-opus-4.7" through us and get the observability without the routing.

Skip routing if your entire workload is one task type

The cheapest model that can handle your one task is already your best answer. If you're running a sentiment-classification service and 100% of calls are "is this tweet positive or negative," the correct model is Llama 3.1 8B and the correct number of routing decisions is zero. A classifier on top of that is pure overhead.

The test: if the cost-per-task-type chart would have one bar on it, you don't need routing. Pin the model, ship it, move on.

Skip routing if your volume is trivial

Some math. If you make 1,000 requests per month to GPT-5.4-mini, your total spend is maybe $5–$10. A router that optimizes that to $3 saves you $2–$7/month. The classifier cost, the latency overhead, and the operational burden of "one more service in the stack" aren't worth $7/month.

Our rough rule: if your model bill is under ~$200/mo, don't bother with routing. Use whatever single provider you like, bookmark this article, come back when your bill gets serious.

Skip routing if you're past product-market-fit but pre-scale

There's a middle-band company state: product works, usage is growing fast, but the model bill isn't yet the thing that keeps the CFO up. Adding a router here is a distraction. You should be shipping product, getting closer to customers, and planning for the scale event. You can always add routing in month 9 when the bill becomes visible. A month-late router captures 98% of the eventual savings.

The flip side: if you are at the scale event and you delayed routing by six months because you were busy, you left meaningful money on the table. The transition signal is "API cost is now more than 30% of our gross margin." When that hits, stop procrastinating.

Skip routing if you have strict single-provider data requirements

Certain regulated workloads have data-residency or provider-specific contracts that effectively require a single provider. If you've signed a BAA with a specific provider that says "this data never leaves our service," routing across providers violates the contract. Don't do that. Route within one provider's model family instead (we do support that — "route within Anthropic only"), or skip routing entirely.

Skip routing if your p99 latency budget is under 150ms

A well-built router adds 30–80ms. For most workloads, that's lost in the noise of a 500–2000ms model response. But if your product is an ultra-low-latency autocomplete or a realtime voice pipeline where every millisecond counts, you can't afford the classifier round-trip. Pin a single model, optimize the hot path, skip routing.

(Side note: for realtime voice specifically, the realtime APIs aren't yet fully routable anyway — models have different voice capabilities and session semantics. This is on our roadmap but it's not a problem we've solved yet.)

Skip routing if your team can't handle non-determinism

This one surprises people. Routing introduces variability: the same prompt may go to Model A on Monday and Model B on Tuesday if the classifier's confidence shifts. The outputs are usually similar; they're not identical.

Some teams have evaluation pipelines, regression tests, or reproducibility requirements that assume a fixed model. If your test suite fails when the model name changes, routing makes your CI flaky until you fix the test suite. The fix is real but it's engineering time. If you don't have the cycles, pin the model for now and add routing later.

Skip routing if you're the one who wrote it

If you're a platform team at a large company, you probably already have a router. You built it. You've been running it for a year. Should you switch to ours?

Usually no. The right answer is "keep yours, we'll talk when you want to stop maintaining it." Our conversion story for these teams isn't "rip out what you have." It's "someone on your team is pagerduty-on-call for routing this weekend; that could be us." That conversion happens when the internal platform owner moves on, or when the router becomes a second full-time project. Until then, your router is fine. Keep shipping.

When you should use a router

For completeness, because the post above sounds like a no-fly-zone: you should route when your workload has meaningful heterogeneity, your volume is non-trivial, and your latency budget allows for it. In practice, that's most production AI products north of a couple thousand users or a couple hundred dollars a month in model spend. The complete guide to LLM routers covers the positive case in depth.

The short version: routing is a cost and quality tool. Like every tool, it has a shape. If your problem doesn't fit the shape, use a different tool.