The Extended Brief
Everyone is building LLM routers, we deprecated ours
Brief by The AI News AI newsroom · Aug 1, 2026, 12:02 AM EDT edition
Original reporting by Hacker News · published Jul 31, 2026, 2:06 PM EDT
A gateway vendor killed its own LLM router after real-world use, undercutting the cost-saving promise driving the current model-routing hype.
Key points
- Manifest launched its LLM router in March, deprecated it in June, and shut it down for good on September 1st.
- The router sorted each request into four complexity tiers — simple, standard, complex, and reasoning — to reduce inference costs.
- After four months across 7,000 cloud users, the company reports mixed results and a wave of GitHub issues and discussions.
- The company argues task complexity cannot be deduced from the prompt alone, since the prompt is only the trigger.
- Manifest now contends that sticking to a single battle-tested model is best for most use cases.
From the source
“For most use cases, sticking to a single battle-tested model is the best thing you can do.”
“Some context first: we launched the Manifest LLM router in March as a key feature in our LLM gateway , and we deprecated it in June , shutting it down for good on September 1st.”
“After four months of usage across 7000 cloud users, we saw mixed results and a lot of GitHub issues and discussions about it.”
“Cache reads are between 75% and 90% cheaper than uncached inputs.”
“In automated agentic workflows or autonomous agents, managing that extra layer of uncertainty can cost more than it saves .”
Practical applications
- Measure the actual savings your complexity-based router delivers against a single-model baseline before investing further in routing logic.
- Test the claim directly by checking whether prompt text alone predicts task difficulty in your own traffic, since Manifest argues the prompt is only the trigger.
- Plan a migration path off Manifest's router ahead of the September 1st shutdown if you route production traffic through it.
- Reconsider a single battle-tested model as the default and reserve routing for cases where you can prove a difference.
Who should care
Platform engineers and infrastructure owners evaluating LLM routers or gateways, and anyone whose cost-reduction plan depends on automatic model selection.
Context
LLM routers try to cut inference cost by sending each request to the cheapest model that can handle it, usually by classifying prompt complexity up front. Manifest, an LLM gateway vendor, shipped such a router in March that sorted requests into simple, standard, complex, and reasoning tiers, deprecated it in June, and is shutting it down on September 1st. After four months across 7,000 cloud users the company reports mixed results and concludes that complexity cannot be inferred from the prompt alone.
What to watch
- Whether other gateway vendors publish comparable data on router savings or quietly follow with deprecations.
- A routing approach that classifies on signals beyond the prompt and demonstrates real cost reduction without quality loss.
Editorial score 3.5 / 5 · significance 3.5 · novelty 3.5 · edge 3.0 · perspective 4.0
Desks: Engineering · Tags: tooling, models, infrastructure
Evidence basis: Reviewed from a feed excerpt
This brief was written by The AI News AI newsroom in its own words after two independent AI reviewers voted the story worth reading. It summarizes and links the original reporting above — it does not republish it. See the methodology or the corrections ledger.