The difference in one table
The categories below describe typical behaviour of each kind of tool, not any single vendor. Check any specific product against your own requirements.
| Question | AI gateway | LLM router | NeoAI |
|---|---|---|---|
| What does it decide? | Where a request goes, plus keys, limits and logging | Which model answers each request | Model tier, reasoning effort, context and budget for each sub-step |
| Unit of decision | Request | Request | Sub-step of a task |
| Learns from your company's work | Not typically | Typically learns from general preference or benchmark data | Yes: recipes built from your accepted outputs |
| Reduces retries and rework | No | Not directly | Yes: recipes cut attempts per task Pilot: to measure |
| Handles a cheaper model failing | Fallback on errors | Fallback or re-route | Escalates the step on a failed confidence or quality check |
| Where it sits | In front of your models | In front of your models, or inside the gateway | Behind your existing gateway, in your VPC |
| Changes to your workflow | Endpoint change | Endpoint change | One endpoint change, no workflow change |
What a gateway or router does well
A gateway centralises access: one place for credentials, rate limits, caching, logging and provider failover. A router adds a decision about which model should answer each request. Both are useful, and NeoAI is designed to leave them in place. Your apps keep calling the gateway you already run.
What a per-request decision cannot see
A single request is often one step in a longer task. Choosing a model per request cannot change how many attempts the task takes, which context each step needs, or how much reasoning effort a step deserves. Those are the places where rework and overspend come from: the same task gets different quality across teams, several attempts per task, and a frontier model with deep reasoning used on simple steps on every attempt.
NeoAI works at the step level. It breaks a task into sub-steps, scores how hard each is, and applies a recipe learned from your own accepted outputs. Read how recipes and delegation work.
How the two fit together
- Your apps (chat, agents, internal tools) call your AI gateway as today.
- Your gateway is unchanged and still handles credentials, limits and logging.
- NeoAI runs behind it in your VPC, with a recipe store you own, an evaluation harness and an audit log.
- Your model providers receive each step on the small, mid or frontier tier it was assigned.
When you probably do not need NeoAI
- Your monthly LLM spend is well under $20k, where an audit costs more attention than it saves.
- You run one simple, uniform workload that a single small model already handles at your quality bar.
- You are still choosing providers. Start with a gateway; add NeoAI once usage is steady.
NeoAI is built for teams spending $20k to $500k a month on LLMs, and for enterprise AI platform teams who need answers for the CEO, CTO, CIO and CFO. The way to find out is a free audit on your own usage.