# NeoAI vs LLM Model Routers: Single-request routing versus step-level workflow delegation
Published: 2026-10-25 | NeoAI (https://recipes.neoai.solutions/vs/martian/)

> Per-request model routers like Martian and RouteLLM predict whether an entire prompt is hard or easy, then route the full request to either a small or frontier model. NeoAI operates at the sub-step level: it decomposes tasks into discrete steps so routine work runs on small models while reserving frontier reasoning for complex sub-steps.

## The Fundamental Problem with Per-Request Routers

First-generation LLM routers operate on a simple premise: use a lightweight classifier to decide whether a user prompt is "simple" (send to a 7B or 8B model) or "hard" (send to GPT-4o or Claude Sonnet).

While this approach works for conversational customer support queries, it fails on enterprise multi-step workloads. Real-world business tasks—such as processing insurance claims, evaluating financial statements, or analyzing software bugs—are inherently heterogeneous.

In a typical document processing task, 80% to 90% of the token volume is spent extracting text, structuring JSON schemas, and formatting summaries. Only 10% to 20% involves complex judgment. A per-request router must classify the whole task as "hard", forcing you to pay frontier rates for all 4,000 tokens.

## How NeoAI Step-Level Delegation Solves This

NeoAI operates inside your private VPC. Instead of treating the request as a monolithic black box, NeoAI splits the workload into an executable directed acyclic graph (DAG):

- Extraction steps run on small, high-throughput models at a fraction of the cost.

- Evaluation steps run on mid-tier models constrained by precise rubrics.

- Frontier reasoning is engaged only for ambiguous edge cases or when automated step verification checks fail.

On standard enterprise workloads, this fine-grained delegation reduces token spend by 55% to 85% compared to monolithic frontier execution, while matching or exceeding output quality.

## Beyond Routing: Rework and Retries

Model routers are blind to what happens after an answer is delivered. If an output is subtly wrong, vague, or missing citations, a human staff member spends 20 minutes redlining the text or re-running prompts.

NeoAI captures these human corrections and distils them into automated quality gates. By cutting both the cost per attempt and the number of attempts required to reach an accepted output, NeoAI scales company-wide throughput with existing headcount.

## Frequently Asked Questions

### Why isn't a per-request router enough for complex workflows?

Most enterprise workflows are mixed: a 10-page document analysis has simple data extraction (cheap) and difficult legal reasoning (expensive). A per-request router must classify the whole task as hard and route 100% of the tokens to a frontier model. NeoAI decomposes the task so each sub-step runs on the cheapest sufficient tier.

### Does NeoAI require training router classifiers?

No. Model routers often require fine-tuning custom routing classifiers or training embedding models on historical data. NeoAI uses explicit, interpretable task recipes with deterministic quality gates (schema tests, citation checks, score bounds) distilled directly from your reviewers' accepted work.

### How does recipe learning carry over between tasks?

Model routers treat every prompt in isolation. NeoAI recipes are constructed from modular blocks (rubric scoring, evidence extraction, calibration). When you launch a related task, it reuses the proven blocks and begins with higher quality from day one.

## Canonical URL
https://recipes.neoai.solutions/vs/martian/