# NeoAI vs LiteLLM: API proxy standardization versus in-VPC workflow step delegation
Published: 2026-10-25 | NeoAI (https://recipes.neoai.solutions/vs/litellm/)

> LiteLLM provides a fast, lightweight proxy that unifies 100+ LLM APIs behind a single OpenAI-compatible interface with load balancing. NeoAI operates in your VPC behind LiteLLM to decompose tasks into discrete steps, route each sub-step across model tiers, and automate quality gates using recipes learned from your accepted runs.

## How NeoAI and LiteLLM Work Together

LiteLLM has become an industry standard for unifying hundreds of LLM APIs behind an OpenAI-compatible interface. It solves the critical developer friction of managing heterogeneous SDKs, provider-specific payload schemas, and API rate limits.

Where LiteLLM ends is where NeoAI begins. LiteLLM is fundamentally an API proxy: when your application sends a 4,000-token prompt asking to analyze an employment contract or summarize a financial filing, LiteLLM forwards that complete request to one model.

Because frontier models cost roughly 20x small models per token, burning a model like Claude 3.5 Sonnet or GPT-4o on every token of a long document is economically unsustainable at scale.

## Decomposing Tasks into Multi-Tier DAGs

By placing NeoAI behind your LiteLLM proxy, incoming requests are analyzed by a lightweight planning agent inside your VPC. NeoAI splits the work into a directed graph of sub-steps:

- Data Extraction: Parsing tables, dates, and names runs on small, fast models (e.g. Claude Haiku or Gemini Flash-Lite).

- Rubric Scoring: Bounded evaluation runs on mid-tier models with calibrated temperature.

- Complex Synthesis: Edge-case reasoning and final judgment are delegated to frontier models only when necessary.

Furthermore, NeoAI distils your human reviewers' edit history into automated assertion checks. If a small model's sub-step output fails an assertion, only that single sub-step escalates to a higher tier.

## Commercial Alignment

LiteLLM is open source with optional enterprise support. NeoAI pairs seamlessly with your self-hosted LiteLLM deployment under a pure gainshare model: if our step-level delegation does not measurably reduce your net AI cost against your baseline evaluation set, you pay nothing.

## Frequently Asked Questions

### Can I use LiteLLM and NeoAI together?

Yes. LiteLLM serves as your OpenAI-compatible interface and connection manager to providers (Bedrock, Vertex, OpenAI, Anthropic). NeoAI sits as the upstream target behind LiteLLM, handling task decomposition and quality gating in your VPC.

### Doesn't LiteLLM already have a router?

LiteLLM has a router for request-level load balancing, cooldowns, and fallbacks (e.g. if Azure OpenAI is down, fallback to Anthropic). It routes the entire request to one model. NeoAI decomposes a single task into multiple sub-steps with varying model sizes and reasoning budgets.

### How does NeoAI handle model keys when using LiteLLM?

NeoAI can either forward its delegated sub-step calls back through LiteLLM's proxy, or call model endpoints directly using your existing credentials. Your model provider relationships and volume discounts remain untouched.

## Canonical URL
https://recipes.neoai.solutions/vs/litellm/