arbiter-cli

arbiter-cli

Cut LLM API costs 69% with one line of code. Smart routing proxy that sends each request to the cheapest model capable of handling it.

Quick Start

# Interactive chat (like Claude CLI, but 69% cheaper)
npx arbiter-cli chat

# AI coding agent (reads files, writes code, runs commands)
npx arbiter-cli code "add error handling to utils.py"

# Set up in your project (zero code changes to your app)
npx arbiter-cli init

# Check your savings
npx arbiter-cli stats

What it does

Arbiter routes every LLM request to the cheapest model that can handle it:

• Simple questions → Gemini Flash / GPT-4o Mini (95% cheaper)
• Medium code tasks → Qwen / Mistral (90% cheaper)
• Complex reasoning → Claude Sonnet 4 / GPT-4o (full quality)

You get the same quality. You pay 69% less on average.

Setup Options

Option 1: Interactive Chat

npx arbiter-cli chat

Chat like you would in Claude CLI. Each response shows which model was picked and how much you saved.

⚡ Arbiter Chat

› What is the capital of France?
  Paris.
  ↳ gemini-2.5-flash · saved <$0.001 (95%)

› Design a CRDT for collaborative editing
  Here's an approach using operation-based CRDTs...
  ↳ claude-sonnet-4.6 · saved $0.00 (0%) — frontier needed

Option 2: Coding Agent

npx arbiter-cli code "fix the bug in main.py"
npx arbiter-cli code   # interactive mode

Reads files, writes code, runs commands. Routes cheap for simple file ops, frontier for architecture decisions.

Option 3: Drop-in Proxy (for your existing code)

npx arbiter-cli init

This adds OPENAI_BASE_URL to your .env. Your existing OpenAI SDK code routes through Arbiter automatically — no code changes.

from openai import OpenAI

# Works unchanged — Arbiter routes behind the scenes
client = OpenAI()  # Reads OPENAI_BASE_URL from .env
response = client.chat.completions.create(
    model="gpt-4o",  # Arbiter overrides intelligently
    messages=[{"role": "user", "content": "What is 2+2?"}]
)
# → Routed to Gemini Flash, saved 95%

CLI Commands

Command	Description
chat	Interactive chat with smart routing
chat --fast	Prefer low-latency models
chat --model claude	Force a specific model
code	AI coding agent (interactive)
code "task"	One-shot coding task
init	Add Arbiter to current project
status	Check proxy connection
stats	View cost savings

Chat Commands

Command	Description
/stats	Session cost breakdown
/model claude	Switch model (claude, gpt4o, flash, haiku, fable, auto)
/good or /bad	Rate response (improves routing)
/copy	Copy last response to clipboard
/save name	Save conversation
/load name	Load conversation
"""	Start/end multi-line input
quit	Exit

How It Works

1. Classify — Each request is analyzed for task type (code, reasoning, analysis, creative, etc.) and complexity (simple/medium/complex) in <1ms
2. Route — Performance matrix picks the cheapest model that meets the quality bar
3. Quality Gate — If cheap model gives garbage, transparently retries on frontier
4. Cache — Identical requests return instantly at $0
5. Compress — Non-frontier responses use concise prompts (fewer output tokens)

Models Available

Model	Best for	Cost
Claude Sonnet 4	Complex reasoning, analysis	$$$
Claude Fable 5	Autonomous coding agents	$$$$
GPT-4o	Complex code, multi-step	$$$
Gemini 2.5 Flash	Simple Q&A, classification	$
GPT-4o Mini	Simple tasks, extraction	$
Qwen 2.5 72B	Code generation, math	$
Llama 3.3 70B	General tasks	$
Mistral Large	Code review, analysis	$$
Claude 3.5 Haiku	Fast responses	$$

Requirements

• Node.js 18+
• An OpenRouter API key (one key, all models)

Set your key:

export OPENROUTER_API_KEY=sk-or-v1-...
# or add to .env in your project directory

Savings Breakdown

From real testing across 90 varied requests:

Traffic Type	Routed To	Savings
Simple Q&A (40%)	Gemini Flash	95%
Classification (15%)	Gemini Flash	95%
Code tasks (25%)	Qwen / GPT-4o	50-93%
Complex reasoning (10%)	Claude Sonnet 4	0%
Analysis (10%)	Claude Sonnet 4	0%
Average	Mixed	69%

Links

• Landing page: https://arbiter-ai.com
• API docs: https://app.arbiter-ai.com/docs
• NPM: https://www.npmjs.com/package/arbiter-cli

License

MIT