Quick-Start Action List
Prioritized by impact and ease of implementation. Start at the top and work down — the first five actions are detailed in the strategy deep-dives below.
| # | Action | Savings | Effort |
|---|
| 1 | Switch to batch API for non-real-time | 50% | Low |
| 2 | Enable prompt caching (structure prompts correctly) | 75–90% input | Low |
| 3 | Downsize model (test quality with cheaper) | 50–95% | Medium |
| 4 | Use effort=medium instead of high | 30–70% | Low |
| 5 | Set max_tokens appropriately | 10–30% | Low |
| 6 | Use Mistral/Gemini/Phi-4 for simple tasks | 80–95% vs frontier | Low |
| 7 | Implement model cascade (cheap → expensive) | 60–80% | High |
| 8 | PTU/Reserved for steady-state (Microsoft) | 20–40% vs on-demand | Medium |
| 9 | Compaction for long conversations (Claude) | 40–60% | Low |
| 10 | Self-host open-weight models (Llama, Qwen, Mistral, Phi, Gemma) | 50–90% at scale | High |
Cost Optimization Strategies
These techniques apply across all AI providers. Applying them in combination produces compound savings.
1. Model Right-Sizing (50–95% savings)
Start with the smallest model that meets quality requirements. Most production workloads do not need frontier-tier models. A practical rule: begin with the cheapest model and only move up if evals or user feedback show a quality gap.
📌 The Single Biggest Cost Decision You Will Make
Model choice is the most impactful cost lever — it determines your base rate before any other optimization applies. The difference between the cheapest and most expensive model in our catalog is 300× ($0.10/MTok for Ministral 3B vs $30/MTok for GPT-5.5 output). No amount of caching, batching, or token trimming can overcome choosing a model that is 300× more expensive than what you need.
Think of it like this: you need to get across town. You could hire a private helicopter, a luxury sedan, a standard taxi, or take the bus. They all get you there. If you just need to get across town reliably, the taxi works fine — the helicopter is not 100× better for this trip. AI models work the same way: a $30/MTok frontier model answering "Is this email spam?" produces the same yes/no answer as a $0.10/MTok model, but costs 300× more.
Why most teams overspend:
- Starting at the top — teams prototype with the most powerful model (GPT-5.5, Claude Opus) and never test whether a cheaper model works just as well
- Assuming bigger is always better — for structured tasks like classification, extraction, and formatting, smaller models often perform identically to flagship models
- Fear of quality loss — in practice, most teams that test cheaper models find the quality difference is undetectable for 70–80% of their workloads
- No evaluation process — without measuring quality (even informally), there is no evidence that the expensive model is actually better for your specific use case
The right-sizing process:
| Step | Action | What You Learn |
|---|
| 1 | Run your workload on the cheapest eligible model | Baseline cost and quality floor |
| 2 | Spot-check 20–50 outputs for quality | Whether the cheap model is "good enough" |
| 3 | If quality is acceptable — stop here, you are done | Most teams stop at this step |
| 4 | If quality falls short, move up one tier and repeat | The minimum model tier that meets your bar |
Model tiers and when to use them:
| Task Complexity | Recommended Tier | Example Models | Monthly Cost at 100M in / 20M out |
|---|
| Simple (classification, extraction) | Nano / Mini | Ministral 3B, Phi-4 mini, GPT-5.4 nano, Flash-Lite | $12 – $35 |
| Standard (chat, summarization) | Mid-tier | Mistral Small 4, Phi-4, GPT-5.4 mini | $16 – $165 |
| Complex (reasoning, coding) | Standard | GPT-5.4, Claude Sonnet 4.6, Mistral Large 3 | $80 – $550 |
| Frontier (research, deep analysis) | Flagship | GPT-5.5, Claude Opus 4.8 | $1,000 – $1,100 |
2. Batch Processing (50% savings)
Use batch for any non-real-time workload. All 5 providers offer ~50% off for batch. This is the easiest lever with zero quality tradeoff — the only cost is latency (typically 24h turnaround).
📌 The Easiest 50% You Will Ever Save
Batch processing is the simplest and most risk-free cost lever. The quality of the output is identical — the model processes the same prompt and produces the same answer. The only difference is timing: instead of getting the response in seconds, you get it in hours. In exchange, providers charge you half price.
Think of it like this: shipping a package. You can pay for next-day delivery or standard ground shipping. The package is the same, the destination is the same — you just wait a bit longer. Batch processing is the "ground shipping" of AI: same result, half the cost, but you cannot stand at the counter and watch it happen.
Is your workload batchable?
| Workload Type | Batchable? | Why |
|---|
| Chatbot responding to live users | No | Users are waiting — they need a response in seconds |
| Nightly document processing | Yes ✅ | Nobody is waiting — results needed by morning, not right now |
| Labeling a backlog of 10,000 records | Yes ✅ | Bulk operation, no real-time requirement |
| Generating weekly reports | Yes ✅ | Scheduled task — hours of delay are fine |
| Content moderation queue | Maybe | Depends on SLA — if 1-hour turnaround is acceptable, batch works |
| Real-time search assistant | No | User expects instant answers |
| Data extraction from uploaded files | Yes ✅ | User uploads a file and checks back later — no need for instant processing |
How easy is it to set up?
| Provider | Setup Difficulty | How It Works | Discount | Turnaround |
|---|
| OpenAI | Easy | Upload a JSONL file of requests, submit as a batch. Poll for completion or use a webhook. | 50% off | Up to 24 hours |
| Anthropic (Claude) | Easy | Message Batches API — submit up to 10,000 requests. Results returned in a single response object. | 50% off | Up to 24 hours |
| Google Gemini | Easy | BatchGenerateContent API. Also supports a "Flex" mode with shorter turnaround at similar discounts. | 50% off | Up to 24 hours |
| Mistral | Easy | Batch inference API — upload JSONL, retrieve results when ready. | 50% off | Up to 24 hours |
| Microsoft Foundry | Easy | Global Batch deployment — same OpenAI batch format, processed on Azure infrastructure. | 50% off | Up to 24 hours |
The key insight: most organizations have a mix of real-time and asynchronous workloads, but run everything through the real-time API because it is the path of least resistance. Simply identifying which workloads do not need instant responses and routing them to batch can cut your total bill by 25–40% with no quality impact and minimal engineering effort. If even 50% of your volume is batchable, that is a 25% reduction in your total spend — for what is often a one-day integration.
3. Prompt Caching (75–90% input savings)
Cache repeated prefixes — system prompts, policy documents, few-shot examples. Cached tokens are served at ~10% of base rate. Place static content at the beginning of prompts and keep prefixes identical to maximize hit rates.
📌 Why Prompt Caching Matters Most
Of all the cost levers, prompt caching often delivers the largest and most immediate savings — because most AI applications send the same instructions, policies, or reference documents with every request. You are paying full price for the same text over and over again. Caching eliminates that waste.
Think of it like this: every time your app calls the AI, it sends a message. The first part of that message — the rules, context, and background information — is usually identical across hundreds or thousands of requests. Without caching, the provider charges you full price to "read" that same text every single time. With caching, the provider remembers it and charges you a fraction of the cost.
How easy is it to set up?
| Provider | Setup Difficulty | How It Works | Discount |
|---|
| OpenAI | Automatic — zero effort | Any repeated prompt prefix over 1,024 tokens is cached automatically. No code changes needed. | 50% off cached input |
| Google Gemini | Automatic + optional manual | Implicit caching works automatically. Explicit caching (for precise control) takes ~5 lines of code. | 75% off cached input |
| Anthropic (Claude) | Small code change | Add a cache_control marker to tell the API what to remember. Takes 2–3 extra lines per request. Automatic caching also available on newer models. | 90% off cached reads (25% write premium on first call) |
| Mistral | Not available | No caching API. However, base rates are already so low that this gap rarely changes the cost winner. | — |
| Microsoft Foundry | Automatic (OpenAI models) | Same automatic caching as OpenAI for Foundry-hosted GPT models. Cached tokens are FREE on reserved capacity (PTU). | 50% off (pay-per-token) / free (PTU) |
| Microsoft Phi-4 | Not available | Phi-4 mini and Phi-4 do not support prompt caching. Like Mistral, they compete on low base rates instead. Self-hostable (MIT license) — infrastructure cost replaces per-token cost at scale. | — |
The real work is not in the code — it is in prompt design:
- Put static content first — system instructions, policies, reference documents, and few-shot examples should always appear at the beginning of the prompt
- Put dynamic content last — the user's specific question or input should come at the end
- Keep prefixes identical — even small differences (extra spaces, reworded instructions) break the cache and you pay full price again
- Reuse large documents — if you send the same 50-page policy document with every question, caching turns a $500/month cost into $50/month
For teams with well-structured prompts (separate system prompt + user message), enabling caching is a one-hour change. For applications where instructions and data are mixed together in every message, it requires a prompt restructuring effort — but the savings typically pay for that effort within the first week.
4. Reasoning Effort Control (30–70% savings)
Use the lowest reasoning effort that still solves the task. Models with effort parameters (OpenAI, Anthropic) can dramatically reduce hidden thinking tokens. Medium is often the best default — reserve high for genuinely hard cases.
📌 Understanding Reasoning Effort
Some AI models do not just produce an answer — they "think" before responding. This thinking uses tokens you pay for, even though you never see them in the output. A model set to "high" effort might spend 5,000 hidden tokens reasoning through a problem that "medium" effort solves in 500 tokens — with the same final answer. That is a 10× cost difference for identical results.
Think of it like this: asking a consultant to write a one-page summary. You can ask them to spend 30 minutes or 3 hours on it. For a routine summary, both versions look the same — but one costs 6× more. Reasoning effort control lets you tell the AI how hard to think before answering.
How easy is it to set up?
| Provider | Setup Difficulty | How It Works | Savings |
|---|
| OpenAI | One parameter | Add reasoning_effort: "medium" or "low" to your API call. Works on reasoning-capable models (o-series, GPT-5.5). One line of code. | 30–70% on reasoning tokens |
| Anthropic (Claude) | One parameter | Set budget_tokens to cap thinking, or use the effort parameter (low/medium/high). Affects all extended thinking models. One line of code. | 30–70% on thinking tokens |
| Google Gemini | Model-dependent | Gemini 2.5 Flash/Pro support a thinking_budget parameter to cap reasoning tokens. Not available on all models. | Varies by budget cap |
| Mistral | Not available | Mistral models do not expose a reasoning effort parameter. Cost is determined by model choice alone. | — |
| Microsoft Foundry | One parameter (GPT models) | Same reasoning_effort parameter as OpenAI for Foundry-hosted GPT models. | 30–70% on reasoning tokens |
| Microsoft Phi-4 | Not available | Phi-4 models do not support reasoning effort control. They are small language models designed for efficiency by default. | — |
When to use each level:
| Effort Level | Best For | Examples |
|---|
| Low | Simple, well-defined tasks where the answer is straightforward | Classification, extraction, formatting, simple Q&A |
| Medium ⭐ | Most production workloads — good balance of quality and cost | Summarization, chat, content generation, standard coding |
| High | Genuinely complex problems requiring deep analysis | Multi-step math, legal reasoning, novel algorithm design, research |
The key insight: most teams default to "high" because they assume more thinking equals better answers. In practice, 80% of production workloads perform identically at "medium". The recommendation is simple — start at medium, only move to high if you measure a quality drop on your specific task. This is a one-line code change with no architectural refactoring needed.
5. Token Optimization (10–30% savings)
Set max_tokens, remove unnecessary boilerplate, prefer structured outputs for machine-consumed responses. Summarize long documents before sending to expensive models.
📌 Why Every Token Counts
AI providers charge per token — roughly one token per word. Every unnecessary word in your prompt costs money going in, and every extra word the model generates costs money coming out (output tokens are typically 3–10× more expensive than input). Most applications waste 20–40% of their token budget on content that does not improve the result.
Think of it like this: you are sending a courier to deliver a message. You pay by the word — both for the instructions you write and the response they bring back. If your instructions include three paragraphs of context the courier does not need, and the response is a five-page essay when a three-line answer would do, you are overpaying for the same outcome.
Five practical techniques:
| Technique | Effort | Savings | How It Works |
|---|
Set max_tokens | One parameter | 10–30% | Cap the response length so the model stops when the answer is complete instead of generating filler. If you need a yes/no answer, set max_tokens to 10 — not 4,000. |
| Use structured output | Small code change | 20–50% | When the response is consumed by code (not humans), request JSON or a specific schema. The model produces compact, parseable output instead of verbose prose. OpenAI, Anthropic, and Gemini all support structured output modes. |
| Trim prompt boilerplate | Prompt review | 10–25% | Audit your prompts for repeated preambles, redundant examples, or overly verbose instructions. Teams often copy-paste prompt templates without removing sections that do not apply. Shorter prompts cost less and often produce better results. |
| Pre-summarize long documents | Pipeline change | 40–80% | Instead of sending a 50-page document to an expensive model, first summarize it with a cheap model (Ministral 3B, Phi-4 mini), then send the summary to the expensive model. You pay nano-tier rates for the bulk processing and flagship rates only for the refined analysis. |
| Use retrieval instead of stuffing | Architecture change | 50–90% | Instead of pasting entire databases or document sets into the prompt, use a retrieval system (RAG) to find only the relevant paragraphs. Sending 3 relevant paragraphs instead of 200 pages reduces input costs by 99%. |
Where the money actually goes:
Output tokens are the hidden cost driver. Across all providers in our catalog, output tokens cost 3× to 10× more than input tokens. A model that generates a 2,000-word essay when a 200-word summary would suffice costs 10× more in output alone. This is why max_tokens and structured outputs have such outsized impact — they directly reduce the most expensive part of every API call.
| Provider | Input $/MTok | Output $/MTok | Output Multiplier |
|---|
| Mistral Small 4 | $0.10 | $0.30 | 3× |
| GPT-5.4 mini | $0.75 | $4.50 | 6× |
| Claude Sonnet 4.6 | $3.00 | $15.00 | 5× |
| GPT-5.5 | $5.00 | $30.00 | 6× |
The bottom line: token optimization is the least glamorous lever, but it compounds with everything else. A team that caches prompts, uses batch, sets a medium reasoning effort, and trims 30% of wasted tokens can reduce their total inference bill by 80–95% compared to a naive implementation using the same model.
Provider Quick Reference
Key differentiators not covered in the universal strategies above.
Mistral
- Lowest per-token rates in the catalog — dominant cost leader at tiers 1–3
- Small 4 is open-weight (Apache 2.0) — self-hostable for fixed-cost deployment at scale
- Codestral: purpose-built for code generation at 3× cheaper output than GPT-5.4 mini
OpenAI
- Widest model range (nano → flagship) — easiest provider to right-size within one ecosystem
- Regional residency adds ~10% premium for models released March 2026+
- Strongest structured output support — best for machine-consumed JSON responses
Anthropic (Claude)
- Best effort control granularity (low → max) — affects tool calls and thinking, not just output
- Especially attractive for agentic workflows with heavy prompt reuse (caching + effort compound well)
- Opus 4.7+ uses a new tokenizer: up to 35% more tokens for equivalent text — factor into migration estimates
Google Gemini
- Flash-Lite 3.5 is a strong low-cost option for high-volume simple tasks
- Four pricing tiers (Standard, Batch, Flex, Priority) — most flexible billing model
- Flash and Pro models omitted from v0.1 pending verified pricing
Microsoft Foundry
- OpenAI models with EU residency — not available on direct OpenAI
- Provisioned Throughput (PTU): fixed hourly capacity, cheaper above ~60–70% utilization; cached tokens are FREE on PTU
- Phi-4 models: Microsoft-native SLMs, open-weight (MIT), self-hostable — Phi-4 mini has the cheapest input rate in the catalog ($0.075/MTok)
Cohere (Excluded from v0.1)
Cohere's cost play is architectural — use Embed + Rerank to reduce candidates before sending to expensive generation models (90%+ token reduction). Pending verified 2026 pricing for inclusion.