Six months ago, I pitched an AI agent to a client, something that would automate their customer support triage. I threw out a number for infrastructure costs that made them flinch. When they asked how I arrived at it, I realized I had no actual model. I'd taken a guess, added 40%, and shipped it. That conversation haunted me. After reading through a detailed cost breakdown for production AI agents, I realized how badly I'd been thinking about this problem. I wasn't alone, most developers I know are still guessing.
The real issue isn't whether AI agents are expensive. They're not, at scale. The issue is that we're conflating token costs with infrastructure costs and pretending they're the same problem. They're not. And once you understand the difference, you stop making mistakes like the one I made with that client.
The Actual Math Behind Running AI Agents
Here's what nobody tells you: if you're running AI agents in production, tokens will eat 95 percent of your bill. Not infrastructure. Not storage. Tokens. And within tokens, it's not equally distributed, output tokens cost 5x more than input tokens in most models.
The worked example that stuck with me was straightforward. Take 300,000 tasks a month, each making 8 API calls to Claude Sonnet. Each call uses 6,000 input tokens and returns 400 output tokens. Without any optimization, that's roughly $38,400 in token costs. But here's the part that changed how I think: prompt caching cuts that by 45 percent. Not through some clever engineering trick. Just by telling the model "hey, don't re-bill me for the same system prompt and tool definitions eight times."
That's $8,280 back in your pocket, every month, for doing something most of us should already be doing.
Infrastructure, the sandboxes that execute model-generated code, the orchestration services, NAT gateways, storage, adds up to less than a thousand dollars a month at this scale. It's real money, but it's not where the hemorrhaging happens.
What Changes When You Run Code Inside Your Agent
The part that actually matters to me as someone who deploys this stuff is the sandbox execution model. If your agent generates code and runs it (and if it doesn't, you're not really using agents), you're spinning up containers. That's compute time you're paying for every minute the code runs.
The model assumes 30 percent of tasks run code for 3 minutes on 2 vCPU and 4 GB. That's about $444 a month. But in practice, sandbox startup time, warm pools you keep for latency, and peak concurrency spikes all add hidden cost. I've burned more money than I care to admit keeping warm pools around that nobody was using.
The real optimization here isn't technical, it's architectural. You need to know upfront: which tasks actually generate executable code? For how long? Can you route simple ones to a smaller model that doesn't even try? Can you cache the code generation results?
My Take: The Uncomfortable Truth About Token Optimization
I agree completely that tokens dominate the cost picture. But I think the article understates how quickly that changes as you scale differently. The worked model assumes steady traffic: 10,000 tasks a day, distributed evenly. Real production workloads don't work that way.
What concerns me is that optimizing for tokens alone might push you toward worse decisions. Yes, you should use prompt caching. Yes, output tokens are expensive, so be concise. But I've seen teams so focused on token count that they split tasks into more API calls to keep individual prompts smaller, and ended up with higher total cost because they hit API latency problems.
The fixed costs, always-on orchestration replicas, NAT gateways, matter more than this model suggests when your traffic is bursty. If you're running three replicas 24/7 for occasional spikes during business hours, you're wasting money. But the article does acknowledge this: at lower volumes, fixed costs are a bigger share of the bill.
Here's what I'd do differently: start with the token calculation, but then run a sensitivity analysis. If I change my call count by 20 percent, how much does cost move? What about output verbosity? What about switching models? That tells you where to actually focus.
The Question I'm Still Sitting With
The model deliberately excludes databases, vector stores, embedding calls, and observability subscriptions. That's pragmatic but misleading. In every agent I've built, those additions have added another 15-25 percent to the bill. They matter.
What's your experience been? Are you running agents in production? What's actually crushing your costs, is it tokens, or is it something the model glossed over?
Source: This post was inspired by "What It Costs to Run AI Agents in Production: A Worked Cost Model for Tokens, Sandboxes, Storage, and Networking" by Dev.to. Read the original article
