Last month, I got a Slack message that made my stomach drop. Our OpenAI bill had tripled overnight. Tripled. I spent four hours digging through logs, trying to figure out which service was hemorrhaging tokens, and I couldn't trace it back to anything. Turns out it was a junior developer's experimental chatbot feature running in production for three weeks. Nobody knew about it. Nobody was monitoring it. I realized then that we weren't actually managing AI costs, we were just absorbing them and hoping they went away.
That's when I stumbled onto this piece about AI FinOps cost attribution, and it hit different. This isn't academic hand-waving. This is the exact problem I was living through, and apparently, most enterprises are too.
The Problem Nobody Wants to Admit
Here's the thing about traditional cloud billing: it was built for instances and infrastructure. You spin up an EC2, you see CPU utilization, you map it to a cost center. Done. But LLM APIs broke that entire model.
When you call OpenAI, you're not provisioning hardware. You're burning tokens, these tiny, invisible units of text that cost fractions of a cent each. The complexity? A single user request might trigger a chain of 50 LLM calls through an agent, each with different model costs. One call could be GPT-4 reasoning ($15 per million input tokens), the next could be GPT-3.5 ($0.50). By the end of it, you have no idea whether that single user interaction cost you a penny or five dollars.
And here's where I got angry reading this: most engineering teams have zero visibility into this. API keys get hardcoded. Agents run unsupervised. Shadow AI experiments proliferate. Then month-end hits and you're explaining a 200% budget overrun to finance.
Building Real Cost Attribution (The Right Way)
The solution proposed in the article is solid: inject correlation IDs at your API gateway, propagate them through your orchestration layer, and intercept every LLM call to record token usage with OpenTelemetry. This creates an immutable trail from user request to actual cost.
I'm actually building this right now. Here's what that looks like in practice:
// API Gateway middleware (Kong/Envoy)
app.use((req, res, next) => {
const correlationId = req.get('x-correlation-id') || uuidv4();
const businessUnit = req.get('x-business-unit') || 'untracked';
const appId = req.get('x-application-id') || 'unknown';
req.locals = {
correlationId,
businessUnit,
appId,
startTime: Date.now()
};
res.setHeader('x-correlation-id', correlationId);
next();
});
// LLM wrapper that emits telemetry
async function callLLM(model, prompt, context) {
const response = await openai.chat.completions.create({
model,
messages: [{ role: 'user', content: prompt }]
});
// Emit OpenTelemetry span with token costs
const span = tracer.startSpan('llm.api_call', {
attributes: {
'correlation_id': context.correlationId,
'business_unit': context.businessUnit,
'application_id': context.appId,
'model': model,
'input_tokens': response.usage.prompt_tokens,
'output_tokens': response.usage.completion_tokens,
'cost_usd': calculateCost(model, response.usage)
}
});
span.end();
return response;
}
This creates an audit trail. Every token call is tagged with who requested it, which business unit it belongs to, and what it cost. No more mystery bills.
Where I Disagree (And Where I'm Skeptical)
The article mentions that 5% of users generate 60% of AI spending. That statistic scares me because it implies there's a "good" way to distribute AI costs, that if we just identify the super-adopters and optimize their models, we're done.
I don't think it's that simple. In my experience, those super-adopters are often the teams generating the most business value. The issue isn't that they're wasteful, it's that we've never built the tools to ask: "Is this expensive because we're using the wrong model, or because this feature is actually valuable and worth the cost?"
Cost attribution isn't about punishment. It's about visibility. Once you see the data, you can make real trade-offs. That requires a different mindset than traditional FinOps.
What I'm Doing Next
I'm implementing this architecture over the next sprint. No more guessing. I want a dashboard where I can see exactly which features, teams, and users are driving token consumption. Then I can actually have data-driven conversations about optimization.
What's your setup like? Are you tracking LLM costs at request-level, or is it still a black box for you?
Source: This post was inspired by "AI FinOps Cost Attribution: Stop Guessing Your LLM Spend" by Dev.to. Read the original article
