Stop Blaming ChatGPT: Your Data Pipeline Is the Real Problem

Last month, I watched a client's "AI-powered" customer classification system confidently assign "fraud" to a legitimate order because the customer's shipping address had a typo. The system worked flawlessly in demos. In production, it was a disaster. Our team spent a week debugging, and I kept hearing the same refrain: "The AI model is broken." It wasn't. The problem was upstream, nobody had bothered to define what valid data actually looked like.

This happens more often than I'd like to admit. We obsess over model accuracy scores and prompt engineering, but we ignore the foundation: the shape of our data. After reading through a particularly thoughtful piece on AI failure, I realized I've been thinking about this wrong. The AI wasn't the villain in that story. It was just faithfully executing a system nobody had properly understood.

The Hidden Cost of Undefined Data Shapes

Here's what I've learned the hard way: before you put any model into production, you need to draw out your entire data pipeline as a sequence of transformations. What goes in at each stage? Where does it come from? What values are valid, missing, contradictory, or stale?

When I started doing this exercise rigorously, I discovered that my systems had massive blind spots. I wasn't handling edge cases, I wasn't even aware they existed. My tests passed. My model performed well in staging. But the moment real data hit production, things broke in ways I hadn't anticipated.

The uncomfortable truth: if your team can't articulate what success looks like, you can't measure whether your system is actually working.

Stabilizing the Target Before You Deploy

I made a mistake early in my career that still bothers me. We built an "intelligent" content moderation system, and the spec was this: "Remove bad content." That's not a specification. That's an aspiration.

We needed to sit down with humans who actually do moderation and ask them to show us examples. Not just the easy cases. Show us content that's clearly bad. Clearly good. And, this is the critical part, genuinely ambiguous. Then we had to verify that the humans could apply those rules consistently themselves.

When we did this exercise, we discovered the humans couldn't agree on 15% of cases. We didn't have a model problem. We had a definition problem. Once we narrowed the scope and anchored it to concrete examples, everything got better.

The Model Is Just One Piece of a Larger System

This is the insight that changed how I architect systems. The model doesn't exist in isolation:

Input → Validation → Retrieval → Normalization → 
Model Inference → Output Validation → Policy Checks → 
Human Escalation

Error can enter at any point. The model output might be fluent and wrong. The retrieval system might return stale data. Normalization might mangle a field. The input validation might miss a malformed record.

In my customer classification example, the model was working correctly. The data validation upstream wasn't catching typos. We needed to ask: should the system accept fuzzy matching? Should it escalate ambiguous cases? Should it fail safely?

Once I started testing each boundary in this pipeline, I caught bugs the model would have taken the blame for.

My Approach: Mutation Testing for Workflows

I'm experimenting with bringing mutation testing principles into AI workflows. Instead of just testing the happy path, I deliberately break things:

• Strip out expected fields
• Inject contradictory data
• Supply stale corpus information
• Feed the model subtle but incorrect context

Then I ask: does the system detect the problem and escalate to a human? Or does it confidently return garbage?

const workflow = async (request) => {
 const shapeCheck = validateInput(request);
 if (!shapeCheck.valid) return escalate(shapeCheck.errors);
 
 const evidence = await retrieveContext(request);
 if (evidence.contradictions.length > 0) {
 return escalate(evidence.contradictions);
 }
 
 const decision = await model.decide(request, evidence);
 const criticism = await critic.review(decision, evidence);
 
 if (criticism) return escalate(criticism);
 return accept(decision);
};

This forces me to think about failure modes explicitly. The model isn't permitted to define its own success criteria.

What I'm Doing Differently Now

My new project starts here: I map data shapes obsessively. I get humans to agree on what success means, then I anchor that agreement to concrete examples. Only then do I build models. And when I do, I wrap them in guardrails that detect when something smells wrong.

The result? Systems that fail predictably and safely. Systems that escalate to humans when they're uncertain. Systems that don't confidently hallucinate their way into production disasters.

What's your biggest AI failure been? I'd bet if you trace it backward, it wasn't the model. It was the shape of something upstream.

Source: This post was inspired by "The Shape of Failure: Before You Blame the AI" by Dev.to. Read the original article