I Built My Own Agent Control Plane So You Don't Have To (But You Probably Will Anyway)

Last month, I watched a startup I know spend three weeks trying to move their Claude-based agent to AWS Bedrock. Three weeks. Not because the agent logic was complicated, it wasn't. But because their entire infrastructure assumed Claude was the runtime. Credentials were hardcoded. Configuration was baked into the agent definition. Session state lived in memory on a single container. By week two, they basically admitted they were rebuilding from scratch.

That conversation stuck with me because I've been there. Six months ago, my team started building agent infrastructure for a fintech client who needed agents across multiple LLM providers, different compliance zones, and a team that could actually operate this thing without me. We didn't know what we didn't know. What I thought would be a nice wrapper around LangGraph turned into a 10-week architectural commitment to build what I'm now convinced is the missing layer in the entire agent platform ecosystem.

I want to walk through what I learned, because the current agent platform market is selling you a lie: that picking one framework or one managed platform solves your infrastructure problems. It doesn't. Not even close.

The Three-Layer Problem (And Why Marketing Obscures It)

Here's what I've come to understand: agent platforms are actually three completely different things pretending to be one.

First, there's the agent logic layer, frameworks like LangGraph and CrewAI. These are legitimately good at what they do: managing reasoning, tool-calling, state machines. They solve the algorithmic problem. I respect this layer.

Second, there's the runtime layer, Claude Managed Agents, Bedrock, Gemini Enterprise. These handle execution sandboxing, session isolation, interrupt/resume. They're production-ready if you're willing to commit to one vendor.

Third, and this is where it gets real, there's the control plane layer. This is credential management, multi-runtime coordination, per-agent configuration, session durability, cost attribution, and observability across heterogeneous infrastructure. Almost nobody is shipping this as a product. Most teams build it themselves around month 3.

The marketing problem is that everyone's calling all three "agent platforms." So teams end up trying to solve infrastructure problems with frameworks, or trying to handle multi-team governance with managed platforms that were designed for single-runtime deployments. It's like trying to use Kubernetes as a networking layer, technically possible, but you're using it wrong.

What I Actually Changed in Production

When we started seeing agents deployed across different runtimes, we had the choice: accept vendor lock-in or build abstraction layers.

The first real pain point was credentials. We had API keys scattered everywhere, AWS Secrets Manager for Bedrock, environment variables for Claude, GitHub OAuth in Vercel. A team member left. Rotating credentials became a nightmare that involved touching deployment configs across five different services.

We built a credential vault layer that sits between our agents and providers. Agents ask for credentials by name, not by provider. Configuration changes, permissions, rotations, all happen centrally without touching agent code. Here's the principle:

# What we STOPPED doing
class Agent:
 def __init__(self):
 self.anthropic_key = os.getenv("ANTHROPIC_API_KEY")
 self.bedrock_config = {"region": "us-east-1"}
 
# What we DO now
class Agent:
 def __init__(self, credential_name: str, runtime: str):
 self.credentials = vault.get(credential_name)
 # Runtime is resolved at invocation time, not init time
 self.runtime = resolve_runtime(runtime)

Second was session durability. The moment we had agents running 24+ hour workflows, losing state mid-execution became expensive. We moved sessions to Postgres with snapshots. Now if a runtime crashes, we replay from the last checkpoint. That's worth more than any marketing claim I've heard.

Third was configuration without redeployment. When GPT-5 shipped (hypothetically speaking), we wanted to swap models in a config file, not redeploy containers. Same with tool permissions and rate limits. Config is now separate from agent definitions entirely.

My Take on This Entire Space

I think the original article is dead accurate, but I'd push further: most teams will build this infrastructure anyway, even knowing about these gaps. We did. And it took longer than we budgeted.

But here's what bothers me: there's a multi-billion dollar market opportunity to ship the control plane layer as a product, and I see maybe two or three companies actually trying. Everyone else is either a framework (which is good but insufficient) or a managed platform (which optimizes for single-runtime convenience).

What I'd do differently if starting now: I'd spend two weeks really understanding my actual multi-runtime requirements before picking frameworks. I'd prototype credential management as a separate concern. I'd build session durability into the architecture from week one, not month three.

The honest thing: you might not need all three layers immediately. But if you're betting on agents for anything beyond a prototype, you need to acknowledge that the gap exists. Plan for it. Budget for it.

What Would You Do?

Are you currently running agents across multiple runtimes or teams? What infrastructure problems have hit you that the marketed platforms didn't solve? I'm genuinely curious what I'm missing because I suspect I'm not alone in this experience.

Source: This post was inspired by "How to Evaluate an Agent Platform Without Getting Sold on Hype: The Real Infrastructure Questions" by Dev.to. Read the original article