The Question Nobody Asks Until Production Breaks: Who Actually Deploys This Thing?
Admin User
Author
I had a small panic attack at 2 AM last month. A payment processing service I'd built was running perfectly in staging, tests passed, the Docker image was sitting in our registry looking clean and ready. Then someone asked the question that made my stomach drop: "Okay, how do we actually put this live without losing money?"
I realized I'd spent weeks optimizing build times and container configurations without ever having a real conversation about how the switch gets flipped. I'd been thinking like a builder, not like someone responsible for keeping the lights on. That distinction matters more than I initially understood.
The Simplest Approach is Also the Most Dangerous One
Let me be blunt: I've seen production deployments that just stop the old container and start the new one. Sometimes it works fine. Sometimes it doesn't, and when it doesn't, the consequences depend entirely on what your service actually does. If you're running an internal admin dashboard? Fine. If you're handling transactions? That gap between "old stopped" and "new started" is now a direct line to unhappy customers and, worse, lost trust.
The article calls this the "recreate" pattern, and there's nothing inherently wrong with it—but using it accidentally is a disaster. The real question you need to answer first is whether your service can afford even a few seconds of downtime. Most production services cannot.
Blue-Green: The Expensive Security Blanket
Blue-green deployment is the approach I think every team should at least understand, even if they don't end up using it. You run two full environments simultaneously. One is live ("blue"), one is idle ("green"). When you deploy, the new version starts on green. You can test it, watch it, verify it's actually working—all while real users continue hitting blue without interruption.
Then you flip the router. Traffic switches instantly. If something goes wrong with green once it's live, you flip back immediately. The old version never stopped running.
The tradeoff is infrastructure cost. You're essentially paying to keep a full backup environment idle most of the time. For a small team or tight budget, that's genuinely expensive. But if you're running something people depend on financially, it's often worth it.
Rolling Deployments: The Practical Middle Ground
Most teams I know use rolling deployments, and honestly, that's what I've gravitated toward too. Instead of an all-or-nothing switch, you replace instances gradually—maybe two at a time across a fleet of ten. For a few minutes, both the old and new versions are handling real traffic simultaneously.
Here's what keeps me up at night about rolling deployments: they expose a category of bugs that don't exist in staging. What happens when the new version writes data that the old version can't read? What if they disagree about API formats? You'll discover this in production, mid-rollout, with real users hitting both versions. I've seen reports like "weird intermittent failures for exactly ninety seconds" and that's usually what's happening.
The hard question rolling deployments force you to answer: can these two versions actually coexist safely? You can't just assume yes.
Canary: Testing With Your Real Users (Carefully)
Canary deployments are what I'm most interested in right now. Instead of committing to a rollout, you route a small percentage of traffic—maybe 5%—to the new version while everyone else stays on the old one. You watch metrics specifically for that slice: error rates, latency, exceptions. If something's wrong, those 5% of users hit it, not everyone.
The psychological difference is huge. You're not making a binary decision to deploy. You're making a series of small decisions: is 5% healthy? Now try 25%. Now try 100%. You can stop at any point and roll back without affecting the majority of your users.
What This Means for How I Deploy Now
I've started asking these questions before I even write deployment code:
- Can we afford downtime? Even 30 seconds?
- How much infrastructure budget do we have?
- How confident am I that the new version won't break mid-rollout?
- If something goes wrong, what's my fastest path back?
The answer to those questions determines everything—blue-green, rolling, or even recreate. The mistake isn't picking the "wrong" strategy. It's not picking one at all and hoping for the best.
What deployment strategy are you using right now, and more importantly—did you choose it deliberately, or did it just happen?
Source: This post was inspired by "Episode 5 — Who Gets to Flip the Switch" by Dev.to. Read the original article