Last month, I was sitting in a standup meeting when someone asked how we'd handle our LLM API if the provider went down. The room went quiet. We'd built our entire feature around OpenAI's reliability, with barely a fallback in sight. That conversation stuck with me, mostly because I realized we'd abdicated responsibility for something critical to a third-party service without really thinking through what happens when it fails.
When I read about Kevin's Haunted AI project, it felt like watching someone solve the exact problem I'd been ignoring. Here's a developer who built a detective game that needs AI to feel natural and responsive, but instead of tying everything to Anthropic or OpenAI, he ran inference locally with Qwen on Ollama. More importantly, he built the entire game architecture so that when the AI provider fails, and it will fail, the experience degrades gracefully rather than breaking entirely.
That's the kind of thinking I need more of in my own work.
The Architecture That Actually Makes Sense
Let me break down what Kevin built, because the design choices matter more than the game itself. The game is a detective mystery where you interrogate ghosts about what they witnessed. The obvious approach would be to let the LLM generate whatever response it wants. Instead, he built something much tighter.
The backend owns all the game state and rules. When you ask a ghost a question, the backend filters what that ghost is allowed to know based on the investigation state. It doesn't send the complete case truth to the model. Instead, it sends a constrained set of facts and lets Qwen choose from them and select an appropriate tone. The model doesn't generate facts, it selects from authored options.
This is the opposite of what I see in most AI feature implementations. We usually throw data at the model and hope it comes back with something acceptable. Kevin inverted it: the model is narrowly responsible for selecting from vetted options, and the game logic handles everything else.
The Fallback That Actually Works
Here's what really got me thinking differently: Kevin built a ten-second timeout. If Qwen doesn't respond in ten seconds, the game shows you a labeled fallback response, still authored, still coherent, but explicitly marked as not coming from the model. He even kept this in his demo video where it happened. Most of us would have re-recorded until the AI worked perfectly.
That's production thinking. That's understanding that reliability matters more than looking smart.
The demo mode also runs the same game with pure authored dialogue, no model calls at all. This means you can test the entire game logic without depending on Ollama being installed. You can ship a fallback experience that's complete.
// Simplified adaptation pattern from the project
async function getGhostResponse(ghostId: string, question: string, context: GameState) {
const permittedFacts = filterKnowledge(ghostId, context);
try {
const response = await ollamaClient.generateStructured(
model: 'qwen2.5:1.5b',
facts: permittedFacts,
userQuestion: question,
timeout: 10000
);
return {
type: 'live',
factIds: response.selectedFacts,
tone: response.selectedTone,
authoredText: renderFacts(response.selectedFacts)
};
} catch (error) {
return {
type: 'fallback',
authoredText: getFallbackResponse(ghostId, question),
sourceLabel: 'Unavailable - showing authored response'
};
}
}
This pattern forces you to think about the fallback path from the beginning, not as an afterthought.
What This Makes Me Question
I've been assuming that bigger LLMs are better. The whole industry pushes us toward GPT-4, Claude, whatever's newest. Kevin used 1.5 billion parameters on a Mac. It worked because the scope was bounded, the model didn't need to solve the entire problem.
I'm now wondering where else in my work I've been overshooting. If I constrain the model's responsibility enough, could I use smaller, faster, cheaper alternatives? Could I ship features that work offline? What would my architecture look like if I designed it around a model that might fail rather than one that probably won't?
The other thing that hits me: the license transparency. Kevin tracked every license explicitly, Apache 2.0 for Qwen, MIT for Ollama, MIT for his own code. No assumption that "the AI generated this, so who owns it?" It matters legally and ethically, and most projects I've seen just... ignore it.
Where I'd Push Back
This architecture requires more engineering upfront. You can't just prompt-engineer your way through. Every interaction needs thought about what's permitted, what's fallback, what's owned by the game versus owned by the model. That's friction that might not feel worth it on a tight deadline.
Also, a small model in isolation might give repetitive responses. The demo shows Qwen's limitations, sometimes it says the same thing twice. That's the tradeoff. You get reliability and inspectability, but not the magic of a massive model.
What I'm Doing Next
I'm going to audit one of our existing AI features this week and ask: what if the API times out? What if it returns garbage? What if we needed to ship this with authored fallbacks instead? That's probably going to make our architecture uglier in the short term. But it'll make it more honest.
Source: This post was inspired by "Haunted AI: Whispers of Blackthorn Manor, A Local Qwen Detective Game" by Dev.to. Read the original article
