Why I'll Never Trust a Live File As a Source of Truth Again

I've been burned by this exact problem twice in production, and both times I blamed the wrong thing. A client's usage numbers drifted by ~15% month-over-month. We assumed it was a calculation error in our reconciliation logic. Spent three days tracing through aggregation queries. Turned out their source system was rewriting its own history, and we were faithfully recording each new version as gospel. That's when I learned: if your data source can edit its past, you don't have an audit trail—you have a story that changes.

Reading about how an append-only log caught accounting bugs in a Rust usage tracker hit different because it validated something I've been preaching internally but struggling to justify in code review. One of our junior developers pushed back last month: "Why are we persisting this to SQLite when we can just recompute from the source files?" Good question, terrible answer to optimize for. This article gave me the words I needed.

The Core Problem: Live Files Aren't Immutable

The original scenario is almost mundane in how it breaks things. Claude Code rewrites session files when you resume or compact conversations. Messages vanish from the historical record. Any tracker that re-reads these files on each run gets different totals over time—not because the code is wrong, but because the source is lying retroactively.

This isn't unique to Claude Code. Every system I've worked with that treats mutable source files as authoritative runs into this. Slack exports that get updated. Log files that get rotated and compressed. Cloud storage APIs that change timestamps when you retry. The pattern repeats: you build your tool, it works great initially, then mysteriously the numbers change and you can't figure out why.

The fix splitrail adopted—a local SQLite history store that persists normalized usage and merges it with current data—is exactly right. It's boring infrastructure. Nobody ships a feature for it. But it catches bugs that would otherwise hide inside your metrics indefinitely.

Append-Only Logs As Insurance

I've started treating append-only logs like you'd treat database backups: non-negotiable. The cost is genuinely small. The author's implementation was "a few hundred lines and a SQLite file." In our stack, we added one table and changed ingest logic to insert-only with a message ID as the stable key. Last-write-wins for partial updates, but the full sequence is always there.

The leverage this gives you is unexpected. When a discrepancy emerges, you can actually audit it. You don't have to shrug and say "the numbers changed." You can decompose the gap: what came from the source? What got deduplicated? What vanished?

This is the part that matters most for cost tracking or any accounting-adjacent feature: reconciliation becomes meaningful. When the author compared their append-only log (SQLite) against splitrail's counts and got token-exact agreement down to the last digit across 13.5k messages, every remaining discrepancy became actionable. That's not coincidence—that's two independent systems telling you where the actual bugs are.

My Take: Do This From Day One

I wish I'd done this from the start instead of retrofitting it. The second bug in the article—subagent transcripts at depth 4 being completely invisible because the directory discovery maxed out at depth 2—is a perfect example. If you're not append-only logging, this bug lives silently in your numbers forever. 54% of messages never get counted. Users who optimize heavily for cheap models (the ones who most need accurate tracking) get the worst visibility.

The phrase that stuck with me: "If your source can rewrite history, recomputation is not accounting." That deserves to be a law. Too many tools treat recomputation from live sources as acceptable for cost tracking. It isn't. Not if accuracy matters at all.

One thing I'd add to the original analysis: start the append-only log before you write the reconciliation queries. Make immutability the primitive, not an afterthought.

A Simple Pattern to Steal

# Instead of: SELECT SUM(tokens) FROM messages WHERE project_id = ?
# (which changes when source files rewrite)

# Do this:
def record_message(message_id, tokens, timestamp):
    # Insert-only. message_id is the stable key.
    db.execute(
        "INSERT INTO audit_log (message_id, tokens, recorded_at) VALUES (?, ?, ?)",
        (message_id, tokens, timestamp)
    )
    
def get_token_total(project_id):
    # Deduplicated at query time, but history is immutable
    return db.execute(
        "SELECT SUM(tokens) FROM audit_log WHERE message_id IN "
        "(SELECT DISTINCT message_id FROM messages WHERE project_id = ?)",
        (project_id,)
    ).scalar()

The deduplication happens at read time, but the log never lies about what happened.

What Questions Does This Raise For You?

I'm curious how many cost-tracking tools in production right now are silently miscounting because they trust mutable sources. How many AI projects are optimizing cost based on incomplete data?

If you're building anything with usage metrics or billing attached, ask yourself: can my source rewrite its history? If yes, you're already broken. If you're unsure, assume yes.

Source: This post was inspired by "An append-only audit log caught two accounting bugs in a 216-star usage tracker" by Dev.to. Read the original article