Last month, I watched a financial audit report lose a transaction ID. A contract number just... vanished. The LLM had read it, understood it, but when it summarized the chunk, that specific fact didn't make it to the final output. My team caught it during QA, but the thought that haunted me was: how many times do we not catch it?
I've been building systems that feed documents into LLMs for three years now. Chatbots, compliance tools, knowledge bases. The pattern was always the same: chunk the document, summarize each chunk, merge the summaries, ship it. It's fast, it feels reasonable, and until you actually measure what's being lost, it seems fine. Then I read about LongLoom, and something clicked. The problem isn't that summarization exists, the problem is that we've normalized it as the default, even when we have no idea what we're losing.
The Uncomfortable Truth About Naive Summarization
Let me be direct: most LLM document processing pipelines are lossy, and nobody wants to admit it. The standard advice is chunk-and-summarize. Works great for general knowledge bases. Catastrophic for anything with compliance or audit requirements.
LongLoom tested this with planted facts, known identifiers, dates, amounts scattered through documents. A naive chunking pipeline lost 2-10% of facts consistently. Some runs lost a contract ID. Others lost a dollar amount. The worst part? It's stochastic. You can't predict what disappears. For customer support? Acceptable loss. For legal discovery? Career-ending loss.
I've shipped systems where I assumed this wasn't happening. I never actually measured it. That's the gap that LongLoom exposes.
The Core Insight: Transport Losslessly, Condense Deliberately
The library's philosophy is elegant: the pipeline should move information without loss. Condensation, summarization, should be optional, explicit, and isolated.
LongLoom splits into two modes. The default still summarizes (useful for readable reports), but the for_extraction() mode copies fact-bearing lines verbatim and deduplicates them. The LLM isn't paraphrasing; it's filtering. Extract the lines that matter, pass them through exactly as written, merge duplicates, done.
They tested it at scale: 83,023 planted facts across 6.7M characters. Result: 100% preservation. Every single fact survived the pipeline.
The architecture is also clean: chunk → map (parallel LLM calls) → hierarchical merge → repair (only if still oversized). The context window never chokes because it only ever sees chunks and compact intermediate results.
Why This Matters for Production Systems
Here's what gets me about this: it's not a wrapper. It works with any LLM. OpenAI, NVIDIA, Anthropic, local vLLM instances, even a bare Python function. Nine storage backends, PostgreSQL, MongoDB, SQLite, whatever you already use. It lives in your database, not some SaaS vendor's.
The security posture is also refreshingly honest. No environment variable reads for secrets. No telemetry. No update pings. Your chunks stay in your tables, your schema. The library validates table names and column names before building queries. It doesn't log credentials or prompts.
That's the difference between "enterprise-ready" and "enterprise-themed."
My Take: Use This, But Know Its Limits
I'm convinced. For any system where you can't afford data loss, legal, financial, compliance, audit trails, this is the pattern you should implement. The benchmarks are reproducible. The code is open. The honesty about limitations is refreshing.
But I notice the real-LLM extraction achieves ~92% accuracy, not 100%. That's the model's quality floor, not the pipeline's fault. Cross-chunk conflicts are preserved side-by-side, not reconciled. That means if two chunks contradict each other, you get both. You need downstream logic to handle that.
I'd also be cautious about tokenizer approximation. They use tiktoken for non-OpenAI models. I'd keep 2× headroom on chunk sizes to be safe.
A Practical Starting Point
If you're processing long documents with any LLM, try the verbatim extraction:
from longloom import MapReduceProcessor, init_llm
processor = MapReduceProcessor(
llm=init_llm("openai", "gpt-4o-mini", api_key="..."),
storage=create_store("sqlite:///my_chunks.db"),
mode="extraction" # Verbatim, not summarization
)
result = processor.process("path/to/large_document.pdf")
# Guaranteed fact preservation
That's it. Your facts stop disappearing.
What I'm Asking Myself Now
Do I have any existing pipelines that are silently losing data? How would I even know? I think I need to audit a few systems this week with the planted-fact approach.
Have you run into this problem in production? I'd genuinely like to know how you caught it, or if you haven't measured it yet.
Source: This post was inspired by "LongLoom: Stop Losing Facts When Your Documents Exceed the Context Window" by Dev.to. Read the original article
