Benchmarks
Every number here comes from a logged run on dated hardware.
Where LATCH loses, the number is published anyway. A benchmark page that only contains wins is a marketing page, and this industry has enough of those.
Corpus-scale run — 347 SEC filings
April 2026, NVIDIA A100 80GB. Corpus: 10-K, 10-Q, 8-K, and DEF 14A filings for Apple, Microsoft, Nvidia, Alphabet, Meta, Amazon, and Tesla — 1,651,666 source tokens across 347 documents. Both systems answered the same cross-document prompts against the same corpus using the same base model family.
| Metric | LATCH | Conventional RAG (BGE-M3 + Qdrant + rerank + vLLM) |
|---|---|---|
| One-time setup | ~50 s compile | ~10.6 min index build |
| Query-time source processing | 0 tokens | 2,500–2,900 tokens retrieved + reranked |
| Time-to-first-token | 0.79 s (selector-routed) | 5.5–8.8 s retrieval before generation begins |
| Compiled-memory load | 18.5 ms (19-document selection) | n/a |
| Decode throughput | ~27 tok/s | ~44 tok/s |
| Determinism | Deterministic artifacts and generation | Probabilistic retrieval |
| Portability | 5.4 GB exportable bundles | Live index + embedding service required |
Runtime profile cdlac_latch_qwen14b_locked_20260317. RAG baseline served by
vLLM 0.19.0 with Qwen 2.5 14B Instruct at max_model_len=8192.
Where LATCH currently loses
- Decode throughput trails a vLLM-served baseline — roughly 27 tok/s against 44 tok/s. LATCH reaches first token far sooner; it generates more slowly after that.
- Total wall time can favor the RAG stack on long generations. When a prompt produces several hundred output tokens, the decode gap can outweigh the time-to-first-token advantage.
- Answer-quality scoring at corpus scale is still being formalized. Internal probe suites pass 10–12 of 12 gates across runs, but internal gates are not a substitute for public benchmarks. LoCoMo and LongMemEval results are planned and will be published on this page — including the losses.
- Selective retrieval of fine details remains the known weak spot. Broad cross-document synthesis is strong; pinpointing a single small fact inside one document is where compiled memory is most likely to miss.
Single-document measurements
- 0.11 s time-to-first-token — single compiled document, warmed runtime.
- 2 ms compiled-memory reload from a
.latchartifact.
These are single-document figures and should not be read as corpus-scale numbers. At 347 documents, measured time-to-first-token is 0.79 s selector-routed and 3.1 s when querying every compiled document without routing.
Changing base models
Compiled artifacts are tied to the model tuple they were compiled against — this is the most common technical objection to LATCH, and it is a fair one. The migration path is recompilation, and it is fast: the full 347-document corpus recompiles in about a minute on one A100.
Bundles record their model tuple, so a mismatch fails loudly at load time rather than silently degrading answer quality.
Reproduction
Every figure above traces to a logged run with recorded hardware, dates, image tags, and runtime profile identifiers. If you are evaluating LATCH and want the underlying run artifacts or a walkthrough of the harness, contact mike@codynamicslab.com.