Clear answers about LATCH, hardware, pricing, and document-memory architecture.

LATCH is a proprietary inference layer that compiles document sets into persistent LLM memory. After a one-time compilation step, every subsequent query runs against the compiled representation without re-reading, re-chunking, or re-embedding source documents. The result is saved as a portable .latch or .latchdoc binary file.

RAG chunks documents, embeds them, retrieves relevant chunks per query, and injects them into the context window every time. LATCH compiles the entire document set once into a persistent model-level representation. There is no chunking, no retrieval step, and no per-query cost after the initial compile.

This eliminates chunking artifacts and enables full cross-document reasoning. For the longer version, see the full LATCH vs RAG comparison.

Standard KV caches are session-bound. They are evicted when the session ends and cannot be persisted to disk or shared. LATCH produces a persistent, hash-validated binary file that can be saved, transferred, and reloaded in 2ms.

LATCH also reduces VRAM usage by 50%, which KV caching does not.

LATCH currently supports four model families: Qwen (2.5 14B benchmarked), Mistral, Llama, and DeepSeek.

LATCH requires an NVIDIA GPU with 80GB VRAM. The H100 and A100 are the recommended and benchmarked GPUs. It runs as a Docker container on Linux.

The deployment path is documented in the Ubuntu GPU quickstart and the RunPod guide.

The evaluation/personal license is $79 one-time, covering up to 3 activations for one user. Commercial deployment and enterprise/OEM licenses are available by contacting sales.

No. LATCH is self-hosted by default. You run the Docker container on your own infrastructure and your documents never leave your environment. A managed hosted option is planned for the future.

PDF, DOCX, XLSX, PPTX, TXT, MD, HTML, CSV, JSON, and XML. The current API surface is documented in the customer API reference.

A .latch file is a portable binary containing only the compiled model-level memory with no source text. It can be shared without exposing the original documents and reloaded in 2ms.

A .latchdoc file includes everything in a .latch file plus embedded raw text, enabling full-text search and automatic quality fallback for edge-case queries. It is the recommended default format.

The format overview on the main site lives in the portable document memory section.

At corpus scale on one NVIDIA A100 80GB — 347 SEC filings, 1,651,666 source tokens — LATCH compiles the full corpus in about 50 seconds, reaches first token in 0.79s, and reprocesses zero source tokens per query. Compiled memory reloads in 2ms for a single document. Generation is deterministic (temperature 0). Decode throughput is about 27 tok/s, which trails a vLLM-served baseline at about 44 tok/s — see the benchmarks page for the full table including where LATCH loses.

See the main-site benchmarks section for the compact summary.

Yes. LATCH exposes an OpenAI-format compatible REST API, so existing tooling and integrations work with minimal changes.

Compiled artifacts are tied to the model they were compiled against. Migration is a recompile, and it is fast: a 347-document corpus recompiles in about a minute on a single A100. Bundles record their model tuple so a mismatch fails loudly instead of silently degrading.

No. RAG retrieves text chunks and re-feeds them to the model on every query. LATCH compiles each document into a fixed-size memory artifact once; at query time zero source tokens are reprocessed. There is no embedding index, no retriever, and no reranker in the query path.

Because the numbers where it wins are only credible if the losses are on the same page. Decode throughput currently trails a vLLM baseline; setup time, time-to-first-token, and query-side token economics are where LATCH wins. See the benchmarks page for the full table.