What Re-ranking means in practice
Re-ranking is a second-stage retrieval pass. The initial dense+sparse retrieval pulls the top 30–100 candidate chunks, then a cross-encoder model rescores each candidate against the query and the top 3–8 are passed to the LLM. The cross-encoder is more expensive per pair than the bi-encoder used for initial retrieval (it attends to query and candidate together), but it only sees a small candidate set, so latency cost is bounded. Re-ranking typically lifts answer accuracy 8–15 points on long-document corpora. The default open-weights re-ranker for sovereign deployments is bge-reranker-v2-m3 — runs acceptably on CPU for low-traffic deployments, on a small GPU for production.
Related terms
Hybrid retrieval →
Combining dense vector search with sparse keyword search (BM25), then fusing the results — typically yielding 15–25% accuracy lift over either alone.
Retrieval-Augmented Generation (RAG) →
A pattern where, instead of relying solely on the LLM's training, the system retrieves relevant documents from a knowledge base and includes them in the prompt as context.
Embedding →
A dense numeric vector representation of a piece of text (or image, or audio) such that semantically similar inputs produce vectors close in the embedding space.
More in this category
The Sovereign AI Playbook.
Enterprise generative AI that runs on your servers.
- 01Definition. The four non-negotiable properties of a sovereign deployment — and the three patterns that look sovereign but aren't.
- 02Why now. Three forces that converged in 2026: regulator pressure, the open-weights capability gap closing, and unit economics flipping.
- 03Reference architecture. The five-layer stack we deploy: applications → agent runtime → sovereign LLM → RAG + KB → data stores. With model and tooling choices for each.
- 04The regulators. SAMA · RBI · UK ICO · EU AI Act · DPDP · HIPAA — what each requires and how it maps to architecture.
All 62 terms, in plain language
Sovereign AI, RAG, agentic AI, IDP, MLOps and the regulations that shape enterprise AI.