What vLLM means in practice
vLLM is the open-source LLM inference server that has become the de facto choice for sovereign deployments. Its core innovations — PagedAttention (memory management for the KV cache borrowed from operating-system virtual memory) and continuous batching (new requests can join a batch mid-flight rather than waiting for the batch to complete) — give 3-5× the throughput of a naive transformers.generate loop. In MindMap deployments vLLM serves Llama 3.3 70B (quantised) on 2× H100, or Llama 3.3 8B on a single A100, with median time-to-first-token under 400ms at 30-50 concurrent users.
Related terms
Sovereign AI →
An architecture where customer data never leaves the network perimeter, model weights run on customer-controlled hardware, inference logs stay in the customer's SIEM, and the entire stack can operate air-gapped.
Large Language Model (LLM) →
A transformer-architecture neural network trained on very large text corpora to predict the next token in a sequence, producing fluent natural-language output across a wide range of tasks.
Kubernetes (in sovereign AI) →
The container orchestration platform that hosts the sovereign AI stack — provides namespace isolation, network-policy enforcement, GPU scheduling, and the lifecycle plumbing for upgrades.
The Sovereign AI Playbook.
Enterprise generative AI that runs on your servers.
- 01Definition. The four non-negotiable properties of a sovereign deployment — and the three patterns that look sovereign but aren't.
- 02Why now. Three forces that converged in 2026: regulator pressure, the open-weights capability gap closing, and unit economics flipping.
- 03Reference architecture. The five-layer stack we deploy: applications → agent runtime → sovereign LLM → RAG + KB → data stores. With model and tooling choices for each.
- 04The regulators. SAMA · RBI · UK ICO · EU AI Act · DPDP · HIPAA — what each requires and how it maps to architecture.
All 62 terms, in plain language
Sovereign AI, RAG, agentic AI, IDP, MLOps and the regulations that shape enterprise AI.