What vLLM means in practice
vLLM is the open-source LLM inference server that has become the de facto choice for sovereign deployments. Its core innovations — PagedAttention (memory management for the KV cache borrowed from operating-system virtual memory) and continuous batching (new requests can join a batch mid-flight rather than waiting for the batch to complete) — give 3-5× the throughput of a naive transformers.generate loop. In MindMap deployments vLLM serves Llama 3.3 70B (quantised) on 2× H100, or Llama 3.3 8B on a single A100, with median time-to-first-token under 400ms at 30-50 concurrent users.
Related terms
Sovereign AI →
An architecture where customer data never leaves the network perimeter, model weights run on customer-controlled hardware, inference logs stay in the customer's SIEM, and the entire stack can operate air-gapped.
Large Language Model (LLM) →
A transformer-architecture neural network trained on very large text corpora to predict the next token in a sequence, producing fluent natural-language output across a wide range of tasks.
Kubernetes (in sovereign AI) →
The container orchestration platform that hosts the sovereign AI stack — provides namespace isolation, network-policy enforcement, GPU scheduling, and the lifecycle plumbing for upgrades.
All 62 terms, in plain language
Sovereign AI, RAG, agentic AI, IDP, MLOps and the regulations that shape enterprise AI.