What LLM observability means in practice
LLM observability is the practice of capturing every model call in a structured store with enough context to inspect production behaviour later. A useful observability record includes the prompt, any retrieved context, the response, latency, token counts and cost, the model and prompt versions in use, the user identifier, and any downstream feedback signal (thumbs-up, escalation, satisfaction score). The canonical open-source tool is Langfuse, which MindMap deploys self-hosted in every sovereign engagement. The point is to be able to answer "why did the model say that" with evidence rather than guesswork — both for debugging and for the regulator who asks the same question.
Related terms
MLOps →
The discipline of taking ML and AI models from development through to reliable production operation — versioning, deployment, monitoring, evaluation, governance.
Evaluation (evals) →
Systematic testing of an AI system against a curated set of inputs to measure quality on the dimensions the business cares about — accuracy, faithfulness, safety, format.
Guardrails →
Runtime checks that intercept LLM inputs and outputs to enforce policy — blocking PII leakage, prompt-injection attempts, off-topic queries, unsafe responses.
More in this category
All 62 terms, in plain language
Sovereign AI, RAG, agentic AI, IDP, MLOps and the regulations that shape enterprise AI.