Evaluation (evals)

What Evaluation (evals) means in practice

Evaluation is the practice of systematically testing an AI system against a curated set of inputs and scoring the outputs on the dimensions the business actually cares about. For RAG: context precision, context recall, answer faithfulness, answer relevance. For classification: standard precision, recall, F1, calibration. For generation: a custom SME-built rubric scored either by humans or by a strong LLM as judge. The discipline that distinguishes a serious deployment from a demo is running the evals on every change — model update, prompt change, retrieval-pipeline change — and blocking deployment on regressions. Teams that ship without evals discover quality regressions in production; teams with evals catch them before merge.