LLM evaluation (evals)
Also known as: evals, LLM evals, AI evals
LLM evaluation, often called evals, is the practice of testing an AI application with a set of realistic inputs and scoring its outputs for accuracy, safety, tone, and other qualities.
Good evals let teams compare models, catch regressions when prompts change, and prove a system is ready to launch. While public benchmarks get headlines, the evals that matter most are the ones built from your own data and users' actual questions.
LLM evaluation (evals), explained at three levels
At home
Evals are like a practice exam for an AI system. You write a set of questions where you already know what a good answer looks like, have the AI answer them all, and mark the results. Because the same exam can be given again after every change, you can tell whether a new prompt or model made things better or quietly worse.
At work
LLM evaluation, or evals, means testing an AI application against a fixed set of realistic inputs and scoring the outputs for accuracy, safety, tone, and format. It is the AI version of a software test suite. Businesses rely on evals to decide whether a tool is ready to launch, to compare vendors, and to catch regressions before customers notice them.
At home
An eval has three parts: test questions, a description of a good answer, and a way to score. Some answers can be checked automatically, like whether code runs or a number matches. Others, like whether an email sounds polite, need judgment, so a person or a second AI grades them against a rubric. Running the same set on several tools gives you a fair, side-by-side comparison on your own tasks.
At work
An eval suite combines a dataset of inputs, expected outputs or grading criteria, and scorers: exact match, code execution, rubric grading, LLM-as-a-judge, or human review. Frameworks such as OpenAI Evals and promptfoo, and platforms such as LangSmith or Braintrust, run the suite and track scores over time. Teams wire evals into continuous integration, so every change to a prompt, model, or retrieval setting triggers a run, and releases are blocked if key scores fall.
At home
Public benchmarks measure narrow skills on fixed questions, and many of those questions have leaked into training data, so models can score well partly from memory. Your own tasks, with your wording and your edge cases, aren't in any benchmark. While leaderboards are a decent first filter, they predict your experience poorly. In my opinion, a quick test with ten of your own real questions tells you more than any leaderboard.
At work
Eval quality hinges on coverage and grader reliability. Benchmark contamination inflates scores, LLM judges carry biases, and small test sets miss rare but costly failures. Agents are harder still, since you must grade multi-step trajectories, not single answers. Offline scores also need linking to real outcomes, such as resolution rates. I'd sample live traffic for review, add every production failure as a new test case, and treat the eval set as a living asset.
Real-world examples
At home
Aisha wanted to choose an AI writing assistant, so she gave three tools the same ten emails to rewrite and scored each result. One tool clearly outperformed the others on her actual work.
At work
Marcus leads AI development at a Toronto insurance company. His team runs 400 test questions through their claims chatbot every time they change a prompt, blocking releases if accuracy drops.
How it compares
| feature | LLM evaluation (evals) | AI Benchmarking |
|---|---|---|
| Built from | Your own tasks and data | Public standard datasets |
| Purpose | Is this system good for our use? | How do models compare generally? |
| Run when | Every change | At model release |
| Risk | Gaps if test set is too small | Contamination and gaming |
Benchmarks compare models. Evals tell you whether your system works.
For analysts
I'd treat an eval set as part of the requirements. Gather real examples and agreed 'good answers' with business users before building, and the project gets a clear finish line.
History & origin
Machine learning has always relied on test sets, but generative AI made scoring harder because there are many acceptable answers. Stanford's HELM project in 2022 evaluated models across many scenarios. OpenAI open-sourced its Evals framework alongside GPT-4 in March 2023. Benchmarks such as SWE-bench, released by Princeton researchers in 2023, tested models on real GitHub issues. Through 2024 and 2025, tools for application-level evaluation spread, including open-source libraries and observability platforms, and 'eval-driven development' became common practice in AI engineering teams. In my view, evals are the difference between an AI demo and an AI product, and most failed projects I've seen skipped them.