Resource
AI Agent Testing Checklist
A practical checklist for testing an AI agent before it reaches real users, data or money. Written by a team that builds agents and tests them — so it covers what actually breaks in production, not theory.
TL;DR: This checklist runs an AI agent through the seven checks that decide whether it survives production: defining what "good" means, catching hallucination, keeping tool calls safe, grounding RAG answers, blocking prompt injection, holding up across multi-turn conversations, and gating every change with evals in CI. Work it top to bottom before your agent touches real users, data or money.
Before you test
- Write down what the agent is supposed to do — and the actions it must never take.
- Collect real inputs and edge cases from actual usage, not just happy-path demos.
- Decide what "good" means per task: correctness, faithfulness, safety, latency, cost.
- Build a labelled golden set from those inputs — your ground truth for every future change.
Behaviour & hallucination
- Grade answers for factual correctness against the golden set.
- Check the agent uses provided context instead of inventing facts.
- Test that it declines questions outside its scope rather than guessing.
- Watch for prompt drift — small wording changes that quietly shift behaviour.
Tool use & actions
- Verify the agent calls the right tool with the right arguments.
- Test that it refuses or confirms before actions touching money, data or production.
- Cover tool failures and timeouts — does it recover or corrupt state?
- Confirm it can't loop or run a destructive action twice.
RAG & retrieval (if applicable)
- Measure faithfulness: are answers grounded in the retrieved context?
- Measure retrieval quality: precision and recall over your corpus.
- Test with questions your documents can't answer — it should say so.
- Re-run evals on every index, chunking or embedding change.
Security: prompt injection & jailbreaks
- Attempt direct injection through user input to override instructions.
- Attempt indirect injection through documents, tools and retrieved content.
- Probe jailbreaks against safety and policy boundaries (OWASP LLM Top 10).
- Confirm an injected instruction can't exfiltrate data or misuse a tool.
Multi-turn & memory
- Test behaviour across long conversations, not just single turns.
- Check the agent doesn't leak or confuse state between sessions or users.
- Verify it recovers gracefully from its own earlier mistakes.
- Confirm memory or history changes don't silently regress answers.
Evals & CI gates
- Wire the eval suite into CI so it runs on every prompt, model or pipeline change.
- Block the merge when a change makes a key metric worse.
- Track scores over time so you can see slow drift, not just pass/fail.
- Keep the golden set versioned and growing as new failures are found.
Get a free audit
Ship the next release without holding your breath.
Tell us what you're building — or what's breaking. We reply within one business day with a concrete plan, not a sales deck.
Or, the direct route
contact@qatestingplus.com