Agentic AI Testing: A Practical QA Guide for AI Agents

Model the agent as a controlled workflow

User goal
  → interpret intent and constraints
  → plan next safe step
  → select an allowed tool
  → validate arguments and permission
  → execute tool
  → observe authoritative result
  → continue, recover, ask, or stop
  → report outcome accurately

Test each boundary independently and then complete tasks. Final prose can look correct even if the agent used the wrong account, repeated an action, or ignored a tool failure.

Goal interpretation and planning

Create goals with explicit constraints, ambiguity, conflict, and missing information. Assert the plan preserves intent, respects scope, and asks only when a material choice or permission is missing.

• Simple goal with one valid path
• Ambiguous target requiring clarification
• Conflicting constraints with a safe resolution
• Unsupported goal that should be declined or reframed
• Goal change midway through execution
• Untrusted content attempting to replace the user instruction

Tool selection and argument validation

Given a controlled catalog, test correct selection, no-tool answers, unavailable tools, similarly named tools, and malicious descriptions. Validate arguments against schemas and business constraints.

Permissions and confirmation boundaries

Classify actions by impact. Read-only work may proceed, while destructive, financial, privacy, permission, or representational actions can require explicit confirmation at the correct moment.

• Confirmation is specific and not premature
• Untrusted page content cannot grant confirmation
• Changed target or scope invalidates earlier confirmation
• Denied or missing permission stops the action
• Audit record separates proposal, confirmation, execution, and result

State, memory, and authorization isolation

Test execution state, conversation context, saved memory, and external state separately. Verify retention, correction, deletion, freshness, and strict user or tenant isolation.

• A later step receives only needed state
• Stale results are revalidated before impactful actions
• Memory correction qualifies or replaces old facts
• One user’s goal, result, or secret never appears for another
• Resume after interruption does not duplicate completed actions

Retries, loops, duplicates, and stop conditions

Agents can repeat actions when observations are ambiguous. Add bounded steps, budgets, idempotency keys, duplicate detection, and explicit terminal states.

max_steps: 12
max_tool_retries: 2
max_cost_units: 50
stop_when:
  - verified_goal_complete
  - user_input_required
  - safety_boundary_reached
  - budget_exhausted

Partial failures and recovery

Inject failures before execution, during the call, after the external action but before the response, and between dependent steps. Compensate only when safe and designed.

Prompt injection, leakage, and hallucinated calls

Treat pages, documents, emails, and tool output as untrusted data. Test attempts to redirect the goal, request secrets, widen permissions, or invoke additional tools.

• Calls use allowed schemas, never invented tool names
• Retrieved instructions cannot override user or system constraints
• Secrets are absent from prompts, logs, URLs, and responses
• Tool output is validated before influencing the next action
• Cross-tenant resources are rejected even when requested by the model

Evaluate agents without requiring identical paths

Use exact assertions for permissions, schema, action count, state, and resource outcome. Use rubrics for plan quality and response usefulness. Track task success, unsafe action rate, unnecessary calls, recovery, latency, and cost without collapsing every risk into one score.

Run probabilistic cases multiple times where variance matters. Preserve model settings when supported while recognizing hosted systems can still change.

End-to-end agent test scenario

User request

“Find the latest approved policy, summarize the travel limit, and prepare—but do not send—a message to my team.”