AI Testing Guide for QA Engineers: How to Test AI-Powered Applications

Why AI testing needs a different oracle

Traditional assertions work for deterministic rules such as access control or JSON structure. Generated answers may be phrased differently while remaining useful, or sound fluent while being wrong. Define observable dimensions: factuality, relevance, completeness, groundedness, safety, tone, and task success.

Build a representative evaluation dataset

Create versioned cases from real product intents, known risks, edge cases, and previously fixed failures. Each case needs the input, context, expected criteria, prohibited behavior, and useful metadata.

• Typical and high-value intents
• Ambiguous, incomplete, noisy, and supported multilingual prompts
• Boundary lengths and unusual formatting
• Known hallucination and safety risks
• Expected refusal or escalation cases
• Different documents, domains, and permission levels

Test factuality, relevance, and hallucination

Factuality asks whether claims are correct. Groundedness asks whether they are supported by the provided source. Relevance asks whether the response addresses the need. These dimensions can fail independently.

Automated graders can prioritize review, but they also make mistakes. Calibrate them against human judgments and inspect disagreements.

Prompt variation, consistency, and adversarial inputs

Vary wording, order, spelling, tone, whitespace, and irrelevant context while preserving intent. Results need not be identical, but critical facts, permissions, and safety boundaries should remain stable.

• Direct and indirect prompt injection attempts
• Instructions embedded in retrieved documents or uploaded text
• Conflicting system, user, and content instructions
• Long context with the key fact in different positions
• Unicode, markup, code blocks, and delimiter confusion
• Requests to reveal hidden instructions or another user’s data

Safety, bias, fairness, and privacy

Translate policy into testable behavior: content to refuse, content to support safely, escalation paths, data retention, and protected operations. For fairness, compare equivalent cases that differ only in a relevant demographic attribute and investigate material disparities.

• No sensitive data in prompts, logs, analytics, or output without authorization
• Tenant and user boundaries hold across retrieval and conversation state
• Safety refusals are useful and reveal no restricted content
• Benign requests are not unnecessarily blocked
• High-impact or ambiguous cases receive human review

Latency, cost, failures, and fallbacks

Measure time to first useful feedback, total latency, failure rate, and token or request consumption under representative loads. Cost is a quality constraint when runaway context or retries can make a feature unsustainable.

Observability and regression testing

Capture model and template versions, retrieval identifiers, tool outcomes, latency, token usage, safety decisions, and user-visible errors without storing unnecessary sensitive content. Compare candidate behavior with an accepted baseline across the versioned dataset.

case_id: rag-policy-017
criteria:
  - cites_retrieved_policy: true
  - invents_policy_clause: false
  - acknowledges_missing_information: true
metrics:
  latency_ms_max: 6000

Testing AI chat interfaces

• Conversation creation, history, retry, stop, and deletion behavior
• Streaming order, partial failure, duplicate chunks, and reconnects
• Clear distinction between user, retrieved, and model content
• Safe Markdown, links, code, tables, and HTML rendering
• Context retention within a chat and isolation across users
• Keyboard, screen-reader announcements, focus, and long-response scrolling

Test whether the interface communicates uncertainty without using disclaimers as a substitute for correct behavior.

Testing RAG systems in practice

Retrieval-augmented generation has at least two quality problems: finding the right evidence and using it faithfully. Diagnose them separately.

Question
  → query transformation
  → retrieval and access filtering
  → ranking/context assembly
  → generation
  → citations and response

• Relevant document appears in top results
• Unauthorized documents never enter results or context
• Citations point to the supporting passage
• Conflicting and outdated sources follow defined precedence
• No-result and low-confidence states are honest
• Retrieved malicious instructions do not control the system

A practical AI feature test plan

1. Map deterministic components, probabilistic behavior, users, harms, and objectives.
2. Define quality dimensions and release thresholds.
3. Build versioned evaluation and holdout datasets.
4. Automate deterministic checks and stable rubric signals.
5. Run sampled human review with disagreement handling.
6. Test safety, privacy, failure, latency, and cost boundaries.
7. Monitor production signals and turn incidents into regression cases.