Google open-sources AQuA, a tool for finding quality failures in production AI agents

Teams running AI agents on Google Cloud get a way to spot quality regressions that infrastructure monitoring misses, though fixes stay with engineers.

Google has released AQuA, an open-source reference tool that runs beside production AI agents in Google Cloud, reads telemetry from Cloud Trace, Cloud Logging or BigQuery, and samples up to 1,000 recent sessions per run to surface quality regressions that ordinary infrastructure monitoring misses. It groups recurring failures, verifies candidate clusters against full transcripts, and compares failing sessions with an immutable source snapshot taken at deployment to point at likely root causes with file and line citations. In a test on 32 sessions from a multi-agent travel application, it produced 42 findings in nine candidate clusters, and the verifier rejected three, leaving six confirmed issues including seat-availability bypasses and a lost vegan dietary preference. After prompt changes and controlled replay, seat-selection bypasses fell from 15 sessions to two and dietary omissions from seven to zero, though those results came from replays rather than live traffic. AQuA does not apply fixes or open pull requests on its own, and Google acknowledges that sampling can miss rare failures and that replay may not reproduce external state.