Simulated Users: Testing AI Agents Before Production
Columbia professor Zhou Yu argues single-turn benchmarks break down for multi-turn agents, and offers simulated users as the fix.
Symbolic image: status lights blink on a server rack in a test lab while two monitors beside it run abstract branching diagrams.
Zhou Yu's answer to the 95 percent of AI agents that reportedly never leave the demo stage is simulation: synthetic users that drive multi-turn conversations inside a CI/CD pipeline before real customers ever do.
At a glance
- Zhou Yu teaches computer science at Columbia University and founded Arklex.ai; InfoQ dates the talk to September 7, 2026.
- The talk claims 95 percent of agents stay in demos, naming compliance and reliability as the bottlenecks.
- Each test scenario is assembled from three parts: a user profile, a user goal, and context information.
- Five scoring dimensions are used: helpfulness, coherence, verbosity, relevance, and faithfulness.
- Five failure types are tracked, among them failure to clarify, repetition, and false information.
Zhou Yu opens the QCon AI talk with a number that reframes the argument: 95 percent of AI agents, by this account, never make it past the demo. The blocker is not model capability but evaluation. A team can show a clean conversation on stage and still have nothing to hand a compliance officer about the thousandth call.
What a single-turn benchmark misses
A static benchmark scores one prompt against one response. An agent handling a credit card application does something else: it asks clarifying questions, calls tools, and adapts to answers it has not seen before. The talk's case is that a fixed test set cannot capture that, because the other side of the conversation keeps moving.
Profiles, goals, context
The proposed fix is to point one agent at another. A simulated user, built from three parts — a profile, a goal, and context information — walks the product agent through realistic exchanges. Because those parts recombine, coverage grows without anyone hand-writing each case.
Scoring runs through an LLM judge tied to the specific task goal rather than a generic sense of quality. Five dimensions are graded: helpfulness, coherence, verbosity, relevance, and faithfulness. Failures are then sorted into five behavior types — failure to clarify, repetition, disobedience, false information, and lack of specificity — so a regression points at something a team can actually fix.
Entropy as a coverage signal
Two entropy measures keep the simulated users off the same happy path: the distribution of tool calls, and the transitions between them. Pushing both up drives runs into the rarer branches where agents tend to break. The suite hangs off a CI/CD pipeline so every change is replayed against the scenarios, and production logs feed back into the scenario set as real user behavior drifts.
Where it is used, and what is unverified
The talk lists e-commerce, finance with credit card applications, education with Pearson as the named example, and mortgage underwriting. ArkSim, the tool built around the method, is described as open source and designed to slot into existing pipelines.
This report rests on the InfoQ page alone. The 95 percent figure is the speaker's framing rather than a measurement we checked, and we could not independently confirm how the entropy metrics behave in production systems beyond the examples cited.
FAQ
Why do AI agents get stuck in the demo phase?
The talk points at evaluation, not model quality. Teams can demo a good conversation but cannot show that an agent behaves reliably across many turns, which stalls approval in regulated workflows.
What is simulation-driven testing for AI agents?
One agent plays the customer against the agent under test. The simulated user is assembled from a profile, a goal, and context information, so many conversations can be generated instead of written by hand.
What is ArkSim?
The open-source tool presented in the talk. It runs simulated user conversations against an agent from a CI/CD pipeline and feeds production logs back into the scenario set as user behavior changes.
