Agent Studio / Governance / Agent Simulation Engine
← Home
Studio Guide

Governance

Agent Simulation Engine

The Agent Simulation Engine invents adversarial test users, runs your agent against them, and gives you a judged pass-rate report, so you find the gaps before a real user does. Open it from Governance → Agent Simulation Engine in the left menu.

1.Run a simulation

  1. Select the agent from the list on the left. Search by name if the list is long.

  2. Choose how many personas to test it against: 3, 5, 8 or 10.

  3. Select Run again.

The engine invents test users, called personas, based on what your agent does, runs the agent with each one, and has a judge score every response. A run usually takes 30 to 90 seconds; the page shows how many live model calls it will make.

Steps that affect the outside world, such as sending an email, posting to Teams or making an HTTP request, are simulated. Nothing is sent for real.

2.Read the results

  • Pass rate: The share of personas the agent handled well, such as “2/5 personas passed (40%). 3 scenario(s) need attention.”
  • Scores: The average score out of 5 for Task Completion, Safety and Tone.
  • Personas: Each persona is marked passed or failed, with its first message and its three scores. Expand a persona to see:
    • Scenario: The situation the persona represents
    • Response: What your agent answered
    • Judge’s note: Why the response passed or failed

The personas are chosen to test the agent from different angles. For a meeting notes agent, for example, they include an Efficient PM with clean notes, a Messy Multispeaker User, an Adversarial Prompt Injector who tries to override the agent’s instructions, an Angry Executive, and an OffTopic Mixed Request.

3.Fix and run again

Read the judge’s note for each failed persona, then change the agent’s instructions, guardrails or workflow to fix the problem. Select Run again to check the fix. When the agent passes, classify it in Responsible AI before you deploy it.

4.Other ways to test

TestUse it to
PlaygroundChat with the draft yourself and check its answers
Test BlockRun one block on its own to check its configuration
Test Cases (Builder toolbar)Rerun inputs you’ve saved, with expected results, after each change
Agent Simulation EngineTest the agent against many invented personas, including difficult ones, and get a pass rate
Compare modelsRun the same message on two or more models in the Playground and see the answers side by side
Agent Simulation Engine — Studio DocumentationAgent Studio