Skip to main content
POST
Run Scenarios

Authorizations

X-API-KEY
string
header
required

Your API key for authentication.

x-org-id
string
header
required

Org impersonation via x-org-id. Superadmins may impersonate any org; admins may impersonate their direct sub-organizations. Enter the target organization ID to act as that org.

Body

application/json
agent_id
string
required

ID of the Agent to run the simulation on

Required string length: 24
scenario_ids
string[] | null

IDs of the simulation scenarios being run in this simulation. Needed if scenario_set_id is not provided.

Required string length: 24
scenario_set_id
string | null

ID of the scenario set to run all scenarios from. Cannot be used together with scenario_ids.

Required string length: 24
simulated_lead_llm_args
AiSettingsModel · object

The model to use for the user

num_trials
integer
default:1

The number of trials to run the simulation on.

Required range: 1 <= x <= 32
max_steps
integer
default:100

The maximum number of steps to run the simulation.

Required range: 1 <= x <= 5000
max_errors
integer
default:5

The maximum number of tool errors allowed in a row.

Required range: x >= 1

Response

Successful Response

The results of a simulation run

_id
string
required
account_id
string
required

Account ID that owns this simulation run

agent_id
string
required

ID of the Agent being simulated

scenario_ids
string[]
required

IDs of the Simulation Scenarios being run

info
SimulationInfo · object
required

Information.

total_runs
integer
required

The total number of simulation runs in this batch

scenario_set_id
string | null

Scenario set that was used for this simulation

created_at
string<date-time>
completed_at
string<date-time> | null

The end time of the simulation.

completed_runs
integer
default:0

The number of completed simulation runs

final_score
number | null

The final score of the simulation. Computed as the average reward over all scenarios.

pass_hat_ks
Pass Hat Ks · object | null

Average score in trial/pass k

avg_agent_cost
number | null

The average cost of the agent over all scenarios.