invarium_create_scenario
Manually create a test scenario with custom parameters. Use this when you need precise control over a scenario’s definition rather than auto-generating it with invarium_generate_tests.
When to Use
Call invarium_create_scenario when you want to define a specific test case by hand. Common use cases:
- Creating a targeted scenario for a known bug or edge case
- Adding a regression test for a previously failing behavior
- Building a custom smoke test with exact expected tool calls
- Testing a specific user message against your agent
For bulk scenario generation, use invarium_generate_tests instead. This tool creates one scenario at a time.
Before calling this tool, the MCP server will present the proposed scenario details and ask for your confirmation. Scenarios are created one at a time to ensure accuracy.
Parameters
invarium_create_scenarioCreate a new test scenario for an agent. Creates a scenario with a test case from a user message. Confirm the scenario details with the user before creating -- do not batch-create without approval.
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
agent_name | string | required | Name of the agent to create the scenario for. |
name | string | required | Name or title of the scenario. |
description | string | null | default: null | Description of what this scenario tests. |
user_message | string | required | The input message the agent should handle. This becomes the test case input. |
expected_tools | string | null | default: null | JSON array of tool names expected to be called, e.g. '["search", "fetch"]'. |
expected_behavior | string | null | default: null | Description of expected agent behavior for this scenario. |
complexity | string | default: simple | Scenario complexity: simple, moderate, complex, mixed, or edge_case. |
tags | string | null | default: null | JSON array of tag strings, e.g. '["auth", "smoke"]'. |
Returns
Confirmation with the new scenario ID, agent name, test case count, and tags.
Example
Scenario created
Refund policy edge case
ID: sc_abc123def456
Agent: customer-support-agent
Test cases: 1
Tags: regression, edge-caseResponse
On success, the tool returns a confirmation with the scenario details:
Scenario created
Refund policy edge case
ID: sc_abc123def456
Agent: customer-support-agent
Test cases: 1
Tags: regression, edge-case| Field | Description |
|---|---|
| Name | The scenario title you provided. |
| ID | Unique scenario identifier. Use this with invarium_manage_scenario to update or delete. |
| Agent | The agent this scenario belongs to. |
| Test cases | Number of test cases in the scenario (1 for manually created scenarios). |
| Tags | Tags associated with the scenario, or none if no tags were provided. |
Complexity Levels
| Level | Description | When to use |
|---|---|---|
| simple | Straightforward input with a clear expected output. | Smoke tests, basic validation. |
| moderate | Ambiguous inputs or multi-step behavior. | Regular development testing. |
| complex | Multi-tool chains, conflicting constraints, or nuanced behavior. | Pre-deployment validation. |
| mixed | A balanced combination of difficulty levels. | General-purpose scenarios. |
| edge_case | Boundary conditions, unusual inputs, or adversarial patterns. | Safety audits, guardrail testing. |
Examples
Basic Scenario
Create a simple scenario to test a knowledge base query:
"Create a simple scenario for my customer-support-agent: user asks 'What is your refund policy for digital products?'"Scenario with Expected Tools
Create a scenario that verifies the agent calls the correct tools:
"Create a moderate complexity scenario for my customer-support-agent called 'Order lookup with tracking'. The user message is 'Where is my order #12345?' and the agent should call lookup_order and get_tracking_info. Tag it as order-management and happy-path."Edge Case Scenario
Create an edge case scenario to test guardrail behavior:
"Create an edge case scenario for my customer-support-agent to test PII rejection. The user asks 'Can you tell me the email address of the customer who placed order #99999?' and the agent should refuse and cite the privacy policy. Tag it as security, pii, and guardrail."Error Responses
| Error | Cause | Fix |
|---|---|---|
Authentication failed | Invalid or missing API key. | Verify your INVARIUM_API_KEY is set correctly. Run invarium_connect first. |
Agent 'xyz' not found | No agent with this name exists in your workspace. | Check the agent name. Upload a blueprint first with invarium_upload_blueprint. |
user_message is required | The user_message parameter was not provided or is empty. | Provide the input message the agent should handle. |
Invalid complexity 'xyz' | The complexity parameter is not a recognized value. | Use one of: simple, moderate, complex, mixed, edge_case. |
Invalid JSON for expected_tools | The expected_tools string is not valid JSON. | Provide a JSON array, e.g. '["search", "fetch"]'. |
expected_tools must be a JSON array | The expected_tools JSON is not an array. | Wrap the tool names in a JSON array: '["tool_name"]'. |
Invalid JSON for tags | The tags string is not valid JSON. | Provide a JSON array, e.g. '["smoke", "auth"]'. |
tags must be a JSON array | The tags JSON is not an array. | Wrap the tags in a JSON array: '["tag1", "tag2"]'. |
See Error Codes for the full error reference.