invarium_generate_tests

invarium_generate_tests

Start generating behavioral test cases for your agent. Triggers the Invarium Scenario Generator and returns immediately with a generation_id. Use invarium_get_tests to check status and retrieve results.

Parameters

Name	Type	Required	Description
`agent_name`	string	required	Name of the agent to generate tests for. Must have a blueprint uploaded first.
`count`	int	default: 10	Number of test cases to generate.
`complexity`	string	default: mixed	Test complexity: simple, moderate, complex, or mixed.

Returns

Generation ID string. Use invarium_get_tests to check status and retrieve results.

Example

Test generation started
  Generation ID: gen_abc123def456
  Agent: customer-support-agent
  Count: 10
  Complexity: mixed

Use invarium_get_tests to check status and retrieve results.

Usage

Call this tool after uploading a blueprint. It starts asynchronous test generation and returns immediately with a generation ID.

invarium_generate_tests(
  agent_name='customer-support-agent',
  count=10,
  complexity='mixed'
)

Then retrieve the results with invarium_get_tests.

Generation typically takes 10-30 seconds. For larger test counts (30+), it may take up to a minute.

Complexity levels

Level	Description	When to use
simple	Straightforward scenarios with clear inputs and expected outputs.	Smoke testing, initial validation.
moderate	Scenarios with ambiguous inputs, edge cases, or multi-step workflows.	Regular development testing.
complex	Adversarial inputs, conflicting constraints, multi-tool chains, and boundary conditions.	Pre-deployment validation, safety audits.
mixed	A balanced mix of simple, moderate, and complex tests.	Recommended for most use cases.

Error responses

Error	Cause	Fix
`Agent not found: customer-support-agent`	No blueprint has been uploaded for this agent name.	Upload a blueprint first with `invarium_upload_blueprint`.
`Invalid complexity value`	The `complexity` parameter is not a recognized value.	Use one of: `simple`, `moderate`, `complex`, `mixed`.
`Count must be between 1 and 50`	Use a value between 1 and 50.	Use a value between 1 and 50.
`Rate limit exceeded`	Too many generation requests in a short period.	Wait a moment and try again. See Rate Limits.

Was this page helpful?

invarium_upload_blueprint invarium_get_tests