Blueprint Schema

A blueprint is a JSON document that describes your AI agent — its name, framework, tools, workflows, constraints, and expected behaviors. Invarium uses this information to generate targeted behavioral test cases.

Full template

Here is the complete blueprint schema with all supported fields:

{
  "agent_name": "my-agent",
  "framework": "langchain",
  "description": "A brief description of what the agent does.",
  "model": "gpt-4o",
  "tools": [
    {
      "name": "tool_name",
      "description": "What this tool does.",
      "parameters": {
        "param_name": "param_type"
      },
      "returns": "Description of what the tool returns.",
      "side_effects": "none",
      "error_handling": "retry",
      "rate_limit": "10 req/min"
    }
  ],
  "workflow_chains": [
    {
      "name": "chain_name",
      "description": "What this workflow does.",
      "steps": ["tool_a", "tool_b", "tool_c"]
    }
  ],
  "constraints": [
    "Never share PII with external services",
    "Always cite sources when answering questions"
  ],
  "guardrails": [
    "Reject requests for harmful content",
    "Rate limit to 100 requests per user per hour"
  ],
  "expected_behaviors": [
    "Greet the user before asking clarifying questions",
    "Escalate to human support when confidence is below 0.5"
  ],
  "failure_modes": [
    "Hallucinating information not in the knowledge base",
    "Calling tools with incorrect parameter types"
  ]
}

Field reference

Field	Type	Required	Description
`agent_name`	string	Yes	Unique name for the agent. Used to identify it across all Invarium tools. Must be URL-safe (alphanumeric, hyphens, underscores).
`framework`	string	No	The agent framework. Helps Invarium generate framework-specific test scenarios.
`description`	string	No	Brief description of what the agent does and its purpose.
`model`	string	No	The LLM model the agent uses (e.g., `gpt-4o`, `claude-3-opus`). Informational only.
`tools`	array	Yes	Array of tool definitions. At least one tool is required.
`tools[].name`	string	Yes	Name of the tool. Must be unique within the blueprint.
`tools[].description`	string	Yes	What the tool does. The more detail, the better the generated tests.
`tools[].parameters`	object	No	Key-value map of parameter names to their types (e.g., `{"query": "string", "limit": "int"}`).
`tools[].returns`	string	No	Description of what the tool returns.
`tools[].side_effects`	string	No	What side effects the tool has.
`tools[].error_handling`	string	No	How errors from this tool are handled.
`tools[].rate_limit`	string	No	Rate limit for this tool, as a human-readable string.
`workflow_chains`	array	No	Ordered sequences of tool calls that represent multi-step workflows.
`workflow_chains[].name`	string	No	Name of the workflow chain.
`workflow_chains[].description`	string	No	What this workflow accomplishes.
`workflow_chains[].steps`	array	No	Ordered list of tool names in the chain.
`constraints`	array	No	Rules the agent must always follow. These become test assertions.
`guardrails`	array	No	Safety boundaries and limits. Invarium generates adversarial tests to probe these.
`expected_behaviors`	array	No	Descriptions of correct agent behavior. Used to evaluate test results.
`failure_modes`	array	No	Known failure modes. Invarium prioritizes testing these scenarios.

Validation rules

Framework values

The framework field accepts the following values:

Value	Framework
`langchain`	LangChain / LangGraph
`crewai`	CrewAI
`autogen`	AutoGen
`openai_agents`	OpenAI Agents SDK
`custom`	Custom / other frameworks

If omitted, Invarium defaults to custom.

Side effects values

The side_effects field on tools accepts:

Value	Meaning
`database_write`	Tool writes to a database.
`database_read`	Tool reads from a database.
`api_call`	Tool makes an external API call.
`email_send`	Tool sends an email.
`file_write`	Tool writes to a file.
`file_read`	Tool reads from a file.
`payment`	Tool processes a payment.
`notification`	Tool sends a notification.
`none`	Tool has no side effects.

Tools with write-related side effects (database_write, file_write, payment, email_send) receive more rigorous testing, including scenarios where the agent should refuse to call the tool.

Error handling values

The error_handling field on tools accepts:

Value	Meaning
`retry`	The agent retries the tool call on failure.
`fallback`	The agent falls back to an alternative approach.
`none`	No specific error handling defined.

Confidence scoring

After uploading a blueprint, Invarium assigns a confidence score based on how much information is provided:

Confidence	Criteria
High	Agent has tools with parameters and return types, constraints or guardrails are defined, and at least one workflow chain is present.
Medium	Agent has tools with descriptions, and at least one constraint or expected behavior is defined.
Low	Agent has only a name and basic tool names with no additional detail.

Higher confidence means Invarium can generate more targeted and relevant test cases. To improve confidence:

Add parameters and returns to every tool
Define constraints and guardrails
Add workflow_chains for multi-step processes
Include failure_modes to guide the scenario generator

You do not need a high-confidence blueprint to get started. Even a minimal blueprint produces useful tests. You can iterate and add detail over time.

Was this page helpful?

invarium_sync_results Blueprint Examples