Framework Integration

Map your agent framework’s concepts to Invarium blueprints — 5 frameworks supported, plus any custom implementation.

Key Takeaways

✓Invarium is framework-agnostic — any agent with tools can be tested
✓Each framework maps its concepts differently: LangChain chains become workflows, CrewAI tasks become tools
✓The blueprint format is the common language — learn the mapping for your framework

Why It Matters

Framework integration translates your agent framework’s native concepts — decorators, agent classes, crew tasks, function definitions — into the universal blueprint format (YAML via MCP, JSON via Dashboard) that Invarium uses for test generation and behavioral auditing.

Every framework has its own conventions for defining tools, composing workflows, and specifying agent behavior. This guide shows you how to translate those conventions into a blueprint so Invarium can understand and test your agent regardless of how it was built.

The blueprint examples below use JSON format, which is used for Dashboard import. When working via MCP, your IDE assistant generates YAML format with the same schema. The structure and field names are identical — only the syntax differs.

Framework Overview

Invarium recognizes 6 framework values in the framework field. Setting the correct framework helps Invarium tailor test scenarios to framework-specific patterns.

Framework	`framework` value	Agent model	Tool definition	Multi-agent
LangChain	`langchain`	AgentExecutor, LCEL chains	`@tool` decorator, Tool class	LangGraph
CrewAI	`crewai`	Crew of agents with roles	`@tool` decorator	Built-in (crews)
AutoGen	`autogen`	Conversable agents	Function definitions	Built-in (group chat)
OpenAI Agents SDK	`openai_agents`	Agent class	Function tools, `@function_tool`	Handoffs
Custom	`custom`	Any implementation	Any callable	Varies
OpenAI (direct)	`openai`	Direct API usage	Function calling	Manual orchestration

Framework Guides

LangChain

Concept Mapping

LangChain Concept	Blueprint Field	Notes
Agent name	`agent_name`	Use the name you give your agent or chain
`@tool` decorated functions	`tools[]`	One blueprint tool per `@tool` function
Tool class instances	`tools[]`	Map name, description, and args_schema
System prompt rules	`constraints.guardrails[]`	Extract each rule as a separate constraint
System prompt text	`constraints.system_prompt_summary`	Paste or summarize your system prompt
LCEL chain / graph steps	`workflow.chains[]`	Map each chain or graph to a workflow
LangGraph nodes	`workflow.chains[].steps[]`	Each node becomes a step in the workflow

Example: LangChain Agent Blueprint

Given this LangChain agent:

@tool
def search_products(query: str) -> str:
    """Search the product catalog by name or category."""
    # ...
 
@tool
def check_inventory(product_id: str) -> dict:
    """Check current inventory levels for a product."""
    # ...
 
@tool
def place_order(product_id: str, quantity: int, customer_id: str) -> str:
    """Place an order for a product."""
    # ...

The blueprint would be:

{
  "agent_name": "ecommerce-assistant",
  "framework": "langchain",
  "description": "An e-commerce assistant that helps customers search products, check availability, and place orders.",
  "tools": [
    {
      "name": "search_products",
      "description": "Search the product catalog by name or category.",
      "parameters": {
        "type": "object",
        "properties": {
          "query": { "type": "string", "description": "Product name, category, or search terms" }
        },
        "required": ["query"]
      },
      "returns": "List of matching products with name, price, and ID.",
      "side_effects": ["database_read"],
      "error_handling": "retry"
    },
    {
      "name": "check_inventory",
      "description": "Check current inventory levels for a specific product.",
      "parameters": {
        "type": "object",
        "properties": {
          "product_id": { "type": "string", "description": "The unique product identifier" }
        },
        "required": ["product_id"]
      },
      "returns": "Object with product_id, available_quantity, and warehouse_location.",
      "side_effects": ["database_read"],
      "error_handling": "retry"
    },
    {
      "name": "place_order",
      "description": "Place an order for a product on behalf of the customer.",
      "parameters": {
        "type": "object",
        "properties": {
          "product_id": { "type": "string", "description": "The product to order" },
          "quantity": { "type": "integer", "description": "Number of units to order (1-100)" },
          "customer_id": { "type": "string", "description": "The customer placing the order" }
        },
        "required": ["product_id", "quantity", "customer_id"]
      },
      "returns": "Order confirmation with order_id and estimated delivery date.",
      "side_effects": ["payment", "database_write"],
      "requires_confirmation": true,
      "error_handling": "fallback"
    }
  ],
  "constraints": {
    "system_prompt_summary": "You are an e-commerce assistant. Help customers find products, check stock, and place orders.",
    "guardrails": [
      "Always check inventory before placing an order",
      "Never place an order without the customer's explicit confirmation",
      "Do not recommend products that are out of stock",
      "Only discuss products available in our catalog"
    ]
  },
  "workflow": {
    "entry_point": "customer_request",
    "chains": [
      {
        "name": "purchase-flow",
        "steps": [
          "Search for the product in the catalog",
          "Check inventory for the matching product",
          "If in stock, confirm the order details with the customer",
          "Place the order after customer confirmation"
        ],
        "condition": "Customer wants to buy a product"
      }
    ]
  }
}

LangChain Tips

Map each @tool decorated function to a tools[] entry — the docstring becomes the description
Extract constraints from your system prompt — every rule in the system prompt should be a separate constraint
If you use LangGraph, map each node to a step in a workflow.chains[] entry
Include side_effects for any tool that writes data or triggers external actions — search_products reads the database, place_order processes a payment
Set requires_confirmation: true on tools that perform irreversible actions (like placing orders or sending emails)

CrewAI

Concept Mapping

CrewAI Concept	Blueprint Field	Notes
Crew name	`agent_name`	Use the crew’s identifier
Agent roles + tools (across all agents)	`tools[]`	Combine tools from all agents in the crew
Agent backstories / goals / rules	`constraints.guardrails[]`	Each rule from any agent becomes a constraint
Task sequences	`workflow.chains[]`	Map the task sequence to workflow steps
Crew process type (sequential/hierarchical)	`workflow.chains[].steps[]`	Note the execution order

Example: CrewAI Crew Blueprint

Given a CrewAI crew with two agents:

researcher = Agent(
    role="Research Analyst",
    goal="Find accurate market data",
    tools=[search_web, analyze_data],
)
 
writer = Agent(
    role="Content Writer",
    goal="Write clear reports from research data",
    tools=[format_report, check_grammar],
)
 
crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, writing_task],
    process=Process.sequential,
)

The blueprint would be:

{
  "agent_name": "market-research-crew",
  "framework": "crewai",
  "description": "A two-agent crew that researches market data and produces formatted reports. The researcher gathers data, and the writer creates the final report.",
  "tools": [
    {
      "name": "search_web",
      "description": "Search the web for market data, news, and competitor information.",
      "parameters": {
        "type": "object",
        "properties": {
          "query": { "type": "string", "description": "Search terms for market research" }
        },
        "required": ["query"]
      },
      "returns": "List of relevant web results with titles, snippets, and URLs.",
      "side_effects": ["api_call"],
      "error_handling": "retry"
    },
    {
      "name": "analyze_data",
      "description": "Perform statistical analysis on collected market data.",
      "parameters": {
        "type": "object",
        "properties": {
          "data": { "type": "array", "description": "Raw data points to analyze" },
          "analysis_type": { "type": "string", "description": "Type of analysis: trend, comparison, or forecast" }
        },
        "required": ["data", "analysis_type"]
      },
      "returns": "Analysis results with key metrics and insights.",
      "side_effects": ["none"],
      "error_handling": "fallback"
    },
    {
      "name": "format_report",
      "description": "Format research findings into a structured report document.",
      "parameters": {
        "type": "object",
        "properties": {
          "content": { "type": "string", "description": "Raw content to format" },
          "template": { "type": "string", "description": "Report template: summary, detailed, or executive" }
        },
        "required": ["content"]
      },
      "returns": "Formatted report as markdown.",
      "side_effects": ["none"],
      "error_handling": "none"
    },
    {
      "name": "check_grammar",
      "description": "Check report text for grammar and style issues.",
      "parameters": {
        "type": "object",
        "properties": {
          "text": { "type": "string", "description": "Text to check for grammar issues" }
        },
        "required": ["text"]
      },
      "returns": "Corrected text with list of changes made.",
      "side_effects": ["none"],
      "error_handling": "none"
    }
  ],
  "constraints": {
    "system_prompt_summary": "Research market data accurately and produce well-formatted reports with citations.",
    "guardrails": [
      "Research data must come from the web search tool, not from training data",
      "Reports must cite all sources used",
      "Do not include speculative information without labeling it as speculation",
      "Reports must follow the specified template format"
    ]
  },
  "workflow": {
    "entry_point": "user_request",
    "chains": [
      {
        "name": "market-report",
        "steps": [
          "Research agent searches the web for relevant market data",
          "Research agent analyzes the collected data",
          "Writer agent formats the analysis into a report",
          "Writer agent checks grammar and style",
          "Final report is delivered to the user"
        ],
        "condition": "User requests a market research report on a topic"
      }
    ]
  }
}

CrewAI Tips

List tools from all agents in the crew in a single tools[] array — Invarium tests the crew as a whole, not individual agents
Include constraints from each agent’s backstory and goal fields
Map the crew’s task sequence to workflow.chains[].steps[]
Note the process type (sequential vs. hierarchical) — sequential maps directly to ordered steps, hierarchical may require multiple workflow chains
If agents share tools, list each tool once in the blueprint

AutoGen

Concept Mapping

AutoGen Concept	Blueprint Field	Notes
Agent group name	`agent_name`	Use a name for the overall agent system
Function definitions in `llm_config`	`tools[]`	Each function becomes a tool
`system_message` rules	`constraints.guardrails[]`	Extract rules as constraints
`system_message` text	`constraints.system_prompt_summary`	Paste or summarize
Conversation flow	`workflow.chains[]`	Map the expected exchange pattern
GroupChat agents	`tools[]`	List all agents’ tools in one array

Example: AutoGen Agent Blueprint

Given an AutoGen setup:

assistant = AssistantAgent(
    name="code_reviewer",
    system_message="You review code for bugs, security issues, and style.",
    llm_config={"functions": [
        {
            "name": "run_linter",
            "description": "Run a linter on the provided code",
            "parameters": {"code": {"type": "string"}, "language": {"type": "string"}}
        },
        {
            "name": "run_security_scan",
            "description": "Run a security vulnerability scan on the code",
            "parameters": {"code": {"type": "string"}}
        }
    ]}
)

The blueprint would be:

{
  "agent_name": "code-reviewer",
  "framework": "autogen",
  "description": "Reviews code submissions for bugs, security vulnerabilities, and style issues. Uses automated linting and security scanning tools.",
  "tools": [
    {
      "name": "run_linter",
      "description": "Run a code linter to check for syntax errors, style issues, and common bugs.",
      "parameters": {
        "type": "object",
        "properties": {
          "code": { "type": "string", "description": "The source code to lint" },
          "language": { "type": "string", "description": "Programming language: python, javascript, typescript, etc." }
        },
        "required": ["code", "language"]
      },
      "returns": "List of linting issues with line numbers, severity, and descriptions.",
      "side_effects": ["none"],
      "error_handling": "retry"
    },
    {
      "name": "run_security_scan",
      "description": "Run a security vulnerability scan to detect common security issues like SQL injection, XSS, and hardcoded secrets.",
      "parameters": {
        "type": "object",
        "properties": {
          "code": { "type": "string", "description": "The source code to scan" }
        },
        "required": ["code"]
      },
      "returns": "List of security findings with severity, CWE ID, and remediation suggestions.",
      "side_effects": ["none"],
      "error_handling": "retry"
    }
  ],
  "constraints": {
    "system_prompt_summary": "You review code for bugs, security issues, and style. Always run both tools before providing a review.",
    "guardrails": [
      "Always run both the linter and security scan before providing a review",
      "Never modify the submitted code — only report findings",
      "Classify every finding by severity: critical, high, medium, low",
      "Do not approve code with critical security findings"
    ]
  },
  "workflow": {
    "entry_point": "code_submission",
    "chains": [
      {
        "name": "code-review",
        "steps": [
          "Run the linter on the submitted code",
          "Run the security scan on the submitted code",
          "Compile findings from both tools",
          "Provide a summary with severity-ranked findings and an approve/reject recommendation"
        ],
        "condition": "User submits code for review"
      }
    ]
  }
}

AutoGen Tips

Map each function in llm_config.functions to a tools[] entry
Extract constraints from the system_message — each sentence that contains a rule (words like “always”, “never”, “must”) should be a separate constraint
For multi-agent conversations, describe the full conversation flow in workflow.chains[]
If using GroupChat, list all agents’ tools in the single blueprint
AutoGen’s function parameters use a simpler format than JSON Schema — when converting, wrap them in { "type": "object", "properties": {...} }

OpenAI Agents SDK

Concept Mapping

OpenAI Agents SDK Concept	Blueprint Field	Notes
Agent `name`	`agent_name`	Use the agent’s name directly
`@function_tool` functions	`tools[]`	One blueprint tool per function tool
Agent `instructions`	`constraints.guardrails[]`	Extract each numbered rule
Agent `instructions` text	`constraints.system_prompt_summary`	Paste or summarize
Handoffs between agents	`workflow.chains[]`	Each handoff path becomes a workflow

Example: OpenAI Agents SDK Blueprint

Given this OpenAI Agents SDK agent:

from agents import Agent, function_tool
 
@function_tool
def lookup_order(order_id: str) -> str:
    """Look up an order by its ID and return order details."""
    # ...
 
@function_tool
def process_refund(order_id: str, reason: str) -> str:
    """Process a refund for a given order."""
    # ...
 
agent = Agent(
    name="order-support",
    instructions="""You are an order support agent. Follow these rules:
    1. Always look up the order before taking any action
    2. Verify the customer's identity before processing refunds
    3. Never process refunds over $500 without manager approval
    4. Always confirm the refund amount with the customer first""",
    tools=[lookup_order, process_refund],
)

The blueprint would be:

{
  "agent_name": "order-support",
  "framework": "openai_agents",
  "description": "An order support agent that looks up orders and processes refunds with identity verification and approval controls.",
  "tools": [
    {
      "name": "lookup_order",
      "description": "Look up an order by its ID and return order details including status, items, and total.",
      "parameters": {
        "type": "object",
        "properties": {
          "order_id": { "type": "string", "description": "The unique order identifier" }
        },
        "required": ["order_id"]
      },
      "returns": "Order details with status, items, total amount, and customer info.",
      "side_effects": ["database_read"],
      "error_handling": "retry"
    },
    {
      "name": "process_refund",
      "description": "Process a refund for a given order. Requires a reason for the refund.",
      "parameters": {
        "type": "object",
        "properties": {
          "order_id": { "type": "string", "description": "The order to refund" },
          "reason": { "type": "string", "description": "Reason for the refund" }
        },
        "required": ["order_id", "reason"]
      },
      "returns": "Refund confirmation with refund_id and expected processing time.",
      "side_effects": ["payment", "database_write"],
      "requires_confirmation": true,
      "error_handling": "fallback"
    }
  ],
  "constraints": {
    "system_prompt_summary": "You are an order support agent. Look up orders and process refunds with identity verification and approval controls.",
    "guardrails": [
      "Always look up the order before taking any action",
      "Verify the customer's identity before processing refunds",
      "Never process refunds over $500 without manager approval",
      "Always confirm the refund amount with the customer first"
    ]
  },
  "workflow": {
    "entry_point": "customer_request",
    "chains": [
      {
        "name": "refund-flow",
        "steps": [
          "Look up the order by ID",
          "Verify customer identity",
          "Confirm refund amount with the customer",
          "If over $500, escalate for manager approval",
          "Process the refund"
        ],
        "condition": "Customer requests a refund for an order"
      }
    ]
  }
}

OpenAI Agents SDK Tips

Map each @function_tool to a tools[] entry — the docstring becomes the description
Extract constraints from the instructions string — numbered rules map directly to constraints.guardrails[]
If using handoffs between agents, describe each handoff path as a separate workflow.chains[] entry
Include side_effects for any tool that writes data or triggers payments
Set requires_confirmation: true on tools that process payments or perform irreversible actions

Custom Agents

Any Agent Works

If your agent is built with a custom framework or no framework at all, you can still create a blueprint. The key is to identify 3 things:

What functions or APIs can your agent call? — These are your tools[].
What rules should your agent follow? — These are your constraints.
What multi-step processes does your agent perform? — These are your workflow.chains[].

Concept Mapping

Custom Agent Concept	Blueprint Field	Notes
Agent/service name	`agent_name`	Use a descriptive identifier
Callable functions, API endpoints	`tools[]`	One tool per callable
Business rules, system prompt	`constraints`	Any rule the agent should follow
Multi-step procedures	`workflow.chains[]`	Ordered task sequences
N/A	`framework`: `"custom"`	Always set framework to `custom`

Example: Custom REST API Agent

{
  "agent_name": "internal-ops-bot",
  "framework": "custom",
  "description": "An internal operations bot that manages deployments, monitors services, and handles incident response through REST API calls.",
  "tools": [
    {
      "name": "deploy_service",
      "description": "Trigger a deployment for a specified service to a target environment.",
      "parameters": {
        "type": "object",
        "properties": {
          "service_name": { "type": "string", "description": "Name of the service to deploy" },
          "environment": { "type": "string", "description": "Target environment: staging or production" },
          "version": { "type": "string", "description": "Version tag to deploy" }
        },
        "required": ["service_name", "environment", "version"]
      },
      "returns": "Deployment status with deployment_id and estimated completion time.",
      "side_effects": ["api_call", "notification"],
      "requires_confirmation": true,
      "error_handling": "fallback"
    },
    {
      "name": "check_service_health",
      "description": "Check the health status of a running service.",
      "parameters": {
        "type": "object",
        "properties": {
          "service_name": { "type": "string", "description": "Name of the service to check" }
        },
        "required": ["service_name"]
      },
      "returns": "Health status (healthy, degraded, down) with CPU, memory, and latency metrics.",
      "side_effects": ["api_call"],
      "error_handling": "retry"
    },
    {
      "name": "create_incident",
      "description": "Create an incident ticket in the incident management system.",
      "parameters": {
        "type": "object",
        "properties": {
          "title": { "type": "string", "description": "Incident title" },
          "severity": { "type": "string", "description": "Incident severity: P1, P2, P3, or P4" },
          "affected_services": { "type": "array", "description": "List of affected service names" }
        },
        "required": ["title", "severity", "affected_services"]
      },
      "returns": "Incident ticket ID and link to the incident page.",
      "side_effects": ["database_write", "notification"],
      "error_handling": "retry"
    }
  ],
  "constraints": {
    "system_prompt_summary": "You are an internal operations bot. Manage deployments, monitor service health, and handle incident response.",
    "guardrails": [
      "Never deploy to production without explicit user confirmation",
      "Always check service health before deploying",
      "P1 incidents must be created immediately, not queued",
      "Never deploy to production during an active P1 incident",
      "Log all deployment actions for audit purposes"
    ]
  },
  "workflow": {
    "entry_point": "user_request",
    "chains": [
      {
        "name": "deploy",
        "steps": [
          "Check the health of the target service",
          "If deploying to production, request explicit user confirmation",
          "Trigger the deployment",
          "Monitor deployment status and report result"
        ],
        "condition": "User requests a service deployment"
      },
      {
        "name": "incident-response",
        "steps": [
          "Identify affected services",
          "Create an incident ticket with appropriate severity",
          "Notify the on-call team",
          "Monitor for resolution"
        ],
        "condition": "Service health check returns degraded or down"
      }
    ]
  }
}

Custom Agent Tips

Set framework to "custom" when not using a standard framework
Be extra detailed in tool descriptions since there is no framework convention for Invarium to infer from
Include side_effects for every tool that changes state — deploy_service triggers deployments, create_incident writes to a database and sends notifications
Document all implicit rules your agent follows as explicit constraints
If your agent calls external REST APIs, set side_effects to ["api_call"] and document what the API does in the tool description

Common Patterns

Regardless of framework, these patterns apply to every blueprint:

Every Callable is a Tool

If your agent can call it, list it in tools[]. This includes functions, API endpoints, database queries, and external service calls. Each tool needs at minimum a name and description.

Every Rule is a Constraint

System prompts, guardrails, and business rules all become entries in constraints. Extract individual rules from your system prompt — “Always check inventory before ordering” is one constraint, “Never process payments without confirmation” is another. The more specific your constraints, the more targeted your test scenarios.

Every Multi-Step Process is a Workflow

If your agent follows a sequence of steps to complete a task, describe it in workflow.chains[]. Workflows help Invarium test your agent’s end-to-end behavior, not just individual tool calls.

Be Specific About Side Effects

Tools that only read data have "side_effects": ["none"] or "side_effects": ["database_read"]. Tools that write data, send messages, or trigger external systems need their side effects documented with specific values:

Side Effect	When to Use
`database_write`	Tool creates, updates, or deletes database records
`database_read`	Tool queries a database
`api_call`	Tool calls an external API
`email_send`	Tool sends an email
`file_write`	Tool creates or modifies files
`file_read`	Tool reads files from disk
`payment`	Tool processes a financial transaction
`notification`	Tool sends a push notification, SMS, or alert
`none`	Tool has no side effects (pure computation)

Not sure if your blueprint is complete? Upload it and generate a small set of 5-10 test scenarios. If the tests feel generic or miss important behaviors, your blueprint needs more detail in tools, constraints, or workflows.

Create & Upload a Blueprint Generate Test Scenarios