Skip to main content

​
Overview

Prompt variables allow you to create dynamic message templates in your router configuration. You can define templates with placeholders (like {{variable_name}}) and then provide the actual values when making API requests. This enables you to reuse the same router with different configurations without recreating it.

​
How It Works

  1. Define templates in router: When creating or updating a router, you can include message_templates with variable placeholders using {{variable_name}} syntax
  2. Provide values in requests: When making chat completion requests, pass prompt_variables in the extra_body parameter
  3. Variables are substituted: The router substitutes the variables in the message templates before sending them to the LLM
Variables are only substituted in prompts specified in the router configuration (via the Router CRUD API), not in messages sent with each request.

​
Router Configuration

When creating or updating a router, you can define message_templates in the defaults section with variable placeholders: 
{
    "name": "routers/router-auto",
    "defaults": {
        "optimize_prompt": true,
        "text_generation_config": {
            "max_new_tokens": 512,
            "temperature": 0.7
        },
        "message_templates": [
            {
                "role": "system",
                "content": "My name is {{agent_name}}. I love {{item}}"
            }
        ]
    },
    "routes": [
        {
            "route": {
                "route_id": "route-1",
                "variants": [
                    {
                        "variant": {
                            "variant_id": "variant-1",
                            "model_id": "openai/gpt-5"
                        },
                        "weight": 50
                    },
                    {
                        "variant": {
                            "variant_id": "variant-2",
                            "model_id": "google-ai-studio/gemini-2.5-flash"
                        },
                        "weight": 50
                    }
                ],
                "text_generation_config": {
                    "max_new_tokens": 1024,
                    "temperature": 0.9,
                    "top_p": 0.95
                }
            },
            "condition": {
                "cel_expression": "true"
            }
        }
    ],
    "defaultRoute": {
        "route_id": "default",
        "variants": [
            {
                "variant": {
                    "variant_id": "default-variant",
                    "model_id": "google-ai-studio/gemini-2.5-flash"
                },
                "weight": 100
            }
        ]
    }
}
Model IDs are specified as strings in the format "provider/model" (e.g., "openai/gpt-5"). Each route has a route object and a condition object. Variants have a variant object and weight at the same level. Routes are evaluated in order, and the first matching route is selected. Weights must sum to exactly 100 within each route - they are not normalized and are independent per route.
In this example, the system message template contains two variables:
  • {{agent_name}} - Will be replaced with the agent’s name
  • {{item}} - Will be replaced with the item the agent loves

​
API Request

When making a chat completion request, provide the variable values in the prompt_variables object within extra_body: 
curl --location 'https://api.inworld.ai/v1/chat/completions' \
--header 'Authorization: Bearer <your-api-key>' \
--header 'Content-Type: application/json' \
--data '{
  "model": "inworld/router-auto",
  "messages": [{"role": "user", "content": "Write a Python sorting algorithm"}],
  "extra_body": {
    "prompt_variables": {
      "agent_name": "Jane",
      "item": "ice scream"
    }
  }
}'

​
Result

The router substitutes the variables in the message templates before sending the request to the LLM. The actual messages sent to the LLM will be: 
[
  {
    "role": "system",
    "content": "My name is Jane. I love ice scream"
  },
  {
    "role": "user",
    "content": "Write a Python sorting algorithm"
  }
]
The variables are substituted in the system message before it’s sent to the LLM. The LLM’s response will reflect the substituted values, though it may not repeat the system message verbatim. You can verify substitution is working by checking that the variable values appear in the assistant’s responses.

​
Use Cases

  • Multi-tenant applications: Use the same router for different customers by passing customer-specific variables
  • A/B testing: Test different prompt configurations without recreating routers
  • Dynamic personas: Create character-specific prompts that change based on context
  • Localization: Use variables for language-specific content or cultural adaptations
  • Environment-specific configurations: Pass different values for development, staging, and production

​
Best Practices

  • Use descriptive variable names: Choose clear names like {{agent_name}} instead of {{name1}}
  • Document your variables: Keep track of which variables are expected in your router templates
  • Validate values: Ensure the values you pass match the expected format and context
  • Handle missing variables: Be aware that if a variable is not provided, it may remain as {{variable_name}} in the template
  • Keep templates simple: Avoid overly complex templates with many variables for better maintainability