Routers allow you to distribute traffic across different model variants within a route based on percentage weights. This is ideal for A/B testing, gradual migrations, or splitting traffic across multiple providers.
How it Works
Routes are defined directly on the Router. Within each route, you can define multiple variants with weights. When a route is selected (either through conditional routing or as the default route), the router selects a variant based on weights.
Important: Weights must sum to exactly 100 within each route. They are not normalized - each route’s variants are weighted independently. For example, if Route A has variants with weights [70, 30] and Route B has variants with weights [80, 20], these are independent - Route A’s weights don’t affect Route B’s weights.
Example Configuration
curl --request POST \
--url https://api.inworld.ai/router/v1/routers \
--header 'Authorization: Basic <your-api-key>' \
--header 'Content-Type: application/json' \
--data '{
"name": "routers/ab-test-router",
"routes": [
{
"route": {
"route_id": "ab-test-route",
"variants": [
{
"variant": {
"variant_id": "variant-a",
"model_id": "openai/gpt-5"
},
"weight": 80
},
{
"variant": {
"variant_id": "variant-b",
"model_id": "anthropic/claude-opus-4-6"
},
"weight": 20
}
]
},
"condition": {
"cel_expression": "true"
}
}
],
"defaultRoute": {
"route_id": "default",
"variants": [
{
"variant": {
"variant_id": "default-variant",
"model_id": "openai/gpt-5"
},
"weight": 100
}
]
}
}'
Model IDs are specified as strings in the format "provider/model" (e.g., "openai/gpt-5").
Weights must sum to exactly 100 within each route. They are not normalized and are independent per route. For example:
- Route 1 with variants [70, 30] ✓ (sums to 100)
- Route 2 with variants [80, 20] ✓ (sums to 100)
- Route 1 with variants [70, 20] ✗ (sums to 90, will fail validation)
Each route’s weights are calculated independently - a weight of 70 in one route does not relate to weights in other routes. User Stickiness
If you specify a user field in your request, the router ensures the same user always gets the same variant (sticky routing):
curl --request POST \
--url https://api.inworld.ai/v1/chat/completions \
--header 'Authorization: Bearer <your-api-key>' \
--header 'Content-Type: application/json' \
--data '{
"model": "inworld/ab-test-router",
"messages": [{"role": "user", "content": "Hello"}],
"user": "user-123"
}'
Use Cases
1. A/B Testing
Send 90% of traffic to your stable model (GPT-5) and 10% to a new candidate (Claude Opus 4) to compare performance and user satisfaction in production.
2. Provider Redundancy
Split traffic 50/50 between two different providers of the same model (e.g., Llama 3 on Groq vs. Llama 3 on Together AI) to mitigate provider-specific rate limits.
3. Gradual Migration
When moving to a new model version, start with a 1% weight and slowly increase it as you verify the new model’s output quality.
4. Conditional Weighted Routing
Combine conditional routing with weighted variants to route based on metadata, then split traffic within that route:
{
"route": {
"route_id": "premium-tier",
"variants": [
{
"variant": {
"variant_id": "gpt5",
"model_id": "openai/gpt-5"
},
"weight": 70
},
{
"variant": {
"variant_id": "claude",
"model_id": "anthropic/claude-opus-4-6"
},
"weight": 30
}
]
},
"condition": {
"cel_expression": "tier == \"premium\""
}
}
