Skip to main content

​
Overview

The extra_body parameter allows you to fine-tune model selection, fallbacks, and optimization preferences. All parameters are optional.

​
Parameters

​
models

An array of model identifiers to consider for selection or use as fallbacks. For specific model requests: These are used as fallback models if the primary model fails. For auto model selection: These define the pool of models to choose from. 
{
  "extra_body": {
    "models": [
      "anthropic/claude-opus-4-6",
      "google-ai-studio/gemini-2.5-pro",
      "openai/gpt-5"
    ]
  }
}

​
ignore

An array of providers or specific models to exclude from selection or fallbacks. You can exclude:
  • Entire providers: "groq" excludes all Groq models
  • Specific models: "groq/llama-3.3-70b-versatile" excludes only that model
{
  "extra_body": {
    "ignore": [
      "groq",                              // Exclude all Groq models
      "groq/llama-3.3-70b-versatile"      // Exclude specific model
    ]
  }
}

​
sort

An array of sorting criteria in priority order. The optimizer uses these to rank and select models. Available criteria:
  • "price" - Cost optimization
  • "latency" - Low latency
  • "throughput" - High throughput
  • "intelligence" - General intelligence/capability
  • "math" - Mathematical reasoning
  • "coding" - Code generation and understanding
{
  "extra_body": {
    "sort": [
      "coding",      // Prioritize coding-optimized models
      "price",       // Then consider cost
      "latency"      // Finally optimize for speed
    ]
  }
}
Format Difference: The sort field in extra_body (for chat completions) uses string arrays like ["price", "latency"]. However, when using sort in router configuration within model_selection, you must use object array format: [{"metric": "SORT_METRIC_PRICE"}, {"metric": "SORT_METRIC_LATENCY"}]. See Create Router for router configuration details.

​
provider

An object that controls provider-level routing when the model is specified without a provider prefix (e.g., "model": "gpt-oss-120b").
FieldTypeDefaultDescription
orderstring[]—Explicit list of providers to try, in order (e.g., ["groq", "fireworks"]).
allow_fallbacksbooleantrueWhether to fall back to the next provider if the current one fails.
{
  "extra_body": {
    "provider": {
      "order": ["groq", "fireworks"],
      "allow_fallbacks": true
    }
  }
}
When no order is specified, the sort criteria determines the order providers are tried. If no sort is specified either, providers are selected by lowest latency. When order is specified, providers are tried in the listed order — sort does not reorder the provider list, but still applies to models fallbacks. See Provider Routing for detailed examples and execution order.

​
metadata

An object containing key-value pairs that can be used for conditional routing with CEL expressions. This is used when routing with Routers that have conditional routes. 
{
  "extra_body": {
    "metadata": {
      "tier": "premium",
      "user_id": "user-123",
      "region": "us-east",
      "domain": "coding",
      "priority": 8
    }
  }
}
The metadata values are evaluated by CEL expressions in your router configuration. For example, a route with "cel_expression": "tier == \"premium\"" will match when metadata.tier is "premium". See Conditional Routing for more details on using metadata for routing.

​
Complete Example

{
  "model": "auto",
  "messages": [{"role": "user", "content": "Write a Python function"}],
  "extra_body": {
    "models": [
      "openai/gpt-5",
      "anthropic/claude-opus-4-6",
      "google-ai-studio/gemini-2.5-pro"
    ],
    "ignore": ["groq"],
    "sort": ["coding", "price", "latency"]
  }
}
This configuration:
  1. Considers only the three specified models
  2. Excludes all Groq models
  3. Prioritizes coding capability, then cost, then latency

​
Best Practices

  • Be specific with models: Limiting the model pool improves selection speed and predictability
  • Use ignore strategically: Exclude providers or models that don’t meet your requirements
  • Order sort by priority: List the most important criteria first
  • Combine criteria: Use multiple sort criteria for balanced optimization