Chat Completions

Complete reference for the POST /v1/chat/completions endpoint, including all parameters, streaming, and AllRoutes-specific extensions.

Endpoint

POST https://api.allroutes.ai/v1/chat/completions

Authentication

Authorization: Bearer allroutes_sk_...
Content-Type: application/json

Request Body

Core Fields

Parameter	Type	Required	Description
`model`	string	Yes	Model identifier (e.g., `gpt-4o`, `claude-sonnet-4-20250514`, `gemini-2.0-flash`)
`messages`	array	Yes	Array of message objects with `role` and `content`
`stream`	boolean	No	Enable streaming via Server-Sent Events (default: `false`)

Sampling Parameters

Parameter	Type	Default	Description
`temperature`	float	Model default	Sampling temperature (0-2). Lower is more deterministic.
`max_tokens`	integer	Model default	Maximum number of tokens to generate
`top_p`	float	1.0	Nucleus sampling: consider tokens with cumulative probability up to `top_p`
`top_k`	integer	--	Limit sampling to top K tokens (supported by some providers)
`frequency_penalty`	float	0	Penalize tokens by frequency (-2 to 2)
`presence_penalty`	float	0	Penalize tokens by presence (-2 to 2)
`repetition_penalty`	float	--	Alternative repetition penalty (some providers)
`min_p`	float	--	Minimum probability threshold for sampling
`seed`	integer	--	Seed for deterministic sampling (where supported)
`stop`	string or array	--	Stop sequences to halt generation
`logit_bias`	object	--	Map of token IDs to bias values

Function / Tool Calling

Parameter	Type	Description
`tools`	array	List of tool definitions (function calling)
`tool_choice`	string or object	Control tool use: `"none"`, `"auto"`, `"required"`, or `{type: "function", function: {name: "..."}}`
`parallel_tool_calls`	boolean	Allow multiple tool calls in a single response

Structured Output

Parameter	Type	Description
`response_format`	object	Force output format: `{"type": "json_object"}` or `{"type": "json_schema", "json_schema": {...}}`

Reasoning / Thinking

Parameter	Type	Description
`reasoning`	object	Configure reasoning/thinking tokens (Anthropic, DeepSeek, etc.)
`include_reasoning`	boolean	Legacy flag to include reasoning in response
`interleaved_thinking`	boolean	Enable interleaved thinking for supported models

AllRoutes-Specific Fields

Parameter	Type	Description
`models`	array	Fallback model array. Tries each in order if the previous fails.
`provider`	object	Provider routing preferences (see Smart Routing)
`plugins`	array	Plugins to apply: `[{"id": "web"}, {"id": "response-healing"}]`
`web_search_options`	object	Configuration for the web search plugin
`cache_control`	object	Anthropic-style per-block prompt caching configuration
`preset`	string	Name or ID of a saved preset to apply
`guardrail`	string	Guardrail profile ID to enforce on this request
`trace`	object	Custom metadata passed to broadcast/observability destinations
`session_id`	string	Group requests into a session for analytics
`route_explain`	boolean	Return verbose routing explanation in response headers
`transforms`	array	Message transforms to apply before sending to provider
`route`	string	Routing hint (e.g., `"fallback"`)
`metadata`	object	Arbitrary key-value metadata for logging
`user`	string	End-user identifier for per-user analytics
`debug`	object	Debug configuration for verbose logging

Output Modalities

Parameter	Type	Description
`modalities`	array	Output types: `["text"]`, `["text", "image"]`
`image_config`	object	Image generation settings when image modality is enabled

Stream Options

Parameter	Type	Description
`stream_options`	object	Configure stream behavior (e.g., include usage in final chunk)

Message Format

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"},
    {"role": "assistant", "content": "Hi there! How can I help?"},
    {"role": "user", "content": "What is 2+2?"}
  ]
}

Supported roles: system, user, assistant, tool.

Response Format

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1714000000,
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The answer is 4."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 24,
    "completion_tokens": 8,
    "total_tokens": 32
  }
}

Model Fallbacks

Use the models array to define an ordered list of fallback models. AllRoutes tries each model in sequence if the previous one fails:

{
  "models": ["gpt-4o", "claude-sonnet-4-20250514", "gemini-2.0-flash"],
  "messages": [{"role": "user", "content": "Hello!"}]
}

When using models, the model field is ignored.

Streaming (SSE)

Set stream: true to receive Server-Sent Events. Each event is a JSON object prefixed with data: :

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" there"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":10,"completion_tokens":5,"total_tokens":15}}

data: [DONE]

AllRoutes-Specific Response Headers

Header	Description
`X-AllRoutes-Request-ID`	Unique request identifier
`X-AllRoutes-Provider`	Provider that served the request
`X-AllRoutes-Model`	Resolved model identifier
`X-AllRoutes-Cache`	`hit` or `miss` (if caching is enabled)
`X-AllRoutes-Cache-Savings`	Cost saved via cache hit (USD)
`X-AllRoutes-Latency-Ms`	Total request latency in milliseconds
`X-AllRoutes-Cost`	Request cost in USD
`X-RateLimit-Limit`	Rate limit ceiling for the key
`X-RateLimit-Remaining`	Remaining requests in the current window
`X-RateLimit-Reset`	Unix timestamp when the rate limit resets

Example: Full Request

curl https://api.allroutes.ai/v1/chat/completions \
  -H "Authorization: Bearer allroutes_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Summarize the theory of relativity."}
    ],
    "temperature": 0.7,
    "max_tokens": 1000,
    "stream": false,
    "plugins": [{"id": "web"}],
    "provider": {
      "order": ["openai", "azure"],
      "allow_fallbacks": true
    }
  }'

Error Codes

Status	Meaning
`400`	Invalid request body (bad parameter, malformed messages)
`401`	Missing or invalid API key
`402`	Insufficient credits or per-key budget exhausted
`403`	Key lacks `chat:completions` scope or model not allowed
`404`	Unknown model ID
`408`	Request timeout (provider did not respond)
`429`	Rate limit exceeded -- check `Retry-After` header
`499`	Client closed the connection mid-stream
`500/502/503`	Upstream provider error -- AllRoutes auto-retries on idempotent retries

See Error Handling for patterns and the canonical error envelope.