Responses API

OpenAI's stateful conversation endpoint with built-in tool execution and conversation persistence.

Overview

The Responses API is the stateful counterpart to Chat Completions. Instead of sending the full message history on every turn, the server stores conversation state and you reference it by previous_response_id. Built-in tools (web search, code interpreter, file search) execute server-side without an extra round trip.

When to choose Responses over Chat Completions:

You want server-managed conversation state (no client-side history bookkeeping)
You're using OpenAI's built-in web_search, code_interpreter, or file_search tools
You need agentic loops where the model autonomously calls multiple tools across turns

When to stay with Chat Completions:

Multi-provider routing matters more than state (Responses is currently OpenAI-only)
You already have client-side message history
You need streaming text (Chat Completions has wider compatibility)

Endpoint

POST https://api.allroutes.ai/v1/responses

Request Body

Field	Type	Required	Description
`model`	string	Yes	Model identifier (e.g., `gpt-4o`, `gpt-4.1`, `o1`, `o3-mini`)
`input`	string or array	Yes	User message text, or an array of typed input items
`instructions`	string	No	System-style instructions for the response
`previous_response_id`	string	No	Continue from a prior response (server-side state lookup)
`tools`	array	No	Tool definitions; supports built-in `web_search`, `code_interpreter`, `file_search`, plus custom functions
`tool_choice`	string or object	No	`auto`, `required`, `none`, or specific tool
`temperature`	float	No	Sampling temperature
`max_output_tokens`	integer	No	Cap on output tokens
`parallel_tool_calls`	boolean	No	Allow multiple tools per turn (default: `true`)
`store`	boolean	No	Persist response server-side for `previous_response_id` chaining (default: `true`)
`metadata`	object	No	Arbitrary key-value tags

Example: Single-Turn

curl https://api.allroutes.ai/v1/responses \
  -H "Authorization: Bearer allroutes_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "instructions": "You are a concise assistant.",
    "input": "Summarize the theory of relativity in 50 words."
  }'

Response

{
  "id": "resp_abc123",
  "object": "response",
  "created_at": 1714000000,
  "model": "gpt-4o",
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {"type": "output_text", "text": "Relativity says that..."}
      ]
    }
  ],
  "usage": {"input_tokens": 18, "output_tokens": 47, "total_tokens": 65},
  "metadata": {}
}

Continuing a Conversation

curl https://api.allroutes.ai/v1/responses \
  -H "Authorization: Bearer allroutes_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "previous_response_id": "resp_abc123",
    "input": "Can you give an example?"
  }'

The server retrieves the prior response (and the chain it descended from), prepends it to the context, and produces a new turn -- no client-side message bookkeeping needed.

Built-In Tools

curl https://api.allroutes.ai/v1/responses \
  -H "Authorization: Bearer allroutes_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "input": "Latest news on the SpaceX Starship launch",
    "tools": [{"type": "web_search"}],
    "tool_choice": "auto"
  }'

Available built-in tools:

Tool	Purpose
`web_search`	Live web search with citations in `output[].content[].annotations`
`code_interpreter`	Sandboxed Python execution; returns plots and computed values
`file_search`	Vector-search over uploaded Files
`computer_use_preview`	Drive a browser via screenshots and actions (gated preview)

Built-in tools execute server-side -- you receive the final assistant message after all tool calls resolve.

Custom Function Calling

Custom functions work the same way as in chat completions:

{
  "model": "gpt-4o",
  "input": "What's the weather in Tokyo?",
  "tools": [{
    "type": "function",
    "name": "get_weather",
    "parameters": {
      "type": "object",
      "properties": {"location": {"type": "string"}},
      "required": ["location"]
    }
  }]
}

When the model calls a function, the response has output[].type: "function_call". Submit results in the next request:

{
  "model": "gpt-4o",
  "previous_response_id": "resp_xyz789",
  "input": [{
    "type": "function_call_output",
    "call_id": "call_abc",
    "output": "{\"temp\": 72, \"condition\": \"sunny\"}"
  }]
}

Streaming

Set stream: true to receive incremental events as Server-Sent Events:

data: {"type":"response.created","response":{...}}
data: {"type":"response.output_text.delta","delta":"Hello"}
data: {"type":"response.output_text.delta","delta":" there"}
data: {"type":"response.completed","response":{...}}
data: [DONE]

The event types are similar to the Realtime API protocol. See Streaming for SSE-parsing patterns.

Retrieve / List / Delete

# Retrieve
curl https://api.allroutes.ai/v1/responses/resp_abc123 \
  -H "Authorization: Bearer allroutes_sk_..."

# List response chain (input_items)
curl https://api.allroutes.ai/v1/responses/resp_abc123/input_items \
  -H "Authorization: Bearer allroutes_sk_..."

# Delete (clears server-side state)
curl -X DELETE https://api.allroutes.ai/v1/responses/resp_abc123 \
  -H "Authorization: Bearer allroutes_sk_..."

SDK Examples

Python

resp = client.responses.create(
    model="gpt-4o",
    instructions="You are a concise assistant.",
    input="Summarize the theory of relativity in 50 words.",
)
print(resp.output[0].content[0].text)

# Continue
followup = client.responses.create(
    model="gpt-4o",
    previous_response_id=resp.id,
    input="Give me an example.",
)

Node.js

const resp = await client.responses.create({
  model: "gpt-4o",
  input: "Summarize the theory of relativity in 50 words.",
});
console.log(resp.output[0].content[0].text);

Storage and Privacy

When store: true (the default), the request and response are retained server-side for 30 days to enable previous_response_id chaining. To prevent storage for sensitive workloads, set store: false -- this disables conversation continuation but matches Chat Completions' privacy semantics.

For full Zero Data Retention, see Guardrails.

Troubleshooting

Symptom	Cause	Fix
`previous_response_id not found`	Past 30-day retention or response was `store: false`	Re-supply full history via `input` array
Built-in tool not running	Model doesn't support the tool	Switch to `gpt-4o` or `gpt-4.1`
Routes only to OpenAI	Responses is OpenAI-only today	Use Chat Completions for multi-provider routing
Streaming events out of order	Buffering proxy	Set `Accept: text/event-stream` and disable proxy buffering