← Back to Docs

Python SDK

Planned Python SDK for AllRoutes. Until it ships, use the OpenAI Python client pointed at https://api.allroutes.ai/v1.

Not yet published. The native allroutes Python package on this page is on the roadmap but is not on PyPI yet. Use the OpenAI Python client pointed at https://api.allroutes.ai/v1 for now -- see the snippet directly below. The rest of this document previews the surface the native SDK will ship with.

Use the OpenAI Client Today

pip install openai

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["ALLROUTES_API_KEY"],
    base_url="https://api.allroutes.ai/v1",
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)

Installation (planned)

pip install allroutes  # not yet published

Requires Python 3.8+.

Quick Start

from allroutes import AllRoutesClient

client = AllRoutesClient(api_key="allroutes_sk_...")

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Explain quantum computing in one paragraph."}],
)

print(response.content)

OpenAI Compatibility

AllRoutes uses the same request/response format as OpenAI. Switching from the OpenAI Python SDK requires minimal changes:

# Before (OpenAI)
from openai import OpenAI
client = OpenAI(api_key="sk-...")

# After (AllRoutes)
from allroutes import AllRoutesClient
client = AllRoutesClient(api_key="allroutes_sk_...")

# The rest of your code stays the same
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)

Use any supported provider by changing the model name:

# Anthropic
response = client.chat.completions.create(
    model="claude-sonnet-4-20250514",
    messages=[{"role": "user", "content": "Hello from Claude!"}],
)

# Google
response = client.chat.completions.create(
    model="gemini-2.0-flash",
    messages=[{"role": "user", "content": "Hello from Gemini!"}],
)

Streaming

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a short story about a robot."}],
    stream=True,
)

for chunk in stream:
    print(chunk.delta_content, end="", flush=True)
print()

Each StreamChunk has:

  • delta_content -- the new text fragment
  • finish_reason -- None until the final chunk (then "stop", "length", etc.)
  • model -- the model that generated the chunk

Embeddings

# Single input
resp = client.create_embedding(
    model="text-embedding-3-small",
    input="The quick brown fox jumps over the lazy dog",
)
vector = resp["data"][0]["embedding"]
print(f"Dimensions: {len(vector)}")

# Batch input
resp = client.create_embedding(
    model="text-embedding-3-small",
    input=["First sentence", "Second sentence", "Third sentence"],
)
for item in resp["data"]:
    print(f"Index {item['index']}: {len(item['embedding'])} dims")

Credits

credits = client.get_credits()
print(f"Balance: ${credits['balance_usd']:.2f} USD")
print(f"Lifetime spend: ${credits['lifetime_spend_usd']:.2f} USD")
print(f"Auto-replenish: {credits['auto_replenish']}")

Models

models = client.list_models()
for model in models:
    print(f"{model['id']} by {model['provider']}")

Provider Keys (BYOK)

keys = client.list_provider_keys()
for key in keys:
    print(f"{key['provider']}: {key['label']} (active: {key['is_active']})")

Presets

presets = client.list_presets()
for preset in presets:
    print(f"{preset['name']} -> {preset['model']}")

Configuration

ParameterTypeDefaultDescription
api_keystr(required)Your AllRoutes API key
base_urlstrhttps://api.allroutes.aiAPI base URL
timeoutfloat120.0Request timeout in seconds
max_retriesint2Max retries for failed requests
client = AllRoutesClient(
    api_key="allroutes_sk_...",
    base_url="https://custom-gateway.example.com",
    timeout=60.0,
    max_retries=3,
)

Context Manager

Use the client as a context manager to ensure proper connection cleanup:

with AllRoutesClient(api_key="allroutes_sk_...") as client:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}],
    )
    print(response.content)
# Connection is automatically closed here

Error Handling

import httpx
from allroutes import AllRoutesClient

client = AllRoutesClient(api_key="allroutes_sk_...")

try:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}],
    )
except httpx.HTTPStatusError as e:
    if e.response.status_code == 401:
        print("Invalid API key")
    elif e.response.status_code == 429:
        print("Rate limit exceeded, try again later")
    else:
        print(f"API error {e.response.status_code}: {e.response.text}")
except httpx.TransportError as e:
    print(f"Network error: {e}")

The client automatically retries on transient errors (5xx and 429) with exponential backoff.

Response Types

CompletionResponse

FieldTypeDescription
contentstrThe generated text
modelstrModel that produced the response
finish_reasonstrWhy generation stopped
usageUsageToken usage statistics
request_idstrUnique request identifier

Usage

FieldTypeDescription
prompt_tokensintTokens in the prompt
completion_tokensintTokens generated
total_tokensintTotal tokens consumed

Environment Variable

The SDK automatically reads ALLROUTES_API_KEY from the environment if no api_key is passed:

export ALLROUTES_API_KEY="allroutes_sk_..."

client = AllRoutesClient()  # reads from environment

Async Client

For asyncio applications, use AsyncAllRoutesClient:

import asyncio
from allroutes import AsyncAllRoutesClient

async def main():
    async with AsyncAllRoutesClient() as client:
        response = await client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": "Hello!"}],
        )
        print(response.content)

        # Async streaming
        stream = await client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": "Stream me a poem."}],
            stream=True,
        )
        async for chunk in stream:
            print(chunk.delta_content, end="", flush=True)

asyncio.run(main())

See Also