Error Handling

Production patterns for catching, retrying, and recovering from AllRoutes API errors -- with typed examples for every SDK.

Overview

Robust AI applications anticipate three kinds of failures:

Client errors (4xx) -- something is wrong with your request; retrying without changes won't help
Transient errors (429, 5xx) -- the upstream provider hiccupped; auto-retry usually fixes it
Network errors -- TCP/TLS failures, DNS issues, client-side timeouts

This guide shows how to detect each, when to retry, and how to fall back gracefully without burning credits on doomed requests.

Error Envelope

Every error response uses the OpenAI-compatible envelope:

{
  "error": {
    "type": "invalid_request_error",
    "code": "model_not_found",
    "message": "The model 'gpt-5o' does not exist or you do not have access to it.",
    "param": "model",
    "allroutes_request_id": "req_abc123"
  }
}

The allroutes_request_id field is unique and lets the support team trace any request end-to-end.

Status Code Reference

Status	Type	Retry?	Common Causes
`400`	`invalid_request_error`	No	Malformed body, missing required fields
`401`	`authentication_error`	No	Invalid or revoked API key
`402`	`insufficient_credits` / `budget_exhausted`	No	Top up or wait for budget reset
`403`	`permission_error`	No	Key lacks required scope
`404`	`not_found_error`	No	Unknown model, file, or resource
`408`	`timeout_error`	Yes	Provider didn't respond in time
`409`	`conflict_error`	No	Duplicate name, already-deleted resource
`413`	`request_too_large`	No	Request body or input over limit
`422`	`validation_error`	No	Schema-valid but semantically invalid
`429`	`rate_limit_error`	Yes (with backoff)	RPM, TPM, or daily quota
`499`	`client_disconnect`	N/A	Client closed before response complete
`500`	`internal_server_error`	Yes	Gateway bug; rare
`502`	`bad_gateway`	Yes	Upstream provider error
`503`	`service_unavailable`	Yes	Provider overloaded
`504`	`gateway_timeout`	Yes	Provider exceeded gateway timeout

SDK-Level Auto-Retry

All three official SDKs auto-retry on 429, 408, and 5xx with exponential backoff:

SDK	Default Retries	Backoff
Python	2	1s, 2s
Node.js	2	1s, 2s
Go	2	1s, 2s

Override per-client:

client = AllRoutesClient(api_key=KEY, max_retries=5)

const client = new AllRoutes({ apiKey: KEY, maxRetries: 5 });

client := allroutes.NewClient(KEY, allroutes.WithMaxRetries(5))

If you've already implemented your own retry layer, set max_retries=0 to avoid double-retry.

Typed Error Classes

Python

import httpx
from allroutes import AllRoutesClient

client = AllRoutesClient()

try:
    response = client.chat.completions.create(model="gpt-4o", messages=msgs)
except httpx.HTTPStatusError as e:
    body = e.response.json().get("error", {})
    code = body.get("code")
    request_id = body.get("allroutes_request_id")

    if e.response.status_code == 401:
        raise RuntimeError("Invalid API key") from e
    elif e.response.status_code == 402:
        # Budget exhausted -- escalate, don't retry
        notify_ops(f"Budget exhausted: {body.get('message')}")
        raise
    elif e.response.status_code == 429:
        retry_after = e.response.headers.get("retry-after", "60")
        raise RuntimeError(f"Rate limited; retry after {retry_after}s") from e
    elif e.response.status_code >= 500:
        # SDK already retried -- this means provider is genuinely down
        fallback_to_static_response()
    else:
        raise
except httpx.TransportError as e:
    # Network failure, not an HTTP error
    log.error(f"Network error reaching AllRoutes: {e}")
    raise

Node.js

import AllRoutes, {
  APIError,
  RateLimitError,
  AuthenticationError,
  PermissionDeniedError,
  BadRequestError,
  InternalServerError,
} from "@allroutes/sdk";

try {
  const completion = await client.chat.completions.create({ model: "gpt-4o", messages });
} catch (err) {
  if (err instanceof AuthenticationError) {
    throw new Error("Invalid API key");
  }
  if (err instanceof RateLimitError) {
    const retryAfter = err.headers?.["retry-after"];
    console.warn(`Rate limited, retry after ${retryAfter}s`);
    throw err;
  }
  if (err instanceof APIError && err.status === 402) {
    notifyOps(`Budget exhausted: ${err.message}`);
    throw err;
  }
  if (err instanceof InternalServerError) {
    return fallbackToStaticResponse();
  }
  throw err;
}

Go

resp, err := client.Chat.Completions.Create(ctx, req)
if err != nil {
    var apiErr *allroutes.APIError
    if errors.As(err, &apiErr) {
        switch apiErr.StatusCode {
        case 401:
            return fmt.Errorf("invalid API key: %w", err)
        case 402:
            notifyOps(apiErr.Message)
            return err
        case 429:
            return fmt.Errorf("rate limited; retry after %s: %w", apiErr.RetryAfter, err)
        case 500, 502, 503, 504:
            return fallbackToStaticResponse()
        default:
            return fmt.Errorf("API error %d: %s", apiErr.StatusCode, apiErr.Message)
        }
    }
    return fmt.Errorf("network error: %w", err)
}

Application-Level Patterns

1. Model Fallback Chain

Use the built-in models array instead of try/catch loops:

response = client.chat.completions.create(
    models=["gpt-4o", "claude-sonnet-4-20250514", "gemini-2.0-flash"],
    messages=msgs,
)

The gateway tries each in order; you only see an exception if all fail.

2. Circuit Breaker

For provider-level outages, the gateway already implements circuit-breaking (Routing). At the application layer, add one for AllRoutes itself:

import time

class CircuitBreaker:
    def __init__(self, threshold=5, cooldown_s=30):
        self.failures = 0
        self.opened_at = None
        self.threshold = threshold
        self.cooldown = cooldown_s

    def call(self, fn, *args, **kwargs):
        if self.opened_at and time.time() - self.opened_at < self.cooldown:
            raise RuntimeError("Circuit open")
        try:
            result = fn(*args, **kwargs)
            self.failures = 0
            self.opened_at = None
            return result
        except Exception:
            self.failures += 1
            if self.failures >= self.threshold:
                self.opened_at = time.time()
            raise

3. Idempotency for Retries

For non-idempotent ops (top-ups, key creation), add a stable idempotency key:

curl -X POST https://api.allroutes.ai/v1/credits/topup \
  -H "Authorization: Bearer allroutes_sk_..." \
  -H "Idempotency-Key: txn-2026-04-05-user-123" \
  -d '{"amount_usd": 50.00}'

Subsequent requests with the same Idempotency-Key within 24h return the original response without re-executing.

4. Stream Disconnects

If a stream disconnects mid-response, the upstream provider call is not auto-resumed. Catch the disconnect and decide whether to retry or surface partial output:

try {
  for await (const chunk of stream) {
    fullText += chunk.choices[0]?.delta?.content ?? "";
  }
} catch (err) {
  if (fullText.length > 0) {
    // Surface partial response to user, retry only if needed
    return { content: fullText, partial: true };
  }
  throw err;
}

Logging Best Practices

Every error response includes allroutes_request_id. Log it. Always.

log.error(
    "AllRoutes call failed",
    extra={
        "request_id": body.get("allroutes_request_id"),
        "status": e.response.status_code,
        "code": body.get("code"),
        "model": req["model"],
    },
)

When you open a support ticket, include the request ID -- the team can trace the exact request in <30 seconds.

Common Mistakes

Retrying 4xx -- 400/401/402/403/404 will never succeed on retry. Validate inputs and budgets before retry logic kicks in.
Tight retry loops -- always backoff. The SDKs do this automatically; don't disable it.
Burning budget on rate-limit -- a 429 doesn't charge you, but if you catch it and immediately retry, you can lock yourself out for hours. Respect Retry-After.
Swallowing 5xx silently -- fall back gracefully, but ensure your monitoring sees the error so you know providers are flaky.
Ignoring partial streams -- a streaming response that ends with error mid-stream still has usable text in fullText.