Broadcast & Observability

Send AI traces and logs to 15+ observability destinations including Datadog, Langfuse, and custom webhooks.

Overview

AllRoutes Broadcast sends real-time traces of every AI request to your observability stack. Connect one or more destinations and AllRoutes streams request metadata, latency, cost, token usage, and optionally prompt/response content to your preferred tools.

Supported Destinations

AllRoutes supports 15+ broadcast destinations out of the box:

Destination	Type	Description
Datadog	APM	Send traces as Datadog APM spans
Langfuse	LLM Observability	Full prompt/response tracing with Langfuse
LangSmith	LLM Observability	Trace and evaluate with LangChain's LangSmith
Braintrust	LLM Observability	Log and evaluate AI outputs
Arize	ML Observability	Monitor model performance and drift
Comet	ML Experiment Tracking	Track AI experiments and metrics
Grafana	Metrics	Push metrics to Grafana Cloud or self-hosted
New Relic	APM	Full-stack observability integration
Sentry	Error Tracking	Track AI errors and exceptions
OpenTelemetry	Standard	Export via OTEL protocol to any OTEL-compatible backend
Amazon S3	Storage	Archive raw traces to S3 buckets
Weights & Biases	ML Tracking	Log to W&B for experiment tracking
BigQuery	Data Warehouse	Stream to Google BigQuery for analytics
Snowflake	Data Warehouse	Stream to Snowflake for analytics
Webhooks	Custom	Send JSON payloads to any HTTP endpoint

Configuration

Configure broadcast destinations in the dashboard under Settings > Broadcast, or via the API:

curl -X POST https://api.allroutes.ai/v1/broadcast/destinations \
  -H "Authorization: Bearer allroutes_mgmt_..." \
  -H "Content-Type: application/json" \
  -d '{
    "type": "langfuse",
    "config": {
      "public_key": "pk-lf-...",
      "secret_key": "sk-lf-...",
      "host": "https://cloud.langfuse.com"
    },
    "sampling_rate": 1.0,
    "privacy_mode": false
  }'

Configuration Fields

Field	Type	Required	Description
`type`	string	Yes	Destination type (e.g., `langfuse`, `datadog`, `webhook`)
`config`	object	Yes	Destination-specific configuration (API keys, endpoints, etc.)
`sampling_rate`	float	No	Fraction of requests to trace (0.0-1.0, default: 1.0)
`privacy_mode`	boolean	No	When `true`, omit prompt/response content from traces
`filter_api_keys`	boolean	No	Strip API key values from traced metadata (default: `true`)
`enabled`	boolean	No	Enable or disable this destination (default: `true`)

Sampling Rate

Control the volume of traces sent to each destination:

1.0 -- trace every request (100%)
0.1 -- trace 10% of requests (random sampling)
0.0 -- disabled

Sampling is applied per-destination, so you can send 100% of traces to Langfuse for debugging while sending only 10% to BigQuery for cost analytics.

Privacy Mode

When privacy_mode is enabled, broadcast payloads include metadata only:

Included: request ID, model, provider, latency, token counts, cost, finish reason, cache status, error codes

Excluded: message content (prompts and responses), system prompts, tool call arguments

This is useful for production environments where you need observability without exposing sensitive data.

API Key Filtering

By default, AllRoutes strips API key values from broadcast payloads. The filter_api_keys setting controls this:

true (default) -- API keys in metadata are replaced with [REDACTED]
false -- API keys are included in broadcast payloads (use with caution)

Per-Request Trace Metadata

Add custom metadata to individual requests using the trace field:

{
  "model": "gpt-4o",
  "messages": [{"role": "user", "content": "Hello!"}],
  "trace": {
    "user_id": "usr_123",
    "session_id": "sess_456",
    "feature": "chat-widget",
    "environment": "production",
    "version": "2.1.0"
  }
}

All trace fields are forwarded to broadcast destinations as metadata tags, enabling filtering and grouping in your observability tools.

Session Grouping

Use the session_id field to group related requests into a conversation or session:

{
  "model": "gpt-4o",
  "messages": [{"role": "user", "content": "Hello!"}],
  "session_id": "sess_456"
}

Destinations like Langfuse and LangSmith will group traces by session, making it easy to view entire conversations.

Trace Payload Format

Each broadcast event includes:

{
  "event": "completion",
  "request_id": "req_abc123",
  "timestamp": "2026-04-05T14:30:00Z",
  "model": "gpt-4o",
  "provider": "openai",
  "latency_ms": 1234,
  "tokens": {
    "prompt": 28,
    "completion": 156,
    "total": 184
  },
  "cost_usd": 0.0032,
  "cache_status": "miss",
  "finish_reason": "stop",
  "status_code": 200,
  "metadata": {
    "user_id": "usr_123",
    "session_id": "sess_456"
  }
}

When privacy_mode is false, the payload also includes prompt and response fields with the full message content.

Webhook Configuration

For custom integrations, use the webhook destination to receive JSON payloads at your endpoint:

{
  "type": "webhook",
  "config": {
    "url": "https://your-app.com/api/ai-traces",
    "headers": {
      "Authorization": "Bearer your-webhook-secret"
    },
    "method": "POST"
  },
  "sampling_rate": 1.0
}

Webhooks are delivered with exponential backoff retry on failure (up to 3 retries).

Managing Destinations

# List all destinations
curl https://api.allroutes.ai/v1/broadcast/destinations \
  -H "Authorization: Bearer allroutes_mgmt_..."

# Update a destination
curl -X PATCH https://api.allroutes.ai/v1/broadcast/destinations/:id \
  -H "Authorization: Bearer allroutes_mgmt_..." \
  -d '{"sampling_rate": 0.5}'

# Delete a destination
curl -X DELETE https://api.allroutes.ai/v1/broadcast/destinations/:id \
  -H "Authorization: Bearer allroutes_mgmt_..."