Fine-Tuning

Create, monitor, and deploy fine-tuned models across OpenAI, Together, Fireworks, and other supported providers.

Overview

The Fine-Tuning API submits a training job to a provider that supports custom training (OpenAI, Together AI, Fireworks, Mistral, Cohere) and exposes the resulting model on the standard /v1/chat/completions endpoint. AllRoutes handles dataset upload, job orchestration, and inference routing -- you don't have to learn each provider's bespoke API.

Fine-tuning works best when you have:

50-1000+ examples of the input/output behavior you want
A consistent system prompt across examples
Output formats (JSON schemas, tone, brand voice) that are hard to express in prompts

If you can solve the task with prompt engineering and few-shot examples, do that first -- fine-tuning is more expensive and less flexible.

Step 1: Upload Training Data

Training data is a JSONL file, one example per line, in OpenAI chat format:

{"messages": [{"role": "system", "content": "You are a JSON-only assistant."}, {"role": "user", "content": "Cat or dog?"}, {"role": "assistant", "content": "{\"answer\":\"cat\"}"}]}
{"messages": [{"role": "system", "content": "You are a JSON-only assistant."}, {"role": "user", "content": "Apple or pear?"}, {"role": "assistant", "content": "{\"answer\":\"apple\"}"}]}

Upload it via the Files API:

curl https://api.allroutes.ai/v1/files \
  -H "Authorization: Bearer allroutes_sk_..." \
  -F purpose=fine-tune \
  -F [email protected]

The response includes a file_id (e.g., file_abc123) that the next step needs.

Step 2: Create a Fine-Tuning Job

POST https://api.allroutes.ai/v1/fine_tuning/jobs

Request Body

Field	Type	Required	Description
`model`	string	Yes	Base model to fine-tune (e.g., `gpt-4o-mini-2024-07-18`, `meta-llama/Llama-3.1-8B-Instruct`)
`training_file`	string	Yes	`file_id` from Step 1
`validation_file`	string	No	Optional validation split for early stopping
`hyperparameters`	object	No	`{"n_epochs": 3, "learning_rate_multiplier": 1.0, "batch_size": "auto"}`
`suffix`	string	No	Custom suffix for the resulting model name
`seed`	integer	No	Reproducible training

Example

curl https://api.allroutes.ai/v1/fine_tuning/jobs \
  -H "Authorization: Bearer allroutes_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini-2024-07-18",
    "training_file": "file_abc123",
    "hyperparameters": {"n_epochs": 3},
    "suffix": "support-bot"
  }'

Response

{
  "id": "ftjob_xyz789",
  "object": "fine_tuning.job",
  "model": "gpt-4o-mini-2024-07-18",
  "training_file": "file_abc123",
  "status": "queued",
  "hyperparameters": {"n_epochs": 3, "batch_size": 4, "learning_rate_multiplier": 1.0},
  "fine_tuned_model": null,
  "created_at": 1714000000,
  "estimated_finish": 1714003600
}

Step 3: Monitor the Job

# Get current status
curl https://api.allroutes.ai/v1/fine_tuning/jobs/ftjob_xyz789 \
  -H "Authorization: Bearer allroutes_sk_..."

# Stream events (SSE)
curl https://api.allroutes.ai/v1/fine_tuning/jobs/ftjob_xyz789/events \
  -H "Authorization: Bearer allroutes_sk_..." \
  -H "Accept: text/event-stream"

Job States

Status	Meaning
`queued`	Waiting for provider capacity
`validating_files`	Schema validation in progress
`running`	Training in progress
`succeeded`	Done; `fine_tuned_model` field is populated
`failed`	Training failed; check `error` field
`cancelled`	Cancelled via the API or dashboard

Step 4: Use the Fine-Tuned Model

When the job succeeds, the fine_tuned_model field becomes a regular model identifier you can pass to /v1/chat/completions:

response = client.chat.completions.create(
    model="ft:gpt-4o-mini-2024-07-18:org:support-bot:abc123",
    messages=[{"role": "user", "content": "Where is my order?"}],
)

Fine-tuned models behave like any other model -- streaming, tool calls, plugins, and caching all work.

List, Cancel, Delete

# List jobs
curl https://api.allroutes.ai/v1/fine_tuning/jobs \
  -H "Authorization: Bearer allroutes_sk_..."

# Cancel a running job
curl -X POST https://api.allroutes.ai/v1/fine_tuning/jobs/ftjob_xyz789/cancel \
  -H "Authorization: Bearer allroutes_sk_..."

# Delete the resulting model
curl -X DELETE https://api.allroutes.ai/v1/models/ft:gpt-4o-mini-2024-07-18:org:support-bot:abc123 \
  -H "Authorization: Bearer allroutes_sk_..."

Supported Base Models

Provider	Base Models
OpenAI	`gpt-4o-mini-2024-07-18`, `gpt-4o-2024-08-06`, `gpt-3.5-turbo-1106`
Together AI	Llama 3.1 8B/70B, Mistral 7B, Mixtral 8x7B, Qwen 2 7B
Fireworks	Llama 3.1 8B/70B, Mixtral 8x7B, custom Hugging Face models
Mistral	`mistral-small`, `open-mistral-7b`
Cohere	`command-r`, `command-r-plus`

Pricing

Fine-tuning is billed by the provider in two parts:

Training tokens -- one-time cost based on dataset size × n_epochs
Inference tokens -- per-token cost on the resulting model (typically 2-3x the base model rate)

AllRoutes adds the standard platform fee on training and inference, or 0% with BYOK. See Models for the live pricing of each base model.

Best Practices

Start small -- 100 high-quality examples beat 10,000 noisy ones
Hold out a validation set -- set validation_file for early stopping and overfit detection
Iterate the system prompt -- a fine-tuned model still uses the system prompt at inference; keep it identical between training and runtime
Pin the base model version -- always include the date suffix (e.g., 2024-07-18) so future base-model rotations don't invalidate your tunes
Use a suffix -- makes the resulting model identifier readable in dashboards and logs

Troubleshooting

Symptom	Likely Cause	Fix
`failed: invalid_format`	JSONL has malformed line(s)	Re-validate with `jq -c < file.jsonl`
`failed: token_limit_exceeded`	Examples exceed the base model's context	Truncate or split long examples
Slow training	Large dataset on busy provider	Try a different provider via the `provider` field
Model performs worse than base	Overfit; too many epochs	Lower `n_epochs` to 2 or add validation file