Migrate this project's LLM inference to Sail (https://sailresearch.com).
Sail docs to consult as you work:
- MCP server: https://docs.sailresearch.com/mcp (connect to it if you support MCP)
- Full docs as plain text: https://docs.sailresearch.com/llms-full.txt
- Key pages: https://docs.sailresearch.com/models (catalog),
https://docs.sailresearch.com/pricing (per-tier rates),
https://docs.sailresearch.com/service-tiers,
https://docs.sailresearch.com/support (API feature matrix)
Sail is drop-in compatible with the OpenAI Responses and Chat Completions
APIs and the Anthropic Messages API, all served from
https://api.sailresearch.com. Keep whichever request shape this code
already uses. Sail's Messages API supports system prompts, tool calling, and
streaming for agentic use. Prompt caching is automatic on every Sail API, so
`cache_control` breakpoints are accepted and have no effect; Messages reports
cache hits in `usage.cache_read_input_tokens` (see /support). The exact base
URL differs by SDK (see step 4).
If you can ask the user questions, ask whenever a step below is ambiguous
instead of guessing. If you can't, make the best-supported choice and flag
it in your final report.
1. Survey the current setup. Find every place this project calls an LLM:
SDK clients, raw HTTP calls, framework configs, env vars, and docs. For
each call site record the provider, API shape, model, and features used
(streaming, tool calls, structured outputs, images).
2. Choose replacement model(s). For each model currently in use, pick the
closest match from https://docs.sailresearch.com/models, comparing
capability tags, context window, and what the model is known to be good
at. If a current model is not one Sail serves, research it (web search
if available) to understand its strengths before choosing. If multiple
Sail models are plausible, ask the user. Otherwise pick the best fit
and explain the choice in your report. Check the /support page for any
features this code uses that Sail doesn't serve, and flag them.
3. Choose a service tier per call site. Omitting service_tier gives
low-latency inference by default, at lower prices than traditional inference
providers for many models. Workloads that can wait save more with balanced
or flex. The live
https://docs.sailresearch.com/service-tiers and
https://docs.sailresearch.com/pricing pages are the source of truth for
public tiers, token prices, and queueing behavior. If you can't fetch
them, use this summary and decide how long each workload can wait:
- asap: no queueing delay; targets 50-100 TPS; foreground-only, not
compatible with background=True or the Batch API. For chat interfaces
and realtime interactivity.
- balanced: queueing of up to 5 minutes for the first turn in a run, then
seconds for follow-up turns of an in-flight run; targets 30-50 TPS;
streaming, foreground, and background-compatible. For subagents,
frequent automations and routines, RL rollouts, evals, and coding agents.
- flex: the lowest prices; queueing of up to 1 hour for the first turn in a
run (guaranteed), then seconds for follow-up turns of an in-flight run;
targets 10-20 TPS; the most generous capacity and highest rate limits;
background-only: send it with background=True on the Responses API or
through the Batch API. For proactive long-horizon agents, background
agents and tasks, rollouts, automations and routines, large-scale evals,
high parallelism work, and offline processing.
Sail uses longest-prefix matching to recognize follow-up turns of an
in-flight run; they are prioritized and see minimal queueing. Experienced
TPS varies by model and workload.
If the workload's latency tolerance isn't obvious from the code, ask
the user. Omit service_tier when the default low-latency behavior is
acceptable. Set it explicitly for balanced or flex, or pin asap when the
request must fail instead of using a fallback tier.
Confirm the chosen tier is available for the chosen model on
https://docs.sailresearch.com/pricing. Some models are Flex-only.
4. Make the changes wherever the client is configured or called:
- base URL: use https://api.sailresearch.com/v1 for OpenAI-compatible
clients (Responses and Chat Completions). For the Anthropic SDK, use
the bare host https://api.sailresearch.com (e.g. set
ANTHROPIC_BASE_URL=https://api.sailresearch.com). The SDK appends
/v1/messages itself, so a /v1 base URL would resolve to /v1/v1/messages
and 404
- API key: read from the SAIL_API_KEY environment variable. Never hardcode
a key or paste a literal key value into the code
(the Anthropic SDK can pass it as `api_key`; `auth_token` also works)
- model: the Sail model(s) chosen in step 2
- service_tier: the tier(s) chosen in step 3. Sail accepts only asap,
balanced and flex, so replace or remove any other value the call site
already sends. The OpenAI and Anthropic SDKs type service_tier with
their own values, so a type checker may reject it. When it does,
silence only that line: add `// @ts-expect-error` on the line above it
in TypeScript, or end the line with `# type: ignore` in Python
- add background=True for flex (required) or very long-running requests;
background requests use balanced or flex, since they do not support asap
Update env var names, .env.example files, config templates, and any
README/docs references. Do not change prompts, tools, or business logic.
5. Estimate the savings. Compare the published per-1M-token list prices
(input, cached input, and output) of the previous model(s) against the
chosen Sail model(s) at the chosen service tier(s) from
https://docs.sailresearch.com/pricing. Research current provider list
prices if you don't know them. State the comparison as a simple table
and an approximate overall multiplier (e.g. "roughly 6x cheaper per
token"). Do not present this as a precise bill forecast.
6. Verify. Run the project's tests. Then make one real smoke request through
the new configuration: if SAIL_API_KEY is already set, use it; if not,
walk the user through creating a key at
https://app.sailresearch.com/api-keys and setting SAIL_API_KEY, then run
the smoke request once they have. Don't ask them to paste the key to you.
Have them export it in their own shell.
Finish with a short migration report: call sites changed; model mapping
with rationale; service tier(s) with rationale; the price comparison
from step 5; and anything that needs human follow-up (unsupported
features, ambiguous choices, untested paths). Close by telling the user
exactly where to set SAIL_API_KEY for their setup (locally and in their
production/deployment environment) so the migrated code can authenticate.