When to use Reserved TPM
- You expect a period of heavy traffic and need guaranteed capacity at a specific time, with as little as one hour’s notice.
- You want guaranteed capacity on demand, with the same serverless APIs and pay-per-token pricing, without renting or managing dedicated servers.
- You want to run at Balanced or Flex TPS targets, with reserved capacity that removes their usual queueing time.
How it works
- You can book Reserved TPM for a start time at least one hour in advance.
You will specify:
- The model and service tier
- Optional: The LoRA name or ID, if reserving throughput for your LoRA fine-tuned model (note - you must upload your LoRA to Sail first).
- The duration of the reservation in hours, from 1 hour to 3 days long
- The reservation start time, from 1 hour to 7 days in advance
- The uncached input TPM amount to reserve
- The output TPM amount to reserve
- The model and service tier
- Eligible inference requests during your reservation are covered by an SLA guaranteeing the uncached input TPM and output TPM you reserved.
Billing
When you book, Sail charges a reservation deposit based on your reserved uncached input and output TPM, multiplied by the duration in minutes, at your normal pay-per-token pricing. There is no surcharge for Reserved TPM. The deposit is nonrefundable except under the free cancellation policy. All eligible inference usage during your reservation draws from the deposit first. This includes cached input tokens, even though only your projected uncached input and output tokens determine the deposit amount. Your deposit balance and your TPM allowances are separate:- The deposit is a monetary balance shared across the reservation. Once it’s exhausted, eligible usage draws from your general credit balance on Pro or is billed to your invoice on Enterprise.
- Your reserved uncached input and output TPM allowances renew each minute and do not roll over to the next minute. Eligible requests within these allowances keep the Reserved TPM SLA until the reservation ends, even after the deposit runs out. Sail serves requests above these allowances when capacity is available, without the Reserved TPM SLA.
Reserved TPM cost calculator
Estimate your deposit and total cost
Estimate your deposit and total cost
The cost calculator requires JavaScript. See Pricing for each model’s pay-per-token pricing.
Book and use Reserved TPM
SendPOST /v1/reserved-tpm with an Idempotency-Key header:
- Base model
- LoRA fine-tune
Reservation fields
Reservation fields
Idempotency-Key will return the same reservation.
Reusing a key with different fields will return HTTP 409.
Errors
HTTP status codes and errors
HTTP status codes and errors
Success
If the Reserved TPM you’ve requested is available, Sail books the reservation and charges the deposit. The response contains the reservation.Use a reservation
Your inference requests need no changes. While your reservation is active, requests on the matching model and service tier automatically use your reserved capacity.Track TPM allowances and SLA coverage
During your reservation, use the response headers on Responses, Chat Completions, and Messages requests to see whether each request uses your reserved TPM or runs as normal traffic.X-Sail-Reserved-Tpm-Coverage reports:
covered: The request uses your reservation and receives the Reserved TPM SLA.excess: The request doesn’t fit within the remaining uncached input allowance, or the output allowance is already exhausted for this minute. Sail can still serve it when capacity is available, without the Reserved TPM SLA.
current_minute
to check its remaining allowances.
Response headers
Response headers
- A retry with the same idempotency key doesn’t return them.
- Polling a background response with
GET /v1/responses/{id}doesn’t return them. - If Sail can’t count a request’s tokens when it arrives, the response has no headers, and the request runs as normal traffic.
prepayment.List reservations
List returns your organization’s reservations, newest first, withhas_more.
limit is 1 to 100 (default 20). To get the next page, pass the last ID as
after_id.
Get a reservation
current_minute: how much of its uncached
input TPM and output TPM is left in the current minute, and when the minute
resets.
current_minuteisnullwhen the reservation isn’t running, or when Sail can’t read the counts.- A covered request’s output counts shortly after it finishes, in the minute it finishes.
- If you have more than one reservation running on the same model and service tier, requests use up the one that ends first, then the next.
- Only getting a single reservation returns
current_minute. Booking, listing and cancelling don’t.
Cancel a reservation
You can cancel abooked reservation to release the capacity, any time until
it ends. Finishing early does not refund any unspent deposit.
Free cancel grace period: you can cancel Reserved TPM within 10 minutes
of booking for a full refund of the deposit in Sail credits. Your organization
gets one free cancellation in a rolling 24-hour window.
Otherwise, the deposit is nonrefundable.
free_cancel_until shows when a free cancel stops being available, or null
when the reservation is not refundable. The response’s refund contains
amount_cents and a status of pending until the credit is added, then
refunded. If nothing is refunded, refund is null.
Reservation details
Reservation object, statuses, and usage
Reservation object, statuses, and usage
prepayment shows how much of the reservation’s deposit is used
(consumed_cents) and left (remaining_cents), as of as_of. Note that
updates to prepayment may continue for up to several hours past the end, because
requests admitted near the end of the reservation can still draw from the
deposit when they finish.FAQ
Access and booking
Who can use Reserved TPM?
Who can use Reserved TPM?
How do I get access?
How do I get access?
How far ahead can I book Reserved TPM?
How far ahead can I book Reserved TPM?
How long can a reservation be?
How long can a reservation be?
What's the smallest reservation?
What's the smallest reservation?
Can I change a reservation?
Can I change a reservation?
Can I reserve TPM for a LoRA fine-tune?
Can I reserve TPM for a LoRA fine-tune?
lora set to the LoRA’s name
or ID. That reservation covers requests that use that LoRA. A
reservation without lora covers only requests that don’t use a LoRA.Usage and allowances
Can I see how much of a reservation I've used?
Can I see how much of a reservation I've used?
prepayment shows how much
of its deposit is left, and current_minute shows how much of its
uncached input and output TPM allowances remain this minute.How can I tell whether a request used my reservation?
How can I tell whether a request used my reservation?
X-Sail-Reserved-Tpm-Coverage response header. covered means
the request used your reservation and receives the Reserved TPM SLA;
excess means it ran as normal traffic without that SLA. For a covered
request, X-Sail-Reserved-Tpm-Id identifies the reservation it used. See
Track TPM allowances and SLA coverage
for the headers and when they’re available.Does a reservation cover other service tiers?
Does a reservation cover other service tiers?
How does a US-only reservation work?
How does a US-only reservation work?
routing.allowed_countries. Still we recommend setting this on the
individual requests as well in case usage exceeds your per-minute TPM
allowances.Billing and cancellation
How does billing work?
How does billing work?
Does the deposit carry across minutes?
Does the deposit carry across minutes?
What happens when usage exceeds the deposit during the reservation?
What happens when usage exceeds the deposit during the reservation?
What if my org's credit balance runs out during a reservation?
What if my org's credit balance runs out during a reservation?
402 until you add credits.What happens to the deposit I don't use?
What happens to the deposit I don't use?
What if my workload finishes early?
What if my workload finishes early?
What happens if I cancel?
What happens if I cancel?