Skip to main content
Reserved TPM is in Beta and available to select pilot customers. To join the waitlist, send us a note from the support form in your Sail dashboard (recommended), or contact support.
With Reserved TPM, you can reserve guaranteed serverless inference capacity ahead of time on your selected model and service tier. You prepay a reservation deposit at your normal pay-per-token pricing in exchange for provisioned throughput, measured in uncached input and output tokens per minute (TPM). Reserved TPM also removes the queueing time associated with Balanced and Flex altogether. Reserved TPM is fully self-serve and available on Pro and Enterprise plans.

When to use Reserved TPM

  • You expect a period of heavy traffic and need guaranteed capacity at a specific time, with as little as one hour’s notice.
  • You want guaranteed capacity on demand, with the same serverless APIs and pay-per-token pricing, without renting or managing dedicated servers.
  • You want to run at Balanced or Flex TPS targets, with reserved capacity that removes their usual queueing time.

How it works

  • You can book Reserved TPM for a start time at least one hour in advance. You will specify:
    • The model and service tier
      • Optional: The LoRA name or ID, if reserving throughput for your LoRA fine-tuned model (note - you must upload your LoRA to Sail first).
    • The duration of the reservation in hours, from 1 hour to 3 days long
    • The reservation start time, from 1 hour to 7 days in advance
    • The uncached input TPM amount to reserve
    • The output TPM amount to reserve
  • Eligible inference requests during your reservation are covered by an SLA guaranteeing the uncached input TPM and output TPM you reserved.

Billing

When you book, Sail charges a reservation deposit based on your reserved uncached input and output TPM, multiplied by the duration in minutes, at your normal pay-per-token pricing. There is no surcharge for Reserved TPM. The deposit is nonrefundable except under the free cancellation policy. All eligible inference usage during your reservation draws from the deposit first. This includes cached input tokens, even though only your projected uncached input and output tokens determine the deposit amount. Your deposit balance and your TPM allowances are separate:
  • The deposit is a monetary balance shared across the reservation. Once it’s exhausted, eligible usage draws from your general credit balance on Pro or is billed to your invoice on Enterprise.
  • Your reserved uncached input and output TPM allowances renew each minute and do not roll over to the next minute. Eligible requests within these allowances keep the Reserved TPM SLA until the reservation ends, even after the deposit runs out. Sail serves requests above these allowances when capacity is available, without the Reserved TPM SLA.
On Pro, Sail deducts the deposit from your credit balance when you book. On Enterprise, Sail bills the deposit on your invoice.

Reserved TPM cost calculator

The cost calculator requires JavaScript. See Pricing for each model’s pay-per-token pricing.

Book and use Reserved TPM

Send POST /v1/reserved-tpm with an Idempotency-Key header:
Retrying with the same Idempotency-Key will return the same reservation. Reusing a key with different fields will return HTTP 409.

Errors

Success

If the Reserved TPM you’ve requested is available, Sail books the reservation and charges the deposit. The response contains the reservation.

Use a reservation

Your inference requests need no changes. While your reservation is active, requests on the matching model and service tier automatically use your reserved capacity.
Using Reserved TPM has the added benefit of removing the request queueing latency that would otherwise be incurred on the Balanced and Flex service tiers.
Don’t use the Batch API for Reserved TPM. Batch requests aren’t eligible for Reserved TPM SLAs.

Track TPM allowances and SLA coverage

During your reservation, use the response headers on Responses, Chat Completions, and Messages requests to see whether each request uses your reserved TPM or runs as normal traffic. X-Sail-Reserved-Tpm-Coverage reports:
  • covered: The request uses your reservation and receives the Reserved TPM SLA.
  • excess: The request doesn’t fit within the remaining uncached input allowance, or the output allowance is already exhausted for this minute. Sail can still serve it when capacity is available, without the Reserved TPM SLA.
The remaining-token headers show your organization’s combined TPM allowances across active reservations on the same model and service tier. They report reserved allowances only, not how much shared capacity is available to your organization. Uncached input counts when a request is admitted. Output counts shortly after a covered request finishes, in the minute it finishes. Both allowances reset at the start of each clock minute (UTC); unused allowances do not roll over. This is not a rolling 60-second window. You can also get a reservation and read current_minute to check its remaining allowances.
Only the response to the original request has the headers:
  • A retry with the same idempotency key doesn’t return them.
  • Polling a background response with GET /v1/responses/{id} doesn’t return them.
  • If Sail can’t count a request’s tokens when it arrives, the response has no headers, and the request runs as normal traffic.
SLA coverage and billing are separate. Both covered and excess usage on the reservation’s model and service tier draw from its deposit first. To check the remaining deposit, get the reservation and read prepayment.

List reservations

List returns your organization’s reservations, newest first, with has_more. limit is 1 to 100 (default 20). To get the next page, pass the last ID as after_id.

Get a reservation

Getting a reservation also returns current_minute: how much of its uncached input TPM and output TPM is left in the current minute, and when the minute resets.
  • current_minute is null when the reservation isn’t running, or when Sail can’t read the counts.
  • A covered request’s output counts shortly after it finishes, in the minute it finishes.
  • If you have more than one reservation running on the same model and service tier, requests use up the one that ends first, then the next.
  • Only getting a single reservation returns current_minute. Booking, listing and cancelling don’t.

Cancel a reservation

You can cancel a booked reservation to release the capacity, any time until it ends. Finishing early does not refund any unspent deposit. Free cancel grace period: you can cancel Reserved TPM within 10 minutes of booking for a full refund of the deposit in Sail credits. Your organization gets one free cancellation in a rolling 24-hour window. Otherwise, the deposit is nonrefundable.
free_cancel_until shows when a free cancel stops being available, or null when the reservation is not refundable. The response’s refund contains amount_cents and a status of pending until the credit is added, then refunded. If nothing is refunded, refund is null.

Reservation details

A reservation looks like:
prepayment shows how much of the reservation’s deposit is used (consumed_cents) and left (remaining_cents), as of as_of. Note that updates to prepayment may continue for up to several hours past the end, because requests admitted near the end of the reservation can still draw from the deposit when they finish.

FAQ

Access and booking

Reserved TPM is currently in waitlisted Beta. Soon, this feature will be open to all users on Pro and Enterprise plans.
Reserved TPM is in Beta, by request. Send us a note from the support form in your Sail dashboard, or contact support.
From 1 hour to 7 days in advance. If you need to book further out, contact support.
From 1 hour to 3 days (72 hours), in whole hours.
Each model and service tier combination has a minimum reservation size, in dollars per hour.
No, reservations can’t be changed after booking. You must cancel and book another reservation instead.
Yes. Book the LoRA’s base model on Flex, with lora set to the LoRA’s name or ID. That reservation covers requests that use that LoRA. A reservation without lora covers only requests that don’t use a LoRA.

Usage and allowances

Yes. Get the reservation: prepayment shows how much of its deposit is left, and current_minute shows how much of its uncached input and output TPM allowances remain this minute.
Check the X-Sail-Reserved-Tpm-Coverage response header. covered means the request used your reservation and receives the Reserved TPM SLA; excess means it ran as normal traffic without that SLA. For a covered request, X-Sail-Reserved-Tpm-Id identifies the reservation it used. See Track TPM allowances and SLA coverage for the headers and when they’re available.
No. The remaining-token headers and current_minute report only your remaining reserved allowances. A falling count means you’ve used some of this minute’s allowance; it doesn’t mean your reservation has become smaller. The allowances reset each minute.Requests above those allowances can still run as normal traffic when capacity is available, without the Reserved TPM SLA. These counts don’t show how much additional shared capacity your organization can receive.
No. It covers only the reservation’s designated model and tier. Requests on other tiers run as normal traffic.
A US-only reservation makes the requests it covers US-only. They run in the US and are billed as US-only, matching the reservation’s price, even without routing.allowed_countries. Still we recommend setting this on the individual requests as well in case usage exceeds your per-minute TPM allowances.

Billing and cancellation

Sail charges a deposit when you book, and eligible usage draws from it first. See Billing for the calculation and how charges work on Pro and Enterprise.
Yes, the deposit is available throughout the reservation, rather than divided into per-minute amounts.For example, with a $100 deposit, eligible usage draws from that $100 until it’s exhausted, regardless of when during the reservation you send the requests.Your uncached input and output TPM allowances renew each minute and do not roll over to the next minute. Sail serves requests above these allowances when capacity is available, without the Reserved TPM SLA.
Further usage draws from your general credit balance on Pro or is billed to your invoice on Enterprise. Eligible requests within your reserved uncached input and output TPM allowances keep the Reserved TPM SLA until the reservation ends. Exhausting the deposit doesn’t reduce those allowances.
Eligible requests continue to run while the reservation has a remaining deposit. Other requests return 402 until you add credits.
Any unspent deposit is nonrefundable. See Billing.
You can optionally cancel your reservation once your workload finishes to release the capacity. If you haven’t used the full reservation deposit, the remaining amount is nonrefundable.
Sail releases the capacity. Cancelling within 10 minutes of booking also refunds the full deposit as Sail credits, if you haven’t already done such a cancellation in the past 24 hours. Any other cancellation is not eligible for a refund.