Output limits and stream lifetime

Three things can end an LLM response before the model decides it is finished: the output ceiling you asked for, the ceiling the model itself enforces, and — on a streamed request — the gateway’s stream lifetime limits. This page documents all three, and how a request that ends early is billed.

Requesting an output ceiling

Endpoint	Field
`/v1/chat/completions`	`max_completion_tokens` (preferred), `max_tokens` (deprecated)
`/v1/messages`	`max_tokens`
`/v1/responses`	`max_output_tokens`

Sunra forwards the ceiling you send to the upstream provider. When a response stops because it hit that ceiling, it reports finish_reason: "length" (Chat Completions and Responses) or stop_reason: "max_tokens" (Messages) — the same signal you would get calling the provider directly.

On Chat Completions, some providers accept only max_tokens and ignore max_completion_tokens — an unbounded completion when you believed you had set a ceiling. For the deepseek/ models the gateway translates max_completion_tokens into max_tokens on your behalf. If you send both, the smaller of the two wins, since either one is an upper bound.Providers that honor max_completion_tokens natively are left alone, so a ceiling you set is enforced exactly once, wherever it is enforced.

Model output ceilings

Each model has its own maximum output length, independent of what you request. Asking for more than a model allows is rejected, not silently reduced — the upstream provider returns 400 with the valid range in the message. For example, deepseek/deepseek-v4-flash and deepseek/deepseek-v4-pro accept up to 393,216 output tokens and reject anything above it:

{
  "error": {
    "message": "Invalid max_tokens value, the valid range of max_tokens is [1, 393216]",
    "type": "invalid_request_error"
  }
}

A 400 is the intended behavior here. A ceiling that is quietly lowered would be indistinguishable from a completion that legitimately ran to its limit: both report finish_reason: "length", and nothing in usage would tell you which happened. Output ceilings are per model and change as providers ship new versions. Treat the 400 as the authoritative answer rather than hard-coding a limit in your client.

Stream lifetime

A streamed request (stream: true) is governed by two limits on the gateway side:

Limit	Value	What it catches
Idle timeout	120 seconds with no bytes from the upstream provider	A provider that has stopped producing output but has not closed the connection
Lifetime ceiling	14 minutes from the start of the upstream response	A stream that is still producing but will not terminate

Neither limit applies to a stream that is actively delivering tokens within the idle window — a long generation is not interrupted just for being long. Non-streaming requests are not subject to these limits; they have their own fixed ceiling of 6 minutes for the whole request.

The abort frame

When either limit fires — or the upstream connection fails mid-stream — the gateway does not drop the connection. It flushes whatever it still holds, sends an error frame in your endpoint’s SSE dialect, and closes the stream cleanly. Everything delivered before the frame is valid output and can be used. On /v1/chat/completions and /v1/responses:

data: {"error":{"message":"Stream aborted by gateway (stream_lifetime_ceiling); partial output above is complete as delivered","type":"gateway_stream_aborted","code":"stream_lifetime_ceiling"}}

data: [DONE]

The [DONE] sentinel is part of the Chat Completions dialect only; Responses streams end after the error frame without one. On /v1/messages, the frame follows the Anthropic dialect — a typed error event, and no [DONE]:

event: error
data: {"type":"error","error":{"type":"gateway_stream_aborted","message":"Stream aborted by gateway (upstream_idle_timeout); partial output above is complete as delivered"}}

type is always gateway_stream_aborted. On the OpenAI-shaped dialects, code names which limit fired:

Code	Retry?	Meaning
`upstream_idle_timeout`	Yes — may be transient	The provider sent nothing for 120 seconds.
`stream_lifetime_ceiling`	Not as-is	The stream reached 14 minutes. The same request will likely reach it again — lower your output ceiling, or split the work into several requests.
`upstream_stream_error`	Yes — may be transient	The connection to the provider failed mid-stream.

A stream that ends without an abort frame ended normally. Because the frame is deterministic, treat its absence together with a truncated stream as a client-side or network problem rather than a gateway abort.

Billing for an aborted stream

An aborted stream is billed for what was delivered to you, never for the full generation the provider may have continued internally.

If the provider had already reported final usage before the abort, that report is billed.
Otherwise the delivered output is settled on a conservative estimate — roughly four characters per token, capped at the output ceiling you requested. The completion records usage_source: "conservative_estimate" for the record.
If nothing was delivered, or the provider reported an explicit error, the request is marked failed and the reserved credits are released. Failed requests are not charged.

Handling this in a client

for line in response.iter_lines():
    if not line.startswith(b"data: "):
        continue
    payload = line[6:]
    if payload == b"[DONE]":
        break

    event = json.loads(payload)
    if error := event.get("error"):
        if error.get("type") == "gateway_stream_aborted":
            # Keep what has been collected so far — it is complete as delivered.
            # error["code"] says whether a retry is worth attempting.
            handle_partial(collected, reason=error["code"])
            break
        raise UpstreamError(error)

    collected.append(event["choices"][0]["delta"].get("content", ""))

The important part is that the partial output is kept rather than discarded. You are billed for it either way, and for long generations it is usually most of the answer.

Overview

Platform

Multimodal

LLM

Client Libraries

Output limits and stream lifetime

Requesting an output ceiling

Model output ceilings

Stream lifetime

The abort frame

Billing for an aborted stream

Handling this in a client

​Requesting an output ceiling

​Model output ceilings

​Stream lifetime

​The abort frame

​Billing for an aborted stream

​Handling this in a client

Requesting an output ceiling

Model output ceilings

Stream lifetime

The abort frame

Billing for an aborted stream

Handling this in a client