Output limits and stream lifetime

Four things can end an LLM response before the model decides it is finished: the output ceiling you asked for, the ceiling the model itself enforces, the edge timeout that governs every non-streaming request, and — on a streamed request — the gateway’s stream lifetime limits. This page documents all four, and how a request that ends early is billed. It also covers what a streamed request looks like before the provider has responded at all, which is where a long-thinking reasoning model spends most of its time.

Requesting an output ceiling

Endpoint	Field
`/v1/chat/completions`	`max_completion_tokens` (preferred), `max_tokens` (deprecated)
`/v1/messages`	`max_tokens`
`/v1/responses`	`max_output_tokens`

Sunra forwards the ceiling you send to the upstream provider. When a response stops because it hit that ceiling, it reports finish_reason: "length" (Chat Completions and Responses) or stop_reason: "max_tokens" (Messages) — the same signal you would get calling the provider directly.

On Chat Completions, some providers accept only max_tokens and ignore max_completion_tokens — an unbounded completion when you believed you had set a ceiling. For the deepseek/ models the gateway translates max_completion_tokens into max_tokens on your behalf. If you send both, the smaller of the two wins, since either one is an upper bound.Providers that honor max_completion_tokens natively are left alone, so a ceiling you set is enforced exactly once, wherever it is enforced.

Model output ceilings

Each model has its own maximum output length, independent of what you request. Asking for more than a model allows is rejected, not silently reduced — the upstream provider returns 400 with the valid range in the message. For example, deepseek/deepseek-v4-flash and deepseek/deepseek-v4-pro accept up to 393,216 output tokens and reject anything above it:

{
  "error": {
    "message": "Invalid max_tokens value, the valid range of max_tokens is [1, 393216]",
    "type": "invalid_request_error"
  }
}

A 400 is the intended behavior here. A ceiling that is quietly lowered would be indistinguishable from a completion that legitimately ran to its limit: both report finish_reason: "length", and nothing in usage would tell you which happened. Output ceilings are per model and change as providers ship new versions. Treat the 400 as the authoritative answer rather than hard-coding a limit in your client.

Non-streaming requests: the edge timeout

api-llm.sunra.ai is served through a CDN edge that gives up on a response which takes too long to arrive. On a non-streaming request, a generation still running when that timer expires is terminated at the edge, before the gateway can say anything about it. Depending on timing and connection reuse, the cut shows up as one of:

HTTP 524 with no body
the connection closed without a response (RemoteDisconnected / ECONNRESET)
a TLS-level EOF with no close_notify (SSL: UNEXPECTED_EOF_WHILE_READING)

None of these carries an error payload — the response never started, so there was nothing to attach one to. When this signature appears only on your long-running requests while short ones sail through, it is this timeout, not a network fault. Do not treat the timer as a precise budget. The CDN’s documented default is 125 seconds, but measured cuts on this endpoint have landed anywhere from about 112 seconds upward, and the value is the CDN’s to change. A request finishing in 83 seconds is comfortably safe; one aiming at 120 is not. The gateway behind the edge allows a non-streaming request 6 minutes, but that ceiling is unreachable from the public endpoint: the edge gives up first, and it is the edge that closes the connection. If a request may generate for more than ~90 seconds, send it with stream: true. On a streamed request the gateway responds within about 60 seconds even if the provider has not — see Before the provider responds — and then keeps bytes flowing, so the edge timer never fires; long generations are instead governed by the far more generous stream lifetime limits below. As a sizing aid: models typically emit 30–80 tokens per second, so an output ceiling in the several thousands of tokens — or a reasoning model that thinks at length before its first output token — will not fit inside the window. If your application wants the whole response as a single object, stream anyway and assemble the deltas client-side; the stream’s final events carry the same finish_reason you would have received non-streaming.

Before the provider responds

A reasoning model often sends nothing at all until it has finished thinking — not a first token, not even HTTP response headers. Measured first-header times on this gateway range from a few seconds to well past six minutes on the heaviest reasoning requests. On a streamed request the gateway no longer waits in silence for that. If the provider has not responded within about 60 seconds, the gateway commits the response on its own:

HTTP/1.1 200 OK
content-type: text/event-stream; charset=utf-8
cache-control: no-cache
x-sunra-prediction-id: chatcmpl_…

and then sends : keepalive comments until the provider starts. Your client sees a normal SSE stream that is briefly quiet; the provider’s own frames follow when they arrive. This has one consequence worth designing for. Once that 200 is on the wire, an error can no longer be an HTTP status code — so a provider error that arrives after the commit is delivered inside the stream, as an error frame carrying prediction_id and the provider’s original status in upstream_status:

data: {"error":{"message":"Rate limit reached for requests","type":"rate_limit_error","code":"rate_limit_exceeded","upstream_status":429,"prediction_id":"chatcmpl_…"}}

data: [DONE]

On /v1/messages the same content arrives in the Anthropic dialect:

event: error
data: {"type":"error","error":{"type":"rate_limit_error","message":"Rate limit reached for requests","code":"rate_limit_exceeded","upstream_status":429,"prediction_id":"msg_…"}}

The official OpenAI and Anthropic SDKs raise on both shapes, so SDK users see an exception either way — only the exception’s class changes (an API-status error becomes a stream error). If you parse the stream by hand, treat any frame carrying an error object as terminal. Errors that arrive within the first 60 seconds are unchanged: they are still native HTTP statuses with the provider’s body, exactly as before. In practice that is the overwhelming majority of them — a provider that is going to reject a request rejects it in milliseconds. Non-streaming requests are not affected by any of this; they are still governed by the edge timeout.

Stream lifetime

A streamed request (stream: true) passes through three phases, each with its own limit:

Phase	Limit	What it catches
Waiting for the provider’s response headers	10 minutes	A provider that never starts responding at all
Between bytes, once the stream has started	120 seconds with no bytes from the provider	A provider that stopped producing output but has not closed the connection
Total stream lifetime, measured from those response headers	14 minutes	A stream that is still producing but will not terminate

The phases are sequential, and the distinction between “response headers” and “first output token” matters: the 120-second idle timeout and the 14-minute ceiling both start when the provider’s response headers arrive, not when the request was sent. A provider that thinks for eight minutes before sending headers therefore still gets its full 14 minutes of stream afterwards. The idle timeout only bites a provider that has gone silent; a stream delivering tokens is never cut for being slow. The lifetime ceiling is absolute: it applies to an actively delivering stream too, which is the point — it is the backstop against a generation that never terminates. Non-streaming requests are subject to none of these; the gateway gives them 6 minutes for the whole request, though the edge timeout above cuts them off well before that.

Heavy reasoning workloads

For openai/gpt-6-astra, reasoning_effort: "xhigh" on a complex prompt may exceed the roughly 10-minute header budget (630 seconds). Prefer high or simplify the task when predictable completion matters. The gateway does not guarantee that every heavy reasoning workload will finish within its limits. Reasoning and visible text share the output-token allowance. Set max_completion_tokens (Chat Completions), max_output_tokens (Responses), or max_tokens (Messages) high enough for both; a small cap can end the response before an answer appears. Reported reasoning usage can still be billable even if no visible answer is returned. On /v1/responses, an early response.created event does not mean an answer is ready. The separate 14-minute stream lifetime applies from upstream response headers, including time spent reasoning after that event.

Keepalive comments

While a stream is open but the provider is producing nothing, the gateway writes a comment line roughly every 20 seconds so the connection never looks idle to the network between us:

: keepalive

Per the SSE specification a line beginning with : is a comment and must be ignored, and every mainstream client does so — the official OpenAI and Anthropic SDKs discard it before you ever see it. You only need to think about this if you parse the stream by hand: skip any line that starts with :, the same way you already skip blank lines. The keepalive carries no meaning. It says nothing about the provider still being alive, and it never resets the idle timeout below.

The abort frame

When either limit fires — or the upstream connection fails mid-stream — the gateway does not drop the connection. It flushes whatever it still holds, sends an error frame in your endpoint’s SSE dialect, and closes the stream cleanly. Everything delivered before the frame is valid output and can be used. On /v1/chat/completions and /v1/responses:

data: {"error":{"message":"Stream aborted by gateway (stream_lifetime_ceiling); partial output above is complete as delivered","type":"gateway_stream_aborted","code":"stream_lifetime_ceiling"}}

data: [DONE]

The [DONE] sentinel is part of the Chat Completions dialect only; Responses streams end after the error frame without one. On /v1/messages, the frame follows the Anthropic dialect — a typed error event, and no [DONE]:

event: error
data: {"type":"error","error":{"type":"gateway_stream_aborted","message":"Stream aborted by gateway (upstream_idle_timeout); partial output above is complete as delivered","code":"upstream_idle_timeout"}}

For a gateway-authored abort, type is always gateway_stream_aborted, and code — present in every dialect — names what happened:

Code	Retry?	Meaning
`upstream_idle_timeout`	Yes — may be transient	The provider sent nothing for 120 seconds.
`stream_lifetime_ceiling`	Not as-is	The stream reached 14 minutes. The same request will likely reach it again — lower your output ceiling, or split the work into several requests.
`upstream_stream_error`	Yes — may be transient	The connection to the provider failed mid-stream.
`upstream_empty_stream`	Yes — may be transient	The provider ended an apparently successful stream without ever producing output or usage. Nothing was delivered, and the request is not billed.
`server_shutdown`	Yes — retry	The gateway instance is shutting down and the request exceeded its five-minute drain window. This is a deployment interruption.
`pre_stream_timeout`	Not as-is	The provider never sent response headers within 10 minutes. Nothing was delivered, and the request is not billed.
`upstream_fetch_failed`	Yes — may be transient	The connection to the provider failed before the stream started. Nothing was delivered, and the request is not billed.
`upstream_not_streaming`	Yes — may be transient	The provider answered with something that is not an SSE stream after the gateway had already committed one. Nothing was delivered, and the request is not billed.

server_shutdown can arrive before any output or during generation. Keep any partial output already received; retrying starts a new request and does not resume that output. Billing follows the aborted-stream rules below. If the gateway has not committed response headers, including for a non-streaming request, shutdown returns HTTP 503 with error.type: "gateway_error" and error.code: "server_shutdown". A request refused after shutdown starts receives the same status and code. upstream_empty_stream is the one code that does not come from a limit or a broken connection: the stream ended cleanly, just empty. Because there is no partial output to keep, its frame carries a different message — Upstream returned an empty stream: the response ended before any billable output or terminal usage was received — and is the entire visible result of the request. The last three codes can only appear on a request that had already been committed before the provider responded, so by construction there is no partial output above them; their message says so — Upstream failed before the stream started (<code>); no output was produced. A provider error delivered after that commit is a different shape again: it keeps the provider’s own type and code rather than gateway_stream_aborted, and adds upstream_status. Use the presence of upstream_status to tell “the provider rejected this” from “the gateway ended the stream”. A stream that ends with neither an abort frame nor its normal terminal events was cut at the transport level. The gateway writes the abort frame whenever the connection to you is still writable, but it cannot promise one in every case: if the connection is already gone — a socket error, or an intermediary that gave up on an idle connection — there is nothing left to write it to. So treat a truncated stream with no abort frame as an interrupted connection, not as a completed response, and not as proof that the fault is on your side. Everything delivered before the cut is still valid output.

Billing for an aborted stream

An aborted stream is billed for what was delivered to you, never for the full generation the provider may have continued internally.

If the provider had already reported final usage before the abort, that report is billed.
Otherwise the delivered output is settled on a conservative estimate — roughly four characters per token, capped at the output ceiling you requested. The completion records usage_source: "conservative_estimate" for the record.
If nothing was delivered, or the provider reported an explicit error, the request is marked failed and the reserved credits are released. Failed requests are not charged.

Handling this in a client

for line in response.iter_lines():
    if not line.startswith(b"data: "):
        continue
    payload = line[6:]
    if payload == b"[DONE]":
        break

    event = json.loads(payload)
    if error := event.get("error"):
        if error.get("type") == "gateway_stream_aborted":
            # Keep what has been collected so far — it is complete as delivered.
            # error["code"] says whether a retry is worth attempting.
            handle_partial(collected, reason=error["code"])
            break
        raise UpstreamError(error)

    collected.append(event["choices"][0]["delta"].get("content", ""))

The important part is that the partial output is kept rather than discarded. You are billed for it either way, and for long generations it is usually most of the answer.

Overview

Platform

Multimodal

LLM

Client Libraries

Output limits and stream lifetime

Requesting an output ceiling

Model output ceilings

Non-streaming requests: the edge timeout

Before the provider responds

Stream lifetime

Heavy reasoning workloads

Keepalive comments

The abort frame

Billing for an aborted stream

Handling this in a client

​Requesting an output ceiling

​Model output ceilings

​Non-streaming requests: the edge timeout

​Before the provider responds

​Stream lifetime

​Heavy reasoning workloads

​Keepalive comments

​The abort frame

​Billing for an aborted stream

​Handling this in a client

Requesting an output ceiling

Model output ceilings

Non-streaming requests: the edge timeout

Before the provider responds

Stream lifetime

Heavy reasoning workloads

Keepalive comments

The abort frame

Billing for an aborted stream

Handling this in a client