FlowModels

Stream responses: Chat Completions with Server-Sent Events

Published on 11. November 2026

With long responses, you don't want to show your users a loading spinner for minutes. With streaming, FlowModels sends you the response piece by piece as the model generates it. As with OpenAI, this works via Server-Sent Events (SSE). Here's how to start a stream, what the individual pieces look like, and where to find costs and tokens at the end.

Start a stream

In your request to POST /v1/chat/completions, set the stream field to true. With curl, you also need -N so the output isn't buffered:

curl -N https://api.flowmodels.one/v1/chat/completions \
  -H "Authorization: Bearer at_your_key" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -d '{"model": "groq/openai/gpt-oss-120b", "stream": true, "messages": [{"role": "user", "content": "Write a short greeting for our customer portal."}]}'

With the OpenAI library for Python, it looks like this:

stream = client.chat.completions.create(
    model="groq/openai/gpt-oss-120b",
    messages=[{"role": "user", "content": "Write a short greeting for our customer portal."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)

Use the OpenAI library for Python and Node.js with FlowModels explains how to set up the library.

What a stream looks like

The response arrives as text/event-stream. Each line starts with data: and contains an object of type chat.completion.chunk. You'll find the new text in choices[0].delta.content:

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","model":"groq/openai/gpt-oss-120b","choices":[{"index":0,"delta":{"content":"Welcome"},"finish_reason":null}]}

data: [DONE]

Simply concatenate the text pieces in your application. As soon as data: [DONE] arrives, the response is complete.

Costs and tokens in a stream

The last chunk before [DONE] contains usage with the token counts as well as cost and currency, which is the amount in euros. That's our default. If you don't need this block, send "stream_options": {"include_usage": false}. stream_options is only allowed together with "stream": true and has only this one field.

If the model searched the web, citations and search_results are also in this last chunk. If the model calls one of your functions, a separate chunk with choices[0].delta.tool_calls arrives right before the end, and the final chunk carries finish_reason: "tool_calls". More on this in Let the model call your own functions: tools and tool_calls.

When the connection drops

If the connection breaks in the middle of a stream, it can't be continued. Whatever the provider generated up to that point is still billed. If you send the request again, it's a new request with new costs.

If you need to fetch long responses reliably, for example over shaky mobile connections, use the /responses endpoint with background: true instead. There, the job keeps running on our side and you can resume the stream at the last position after an interruption. Long requests in the background: Responses with background, polling and cancelling shows how.

Good to know

  • Each running stream takes up one slot in your tier's number of concurrent requests. If all are taken, you get 429 concurrent_requests_exceeded. Request limit: how the tiers work explains the limits per tier.
  • Streaming also works with the Brain, so together with external_identifier and conversation.
  • A proxy or web server between your application and the browser must not buffer the response, or the text will only arrive at the end after all.

Was this article helpful?

Back to Getting started & API keys