With long responses, you don't want to show your users a loading spinner for minutes. With streaming, FlowModels sends you the response piece by piece as the model generates it. As with OpenAI, this works via Server-Sent Events (SSE). Here's how to start a stream, what the individual pieces look like, and where to find costs and tokens at the end.
Start a stream
In your request to POST /v1/chat/completions, set the stream field to true. With curl, you also need -N so the output isn't buffered:
curl -N https://api.flowmodels.one/v1/chat/completions \
-H "Authorization: Bearer at_your_key" \
-H "Content-Type: application/json" \
-H "Accept: text/event-stream" \
-d '{"model": "groq/openai/gpt-oss-120b", "stream": true, "messages": [{"role": "user", "content": "Write a short greeting for our customer portal."}]}'
With the OpenAI library for Python, it looks like this:
stream = client.chat.completions.create(
model="groq/openai/gpt-oss-120b",
messages=[{"role": "user", "content": "Write a short greeting for our customer portal."}],
stream=True,
)
for chunk in stream:
if chunk.choices:
print(chunk.choices[0].delta.content or "", end="", flush=True)
Use the OpenAI library for Python and Node.js with FlowModels explains how to set up the library.
What a stream looks like
The response arrives as text/event-stream. Each line starts with data: and contains an object of type chat.completion.chunk. You'll find the new text in choices[0].delta.content:
data: {"id":"chatcmpl-…","object":"chat.completion.chunk","model":"groq/openai/gpt-oss-120b","choices":[{"index":0,"delta":{"content":"Welcome"},"finish_reason":null}]}
data: [DONE]
Simply concatenate the text pieces in your application. As soon as data: [DONE] arrives, the response is complete.
Costs and tokens in a stream
The last chunk before [DONE] contains usage with the token counts as well as cost and currency, which is the amount in euros. That's our default. If you don't need this block, send "stream_options": {"include_usage": false}. stream_options is only allowed together with "stream": true and has only this one field.
If the model searched the web, citations and search_results are also in this last chunk. If the model calls one of your functions, a separate chunk with choices[0].delta.tool_calls arrives right before the end, and the final chunk carries finish_reason: "tool_calls". More on this in Let the model call your own functions: tools and tool_calls.
When the connection drops
If the connection breaks in the middle of a stream, it can't be continued. Whatever the provider generated up to that point is still billed. If you send the request again, it's a new request with new costs.
If you need to fetch long responses reliably, for example over shaky mobile connections, use the /responses endpoint with background: true instead. There, the job keeps running on our side and you can resume the stream at the last position after an interruption. Long requests in the background: Responses with background, polling and cancelling shows how.
Good to know
- Each running stream takes up one slot in your tier's number of concurrent requests. If all are taken, you get
429 concurrent_requests_exceeded. Request limit: how the tiers work explains the limits per tier. - Streaming also works with the Brain, so together with
external_identifierandconversation. - A proxy or web server between your application and the browser must not buffer the response, or the text will only arrive at the end after all.