A Guide to Streaming Responses

The Chat API is capable of streaming events (such as text generation) as they come. This means that partial results from the model can be displayed within moments, even if the full generation takes longer.

You're likely already familiar with streaming. When you ask the model a question using the Cohere playground, the interface doesn't output a single block of text, instead it streams the text out a few words at a time. In many user interfaces enabling streaming improves the user experience by lowering the perceived latency.

When streaming is enabled, the API sends events down one by one. Each event has a type. Events of different types need to be handled correctly.

The following is an example of printing the content-delta event type from a streamed response, which contains the text contents of an LLM's response.

import cohere

co = cohere.ClientV2(api_key="<YOUR API KEY>")

res = co.chat_stream(
    model="command-a-plus-05-2026",
    messages=[{"role": "user", "content": "What is an LLM?"}],
)

for event in res:
    if event:
        if event.type == "content-delta":
            print(event.delta.message.content.text, end="")

curl --request POST \
  --url https://api.cohere.ai/v2/chat \
  --header 'accept: text/event-stream' \
  --header 'content-type: application/json' \
  --header "Authorization: bearer $CO_API_KEY" \
  --data '{
    "model": "command-a-plus-05-2026",
    "messages": [
      {
        "role": "user",
        "content": "What is an LLM?"
      }
    ],
    "stream": true
  }'

# Sample output (streamed)

A large language model (LLM) is a type of artificial neural network model that has been trained on massive amounts of text data ...

The following sections describe the different types of events that are emitted during a streaming session.

The first event in the stream containing metadata for the request such as the id. Only one message-start event will be emitted.

The event that indicates the start of the content block of the message. Only one content-start event will be emitted.

The event that is emitted whenever the next chunk of text comes back from the model. As the model continues generating text, multiple events of this type will be emitted. Each event generates one token through the delta.message.content.text field.

# Sample events

type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text='A')))

type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text=' large')))

type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text=' language')))

...

The event that indicates the end of the content block of the message. Only one content-end event will be emitted.

The final event in the stream indicating the end of the streamed response. Only one message-end event will be emitted.

Same as in a basic chat stream event.

Same as in a basic chat stream event.

Same as in a basic chat stream event.

Emitted for every citation generated in the response.

# Sample event

type='citation-start' index=0 delta=CitationStartEventDelta(message=CitationStartEventDeltaMessage(citations=Citation(start=14, end=29, text='gym memberships', sources=[DocumentSource(type='document', id='doc:1', document={'id': 'doc:1', 'text': 'Health and Wellness Benefits: We care about your well-being and offer gym memberships, on-site yoga classes, and comprehensive health insurance.'})])))

Emitted to indicate the end of a citation. If there are multiple citations generated, the events will come as a sequence of citation-start and citation-end pairs.

Same as in a basic chat stream event.

Same as in a basic chat stream event.

Same as in a basic chat stream event.

Emitted when the next token of the tool plan is generated.

# Sample events

type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(tool_plan=None, message={'tool_plan': 'I'})

type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(tool_plan=None, message={'tool_plan': ' will'})

type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(tool_plan=None, message={'tool_plan': ' use'})

...

Emitted when the model generates tool calls that require actioning upon. The event contains a list of tool_calls containing the tool name and tool call ID of the tool.

# Sample event

type='tool-call-start' index=0 delta=ChatToolCallStartEventDelta(tool_call=None, message={'tool_calls': {'id': 'get_weather_nsz5zm3w56q3', 'type': 'function', 'function': {'name': 'get_weather', 'arguments': ''}}})

Emitted when the next token of the the tool call is generated.

# Sample events

type='tool-call-delta' index=0 delta=ChatToolCallDeltaEventDelta(tool_call=None, message={'tool_calls': {'function': {'arguments': '{\n    "'}}})

type='tool-call-delta' index=0 delta=ChatToolCallDeltaEventDelta(tool_call=None, message={'tool_calls': {'function': {'arguments': 'location'}}})

type='tool-call-delta' index=0 delta=ChatToolCallDeltaEventDelta(tool_call=None, message={'tool_calls': {'function': {'arguments': '":'}}})

...

Emitted when the tool call is finished.

Same as in a basic chat stream event.

Same as in a basic chat stream event.

Same as in a basic chat stream event.

Same as in a basic chat stream event.

Emitted for every citation generated in the response.

Emitted to indicate the end of a citation. If there are multiple citations generated, the events will come as a sequence of citation-start and citation-end pairs.

Same as in a basic chat stream event.

Same as in a basic chat stream event.