Skip to main content
Cohere

Search documentation

Type to search this documentation.

Chat API (v2)

POST/v2/chatChat API (v2)

Generates a text response to a user message and streams it down, token by token. To learn how to use the Chat API with streaming follow our Text Generation guides.

Follow the Migration Guide for instructions on moving from API v1 to API v2.

Parameters

X-Client-Namestringheader

The name of the project that is making the request.

Request body

application/json
object
citation_optionsobject

Options for controlling citation generation.

Show child attributes
modestring

Defaults to `"enabled"`. Citations are enabled by default for models that support it, but can be turned off by setting `"type": "disabled"`.

one of "ENABLED", "DISABLED", "FAST", "ACCURATE", "OFF"

documentsarray of value

A list of relevant documents that the model can cite to generate a more accurate reply. Each document is either a string or document object with content and metadata.

Show child attributes
Show array items
oneOf · 2 options
Option 1string
Option 2objectDocument

Relevant information that could be used by the model to generate a more accurate reply. The content of each document are generally short (should be under 300 words). Metadata should be used to provide additional information, both the key name and the value will be passed to the model.

dataobjectrequired

A relevant document that the model can cite to generate a more accurate reply. Each document is a string-any dictionary.

idstring

Unique identifier for this document which will be referenced in citations. If not provided an ID will be automatically generated.

frequency_penaltynumber · float

Defaults to `0.0`, min value of `0.0`, max value of `1.0`. Used to reduce repetitiveness of generated tokens. The higher the value, the stronger a penalty is applied to previously present tokens, proportional to how many times they have already appeared in the prompt or prior generation.

kinteger

Ensures that only the top `k` most likely tokens are considered for generation at each step. When `k` is set to `0`, k-sampling is disabled. Defaults to `0`, min value of `0`, max value of `500`.

default 0 · maximum 500 · minimum 0

logprobsboolean

Defaults to `false`. When set to `true`, the log probabilities of the generated tokens will be included in the response.

max_tokensinteger

The maximum number of output tokens the model will generate in the response. If not set, `max_tokens` defaults to the model's maximum output token limit. You can find the maximum output token limits for each model in the [model documentation](https://docs.cohere.com/docs/models). **Note**: Setting a low value may result in incomplete generations. In such cases, the `finish_reason` field in the response will be set to `"MAX_TOKENS"`. **Note**: If `max_tokens` is set higher than the model's maximum output token limit, the generation will be capped at that model-specific maximum limit.

messagesarray of valuerequired

A list of chat messages in chronological order, representing a conversation between the user and the model. Messages can be from `User`, `Assistant`, `Tool` and `System` roles. Learn more about messages and roles in [the Chat API guide](https://docs.cohere.com/v2/docs/chat-api).

Show child attributes
Show array items

Represents a single message in the chat history from a given role.

oneOf · 4 options
Option 1objectUserMessageV2

User Message

A message from the user.

contentvaluerequired

The content of the message. This can be a string or a list of content blocks. If a string is provided, it will be treated as a text content block.

Show child attributes
oneOf · 2 options
Option 1string
Option 2array of value
Show array items

A Content block which contains information about the content type and the content itself.

oneOf · 2 options
Option 1objectChatTextContent

Text content of the message.

textstringrequired
typestringrequired

one of "text"

Option 2objectImageContent

Image content of the message.

image_urlobjectrequired
Show child attributes
detailstring

Controls the level of detail in image processing. `"auto"` is the default and lets the system choose, `"low"` is faster but less detailed, and `"high"` preserves maximum detail. You can save tokens and speed up responses by using detail: `"low"`.

one of "auto", "low", "high"

urlstringrequired

URL of an image. Can be either a base64 data URI or a web URL.

typestringrequired

Type of message content

one of "text", "image_url"

rolestringrequired

one of "user"

Option 2objectAssistantMessageV2

Assistant Message

A message from the assistant role can contain text and tool call information.

citationsarray of object
Show child attributes
Show array items

Citation information containing sources and the text cited.

content_indexinteger

Index of the content block in which this citation appears.

endinteger

End index of the cited snippet in the original source text.

sourcesarray of value
Show child attributes
Show array items

A source object containing information about the source of the data cited.

oneOf · 2 options
Option 1objectChatToolSource

Tool Output

idstring

The unique identifier of the document

tool_outputobject
Option 2objectChatDocumentSource

A document source object containing the unique identifier of the document and the document itself.

documentobject
idstring

The unique identifier of the document

startinteger

Start index of the cited snippet in the original source text.

textstring

Text snippet that is being cited.

typestring

The type of citation which indicates what part of the response the citation is for.

one of "TEXT_CONTENT", "THINKING_CONTENT", "PLAN"

contentvalue
Show child attributes
oneOf · 2 options
Option 1string
Option 2array of value
Show array items
oneOf · 2 options

Option 1: ChatTextContent ↑

Option 2objectChatThinkingContent

Thinking content of the message. This will be present when `thinking` is enabled, and will contain the models internal reasoning.

thinkingstringrequired
typestringrequired

one of "thinking"

rolestringrequired

one of "assistant"

tool_callsarray of object
Show child attributes
Show array items

An array of tool calls to be made.

functionobject
Show child attributes
argumentsstring
namestring
idstringrequired
typestringrequired

one of "function"

tool_planstring

A chain-of-thought style reflection and plan that the model generates when working with Tools.

Option 3objectSystemMessageV2

System Message

A message from the system.

contentvaluerequired
Show child attributes
oneOf · 2 options
Option 1string
Option 2array of value
Show array items
oneOf · 1 option

Option 1: ChatTextContent ↑

rolestringrequired

one of "system"

Option 4objectToolMessageV2

Tool Message

A message with Tool outputs.

contentvaluerequired

Outputs from a tool. The content should formatted as a JSON object string, or a list of tool content blocks

Show child attributes
oneOf · 2 options
Option 1string
Option 2array of value
Show array items

A content block which contains information about the content of a tool result

oneOf · 2 options

Option 1: ChatTextContent ↑

Option 2objectDocumentContent

Document content.

documentobjectrequiredDocument ↑

Relevant information that could be used by the model to generate a more accurate reply. The content of each document are generally short (should be under 300 words). Metadata should be used to provide additional information, both the key name and the value will be passed to the model.

typestringrequired

one of "document"

rolestringrequired

one of "tool"

tool_call_idstringrequired

The id of the associated tool call that has provided the given content

modelstringrequired

The name of a compatible [Cohere model](https://docs.cohere.com/v2/docs/models).

pnumber · float

Ensures that only the most likely tokens, with total probability mass of `p`, are considered for generation at each step. If both `k` and `p` are enabled, `p` acts after `k`. Defaults to `0.75`. min value of `0.01`, max value of `0.99`.

default 0.75 · maximum 0.99 · minimum 0.01

presence_penaltynumber · float

Defaults to `0.0`, min value of `0.0`, max value of `1.0`. Used to reduce repetitiveness of generated tokens. Similar to `frequency_penalty`, except that this penalty is applied equally to all tokens that have already appeared, regardless of their exact frequencies.

priorityinteger

Controls how early the request is handled. Lower numbers indicate higher priority (default: 0, the highest). When the system is under load, higher-priority requests are processed first and are the least likely to be dropped.

default 0 · maximum 999 · minimum 0

response_formatvalue

Configuration for forcing the model output to adhere to the specified format. Supported on [Command R](https://docs.cohere.com/v2/docs/command-r), [Command R+](https://docs.cohere.com/v2/docs/command-r-plus) and newer models. The model can be forced into outputting JSON objects by setting `{ "type": "json_object" }`. A [JSON Schema](https://json-schema.org/) can optionally be provided, to ensure a specific structure. **Note**: When using `{ "type": "json_object" }` your `message` should always explicitly instruct the model to generate a JSON (eg: _"Generate a JSON ..."_) . Otherwise the model may end up getting stuck generating an infinite stream of characters and eventually run out of context length. **Note**: When `json_schema` is not specified, the generated object can have up to 5 layers of nesting. **Limitation**: The parameter is not supported when used in combinations with the `documents` or `tools` parameters.

Show child attributes
oneOf · 2 options
Option 1objectChatTextResponseFormatV2
typestringrequired

Defaults to `"text"`. When set to `"json_object"`, the model's output will be a valid JSON Object.

one of "text", "json_object"

Option 2objectJsonResponseFormatV2
json_schemaobject

A [JSON schema](https://json-schema.org/overview/what-is-jsonschema) object that the output will adhere to. There are some restrictions we have on the schema, refer to [our guide](https://docs.cohere.com/docs/structured-outputs-json#schema-constraints) for more information. Example (required name and age object): ```json { "type": "object", "properties": { "name": {"type": "string"}, "age": {"type": "integer"} }, "required": ["name", "age"] } ``` **Note**: This field must not be specified when the `type` is set to `"text"`.

typestringrequired

Defaults to `"text"`. When set to `"json_object"`, the model's output will be a valid JSON Object.

one of "text", "json_object"

safety_modevalue

Used to select the [safety instruction](https://docs.cohere.com/v2/docs/safety-modes) inserted into the prompt. Defaults to `CONTEXTUAL`. When `OFF` is specified, the safety instruction will be omitted. Safety modes are not yet configurable in combination with `tools` and `documents` parameters. **Note**: This parameter is only compatible newer Cohere models, starting with [Command R 08-2024](https://docs.cohere.com/docs/command-r#august-2024-release) and [Command R+ 08-2024](https://docs.cohere.com/docs/command-r-plus#august-2024-release). **Note**: `command-r7b-12-2024` and newer models only support `"CONTEXTUAL"` and `"STRICT"` modes.

one of "CONTEXTUAL", "STRICT", "OFF"

seedinteger

If specified, the backend will make a best effort to sample tokens deterministically, such that repeated requests with the same seed and parameters should return the same result. However, determinism cannot be totally guaranteed.

minimum 0

stop_sequencesarray of string

A list of up to 5 strings that the model will use to stop generation. If the model generates a string that matches any of the strings in the list, it will stop generating tokens and return the generated text up to that point not including the stop sequence.

Show child attributes
streamboolean

Defaults to `false`. When `true`, the response will be a SSE stream of events. Streaming is beneficial for user interfaces that render the contents of the response piece by piece, as it gets generated.

strict_toolsboolean

When set to `true`, tool calls in the Assistant message will be forced to follow the tool definition strictly. Learn more in the [Structured Outputs (Tools) guide](https://docs.cohere.com/docs/structured-outputs-json#structured-outputs-tools). **Note**: The first few requests with a new set of tools will take longer to process.

temperaturenumber · float

Defaults to `0.3`. A non-negative float that tunes the degree of randomness in generation. Lower temperatures mean less random generations, and higher temperatures mean more random generations. Randomness can be further maximized by increasing the value of the `p` parameter.

maximum 1 · minimum 0

thinkingobject

Configuration for [reasoning features](https://docs.cohere.com/docs/reasoning).

Show child attributes
token_budgetinteger

The maximum number of tokens the model can use for thinking, which must be set to a positive integer. The model will stop thinking if it reaches the thinking token budget and will proceed with the response.

minimum 1

typestringrequired

Reasoning is enabled by default for models that support it, but can be turned off by setting `"type": "disabled"`.

one of "enabled", "disabled"

tool_choicevalue

Used to control whether or not the model will be forced to use a tool when answering. When `REQUIRED` is specified, the model will be forced to use at least one of the user-defined tools, and the `tools` parameter must be passed in the request. When `NONE` is specified, the model will be forced **not** to use one of the specified tools, and give a direct response. If tool_choice isn't specified, then the model is free to choose whether to use the specified tools or not. **Note**: This parameter is only compatible with models [Command-r7b](https://docs.cohere.com/v2/docs/command-r7b) and newer.

one of "REQUIRED", "NONE"

toolsarray of object

A list of tools (functions) available to the model. The model response may contain 'tool_calls' to the specified tools. Learn more in the [Tool Use guide](https://docs.cohere.com/docs/tools).

Show child attributes
Show array items
functionobject

The function to be executed.

Show child attributes
descriptionstring

The description of the function.

namestringrequired

The name of the function.

parametersobjectrequired

The parameters of the function as a JSON schema.

typestringrequired

one of "function"

Example request
{
  "citation_options": {
    "mode": "ACCURATE"
  },
  "documents": [
    {
      "data": {},
      "id": "string"
    }
  ],
  "frequency_penalty": 0,
  "k": 0,
  "logprobs": true,
  "max_tokens": 0,
  "messages": [
    {
      "content": "string",
      "role": "system"
    }
  ],
  "model": "string",
  "p": 0.75,
  "presence_penalty": 0,
  "priority": 0,
  "response_format": {
    "type": "text"
  },
  "safety_mode": "CONTEXTUAL",
  "seed": 0,
  "stop_sequences": [
    "string"
  ],
  "stream": true,
  "strict_tools": true,
  "temperature": 0,
  "thinking": {
    "token_budget": 0,
    "type": "disabled"
  },
  "tool_choice": "NONE",
  "tools": [
    {
      "function": {
        "description": "string",
        "name": "string",
        "parameters": {}
      },
      "type": "function"
    }
  ]
}

Responses

200OKapplication/json
value
oneOf · 2 options
Option 1objectChatResponseV2
finish_reasonstringrequired

The reason a chat request has finished. - **complete**: The model finished sending a complete message. - **max_tokens**: The number of generated tokens exceeded the model's context length or the value specified via the `max_tokens` parameter. - **stop_sequence**: One of the provided `stop_sequence` entries was reached in the model's generation. - **tool_call**: The model generated a Tool Call and is expecting a Tool Message in return - **error**: The generation failed due to an internal error - **timeout**: The generation was stopped because it exceeded the allowed time limit.

one of "COMPLETE", "STOP_SEQUENCE", "MAX_TOKENS", "TOOL_CALL", "ERROR", "TIMEOUT"

idstringrequired

Unique identifier for the generated reply. Useful for submitting feedback.

logprobsarray of object
Show child attributes
Show array items
logprobsarray of number · float

The log probability of each token used to construct the text chunk.

Show child attributes
textstring

The text chunk for which the log probabilities was calculated.

token_idsarray of integerrequired

The token ids of each token used to construct the text chunk.

Show child attributes
messageobjectrequired

A message from the assistant role can contain text and tool call information.

Show child attributes
citationsarray of object
Show child attributes

Items: Citation ↑

contentarray of value
Show child attributes
Show array items
oneOf · 2 options

Option 1: ChatTextContent ↑

Option 2: ChatThinkingContent ↑

rolestringrequired

one of "assistant"

tool_callsarray of object
Show child attributes

Items: ToolCallV2 ↑

tool_planstring

A chain-of-thought style reflection and plan that the model generates when working with Tools.

usageobject
Show child attributes
billed_unitsobject
Show child attributes
classificationsnumber

The number of billed classifications units.

input_tokensnumber

The number of billed input tokens.

output_tokensnumber

The number of billed output tokens.

search_unitsnumber

The number of billed search units.

cached_tokensnumber

The number of prompt tokens that hit the inference cache.

tokensobject
Show child attributes
input_tokensnumber

The number of tokens used as input to the model.

output_tokensnumber

The number of tokens produced by the model.

Option 2valueStreamedChatResponseV2

StreamedChatResponse is returned in streaming mode (specified with `stream=True` in the request).

oneOf · 10 options
Option 1valueChatMessageStartEvent

A streamed event which signifies that a stream has started.

allOf · 2 options
Option 1objectChatStreamEventType

The streamed event types

typestringrequired

one of "message-start", "content-start", "content-delta", "content-end", "tool-call-start", "tool-call-delta", "tool-call-end", "tool-plan-delta", "citation-start", "citation-end", "message-end"

Option 2object
deltaobject
Show child attributes
messageobject
Show child attributes
rolestring

The role of the message.

one of "assistant"

idstring

Unique identifier for the generated reply.

Option 2valueChatContentStartEvent

A streamed delta event which signifies that a new content block has started.

allOf · 2 options

Option 1: ChatStreamEventType ↑

Option 2object
deltaobject
Show child attributes
messageobject
Show child attributes
contentobject
Show child attributes
textstring
thinkingstring
typestring

one of "text", "thinking"

indexinteger
Option 3valueChatContentDeltaEvent

A streamed delta event which contains a delta of chat text content.

allOf · 2 options

Option 1: ChatStreamEventType ↑

Option 2object
deltaobject
Show child attributes
messageobject
Show child attributes
contentobject
Show child attributes
textstring
thinkingstring
indexinteger
logprobsobjectLogprobItem ↑
Option 4valueChatContentEndEvent

A streamed delta event which signifies that the content block has ended.

allOf · 2 options

Option 1: ChatStreamEventType ↑

Option 2object
indexinteger
Option 5valueChatToolPlanDeltaEvent

A streamed event which contains a delta of tool plan text.

allOf · 2 options

Option 1: ChatStreamEventType ↑

Option 2object
deltaobject
Show child attributes
messageobject
Show child attributes
tool_planstring
Option 6valueChatToolCallStartEvent

A streamed event delta which signifies a tool call has started streaming.

allOf · 2 options

Option 1: ChatStreamEventType ↑

Option 2object
deltaobject
Show child attributes
messageobject
Show child attributes
tool_callsobjectToolCallV2 ↑

An array of tool calls to be made.

indexinteger
Option 7valueChatToolCallDeltaEvent

A streamed event delta which signifies a delta in tool call arguments.

allOf · 2 options

Option 1: ChatStreamEventType ↑

Option 2object
deltaobject
Show child attributes
messageobject
Show child attributes
tool_callsobject
Show child attributes
functionobject
Show child attributes
argumentsstring
indexinteger
Option 8valueChatToolCallEndEvent

A streamed event delta which signifies a tool call has finished streaming.

allOf · 2 options

Option 1: ChatStreamEventType ↑

Option 2object
indexinteger
Option 9valueChatMessageEndEvent

A streamed event which signifies that the chat message has ended.

allOf · 2 options

Option 1: ChatStreamEventType ↑

Option 2object
deltaobject
Show child attributes
errorstring

An error message if an error occurred during the generation.

finish_reasonstring

The reason a chat request has finished. - **complete**: The model finished sending a complete message. - **max_tokens**: The number of generated tokens exceeded the model's context length or the value specified via the `max_tokens` parameter. - **stop_sequence**: One of the provided `stop_sequence` entries was reached in the model's generation. - **tool_call**: The model generated a Tool Call and is expecting a Tool Message in return - **error**: The generation failed due to an internal error - **timeout**: The generation was stopped because it exceeded the allowed time limit.

one of "COMPLETE", "STOP_SEQUENCE", "MAX_TOKENS", "TOOL_CALL", "ERROR", "TIMEOUT"

usageobjectUsage ↑
idstring
Option 10valueChatDebugEvent
allOf · 2 options
Option 1objectChatStreamEvent
event_typestringrequired

one of "stream-start", "search-queries-generation", "search-results", "text-generation", "citation-generation", "stream-end", "debug"

Option 2object
promptstring
Example response
{
  "finish_reason": "COMPLETE",
  "id": "string",
  "logprobs": [
    {
      "logprobs": [
        0
      ],
      "text": "string",
      "token_ids": [
        0
      ]
    }
  ],
  "message": {
    "citations": [
      {
        "content_index": 0,
        "end": 0,
        "sources": [
          {
            "document": {
              "additionalProp1": null
            },
            "id": "string",
            "type": "document"
          }
        ],
        "start": 0,
        "text": "string",
        "type": "PLAN"
      }
    ],
    "content": [
      {
        "text": "string",
        "type": "text"
      }
    ],
    "role": "assistant",
    "tool_calls": [
      {
        "function": {
          "arguments": "string",
          "name": "string"
        },
        "id": "string",
        "type": "function"
      }
    ],
    "tool_plan": "string"
  },
  "usage": {
    "billed_units": {
      "classifications": 0,
      "input_tokens": 0,
      "output_tokens": 0,
      "search_units": 0
    },
    "cached_tokens": 0,
    "tokens": {
      "input_tokens": 0,
      "output_tokens": 0
    }
  }
}
400This error is returned when the request is not well formed. This could be because: - JSON is invalid - The request is missing required fields - The request contains an invalid combination of fields application/json
object
idstring
messagestring
Example response
{
  "id": "string",
  "message": "string"
}
401This error indicates that the operation attempted to be performed is not allowed. This could be because: - The api token is invalid - The user does not have the necessary permissions application/json
object
idstring
messagestring
Example response
{
  "id": "string",
  "message": "string"
}
403This error indicates that the operation attempted to be performed is not allowed. This could be because: - The api token is invalid - The user does not have the necessary permissions application/json
object
idstring
messagestring
Example response
{
  "id": "string",
  "message": "string"
}
404This error is returned when a resource is not found. This could be because: - The endpoint does not exist - The resource does not exist eg model id, dataset id application/json
object
idstring
messagestring
Example response
{
  "id": "string",
  "message": "string"
}
422This error is returned when the request is not well formed. This could be because: - JSON is invalid - The request is missing required fields - The request contains an invalid combination of fields application/json
object
idstring
messagestring
Example response
{
  "id": "string",
  "message": "string"
}
429Too many requestsapplication/json
object
idstring
messagestring
Example response
{
  "id": "string",
  "message": "string"
}
498This error is returned when a request or response contains a deny-listed token. application/json
object
idstring
messagestring
Example response
{
  "id": "string",
  "message": "string"
}
499This error is returned when a request is cancelled by the user. application/json
object
idstring
messagestring
Example response
{
  "id": "string",
  "message": "string"
}
500This error is returned when an uncategorised internal server error occurs. application/json
object
idstring
messagestring
Example response
{
  "id": "string",
  "message": "string"
}
501This error is returned when the requested feature is not implemented. application/json
object
idstring
messagestring
Example response
{
  "id": "string",
  "message": "string"
}
503This error is returned when the service is unavailable. This could be due to: - Too many users trying to access the service at the same time application/json
object
idstring
messagestring
Example response
{
  "id": "string",
  "message": "string"
}
504This error is returned when a request to the server times out. This could be due to: - An internal services taking too long to respond application/json
object
idstring
messagestring
Example response
{
  "id": "string",
  "message": "string"
}
Documentation menu