Skip to main content
Cohere

Search documentation

Type to search this documentation.

On this pageOverview

Tool use & agents - quickstart

About tool use & agents

Section titled “About tool use & agents”

Tool use enables developers to build agentic applications that connect to external tools, do reasoning, and perform actions.

The Chat endpoint comes with built-in tool use capabilities such as function calling, multi-step reasoning, and citation generation.

This quickstart guide shows you how to utilize tool use with the Chat endpoint.

  1. Setup

    First, install the Cohere Python SDK with the following command.

    Bash
    pip install -U cohere

    Next, import the library and create a client.

    PYTHON
    import cohere

co = cohere.ClientV2(
    "COHERE_API_KEY"
)  # Get your free API key here: https://dashboard.cohere.com/api-keys
    PYTHON
    import cohere

co = cohere.ClientV2(
    api_key="",  # Leave this blank
    base_url="<YOUR_DEPLOYMENT_URL>",
)
    PYTHON
    import cohere

co = cohere.BedrockClientV2(
    aws_region="AWS_REGION",
    aws_access_key="AWS_ACCESS_KEY_ID",
    aws_secret_key="AWS_SECRET_ACCESS_KEY",
    aws_session_token="AWS_SESSION_TOKEN",
)

# Get the model name: https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html
    PYTHON
    import cohere

co = cohere.SagemakerClientV2(
    aws_region="AWS_REGION",
    aws_access_key="AWS_ACCESS_KEY_ID",
    aws_secret_key="AWS_SECRET_ACCESS_KEY",
    aws_session_token="AWS_SESSION_TOKEN",
)
    PYTHON
    import cohere

co = cohere.ClientV2(
    api_key="AZURE_API_KEY",
    base_url="AZURE_ENDPOINT",  # example: "https://cohere-command-r-plus-08-2024-xyz.eastus.models.ai.azure.com/"
)

  2. Tool Definition

    First, we need to set up the tools. A tool can be any function or service that can receive and send objects.

    We also need to define the tool schemas in a format that can be passed to the Chat endpoint. The schema must contain the following fields: name, description, and parameters.

    PYTHON
    def get_weather(location):
    # Implement your tool calling logic here
    return [{"temperature": "20C"}]
    # Return a list of objects e.g. [{"url": "abc.com", "text": "..."}, {"url": "xyz.com", "text": "..."}]


functions_map = {"get_weather": get_weather}

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "gets the weather of a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "the location to get weather, example: San Fransisco, CA",
                    }
                },
                "required": ["location"],
            },
        },
    },
]

  3. Tool Calling

    Next, pass the tool schema to the Chat endpoint together with the user message.

    The LLM will then generate the tool calls (if any) and return the tool_plan and tool_calls objects.

    PYTHON
    messages = [
    {"role": "user", "content": "What's the weather in Toronto?"}
]

response = co.chat(
    model="command-a-plus-05-2026", messages=messages, tools=tools
)

if response.message.tool_calls:
    messages.append(response.message)
    print(response.message.tool_calls)
    PYTHON
    messages = [
    {"role": "user", "content": "What's the weather in Toronto?"}
]

response = co.chat(
    model="command-a-plus-05-2026", messages=messages, tools=tools
)

if response.message.tool_calls:
    messages.append(response.message)
    print(response.message.tool_calls)
    PYTHON
    messages = [
    {"role": "user", "content": "What's the weather in Toronto?"}
]

response = co.chat(
    model="YOUR_MODEL_NAME", messages=messages, tools=tools
)

if response.message.tool_calls:
    messages.append(response.message)
    print(response.message.tool_calls)
    PYTHON
    messages = [
    {"role": "user", "content": "What's the weather in Toronto?"}
]

response = co.chat(
    model="YOUR_ENDPOINT_NAME", messages=messages, tools=tools
)

if response.message.tool_calls:
    messages.append(response.message)
    print(response.message.tool_calls)
    PYTHON
    messages = [
    {"role": "user", "content": "What's the weather in Toronto?"}
]

response = co.chat(
    model="model",  # Pass a dummy string
    messages=messages,
    tools=tools,
)

if response.message.tool_calls:
    messages.append(response.message)
    print(response.message.tool_calls)
    wordWrap
    [ToolCallV2(id='get_weather_776n8ctsgycn', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"Toronto"}'))]

  4. Tool Execution

    Next, the tools called will be executed based on the arguments generated in the tool calling step earlier.

    PYTHON
    import json

if response.message.tool_calls:
    for tc in response.message.tool_calls:
        tool_result = functions_map[tc.function.name](
            **json.loads(tc.function.arguments)
        )
        tool_content = []
        for data in tool_result:
            tool_content.append(
                {
                    "type": "document",
                    "document": {"data": json.dumps(data)},
                }
            )
            # Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated
        messages.append(
            {
                "role": "tool",
                "tool_call_id": tc.id,
                "content": tool_content,
            }
        )

  5. Response Generation

    The results are passed back to the LLM, which generates the final response.

    PYTHON
    response = co.chat(
    model="command-a-plus-05-2026", messages=messages, tools=tools
)
print(response.message.content[0].text)
    PYTHON
    response = co.chat(
    model="command-a-plus-05-2026", messages=messages, tools=tools
)
print(response.message.content[0].text)
    PYTHON
    response = co.chat(
    model="YOUR_MODEL_NAME", messages=messages, tools=tools
)
print(response.message.content[0].text)
    PYTHON
    response = co.chat(
    model="YOUR_ENDPOINT_NAME", messages=messages, tools=tools
)
print(response.message.content[0].text)
    PYTHON
    response = co.chat(
    model="model",  # Pass a dummy string
    messages=messages,
    tools=tools,
)
print(response.message.content[0].text)
    wordWrap
    It is 20C in Toronto.

  6. Citation Generation

    The response object contains a citations field, which contains specific text spans from the documents on which the response is grounded.

    PYTHON
    if response.message.citations:
    for citation in response.message.citations:
        print(citation, "\n")
    wordWrap
    start=6 end=9 text='20C' sources=[ToolSource(type='tool', id='get_weather_776n8ctsgycn:0', tool_output={'temperature': '20C'})]

Further Resources

Section titled “Further Resources”
Suggest an edit

Propose a replacement for this page. The site team reviews it before applying any changes.

Export
Documentation menu