Multi-step Tool Use (Agents)
"Cohere's tool use feature enhances AI capabilities by connecting external tools for dynamic, adaptable, and sequential actions."
Tool use is a technique which allows Cohere's models to invoke external tools: search engines, APIs, functions, databases, and so on.
Multi-step tool use happens when the output of one tool calling step is needed as the input to the another. In other words, tool-calling needs to happen in a sequence.
For example, given the web-search tool, the model can start answering complex questions that require performing internet searches.
Notice that the model learned information from the first search, which it then used to perform a second web search. This behavior is called multi-step because the model tackles the task step by step.
Also, note that multi-step is enabled in the Chat API by default.
Multi-step Tool Use With the Chat API
Section titled “Multi-step Tool Use With the Chat API”Step 1: Define the tools
Section titled “Step 1: Define the tools”# define the `web_search` tool.
def web_search(query: str) -> list[dict]:
pass
# your code for performing a web search goes here
# return [{
# "url": "https://en.wikipedia.org/wiki/Ontario",
# "text": "The capital of Ontario is Toronto, ..."
# }]
web_search_tool = {
"type": "function",
"function": {
"name": "web_search",
"description": "performs a web search with the specified query",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "the query to look up",
}
},
"required": ["query"],
},
},
}Step 2: Run the tool use workflow
Section titled “Step 2: Run the tool use workflow”import json
import cohere
co = cohere.ClientV2(api_key="<YOUR API KEY>")
# 1 - Add the user message
message = "Who is the mayor of the capital of Ontario?"
messages = [{"role": "user", "content": message}]
# 2 - Model generates tool calls, if any
model = "command-a-plus-05-2026"
res = co.chat(model=model, messages=messages, tools=[web_search_tool])
# As long as the model sends back tool_calls,
# keep invoking tools and sending the results back to the model
while res.message.tool_calls:
print("\nTool plan:")
print(
res.message.tool_plan
) # This will be an observation and a plan with next steps
print("\nTool calls:")
for tc in res.message.tool_calls:
print(
f"Tool name: {tc.function.name} | Parameters: {tc.function.arguments}"
)
messages.append(res.message)
# 3 - Execute tools based on the tool calls generated by the model
print("\nTool results:")
for tc in res.message.tool_calls:
tool_result = web_search(**json.loads(tc.function.arguments))
print(tool_result)
tool_content = []
for data in tool_result:
tool_content.append(
{
"type": "document",
"document": {"data": json.dumps(data)},
}
)
# Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated
messages.append(
{
"role": "tool",
"tool_call_id": tc.id,
"content": tool_content,
}
)
# 4 - Model either generates more tool calls or returns a response
res = co.chat(
model=model, messages=messages, tools=[web_search_tool]
)
print("\nResponse:")
print(res.message.content[0].text)
if res.message.citations:
print("\nCitations:")
for citation in res.message.citations:
print(citation, "\n")# EXAMPLE RESPONSE
Tool plan:
I will search for the capital of Ontario, then search for the mayor of that city.
Tool calls:
Tool name: web_search | Parameters: {"query":"capital of ontario"}
Tool results:
[{'url': 'https://simple.wikipedia.org/wiki/Toronto', 'text': 'Toronto is the capital city of the province of Ontario.It is also the largest city in Ontario. It is found on the north-west side of Lake Ontario behind New York City and Chicago.. The City of Toronto itself has a population of almost 3 million people. Even more people live in the regions around it. All together, the Greater Toronto Area is home to over 6.2 million people.'}]
Tool plan:
I have found that Toronto is the capital of Ontario. I will now search for the mayor of Toronto.
Tool calls:
Tool name: web_search | Parameters: {"query":"mayor of toronto"}
Tool results:
[{'url': 'https://en.wikipedia.org/wiki/Mayor_of_Toronto', 'text': 'The mayor of Toronto is the head of Toronto City Council and chief executive officer of the municipal government.The mayor is elected alongside city council every four years on the fourth Monday of October; there are no term limits. [2] While in office, mayors are styled His/Her Worship. [3]Olivia Chow has served as the 66th and current mayor of Toronto since July 12, 2023, after winning the ...'}]
Response:
The capital of Ontario is Toronto. The current mayor of Toronto is Olivia Chow, who has been in office since July 12, 2023.
Citations:
start=26 end=34 text='Toronto.' sources=[ToolSource(type='tool', id='web_search_p7f8t49dzs8e:0', tool_output={'text': 'Toronto is the capital city of the province of Ontario.It is also the largest city in Ontario. It is found on the north-west side of Lake Ontario behind New York City and Chicago.. The City of Toronto itself has a population of almost 3 million people. Even more people live in the regions around it. All together, the Greater Toronto Area is home to over 6.2 million people.', 'url': 'https://simple.wikipedia.org/wiki/Toronto'})]
start=67 end=78 text='Olivia Chow' sources=[ToolSource(type='tool', id='web_search_6p4e6ba9e2cv:0', tool_output={'text': 'The mayor of Toronto is the head of Toronto City Council and chief executive officer of the municipal government.The mayor is elected alongside city council every four years on the fourth Monday of October; there are no term limits. [2] While in office, mayors are styled His/Her Worship. [3]Olivia Chow has served as the 66th and current mayor of Toronto since July 12, 2023, after winning the ...', 'url': 'https://en.wikipedia.org/wiki/Mayor_of_Toronto'})]
start=109 end=123 text='July 12, 2023.' sources=[ToolSource(type='tool', id='web_search_6p4e6ba9e2cv:0', tool_output={'text': 'The mayor of Toronto is the head of Toronto City Council and chief executive officer of the municipal government.The mayor is elected alongside city council every four years on the fourth Monday of October; there are no term limits. [2] While in office, mayors are styled His/Her Worship. [3]Olivia Chow has served as the 66th and current mayor of Toronto since July 12, 2023, after winning the ...', 'url': 'https://en.wikipedia.org/wiki/Mayor_of_Toronto'})]
How Does Multi-step Tool Use Work?
Section titled “How Does Multi-step Tool Use Work?”
Here’s an outline of the basic steps involved in multi-step tool use:
- Given a user request, the model comes up with a plan to solve the problem which answers questions such as "Which tools should be used," and "In what order should they be used."
- The model then carries out the plan by repeatedly executing actions (using whatever tools are appropriate), reasoning over the results, and re-evaluating the plan.
- After each Action -> Observation ->Reflection cycle, the model reflects about what to do next. This reflection involves analyzing what has been figured out so far, determining whether any changes need to be made to the plan, and what to do next. The model can take as many steps as it deems necessary.
- Once the model decides it knows how to answer the user question, it proceeds to generating the final response.
What is the difference between tool use and Retrieval Augmented Generation (RAG)?
Section titled “What is the difference between tool use and Retrieval Augmented Generation (RAG)?”Tool use is a natural extension of retrieval augmented generation (RAG). RAG is about enabling the model to interact with an information retrieval system (like a vector database). Our models are trained to be excellent at RAG use cases.
Tool use pushes this further, allowing Cohere models to go far beyond information retrieval, interact with search engines, APIs, functions, databases, and many other tools.
Structured Outputs (Tools)
Section titled “Structured Outputs (Tools)”Setting the strict_tools parameter to True will enforce each tool call to follow the specified tool schema. To learn more about this feature, visit the Structured Outputs documentation.
Note that strict_tools is currently an experimental feature.
response = co.chat(model="command-a-plus-05-2026", messages=messages, tools=tools, strict_tools=True)A Further Example With Multiple Tools
Section titled “A Further Example With Multiple Tools”This section provides another example of multi-step tool use, this time with multiple tools. The notebook for this example can be found in our repo.
This example demonstrates an agent that performs analysis on a Spotify tracks dataset (via a Python interpreter tool) while also having access to another tool: web search tool.
Step 1: Define the tools
Section titled “Step 1: Define the tools”Here, we define the web search tool, which uses the Tavily Python client to perform web searches.
# ! pip install tavily-python --q --disable-pip-version-check
from tavily import TavilyClient
tavily_client = TavilyClient(api_key="TAVILY_API_KEY")
# here's a web search engine
def web_search(query: str) -> list[dict]:
results = tavily_client.search(query, max_results=3)["results"]
return results
# the LLM is equipped with a description of the web search engine
web_search_tool = {
"type": "function",
"function": {
"name": "web_search",
"description": "Returns a list of relevant document snippets for a textual query retrieved from the internet",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Query to search the internet with",
}
},
"required": ["query"],
},
},
}Here, we define the Python interpreter tool, which uses the exec function to execute Python code.
# here's a python console, which can be used to access the spreadsheet, but also more generally to code and plot stuff
import io, contextlib
def python_interpreter(code: str) -> list[dict]:
output = io.StringIO()
try:
# Redirect stdout to capture print statements
with contextlib.redirect_stdout(output):
exec(code, globals())
except Exception as e:
return {"error": str(e), "executed_code": code}
# Get stdout
return [
{"console_output": output.getvalue(), "executed_code": code}
]
# the LLM is equipped with a description of a python console
python_interpreter_tool = {
"type": "function",
"function": {
"name": "python_interpreter",
"description": "Executes python code and returns the result. The code runs in a static sandbox without internet access and without interactive mode, so print output or save output to a file.",
"parameters": {
"type": "object",
"properties": {
"code": {
"type": "string",
"description": "Python code to execute",
}
},
"required": ["code"],
},
},
}
functions_map = {
"web_search": web_search,
"python_interpreter": python_interpreter,
}We'll also need the spotify_data dataset, which contains information about Spotify tracks such as the track information, release information, popularity metrics, and musical characteristics. You can find the dataset in our repo.
Here is the task that the agent needs to perform:
message = """What's the age and citizenship of the artists who had the top 3 most streamed songs on Spotify in 2023?
You have access to a dataset with information about Spotify songs from the past 10 years, located at ./spotify_dataset.csv.
You also have access to the internet to search for information not available in the dataset.
You must use the dataset when you can, and if stuck you can use the internet.
Remember to inspect the dataset and get a list of its columnsto understand its structure before trying to query it. Take it step by step.
"""Step 2: Run the tool use workflow
Section titled “Step 2: Run the tool use workflow”Next, we run the tool use workflow involving for steps:
- Get the user message
- Model generates tool calls, if any
- Execute tools based on the tool calls generated by the model
- Model either generates more tool calls or returns a response with citations
model = "command-a-plus-05-2026"
tools = [web_search_tool, python_interpreter_tool]
# Step 1: get user message
print(f"USER MESSAGE:\n{message}")
print("=" * 50)
messages = [{"role": "user", "content": message}]
# 2 - Model generates tool calls, if any
res = co.chat(
model=model, messages=messages, tools=tools, temperature=0.1
)
# Keep invoking tools as long as the model generates tool calls
while res.message.tool_calls:
# Tool plan and tool calls
print("\nTOOL PLAN:")
print(res.message.tool_plan)
print("\nTOOL CALLS:")
for tc in res.message.tool_calls:
if tc.function.name == "python_interpreter":
print(f"Tool name: {tc.function.name}")
tool_call_prettified = print(
"\n".join(
f" {line}"
for line_num, line in enumerate(
json.loads(tc.function.arguments)[
"code"
].splitlines()
)
)
)
print(tool_call_prettified)
else:
print(
f"Tool name: {tc.function.name} | Parameters: {tc.function.arguments}"
)
messages.append(res.message)
# 3 - Execute tools based on the tool calls generated by the model
print("\nTOOL RESULTS:")
for tc in res.message.tool_calls:
tool_result = functions_map[tc.function.name](
**json.loads(tc.function.arguments)
)
print(tool_result, "\n")
tool_content = []
for data in tool_result:
tool_content.append(
{
"type": "document",
"document": {"data": json.dumps(data)},
}
)
# Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated
messages.append(
{
"role": "tool",
"tool_call_id": tc.id,
"content": tool_content,
}
)
# 4 - Model either generates more tool calls or returns a response
res = co.chat(
model=model, messages=messages, tools=tools, temperature=0.1
)
messages.append(
{"role": "assistant", "content": res.message.content[0].text}
)
print("\nRESPONSE:")
print(res.message.content[0].text)
if res.message.citations:
print("\nCITATIONS:")
for citation in res.message.citations:
print(
f"Start: {citation.start} | End: {citation.end} | Text: '{citation.text}'"
)
print("Sources:")
if citation.sources:
for source in citation.sources:
print(source.id)
print("-" * 50)And here is an example output. In summary, the agent performs the task in a sequence of 3 steps:
- Inspect the dataset and get a list of its columns.
- Write and execute Python code to find the top 3 most streamed songs on Spotify in 2023 and their respective artists.
- Search for the age and citizenship of each artist on the internet.
USER MESSAGE:
What's the age and citizenship of the artists who had the top 3 most streamed songs on Spotify in 2023?
You have access to a dataset with information about Spotify songs from the past 10 years, located at ./spotify_dataset.csv.
You also have access to the internet to search for information not available in the dataset.
You must use the dataset when you can, and if stuck you can use the internet.
Remember to inspect the dataset and get a list of its columnsto understand its structure before trying to query it. Take it step by step.
==================================================
TOOL PLAN:
I will first inspect the dataset to understand its structure. Then, I will write and execute Python code to find the top 3 most streamed songs on Spotify in 2023. After that, I will search for the age and citizenship of the artists of these songs.
TOOL CALLS:
Tool name: python_interpreter
import pandas as pd
df = pd.read_csv("spotify_dataset.csv")
# Print the first 5 rows of the dataset
print(df.head())
# Print the column names
print(df.columns)
None
TOOL RESULTS:
[{'console_output': " track_name artist(s)_name artist_count \\\n0 Seven (feat. Latto) (Explicit Ver.) Latto, Jung Kook 2 \n1 LALA Myke Towers 1 \n2 vampire Olivia Rodrigo 1 \n3 Cruel Summer Taylor Swift 1 \n4 WHERE SHE GOES Bad Bunny 1 \n\n released_year released_month released_day in_spotify_playlists \\\n0 2023 7 14 553 \n1 2023 3 23 1474 \n2 2023 6 30 1397 \n3 2019 8 23 7858 \n4 2023 5 18 3133 \n\n in_spotify_charts streams in_apple_playlists ... key mode \\\n0 147 141381703.0 43 ... B Major \n1 48 133716286.0 48 ... C# Major \n2 113 140003974.0 94 ... F Major \n3 100 800840817.0 116 ... A Major \n4 50 303236322.0 84 ... A Minor \n\n danceability valence energy acousticness instrumentalness liveness \\\n0 80 89 83 31 0 8 \n1 71 61 74 7 0 10 \n2 51 32 53 17 0 31 \n3 55 58 72 11 0 11 \n4 65 23 80 14 63 11 \n\n speechiness release_date \n0 4 2023-07-14 \n1 4 2023-03-23 \n2 6 2023-06-30 \n3 15 2019-08-23 \n4 6 2023-05-18 \n\n[5 rows x 25 columns]\nIndex(['track_name', 'artist(s)_name', 'artist_count', 'released_year',\n 'released_month', 'released_day', 'in_spotify_playlists',\n 'in_spotify_charts', 'streams', 'in_apple_playlists', 'in_apple_charts',\n 'in_deezer_playlists', 'in_deezer_charts', 'in_shazam_charts', 'bpm',\n 'key', 'mode', 'danceability', 'valence', 'energy', 'acousticness',\n 'instrumentalness', 'liveness', 'speechiness', 'release_date'],\n dtype='object')\n", 'executed_code': 'import pandas as pd\r\n\r\ndf = pd.read_csv("spotify_dataset.csv")\r\n\r\n# Print the first 5 rows of the dataset\r\nprint(df.head())\r\n\r\n# Print the column names\r\nprint(df.columns)'}]
TOOL PLAN:
The dataset contains a column called 'artist(s)_name' which contains the names of the artists. It also contains a column called 'released_year' which contains the year the song was released.
I will now write and execute Python code to find the top 3 most streamed songs on Spotify in 2023.
TOOL CALLS:
Tool name: python_interpreter
import pandas as pd
df = pd.read_csv("spotify_dataset.csv")
# Filter the dataset to only include songs released in 2023
df_2023 = df[df["released_year"] == 2023]
# Sort the dataset by the number of streams in descending order
df_2023_sorted = df_2023.sort_values("streams", ascending=False)
# Print the top 3 most streamed songs
print(df_2023_sorted.head(3))
# Print the names of the artists of the top 3 most streamed songs
print(df_2023_sorted.head(3)["artist(s)_name"])
None
TOOL RESULTS:
[{'console_output': ' track_name artist(s)_name \\\n12 Flowers Miley Cyrus \n6 Ella Baila Sola Eslabon Armado, Peso Pluma \n133 Shakira: Bzrp Music Sessions, Vol. 53 Shakira, Bizarrap \n\n artist_count released_year released_month released_day \\\n12 1 2023 1 12 \n6 2 2023 3 16 \n133 2 2023 1 11 \n\n in_spotify_playlists in_spotify_charts streams \\\n12 12211 115 1.316856e+09 \n6 3090 50 7.259801e+08 \n133 5724 44 7.219756e+08 \n\n in_apple_playlists ... key mode danceability valence energy \\\n12 300 ... NaN Major 71 65 68 \n6 34 ... F Minor 67 83 76 \n133 119 ... D Minor 78 50 63 \n\n acousticness instrumentalness liveness speechiness release_date \n12 6 0 3 7 2023-01-12 \n6 48 0 8 3 2023-03-16 \n133 27 0 9 5 2023-01-11 \n\n[3 rows x 25 columns]\n12 Miley Cyrus\n6 Eslabon Armado, Peso Pluma\n133 Shakira, Bizarrap\nName: artist(s)_name, dtype: object\n', 'executed_code': 'import pandas as pd\r\n\r\ndf = pd.read_csv("spotify_dataset.csv")\r\n\r\n# Filter the dataset to only include songs released in 2023\r\ndf_2023 = df[df["released_year"] == 2023]\r\n\r\n# Sort the dataset by the number of streams in descending order\r\ndf_2023_sorted = df_2023.sort_values("streams", ascending=False)\r\n\r\n# Print the top 3 most streamed songs\r\nprint(df_2023_sorted.head(3))\r\n\r\n# Print the names of the artists of the top 3 most streamed songs\r\nprint(df_2023_sorted.head(3)["artist(s)_name"])'}]
TOOL PLAN:
I have found that the top 3 most streamed songs on Spotify in 2023 are:
1. Flowers by Miley Cyrus
2. Ella Baila Sola by Eslabon Armado, Peso Pluma
3. Shakira: Bzrp Music Sessions, Vol. 53 by Shakira, Bizarrap
I will now search the internet for the age and citizenship of these artists.
TOOL CALLS:
Tool name: web_search | Parameters: {"query":"Miley Cyrus age"}
Tool name: web_search | Parameters: {"query":"Miley Cyrus citizenship"}
Tool name: web_search | Parameters: {"query":"Eslabon Armado age"}
Tool name: web_search | Parameters: {"query":"Eslabon Armado citizenship"}
Tool name: web_search | Parameters: {"query":"Peso Pluma age"}
Tool name: web_search | Parameters: {"query":"Peso Pluma citizenship"}
Tool name: web_search | Parameters: {"query":"Shakira age"}
Tool name: web_search | Parameters: {"query":"Shakira citizenship"}
Tool name: web_search | Parameters: {"query":"Bizarrap age"}
Tool name: web_search | Parameters: {"query":"Bizarrap citizenship"}
TOOL RESULTS:
[{'title': 'Miley Cyrus Height, Weight, Age, Husband, Bio, Net Worth, Facts', 'url': 'https://www.celebsline.com/miley-cyrus/', 'content': 'Miley Cyrus Height, Weight, Age, Husband, Bio, Net Worth, Facts Miley Cyrus ...', 'score': 0.853669, 'raw_content': None}]
[{'title': "Miley Cyrus Talks Global Citizen's Fight to End Inequality, COVID-19", 'url': 'https://www.rollingstone.com/music/music-news/miley-cyrus-global-citizen-global-goal-interview-1021734/', 'content': "Miley Cyrus Talks Fighting Inequality, ...", 'score': 0.39589298, 'raw_content': None}]
[{'title': 'Eslabon Armado - Wikipedia', 'url': 'https://en.wikipedia.org/wiki/Eslabon_Armado', 'content': 'Eslabon Armado is an American regional Mexican group from Patterson, California, formed in 2017.The group consists of Pedro Tovar ...', 'score': 0.9923638, 'raw_content': None}]
...
...
RESPONSE:
The top 3 most streamed songs on Spotify in 2023 were:
1. 'Flowers' by Miley Cyrus
2. 'Ella Baila Sola' by Eslabon Armado and Peso Pluma
3. 'Shakira: Bzrp Music Sessions, Vol. 53' by Shakira and Bizarrap
Miley Cyrus is 31 years old and is an American citizen.
Eslabon Armado is a group of musicians from California, USA, formed in 2017. The group consists of Pedro Tovar (lead vocals), Brian Tovar (bass), Ulises González (acoustic guitar), and Damián Pacheco (twelve-string guitar). The members of the group are all in their early 20s.
Peso Pluma is a 24-year-old rapper, singer, and songwriter from Mexico.
Shakira is a 46-year-old Colombian singer, songwriter, and philanthropist.
Bizarrap is a 25-year-old Argentine DJ and producer.
CITATIONS:
Start: 58 | End: 67 | Text: ''Flowers''
Sources:
python_interpreter_sbccqz8vt4yj:0
--------------------------------------------------
Start: 71 | End: 82 | Text: 'Miley Cyrus'
Sources:
python_interpreter_sbccqz8vt4yj:0
--------------------------------------------------
Start: 86 | End: 103 | Text: ''Ella Baila Sola''
Sources:
python_interpreter_sbccqz8vt4yj:0
--------------------------------------------------
Start: 107 | End: 136 | Text: 'Eslabon Armado and Peso Pluma'
Sources:
python_interpreter_sbccqz8vt4yj:0
--------------------------------------------------
Start: 140 | End: 179 | Text: ''Shakira: Bzrp Music Sessions, Vol. 53''
Sources:
python_interpreter_sbccqz8vt4yj:0
--------------------------------------------------
Start: 183 | End: 203 | Text: 'Shakira and Bizarrap'
Sources:
python_interpreter_sbccqz8vt4yj:0
--------------------------------------------------
Start: 220 | End: 232 | Text: '31 years old'
Sources:
web_search_kvbq3v3yexkc:1
--------------------------------------------------
Start: 243 | End: 260 | Text: 'American citizen.'
Sources:
web_search_kvbq3v3yexkc:1
web_search_kvbq3v3yexkc:2
...