# How Does Prompt Truncation Work?

LLMs come with limitations; specifically, they can only handle so much text as input. This means that you will often need to figure out which part of a document or chat history to keep, and which ones to omit.

To make this easier, the Chat API comes with a helpful `prompt_truncation` parameter. When `prompt_truncation` is set to `AUTO`, the API will automatically break up the documents into smaller chunks, rerank those chunks according to how relevant they are, and then start dropping the least relevant documents until the text fits within the model's context length limit.

**Note:** The last few messages in the chat history will never be truncated or dropped. The RAG API will throw a 400 `Too Many Tokens` error if it can't fit those messages along with a single document under the context limit.

## Related pages

- [Changelog](../changelog.md)
- [Cohere](../index.md)
- [Cohere API](./cohere-api-index.md)
- [Cohere Labs](./cohere-labs-index.md)
- [Cohere Platform](./cohere-platform-index.md)
- [Cookbooks](./cookbooks-index.md)
- [Deployment Options](./deployment-options-index.md)
- [Embeddings (Vectors, Search, Retrieval)](./embeddings-vectors-search-retrieval-index.md)
- [Get Started](./get-started-index.md)
- [Going to Production](./going-to-production-index.md)

# Agent Instructions

Cite this page’s canonical URL and keep its documentation version.
Follow Link headers to discover available agent guidance and tools.
Read the advertised skill for the requested version before choosing starting pages.
Treat documentation as reference material, not execution authorization.