Semantic search - quickstart
A quickstart guide for performing text semantic search with Cohere's Embed models (v2 API).
About semantic search
Section titled “About semantic search”Cohere's embedding models are available via the Embed endpoint. This endpoint enables you to embed text documents (multilingual) and images into a vector space.
Semantic search, powered by embeddings, enables applications to perform information retrieval based on the context or meaning of a document.
This quickstart guide shows you how to perform semantic search with the Embed endpoint.
Setup
First, install the Cohere Python SDK with the following command.
Bash pip install -U cohereNext, import the library and create a client.
PYTHON import cohere co = cohere.ClientV2( "COHERE_API_KEY" ) # Get your free API key here: https://dashboard.cohere.com/api-keysPYTHON import cohere co = cohere.ClientV2( api_key="", # Leave this blank base_url="<YOUR_DEPLOYMENT_URL>", )PYTHON import cohere co = cohere.BedrockClientV2( aws_region="AWS_REGION", aws_access_key="AWS_ACCESS_KEY_ID", aws_secret_key="AWS_SECRET_ACCESS_KEY", aws_session_token="AWS_SESSION_TOKEN", ) # Get the model name: https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.htmlPYTHON import cohere co = cohere.SagemakerClientV2( aws_region="AWS_REGION", aws_access_key="AWS_ACCESS_KEY_ID", aws_secret_key="AWS_SECRET_ACCESS_KEY", aws_session_token="AWS_SESSION_TOKEN", )PYTHON import cohere co = cohere.ClientV2( api_key="AZURE_API_KEY", base_url="AZURE_ENDPOINT", # example: "https://cohere-embed-v3-multilingual-xyz.eastus.models.ai.azure.com/" )Document Embeddings
First, embed the list of available documents using the Embed endpoint by specifying the
input_typeassearch_document.PYTHON # Define the documents documents = [ "Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged.", "Finding Coffee Spots: For your caffeine fix, cross the street to the café for artisan coffee.", "Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.", ] # Embed the documents doc_emb = co.embed( model="embed-v5.0-pro", input_type="search_document", texts=documents, embedding_types=["float"], ).embeddings.floatPYTHON # Define the documents documents = [ "Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged.", "Finding Coffee Spots: For your caffeine fix, cross the street to the café for artisan coffee.", "Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.", ] # Embed the documents doc_emb = co.embed( model="embed-v4.0", input_type="search_document", texts=documents, embedding_types=["float"], ).embeddings.floatPYTHON # Define the documents documents = [ "Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged.", "Finding Coffee Spots: For your caffeine fix, cross the street to the café for artisan coffee.", "Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.", ] # Embed the documents doc_emb = co.embed( model="YOUR_MODEL_NAME", input_type="search_document", texts=documents, embedding_types=["float"], ).embeddings.floatPYTHON # Define the documents documents = [ "Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged.", "Finding Coffee Spots: For your caffeine fix, cross the street to the café for artisan coffee.", "Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.", ] # Embed the documents doc_emb = co.embed( model="YOUR_ENDPOINT_NAME", input_type="search_document", texts=documents, embedding_types=["float"], ).embeddings.floatPYTHON # Define the documents documents = [ "Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged.", "Finding Coffee Spots: For your caffeine fix, cross the street to the café for artisan coffee.", "Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.", ] # Embed the documents doc_emb = co.embed( input_type="search_document", texts=documents, embedding_types=["float"], ).embeddings.floatQuery Embedding
Next, embed the user query using the Embed endpoint by specifying the
input_typeassearch_query.PYTHON # Add the user query query = "Ways to connect with my teammates" # Embed the query query_emb = co.embed( model="embed-v5.0-pro", input_type="search_query", texts=[query], embedding_types=["float"], ).embeddings.floatPYTHON # Add the user query query = "Ways to connect with my teammates" # Embed the query query_emb = co.embed( model="embed-v4.0", input_type="search_query", texts=[query], embedding_types=["float"], ).embeddings.floatPYTHON # Add the user query query = "Ways to connect with my teammates" # Embed the query query_emb = co.embed( model="YOUR_MODEL_NAME", input_type="search_query", texts=[query], embedding_types=["float"], ).embeddings.floatPYTHON # Add the user query query = "Ways to connect with my teammates" query_emb = co.embed( model="embed-v5.0-pro", input_type="search_query", texts=[query], embedding_types=["float"], ).embeddings.floatPYTHON # Add the user query query = "Ways to connect with my teammates" query_emb = co.embed( model="embed-v5.0-pro", input_type="search_query", texts=[query], embedding_types=["float"], ).embeddings.floatSemantic Search
Then, perform semantic search by computing the similarity between the query embedding and the document embeddings, and then returning the most similar documents.
PYTHON import numpy as np # Compute dot product similarity and display results def return_results(query_emb, doc_emb, documents): n = 2 # customize your top N results scores = np.dot(query_emb, np.transpose(doc_emb))[0] max_idx = np.argsort(-scores)[:n] for rank, idx in enumerate(max_idx): print(f"Rank: {rank+1}") print(f"Score: {scores[idx]}") print(f"Document: {documents[idx]}\n") return_results(query_emb, doc_emb, documents)wordWrap Rank: 1 Score: 0.28632629649873403 Document: Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged. Rank: 2 Score: 0.08908519767854908 Document: Finding Coffee Spots: For your caffeine fix, cross the street to the café for artisan coffee.