Semantic search - quickstart
About semantic search
Section titled “About semantic search”Cohere's embedding models are available via the Embed endpoint. This endpoint enables you to embed text documents (multilingual) and images into a vector space.
Semantic search, powered by embeddings, enables applications to perform information retrieval based on the context or meaning of a document.
This quickstart guide shows you how to perform semantic search with the Embed endpoint.
Setup
First, install the Cohere Python SDK with the following command.
Bash pip install -U cohereNext, import the library and create a client.
PYTHON import cohere co = cohere.ClientV2( "COHERE_API_KEY" ) # Get your free API key here: https://dashboard.cohere.com/api-keysPYTHON import cohere co = cohere.ClientV2( api_key="", # Leave this blank base_url="<YOUR_DEPLOYMENT_URL>", )PYTHON import cohere co = cohere.BedrockClientV2( aws_region="AWS_REGION", aws_access_key="AWS_ACCESS_KEY_ID", aws_secret_key="AWS_SECRET_ACCESS_KEY", aws_session_token="AWS_SESSION_TOKEN", ) # Get the model name: https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.htmlPYTHON import cohere co = cohere.SagemakerClientV2( aws_region="AWS_REGION", aws_access_key="AWS_ACCESS_KEY_ID", aws_secret_key="AWS_SECRET_ACCESS_KEY", aws_session_token="AWS_SESSION_TOKEN", )PYTHON import cohere co = cohere.ClientV2( api_key="AZURE_API_KEY", base_url="AZURE_ENDPOINT", # example: "https://cohere-embed-v3-multilingual-xyz.eastus.models.ai.azure.com/" )Document Embeddings
First, embed the list of available documents using the Embed endpoint by specifying the
input_typeassearch_document.PYTHON # Define the documents documents = [ "Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged.", "Finding Coffee Spots: For your caffeine fix, cross the street to the café for artisan coffee.", "Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.", ] # Embed the documents doc_emb = co.embed( model="embed-v4.0", input_type="search_document", texts=documents, embedding_types=["float"], ).embeddings.floatPYTHON # Define the documents documents = [ "Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged.", "Finding Coffee Spots: For your caffeine fix, cross the street to the café for artisan coffee.", "Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.", ] # Embed the documents doc_emb = co.embed( model="embed-v4.0", input_type="search_document", texts=documents, embedding_types=["float"], ).embeddings.floatPYTHON # Define the documents documents = [ "Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged.", "Finding Coffee Spots: For your caffeine fix, cross the street to the café for artisan coffee.", "Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.", ] # Embed the documents doc_emb = co.embed( model="YOUR_MODEL_NAME", input_type="search_document", texts=documents, embedding_types=["float"], ).embeddings.floatPYTHON # Define the documents documents = [ "Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged.", "Finding Coffee Spots: For your caffeine fix, cross the street to the café for artisan coffee.", "Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.", ] # Embed the documents doc_emb = co.embed( model="YOUR_ENDPOINT_NAME", input_type="search_document", texts=documents, embedding_types=["float"], ).embeddings.floatPYTHON # Define the documents documents = [ "Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged.", "Finding Coffee Spots: For your caffeine fix, cross the street to the café for artisan coffee.", "Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.", ] # Embed the documents doc_emb = co.embed( input_type="search_document", texts=documents, embedding_types=["float"], ).embeddings.floatQuery Embedding
Next, embed the user query using the Embed endpoint by specifying the
input_typeassearch_query.PYTHON # Add the user query query = "Ways to connect with my teammates" # Embed the query query_emb = co.embed( model="embed-v4.0", input_type="search_query", texts=[query], embedding_types=["float"], ).embeddings.floatPYTHON # Add the user query query = "Ways to connect with my teammates" # Embed the query query_emb = co.embed( model="embed-v4.0", input_type="search_query", texts=[query], embedding_types=["float"], ).embeddings.floatPYTHON # Add the user query query = "Ways to connect with my teammates" # Embed the query query_emb = co.embed( model="YOUR_MODEL_NAME", input_type="search_query", texts=[query], embedding_types=["float"], ).embeddings.floatPYTHON # Add the user query query = "Ways to connect with my teammates" query_emb = co.embed( model="embed-v4.0", input_type="search_query", texts=[query], embedding_types=["float"], ).embeddings.floatPYTHON # Add the user query query = "Ways to connect with my teammates" query_emb = co.embed( model="embed-v4.0", input_type="search_query", texts=[query], embedding_types=["float"], ).embeddings.floatSemantic Search
Then, perform semantic search by computing the similarity between the query embedding and the document embeddings, and then returning the most similar documents.
PYTHON import numpy as np # Compute dot product similarity and display results def return_results(query_emb, doc_emb, documents): n = 2 # customize your top N results scores = np.dot(query_emb, np.transpose(doc_emb))[0] max_idx = np.argsort(-scores)[:n] for rank, idx in enumerate(max_idx): print(f"Rank: {rank+1}") print(f"Score: {scores[idx]}") print(f"Document: {documents[idx]}\n") return_results(query_emb, doc_emb, documents)wordWrap Rank: 1 Score: 0.262197161387274 Document: Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged. Rank: 2 Score: 0.1266074257723145 Document: Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.