Bulk Embedding (might be redundant)
This document explains how to efficiently embed text in bulk using an endpoint that returns a JSONL file of text embeddings with their respective text, which can be downloaded within 2 months.
If you have a large collection of text rather than a single file, bulk embedding may be more efficient for you. This endpoint returns a JSONL file of text embeddings with their respective text. The JSONL will be available for 2 months and downloadable via the download_file().
Example Requests
Section titled “Example Requests”Create A Bulk Embed
Section titled “Create A Bulk Embed”Request
Section titled “Request”# Request
co.bulk_embed(
model="string",
url="string", # (url to gcp file)
file_id="string", # (id to uploaded file)
text_field="string", # (column to get text from)
truncate="string", # {LEFT:DEFAULT,RIGHT}
)Response
Section titled “Response”{job_id: "string"}Retrieve A Bulk Embed
Section titled “Retrieve A Bulk Embed”Request
Section titled “Request”co.get_bulk_embed(job_id="string")Response
Section titled “Response”# Response of co.get_bulk_embed():
{
job_id: "string",
status: "string", # (COMPLETE, QUEUED, FAILED or RUNNING)
created_at: "string", # (timestamp)
updated_at: "string", # (timestamp)
input_url: "string",
input_file_id: "string",
output_file_id: "string",
model: "string",
truncate: "string", # (LEFT or RIGHT)
percent_complete: float,
}List a Bulk Embed
Section titled “List a Bulk Embed”Request
Section titled “Request”co.list_bulk_embed()Response
Section titled “Response”# Response of co.list_bulk_embeds():
{
bulk_embed: [
{
job_id: "string",
status: "string", # (COMPLETE, QUEUED, FAILED or RUNNING)
created_at: "string", # (timestamp)
updated_at: "string", # (timestamp)
input_url: "string",
input_file_id: "string",
output_file_id: "string",
model: "string",
truncate: "string", # (LEFT or RIGHT)
percent_complete: float,
},
...,
]
}