Vector databases for embedding systems

Introduction to Embeddings with the OpenAI API

Emmanuel Pire

Senior Software Engineer, DataCamp

Limitations of the current approach

Loading all the embeddings into memory (1536 floats ~ 13kB/embedding)
Recalculated embeddings for each new query
Calculating cosine distances for every embedding and sorting is slow and scales linearly

A diagram showing how the documents and query are embedded with every query with no storage solution.

Vector databases

Embedded documents are stored and queried from the vector database

A diagram showing how a vector database is used to store document embeddings and returned results based on an embedded query back to the user.

NoSQL Database

More flexible structure that allows for faster querying

Examples of NoSQL database schemas, including key:value, document, and graph databases.

SQL/Relational Database

Structured data into tables, rows, and columns

Tables with columns and rows in a SQL, or relational, database.

Components to store

Embeddings
Source texts
Metadata
- IDs and references
- Additional data useful for filtering results

Top tip: Don't store the source text as metadata!

A vector database icon.

The vector database landscape

Many of the most popular vector database solutions, sorted by whether they are closed source or open source, and whether they are dedicated vectors databases or not.

¹ Image Credit: Yingjun Wu

Which solution is best?

Database management:
- Managed → more expensive but lowers workload
- Self-managed → cheaper but requires time and expertise
Open source or commercial?
- Open source → flexible and cost-effective
- Commercial → better support, more advanced features, and compliance

Data models: does the type of data lend itself to a particular database type?
Specific features: does your use case depend on specific functionality, such as multi-modal storage?

Let's practice!

Introduction to Embeddings with the OpenAI API