Vector Databases & Embeddings

Understand how vector databases store numerical representations of data to enable semantic search, allowing analysts to find information by meaning rather than exact keyword matches.

What it is

A vector database is a specialized system designed to store and query high-dimensional vectors. A vector embedding is a numerical representation (an array of numbers) generated by an AI model that captures the semantic meaning of text, images, or other data points. Unlike traditional relational databases that match exact strings, vector databases calculate mathematical similarity between these embeddings to find related items.

The mental model is a map where similar concepts are located close together. For example, "car" and "automobile" will have vectors that are mathematically near each other, while "banana" will be far away. Key terms include embedding model, dimensionality, and similarity metric (such as cosine similarity).

Why it matters

• Semantic Search: Users can search for "fast cars" and find documents about "high-performance vehicles" even if those exact words aren't present.
• Unstructured Data Analysis: Enables querying large volumes of text logs, customer feedback, or reports without manual tagging.
• Recommendation Systems: Finds similar products or articles based on user behavior patterns encoded in vectors.
• RAG Pipelines: Essential for Retrieval-Augmented Generation, where relevant context is fetched from a knowledge base to answer questions accurately.

Syntax or steps

1. Generate Embeddings: Use a pre-trained model to convert raw text into fixed-length numerical arrays.
2. Store Vectors: Insert the embeddings along with metadata into the vector database.
3. Query Vector: Convert the search term into an embedding using the same model.
4. Calculate Similarity: The database compares the query vector against stored vectors using a distance function.
5. Retrieve Results: Return the top k most similar items.

Example

This Python example uses chromadb (a lightweight vector DB) and sentence-transformers to demonstrate semantic search.

import chromadb
from sentence_transformers import SentenceTransformer

# 1. Initialize client and embedding model
client = chromadb.Client()
collection = client.create_collection(name="analyst_notes")
model = SentenceTransformer('all-MiniLM-L6-v2')

# 2. Prepare data and generate embeddings
documents = [
    "Q3 revenue increased due to new product launch.",
    "Server latency issues caused downtime last night.",
    "Customer satisfaction scores dropped in APAC region."
]
embeddings = model.encode(documents).tolist()

# 3. Add to database
collection.add(
    ids=["doc1", "doc2", "doc3"],
    embeddings=embeddings,
    documents=documents
)

# 4. Perform semantic search
query_text = "sales performance went up"
query_embedding = model.encode([query_text]).tolist()

results = collection.query(
    query_embeddings=query_embedding,
    n_results=1
)

print(results['documents'][0][0])
# Output: Q3 revenue increased due to new product launch.

Explanation: We initialize a local ChromaDB instance. The SentenceTransformer converts three distinct business notes into vectors. When we query with "sales performance went up," the system does not look for the word "sales." Instead, it finds the vector closest to this phrase, which corresponds to the document about "revenue increased," demonstrating semantic understanding.

Common mistakes

• Mismatched Models: Using different embedding models for indexing and querying. You must use the exact same model version for both steps.
• Ignoring Metadata: Storing only vectors makes filtering difficult. Always attach metadata (like date or source) to allow hybrid searches.
• High Dimensionality Without Need: Using massive 768-dim vectors when smaller ones suffice increases storage costs and slows down queries.
• Assuming Perfect Accuracy: Semantic search is probabilistic. It may return irrelevant results if the training data didn't cover specific jargon.

When to use it

Use vector databases when users ask natural language questions or when synonyms matter. Use traditional databases when you need precise counts, sums, or exact ID lookups.

Practice

Guided Exercise: Modify the example above to add two more documents about "marketing budget cuts" and "employee retention rates." Query for "staff staying longer" and observe which document is returned.

Challenge: Implement a filter so that the search only returns documents added after a specific date. Hint: Use the where parameter in the collection.query() method with metadata.

Quick check

Question: Why must the embedding model used during indexing be identical to the one used during querying?

Answer: Because the vector space coordinates are relative to the specific model's training. Different models produce different numerical representations for the same text, making them incomparable.

Summary

Vector databases transform unstructured data into numerical embeddings to enable semantic search, finding content by meaning rather than keywords. They are essential for modern analytics involving natural language but require careful management of embedding models and metadata to be effective.