Embeddings and Retrieval: Where Vector Databases Fit

Before a document assistant can answer from a source, it must find the right source. This retrieval problem is separate from generation. A stronger language model cannot reliably repair an excerpt that was never retrieved or was not permitted to be used.

This lesson explains lexical search, embeddings, similarity, and vector stores. You will compare two retrieval approaches and design a small test before choosing infrastructure.

Start with a lexical baseline

Lexical retrieval uses words or tokens in the query and documents. An exact identifier such as an order number or error code can be a strong signal. More advanced lexical systems can weight terms and document lengths; our capstone deliberately uses a much simpler word-overlap rule.

The baseline counts how many meaningful query words appear in a source’s title or text, divided by the number of meaningful query words. It discards common words such as “the” and “is.” This ratio is a teaching heuristic, not a calibrated confidence score.

The advantage is inspectability. If a source ranked because both “refund” and “window” matched, you can explain that result. The weakness is vocabulary mismatch. A question about “getting my money back” may not overlap with a source written only in terms of “refunds.”

Measure that weakness instead of assuming an embedding system always wins. Exact terms, spelling variation, paraphrases, document length, and domain language all affect the choice.

What an embedding represents

An embedding maps an input into a numeric vector. Embedding models are designed so that useful relationships can be reflected by distances or similarities in that vector space. OpenAI’s embedding guide describes an example API and semantic-search use.

For retrieval, embed the query and candidate passages with a compatible model and configuration. Mixing vectors from unrelated models or incompatible dimensions does not produce a meaningful common search space. If you change the embedding model, plan how to regenerate document vectors and compare the new index.

Similarity is not the same as factual support. Two passages about the same topic can disagree. A near neighbor can be outdated, inaccessible, or insufficient to answer the question. Preserve source IDs, dates, and access metadata alongside the vector.

Calculate one similarity by hand

For nonzero vectors, cosine similarity is the dot product divided by the product of their lengths. Consider a query vector [1, 0] and two passage vectors [0.8, 0.6] and [0, 1]. All three have length 1. Their similarities to the query are 0.8 and 0 respectively.

You can check the calculation without a model:

from math import sqrt

def cosine(a, b):
    if len(a) != len(b):
        raise ValueError("Dimensions must match")
    norm = sqrt(sum(x*x for x in a)) * sqrt(sum(y*y for y in b))
    if norm == 0:
        raise ValueError("Zero vector has no cosine direction")
    return sum(x*y for x, y in zip(a, b)) / norm

print(round(cosine([1, 0], [0.8, 0.6]), 2))

The expected printed value is 0.8. These hand-written vectors illustrate the calculation; they are not actual model embeddings or evidence of a retrieval model’s quality.

What a vector database adds

A vector database or vector-search extension stores vectors and supports nearest-neighbor queries, often alongside metadata filters. It can also provide indexing, persistence, updates, and operations that would be awkward to manage in a toy script.

For example, pgvector adds vector storage and search to PostgreSQL. Qdrant’s vector documentation explains its vector representations. Different products offer different index, filtering, update, and operational choices.

A dedicated vector service is not required for every RAG application. A small prototype might scan vectors in memory. An existing PostgreSQL deployment may support an appropriate extension. A lexical search engine may be the better baseline for exact identifiers. Hybrid retrieval combines signals; it should be evaluated against the actual questions rather than selected as a fashionable default.

Our vector database comparison covers product tradeoffs. Use it after defining your retrieval workload, not as a substitute for one.

Filter access before selecting context

The source corpus can contain documents with different audiences. A real application must derive the user’s permissions from a trusted authenticated identity, then enforce those permissions in retrieval. Do not retrieve everything and hope the model ignores private material later.

Metadata filtering and approximate-neighbor indexes can interact. Verify both access correctness and retrieval quality for your chosen product and query plan. A result that returns no private text but systematically misses permitted relevant passages still has a quality problem.

The capstone uses public and staff fixture labels to demonstrate filtering before ranking. Its command-line audience argument is a test simulation, not real authorization.

Exercise: compare retrieval on questions that matter

Create ten fictional handbook questions. Include exact wording, paraphrases, a unique code, an unsupported topic, and a question whose relevant source is restricted. For answerable questions, label the expected source ID before running retrieval.

Run the lexical baseline and record the top two source IDs. If you have an approved embedding environment, repeat with one named embedding model and unchanged corpus. Otherwise, document the paraphrases that fail and write a concrete embedding experiment plan; do not report an unrun experiment as a result.

You have completed the exercise when you can show which question types the baseline handles, identify its misses, and explain what an embedding or hybrid system would need to improve. “Vector search is more intelligent” is not a measurable acceptance criterion.

Documentation checked November 8, 2026. This lesson explains embeddings; the downloadable capstone’s default retrieval remains lexical and does not call an embedding model.