Vector Database for RAG: Embeddings Explained for Beginners

Say you've built an HR chatbot. An employee types, "How many days off do I get?"

Your policy document has the answer. It says, "Annual leave: 24 days per year." A person reads both lines and knows they mean the same thing. A keyword search reads both and shrugs, because the two sentences share almost no words.

Closing that gap is why embeddings and vector databases exist. This guide covers what an embedding is, what dimensions and similarity metrics do, how dense and sparse vectors differ, and where an index stops and a namespace starts. If you're planning to build RAG features into a product, this is the vocabulary you'll need first.

Key Takeaways

• An embedding is a fixed-length list of numbers that captures meaning. Popular models output 384, 1,024 or 1,536 numbers per text.
• Your index dimension must match your embedding model exactly, and Pinecone won't let you change the metric after creation.
• Start with dense vectors and cosine similarity. Add sparse vectors for hybrid search only when exact-keyword misses start to hurt.

Why Does RAG Need a Vector Database?

A vector database for RAG exists because LLMs haven't read your private data. ChatGPT and Claude were trained on public text with a fixed cutoff, so they've never seen your HR policy, your product manuals or last week's PDFs. Ask about those, and the model guesses.

RAG (retrieval-augmented generation) fixes that with a search step. The user asks a question, your system digs out the few chunks of text that look relevant, and those chunks go to the LLM along with the question. The model then answers from your text instead of from memory.

That digging step is the vector database's job. It's also where most of the generative AI tools and frameworks you've heard of plug in.

Why not plain keyword search? Because keyword search matches words, while semantic search matches meaning. Here's the same question both ways:

What Is an Embedding?

An embedding is a list of numbers that represents the meaning of a piece of text. OpenAI's text-embedding-3-small, for example, turns any input of up to 8,191 tokens into exactly 1,536 numbers, whether you feed it one word or three pages.

Think of latitude and longitude. Just two numbers pinpoint any spot on Earth, and cities close to each other have similar coordinates. An embedding model does the same thing for sentences, so sentences with similar meanings end up with similar numbers.

"I love pizza"           ->  [ 0.21, -0.53,  0.88, ... ]
"Pizza is my favorite"   ->  [ 0.20, -0.50,  0.90, ... ]   # very close
"The stock market fell"  ->  [-0.75,  0.12, -0.30, ... ]   # far away

These are short sentences to keep the examples simple. In a real RAG app, you embed chunks, which are larger pieces of a document. We cover how to split them in our guide to chunking strategies for RAG

What tools create vector embeddings?

The tool that handles this conversion is an embedding model. You've probably heard of a few: OpenAI's text-embedding-3-small, the open-source all-MiniLM-L6-v2 on Hugging Face, or models from Cohere and Pinecone. Frameworks such as Spring AI wrap these models behind one API, and Postgres-based stacks like Supabase can store the output with the pgvector extension.

What comes out is a vector. Technically the two words aren't identical, but people say "embedding" and "vector" to mean the same thing. You store a large collection of them in a vector database, so every lookup returns the closest matches in milliseconds.

Don't LLMs use vectors too?

Yes, and you're right to ask. Both kinds of model turn words into numbers behind the scenes. They just do very different things with them.

An embedding model is built to give you one final, fixed-size vector for your text. That vector is the whole point: it's what you save and search later. An LLM, on the other hand, pushes its numbers through dozens of layers to predict the next word, then the next, until it writes a full response. Those internal numbers aren't kept in a database once the answer is done.

In a RAG setup, one model finds and the other writes.

What does vector similarity look like in practice?

Here's an example with three numbers per vector. Real models use hundreds.

Cosine similarity between the first two rows comes out at about 0.997, which is nearly identical. The first row against the weather question scores about -0.01, so nothing in common. Semantic search is that same comparison, repeated across thousands or millions of records.

What Are Embedding Dimensions?

Dimensions are simply how many numbers sit in the vector. A model that outputs 384 numbers produces 384-dimension embeddings. Popular models range from 384 dimensions (all-MiniLM-L6-v2) to 1,536 (OpenAI's text-embedding-3-small), and that number decides how you configure your index.

More dimensions can hold more nuance, the same way 384 facts about a person say more than two. They also cost more to store and to search. Bigger isn't automatically better.

One rule you can't bend: the index dimension has to equal the model's output. A model that returns 384 numbers needs a 384-dimension index. Mix them up and Pinecone throws a dimension mismatch error. Each model also has a maximum input length, so your chunks must fit inside it, or the extra text is silently dropped.

What Is the Difference Between Dense and Sparse Vectors?

Dense vectors capture meaning; sparse vectors capture exact keywords. A dense vector fills nearly every one of its hundreds of positions with a number. A sparse vector can span tens of thousands of positions, one per vocabulary term, yet hold values in only the handful that match words actually in the text.

Dense vectors are the ones we've been looking at so far. A neural model produces them, and they pick up on meaning, so "refund" and "money back" land close together.

Sparse vectors are the opposite shape. They're very long and mostly zeros, with weight only on the words that really appear. BM25, TF-IDF and SPLADE are the usual methods. Because they're so empty, they're stored as pairs of position and weight rather than as full lists.

When should you use hybrid search?

Plenty of production systems run both and call it hybrid search. Dense handles meaning; sparse handles product codes, SKUs and error strings where the exact text matters.

Just starting out? Skip sparse. In our experience, a dense-only setup answers most natural-language questions well, and you'll know when you need hybrid: users start searching for codes and part numbers and getting near-misses back.

Cosine Similarity vs Dot Product vs Euclidean: Which Metric Should You Use?

Once text becomes vectors, the database still needs a rule for "how alike are these two?" That rule is the metric, and Pinecone offers three: cosine, dot product and Euclidean. You pick one when the index is created and can't change it afterwards, so it's worth two minutes of thought.

Cosine similarity looks only at the direction of two vectors and ignores their length. Picture two arrows: if they point roughly the same way, the score stays high even when one arrow is far longer. Scores run from 1.0 (same meaning) through 0 (unrelated) to -1.0 (opposite). It's the usual choice for text embeddings, so start here if you're unsure.

Dot product cares about direction and length. Two vectors pointing the same way score higher if they're also longer. Sparse vectors and hybrid setups need it, and some embedding models recommend it outright. The numbers can get large, so the scale isn't the same as cosine.

Euclidean distance is the straight-line distance between two points. Smaller means more similar. It's sensitive to magnitude, which is why you'll see it less often with text.

Here's a detail that saves confusion: if your vectors are normalized to length 1, cosine and dot product return the same ranking. OpenAI's embeddings come normalized, which is why either metric works with them.

Heads up: the metric is fixed at index creation. Read your embedding model's docs before you hit create. Most text models are happiest with cosine.

How Do an Index, a Namespace and Metadata Fit Together?

An index is the container, a namespace is a partition inside it, and metadata is the label on each record. In Pinecone, a single record can carry up to 40 KB of metadata alongside its vector, which is where the original chunk text usually lives.

What is an index?

The index is the main container for your vectors, a bit like a database or a table. Creating one means choosing a name, a dimension and a metric. On Pinecone serverless, you also pick a cloud and a region.

What is a namespace?

A namespace is a partition inside one index, closer to a folder. Each query searches a single namespace.

Index: "rag-demo" (dimension=384, metric=cosine)
  |-- Namespace: "hr-policies"  -> leave policy, salary policy
  |-- Namespace: "tech-docs"    -> API guide, setup guide
  |-- Namespace: "customer-A"   -> handy for multi-tenant apps

If it helps, think of the index as an apartment building and each namespace as one flat. Same building, so the same dimension and metric. Everyone's stuff stays behind their own door.

Why bother with namespaces? Isolation, mainly. Each customer only searches their own data, which matters for any multi-tenant AI chatbot or for SaaS products where one tenant must never see another's records. Searches also get faster because there are fewer vectors to scan. And cleanup is painless: delete one namespace and the rest stay untouched.

What is metadata, and why do you need it?

Metadata is extra information stored beside a vector: the original text, the source file, a page number or a category.

You need it for two reasons. First, a vector can't be turned back into text. If you want to send the original chunk to the LLM, you have to store that text yourself. Second, metadata lets you filter, for example "only documents from hr-policy.pdf." That filtering is also how teams enforce access rules, a core part of AI governance. The vector finds the record; the metadata tells you what the record is.

{
  "id": "hr-policy-p3-c2",
  "values": [0.21, -0.53, ...],
  "metadata": {
    "text": "Annual leave: 24 days per year",
    "section": "Leave Policy",
    "source": "hr-policy.pdf",
    "page": 3
  }
}

How Does the Whole Vector Database Pipeline Work?

Every RAG query runs the same eight steps, and the embedding model appears twice: once when you load documents and again for each question. Using the same model both times is non-negotiable, since vectors from two different models can't be compared.

Your documents are first split into chunks. Each chunk goes through an embedding model and becomes a vector. That vector is saved in an index, inside a namespace, with its metadata. When someone asks a question, the same model turns the question into a vector too. The index uses its metric to find the closest matches, and the LLM receives the original text of those matches and writes the answer.

The short version:

• Chunk: a piece of a document
• Embedding: meaning as numbers
• Dimension: how many numbers (match your model)
• Dense: meaning; sparse: keywords
• Metric: how closeness gets scored
• Namespace: a folder inside an index

Conclusion

When I first heard "vector database," I assumed only ML engineers needed to care. That isn't the case. The idea is small: turn text into numbers, find the numbers that sit closest together, and hand the matching text to an LLM.

If one thing sticks, make it this flow:

Text → Chunks → Embedding → Vector → Search → Relevant context → LLM → Answer

Dimensions, metrics and namespaces are settings you tune. But before any text can be embedded, long documents have to be split into the right-sized pieces. That step is called chunking, and it moves retrieval quality more than most people expect. Read our guide to chunking strategies for RAG next.