You've probably seen a demo like this: someone connects a chatbot to their company's internal documents, then asks it "what's our refund policy for enterprise customers?" — and it answers correctly, quoting the actual policy. If you stop and think about it, that should be impossible. The AI was trained months ago on public internet text. It has never seen your company's refund policy. Nobody retrained it on your files. So how does it know?

The answer is the single most useful thing to understand about how modern AI apps are built, and it's much simpler than it looks. The AI doesn't know your policy. It looks it up — a fraction of a second before answering — and vector search is the part that does the looking.

Closed-book exam vs. open-book exam

Think about two kinds of test.

A closed-book exam tests what you memorized. You walk in with only what's in your head. If the question is about something you never studied, you're stuck — or worse, you make up a confident-sounding answer and hope. That's a plain chatbot answering from training alone: it only knows what it happened to read while being built, and when it doesn't know, it has an unfortunate habit of inventing something plausible. (You may have heard this called "hallucinating.")

An open-book exam is a different game. You can bring the textbook. You don't need to have memorized the refund policy — you need to know how to flip to the right page fast, read it, and answer from what it says. Suddenly the test isn't about memory at all. It's about lookup.

Connecting an AI to your documents turns a closed-book exam into an open-book one. And the "flip to the right page" step — finding the handful of pages relevant to the question, out of thousands — is exactly what vector search is good at. That whole approach even has a name you'll see everywhere: RAG, which stands for retrieval-augmented generation. Retrieval is just a formal word for "look it up first."

Closed-book versus open-book. On the left, a closed-book AI answers the question "What does the policy say?" from memory alone — and may confidently make things up. On the right, an open-book AI is handed the relevant document first and answers from the page in front of it. Connecting your documents turns a memory test into a lookup.

The three steps, in plain words

When you ask that question about the refund policy, three things happen in quick succession.

One: find the relevant pages. Your question gets turned into a point on the map — the same embedding trick used for everything else — and the system searches your documents for the passages that sit nearest to it in meaning. Not passages that contain your exact words; passages that are about what you asked. Your documents were embedded and stored the same way ahead of time, so this search is fast even across a huge pile of files. Out comes a short stack: the few paragraphs most likely to hold the answer.

Two: hand those pages to the AI. The system takes that short stack of relevant passages and gives them to the AI along with your question — essentially saying, "Here's the question, and here are the specific pages from their documents that seem relevant. Answer using these."

Three: answer from the pages. Now the AI does what it's genuinely great at — reading passages and writing a clear, direct answer — but grounded in your text instead of its memory. It's not recalling your refund policy from training. It's reading the policy you just handed it and summarizing it back.

That's the entire trick. Find the right pages, hand them over, answer from them.

How the answer actually gets made — retrieval-augmented generation in three plain steps. Step one: find the few relevant pages (a magnifier over a stack of documents, one highlighted). Step two: hand them to the AI. Step three: answer from those pages (a clean answer). Find the right pages, hand them over, answer from them — nothing memorized.

Why this fixes the "makes stuff up" problem

The made-up-answer problem comes from a closed-book AI reaching for something it doesn't actually have. The open-book setup attacks that at the root: instead of hoping the answer is somewhere in the model's memory, you put the answer in front of it right before it responds. Ask about something that genuinely isn't in your documents, and a well-built system can now say "I couldn't find anything about that" — because it looked, and the lookup came back empty. That's a far more trustworthy failure than a confident invention.

It also means the AI can answer about things that didn't exist when it was trained — a policy you wrote last week, a product you shipped yesterday, a document from this morning. The model never has to learn any of it. It just has to be handed the right pages, and vector search is what finds them.

Why vector search specifically

You might wonder why this needs anything fancy — why not just keyword-search the documents for the words in the question? Because the words in a question rarely match the words in the answer. Someone asks "can I get my money back?" and the relevant document is titled "Refund Eligibility." No shared keywords, same meaning. Keyword search flips to the wrong page, or no page at all. Vector search flips to the right one, because it's matching on meaning rather than exact wording — which is the whole reason it exists. (For where keyword search still wins, see when to use vector search.)

Closing

So the impossible-seeming demo is really just a well-run open-book exam. The AI supplies the language skills; your documents supply the facts; and vector search is the quiet step in the middle that flips to the right page in time. Nothing was memorized, nothing was retrained — the answer was looked up, freshly, the moment you asked.

If you want to see the lookup half working on its own, the projects walk through building a searchable collection end to end — the same "find the relevant items by meaning" step that sits at the heart of every one of these assistants.