Top 15 RAG Interview Questions, Answered by Building One

Key takeaway: Most RAG interview questions are about the pipeline before the model. Of the 15 below, 10 sit in ingestion, chunking, indexing, or retrieval. A list of definitions will not get you through them. A small doc Q&A app that you have broken on purpose will. The decision this post helps you make: what to build this weekend, and which failures to cause on purpose so you have a real answer when the interviewer asks.

RAG interview questions look like trivia ("what is chunking?") and get scored like a design review ("why that chunk size, and how did you know?"). The follow-ups keep going until it is clear whether you shipped something. The quickest way to have real answers is to build one doc Q&A app, break it at each stage, and keep notes.

This post groups 15 questions worth preparing by pipeline stage. Each one gets an answer skeleton you can fill with your own numbers.

Previous post: AI Engineer Interview Guide 2026: Build These 5 Apps.

What this post covers

• What RAG is, where it came from, and why interviewers care
• The app to build and the five failures to cause on purpose
• The 15 questions, by pipeline stage, with answer skeletons
• A worked example: one stale-answer question, answered well
• How to demo it in five minutes

RAG is a retrieval step bolted to a generator, and interviews test the bolt

What it literally is. Retrieval-augmented generation fetches passages from an external store and puts them in the model's prompt before it answers. Lewis et al. (2020) coined the term. Their model paired a dense retriever over Wikipedia with a sequence-to-sequence generator, so knowledge could be updated by swapping the index, not retraining the weights.

Where it sits. Today's version is a pipeline you own: parse documents, split them into chunks, embed and index them, retrieve and rerank, assemble context, generate, cite. The model is usually rented through an API. Everything else is yours.

Technical explanation: every stage before the model call decides what the model is allowed to know. If the right passage is not in the context window, no prompt can recover it.

Business explanation: picture an HR policy assistant. The leave policy changed on Monday. If the index still holds last month's chunk, the assistant tells an employee the wrong number of leave days, cites the document to back it up, and sounds certain.

Build one doc Q&A app, then break it five ways

Scope (one weekend): about 200 real documents (PDFs with tables, slide decks, a wiki export). Hybrid search, a reranker, answers with chunk-level citations, and a 50-question golden set with known source chunks.

Break it on purpose and write down what happened:

1. Ask a question the corpus cannot answer.
2. Edit a source document and ask again.
3. Ask about a number buried in a table.
4. Run the same query twice and diff the retrieved chunk IDs.
5. Add a second user and try to retrieve their documents.

Each break gives you a real story to tell for at least one of the questions below.

The 15 questions, grouped by where the pipeline breaks

Ingest and parse

1. How do you handle tables and scanned PDFs? Skeleton: which parser, one table it mangled, how you checked the fix.
2. A document is updated. How does the old answer disappear? Skeleton: document IDs, a content hash, delete-then-reindex, and how long the stale window was.
3. What happens when the answer is not in the corpus? Skeleton: a "not found" path, plus the threshold that triggers it. Barnett et al. list this as failure point one, "missing content" (2024).

Chunk

4. How did you pick the chunk size? Skeleton: two or three sizes tried, recall@k on your golden set for each, the one you kept.
5. Fixed, recursive, or structure-aware chunking? Skeleton: what broke with fixed windows (split tables, orphan headings) and what structure-aware chunking fixed. Go deeper: RAG Architecture Tradeoffs in Plain English.

Embed and index

6. How do you choose an embedding model? Skeleton: recall on your own queries, not a public leaderboard, plus cost per million tokens.
7. How do you change embedding models without downtime? Skeleton: build a second index, dual-read, compare recall, cut over.

Retrieve and rank

8. Why hybrid search? Skeleton: BM25 catches exact terms (error codes, SKUs), dense vectors catch paraphrase, fused with reciprocal rank fusion (Cormack et al., 2009). Anthropic reports failed retrievals dropping 49% with contextual embeddings plus contextual BM25, versus 35% with contextual embeddings alone (Contextual Retrieval).
9. What does a reranker add? Skeleton: a cross-encoder rescoring the top 50 to 100 candidates. The same write-up reports 67% fewer failed retrievals with reranking on top.
10. How do you stop user A from retrieving user B's documents? Skeleton: filter by tenant and permission inside the retrieval query, never after generation. Go deeper: Agent Memory Without Risk: A Plain-English Guide.

Generate and cite

11. Why not just stuff more chunks into a long context? Skeleton: cost, latency, and Liu et al. (2023), who found models use information at the start and end of a long context better than in the middle.
12. A customer disputes an answer. Show me where it came from. Skeleton: chunk-level citations, plus a log of which chunks were retrieved for that request.

Evaluate and operate

13. How do you know a change helped? Skeleton: score retrieval (recall@k) and generation (faithfulness) separately on a fixed golden set. Ragas defines faithfulness as whether the answer's claims are supported by the retrieved context (Es et al., 2023). Go deeper: RAG Evaluation: Prove a Retrieval Change Actually Helped.
14. The same question returned different chunks yesterday. Why? Skeleton: index rebuilt mid-day, ties in approximate nearest-neighbour search, or a reranker change. Fix: version the index and log the retrieved chunk IDs.
15. Build or buy the retrieval layer? Skeleton: build it once to learn the failures. At work, price the ongoing parsing, re-indexing, and permission work against managed options (LlamaCloud, Vectara, Ragie, Pinecone Assistant, or a context engine like Inherent).

Worked example: Aarav and the stale leave policy

Fictional scenario. Aarav, a backend engineer at a large IT services company, demos his HR policy bot in an AI engineer interview. The interviewer edits the leave policy PDF from 18 to 21 days, re-uploads it, and asks the bot. It answers 18, with a citation.

"Why?" she asks.

The weak answer: "Maybe the prompt needs tuning."

Aarav's answer: "The upload added new chunks but never deleted the old ones, because I keyed chunks by position, not by document ID. Both versions are in the index and the old one ranked higher. I'd key every chunk by document ID plus a content hash, delete by document ID on update, and add this exact edit to my golden set so it can't regress."

Things to note: he names the stage (ingestion), the mechanism (no delete path), the fix, and the test. He never touches the prompt. Go deeper: Batch Ingestion Is Why Your RAG Answers Go Stale.

How to demo it in five minutes

1. Ask a question and show the cited chunk (one minute).
2. Edit the source, re-ingest, ask again, and show the new answer (two minutes).
3. Ask something out of corpus and show the "not found" path (one minute).
4. Run the golden set and show recall@k and faithfulness did not drop (one minute).

TLDR

• 10 of these 15 RAG interview questions sit before the model call.
• Build one doc Q&A app on messy documents, then break it five ways on purpose.
• Every answer skeleton follows one pattern: stage, mechanism, fix, test.
• Score retrieval and generation separately, or you cannot say which one moved.
• Build retrieval yourself once; know which parts teams buy as managed infrastructure.

Where to start

Build the app this weekend and run break number two (edit a document, ask again) first. It exercises ingestion, indexing, and retrieval in one test, and it makes the best five-minute demo.

This is Day 2 of the 30-day series on interview prep and build guides for AI app builders. Follow Flow on X for the rest. Already fighting stale chunks at work? Inherent's context engine handles managed ingestion, deterministic retrieval, and a receipt for every answer. DM for a design-partner slot.

Next read: RAG Ingestion Pipeline: A Field Guide for FDEs