AI Engineer Interview Guide 2026: Build These 5 Apps

Key takeaway: The AI engineer interview rewards what you have built and can defend. Memorized definitions score very little. Five small apps cover almost every question in the loop: a structured extractor, a document Q&A (RAG) app, an eval harness, a tool-using agent, and a production wrapper. The RAG app is the one candidates get wrong, because its worst failures happen before the model is ever called. The decision this post helps you make: which app to build first, and what to be ready to defend when the interviewer pokes at it.

Most AI engineer interview prep is a list of 50 questions to memorize. That prep ages badly. Current AI engineer job posts ask for RAG architecture, retrieval evaluation, and eval pipelines, and the loop reads like a code review of something you shipped: "you said you built a RAG app, so what happens when a document changes?" A candidate who built the thing can answer from memory of a bug. A candidate who read a list cannot.

So this guide flips the usual format. Instead of questions first, it gives you five apps to build, the questions each one prepares you for, and the trap in each. One app, retrieval-augmented generation over your own documents, gets the most space. It looks like 20 lines of code in a tutorial and turns out to be the hardest system in the portfolio to get right.

Previous post: Top 10 Inference Engineering Interview Questions for 2026.

What this post covers

• What an AI engineer is, and how the role differs from ML and inference engineering
• The five-app portfolio and what each app proves
• Why the RAG app is the hardest, and the seven places it breaks
• The questions and traps for the other four apps
• A four-week build plan and a five-minute demo script

An AI engineer builds products on top of models, so the loop tests product judgment

Where the role came from. In 2023, swyx's essay The Rise of the AI Engineer named a new role: engineers who build products on top of foundation models through APIs, rather than training models. Chip Huyen's O'Reilly book AI Engineering (2025) draws the same line: the work is "building applications on top of foundation models," through prompt engineering, context construction, and parameter-efficient fine-tuning, rather than feature engineering and model training.

Where it sits. Three roles now split the stack:

• ML engineer: trains and fine-tunes the model.
• Inference engineer: serves it fast and cheap (the previous post).
• AI engineer: turns it into a product that gives correct answers on a company's own data.

Technical explanation: the AI engineer owns everything between the user and the model call: what context goes in, which tools the model can use, how outputs are checked, and how quality is measured over time.

Business explanation: picture a support team adopting an AI assistant. The model is rented. What the company actually pays an AI engineer for is whether the assistant quotes the current refund policy, only shows each customer their own tickets, and can prove where an answer came from when a customer escalates.

Five apps cover the loop, and each one proves a different skill

The five apps, in build order: a structured extractor, a RAG app, an eval harness, a tool-using agent, and a production wrapper. Apps 3 to 5 all build on the RAG app, so by week four you have one system rather than five toy demos. These are the AI engineer portfolio projects worth your weekends.

App 2, the RAG app, is the hardest because it fails upstream of the model

What to build (week 2): of all the RAG project ideas, pick the unglamorous one: a chat-with-your-docs app over about 200 real, messy documents: PDFs with tables, slide decks, a wiki export. Return an answer with citations to the exact chunk.

Why it is hard. The tutorial version (split text, embed, top-k search, stuff the prompt) works on a clean demo. Real documents break it in ways the model cannot fix. Barnett et al. (2024) studied RAG systems built for research, education, and biomedical use and catalogued seven failure points. The first three happen before the model reads a single token:

1. Missing content: the answer is not in the corpus, and the system invents one.
2. Missed top-ranked documents: the right chunk exists but ranks below the cut-off.
3. Not in context: it was retrieved but dropped when the context was assembled.
4. Not extracted: it reached the model, but the model missed it among noise.
5. Wrong format: the answer ignores the requested structure.
6. Incorrect specificity: too vague or too detailed for the question.
7. Incomplete: a partial answer when the full one was in context.

Two more that the interviewer will add from production experience:

• Staleness: a policy doc changed last week; the index still holds the old chunk, so the answer is confidently out of date.
• Non-reproducibility: the same question returns different chunks on two runs, so nobody can debug the bad answer a customer saw.

RAG interview questions this prepares you for:

• How did you chunk, and what did you measure to pick the chunk size?
• Why hybrid search (BM25 plus embeddings)? What does a reranker add?
• A document is updated. Walk me through how the old answer disappears.
• How do you stop user A from retrieving user B's documents?
• A customer disputes an answer. Show me which chunk it came from.

For the hybrid search question, have a number ready. Anthropic's Contextual Retrieval write-up reports failed retrievals falling 35% with contextual embeddings, 49% with contextual BM25 added, and 67% with reranking on top.

The trap: tuning the prompt when the retrieval is wrong. If the right chunk never reached the model, no prompt fixes it. Liu et al. (2023) also showed models use information at the start and end of a long context better than information in the middle, so "just retrieve more" fails too.

What good sounds like: build it yourself once, because the interview is about the failure modes, not the vendor. Then say what you would do at work. Most teams stop hand-rolling parsing, re-indexing, and permission filters for every project. Managed retrieval services (LlamaCloud, Vectara, Ragie, Pinecone Assistant) and context engines like Inherent each take on parts of this; Inherent's slice is managed ingestion, deterministic retrieval, and a receipt for every answer. Knowing when to build and when to buy is itself a senior-level answer.

Go deeper: RAG Ingestion Pipeline: A Field Guide and Batch Ingestion Is Why Your RAG Answers Go Stale

The other four apps each test one skill the RAG app depends on

App 1, structured extractor (weekend 1). Turn invoices or resumes into validated JSON.

• Asked: how do you guarantee a schema? What happens on a malformed response?
• Trap: trusting the model's JSON without a validator and a retry path.

App 3, eval harness for the RAG app (week 3). A golden set of 50-100 questions with known source chunks. Measure recall@k for retrieval and faithfulness for answers, and fail CI when either drops.

• Asked: how do you know a change helped rather than just looked better?
• Trap: grading only final answers, which hides whether retrieval or generation moved. See RAG Evaluation: Prove a Retrieval Change Actually Helped.

App 4, tool-using agent (week 4). Give an agent three tools, one of which is your RAG app, plus a step budget.

• Asked: how does it stop? What happens when a tool fails? What does it remember between turns?
• Trap: forgetting that an agent built on flaky retrieval is flaky retrieval with more steps.

App 5, production wrapper (ongoing). Add request tracing, cost per answer, caching, and model routing with a fallback.

• Asked: a user reports a bad answer from last Tuesday. Reproduce it.
• Trap: logs that record the answer but not the retrieved context, so the trace cannot be replayed. See LLM Observability: A Trace You Can't Replay Is a Screenshot.

A four-week plan and a five-minute demo

The demo that lands in five minutes: ask a question, show the cited chunk, edit the source document, ask again, and show the answer change. Then show the eval run that proves retrieval did not regress. That sequence answers most of the loop before it is asked.

TLDR

• AI engineers build products on top of models; interviews test what you built and can defend.
• Five apps cover the loop: extractor, RAG app, eval harness, agent, production wrapper.
• The RAG app is the hardest: its most painful failures (missed chunks, stale chunks, irreproducible retrieval) happen before the model runs.
• Build RAG yourself once to learn the failure modes; know which parts teams buy as managed infrastructure.
• Close every demo with evidence: the source chunk, a document update, and an eval run.

Where to start

If you have shipped structured output before, skip App 1 and start the RAG app this week, on documents you did not write. The first stale answer you hit will teach you more than a question list.

This is Day 1 of a 30-day series on interview prep and build guides for people building AI apps. Follow Flow on X for the next 29 days. Already building RAG at work and tired of rebuilding ingestion? DM for a design-partner slot on Inherent's context engine.

Next read: Production RAG Needs Truth and Memory