Gradient
Lesson 10.3ModelCode~1 h

Retrieval & RAG

Give a model an open-book exam: chunk documents, search them, and ground its answers in the sources it found.

Module 10 · 0/11 done

Open-book answers

A language model only knows what was in its training data. It doesn’t know your company’s documents, yesterday’s news, or anything after its training cutoff. Asked anyway, it may hallucinate: produce a fluent, confident, wrong answer.

Retrieval-augmented generation (RAG) turns the exam into an open-book one:

  1. Retrieve: find the passages most relevant to the question.
  2. Augment: paste them into the prompt, with an instruction to use them.
  3. Generate: the model answers, citing what it was given.
Fig 10.4Type any question. The pipeline scores all 14 documents against it, keeps the best 3, and builds the prompt the model would see.