Week 8 Building and Evaluating AI Systems
Retrieval, Grounding, and Provenance
Retrieval-augmented generation (RAG) connects an answer model to an external collection. Its value is not that retrieval makes an answer true, but that the system can expose what it searched, what it selected, what each claim depends on, and why it abstained.
Core question: How can a model answer from an inspectable source collection?
By the end
- Trace ingestion, chunking, indexing, retrieval, context assembly, answer generation, citation, and verification.
- Compare sparse, dense, and hybrid retrieval without confusing similarity with support.
- Compute retrieval metrics and audit claim-level grounding, currency, conflict, and abstention.
- Design provenance, security, access, and update boundaries for a small grounded system.
System map
RAG is an evidence pipeline, not a truth switch
A language model stores general patterns in parameters. RAG supplies selected external passages at answer time, which can improve currency, domain coverage, and inspectability without retraining the base model.
sourceschunksindexretrievecontextanswerverify
Collection contract
Approval, license, privacy, access, version, effective date, status, and non-use determine what may enter the index.
Retrieval contract
Query, filters, representation, ranking, top-\(k\), and score threshold determine which evidence is offered.
Answer contract
Claim support, citation, conflict handling, uncertainty, and abstention determine what the system may say.
Retrieval does not guarantee truth
The collection may be incomplete or poisoned; the retriever may rank an irrelevant passage; context assembly may omit a qualification; the generator may distort the context; and a real citation may still fail to support its nearby claim.
Collection design
Grounding begins with an inventory
Before embedding anything, define the evidence universe. A source inventory makes inclusion decisions, ownership, update cycles, and deletion requirements visible.
| Field | Why it matters | Example |
|---|---|---|
| source_id and URI | Stable identity for citations and deletion | loan-policy-2026 |
| status and effective dates | Separate current, future, and archived rules | current from 2026-04-01 |
| owner and authority | Resolve conflicts and update responsibility | Student Services |
| license and access | Control lawful ingestion and disclosure | public / internal / restricted |
| checksum and indexed_at | Reproduce exactly what the system saw | content hash plus timestamp |
Ingestion is not neutral
Parsing can drop headings, footnotes, tables, or reading order. Optical character recognition can corrupt names and numbers. Deduplication can remove useful variants or retain near-duplicates that dominate ranking. Sample and visually inspect parsed content before indexing.
Access control must survive retrieval
If users have different permissions, the index and query filters must enforce document-level or chunk-level access. A model prompt is not an authorization mechanism.
Ingestion
Chunking changes what can be found
A chunk should be small enough to rank precisely and large enough to preserve the conditions around a claim. Boundaries are a model-design choice because they change the retriever's candidate space.
\[\operatorname{step}=\operatorname{chunk\ size}-\operatorname{overlap}\]
Fixed windows are transparent but can split a heading from its rule or a rule from its exception. Structure-aware chunking preserves sections, lists, and tables but depends on reliable parsing. Overlap can protect boundary context, yet excessive overlap expands the index and creates redundant results.
Chunk-quality questions
- Can each chunk be interpreted without hidden context?
- Are title, document identity, section path, date, and status retained as metadata?
- Are tables and their headers kept together?
- Can one source be updated or deleted without rebuilding unrelated content?
Chunk boundary explorer
Change the boundary.
Inspect overlapping word chunks
text = "The current laptop loan is seven days. Renewals require staff approval. Emergency support is outside this service."
words = text.split()
chunk_size, overlap = 8, 2
step = chunk_size - overlap
for start in range(0, len(words), step):
chunk = words[start:start + chunk_size]
if chunk:
print(start, "::", " ".join(chunk))output
Click Run to see the results.
Evidence selection
Represent, rank, filter, then inspect
Retrieval maps a query and each chunk into comparable representations. A ranking score orders candidates; metadata filters and thresholds enforce additional boundaries.
Sparse retrieval
TF–IDF or BM25 emphasizes exact terms. It is efficient and inspectable, but vocabulary mismatch can miss paraphrases.
Dense retrieval
Embeddings can connect semantically related phrasing, but similarity is harder to explain and can retrieve a fluent non-answer.
Hybrid retrieval
Combine sparse and dense candidates, then fuse or rerank them. More components require component-specific evaluation.
\[\operatorname{tfidf}(t,d)=\operatorname{tf}(t,d)\log\!\left(\frac{N}{\operatorname{df}(t)}\right),\qquad \operatorname{cos}(q,d)=\frac{q\cdot d}{\lVert q\rVert_2\lVert d\rVert_2}\]
TF–IDF gives more weight to terms frequent in one document but rare across the collection. Cosine similarity compares direction rather than raw vector length. Neither metric establishes factual entailment.
Grounded-answer explorer
Choose a question.
Top evidence candidates
Run a transparent TF–IDF retriever
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
documents = [
"The current laptop loan is seven days.",
"Language exchange meets on Friday afternoons.",
"Emergency support is outside this service."
]
query = "How long can I borrow a laptop?"
matrix = TfidfVectorizer().fit_transform(documents + [query])
scores = cosine_similarity(matrix[-1], matrix[:-1]).ravel()
for rank in scores.argsort()[::-1]:
print(round(scores[rank], 3), "::", documents[rank])output
Click Run to see the results.
Context and answer
Grounding requires claim-level support
Context assembly selects and orders passages within a token budget. The answer instruction should distinguish evidence from instructions, require citations, preserve qualifications, surface conflict, and permit abstention.
Supported
Every consequential claim is entailed by eligible cited evidence, including dates, scope, and exceptions.
Faithfully cited
The citation identifies a source actually retrieved, and that source supports the nearby claim rather than merely sharing a topic.
Bounded
When evidence is absent, conflicting, stale, below threshold, or outside scope, the response says so and names a safe next step.
Provenance record
For each answer, preserve the query, user/access context, collection and index version, retrieval settings, retrieved source and chunk identifiers with scores, exact context, answer, citations, model version, and verification result. Exclude or protect sensitive content according to the data policy.
Apply an explicit support and abstention rule
question = "What is the lost-charger fee?"
candidates = [
{"source": "loan-policy", "score": 0.19,
"text": "The current laptop loan is seven days."}
]
threshold = 0.28
if not candidates or candidates[0]["score"] < threshold:
print("ABSTAIN: insufficient evidence in the approved collection")
else:
top = candidates[0]
print(top["text"], "[source:", top["source"], "]")output
Click Run to see the results.
Verify
Evaluate components before the end-to-end impression
A realistic test set contains answerable, unanswerable, stale, conflicting, paraphrased, multilingual, adversarial, and access-restricted queries. Expected evidence should be labeled independently of the system output.
| Layer | Question | Example evidence |
|---|---|---|
| Collection | Is needed, allowed, current evidence present? | coverage, provenance, version/status audit |
| Retrieval | Did expected evidence appear and rank highly? | Recall@(k), MRR, filter failures, hard negatives |
| Context | Were qualifications and conflicts preserved? | context recall, redundancy, token-budget loss |
| Answer | Are claims supported and citations faithful? | claim support, citation precision, contradiction |
| Abstention | Does the system refuse unsupported questions? | coverage–risk curve, answerable/unanswerable set |
| Workflow | Does the complete system help safely? | human outcomes, latency, privacy, access, incidents |
\[\operatorname{Recall@}k=\frac{\#\{\text{answerable queries with expected evidence in top }k\}}{\#\{\text{answerable queries}\}}\]
\[\operatorname{MRR}=\frac{1}{|Q|}\sum_{q\in Q}\frac{1}{\operatorname{rank}_q},\qquad \operatorname{Citation\ precision}=\frac{\text{supporting citations}}{\text{citations checked}}\]
High Recall@(k) only shows that evidence was available to later stages. It does not show the answer used it correctly. Report uncertainty and failure examples, and use human review for nuanced entailment judgments.
Failure and security
Design hostile and inconvenient evidence into the tests
A system tested only on clean, answerable demonstrations will conceal its most consequential boundaries.
Missing
The collection has no lost-charger fee. Correct behavior is abstention, not a plausible number.
Stale
An archived policy says three days; current policy says seven. Status filtering is part of correctness.
Conflicting
Two eligible sources disagree. Surface the conflict and authority rather than silently merging them.
Irrelevant
Keyword or semantic similarity ranks a non-answer. Retrieval score is not entailment.
Injected
A retrieved document contains instructions to ignore rules or reveal data. Treat retrieved text as untrusted data.
Unsupported citation
A real source is attached to a claim it does not establish. Verify claim-to-passage support.
Security is architectural
Separate system instructions from retrieved text, sanitize and label untrusted content, enforce authorization before retrieval, minimize tool permissions, validate structured outputs, isolate side effects, log decisions, and require confirmation before consequential action. RAG and fine-tuning do not remove prompt injection.
Lifecycle
A grounded system needs update and deletion paths
Grounding is only as current as its collection and index. Define ownership and service levels before the first release.
Update
Detect changed sources, parse and validate them, build a versioned index, run regression tests, then switch traffic reversibly.
Delete
Remove revoked or expired material from source storage, chunks, indexes, caches, and derived artifacts; record completion.
Monitor
Track no-result rate, score and source distributions, stale citations, unsupported answers, latency, cost, and incidents.
Keep a small regression suite with known evidence and expected abstentions. Re-run it after collection, parser, embedding, ranking, prompt, model, or threshold changes.
Self-check
Try before revealing the answer
1. What does retrieval add that a language model's parameters do not provide?
It supplies selected external evidence at answer time and can expose source identity, version, and passage-level provenance. It does not itself guarantee correctness.
2. Why can a larger chunk reduce retrieval precision?
It mixes more topics and irrelevant words into one candidate, so a match may be less specific. A chunk that is too small can lose qualifications, so the boundary must be evaluated.
3. What is one key difference between sparse and dense retrieval?
Sparse retrieval emphasizes explicit token overlap and is easy to inspect; dense retrieval can match semantic paraphrases but makes the basis of similarity less transparent.
4. Recall@3 is high, but answers remain unsupported. Where should you look?
Inspect context assembly, claim generation, citation alignment, conflicts, and answer verification. Retrieval success is necessary for those cases but not sufficient.
5. The correct policy ranks second behind an archived version. Is generation the main problem?
No. Collection metadata, status filtering, or ranking failed before generation. Fix and test that evidence path first.
6. Why is a citation not automatically evidence of grounding?
The source may be real but irrelevant, stale, or contradictory, or it may not support the nearby claim. Grounding requires claim-to-passage support.
7. The system abstains on every question and never hallucinates. Is it successful?
No. Report coverage as well as risk. A useful bounded system answers supported questions while abstaining appropriately on unsupported ones.
8. Why must access control be enforced before or during retrieval?
Once restricted content reaches model context it may be disclosed or influence the answer. Prompt instructions are not a dependable authorization boundary.
9. How should retrieved instructions be treated?
As untrusted data, not system authority. Isolate them from instructions, restrict tools and side effects, validate outputs, and require confirmation where consequences matter.
10. What should trigger regression testing of a grounded system?
Changes to sources, parsing, chunking, embeddings, indexes, ranking, filters, prompts, answer models, or thresholds can alter behavior and should trigger relevant tests.
Continue in Python
Build and evaluate a small grounded system
The notebook constructs a CPU-friendly sparse retriever, evaluates known questions, produces extractive grounded answers, tests currency and abstention, and exports a system card.
Essential path
Use the approved fictional collection. Separate retrieval evaluation from answer support, citation accuracy, and abstention.
Evidence artifact: retrieval table, answer audit, failure log, and compact system card.
Further exploration
Compare a dense embedding retriever if the optional model is already cached. A network download or paid API is never required for the essential path.
Glossary
Key terminology
- Retrieval-augmented generation
- An answer system that conditions generation on externally retrieved evidence.
- Corpus / collection
- The governed set of sources eligible for retrieval.
- Chunk
- A retrievable passage produced from a source document.
- Embedding
- A numerical vector representation used to compare meaning or features.
- Sparse retrieval
- Retrieval based mainly on weighted token occurrence, such as TF–IDF or BM25.
- Dense retrieval
- Retrieval using learned, mostly nonzero vector representations.
- Reranker
- A model or rule that reorders an initial candidate set.
- Grounding
- Constraining claims to evidence that supports them.
- Provenance
- A record of source identity, version, transformations, and use in an output.
- Recall@k
- The fraction of answerable queries whose expected evidence appears in the first k results.
- Abstention
- A designed refusal to answer when evidence or authority is insufficient.
- Prompt injection
- Untrusted content attempting to alter system instructions or induce unauthorized behavior.
References
Key references
- Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.
- scikit-learn User Guide: Text feature extraction.
- scikit-learn API: Cosine similarity.
- OWASP GenAI Security Project: Prompt Injection.
- OWASP Top 10 for LLM Applications 2025.
- Harbor City Student Services is an original fictional teaching collection; all organizations, policies, dates, contacts, and services in it are fictional.