Week 6 Building and Evaluating AI Systems
Representations, Embeddings, and Attention
Embeddings turn objects into vectors so that geometry can support retrieval, clustering, classification, and generation. Attention then constructs context-sensitive mixtures—but neither proximity nor attention weight is automatically an explanation.
Core question: How can text, images, and other objects become useful vectors?
By the end
- Compare designed features with learned representations.
- Compute and audit cosine similarity and nearest neighbors.
- Trace scaled dot-product self-attention through queries, keys, and values.
- Identify misleading neighborhoods, projection artifacts, and attention overclaiming.
Course progression
Build one evidence chain
This week uses the course-wide sequence: frame the use, establish a baseline, build or compare, evaluate failure, document boundaries, and decide.
framebaselinebuildevaluatedocumentdecidemonitor
Connection from the course
Week 7 connects transformer representations to pretraining objectives and evidence-based adaptation choices.
Essential concepts
Understand the parts before combining them
The core lane focuses on transferable judgment. Optional formal or engineering depth belongs in the companion notebook's stretch lane.
Embedding geometry
Distances reflect the training objective and data, not universal meaning. Useful neighbors in one task can be harmful or irrelevant in another.
Similarity audit
Inspect nearest neighbors, positive and negative pairs, slices, hubness, and stability rather than trusting a two-dimensional projection.
Self-attention
Queries ask what is relevant, keys advertise matchable features, and values carry information into a weighted mixture.
Mathematical intuition
One relationship worth keeping
\[\operatorname{Attention}(Q,K,V)=\operatorname{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V\]
Scaling prevents dot products from becoming so large that softmax collapses too early. Masks can block future or padded positions; multiple heads learn different projections.
Stretch: what the notation leaves out
The weights show how this layer combined value vectors for a given input; they do not by themselves establish causal importance or a faithful human explanation.
Interactive explorer
Change assumptions and inspect the decision
Change the attention temperature and query position. Observe how sharply the fixed keys are weighted.
Current signal—
Recommended response—
Lower effective temperature makes weights sharper; higher temperature spreads mass. A changed query changes compatibility even when the keys remain fixed.
Evidence workflow
Move from a claim to a decision
Use this order in the chapter, notebook, and project record. Skipping an earlier step weakens every later claim.
- 1. State the representation objective and source data.
- 2. Normalize or select a distance measure appropriate to the task.
- 3. Inspect neighborhoods with known relevant and irrelevant pairs.
- 4. Trace Q, K, V, masking, softmax weights, and the output mixture.
- 5. Evaluate downstream usefulness and harms; do not substitute a projection for evidence.
Weekly evidence artifact
Embedding-neighborhood audit and attention-block trace
Project connection
No new PBL deliverable; use the neighborhood audit as transferable evaluation practice.
Failure analysis
Deliberately look for the claim's boundary
A failure case is useful when the setup, expected behavior, observation, severity, and response are recorded.
Semantic overclaim
Failure: Two items are close, so the system claims they mean the same thing.
Evidence: A task-specific label or human review contradicts the neighborhood.
Response: Treat similarity as candidate evidence, not identity or entailment.
Projection illusion
Failure: A 2D plot appears to show clean clusters.
Evidence: Distances and neighbors change under another seed or projection.
Response: Audit the original space and report projection settings.
Attention as explanation
Failure: A high attention weight is presented as the reason for a decision.
Evidence: Perturbation or alternative attribution does not support the claim.
Response: Describe the computation narrowly and test explanations separately.
Use and non-use
Keep authority proportional to evidence
Intended use
Use embeddings for candidate organization and attention for learned contextual mixing.
Do not use
Do not treat vector proximity or attention weights as truth, causality, fairness, or authorization.
Human responsibility
A human defines acceptable similarity, inspects slices, and approves high-stakes uses.
Small Python demonstrations
Predict, run, and interpret
Each button calls one fixed, allowlisted computation. Use the notebook for longer experiments and saved evidence.
Compute cosine similarity
import math
a, b = [1, 2, 0], [2, 1, 1]
dot = sum(x*y for x,y in zip(a,b))
cosine = dot / math.sqrt(sum(x*x for x in a)*sum(y*y for y in b))
print(round(cosine, 3))output
Run this fixed example to compare your prediction with the result.
Calculate attention weights
import math
scores = [1.2, 0.3, -0.4]
exps = [math.exp(s) for s in scores]
weights = [v/sum(exps) for v in exps]
print([round(w, 3) for w in weights])output
Run this fixed example to compare your prediction with the result.
Audit nearest neighbors
candidates = [(0.91, "policy summary"), (0.88, "old policy"), (0.42, "sports notice")]
for score, label in sorted(candidates, reverse=True)[:2]:
print(score, label)output
Run this fixed example to compare your prediction with the result.
Check your understanding
Ten questions with standard answers
Answer before opening each panel. A good answer connects the concept to evidence, failure, and a bounded decision.
1. How can text, images, and other objects become useful vectors?
Standard answer: Vectors are useful when their learned geometry supports a defined task under audited neighborhoods; attention constructs context by learned compatibility but requires independent evaluation.
2. What is the role of embedding geometry in this chapter?
Standard answer: Distances reflect the training objective and data, not universal meaning. Useful neighbors in one task can be harmful or irrelevant in another.
3. Why does similarity audit require evidence rather than intuition?
Standard answer: Inspect nearest neighbors, positive and negative pairs, slices, hubness, and stability rather than trusting a two-dimensional projection.
4. How should a practitioner use self-attention?
Standard answer: Queries ask what is relevant, keys advertise matchable features, and values carry information into a weighted mixture.
5. What does the chapter's main formula clarify—and what does it not prove?
Standard answer: The weights show how this layer combined value vectors for a given input; they do not by themselves establish causal importance or a faithful human explanation.
6. What should change in the explorer as its risk or complexity controls increase?
Standard answer: Lower effective temperature makes weights sharper; higher temperature spreads mass. A changed query changes compatibility even when the keys remain fixed.
7. How should the system respond to: Semantic overclaim?
Standard answer: Treat similarity as candidate evidence, not identity or entailment.
8. What evidence reveals the failure called Projection illusion?
Standard answer: Distances and neighbors change under another seed or projection.
9. When should the system not be used or allowed to proceed?
Standard answer: Do not treat vector proximity or attention weights as truth, causality, fairness, or authorization.
10. How does this week prepare the next stage of the course?
Standard answer: Week 7 connects transformer representations to pretraining objectives and evidence-based adaptation choices.
Terminology
Glossary
- Representation
- A set of features used by a model or comparison.
- Embedding
- A learned vector representation of an object.
- Cosine similarity
- Normalized dot-product similarity based on vector angle.
- Nearest neighbor
- An item with small distance or high similarity under a chosen metric.
- Hubness
- A high-dimensional effect where some points become neighbors of many others.
- Projection
- A mapping from high dimensions to fewer dimensions for inspection.
- Query
- A vector expressing what information is being sought in attention.
- Key
- A vector used to calculate compatibility with a query.
- Value
- The information vector mixed according to attention weights.
- Attention mask
- A constraint preventing selected positions from influencing an attention output.
Continue learning
Key references
These primary papers, standards, or official technical documents anchor the chapter. Product names and current legal timelines should be rechecked when used in a real project.