Week 4 Building and Evaluating AI Systems
Specialized Neural Architectures in Context
CNNs and recurrent networks are not museum pieces. They encode locality, parameter sharing, order, and state—assumptions that can still make compact systems data-efficient, fast, and appropriate.
Core question: What inductive bias does an architecture add, and when is it still useful?
By the end
- Explain locality, shared filters, pooling, recurrence, and state.
- Compare training from scratch, transfer learning, recurrence, and attention.
- Use class, group, and temporal evidence to audit specialized models.
- Defend an architecture choice against a simpler baseline.
Course progression
Build one evidence chain
This week uses the course-wide sequence: frame the use, establish a baseline, build or compare, evaluate failure, document boundaries, and decide.
framebaselinebuildevaluatedocumentdecidemonitor
Connection from the course
Week 5 supplies the evaluation harness required to decide whether any architectural gain is trustworthy and useful.
Essential concepts
Understand the parts before combining them
The core lane focuses on transferable judgment. Optional formal or engineering depth belongs in the companion notebook's stretch lane.
Convolutional bias
A filter examines local neighborhoods and shares its weights across positions. This reduces parameters and encodes translation-related assumptions.
Recurrent bias
A recurrent cell updates state sequentially. It naturally represents order and streaming but can struggle with long dependencies and parallelism.
Transfer learning
A pretrained representation can reduce data and compute needs, but the source–target mismatch and frozen-feature failures must be tested.
Mathematical intuition
One relationship worth keeping
\[h_t=\phi\!\left(W_xx_t+W_hh_{t-1}+b\right)\]
The recurrent state compresses the past. Gated cells add learned paths for retaining, forgetting, and exposing information; attention instead relates positions directly.
Stretch: what the notation leaves out
A compact state can lose distant information, and repeated multiplication through time can shrink or amplify gradients.
Interactive explorer
Change assumptions and inspect the decision
Change locality, sequence length, and compute budget. Which inductive bias is most defensible?
Current signal—
Recommended response—
Strong locality favors a CNN; short streaming sequences can favor recurrence; long-range parallel relationships favor attention when compute and data support it.
Evidence workflow
Move from a claim to a decision
Use this order in the chapter, notebook, and project record. Skipping an earlier step weakens every later claim.
- 1. Start with a linear, rule-based, or shallow baseline.
- 2. Name the structure the architecture assumes: locality, order, invariance, or long-range relation.
- 3. Compare parameter count, data need, latency, and held-out behavior.
- 4. Audit imbalance, temporal leakage, and failures under source–target shift.
- 5. Choose the smallest model whose bias is supported by evidence.
Weekly evidence artifact
Architecture-selection memo supported by validation evidence
Project connection
Review the PBL1 architecture and comparison plan; require a baseline and a split rationale before experimentation expands.
Failure analysis
Deliberately look for the claim's boundary
A failure case is useful when the setup, expected behavior, observation, severity, and response are recorded.
Architecture zoo reasoning
Failure: A model is chosen because it is newer or deeper.
Evidence: No controlled baseline or task-specific evidence supports the choice.
Response: Reduce the comparison to assumptions, resources, and held-out failures.
Invalid temporal validation
Failure: A sequence model trains on observations later than its test cases.
Evidence: Chronological replay reveals future leakage.
Response: Use rolling or forward-chaining evaluation.
Transfer mismatch
Failure: Pretrained visual features ignore a small but consequential domain cue.
Evidence: Slice errors cluster in the target domain despite strong average accuracy.
Response: Fine-tune cautiously, add target evidence, or use a specialized compact model.
Use and non-use
Keep authority proportional to evidence
Intended use
Use specialized models when their structural assumptions match the data and constraints.
Do not use
Do not select an architecture by historical prestige, parameter count, or leaderboard position alone.
Human responsibility
A human owns the architecture rationale, validation design, and error review.
Small Python demonstrations
Predict, run, and interpret
Each button calls one fixed, allowlisted computation. Use the notebook for longer experiments and saved evidence.
Apply a shared local filter
signal = [2, 1, 3, 0, 2]
kernel = [1, 0, -1]
output = [sum(a*b for a,b in zip(signal[i:i+3], kernel)) for i in range(3)]
print(output)output
Run this fixed example to compare your prediction with the result.
Update a recurrent state
import math
state = 0.0
for value in [0.4, -0.2, 0.9]:
state = math.tanh(0.8*state + value)
print(round(state, 4))output
Run this fixed example to compare your prediction with the result.
Build a temporal split
years = [2021, 2022, 2023, 2024, 2025]
print("train:", years[:-1])
print("test:", years[-1:])output
Run this fixed example to compare your prediction with the result.
Check your understanding
Ten questions with standard answers
Answer before opening each panel. A good answer connects the concept to evidence, failure, and a bounded decision.
1. What inductive bias does an architecture add, and when is it still useful?
Standard answer: An architecture is useful when its inductive bias matches the observable structure and improves a relevant baseline under valid evaluation and resource constraints.
2. What is the role of convolutional bias in this chapter?
Standard answer: A filter examines local neighborhoods and shares its weights across positions. This reduces parameters and encodes translation-related assumptions.
3. Why does recurrent bias require evidence rather than intuition?
Standard answer: A recurrent cell updates state sequentially. It naturally represents order and streaming but can struggle with long dependencies and parallelism.
4. How should a practitioner use transfer learning?
Standard answer: A pretrained representation can reduce data and compute needs, but the source–target mismatch and frozen-feature failures must be tested.
5. What does the chapter's main formula clarify—and what does it not prove?
Standard answer: A compact state can lose distant information, and repeated multiplication through time can shrink or amplify gradients.
6. What should change in the explorer as its risk or complexity controls increase?
Standard answer: Strong locality favors a CNN; short streaming sequences can favor recurrence; long-range parallel relationships favor attention when compute and data support it.
7. How should the system respond to: Architecture zoo reasoning?
Standard answer: Reduce the comparison to assumptions, resources, and held-out failures.
8. What evidence reveals the failure called Invalid temporal validation?
Standard answer: Chronological replay reveals future leakage.
9. When should the system not be used or allowed to proceed?
Standard answer: Do not select an architecture by historical prestige, parameter count, or leaderboard position alone.
10. How does this week prepare the next stage of the course?
Standard answer: Week 5 supplies the evaluation harness required to decide whether any architectural gain is trustworthy and useful.
Terminology
Glossary
- Inductive bias
- An assumption that makes some patterns easier for a model to learn.
- Convolution
- A shared local transformation applied across positions.
- Kernel
- The learned local weights used by a convolution.
- Pooling
- A spatial summary that reduces resolution and can add invariance.
- Receptive field
- The input region that can influence a representation.
- Recurrence
- Sequential state updating using the current input and previous state.
- Hidden state
- A learned summary passed across sequence steps.
- Gating
- Learned control of information retention, forgetting, and output.
- Transfer learning
- Adapting or reusing a representation learned on another task or domain.
- Forward chaining
- Temporal validation that repeatedly trains on the past and tests on the future.
Continue learning
Key references
These primary papers, standards, or official technical documents anchor the chapter. Product names and current legal timelines should be rechecked when used in a real project.