Week 14 Building and Evaluating AI Systems
PBL2 Demonstration, Defense, and Synthesis
A final demonstration is not a performance. It is a compact evidence argument connecting intended use, architecture, sources, evaluation, failures, mitigations, fallback, and human responsibility.
Core question: What evidence supports this system, and what should never be delegated to it?
By the end
- Present a claim–evidence–boundary demonstration within a fixed time.
- Answer individual questions about decisions, failures, and AI assistance.
- Review peers by evidence quality rather than polish or novelty.
- State what the system must never decide or do autonomously.
Course progression
Build one evidence chain
This week uses the course-wide sequence: frame the use, establish a baseline, build or compare, evaluate failure, document boundaries, and decide.
framebaselinebuildevaluatedocumentdecidemonitor
Connection from the course
The course ends by returning to Week 1: choose the least complex system, support bounded claims, preserve evidence, and keep responsibility human.
Essential concepts
Understand the parts before combining them
The core lane focuses on transferable judgment. Optional formal or engineering depth belongs in the companion notebook's stretch lane.
Evidence argument
A claim is supported by traceable outputs and tests under stated conditions. Limitations define where the argument stops.
Individual accountability
Each student explains a substantive contribution, one design choice, one failure, one correction, and AI assistance without relying on the team spokesperson.
Non-delegation
Some decisions remain human because they require authority, care, rights, contextual judgment, or acceptance of consequences.
Mathematical intuition
One relationship worth keeping
\[\text{defensible claim}=\text{evidence}+\text{conditions}+\text{limitations}+\text{responsibility}\]
This is an argument structure rather than an arithmetic identity. Removing any term turns a bounded claim into overstatement.
Stretch: what the notation leaves out
A score without conditions and limitations cannot identify where performance transfers; a claim without responsibility cannot identify who acts when the system fails.
Interactive explorer
Change assumptions and inspect the decision
Rate evidence coverage, failure testing, reproducibility, and boundary clarity. Identify what still blocks a defended release claim.
Current signal—
Recommended response—
Averages can guide rehearsal, but a single missing release gate—such as reproducibility, fallback, or authority—can block the claim regardless of presentation quality.
Evidence workflow
Move from a claim to a decision
Use this order in the chapter, notebook, and project record. Skipping an earlier step weakens every later claim.
- 1. Open with intended use, explicit non-use, baseline, and one bounded claim.
- 2. Demonstrate one success, one failure or abstention, and one mitigation with source evidence.
- 3. Show the system card, risk register, version fingerprint, and fallback.
- 4. Sample individual defenses across contribution, failure, AI-use verification, and human boundary.
- 5. Submit the individual synthesis memo and archive or retire unsupported components.
Weekly evidence artifact
PBL2 demonstration, sampled individual defense, peer evidence review, and synthesis memo
Project connection
Complete PBL2 demonstrations and defenses, peer evidence review, individual synthesis memo, and final archive/retirement decisions.
Failure analysis
Deliberately look for the claim's boundary
A failure case is useful when the setup, expected behavior, observation, severity, and response are recorded.
Demo as proof
Failure: One successful live example is treated as validation.
Evidence: The evaluation set, slices, and failure rate are absent.
Response: Use the demo only as an illustration of the frozen evidence dossier.
Team opacity
Failure: Only one student can explain the system.
Evidence: Sampled questions reveal missing individual understanding.
Response: Require responsibility records and individual defense.
Boundary omission
Failure: The presentation ends with benefits and no non-use or escalation.
Evidence: No one is accountable for uncertain or harmful cases.
Response: Make non-delegation and named human authority part of the claim.
Use and non-use
Keep authority proportional to evidence
Intended use
Use the system only within the conditions supported by the frozen dossier and release decision.
Do not use
Never delegate rights-affecting authority, irreversible actions, or responsibility for consequences to the model.
Human responsibility
Named people remain responsible for use, review, escalation, incident response, and retirement.
Small Python demonstrations
Predict, run, and interpret
Each button calls one fixed, allowlisted computation. Use the notebook for longer experiments and saved evidence.
Check evidence coverage
required = {"baseline", "evaluation", "failure", "mitigation", "fallback", "owner"}
present = {"baseline", "evaluation", "failure", "mitigation", "owner"}
print("coverage:", round(len(required & present)/len(required), 2))
print("missing:", sorted(required-present))output
Run this fixed example to compare your prediction with the result.
Sample a defense prompt
prompts = ["Defend the split.", "Explain one failure.", "Verify one AI suggestion."]
print(prompts[1])output
Run this fixed example to compare your prediction with the result.
Enforce non-delegation
action = {"consequential": True, "human_approval": False}
print("authorized:", not action["consequential"] or action["human_approval"])output
Run this fixed example to compare your prediction with the result.
Check your understanding
Ten questions with standard answers
Answer before opening each panel. A good answer connects the concept to evidence, failure, and a bounded decision.
1. What evidence supports this system, and what should never be delegated to it?
Standard answer: A system is defensible only through traceable evidence under stated conditions, honest limitations, tested fallback, and named human responsibility; consequential authority is never transferred to fluency or automation.
2. What is the role of evidence argument in this chapter?
Standard answer: A claim is supported by traceable outputs and tests under stated conditions. Limitations define where the argument stops.
3. Why does individual accountability require evidence rather than intuition?
Standard answer: Each student explains a substantive contribution, one design choice, one failure, one correction, and AI assistance without relying on the team spokesperson.
4. How should a practitioner use non-delegation?
Standard answer: Some decisions remain human because they require authority, care, rights, contextual judgment, or acceptance of consequences.
5. What does the chapter's main formula clarify—and what does it not prove?
Standard answer: A score without conditions and limitations cannot identify where performance transfers; a claim without responsibility cannot identify who acts when the system fails.
6. What should change in the explorer as its risk or complexity controls increase?
Standard answer: Averages can guide rehearsal, but a single missing release gate—such as reproducibility, fallback, or authority—can block the claim regardless of presentation quality.
7. How should the system respond to: Demo as proof?
Standard answer: Use the demo only as an illustration of the frozen evidence dossier.
8. What evidence reveals the failure called Team opacity?
Standard answer: Sampled questions reveal missing individual understanding.
9. When should the system not be used or allowed to proceed?
Standard answer: Never delegate rights-affecting authority, irreversible actions, or responsibility for consequences to the model.
10. How does this week prepare the next stage of the course?
Standard answer: The course ends by returning to Week 1: choose the least complex system, support bounded claims, preserve evidence, and keep responsibility human.
Terminology
Glossary
- Evidence argument
- A bounded claim connected to traceable observations and tests.
- Demonstration
- An illustration of system behavior, not a substitute for evaluation.
- Oral defense
- Individual explanation and justification of substantive work and decisions.
- Responsibility record
- A trace of each person's contribution, decisions, and verification.
- Peer evidence review
- Review focused on claim support, failures, and boundaries rather than polish.
- Synthesis memo
- An individual account connecting course concepts to system evidence and responsibility.
- Non-delegation
- A decision or authority that remains human.
- Release decision
- Documented authorization, limitation, revision, or refusal to deploy.
- Incident response
- Defined action when harmful, insecure, or invalid behavior occurs.
- Retirement
- Controlled end of system use, integrations, data access, and monitoring.
Continue learning
Key references
These primary papers, standards, or official technical documents anchor the chapter. Product names and current legal timelines should be rechecked when used in a real project.