Week 12 Building and Evaluating AI Systems

System Selection, Governance, and Release Decisions

Model quality is only one system criterion. A release decision must connect intended use to data rights, performance, uncertainty, security, privacy, accessibility, cost, latency, control, monitoring, and fallback.

Core question: Which system should we build, and under what conditions should it be released?

SelectCompare the smallest viable systems against quality and operational constraints.

GovernAssign owners, map context, measure risks, and manage mitigations.

ReleaseRequire evidence gates, monitoring, fallback, incident response, and retirement.

By the end

Compare no-AI, specialized, smaller, open-weight, and hosted components.
Apply weighted criteria without hiding disqualifying constraints.
Build a system card and risk register using lifecycle evidence.
Defend release, limited release, revision, or non-release.

Course progression

Build one evidence chain

This week uses the course-wide sequence: frame the use, establish a baseline, build or compare, evaluate failure, document boundaries, and decide.

framebaselinebuildevaluatedocumentdecidemonitor

Connection from the course

Week 13 freezes the PBL2 system and subjects its integrated evidence, controls, and fallback to adversarial testing.

Essential concepts

Understand the parts before combining them

The core lane focuses on transferable judgment. Optional formal or engineering depth belongs in the companion notebook's stretch lane.

System-level comparison

A hosted frontier model may score well but conflict with privacy, latency, cost, or control; a smaller local model may be sufficient and easier to govern.

Risk management lifecycle

Govern establishes accountability; Map defines context; Measure evaluates risks; Manage prioritizes treatment and monitoring.

System documentation

A system card links intended use, architecture, data, evaluation, limitations, security, human boundaries, monitoring, and change history.

Mathematical intuition

One relationship worth keeping

\[S_j=\sum_k w_k\,s_{jk}\quad\text{subject to mandatory constraints}\]

A weighted score can organize trade-offs, but privacy, legality, accessibility, or safety constraints should not be averaged away. Failing a mandatory gate removes an option.

Stretch: what the notation leaves out

Weights make priorities visible, but a high total cannot compensate for a violated non-negotiable requirement such as unlawful data use or missing human oversight.

Interactive explorer

Change assumptions and inspect the decision

Change the importance of quality, privacy/control, and cost/latency. Compare three stylized system options.

Quality importance 7Privacy and control importance 8Cost and latency importance 6

Current signal—

Recommended response—

The preferred option can change with legitimate priorities. Mandatory constraints must be checked before weighted comparison, and sensitivity analysis should accompany the choice.

Evidence workflow

Move from a claim to a decision

Use this order in the chapter, notebook, and project record. Skipping an earlier step weakens every later claim.

1. List no-AI and simple alternatives before model options.
2. Define weighted criteria plus non-negotiable privacy, safety, legal, and accessibility gates.
3. Evaluate quality, slices, uncertainty, security, latency, cost, and operational control.
4. Create a risk register with owner, likelihood, impact, control, residual risk, and trigger.
5. Issue a documented release, limited-release, revise, or do-not-release decision.

Weekly evidence artifact

Model-selection memo, system card, risk register, and defended release decision

Project connection

Approve PBL2 scope and release boundary only after the system card, evaluation plan, risk register, and fallback are inspectable.

Failure analysis

Deliberately look for the claim's boundary

A failure case is useful when the setup, expected behavior, observation, severity, and response are recorded.

Benchmark substitution

Failure: A public benchmark score replaces evaluation on the intended population.

Evidence: Task-specific failures and slices are unknown.

Response: Require local use-case evidence before release.

Average-away governance

Failure: A weighted score compensates for a privacy violation.

Evidence: A mandatory gate is failed despite a high total.

Response: Separate disqualifying constraints from trade-off scores.

Static documentation

Failure: A system card is written once and never updated.

Evidence: Model, data, prompt, or policy versions drift from the card.

Response: Link change control, monitoring, incidents, and retirement to documentation.

Use and non-use

Keep authority proportional to evidence

Intended use

Release only the smallest system that meets intended-use evidence and mandatory constraints.

Do not use

Do not treat vendor claims, benchmark rank, or documentation alone as validation.

Human responsibility

Named owners approve data, risk acceptance, release scope, monitoring, incidents, and retirement.

Small Python demonstrations

Predict, run, and interpret

Each button calls one fixed, allowlisted computation. Use the notebook for longer experiments and saved evidence.

Compare weighted options

weights = {"quality":0.4, "privacy":0.35, "cost":0.25}
options = {"hosted":{"quality":9,"privacy":4,"cost":5}, "local":{"quality":7,"privacy":9,"cost":7}}
for name, scores in options.items():
    total = sum(weights[k]*scores[k] for k in weights)
    print(name, round(total, 2))

output

Run this fixed example to compare your prediction with the result.

Prioritize a risk

likelihood, impact, detectability = 3, 5, 4
print("risk priority:", likelihood*impact*detectability)

output

Run this fixed example to compare your prediction with the result.

Apply release gates

gates = {"task evidence":True, "privacy":True, "fallback":False, "owner":True}
print("release:", all(gates.values()))
print("missing:", [k for k,v in gates.items() if not v])

output

Run this fixed example to compare your prediction with the result.

Check your understanding

Ten questions with standard answers

Answer before opening each panel. A good answer connects the concept to evidence, failure, and a bounded decision.

1. Which system should we build, and under what conditions should it be released?

Standard answer: Select the smallest viable system using task evidence and operational criteria, enforce non-negotiable gates, document residual risk, and release only within a monitored human-governed boundary.

2. What is the role of system-level comparison in this chapter?

Standard answer: A hosted frontier model may score well but conflict with privacy, latency, cost, or control; a smaller local model may be sufficient and easier to govern.

3. Why does risk management lifecycle require evidence rather than intuition?

Standard answer: Govern establishes accountability; Map defines context; Measure evaluates risks; Manage prioritizes treatment and monitoring.

4. How should a practitioner use system documentation?

Standard answer: A system card links intended use, architecture, data, evaluation, limitations, security, human boundaries, monitoring, and change history.

5. What does the chapter's main formula clarify—and what does it not prove?

Standard answer: Weights make priorities visible, but a high total cannot compensate for a violated non-negotiable requirement such as unlawful data use or missing human oversight.

6. What should change in the explorer as its risk or complexity controls increase?

Standard answer: The preferred option can change with legitimate priorities. Mandatory constraints must be checked before weighted comparison, and sensitivity analysis should accompany the choice.

7. How should the system respond to: Benchmark substitution?

Standard answer: Require local use-case evidence before release.

8. What evidence reveals the failure called Average-away governance?

Standard answer: A mandatory gate is failed despite a high total.

9. When should the system not be used or allowed to proceed?

Standard answer: Do not treat vendor claims, benchmark rank, or documentation alone as validation.

10. How does this week prepare the next stage of the course?

Standard answer: Week 13 freezes the PBL2 system and subjects its integrated evidence, controls, and fallback to adversarial testing.

Terminology

Glossary

System selection: Choice among technical and non-technical ways to meet a defined need.
Mandatory gate: A requirement that cannot be compensated for by other strengths.
Weighted score: A transparent aggregation of criterion scores and priorities.
Sensitivity analysis: Checking whether a decision changes under reasonable weights or assumptions.
Risk register: A living record of risks, owners, controls, residual exposure, and triggers.
Residual risk: Risk remaining after controls are applied.
System card: Documentation of a deployed system in its use context.
Release boundary: The users, conditions, actions, and limits authorized for operation.
Monitoring trigger: A condition requiring investigation, rollback, retraining, or suspension.
Retirement: Controlled removal of a system and its data, integrations, and responsibilities.

Continue learning

Key references

These primary papers, standards, or official technical documents anchor the chapter. Product names and current legal timelines should be rechecked when used in a real project.