Week 9 · Hands-on foundations

Convolution and a Compact CNN Workflow

How does convolution use image structure, and what evidence shows that a compact CNN helps?

The historical image unit is long enough to fill more than one week, but the semester also reserves Weeks 10 and 14 for PBL presentations. This compressed week keeps the durable path: local filters, shared weights, output shapes, one compact CNN, a matched baseline, and error inspection. Architecture surveys and large downloads remain optional.

This chapter condenses materials taught in the 2023–2025 Advanced Data Science course into a smaller core path. Optional depth remains available in the companion notebook.

By the end

Apply a kernel and calculate its output size.
Trace channel, height, and width through a compact CNN.
Compare a CNN with an MLP under the same split and training budget.
Inspect class-specific errors before claiming that convolution helps.

Low floor

Use the explorer and three short Python examples before changing a longer model.

High ceiling

Add one second convolution block only after documenting how it changes receptive field, parameter count, runtime, and held-out errors.

Core ideas

Three concepts to keep

Locality and sharing

A convolution reuses one small kernel across positions. This reduces parameters and encodes the assumption that a useful local pattern may appear in different locations.

Shape through the network

Convolution, activation, and pooling change channels and spatial resolution in predictable ways. Calculate each shape before connecting the classifier head.

Controlled comparison

Architecture is not evidence by itself. A CNN should be compared with a simple image baseline using the same data, split, preprocessing, budget, and metric.

Mathematical intuition

One relationship worth keeping

\[H_{\mathrm{out}}=\left\lfloor\frac{H+2P-K}{S}\right\rfloor+1\]

Input height H, padding P, kernel size K, and stride S determine the number of vertical filter positions. Width follows the same rule.

Stretch: read the symbols slowly

Identify the input, learned quantity, output, and aggregation. Re-create the relationship with a tiny hand-checkable example before using a library layer.

Interactive explorer

Change one assumption at a time

Predict the direction of change before moving a control.

Image size 12Kernel size 3Stride 1

Current evidence—

Interpretation—

Change a control to inspect the relationship.

Hands-on path

A reusable five-step routine

1. Represent images as batch × channel × height × width.
2. Calculate one filter response and output shape by hand.
3. Build one convolution–activation–pooling block and print every shape.
4. Fit an MLP baseline and compact CNN under comparable conditions.
5. Inspect confusion patterns and example mistakes, then state a bounded conclusion.

Minimal Python

Predict, run, and explain

Each example is independent. Explain its output before copying it into a larger workflow.

Calculate one filter response

patch = [[1,2],[3,4]]
kernel = [[1,0],[0,-1]]
value = sum(patch[i][j]*kernel[i][j] for i in range(2) for j in range(2))
print(value)

output

Predict the result, then click Run.

Calculate output size

image, kernel, padding, stride = 28, 3, 1, 2
out = (image + 2*padding - kernel)//stride + 1
print(out)

output

Predict the result, then click Run.

Count convolution parameters

in_channels, out_channels, kernel = 3, 16, 3
print(out_channels * (in_channels*kernel*kernel + 1))

output

Predict the result, then click Run.

Failure checks

What can look correct while being wrong?

Channel order confusion

What goes wrong: Height is treated as the channel dimension.

Check: Print the batch × channel × height × width shape.

Unexpected output size

What goes wrong: Stride or missing padding shrinks the image too quickly.

Check: Calculate the formula before running.

Kernel as a picture label

What goes wrong: A learned feature map is overinterpreted as a human concept.

Check: Treat visualization as evidence about response, not a semantic guarantee.

Check your understanding

Ten questions with standard answers

Answer in your own words before opening each panel.

1. How does convolution use image structure, and what evidence shows that a compact CNN helps?

Standard answer: A strong answer connects the central idea to a visible computation and a held-out or shape-based check. Architecture surveys and large downloads remain optional.

2. What does locality and sharing mean here?

Standard answer: A convolution reuses one small kernel across positions. This reduces parameters and encodes the assumption that a useful local pattern may appear in different locations.

3. Why is shape through the network useful?

Standard answer: Convolution, activation, and pooling change channels and spatial resolution in predictable ways. Calculate each shape before connecting the classifier head.

4. How should you interpret controlled comparison?

Standard answer: Architecture is not evidence by itself. A CNN should be compared with a simple image baseline using the same data, split, preprocessing, budget, and metric.

5. What does the main equation clarify—and what does it not prove?

Standard answer: Input height H, padding P, kernel size K, and stride S determine the number of vertical filter positions. Width follows the same rule. It does not by itself prove useful behavior on unseen intended-use cases.

6. What should change when you move the explorer controls?

Standard answer: The visible calculation and interpretation should change together. Predict the direction first, then use the result to correct your mental model.

7. What is the first hands-on check you should perform?

Standard answer: Represent images as batch × channel × height × width.

8. How can you detect the failure called “Channel order confusion”?

Standard answer: Print the batch × channel × height × width shape.

9. What evidence should be recorded before making a claim?

Standard answer: Record data and split assumptions, input/output shapes, settings, the baseline, held-out metrics, representative failures, and the decision supported by that evidence.

10. How does this week prepare the next topic?

Standard answer: Week 10 is reserved for both classes of PBL1 presentations. Week 11 then compresses simple RNN and LSTM concepts into one sequence-model week.

Terminology

Glossary

Convolution: Local weighted operation applied across positions.
Kernel: Small shared weight array used by a convolution.
Filter: Common name for a kernel producing a feature map.
Receptive field: Input region affecting one output location.
Feature map: Spatial array of filter responses.
Channel: One plane of input or learned features.
Stride: Number of positions moved between filter applications.
Padding: Values added around input boundaries.
Pooling: Local summarization that reduces spatial resolution.
Translation equivariance: Input shifts produce corresponding shifts in feature maps under suitable conditions.

Go further

Key references

Next: Week 10 is reserved for both classes of PBL1 presentations. Week 11 then compresses simple RNN and LSTM concepts into one sequence-model week.