Exercises Notebook

Converted from exercises.ipynb for web reading.

CNN and Convolution Math: Exercises

Ten exercises cover convolution indexing, output shapes, parameter counts, pooling, receptive fields, depthwise separable convolution, im2col, residuals, and diagnostics.

Code cell 2

import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl

try:
    import seaborn as sns
    sns.set_theme(style="whitegrid", palette="colorblind")
    HAS_SNS = True
except ImportError:
    plt.style.use("seaborn-v0_8-whitegrid")
    HAS_SNS = False

mpl.rcParams.update({
    "figure.figsize":    (10, 6),
    "figure.dpi":         120,
    "font.size":           13,
    "axes.titlesize":      15,
    "axes.labelsize":      13,
    "xtick.labelsize":     11,
    "ytick.labelsize":     11,
    "legend.fontsize":     11,
    "legend.framealpha":   0.85,
    "lines.linewidth":      2.0,
    "axes.spines.top":     False,
    "axes.spines.right":   False,
    "savefig.bbox":       "tight",
    "savefig.dpi":         150,
})
np.random.seed(42)
print("Plot setup complete.")

Exercise 1: 1D cross-correlation

Compute valid cross-correlation.

Code cell 4

# Your Solution
x = np.array([1, 2, 3])
w = np.array([1, -1])
print("Starter: output length is len(x)-len(w)+1.")

Code cell 5

# Solution
x = np.array([1, 2, 3])
w = np.array([1, -1])
y = np.array([np.sum(x[i:i+2] * w) for i in range(2)])
print("y:", y)

Exercise 2: Output size

Compute output size for H=32,K=3,P=1,S=1,D=1.

Code cell 7

# Your Solution
H, K, P, S, D = 32, 3, 1, 1, 1
print("Starter: floor((H+2P-D(K-1)-1)/S)+1.")

Code cell 8

# Solution
H, K, P, S, D = 32, 3, 1, 1, 1
out = int(np.floor((H + 2*P - D*(K-1) - 1)/S) + 1)
print("out:", out)

Exercise 3: Conv parameters

Count parameters for 3x3 conv from 16 to 32 channels.

Code cell 10

# Your Solution
C_in, C_out, K = 16, 32, 3
print("Starter: C_out*C_in*K*K + C_out.")

Code cell 11

# Solution
C_in, C_out, K = 16, 32, 3
params = C_out * C_in * K * K + C_out
print("params:", params)

Exercise 4: Dense versus conv

Compare dense and conv params for 32x32x3 to 64 features.

Code cell 13

# Your Solution
H, W, C, out = 32, 32, 3, 64
K = 3
print("Starter: dense=(H*W*C)*out; conv=out*C*K*K.")

Code cell 14

# Solution
H, W, C, out = 32, 32, 3, 64
K = 3
dense = H * W * C * out
conv = out * C * K * K
print("dense:", dense, "conv:", conv)

Exercise 5: Pooling

Compute max and average of a 2x2 patch.

Code cell 16

# Your Solution
patch = np.array([[1, 5], [2, 4]])
print("Starter: patch.max() and patch.mean().")

Code cell 17

# Solution
patch = np.array([[1, 5], [2, 4]])
print("max:", patch.max(), "avg:", patch.mean())

Exercise 6: Receptive field

Compute R after two 3x3 stride-1 layers.

Code cell 19

# Your Solution
R, J = 1, 1
print("Starter: repeat R = R + (K-1)*J.")

Code cell 20

# Solution
R, J = 1, 1
for _ in range(2):
    R = R + (3 - 1) * J
    J = J * 1
print("R:", R)

Exercise 7: Depthwise separable count

Count depthwise separable params.

Code cell 22

# Your Solution
C_in, C_out, K = 32, 64, 3
print("Starter: C_in*K*K + C_in*C_out.")

Code cell 23

# Solution
C_in, C_out, K = 32, 64, 3
params = C_in * K * K + C_in * C_out
print("params:", params)

Exercise 8: im2col shape

Find im2col shape for 4x4 input and 2x2 kernel.

Code cell 25

# Your Solution
H, W, K = 4, 4, 2
print("Starter: rows=K*K, cols=(H-K+1)*(W-K+1).")

Code cell 26

# Solution
H, W, K = 4, 4, 2
shape = (K * K, (H - K + 1) * (W - K + 1))
print("im2col shape:", shape)

Exercise 9: Residual output

Compute y=x+F(x).

Code cell 28

# Your Solution
x = np.array([1.0, 2.0])
F = np.array([-0.5, 0.25])
print("Starter: y=x+F.")

Code cell 29

# Solution
x = np.array([1.0, 2.0])
F = np.array([-0.5, 0.25])
y = x + F
print("y:", y)

Exercise 10: CNN checklist

Write four CNN debugging checks.

Code cell 31

# Your Solution
print("Starter: include shape, output size, receptive field, activation stats.")

Code cell 32

# Solution
checks = [
    "track N,C,H,W axes explicitly",
    "verify output sizes after padding, stride, and dilation",
    "test receptive field with controlled inputs",
    "inspect activation statistics for dead channels",
]
for check in checks:
    print("-", check)

Closing Reflection

CNNs are shape discipline plus spatial inductive bias. Most errors become visible when you write down the axes and output-size formula.