Lesson overview | Previous part | Lesson overview

Linear Transformations: Appendix N: Notation Summary for This Section to Appendix P: Quick Reference - Common Linear Maps in $\mathbb{R}^2$ and $\mathbb{R}^3$

Appendix N: Notation Summary for This Section

---

Appendix O: Linear Maps and Symmetry

O.1 Equivariant Maps

A linear map $T: V \to W$ is equivariant with respect to a group $G$ if, for every $g \in G$ and every $\mathbf{v} \in V$:

$$T(\rho_V(g)\mathbf{v}) = \rho_W(g) T(\mathbf{v})$$

where $\rho_V: G \to GL(V)$ and $\rho_W: G \to GL(W)$ are representations of $G$ on $V$ and $W$.

Intuitively: "applying the group action then the map = applying the map then the group action." The map commutes with the symmetry.

Examples:

• Translation equivariance: $T(v + c) = T(v) + T(c)$... but this is additivity - every linear map is equivariant to the translation group on vector spaces.
• Rotation equivariance: $T(R\mathbf{v}) = RT(\mathbf{v})$ for all rotations $R$. In 3D: implies $T = \lambda I$ for some scalar $\lambda$ (Schur's lemma for the rotation representation).
• Permutation equivariance: $T(P\mathbf{v}) = PT(\mathbf{v})$ for all permutation matrices $P$. Implies $T$ is a sum of a "same-position" term and a "mean-field" term - this is why mean pooling and attention with tied weights are permutation equivariant.

For AI: CNNs achieve translation equivariance by using convolutional (shared-weight) linear maps. Equivariant graph neural networks use permutation-equivariant maps. Geometric deep learning is the systematic study of building neural networks as compositions of equivariant linear maps. Transformer attention (without positional encoding) is permutation equivariant - adding positional encodings explicitly breaks this symmetry.

O.2 Schur's Lemma and Irreducible Representations

Schur's Lemma. Let $T: V \to V$ be a linear map that commutes with all maps in an irreducible representation of a group $G$ (i.e., $T\rho(g) = \rho(g)T$ for all $g \in G$). Then $T = \lambda I$ for some scalar $\lambda$.

This powerful result says: the only linear maps that commute with all symmetries of an irreducible representation are scalar multiples of the identity. This constrains the form of equivariant maps.

Application: If attention weights must be equivariant to the representation of a certain symmetry group acting on the heads, Schur's lemma constrains the possible attention patterns.

O.3 Representation Theory Preview

Representation theory studies how groups act on vector spaces via linear maps. Every group representation $\rho: G \to GL(V)$ is a group homomorphism - a map that takes group elements to invertible linear maps, preserving the group structure:

$$\rho(gh) = \rho(g)\rho(h) \quad \text{(composition respects group multiplication)}$$

This is the language in which equivariant neural networks (E(3)-equivariant networks for molecular property prediction, SE(3)-equivariant networks for robotics, permutation-equivariant networks for sets) are designed. The "weights" of an equivariant linear layer are constrained to be equivariant - and representation theory tells you exactly what form these weights can take.

---

Appendix P: Quick Reference - Common Linear Maps in $\mathbb{R}^2$ and $\mathbb{R}^3$

Common $2 \times 2$ Linear Maps

All these are linear maps. To make them affine (include translation), append a row and column in homogeneous form.

Common $3 \times 3$ Linear Maps

---

End of Linear Transformations section. Continue to 05: Orthogonality and Orthonormality.