Lesson overview | Previous part | Next part

Sigma Algebras: Part 2: Formal Definitions

2. Formal Definitions

Formal Definitions develops the part of sigma algebras specified by the approved Chapter 24 table of contents. The treatment is measure-theoretic and AI-facing: every concept is tied to probability, expectation, density, or learning systems.

2.1 Algebras of sets vs sigma algebras

Algebras of sets vs sigma algebras belongs to the canonical scope of Sigma Algebras. Here the point is not to repeat introductory probability, but to expose the measurable structure that makes the probability statement valid.

Working scope for this subsection: measurable spaces, generated sigma algebras, Borel sets, product sigma algebras, measurable maps, and AI observability. The mathematical habit is to name the space, the sigma algebra, the measure, and the map before writing probabilities or expectations.

$$\sigma(\mathcal{C})=\bigcap\{\mathcal{F}:\mathcal{F}\text{ is a sigma algebra and }\mathcal{C}\subseteq\mathcal{F}\}.$$

Operational definition.

A sigma algebra is a collection of subsets closed under complements and countable unions. It is the list of events for which the model agrees that probability, integration, and observation are meaningful.

Worked reading.

On a finite universe, a generator such as a model flag partitions examples into visible cells. The generated sigma algebra contains every union of those cells, because any observable event must be expressible from the available information.

Three examples of algebras of sets vs sigma algebras:

1. All subsets of a finite dataset.
2. Borel sets generated by open intervals in $\mathbb{R}$.
3. Events determined by the first $n$ tokens of a sequence.

Two non-examples clarify the boundary:

1. A collection closed under finite unions but not countable unions.
2. A feature filter whose inverse image is not in the source sigma algebra.

Proof or verification habit for algebras of sets vs sigma algebras:

Most sigma algebra proofs use closure and minimality: show a family is closed, then use intersection of all eligible closed families to prove generated objects exist.

set question        -> is the subset measurable?
function question   -> are inverse images measurable?
integral question   -> is the function measurable and integrable?
density question    -> is absolute continuity satisfied?
ML question         -> which measure defines the population claim?

In AI systems, algebras of sets vs sigma algebras matters because probability language is constantly compressed into informal notation. Measure theory expands the notation so support, observability, null sets, and convergence assumptions are visible.

In AI, sigma algebras describe what information a model, evaluator, or monitoring system can distinguish.

Practical checklist:

• Name the measurable space before naming the probability.
• Identify whether the object is a set, function, measure, distribution, or derivative of measures.
• Check whether equality is pointwise, almost everywhere, or distributional.
• Check whether limits are moved through integrals and which theorem justifies the move.
• For density ratios, check support and absolute continuity before dividing.
• For ML claims, distinguish population measure, empirical measure, model measure, and proposal measure.

Local diagnostic: Ask which subsets of examples are observable from the features or logs.

The notebook version of this subsection uses finite spaces, step functions, empirical measures, or simple density ratios. These toy cases keep the objects visible while preserving the exact logic used in continuous ML models.

The learner should leave this subsection able to translate between the compact ML notation and the full measure-theoretic statement.

A useful way to study this subsection is to keep three layers separate:

1. Semantic layer: what real-world question is being asked?
2. Measurable layer: which event, function, or measure represents that question?
3. Computational layer: which sum, integral, sample average, or ratio estimates it?

For example, the semantic question may be whether a guardrail fails on a class of prompts. The measurable layer is an event in the prompt space. The computational layer is an empirical estimate under a validation or red-team distribution. Mixing these layers is how many probability arguments become ambiguous.

The same discipline applies to generative models. A generator is a measurable transformation of latent randomness. The generated distribution is the pushforward measure. A likelihood, density, or divergence is only meaningful after the target space, base measure, and support relation are clear.

When reading ML papers, silently expand phrases like "sample from the model," "take expectation over data," and "density ratio" into this measure-theoretic checklist. This turns informal notation into a statement that can be checked.

This is the level of precision needed for high-stakes evaluation, off-policy learning, variational inference, and theoretical generalization arguments.

A final question to ask is whether the claim would still be meaningful if the dataset were infinite, the model output lived in a function space, or the event being queried were defined by a limiting process. Measure theory is what keeps the answer honest.

2.2 Generated sigma algebra $\sigma(\mathcal{C})$

Generated sigma algebra $\sigma(\mathcal{C})$ belongs to the canonical scope of Sigma Algebras. Here the point is not to repeat introductory probability, but to expose the measurable structure that makes the probability statement valid.

Working scope for this subsection: measurable spaces, generated sigma algebras, Borel sets, product sigma algebras, measurable maps, and AI observability. The mathematical habit is to name the space, the sigma algebra, the measure, and the map before writing probabilities or expectations.

$$\mathcal{B}(\mathbb{R}^n)=\sigma\bigl(\{\text{open subsets of }\mathbb{R}^n\}\bigr).$$

Operational definition.

A sigma algebra is a collection of subsets closed under complements and countable unions. It is the list of events for which the model agrees that probability, integration, and observation are meaningful.

Worked reading.

On a finite universe, a generator such as a model flag partitions examples into visible cells. The generated sigma algebra contains every union of those cells, because any observable event must be expressible from the available information.

Three examples of generated sigma algebra $\sigma(\mathcal{c})$:

1. All subsets of a finite dataset.
2. Borel sets generated by open intervals in $\mathbb{R}$.
3. Events determined by the first $n$ tokens of a sequence.

Two non-examples clarify the boundary:

1. A collection closed under finite unions but not countable unions.
2. A feature filter whose inverse image is not in the source sigma algebra.

Proof or verification habit for generated sigma algebra $\sigma(\mathcal{c})$:

Most sigma algebra proofs use closure and minimality: show a family is closed, then use intersection of all eligible closed families to prove generated objects exist.

set question        -> is the subset measurable?
function question   -> are inverse images measurable?
integral question   -> is the function measurable and integrable?
density question    -> is absolute continuity satisfied?
ML question         -> which measure defines the population claim?

In AI systems, generated sigma algebra $\sigma(\mathcal{c})$ matters because probability language is constantly compressed into informal notation. Measure theory expands the notation so support, observability, null sets, and convergence assumptions are visible.

In AI, sigma algebras describe what information a model, evaluator, or monitoring system can distinguish.

Practical checklist:

• Name the measurable space before naming the probability.
• Identify whether the object is a set, function, measure, distribution, or derivative of measures.
• Check whether equality is pointwise, almost everywhere, or distributional.
• Check whether limits are moved through integrals and which theorem justifies the move.
• For density ratios, check support and absolute continuity before dividing.
• For ML claims, distinguish population measure, empirical measure, model measure, and proposal measure.

Local diagnostic: Ask which subsets of examples are observable from the features or logs.

The notebook version of this subsection uses finite spaces, step functions, empirical measures, or simple density ratios. These toy cases keep the objects visible while preserving the exact logic used in continuous ML models.

The learner should leave this subsection able to translate between the compact ML notation and the full measure-theoretic statement.

A useful way to study this subsection is to keep three layers separate:

1. Semantic layer: what real-world question is being asked?
2. Measurable layer: which event, function, or measure represents that question?
3. Computational layer: which sum, integral, sample average, or ratio estimates it?

For example, the semantic question may be whether a guardrail fails on a class of prompts. The measurable layer is an event in the prompt space. The computational layer is an empirical estimate under a validation or red-team distribution. Mixing these layers is how many probability arguments become ambiguous.

The same discipline applies to generative models. A generator is a measurable transformation of latent randomness. The generated distribution is the pushforward measure. A likelihood, density, or divergence is only meaningful after the target space, base measure, and support relation are clear.

When reading ML papers, silently expand phrases like "sample from the model," "take expectation over data," and "density ratio" into this measure-theoretic checklist. This turns informal notation into a statement that can be checked.

This is the level of precision needed for high-stakes evaluation, off-policy learning, variational inference, and theoretical generalization arguments.

A final question to ask is whether the claim would still be meaningful if the dataset were infinite, the model output lived in a function space, or the event being queried were defined by a limiting process. Measure theory is what keeps the answer honest.

2.3 Borel sigma algebra on $\mathbb{R}^n$

Borel sigma algebra on $\mathbb{R}^n$ belongs to the canonical scope of Sigma Algebras. Here the point is not to repeat introductory probability, but to expose the measurable structure that makes the probability statement valid.

Working scope for this subsection: measurable spaces, generated sigma algebras, Borel sets, product sigma algebras, measurable maps, and AI observability. The mathematical habit is to name the space, the sigma algebra, the measure, and the map before writing probabilities or expectations.

$$X:(\Omega,\mathcal{F})\to(\mathcal{X},\mathcal{G})\text{ is measurable iff }X^{-1}(B)\in\mathcal{F}\text{ for every }B\in\mathcal{G}.$$

Operational definition.

A sigma algebra is a collection of subsets closed under complements and countable unions. It is the list of events for which the model agrees that probability, integration, and observation are meaningful.

Worked reading.

On a finite universe, a generator such as a model flag partitions examples into visible cells. The generated sigma algebra contains every union of those cells, because any observable event must be expressible from the available information.

Three examples of borel sigma algebra on $\mathbb{r}^n$:

1. All subsets of a finite dataset.
2. Borel sets generated by open intervals in $\mathbb{R}$.
3. Events determined by the first $n$ tokens of a sequence.

Two non-examples clarify the boundary:

1. A collection closed under finite unions but not countable unions.
2. A feature filter whose inverse image is not in the source sigma algebra.

Proof or verification habit for borel sigma algebra on $\mathbb{r}^n$:

Most sigma algebra proofs use closure and minimality: show a family is closed, then use intersection of all eligible closed families to prove generated objects exist.

set question        -> is the subset measurable?
function question   -> are inverse images measurable?
integral question   -> is the function measurable and integrable?
density question    -> is absolute continuity satisfied?
ML question         -> which measure defines the population claim?

In AI systems, borel sigma algebra on $\mathbb{r}^n$ matters because probability language is constantly compressed into informal notation. Measure theory expands the notation so support, observability, null sets, and convergence assumptions are visible.

In AI, sigma algebras describe what information a model, evaluator, or monitoring system can distinguish.

Practical checklist:

• Name the measurable space before naming the probability.
• Identify whether the object is a set, function, measure, distribution, or derivative of measures.
• Check whether equality is pointwise, almost everywhere, or distributional.
• Check whether limits are moved through integrals and which theorem justifies the move.
• For density ratios, check support and absolute continuity before dividing.
• For ML claims, distinguish population measure, empirical measure, model measure, and proposal measure.

Local diagnostic: Ask which subsets of examples are observable from the features or logs.

The notebook version of this subsection uses finite spaces, step functions, empirical measures, or simple density ratios. These toy cases keep the objects visible while preserving the exact logic used in continuous ML models.

The learner should leave this subsection able to translate between the compact ML notation and the full measure-theoretic statement.

A useful way to study this subsection is to keep three layers separate:

1. Semantic layer: what real-world question is being asked?
2. Measurable layer: which event, function, or measure represents that question?
3. Computational layer: which sum, integral, sample average, or ratio estimates it?

For example, the semantic question may be whether a guardrail fails on a class of prompts. The measurable layer is an event in the prompt space. The computational layer is an empirical estimate under a validation or red-team distribution. Mixing these layers is how many probability arguments become ambiguous.

The same discipline applies to generative models. A generator is a measurable transformation of latent randomness. The generated distribution is the pushforward measure. A likelihood, density, or divergence is only meaningful after the target space, base measure, and support relation are clear.

When reading ML papers, silently expand phrases like "sample from the model," "take expectation over data," and "density ratio" into this measure-theoretic checklist. This turns informal notation into a statement that can be checked.

This is the level of precision needed for high-stakes evaluation, off-policy learning, variational inference, and theoretical generalization arguments.

A final question to ask is whether the claim would still be meaningful if the dataset were infinite, the model output lived in a function space, or the event being queried were defined by a limiting process. Measure theory is what keeps the answer honest.

2.4 Measurable spaces $(\Omega,\mathcal{F})$

Measurable spaces $(\Omega,\mathcal{F})$ belongs to the canonical scope of Sigma Algebras. Here the point is not to repeat introductory probability, but to expose the measurable structure that makes the probability statement valid.

Working scope for this subsection: measurable spaces, generated sigma algebras, Borel sets, product sigma algebras, measurable maps, and AI observability. The mathematical habit is to name the space, the sigma algebra, the measure, and the map before writing probabilities or expectations.

$$\mathcal{F}\subseteq 2^{\Omega},\quad \Omega\in\mathcal{F},\quad A\in\mathcal{F}\Rightarrow A^c\in\mathcal{F},\quad A_n\in\mathcal{F}\Rightarrow \bigcup_{n=1}^{\infty}A_n\in\mathcal{F}.$$

Operational definition.

Measurable spaces $(\Omega,\mathcal{F})$ is part of the canonical scope of Sigma Algebras: measurable spaces, generated sigma algebras, Borel sets, product sigma algebras, measurable maps, and AI observability.

Worked reading.

Begin with the measurable objects, identify the measure, then state which integral or probability claim is being made.

Three examples of measurable spaces $(\omega,\mathcal{f})$:

1. A finite synthetic example.
2. A probability model used in ML.
3. A measurable transformation of model outputs.

Two non-examples clarify the boundary:

1. An undefined probability claim.
2. A density written without a base measure.

Proof or verification habit for measurable spaces $(\omega,\mathcal{f})$:

The proof habit is to reduce the claim to measurable sets, simple functions, or finite partitions before passing to limits.

set question        -> is the subset measurable?
function question   -> are inverse images measurable?
integral question   -> is the function measurable and integrable?
density question    -> is absolute continuity satisfied?
ML question         -> which measure defines the population claim?

In AI systems, measurable spaces $(\omega,\mathcal{f})$ matters because probability language is constantly compressed into informal notation. Measure theory expands the notation so support, observability, null sets, and convergence assumptions are visible.

The AI role is to make probabilistic modeling assumptions explicit rather than hidden in notation.

Practical checklist:

• Name the measurable space before naming the probability.
• Identify whether the object is a set, function, measure, distribution, or derivative of measures.
• Check whether equality is pointwise, almost everywhere, or distributional.
• Check whether limits are moved through integrals and which theorem justifies the move.
• For density ratios, check support and absolute continuity before dividing.
• For ML claims, distinguish population measure, empirical measure, model measure, and proposal measure.

Local diagnostic: Name the measurable space, the measure, and the map.

The notebook version of this subsection uses finite spaces, step functions, empirical measures, or simple density ratios. These toy cases keep the objects visible while preserving the exact logic used in continuous ML models.

The learner should leave this subsection able to translate between the compact ML notation and the full measure-theoretic statement.

A useful way to study this subsection is to keep three layers separate:

1. Semantic layer: what real-world question is being asked?
2. Measurable layer: which event, function, or measure represents that question?
3. Computational layer: which sum, integral, sample average, or ratio estimates it?

For example, the semantic question may be whether a guardrail fails on a class of prompts. The measurable layer is an event in the prompt space. The computational layer is an empirical estimate under a validation or red-team distribution. Mixing these layers is how many probability arguments become ambiguous.

The same discipline applies to generative models. A generator is a measurable transformation of latent randomness. The generated distribution is the pushforward measure. A likelihood, density, or divergence is only meaningful after the target space, base measure, and support relation are clear.

When reading ML papers, silently expand phrases like "sample from the model," "take expectation over data," and "density ratio" into this measure-theoretic checklist. This turns informal notation into a statement that can be checked.

This is the level of precision needed for high-stakes evaluation, off-policy learning, variational inference, and theoretical generalization arguments.

A final question to ask is whether the claim would still be meaningful if the dataset were infinite, the model output lived in a function space, or the event being queried were defined by a limiting process. Measure theory is what keeps the answer honest.

2.5 Measurable maps and random variables

Measurable maps and random variables belongs to the canonical scope of Sigma Algebras. Here the point is not to repeat introductory probability, but to expose the measurable structure that makes the probability statement valid.

Working scope for this subsection: measurable spaces, generated sigma algebras, Borel sets, product sigma algebras, measurable maps, and AI observability. The mathematical habit is to name the space, the sigma algebra, the measure, and the map before writing probabilities or expectations.

$$\sigma(\mathcal{C})=\bigcap\{\mathcal{F}:\mathcal{F}\text{ is a sigma algebra and }\mathcal{C}\subseteq\mathcal{F}\}.$$

Operational definition.

A measurable map is a function whose observable target events pull back to observable source events.

Worked reading.

If $X$ maps raw prompts to toxicity scores, then $\{X>0.8\}$ must be an event in the raw prompt space. Otherwise the probability of high toxicity is not defined by the model.

Three examples of measurable maps and random variables:

1. A tokenizer from strings to token ids.
2. An embedding map from text to $\mathbb{R}^d$.
3. A classifier score whose threshold events are measurable.

Two non-examples clarify the boundary:

1. A function whose threshold set is not an event.
2. A hidden logging transformation with no specified event space.

Proof or verification habit for measurable maps and random variables:

To prove measurability into a generated sigma algebra, it is enough to check preimages of the generating class.

set question        -> is the subset measurable?
function question   -> are inverse images measurable?
integral question   -> is the function measurable and integrable?
density question    -> is absolute continuity satisfied?
ML question         -> which measure defines the population claim?

In AI systems, measurable maps and random variables matters because probability language is constantly compressed into informal notation. Measure theory expands the notation so support, observability, null sets, and convergence assumptions are visible.

This is the formal reason feature engineering and preprocessing must preserve measurable events.

Practical checklist:

• Name the measurable space before naming the probability.
• Identify whether the object is a set, function, measure, distribution, or derivative of measures.
• Check whether equality is pointwise, almost everywhere, or distributional.
• Check whether limits are moved through integrals and which theorem justifies the move.
• For density ratios, check support and absolute continuity before dividing.
• For ML claims, distinguish population measure, empirical measure, model measure, and proposal measure.

Local diagnostic: For every target event you will query, can you pull it back to a source event?

The notebook version of this subsection uses finite spaces, step functions, empirical measures, or simple density ratios. These toy cases keep the objects visible while preserving the exact logic used in continuous ML models.

The learner should leave this subsection able to translate between the compact ML notation and the full measure-theoretic statement.

A useful way to study this subsection is to keep three layers separate:

1. Semantic layer: what real-world question is being asked?
2. Measurable layer: which event, function, or measure represents that question?
3. Computational layer: which sum, integral, sample average, or ratio estimates it?

For example, the semantic question may be whether a guardrail fails on a class of prompts. The measurable layer is an event in the prompt space. The computational layer is an empirical estimate under a validation or red-team distribution. Mixing these layers is how many probability arguments become ambiguous.

The same discipline applies to generative models. A generator is a measurable transformation of latent randomness. The generated distribution is the pushforward measure. A likelihood, density, or divergence is only meaningful after the target space, base measure, and support relation are clear.

When reading ML papers, silently expand phrases like "sample from the model," "take expectation over data," and "density ratio" into this measure-theoretic checklist. This turns informal notation into a statement that can be checked.

This is the level of precision needed for high-stakes evaluation, off-policy learning, variational inference, and theoretical generalization arguments.

A final question to ask is whether the claim would still be meaningful if the dataset were infinite, the model output lived in a function space, or the event being queried were defined by a limiting process. Measure theory is what keeps the answer honest.