Lesson 2.5: The Chi-Squared (χ²) Distribution

We now meet the first and most fundamental of the 'sampling distributions.' The Chi-Squared distribution is built from the Normal distribution and its primary job is to model the behavior of sample variance. It is the essential building block for constructing both the t-distribution and the F-distribution.

Part 1: Constructing the Chi-Squared Distribution

Imagine we take a standard normal variable, $Z \sim \mathcal{N}(0, 1)$ , and we square it. What is the distribution of $Z^2$ ? It's certainly not Normal anymore (it can't be negative!). This simple question leads us directly to the definition of the Chi-Squared distribution.

The Core Idea: A Chi-Squared distribution is the distribution of a sum of squared independent standard normal variables. It is the fundamental distribution for anything involving variance.

Definition: The χ² (Chi-Squared) Distribution

Let $Z_1, Z_2, \dots, Z_{\nu}$ are $\nu$ independent random variables, and each is distributed Standard Normal ( $Z_i \sim \mathcal{N}(0, 1)$ ), then the sum of their squares follows a $\chi^2$ distribution with $\nu$ degrees of freedom ( $\text{df}$ ).

\chi^2_{\nu} = Z_1^2 + Z_2^2 + \dots + Z_{\nu}^2 = \sum_{i=1}^{\nu} Z_i^2

Part 2: Properties of the Chi-Squared Distribution

The Single Parameter: Degrees of Freedom (

k

)

The entire shape of the distribution is determined by how many squared Normals you add together.

Imagine a plot showing several Chi-Squared curves. For low k (like 2), it's highly skewed right. As k increases to 10 or 20, the curve becomes more symmetric and bell-shaped.

Shape: The distribution is always skewed to the right and is only defined for positive values (since it's a sum of squares).
Symmetry: It becomes less skewed and more symmetric as the degrees of freedom ( $k$ ) increase.

Moments of the Chi-Squared Distribution

The mean and variance have a beautifully simple relationship with the degrees of freedom.

Chi-Squared Moments

If $X \sim \chi^2_k$ :

Expected Value: $E[X] = k$

Variance: $\text{Var}(X) = 2k$

Part 3: The Critical Connection to Sample Variance

The abstract definition is powerful, but the reason the Chi-Squared distribution is a cornerstone of statistics is its relationship to the sample variance ( $s^2$ ).

Theorem: The Distribution of Sample Variance

Let $X_1, X_2, \dots, X_n$ be an i.i.d. sample from a $\mathcal{N}(\mu, \sigma^2)$ population.

Let $s^2 = \frac{1}{n-1}\sum(X_i - \bar{X})^2$ be the sample variance.

Then the following quantity has a Chi-Squared distribution with $n-1$ degrees of freedom:

\frac{(n-1)s^2}{\sigma^2} \sim \chi^2_{n-1}

Intuition: Why n-1 Degrees of Freedom?

We start with $n$ independent pieces of information. However, to calculate the sample variance $s^2$ , we first have to calculate the sample mean $\bar{X}$ from the same data. The sample mean acts as one constraint on the data. For example, once you know $\bar{X}$ and the first $n-1$ data points, the last data point is no longer free to vary; its value is fixed. Therefore, only $n-1$ pieces of information are "free" to determine the sample variance.

The Payoff: Why the Chi-Squared is Essential

Foundation for Other Tests: This is its most important role. The Chi-Squared distribution is a prerequisite for understanding the next two distributions in our toolbox.
- The **t-distribution** is formed by a ratio involving a Normal and a Chi-Squared variable.
- The **F-distribution** is formed by a ratio of two Chi-Squared variables.
Hypothesis Testing for Variance: This theorem allows us to construct confidence intervals and perform hypothesis tests on a population variance $\sigma^2$ , a key task in quality control and financial risk assessment (e.g., "Is the volatility of our new trading strategy significantly lower than the old one?").
Goodness-of-Fit Tests: In machine learning and data analysis, the famous Pearson's Chi-Squared test uses this distribution to check if the observed counts in different categories match the expected counts from a theory. It's a fundamental tool for A/B testing and analyzing categorical data.

What's Next? Building the t-Distribution

We've now mastered the distribution that governs sample variance. But in the real world, we almost never know the true population variance $\sigma^2$ .

So what happens when we try to standardize our sample mean ( $\bar{X}$ ) using our *estimate* of the standard deviation ( $s$ ) instead of the true value ( $\sigma$ )? The result is no longer a perfect Z-distribution. It follows a new, slightly wider distribution designed to account for this extra uncertainty.

In the next lesson, we will combine the Normal and the Chi-Squared to derive the workhorse of all statistical inference: the Student's t-Distribution.

Lesson 2.4: Capstone 1: The MVN in Action: Portfolio Theory

Lesson 2.6: The t-Distribution (Student's t)