Lesson 2.6: The t-Distribution (Student's t)

We now introduce the workhorse of all statistical inference. The t-distribution is what we use when we want to test hypotheses but don't know the true population variance (which is always). We'll derive its structure from the Normal and Chi-Squared distributions and understand why its 'fatter tails' are the key to honest statistical testing with real-world data.

Part 1: The Problem with the Real World

In a perfect theoretical world, we could test a hypothesis about a sample mean $\bar{X}$ using the Z-statistic we know and love:

Z = \frac{\bar{X} - \mu}{\sigma / \sqrt{n}} \sim \mathcal{N}(0, 1)

But this formula has a fatal flaw for practical use: it requires $\sigma$ , the true population standard deviation. In 99.9% of real-world scenarios, from analyzing stock returns to medical trials, $\sigma$ is unknown.

The Core Idea: The t-distribution is what you get when you build a Z-statistic but are forced to use the sample standard deviation ( $s$ ) as a plug-in estimate for the true population standard deviation ( $\sigma$ ).

This simple substitution changes everything. The new statistic no longer follows a perfect Normal distribution.

Definition: The Student's t-statistic

t = \frac{\bar{X} - \mu}{s / \sqrt{n}} \sim t_{n-1}

This new statistic follows a t-distribution with $n-1$ degrees of freedom.

Part 2: Understanding the t-Distribution

The Z-Distribution's Humble Cousin

The t-distribution looks just like the standard Normal, but with one crucial difference.

Imagine a plot showing a standard Normal curve (Z) in blue. Overlaid in red is a t-distribution curve with low df (e.g., 5 df), which is slightly shorter at the peak and visibly fatter in both tails.

The 'Uncertainty Tax': Fatter Tails

Using an estimate $s$ instead of the true value $\sigma$ introduces extra uncertainty into our calculation. The t-distribution accounts for this by having **fatter tails** than the Normal distribution.

Think of this as an "uncertainty tax": for the convenience of using an estimate, we have to be more conservative. The fatter tails mean that more extreme values are more likely, so we'll need stronger evidence (a larger t-statistic) to reject a null hypothesis.

The Role of Degrees of Freedom (

\nu

)

The "fatness" of the tails is controlled by the degrees of freedom ( $\nu = n-1$ ).

Low df (small sample): Our estimate $s$ is unreliable. The uncertainty tax is high, and the tails are very fat.
High df (large sample): Our estimate $s$ becomes very accurate. The uncertainty tax shrinks, and the t-distribution converges to become identical to the standard Normal Z-distribution. (Generally, for $\nu > 120$ , they are practically the same).

Part 3: The Formal Derivation

The t-distribution is a beautiful composite of the two distributions we just learned.

Formal Definition: The t-Distribution

If $Z \sim \mathcal{N}(0, 1)$ and $V \sim \chi^2_{\nu}$ are independent, then the variable $T$ defined below follows a t-distribution with $\nu$ degrees of freedom:

T = \frac{Z}{\sqrt{V / \nu}} \sim t_{\nu}

Proof: How this definition creates our t-statistic

We need to show that $(\bar{X} - \mu) / (s/\sqrt{n})$ fits this structure.

Step 1: Identify the Z component (the numerator).

Z = \frac{\bar{X} - \mu}{\sigma / \sqrt{n}} \sim \mathcal{N}(0, 1)

Step 2: Identify the V component (related to the denominator). From the previous lesson, we know:

V = \frac{(n-1)s^2}{\sigma^2} \sim \chi^2_{n-1}

Here, the degrees of freedom $\nu = n-1$ .

Step 3: Construct the ratio $\sqrt{V/\nu}$ .

\sqrt{\frac{V}{\nu}} = \sqrt{\frac{(n-1)s^2 / \sigma^2}{n-1}} = \sqrt{\frac{s^2}{\sigma^2}} = \frac{s}{\sigma}

Step 4: Assemble the final T statistic.

T = \frac{Z}{\sqrt{V/\nu}} = \frac{ (\bar{X} - \mu) / (\sigma/\sqrt{n}) }{ s/\sigma }

The unknown $\sigma$ terms cancel out perfectly, leaving:

T = \frac{\bar{X} - \mu}{s / \sqrt{n}}

This proves that our practical statistic has the exact structure of a formal t-distribution.

The Payoff: The Workhorse of Inference

The t-test is arguably the most widely used statistical test in the world.

Econometrics & Finance: Every time you look at a regression output, the t-statistic and p-value for each coefficient ( $\hat{\beta}_j$ ) are calculated using the t-distribution. It tells you if a factor is a statistically significant predictor of the outcome.
A/B Testing: When comparing the means of two groups (e.g., click-through rate of website version A vs. version B), the two-sample t-test is the standard method for determining if the observed difference is real or just due to random chance.

What's Next? Testing the Whole Model

The t-test is perfect for checking the significance of a *single* variable. But how do we test the significance of our *entire regression model* at once? How do we test if a *group* of variables is jointly significant?

For that, we need a new tool. We need to compare the variance explained by our model to the residual variance. This requires taking a ratio of two Chi-Squared variables, which leads us to the final member of the sampling family: the F-Distribution.

Lesson 2.5: The χ² (Chi-Squared) Distribution

Lesson 2.7: The F-Distribution (Fisher-Snedecor)