Central Limit Theorem (CLT)

The Central Limit Theorem is a fundamental result in probability and statistics that describes the behavior of the sum (or average) of a large number of independent and identically distributed (i.i.d.) random variables. It states:

• If we take a large enough sample size (n) from a population with any distribution (with finite mean $\mu$ and variance $\sigma^2$), the sampling distribution of the sample mean $\bar{X}$ will approach a normal distribution, regardless of the population's original distribution.**

Mathematical Formulation:

1. Suppose $X_1, X_2, ..., X_n$ are i.i.d. random variables with:

   • Mean $\mu$,
   • Variance $\sigma^2$.

2. Let the sample mean $\bar{X}$ be: $\bar{X} = \frac{1}{n} \sum_{i=1}^{n} X_i$

3. As $n \to \infty$, the distribution of $\bar{X}$ converges to: $\bar{X} \sim \mathcal{N} \left( \mu, \frac{\sigma^2}{n} \right)$

Key Implications:

• The larger the sample size $n$, the closer the sample mean will be to the normal distribution.
• This applies to the sum $S_n = \sum_{i=1}^n X_i$, which will also follow a normal distribution with mean $n\mu$ and variance $n\sigma^2$.

---

Binomial Approximation to Normal Distribution

The binomial approximation to the normal distribution is a practical application of the CLT. It allows us to approximate a binomial distribution $\text{Bin}(n, p)$ using a normal distribution when $n$ (number of trials) is large and $p$ (success probability) is not too close to 0 or 1.

Conditions for Approximation:

1. $n$ is large.
2. $np \geq 5$ and $n(1-p) \geq 5$ (ensuring sufficient symmetry in the binomial distribution).

Approximation:

The binomial random variable $X \sim \text{Bin}(n, p)$ can be approximated by: $X \sim \mathcal{N} \left( \mu, \sigma^2 \right)$ where:

• $\mu = np$ (mean of the binomial distribution),
• $\sigma^2 = np(1-p)$ (variance of the binomial distribution).

Continuity Correction:

• Since the binomial distribution is discrete and the normal distribution is continuous, we apply a continuity correction by adjusting for the width of each binomial "step": $P(a \leq X \leq b) \approx P(a - 0.5 \leq Z \leq b + 0.5)$ where $Z$ is the standard normal random variable.

---

Practical Example:

1. Central Limit Theorem:

   • Suppose the population of test scores has a mean $\mu = 70$ and standard deviation $\sigma = 10$. If we randomly sample $n = 30$ scores, the sampling distribution of the mean $\bar{X}$ will be approximately $\mathcal{N}(70, \frac{10^2}{30})$.

2. Binomial Approximation:

   • Let $X \sim \text{Bin}(n=100, p=0.6)$. Here, $\mu = np = 60$, $\sigma^2 = np(1-p) = 24$, and $\sigma = \sqrt{24}$. For large $n$, $X$ can be approximated by $\mathcal{N}(60, 24)$.

---

Do you have further questions, or want additional details on these concepts?

---

5 Related Questions:

1. How does the Central Limit Theorem apply to non-i.i.d. variables?
2. What are practical limitations of the binomial-to-normal approximation?
3. How is the continuity correction applied in hypothesis testing?
4. How can the Central Limit Theorem be used in confidence interval estimation?
5. How do we determine "large enough $n$" for the CLT to apply?

Tip:

When using the binomial-to-normal approximation, always check the $np \geq 5$ and $n(1-p) \geq 5$ condition for validity!