To prove that the sum of residuals in a simple linear regression model is equal to zero, we’ll go through the basic derivation.

Step 1: Setup and Definitions

In a linear regression model, the predicted values $\hat{y}_i$ are given by: $\hat{y}_i = a + b x_i$ where:

• $a$ is the intercept,
• $b$ is the slope,
• $x_i$ are the independent variable values,
• $y_i$ are the observed dependent variable values.

The residual for each observation $i$ is defined as: $e_i = y_i - \hat{y}_i$

Step 2: Expressing the Sum of Residuals

We want to show that the sum of residuals is zero: $\sum_{i=1}^n e_i = \sum_{i=1}^n (y_i - \hat{y}_i) = 0$

Step 3: Expand the Summation

Expanding the summation: $\sum_{i=1}^n (y_i - \hat{y}_i) = \sum_{i=1}^n y_i - \sum_{i=1}^n \hat{y}_i$

Since $\hat{y}_i = a + b x_i$, we can substitute to get: $\sum_{i=1}^n \hat{y}_i = \sum_{i=1}^n (a + b x_i) = \sum_{i=1}^n a + \sum_{i=1}^n b x_i$

Step 4: Simplify Each Term

Notice that $a$ and $b$ are constants: $\sum_{i=1}^n a = n \cdot a$ and $\sum_{i=1}^n b x_i = b \sum_{i=1}^n x_i$

Thus, we can rewrite $\sum_{i=1}^n \hat{y}_i$ as: $\sum_{i=1}^n \hat{y}_i = n \cdot a + b \sum_{i=1}^n x_i$

Step 5: Use the Least Squares Condition

One of the properties of the least squares estimates is that they make the sum of residuals zero. This property results from the normal equations derived in the least-squares estimation: $\sum_{i=1}^n y_i = n \cdot a + b \sum_{i=1}^n x_i$

Thus, $\sum_{i=1}^n y_i = \sum_{i=1}^n \hat{y}_i$

Substituting back, we find: $\sum_{i=1}^n (y_i - \hat{y}_i) = 0$

This proves that the sum of residuals in a linear regression model is equal to zero.

Would you like additional details or have questions on any part of this proof?

---

Here are five related questions for further exploration:

1. Why is the sum of residuals in polynomial regression not always zero?
2. How does this property of residuals affect the interpretation of the model's fit?
3. Can this proof be generalized to multiple linear regression?
4. How does the sum of squared residuals relate to this proof?
5. What role do residuals play in determining R-squared?

Tip: In least squares regression, minimizing the sum of squared residuals gives the best linear unbiased estimator, reinforcing this zero-sum property for residuals.