1. What is Regression Analysis?

Definition: Regression analysis is a statistical technique used to study the relationship between a dependent variable (response) and one or more independent variables (predictors).

• It helps in predicting the value of the dependent variable based on known values of independent variables.
• It also quantifies the strength and nature of relationships between variables.

2. Key Terms

3. Types of Regression

A. Simple Linear Regression

• Involves one independent variable and one dependent variable.
• Model: $$Y = β_0 + β_1 X + \varepsilon$$
• Goal: Estimate $\beta_0$ (intercept) and $\beta_1$ (slope) to best fit the data.

B. Multiple Linear Regression

• Involves two or more independent variables predicting a dependent variable.
• Model: $$Y = β_0 + β_1 X_1 + β_2 X_2 + \dots + β_k X_k + \varepsilon$$
• Allows for more accurate predictions and understanding of combined effects of variables.

4. Assumptions of Linear Regression

1. Linearity – Relationship between X and Y is linear.
2. Independence – Observations are independent.
3. Homoscedasticity – Constant variance of residuals.
4. Normality – Residuals are normally distributed.
5. No multicollinearity (for multiple regression) – Independent variables are not highly correlated.

5. Purpose of Regression Analysis

1. Prediction:

   • Predicting future outcomes based on independent variables.
   • Example: Predicting sales based on advertising expenditure.

2. Estimation:

   • Estimating the strength of relationships between variables.
   • Example: How much does temperature affect ice cream sales?

3. Hypothesis Testing:

   • Testing whether independent variables significantly influence the dependent variable.

6. Goodness of Fit

• R-squared ($R^2$): Proportion of variation in Y explained by X.
• Adjusted R-squared: Adjusted for the number of predictors, used in multiple regression.
• Standard Error of Estimate: Measures the average distance of observed values from the regression line.

7. Example: Simple Linear Regression

Suppose we want to predict students’ test scores (Y) based on hours studied (X):

• Sample data:

  • Hours studied: 2, 4, 6, 8
  • Scores: 50, 60, 65, 80

• Regression equation:

  $$\hat{Y} = 40 + 5X$$

• Interpretation:

  • Intercept (40): Predicted score if 0 hours studied.
  • Slope (5): Each additional hour of study increases score by 5 points.

8. Summary

• Regression analysis helps in modeling relationships, prediction, and decision-making.
• Simple regression → one predictor, multiple regression → several predictors.
• Assumptions must be checked to ensure validity of results.
• Key outputs: Regression coefficients, R-squared, residuals, significance of predictors.