Linear Regression is a statistical method to model the relationship between two numerical variables by fitting a linear equation to the observed data. This calibrated line serves as a predictive model to forecast future outcomes based on input features.

Key Components of Linear Regression

• Dependent Variable: Denoted as $Y$, the variable being predicted.
• Independent Variable(s): Denoted as $X$ or $\vec{x}$, the predictor variable(s).
• Coefficients: Denoted as $B_0$ (intercept) and $B_i$ (slopes), the parameters estimated by the model.
• Residuals: The gaps between predicted and observed values.

Core Attributes of the Model

• Linearity: The relationship between $X$ and $Y$ is linear.
• Independence: Each input feature $X_i$ is independent of one another.
• Homoscedasticity: Consistent variability in residuals along the entire range of predictors.
• Poor Performance in the Presence of Outliers: Sensitive to outliers during model training.

Computational Approach

• Ordinary Least Squares (OLS): Minimizes the sum of squared differences between observed and predicted values.
• Coordinate Descent: Iteratively adjusts coefficients to minimize a specified cost function.

Model Performance Metrics

• Coefficient of Determination ($R^2$): Measures the proportion of the variance in the dependent variable that is predictable from the independent variables.
• Root Mean Squared Error (RMSE): Provides the standard deviation of the residuals, effectively measuring the average size of the error in the forecasted values.

Practical Applications

• Sales Forecasting: To predict future sales based on advertising expenditures.
• Risk Assessment: For evaluating the potential for financial events, such as loan defaults.
• Resource Allocation: To determine optimal usage of resources regarding output predictions.