Linear Regression Primer

Linear regression is the simplest useful model in machine learning. It contains most of the ideas that show up everywhere else in ML: a loss function, an optimizer, regularization, and the bias-variance trade-off. Understand linear regression well and a lot of other models start to look like variations on the same theme. The Model And The Intuition Loss Function Solving It: The Normal Equation Solving It: Gradient Descent Assumptions And Diagnostics Regularization Evaluation Metrics Robustness And Feature Handling Connections Quick Reference The Model And The Intuition Linear regression predicts a number from a set of inputs. The prediction is, \(\hat{y} = w^Tx + b\). \(w\) is a vector of weights, one per feature. \(b\) is the intercept. Each weight tells you how much the prediction changes when that feature goes up by one unit. Everything else is held fixed. Fitting the model means picking \(w\) and \(b\) so predictions land close to the real targets \(y\). “Close” is defined by the…