Linear Regression: Your First Model

Linear regression is the simplest model and the best first one. It draws a line that best fits your data and uses that line to predict new points. For housing, it might learn that each bedroom adds $30k and each sqft adds $150.

You will fit it in three lines with scikit-learn. The model learns coefficients during fit, and you check them after. If a coefficient is huge or negative when it should not be, your features need scaling or you have leakage.

Do not be fooled by high training accuracy. Always test on data the model has never seen. A line that perfectly fits training noise will fail on new houses.

Fit and predict

Split first, fit on train, predict on test, then check error. Use mean squared error or mean absolute error to see how far off you are.

from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error
model = LinearRegression()
model.fit(X_train, y_train)
pred = model.predict(X_test)
print(mean_absolute_error(y_test, pred))

Overfitting with a line

Even a simple line can overfit if you add too many features like polynomial expansions without validation. Start simple, add complexity only if test error drops.

print(model.coef_)  # check coefficients
print(model.intercept_)

Syntax

from sklearn.linear_model import LinearRegression
model = LinearRegression()
model.fit(X_train, y_train)
model.predict([[3, 1500]])  # 3 beds, 1500 sqft

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)