Training the Baseline Linear Model: A Practical Guide

Previously in this course, we covered the mechanics of linear regression and the importance of training and testing data splits. Now that your data is cleaned and partitioned, it's time to build your first baseline model.

Establishing a baseline is the most critical step in any machine learning project. It provides a "floor" for performance—a simple, interpretable model against which you can measure the effectiveness of more complex techniques.

Instantiating and Fitting Your First Model

In Scikit-Learn, the workflow follows a consistent API: you instantiate an estimator object, call .fit() to train it on your data, and call .predict() to generate outputs. When you build Scikit-Learn pipelines, this process becomes even more robust because the pipeline handles the transformation steps automatically.

The Baseline Linear Model in Practice

For our running project, we will use a LinearRegression model. Since we have already preprocessed our features—handling missing data and feature selection—we can feed our training set directly into the pipeline.

PYTHON
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import Pipeline

# 1. Instantiate the model
model = LinearRegression()

# 2. Create the pipeline(assuming you have a preprocessor defined)
# If you haven't defined a preprocessor yet, use a simple identity or 
# just the model itself for the absolute baseline.
baseline_pipeline = Pipeline([
    (CE9178">'regressor', model)
])

# 3. Fit the model
# X_train and y_train come from your previous train-test split step
baseline_pipeline.fit(X_train, y_train)

print("Model training complete.")

Generating Initial Predictions

Once the model is fitted, it has "learned" the coefficients (weights) that minimize the error on your training data. To see how it performs on unseen data, we pass the test set to the .predict() method.

PYTHON
# 4. Generate predictions on the test set
y_pred = baseline_pipeline.predict(X_test)

# Compare the first 5 predictions to actual values
import pandas as pd
comparison = pd.DataFrame({CE9178">'Actual': y_test, CE9178">'Predicted': y_pred})
print(comparison.head())

These initial predictions are your first real look at how well your features capture the underlying patterns in the target variable.

Hands-on Exercise: Run Your Baseline

Using the dataset you cleaned in the project dataset initialization lesson:

1. Import LinearRegression from sklearn.linear_model.
2. Instantiate the model and wrap it in a Pipeline.
3. Fit the pipeline using your X_train and y_train variables.
4. Generate predictions for X_test and store them in a variable called y_pred.
5. Calculate the difference (residuals) between y_test and y_pred.

Common Pitfalls

• Data Leakage: Ensure your training set does not contain information from the future or the test set. If you use a pipeline, ensure that any scaling or imputation is fitted only on the training set.
• Dimensionality Mismatch: Always double-check the shape of your input arrays. Scikit-Learn expects X to be a 2D array (samples, features) and y to be a 1D array (samples).
• Ignoring the Baseline: Don't be tempted to jump straight into complex models like Gradient Boosting or Neural Networks. If your complex model performs similarly to your simple linear baseline, you've likely over-engineered the solution.

Recap

In this lesson, we transitioned from theory to application by instantiating a LinearRegression model, fitting it via a pipeline, and generating predictions on the test set. This baseline acts as your primary performance metric. By establishing this foundation, you now have a clear target to beat as you experiment with feature engineering and more advanced algorithms in the coming lessons.

Up next: We will examine the gap between your training results and test results to discuss training error vs generalization error.