Linear Regression 101: A Python scikit-learn Tutorial

Linear regression is a statistical method used to model the relationship between a dependent variable and one or more independent variables. It is a commonly used technique in data science and is often used as a baseline model for comparison with more complex models. In this tutorial, we will learn how to perform linear regression in Python using the scikit-learn library.

Prerequisites

Before we start, make sure you have the following libraries installed:

• NumPy (http://www.numpy.org/)
• pandas (https://pandas.pydata.org/)
• scikit-learn (https://scikit-learn.org/stable/)

You can install these libraries using pip:

1
pip install numpy pandas scikit-learn

Step 1: Import libraries and load the data

First, we need to import the necessary libraries and load the data into a pandas DataFrame.

1
import pandas as pd
2
import numpy as np
3
from sklearn.linear_model import LinearRegression
4

5
# Load the data into a pandas DataFrame
6
df = pd.read_csv('data.csv')

Step 2: Prepare the data

Next, we need to prepare the data for modeling. This involves splitting the data into the independent variables (also known as the predictor variables) and the dependent variable.

1
# Split the data into the predictor variables and the dependent variable
2
X = df[['x1', 'x2', ...]] # predictor variables
3
y = df['y'] # dependent variable

It is also a good idea to split the data into training and test sets. This allows us to evaluate the performance of our model on unseen data.

1
from sklearn.model_selection import train_test_split
2

3
# Split the data into training and test sets
4
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

Step 3: Fit the model

Now, we can fit the linear regression model to the training data.

1
# Create a linear regression object
2
model = LinearRegression()
3

4
# Fit the model to the training data
5
model.fit(X_train, y_train)

Step 4: Predict and evaluate the model

Finally, we can use the model to make predictions on the test data and evaluate the performance of the model.

1
# Make predictions on the test data
2
y_pred = model.predict(X_test)
3

4
# Evaluate the performance of the model
5
from sklearn.metrics import mean_squared_error
6

7
mse = mean_squared_error(y_test, y_pred)
8
print('Mean Squared Error:', mse)

Step 5: Interpret the coefficients

Once the model is fit, we can access the coefficients (b1, b2, …) using the coef_ attribute of the model. These coefficients represent the strength and direction of the relationship between each independent variable and the dependent variable.

1
# Print the coefficients
2
print(model.coef_)

It is also possible to retrieve the intercept (b0) using the intercept_ attribute:

1
# Print the intercept
2
print(model.intercept_)

Step 6: Check the assumptions of the model

It is important to check that the assumptions of the linear regression model are met in order to ensure that the results are reliable. One of the key assumptions is that the errors, or residuals, are normally distributed and have a mean of zero. We can check this using a histogram of the residuals.

1
# Plot a histogram of the residuals
2
import seaborn as sns
3
sns.distplot(y_test - y_pred)

We can also check the assumptions using statistical tests such as the Durbin-Watson test or the Breusch-Pagan test. If the assumptions are not met, it may be necessary to transform the data or use a different model.

Step 7: Improve the model

There are several ways to improve the performance of the linear regression model. Some common techniques include:

• Adding more predictor variables
• Adding polynomial terms or interaction terms to account for nonlinear relationships
• Transforming the data (e.g. using a log transformation)
• Regularization (e.g. Lasso or Ridge regression) to prevent overfitting

By iteratively trying different techniques and evaluating the performance of the model, it is possible to improve the accuracy of the predictions.

Conclusion

In this tutorial, we learned how to perform linear regression in Python using the scikit-learn library. We covered the steps of preparing the data, fitting the model, making predictions, and evaluating the performance of the model. We also discussed how to interpret the coefficients and check the assumptions of the model, as well as how to improve the model. Linear regression is a powerful and widely used statistical tool for modeling and predicting the relationship between a dependent variable and one or more independent variables.