Logistic Regression 101: A Beginner’s Guide with Python

Logistic Regression is a popular statistical model that is often used for binary classification tasks. In this tutorial, we will learn how to implement Logistic Regression in Python using the scikit-learn library.

We will start by importing the necessary libraries and loading the data. The scikit-learn library provides several datasets that we can use for this tutorial. For this example, we will use the “Iris” dataset, which consists of 150 observations of iris flowers with four features: sepal length, sepal width, petal length, and petal width. The goal is to predict the species of the iris flower based on these features.

1
import pandas as pd
2
from sklearn.datasets import load_iris
3

4
# Load the Iris dataset
5
iris = load_iris()
6

7
# Create a DataFrame from the Iris data
8
df = pd.DataFrame(iris.data, columns=iris.feature_names)
9

10
# Add the target column to the DataFrame
11
df['target'] = iris.target
12

13
# Print the first 5 rows of the DataFrame
14
df.head()

The next step is to split the data into training and testing sets. We will use the training set to fit the model, and the testing set to evaluate its performance.

1
from sklearn.model_selection import train_test_split
2

3
# Split the data into a training set and a testing set
4
X = df.drop('target', axis=1)
5
y = df['target']
6
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)

Now that we have our data ready, we can fit a Logistic Regression model to the training data. We will use the LogisticRegression class from the sklearn.linear_model module.

1
from sklearn.linear_model import LogisticRegression
2

3
# Create a Logistic Regression model
4
model = LogisticRegression()
5

6
# Fit the model to the training data
7
model.fit(X_train, y_train)

After fitting the model to the training data, we can use it to make predictions on the testing data. We can do this using the predict method of the model.

1
# Make predictions on the testing data
2
y_pred = model.predict(X_test)

Finally, we can evaluate the performance of the model using various metrics such as accuracy, precision, and recall. The sklearn.metrics module provides several functions for computing these metrics.

1
from sklearn.metrics import accuracy_score, precision_score, recall_score
2

3
# Calculate the accuracy of the model
4
accuracy = accuracy_score(y_test, y_pred)
5

6
# Calculate the precision of the model
7
precision = precision_score(y_test, y_pred, average='micro')
8

9
# Calculate the recall of the model
10
recall = recall_score(y_test, y_pred, average='micro')
11

12
# Print the results
13
print("Accuracy:", accuracy)
14
print("Precision:", precision)
15
print("Recall:", recall)

That’s it! You now know how to implement Logistic Regression in Python using the scikit-learn library. However, this is just the basic implementation of Logistic Regression. There are several parameters and options that you can adjust to improve the performance of the model.

For example, you can adjust the regularization strength of the model using the C parameter. By default, the C parameter is set to 1.0, which means that the model is not regularized. You can increase the value of C to increase the strength of the regularization, or decrease it to decrease the strength.

1
# Create a Logistic Regression model with a stronger regularization
2
model = LogisticRegression(C=10.0)
3

4
# Fit the model to the training data
5
model.fit(X_train, y_train)

You can also adjust the solver algorithm that is used to fit the model. By default, the solver parameter is set to “lbfgs”, which is a good choice for small datasets. However, for larger datasets, you may want to use a different solver such as “sag” or “liblinear”.

1
# Create a Logistic Regression model with the "sag" solver
2
model = LogisticRegression(solver='sag')
3

4
# Fit the model to the training data
5
model.fit(X_train, y_train)

There are many other parameters and options that you can adjust to improve the performance of the model. For a complete list of parameters and options, you can refer to the documentation of the LogisticRegression class in the scikit-learn library.

In conclusion, Logistic Regression is a powerful and widely-used tool for binary classification tasks. It is relatively simple to implement and interpret, and is robust to noise and outliers. By adjusting the various parameters and options, you can improve the performance of the model and achieve better results on your classification tasks.