From Zero to Hero: A Comprehensive Guide to Feature Engineering in Python scikit-learn

Feature engineering is the process of transforming raw data into features that better represent the underlying problem to the predictive model. It is a crucial step in the machine learning workflow, as the quality of the features can greatly affect the performance of the model. In this tutorial, we will learn how to do feature engineering in Python.

Before we begin, it is important to understand that feature engineering is an iterative process and requires a good understanding of the problem and the data. It is also important to keep in mind that there is no one-size-fits-all approach to feature engineering and that the methods used will depend on the specific problem and data at hand.

Importing libraries and loading the data

First, let’s start by importing the necessary libraries and loading the data. We will use the popular pandas library to load and manipulate the data, and the sklearn library to perform some preprocessing steps.

1
import pandas as pd
2
from sklearn import preprocessing
3

4
# Load the data
5
data = pd.read_csv("data.csv")

Exploratory Data Analysis (EDA)

Before we start transforming the data, it is important to perform some exploratory data analysis (EDA) to get a better understanding of the data. This includes analyzing the distribution of the features, identifying missing values, and checking for outliers.

Here are a few common techniques for EDA:

• data.info(): This function displays a summary of the data, including the data types of each column and the number of non-null values.
• data.describe(): This function computes basic statistics of the numerical columns, including the count, mean, standard deviation, minimum, and maximum value.
• data.plot(): This function plots the data using various plots, such as histograms, scatter plots, and box plots.

1
# Get information about the data
2
data.info()
3

4
# Get summary statistics of the numerical columns
5
data.describe()
6

7
# Plot the distribution of a numerical column
8
data["column_name"].plot.hist()

For more information about EDA, please check out my previous blog on “Exploratory Data Analysis (EDA) in python”

Handling missing values

Missing values can have a significant impact on the performance of a machine learning model. Therefore, it is important to handle missing values before training the model.

There are several ways to handle missing values, including:

• Deleting rows: This is usually not recommended as it can result in a loss of valuable information.
• Imputing missing values: This involves replacing the missing values with a substitute value, such as the mean, median, or mode of the column.

1
# Count the number of missing values in each column
2
data.isnull().sum()
3

4
# Impute missing values with the mean
5
data = data.fillna(data.mean())

For more information about dealing with missing values, please check out my previous blog on “How to do feature imputation in python scikit-learn”.

Encoding categorical features

Categorical features are features that take on a limited number of values. These features need to be encoded before they can be used in a machine learning model. There are several ways to encode categorical features, including:

• One-hot encoding: This method creates a new binary column for each unique category in the feature. For example, if the feature has three categories A, B, and C, three new columns will be created: one for A, one for B, and one for C.
• Label encoding: This method assigns a unique integer to each category in the feature. For example, if the feature has three categories A, B, and C, they will be encoded as 0, 1, and 2, respectively.

Here is an example of how to encode categorical features using one-hot encoding and label encoding in Python:

1
# One-hot encode the categorical column
2
one_hot = pd.get_dummies(data["column_name"])
3

4
# Add the one-hot encoded columns to the data
5
data = pd.concat([data, one_hot], axis=1)
6

7
# Label encode the categorical column
8
le = preprocessing.LabelEncoder()
9
data["column_name"] = le.fit_transform(data["column_name"])

For more information about dealing with feature encoding, please check out my previous blog on “How to do feature encoding in python scikit-learn”.

Normalizing numerical features

It is often a good idea to normalize numerical features, especially if they have different scales. Normalization can help the model converge faster and perform better.

There are several ways to normalize numerical features, including:

• Min-max scaling: This method scales the values between 0 and 1.
• Standardization: This method scales the values so that they have zero mean and unit variance.

Here is an example of how to normalize numerical features using min-max scaling and standardization in Python:

1
# Min-max scaling
2
min_max_scaler = preprocessing.MinMaxScaler()
3
data["column_name"] = min_max_scaler.fit_transform(data["column_name"].values.reshape(-1,1))
4

5
# Standardization
6
scaler = preprocessing.StandardScaler()
7
data["column_name"] = scaler.fit_transform(data["column_name"].values.reshape(-1,1))

For more information about dealing with feature normalization, please check out my previous blog on “How to do feature normalization in python scikit-learn”.

Creating new features

Creating new features, also known as feature engineering, can often improve the performance of the model. There are several ways to create new features, including:

• Combining features: This involves combining two or more features to create a new feature. For example, we can create a new feature that represents the sum of two numerical features.
• Transforming features: This involves applying a mathematical transformation to a feature to create a new feature. For example, we can apply the log transformation to a numerical feature to reduce the impact of outliers.

Here is an example of how to create new features in Python:

1
# Combine two features
2
data["new_feature"] = data["feature_1"] + data["feature_2"]
3

4
# Apply a transformation to a feature
5
data["new_feature"] = np.log(data["feature"])

Wrapping up

In this tutorial, we learned how to do feature engineering in Python. We saw how to perform EDA, handle missing values, encode categorical features, normalize numerical features, and create new features.

Remember, feature engineering is an iterative process and requires a good understanding of the problem and the data. It is important to try out different approaches and see what works best for your specific problem and data.