The importance of uncertainty sampling: A deep dive into active learning query strategies

Active learning is a powerful approach for improving the performance of machine learning models by selectively labeling the most informative samples. One of the key components of active learning is the query strategy, which determines which samples to label next. Among the most popular query strategies is uncertainty sampling, which selects samples where the model is least certain of its predictions.

Uncertainty sampling is based on the idea that samples where the model is uncertain are the most informative for improving its performance. By providing labels for these samples, the model can learn from its mistakes and improve its performance on similar samples in the future. Additionally, by focusing on the samples where the model is uncertain, active learning can make the most efficient use of the limited labeled data available.

There are several ways to measure uncertainty in active learning using uncertainty sampling such as least confidence, margin-based, entropy-based sampling. Each method measures the uncertainty of a model’s predictions in a different way and can be used in different scenarios.

Least confidence sampling is a method for measuring uncertainty in active learning that is based on the model’s confidence in its predictions. The idea behind this method is that the samples where the model is least confident of its predictions are the ones that are most likely to contain errors and thus are the most informative for improving the model’s performance.

Here is how least confidence sampling works in active learning:

1. The model makes predictions on a set of unlabeled samples.
2. For each sample, the model calculates the predicted class probability for the most likely class.
3. The sample with the lowest predicted class probability is selected for labeling.
4. The labeled sample is added to the training dataset, and the model is retrained.
5. The process is repeated, iteratively selecting and labeling the sample with the lowest predicted class probability until the desired number of labeled samples is reached.

Margin-based sampling is a method for measuring uncertainty in active learning that is based on the difference between the highest and second-highest predicted class probabilities. The idea behind this method is that the samples where the difference between the highest and second-highest predicted class probabilities is small (i.e., the margin is small) are the most informative for improving the model’s performance.

Here is how margin-based sampling works in active learning:

1. The model makes predictions on a set of unlabeled samples.
2. For each sample, the model calculates the class probabilities for all classes.
3. The difference between the highest and second-highest predicted class probabilities is calculated for each sample. This difference is known as the margin.
4. The sample with the smallest margin is selected for labeling.
5. The labeled sample is added to the training dataset, and the model is retrained.
6. The process is repeated, iteratively selecting and labeling the sample with the smallest margin until the desired number of labeled samples is reached.

Entropy-based sampling is a method for measuring uncertainty in active learning that is based on the concept of entropy from information theory. The idea behind this method is that the samples where the model’s predictions have the highest entropy (i.e., the most uncertainty) are the most informative for improving the model’s performance.

Here is how entropy-based sampling works in active learning:

1. The model makes predictions on a set of unlabeled samples.
2. For each sample, the model calculates the class probabilities for all classes.
3. The entropy of the class probabilities is calculated for each sample. The entropy of a set of class probabilities is a measure of the uncertainty of the predictions.
4. The sample with the highest entropy is selected for labeling.
5. The labeled sample is added to the training dataset, and the model is retrained.
6. The process is repeated, iteratively selecting and labeling the sample with the highest entropy until the desired number of labeled samples is reached.

Entropy is a measure of the uncertainty or randomness of a set of class probabilities. To calculate the entropy of the class probabilities for each sample in active learning using entropy-based sampling, you can use the following formula:

Entropy = — Σ(p_i * log2(p_i))

where p_i is the class probability for class i and Σ represents the sum over all classes.

Here is a step-by-step process for calculating the entropy of the class probabilities for each sample:

1. For each sample, the model makes predictions and returns the class probabilities for all classes.
2. For each class probability, p_i, calculate p_i * log2(p_i).
3. Sum up the results from step 2 for all classes.
4. The resulting value is the entropy of the class probabilities for that sample.

It’s important to note that entropy is a non-negative value that ranges between 0 and log2(n), where n is the number of classes. The entropy is 0 when all class probabilities are 1 (i.e., the model is completely certain of its prediction) and log2(n) when all class probabilities are equal (i.e., the model is completely uncertain of its prediction).

It’s also worth noting that if the model returns a probability of 0 for any class, the log2(p_i) will be undefined, in this case, it’s common to add a small value known as Laplace smoothing to avoid this.

Choosing the right uncertainty sampling method can be crucial for the performance of the active learning system. Each method has its own strengths and weaknesses, and the best method will depend on the specific dataset and model being used. Therefore, it is always a good idea to experiment with multiple methods to find the one that works best for a specific application.

In conclusion, uncertainty sampling is an important query strategy in active learning as it allows to select samples where the model is uncertain and therefore the most informative for improving the model’s performance. Different uncertainty sampling methods are available in modAL and the choice of the method should be based on the specific dataset and model. Experimenting with different methods can help to find the best strategy for the specific application. Feel free to check out my previous blog post on how to do active learning with modAL python package!