Goodness-of-Fit Testing in Python: Tips and Tricks for Data Scientists

Goodness-of-fit tests are statistical tests that are used to evaluate whether a sample comes from a population with a specific distribution. These tests compare the observed data to what is expected if the data follow a certain distribution.

In this tutorial, we will learn how to perform goodness-of-fit tests in Python using the scipy module.

Kolmogorov-Smirnov Test

The Kolmogorov-Smirnov (K-S) test is a nonparametric test that can be used to evaluate whether a sample comes from a population with a specific continuous distribution.

To perform the K-S test in Python, we can use the scipy.stats.kstest function from the scipy module.

1
from scipy.stats import kstest
2

3
# Sample data
4
sample = [0.5, 0.4, 0.35, 0.3, 0.25]
5

6
# Perform the K-S test
7
statistic, p_value = kstest(sample, 'norm')
8

9
print(statistic)
10
print(p_value)

The kstest function takes two arguments: the sample data and the name of the distribution to test against. In this case, we are testing whether the sample comes from a normal distribution.

The function returns the K-S statistic and the p-value. The K-S statistic represents the maximum difference between the observed cumulative distribution function and the expected cumulative distribution function for the given distribution. The p-value is the probability that the difference between the observed and expected cumulative distribution functions is due to chance.

If the p-value is below a certain threshold (usually 0.05), we can reject the null hypothesis that the sample comes from the specified distribution and conclude that there is a significant difference between the two.

Anderson-Darling Test

The Anderson-Darling test is another goodness-of-fit test that can be used to evaluate whether a sample comes from a population with a specific distribution. It is a more powerful test than the K-S test, but it is also more sensitive to deviations from normality.

To perform the Anderson-Darling test in Python, we can use the scipy.stats.anderson function from the scipy module.

1
from scipy.stats import anderson
2

3
# Sample data
4
sample = [0.5, 0.4, 0.35, 0.3, 0.25]
5

6
# Perform the Anderson-Darling test
7
result = anderson(sample)
8

9
print(result)

The anderson function returns a tuple containing the test statistic and the critical values for a given significance level. The critical values represent the boundaries for the test statistic beyond which the null hypothesis can be rejected.

For example, if the test statistic is greater than the critical value for a significance level of 0.05, we can reject the null hypothesis that the sample comes from a normally distributed population.

It’s important to note that the Anderson-Darling test is sensitive to the sample size, so it is recommended to use a sample size of at least 20 for the test to be reliable.

Chi-Square Test

The chi-square (χ²) test is a goodness-of-fit test that can be used to evaluate whether a sample comes from a population with a specific discrete distribution.

To perform the chi-square test in Python, we can use the scipy.stats.chisquare function from the scipy module.

1
from scipy.stats import chisquare
2

3
# Sample data
4
sample = [0.5, 0.4, 0.35, 0.3, 0.25]
5

6
# Expected frequencies
7
expected = [1, 1, 1, 1, 1]
8

9
# Perform the chi-square test
10
statistic, p_value = chisquare(sample, expected)
11

12
print(statistic)
13
print(p_value)

The chisquare function takes two arguments: the sample data and the expected frequencies. The function returns the chi-square statistic and the p-value.

The chi-square statistic represents the difference between the observed and expected frequencies. The p-value is the probability that the difference between the observed and expected frequencies is due to chance.

If the p-value is below a certain threshold (usually 0.05), we can reject the null hypothesis that the sample comes from the specified distribution and conclude that there is a significant difference between the two.

Conclusion

In this tutorial, we learned how to perform goodness-of-fit tests in Python using the scipy module. We saw examples of the Kolmogorov-Smirnov test, the Anderson-Darling test, and the chi-square test.

It’s important to choose the appropriate goodness-of-fit test based on the nature of the data and the distribution being tested. The K-S test is suitable for continuous data, the Anderson-Darling test is suitable for normally distributed data, and the chi-square test is suitable for discrete data.

References:

Scipy documentation: https://docs.scipy.org/doc/scipy/reference/

“Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach” by Anne Salib (2007)