# Using a Chi-Square Test to Satisfy Benford's Law With Python

# Using a Chi-Square Test to Satisfy Benford's Law With Python

### Learn how to run a chi-square test in Python to analyze the distribution of factorials to satisfy Benford's Law, which is often used in fraud detection.

Join the DZone community and get the full member experience.

Join For FreeHortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

*Here's a fun quick excursion using a very short Python program that illuminates the somewhat counterintuitive basis for Benford's law. Quick, interesting and refreshing. We're all geeks here right?*

A while back I wrote about how the leading digits of factorials follow Benford’s law. That is, if you look at just the first digit of a sequence of factorials, they are not evenly distributed. Instead, 1’s are most popular, then 2’s, etc. Specifically, the proportion of factorials starting with *n* is roughly log_{10}(1 + 1/*n*).

Someone has proved that the limiting distribution of leading digits of factorials exactly satisfies Benford’s law. But if we didn’t know this, we might use a chi-square statistic to measure how well the empirical results match expectations. As I argued in the previous post, statistical tests don’t apply here, but they can be useful anyway in giving us a way to measure the size of the deviations from theory.

Benford’s law makes a better illustration of the chi-square test than the example of prime remainders because the bins are unevenly sized, which they’re allowed to be in general. In the prime remainder post, they were all the same size.

The original post on leading digits of factorial explains why we compute the leading digits the way we do. Only one detail has changed: The original post used Python 2 and this one uses Python 3. Integer division was the default in Python 2, but now in Python 3 we have to use `//`

to explicitly ask for integer division, floating point division being the new default.

Here’s a plot of the distribution of the leading digits for the first 500 factorials.

And here’s code to compute the chi-square statistic:

```
from math import factorial, log10
def leading_digit_int(n):
while n > 9:
n = n//10
return n
def chisq_stat(O, E):
return sum( [(o - e)**2/e for (o, e) in zip(O, E)] )
# Waste the 0th slot to make the code simpler.
digits = [0]*10
N = 500
for i in range(N):
digits[ leading_digit_int( factorial(i) ) ] += 1
expected = [ N*log10(1 + 1/n) for n in range(1, 10) ]
print( chisq_stat(digits[1:], expected) )
```

This gives a chi-square statistic of 7.693, very near the mean value of 8 for a chi-square distribution with eight degrees of freedom. (There are eight degrees of freedom, not nine, because if we know how many factorials start with the digits 1 through 8, we know how many start with 9.)

So the chi-square statistic suggests that the deviation from Benford’s law is just what we’d expect from random data following Benford’s law. And as we said before, this suggestion turns out to be correct.

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub. Join the discussion.

Published at DZone with permission of John Cook , DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

## {{ parent.title || parent.header.title}}

## {{ parent.tldr }}

## {{ parent.linkDescription }}

{{ parent.urlSource.name }}