Computing Skewness and Kurtosis in One Pass
Join the DZone community and get the full member experience.Join For Free
If you compute the standard deviation of a data set by directly implementing the definition, you’ll need to pass through the data twice: once to find the mean, then a second time to accumulate the squared differences from the mean. But there is an equivalent algorithm that requires only one pass and that is more accurate than the most direct method. You can find the code for implementing it here.
You can also compute the higher sample moments in one pass. I’ve extended the previous code to compute skewness and kurtosis in one pass as well.
The new code also lets you split your data, say to process it in parallel on different threads, and then combine the statistics, in the spirit of map-reduce.
Lastly, I’ve posted analogous code for simple linear regression.
Published at DZone with permission of John Cook, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Seven Steps To Deploy Kedro Pipelines on Amazon EMR
Authorization: Get It Done Right, Get It Done Early
Essential Architecture Framework: In the World of Overengineering, Being Essential Is the Answer
Health Check Response Format for HTTP APIs