19 Free Public Data Sets for Your Data Science Project

Completing your first project is a major milestone on the road to becoming a data scientist and helps to both reinforce your skills and provide something you can discuss during the interview process. It’s also an intimidating process. The first step is to find an appropriate, interesting data set. You should decide how large and how messy a data set you want to work with; while cleaning data is an integral part of data science, you may want to start with a clean data set for your first project so that you can focus on the analysis rather than on cleaning the data.

Based on the learnings from our courses, we’ve selected data sets of varying types and complexity that we think work well for first projects. (Some of them work for research projects as well!) These data sets cover a variety of sources: demographic data, economic data, text data, and corporate data.

You may also like: Practical Data Mining with Python.

Free Data Sets for Students

1. 
   
   1. United States Census Data: The U.S. Census Bureau publishes reams of demographic data at the state, city, and even zip code level. It is a fantastic data set for students interested in creating geographic data visualizations and can be accessed on the Census Bureau website. Alternatively, the data can be accessed via an API. One convenient way to use that API is through choroplethr. In general, this data is very clean and very comprehensive.
   2. FBI Crime Data: The FBI crime data is fascinating and one of the most interesting data sets on this list. If you’re interested in analyzing time series data, you can use it to chart changes in crime rates at the national level over a 20-year period. Alternatively, you can look at the data geographically.
   3. CDC Cause of Death: The Centers for Disease Control and Prevention maintains a database on cause of death. The data can be segmented in almost every way imaginable: age, race, year, and so on.
   4. Medicare Hospital Quality: The Centers for Medicare & Medicaid Services maintains a database on quality of care at more than 4,000 Medicare-certified hospitals across the U.S., providing for interesting comparisons.
   5. SEER Cancer Incidence: The U.S. government also has data about cancer incidence, again segmented by age, race, gender, year, and other factors. It comes from the National Cancer Institute’s Surveillance, Epidemiology, and End Results Program.
   6. Bureau of Labor Statistics: Many important economic indicators for the United States (like unemployment and inflation) can be found on the Bureau of Labor Statistics website. Most of the data can be segmented both by time and by geography.
   7. Bureau of Economic Analysis: The Bureau of Economic Analysis also has national and regional economic data, including gross domestic product and exchange rates.
   8. IMF Economic Data: For access to global financial statistics and other data, check out the International Monetary Fund’s website.
   9. Dow Jones Weekly Returns: Predicting stock prices is a major application of data analysis and machine learning. One relevant data set to explore is the weekly returns of the Dow Jones Index from the Center for Machine Learning and Intelligent Systems at the University of California, Irvine.
   10. Data.gov.uk: The British government’s official data portal offers access to tens of thousands of data sets on topics such as crime, education, transportation, and health.
   11. Enron Emails: After the collapse of Enron, a free data set of roughly 500,000 emails with message text and metadata were released. The data set is now famous and provides an excellent testing ground for text-related analysis. You also can explore other research uses of this data set through the page.
   12. Google Books Ngrams: If you’re interested in truly massive data, the Ngram viewer data set counts the frequency of words and phrases by year across a huge number of text sources. The resulting file is 2.2 TB.
   13. UNICEF: If data about the lives of children around the world is of interest, UNICEF is one of the most credible sources. The organization’s public data sets touch upon nutrition, immunization, and education, among others.
   14. Reddit Comments: Reddit released a really interesting data set of every comment that has ever been made on the site. It’s over a terabyte of data uncompressed, so if you want a smaller data set to work with Kaggle has hosted the comments from May 2015 on their site.
   15. Wikipedia: Wikipedia provides instructions for downloading the text of English-language articles, in addition to other projects from the Wikimedia Foundation.
   16. Lending Club: Lending Club provides data about loan applications it has rejected as well as the performance of loans that it has issued. The free data set lends itself both to categorization techniques (will a given loan default) as well as regressions (how much will be paid back on a given loan).
   17. Walmart: Walmart has released historical sales data for 45 stores located in different regions across the United States.
   18. Airbnb: Airbnb offers different data sets related to Airbnb listings in dozens of cities around the world.
   19. Yelp: Yelp maintains a free dataset for use in personal, educational, and academic purposes. It includes 6 million reviews spanning 189,000 businesses in 10 metropolitan areas. Students are welcome to participate in Yelp’s dataset challenge.

Further Reading

• Introduction to R: The Statistical Programming Language.
• The Best Python Libraries for Data Science and Machine Learning.
• TensorFlow in a Nutshell.