Correlations Made Easy

Prelude

In the connected world, the internet and social networks give an additional push to generate humongous data. There is no such great challenge to store all the data that was generated, and one can leverage the latest advancements in terms of storing data. The real challenge is to understand and identify the possibility to correlate this data and draw meaningful insights. In this article, we will check out in more detail correlations and how to identify the correlation coefficient between given variables.

Data Analytics

Data Analytics plays a key role in deep diving into data, identifying trends, discovering data patterns, and extracting value out of data. Here, I would like to give a real-time example instead of regular correlation examples (sales of umbrellas during rainstorms and sales of ice cream during summer). The two variables we consider here to correlate are:

1. Life Expectancy
2. Public Health Care Expenditure % of GDP

We are analyzing data sets from “Our World in Data” the “life expectancy” of individuals in a country is correlated to “Public Health Care Expenditure % of GDP” or not.

There are four different types of analytics, and we just try to understand them with the same example in simple statements.

1. Descriptive Analytics: “What happened?” 

• What is life expectancy in a specific continent or country? 
• How much government expenditure on healthcare as a percent of gross domestic product (GDP) is spent?

2. Diagnostic Analytics: “What could be the root cause?”

• Why the life expectancy in some countries is very low?
• Is there a correlation between life expectancy and government spending on public health?
• Identifying data patterns and correlations is key as part of Diagnostic Analytics. 

3. Predictive Analytics: “What might possibly happen in the future?”

• What will be the life expectancy of a specific country in the next two years?
• What percentage of the amount will be spent in the next five years of time?

4. Prescriptive Analytics: “What are the possible action items?”

• Whether to start a program globally that can help countries to focus on healthcare as a priority or a program to collaborate with NGOs to develop local health facilities.

Diagnostic Analytics — Correlation Analysis

Let’s talk about correlation analysis in a bit more detail on how it is used to identify the strength of correlation between two different variables.

The two different variables we are considering here:

1. Public Health Care Expenditure % of GDP (2019 — “Our World in Data”) — Independent Variable
2. Life Expectancy (2019 — “Our World in Data”) — Dependent Variable

We will identify if the dependent variable is changing as the value changes in independent variables.

The correlation coefficient (r) is calculated to identify the strength of the given two variables.

Correlation is identified based on the r value:

• r towards 1: Positive Correlation
• r towards -1: Negative Correlation
• r towards 0: No Correlation

Access the dataset here.

Sample data for a few countries and aggregation values for 52 countries:

We have all the required X and Y values available to quantify the correlation r value between the given two variables.

There are several different types to calculate the correlation coefficient (ex: Pearson, Rank, Intra-class, etc..).

In this article, we will use the Pearson Correlation coefficient. 

Pearson Correlation Coefficient Formulae

                                                    r =   0.61564

Result of r, which is towards 1 and says the given 2 variables have a positive correlation. In the same way, we can quantify the correlation between any given two variables. It’s not only about identifying correlations but visualizing them with proper storytelling plays a key role in decision-making. 

Let us see another example where we can visualize two different variables.

1. Life Expectancy
2. No. of people having access to safe water

Here, we not only quantify the correlation between having access to safe water and the life expectancy of the population in any country, but we also visualize those correlations. Visualizations were created using Tableau.

Visualize

Life Expectancy:

The complete interactive dashboard can be accessed here.

Correlation Analysis plays a vital role as part of Diagnostic Analysis. Pearson Correlation is widely used to identify and quantify the correlation between given two variables.