Deciphering Data to Uncover Hidden Insights: Understanding the Data
In this article, we will walk you through the process of deciphering data in order to uncover hidden insights from their data.
Join the DZone community and get the full member experience.Join For Free
" The best vision is insight" – Malcolm Forbes.
By Ranjith Udayakumar, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud's incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.
When it comes to data analytics for enterprises, nothing is more important than making accurate and reliable inferences from data. It is no surprise that enterprises are investing heavily in big data analytics as they can reap large profits with accurate insights. However, this is often easier said than done. Data collected from real-world applications is affected by many variables, making data prediction challenging. Regardless, data analytics remains essential for many, if not all, businesses around the world.
In this article, I will walk you through the process of deciphering data for uncovering hidden insights.
Is This Article for Me?
This article is meant for everyone! This includes students who just want to familiarize themselves with general concepts, professional data analysts who want to learn new ways to analyze data, and business decision makers who want to know how to get better insights from business data.
This article covers the overall process of deciphering data from conceptual, practical, and best practice perspectives. Anyone with valid data can use this article as a guide to get insights from data with the help of open-source technologies. However, in this tutorial, I'll be using Alibaba Cloud QuickBI.
To use Alibaba Cloud QuickBI, you need to do the following:
- Create Account in Alibaba Cloud.
- Add a valid Payment Method to your account.
- Enroll yourself for a free trial of QuickBI Pro in your console.
Overview of the Article
For this article, we are going to be looking at:
- Domain - BFSI (Banking, Financial Services, and Insurance)
- Modules - From Understanding Data to Visual Stories
- Use cases - ATM Analytics, Customer 360
We will be covering the entire process of deciphering data. The overall process involves:
- Understanding the data
- Wrangling the data according to your business scenario (if needed)
- Ingesting the data
- Modeling the data
- Visualizing the data
Understanding the Data (Conceptual)
When it comes to big data, more data isn't necessarily better. Your data is only as good as your ability to understand and communicate it, which is why understanding the data is so essential.
Once you've got your data, you need to consider the following problems:
- What do you do with it?
- What should you look for?
- Which tools should you use?
You will need to address these questions for your data analysis to be effective. We will provide some generalized answers for the above questions in this article.
What Do You Do With It?
We should analyze the data to understand the domain it belongs to. With the domain in mind we should ask the right questions against the data to get insights out of it. For example, if the data shows ATM location details, transaction types, the number of transactions, and transaction amounts, it clearly shows that the data belongs to the BFSI domain.
After we determine the domain, it's now our turn to decide what type of insights we can infer out of it from the given data. We will do this in our practical section.
What Should You Look For?
We should look for some "interesting" insights. As we discussed earlier, we need to ask the right questions against the data to understand it better and decipher insights.
For example, let's assume you have some understanding of the BFSI domain. Then, we should able to differentiate the Facts (measures) and Dimensions (other than measures) from the data to get a clear idea about the data.
It's now our turn to understand the facts and dimensions available, and the right questions that we need to ask about the given data. We can do this in our practical section.
What Tools Should You Use?
We need to choose the right tool to wrangle, process, and visualize the data effectively. There are a lot of tools available in the market, all of them with their own unique strengths.
When deploying on the cloud, I prefer using Alibaba Cloud Quick BI, which easily covers the majority of tasks needed at an affordable price.
- Quick BI allows you to perform data analytics, exploration, and reporting on mass data with drag-and-drop features and a rich variety of visuals.
- Quick BI enables users to perform data analytics, exploration, and reporting and empowers enterprise users to view and explore data and make informed, data-driven decisions.
In this article, we are going to utilize Alibaba Cloud QuickBI as a tool to decipher the data to get insights out of it. We will explore how to do this in our practical section.
Understanding the Data (Practical)
As we discussed earlier, we are going to understand the data better with real use cases.
Use Case 1: ATM Analytics
Here we will use the data from an ATM Dataset.
What Do You Do With This Data?
As mentioned previously, we know that this data belongs to the BFSI domain. Specifically, this data talks about ATM transactions. Now, before digging deeper, we need to understand the domain basics and how the business users will see it proceed with the next question.
What Should You Look For?
As we discussed earlier, we need to ask the right questions to understand the data better. We need to differentiate the Facts (measures) and Dimensions (other than the measures).
The Facts include:
The Dimensions include:
After separating the facts and dimensions, we can now ask questions about the data. Questions may include:
- Total number of transactions.
- Total transaction amount.
- Top 5 ATMs by transaction volume.
- Top 5 ATMs by transaction amount.
- Lowest 5 ATMs by transaction volume.
- Lowest 5 ATMs by transaction amount.
- Number of different transactions by ATM.
These questions are key to deriving insights from the data. Without the right questions, we can't derive the value we need from the data.
Use Case 2: Customer 360
Here we will use the data from Customer360.
What Do You Do With It?
Similar to the previous use case, we know the data belongs to the BFSI domain, specifically on bank customer details. Now before digging deeper, we need to understand the domain basics and how the business users will see it to proceed with next question.
What Should You Look For?
Similarly, we need to differentiate the Facts (measures) and Dimensions (other than the measures).
The Facts are:
The Dimensions are:
- Marital status
After separating the facts and dimensions, we can ask questions such as:
- Balance by job
- Balance by marital status
- Loan by age
- Loan by job
These questions are key to deriving insights from the data. Let's now look at the best practices of understanding data.
Understanding the Data (Best Practices)
Here are some of the best practices when trying to make sense out of data, particularly data relating to the two use cases above.
- Determine the appropriate domain, and understand the domain basics.
- Always ask the right questions about the data.
- Which ATMs fall under the Transaction Volume Benchmark?
- Which ATMs fall under the Transaction Amount Benchmark?
- Which ATMs fall under the Hit Rate Benchmark?
- Which ATMs perform well irrespective of External Influences?
- Top Violators.
- Income or Profitability of ATMs.
- Have a clear understanding of Facts and Dimensions.
- Name the columns meaningfully.
- "Job" as "Job Category"
- "Marital" as "Marital Status"
- "pdays" as "Previous Days"
- "poutcome" as "Previous Outcome"
- Name the columns in sentence case and always use space instead of underscore
- "Job_Category" as "Job Category"
I hope that this article gives you a better grasp of the basic principles of data analytics, specifically on understanding your data.
In the next article of this series, we will be exploring how to wrangle the data. Stay tuned!
Reference: Originally published on Alibaba Cloud
Published at DZone with permission of Ranjith Udayakumar, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.