DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Securing Your Software Supply Chain with JFrog and Azure
Register Today

Trending

  • Superior Stream Processing: Apache Flink's Impact on Data Lakehouse Architecture
  • Mastering Time Series Analysis: Techniques, Models, and Strategies
  • Leveraging FastAPI for Building Secure and High-Performance Banking APIs
  • What to Pay Attention to as Automation Upends the Developer Experience

Trending

  • Superior Stream Processing: Apache Flink's Impact on Data Lakehouse Architecture
  • Mastering Time Series Analysis: Techniques, Models, and Strategies
  • Leveraging FastAPI for Building Secure and High-Performance Banking APIs
  • What to Pay Attention to as Automation Upends the Developer Experience

Kernel Density Estimation

Learn how to create an interactive Gaussian Kernel Density Estimation plot with Highcharts.

Mustapha Mekhatria user avatar by
Mustapha Mekhatria
CORE ·
Jul. 20, 20 · Tutorial
Like (3)
Save
Tweet
Share
4.59K Views

Join the DZone community and get the full member experience.

Join For Free

Kernel density estimation is a useful statistical method to estimate the overall shape of a random variable distribution. In other words, kernel density estimation, also known as KDE, helps us to “smooth” and explore data that doesn’t follow any typical probability density distribution, such as normal distribution, binomial distribution, etc. In this tutorial, we will show you how to create an interactive kernel density estimation in Javascript and plot the result using the Highcharts library. Let’s first explore the KDE plot; then we will dive into the code. The demo below displays a Gaussian kernel density estimate of a random dataset:

This chart helps us to estimate the probability distribution of our random data set, and we can see that the data are concentrated mainly at the beginning and at the end of the chart. Basically, for each data points in red, we plot a Gaussian kernel function in orange, then we sum all the kernel functions together to create the density estimate in blue (see demo):

By the way, there are many kernel function types such as Gaussian, Uniform, Epanechnikov, etc. The one we use is the Gaussian kernel, as it offers a smooth pattern. The mathematical representation of the Gaussian kernel is: Gaussian kernel Now, you have an idea about how the kernel density estimation looks like, let’s take a look at the code behind it. There are four main steps in the code:

  1. Create the Gaussian kernel function.
  2. Process the density estimate points.
  3. Process the kernel points.
  4. Plot the whole data points.

Gaussian Kernel Function

The following code represents the Gaussian kernel function:

Java
 




x


 
1
function GaussKDE(xi, x) {
2
  return (1 / Math.sqrt(2 * Math.PI)) * Math.exp(Math.pow(xi - x, 2) / -2);
3
}


Where x represents the main data (observation), and xi represents the range to plot the kernels and the density estimate function. In our case, the xi range is from 88 to 107 to be sure to cover the range of the observation data that is from 93 to 102.

Density Estimate Points

The following loop creates the density estimate points using the GaussKDE() function and the range represented by the array xiData:

Java
 




xxxxxxxxxx
1
11


 
1
//Create the density estimate
2
for (i = 0; i < xiData.length; i++) {
3
  let temp = 0;
4
  kernel.push([]);
5
  kernel[i].push(new Array(dataSource.length));
6
  for (j = 0; j < dataSource.length; j++) {
7
    temp = temp + GaussKDE(xiData[i], dataSource[j]);
8
    kernel[i][j] = GaussKDE(xiData[i], dataSource[j]);
9
  }
10
  data.push([xiData[i], (1 / N) * temp]);
11
}


Kernels Points

This step is required only if you would like to display the kernel points (orange charts); otherwise, you are already good with the density estimate step. Here is the code to process the data points for each kernel:

Java
 




xxxxxxxxxx
1


 
1
//Create the kernels
2
for (i = 0; i < dataSource.length; i++) {
3
  kernelChart.push([]);
4
  kernelChart[i].push(new Array(kernel.length));
5
  for (j = 0; j < kernel.length; j++) {
6
    kernelChart[i].push([xiData[j], (1 / N) * kernel[j][i]]);
7
  }
8
}
9

          


Basically, this loop is just about adding the range xiData to each kernel array that was already processed in the density estimate step.

Plot the Points

Once all the data points are processed, it is time to use Highcharts to render the series. The density estimate and the kernels are spline chart types, whereas the observations are plotted as a scatter plot:

Java
 




xxxxxxxxxx
1
55


 
1
Highcharts.chart("container", {
2
  chart: {
3
    type: "spline",
4
    animation: true
5
  },
6
  title: {
7
    text: "Gaussian Kernel Density Estimation (KDE)"
8
  },
9
  yAxis: {
10
    title: { text: null }
11
  },
12
  tooltip: {
13
    valueDecimals: 3
14
  },
15
  plotOptions: {
16
    series: {
17
      marker: {
18
        enabled: false
19
      },
20
      dashStyle: "shortdot",
21
      color: "#ff8d1e",
22
      pointStart: xiData[0],
23
      animation: {
24
        duration: animationTime
25
      }
26
    }
27
  },
28
  series: [
29
    {
30
      type: "scatter",
31
      name: "Observation",
32
      marker: {
33
        enabled: true,
34
        radius: 5,
35
        fillColor: "#ff1e1f"
36
      },
37
      data: dataPoint,
38
      tooltip: {
39
        headerFormat: "{series.name}:",
40
        pointFormat: "<b>{point.x}</b>"
41
      },
42
      zIndex: 9
43
    },
44
    {
45
      name: "KDE",
46
      dashStyle: "solid",
47
      lineWidth: 2,
48
      color: "#1E90FF",
49
      data: data
50
    },
51
    {
52
      name: "k(" + dataSource[0] + ")",
53
      data: kernelChart[0]
54
    },...  ]
55
});


Now, you are ready to explore your own data using the power of the Kernel density estimation plot. Feel free to share your comments or questions in the comment section below.

Kernel (operating system)

Opinions expressed by DZone contributors are their own.

Trending

  • Superior Stream Processing: Apache Flink's Impact on Data Lakehouse Architecture
  • Mastering Time Series Analysis: Techniques, Models, and Strategies
  • Leveraging FastAPI for Building Secure and High-Performance Banking APIs
  • What to Pay Attention to as Automation Upends the Developer Experience

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: