Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Analyzing Data With IP2Location and the ELK Stack

DZone's Guide to

Analyzing Data With IP2Location and the ELK Stack

Geolocation data is incredibly valuable. In this tutorial, learn how to use the IP2Location filter plugin with Elasticsearch, Logstash, and Kibana to analyze geolocation data.

· Big Data Zone
Free Resource

Access NoSQL and Big Data through SQL using standard drivers (ODBC, JDBC, ADO.NET). Free Download 

Geolocation data has become increasingly important over the past few years, regardless of whether you are a marketer, business owner, or developer. We understand that geolocation provides important data about generating sales from potential leads. It helps us learn more about our online visitors and customers, such as where are they coming from, their time zone, and so on.

In this article, we will guide you on how to use IP2Location filter plugin with Elasticsearch, Logstash, and Kibana to monitor your network traffic data.

To follow this tutorial, you must have a working ELK environment. You can install the ELK stack here.

Configure Filebeat for Logging With Logstash

  1. Download the suitable package for Filebeat and install it on your machine.
  2. Configure the filebeat.yml file that is located in your Filebeat root directory. Replace the content as follows:
    filebeat.prospectors:
    - input_type: log
      paths:
        - /path/to/log/file
    output.logstash:
      hosts: ["localhost:5043"]
  3. You can get the sample dataset that was used in this tutorial here.
  4. Unpack the file and make sure the paths field in the filebeat.yml is pointing correctly to the downloaded sample dataset log file.
  5. Save the filebeat.yml file.
  6. Run the command below on your machine:
    sudo ./filebeat -e -c filebeat.yml -d "publish"

Configure Logstash to Use IP2Location Filter Plugin

  1. Install the IP2Location filter plugin with:
    bin/logstash-plugin install logstash-filter-ip2location
  2. Create a Logstash configuration pipeline named test-pipeline.conf with the following content:
    input {
      beats {
    	port => "5043"
      }
    }
    
    filter {
      grok {
    	match => { "message" => "%{COMBINEDAPACHELOG}"}
      }
      ip2location {
    	source => "clientip"
      }
    }
    
    
    output {
      elasticsearch {
    	hosts => [ "localhost:9200" ]
      }
    }
  3. Save the test-pipeline.conf file.
  4. Run the command below at your machine to start Logstash:
    bin/logstash -f test-pipeline.conf --config.reload.automatic

Visualize Data in Kibana

After you have successfully installed Kibana on your machine, you can use Kibana to view the data by opening Kibana in a web browser.

Example obtained from IP2Location filter plugin with ELK using IP2Location DB24 BIN:

Image title

And that's it!

The fastest databases need the fastest drivers - learn how you can leverage CData Drivers for high performance NoSQL & Big Data Access.

Topics:
big data ,data analytics ,logging ,filebeat ,logstash ,elk ,kibana ,ip2location ,geolocation ,tutorial

Published at DZone with permission of camimi Morales. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}