Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Log Management With the ELK Stack on Windows Server — Part 2 — Installation

DZone's Guide to

Log Management With the ELK Stack on Windows Server — Part 2 — Installation

Learn how to install the components of the ELK Stack log management platform.

· Performance Zone ·
Free Resource

In Part 1, we spoke about the ELK Stack. In this post, we will go through the ELK Stack installation process on Windows Server 2012 R2 platform. We will go step-by-step through the installation process with a brief description of each step.

Let’s take a look at our checklist. What do we need?

  1. Windows Server 2012 R2
  2. Elasticsearch
  3. Logstash
  4. Kibana
  5. The Non-Sucking Service Manager (NSSM)
  6. Java (version 8)
  7. WinPcap

Also, for demonstration, we will need the following log shippers:

  1. Filebeat
  2. Packetbeat
  3. Winlogbeat

Before starting, let me share with you some notes:

  • I’ll be using a Windows 2012 R2 VM with 4 vCPUs, 8 Gb RAM. For testing, it will be comfortable if you have a minimum of 35–40 Gb free space.
  • I recommend you download the same version of all components (Elasticsearch, Logstash, Kibana, and beats). Of course, you can mix different version of particular components, but this may result in failure. Anyway, it is totally up to you.
  • I will use version 6.4.0, but you can use the latest release.
  • I also recommend unzipping all of the components (zips, including the beats agents) to a common local directory that will be home for the installs — For example: “C:\ELK\”. Again, it’s totally up to you, so if you don’t like name please feel free to change what you want.

Let’s get started.

After we download and unzip all components, you will probably have this look:

ELK Stack home directory

Step 1 — Installation of Java JDK

The first step we is installing the latest version of the Java JDK and creating the JAVA_HOME system variable.

1. Install the Java JDK and copy the installation directory (for me, it will be: “C:\Program Files\Java\jre1.8.0_181\”). In your case, it may be another number after “_” (underscore).

Installation of JDK

2. Open “System Properties” -> “Environment Variables” and create a new “System variable” named JAVA_HOME with a value of the path from the Java install.

Add new “System variable”


Step 2 — Installing Elasticsearch

1) To install Elasticsearch, open a Windows PowerShell prompt (Run as Administrator) and type the following commands:

Invoke-Expression -command “c:\ELK\elasticsearch\bin\elasticsearch service install”

If installation finishes successfully, PowerShell should display something similar to this:

Installing service: “elasticsearch-service-x64”
Using JAVA_HOME (64-bit): “C:\Program Files\Java\jdk1.8.0_181\”
The service ‘elasticsearch-service-x64’ has been installed.


2) After Elasticsearch is installed, open the Elasticsearch service properties with the below command, change the Startup type to Automatic, and start the service.

Invoke-Expression -command “c:\ELK\elasticsearch\bin\elasticsearch-service manager”

Image title

3) At last, let’s verify that the Elasticsearch is running. Open a browser (it may any browser, but let’s use Google Chrome) and go to http://127.0.0.1:9200. If you see something similar to the following, congrats! Elasticsearch works and we can move to the next step.

Elasticsearch

Step 3. Installing Logstash

1) Before installing Logstash, you will need to create a “logstash.json” configuration file in the bin folder inside an unzipped Logstash folder (in my case: C:\ELK\logstash\bin\).

For now, let’s create a file and fill the configuration details in the JSON file. You can copy the content from this link.

2) To install Logstash, open a Windows PowerShell prompt (Run as Administrator) and type the following commands. We will use NSSM (Non-Sucking Service Manager) to install it as a Windows Service:

a. Run the following command:

Invoke-Expression -command “c:\ELK\nssm\win64\nssm install Logstash”

b. In the Application Tab, set the following settings:

Path: C:\ELK\logstash\bin\logstash.bat
Startup directory: C:\ELK\logstash\bin
Arguments: -f C:\ELK\logstash\bin\logstash.bat

c. In the Details Tab, set the following:

Display Name: Logstash
Description: Logstash Service
Startup type: Automatic

d. In Dependencies Tab, set the following:
This service depends……:
    elasticsearch-service-x64

e. After successfully installing, run the following command:

Invoke-Expression –command “C:\ELK\logstash\bin\logstash-plugin install logstash-input-beats”

Next, I'll provide screenshots to better understand the above points:

Image title

Image title

Image title

At last, click “Install Service.” If you see << Service “Logstash” installed successfully! >>, congrats! Logstash works and we can move to the next step.

Step 4. Installing Kibana

To install Kibana, open a Windows PowerShell prompt (Run as Administrator) and type the following command. We will use again NSSM (Non-Sucking Service Manager) to install it as a Windows Service:

a. Run the following command:

Invoke-Expression -command “c:\ELK\nssm\win64\nssm install Kibana”

a. In the Application Tab, set the following settings:

Path: C:\ELK\kibana\bin\kibana.bat
Startup directory: C:\ELK\kibana\bin
Arguments: “In this case, we DIDN’T SET ANY ARGUMENT. Let it be empty”

b. In the Details Tab, set the following:

Display Name: Kibana
Description: Kibana Service
Startup type: Automatic

c. In the Dependencies Tab, set the following:

This service depends……:
    elasticsearch-service-x64
    Logstash

Note: Be careful, here we need to set 2 services as dependencies. Write each of them in a new line.

Next, I was providing screenshots, which help to better understand the above points:

Kibana installing process

Image title

At last, click “Install Service.” And if you see << Service “Kibana” installed successfully! >>, we successfully install Kibana and we can move to the next step.

Before moving to the next step, let’s verify the services are started. To do this, open “Run” (Windows + R) and type “services.msc.” In the open windows, find the following services and check the “Status” of each:

  • Elasticsearch
  • Logstash
  • Kibana

Open a browser and go to http://localhost:5601 (or http://127.0.0.1:5601) — you should see something similar to the following screenshot:

Kibana interface (Management)

Step 5. Installing Beats

Before starting the installation process, let me tell you that in this documentation, I didn’t explain deeply about beats. I will give some brief information about beats, but we will discuss this topic more deeply at another time.

What Are Beats?

The beats are a collection of lightweight (resource-efficient, no dependencies, small) and open-source log shippers installed on servers for data collection so that it can be indexed and analyzed via the ELK Stack.

  • Packetbeat — Packetbeat captures network traffic between servers and, as such, can be used for application and performance monitoring.
  • Filebeat — Filebeat, as its name implies, is used for collecting and shipping log files.
  • Winlogbeat — It is a beat designed specifically for collecting Windows Event logs. It can be used to analyze security events, updates installed, and so forth.

Now to install beats, open a Windows PowerShell prompt (Run as Administrator) and type the following commands:

a. Installing Filebeat:

PowerShell.exe -ExecutionPolicy UnRestricted -File C:\ELK\filebeat\.\install-service-filebeat.ps1

b. Installing WinPcap:

c:\ELK\WinPcap_4_1_3.exe

c. Installing Packetbeat:

PowerShell.exe -ExecutionPolicy UnRestricted -File C:\ELK\packetbeat\.\install-service-packetbeat.ps1

d. Installing Winlogbeat:

PowerShell.exe -ExecutionPolicy UnRestricted -File C:\ELK\winlogbeat\.\install-service-winlogbeat.ps1

At last, let’s verify the services are started. To do this, open “Run” (Windows + R) and type “services.msc.” In the open windows, find the following services and check “Status” each of them:

  • filebeat
  • packetbeat
  • winlogbeat

Step 6. Visualizing Beats

Now we will go back into Kibana and under “Management” -> “Kibana” -> “Index Patterns” configure the following index patterns:

  1. filebeat-*
  2. packetbeat-*
  3. winlogbeat-*

And cheers, if you see screens like below, let me congratulate you. You install ELK Stack successfully.

Management tab

And under “Discover,” we will see magic!

Image title

Conclusion

I hope, this post was useful for you. Please feel free to share with me your comments and feedback.

In Part 3, you can find the customization process of “Logstash” and visualization it in Kibana.

References:

Installing Elasticsearch, Logstash, and Kibana on Windows Server 2012 R2 2016 by Rob Willis (thanks to Rob Willis, his video tutorial and blog material helped me to better understand the whole process)

Topics:
elk stack ,tutorial ,performance ,log management ,data processing ,logging

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}