Install Elasticsearch on Ubuntu 18.04.1

In this post, we will Install Elasticsearch on Ubuntu 18.04.1 including Logstash and Kibana. Elasticsearch lets you search and visualize your data.

Prepare

We will begin by starting with a fresh installation of Ubuntu Server 18.04.1 and running all the updates.

I created a virtual machine with 8 vCPUs, 4 GB of memory, and 200 GB of drive space. I also set up an A record on my internal DNS that set the logging host to 192.168.1.15.

If you would like to know how to install Ubuntu Server 18.04.1 I have a video available on my YouTube Channel: AdminTome Blog TV - Installing Ubuntu Server 18.04.1 Tutorial Video

During the install, I set the hostname as logging.admintome.lab and set the static IP to the IP mentioned above.

After Ubuntu Server is finished installing, we need to install all the updates.

apt update && apt upgrade -y

When it is completed updating, go ahead and reboot the system.

reboot

Elasticsearch requires Java SDK 8 to be installed.

Run these commands to install the Java 8 SDK.

add-apt-repository ppa:webupd8team/java
apt install -y oracle-java8-set-default

You can verify that we have Java 8 installed by running this command:

# java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

We are now ready to download and install Elasticsearch.

Install Elasticsearch

We are going to install the latest version of Elasticsearch which as of this writing is 6.3.2.

If you want to check for the latest version go to the Elastic Downloads page.

We will want to download the .DEB package.

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.2.deb

When it is done downloading, install it using DPKG:

dpkg -i elasticsearch-6.3.2.deb

After it is done installing, open /etc/elasticsearch/elasticsearch.yml and edit this line:

# network.host: 192.168.0.1

Uncomment it and set the IP to your server.

network.host: 192.168.1.15

Save and exit the file.

Finally, start and enable the Elasticsearch service.

systemctl enable elasticsearch.service
systemctl start elasticsearch.service

Verify that everything is working by browsing to this url:

http://<your-ip>:9200/_cat/health?v

You should see a page similar to this.

We can see that our node status is green, which is great.

Next we will install Kibana which will give us a graphical font-end.

Install Kibana

We will download and install Kibana exactly the same way we installed Elasticsearch.

wget https://artifacts.elastic.co/downloads/kibana/kibana-6.3.2-amd64.deb
dpkg -i kibana-6.3.2-amd64.deb

Next, open /etc/kibana/kibana.yml and update these two lines. Uncommenting as necessary.

server.host: "192.168.1.15"
elasticsearch.url: "http://192.168.1.15:9200"

Save and exit.

Configure the VM Heap Size for JVMs. We only have to do this now so we don't have to reboot.

After a reboot, this setting will be configured for us already.

sysctl -w vm.max_map_count=262144

Finally, start and enable the Kibana service.

systemctl enable kibana.service
systemctl start kibana.service

Browse to this URL:

http://<your-ip>:5601

And you will see the Kibana Dashboard.

Our last step is to install Logstash.

Install Logstash

Logstash is the agent that will put our data into Elasticsearch and displayed using Kibana.

As with the other applications, we will download and install Logstash in the same manner.

wget https://artifacts.elastic.co/downloads/logstash/logstash-6.3.2.deb
dpkg -i logstash-6.3.2.deb

Open the vim /etc/logstash/logstash.yml file and change this setting:

http.host: "192.168.1.15"

Save the file and exit.

Start and enable the Logstash service.

systemctl enable logstash.service
systemctl start logstash.service

Logstash is now installed. Next we will put some data into Elasticsearch using FileBeat.

Installing FileBeats

First, let's change to the logstash installation directory.

cd /usr/share/logstash/

Run the following command to install the FileBeat Logstash plugin:

bin/logstash-plugin install logstash-input-beats

Now we will configure Logstash to receive FileBeat data.

Create /etc/logstash/conf.d/beats.conf and add the following contents.

input {
  beats {
    port => "5044"
  }
}
output {
  elasticsearch {
    hosts => ["192.168.1.15:9200"]
    manage_template => false
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
  }
}

Be sure to set your actual IP address in the hosts line.

Save and exit the file.

Sending Logs

For this tutorial, we are going to configure the Elasticsearch server to send it's own logs to demonstrate how to configure your systems to send logs to the server.

If you want to configure sending logs from another server, follow these steps on the other server.

Download and install the FileBeat agent just like we have done for the other applications.

wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.3.2-amd64.deb
dpkg -i filebeat-6.3.2-amd64.deb

Next, we need to configure it.

Open up /etc/filebeat/filebeat.yml and configure it to look like this:

filebeat.prospectors:
- input_type: log
  paths:
    - /var/log/*/*.log

output.logstash:
  hosts: ["192.168.1.15:5044"]

This will configure FileBeat to send logs from /var/log/ to our Elasticsearch server on port 5044 (the port we configured in the last section).

Start and enable the FileBeat service.

systemctl enable filebeat.service
systemctl start filebeat.service

Finally, we need to install the FileBeat template to Kibana. You only have to do this once if you are setting up logging for multiple servers using FileBeat.

filebeat setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'

Wait a couple minutes for the logs to end up in Elasticsearch then go back to Kibana.

Click on the Management menu item on the left.

Next click on Index Patterns.

You should see our filebeat index pattern listed.

In the index pattern type in filebeat-* as shown here.

Click the Next step button.

In the next step, select @timestamp for the Time Filter field name.

Finally, click on the Create index pattern button.

Click on the Discover menu item on the left and you will see your logs.

Follow these same steps to get logs from other systems into Elasticsearch/Kibana.

Conclusion

In this post, we learned how to install Elasticsearch on Ubuntu 18.04.1.