[keen]
Introduction
I have recently been playing around with Elastisearch, Logstash and Kibana, often referred to as the ELK stack.
Installing ELK
Installing ELK is rather simple. To do so, go to the Elastic website (here) and download each package. As I’m running on Ubuntu (fresh install on Openstack), I’m downloading the equivalent packages for each service:
ubuntu@elk:~$ sudo apt-get update ubuntu@elk:~$ wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.6.0.tar.gz ubuntu@elk:~$ wget https://download.elastic.co/kibana/kibana/kibana-4.1.1-linux-x64.tar.gz ubuntu@elk:~$ wget https://download.elastic.co/logstash/logstash/logstash-1.5.2.tar.gz
Next, move ahead on untar these packages:
ubuntu@elk:~$ tar xvzf elasticsearch-1.6.0.tar.gz ubuntu@elk:~$ tar xvzf logstash-1.5.2.tar.gz ubuntu@elk:~$ tar xvzf kibana-4.1.1-linux-x64.tar.gz
When all is well, you’ll see the packages in your directory.
ubuntu@elk:~$ ls -l total 16 drwxrwxr-x 7 ubuntu ubuntu 4096 Jul 16 15:59 elasticsearch-1.6.0 drwxrwxr-x 7 ubuntu ubuntu 4096 Jun 29 18:07 kibana-4.1.1-linux-x64 drwxrwxr-x 5 ubuntu ubuntu 4096 Jul 17 10:21 logstash-1.5.2
Don’t forget to also install Java in case you haven’t done so:
ubuntu@elk:~$ sudo add-apt-repository -y ppa:webupd8team/java ubuntu@elk:~$ sudo apt-get update ubuntu@elk:~$ sudo apt-get -y install oracle-java8-installer
In order to make things a bit easier later on, let’s also create a startup script:
ubuntu@elk:~$ cat startup.sh echo "----Starting ElasticSearch----" cd ~/elasticsearch-1.6.0/ ./bin/elasticsearch & echo "----Starting Kibana----" cd ~/kibana-4.1.1-linux-x64/ ./bin/kibana & echo "----Starting Logstash----" cd ~/logstash-1.5.2/ ./bin/logstash -f logstash-access.conf echo "----Finished booting the ELK stack"
The script obviously is not strictly necessary and can probably be improved dramatically, but for now it does the trick quite well.
If all goes well, you will see messages appearing like below:
ubuntu@elk:~/elasticsearch-1.6.0$ sudo ./bin/elasticsearch
[2015-07-17 11:46:42,569][INFO ][node ] [Mindworm] version[1.6.0], pid[3797], build[cdd3ac4/2015-06-09T13:36:34Z]
[2015-07-17 11:46:42,577][INFO ][node ] [Mindworm] initializing ...
[2015-07-17 11:46:42,581][INFO ][plugins ] [Mindworm] loaded [], sites []
[2015-07-17 11:46:42,664][INFO ][env ] [Mindworm] using [1] data paths, mounts [[/ (/dev/vda1)]], net usable_space [45.5gb], net total_space [49.1gb], types [ext4]
[2015-07-17 11:46:46,591][INFO ][node ] [Mindworm] initialized
[2015-07-17 11:46:46,592][INFO ][node ] [Mindworm] starting ...
[2015-07-17 11:46:46,876][INFO ][transport ] [Mindworm] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/173.39.243.147:9300]}
[2015-07-17 11:46:46,929][INFO ][discovery ] [Mindworm] elasticsearch/pQUGKeJCT4CbNxjWZK33Jw
[2015-07-17 11:46:50,745][INFO ][cluster.service ] [Mindworm] new_master [Mindworm][pQUGKeJCT4CbNxjWZK33Jw][elk][inet[/173.39.243.147:9300]], reason: zen-disco-join (elected_as_master)
[2015-07-17 11:46:50,811][INFO ][http ] [Mindworm] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/173.39.243.147:9200]}
....
Let’s now also test if ElasticSearch is really running and available. Go to http://
In order to reach Kibana, you’ll go to http://
You have now a full ELK stack up and running. In part 2 of this tutorial, we are going to configure ELK to monitor apache access log files and visualize then using Kibana.
