07.06.2023

ELK Stack on Centos 7

The instructions describe how to install ELK Stack on a virtual server running Centos 7.

What is ELK Stack?

"ELK" is short for three open source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine. Logstash is a server-side data processing pipeline that receives data from multiple sources at the same time, transforms it, and then sends it to a "stash" like Elasticsearch. Kibana allows users to visualize data using charts and graphs in Elasticsearch.

Initial requirements

The minimum system requirements for normal operation of ELK are 4Gb RAM and 2 CPUs

Java installation

Deploying an Elastic stack requires Java to be installed. Run the following command:

yum -y install java-1.8.0

You can check the installed version with the command:

java -version

Installing and configuring Elasticsearch

Use the rpm command to download the Elasticsearch key:

Add a repository:

sudo vi /etc/yum.repos.d/elasticsearch.repo

Paste the following content into it and save the file:

[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Start the installation process:

sudo yum install elasticsearch

Next, you need to edit the configuration file, first you need to open it with the vi text editor:

vi /etc/elasticsearch/elasticsearch.yml

Uncomment the following line in the file:

bootstrap.memory_lock: true

Uncomment and set values for the following parameters:

network.host: localhost
http.port: 9200

Save your changes and close the text editor.

The next step is to edit the sysconfig configuration file for Elasticsearch:

vi /etc/sysconfig/elasticsearch

Uncomment the MAX_LOCKED_MEMORY setting and make sure it's set to unlimited:

MAX_LOCKED_MEMORY=unlimited

Before starting the service, first reboot the systems and allow Elasticsearch to start at boot time:

systemctl daemon-reload
systemctl enable elasticsearch
systemctl start elasticsearch

Installing and configuring Kibana

Use wget to download Kibana 6.2.4 and then install it with the rpm command:

wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.4-x86_64.rpm
rpm -ivh kibana-6.2.4-x86_64.rpm

Next, you need to edit the Kibana configuration file:

vi /etc/kibana/kibana.yml

Uncomment the following lines and set the values:

server.port: 5601
server.host: "[IP - adress of your server]"
elasticsearch.url: "http://localhost:9200"

For example:

server.port: 5601
server.host: "123.234.123.234"
elasticsearch.url: "http://localhost:9200"

Start the Kibana service and set it to start automatically on boot:

systemctl enable kibana
systemctl start kibana

Open port 5601 to connect to the Kibana web interface:

sudo firewall-cmd --add-port=5601/tcp --permanent
sudo firewall-cmd --reload

Setting up Nginx

On your virtual server, the NGINX web server must be installed, how to do this is written in our instructions(нужна гиперссылочка).

Install additional tools for working with the web server:

yum install httpd-tools

Next, you need to create a new virtual host configuration file in the conf.d directory:

vi /etc/nginx/conf.d/[Домен или IP-адрес].conf

For example:

vi /etc/nginx/conf.d/123.234.123.234.conf

And insert the following data:

server {
listen 80;    server_name [Domen or IP-adress];
auth_basic "Restricted Access";
auth_basic_user_file /etc/nginx/htpasswd.kibana;
location / {
proxy_pass http://localhost:5601;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}

Then create a new basic authentication file using the htpassw command. You will need to create a password for the admin user:

sudo htpasswd -c /etc/nginx/htpasswd.kibana admin

Restart the nginx server:

systemctl restart nginx

Installing and configuring Logstash

Download and install Logstash:

wget https://artifacts.elastic.co/downloads/logstash/logstash-6.2.4.rpm
rpm -ivh logstash-6.2.4.rpm

Once Logstash is installed, start the service:

systemctl restart logstash
systemctl enable logstash

Connecting to Kibana

After completing the settings, connect to Kibana in the browser, to do this, go to the following link:

http://[IP-adress of your server]:5601

For example:
http://123.234.123.234:5601