什么是 ELK Stack？

“ELK”是三个开源项目的首字母缩写，这三个项目分别是：Elasticsearch、Logstash 和 Kibana。Elasticsearch 是一个搜索和分析引擎。Logstash 是服务器端数据处理管道，能够同时从多个来源采集数据，转换数据，然后将数据发送到诸如 Elasticsearch 等“存储库”中。Kibana 则可以让用户在 Elasticsearch 中使用图形和图表对数据进行可视化。

一、环境准备

1.运行环境

http://mirrors.aliyun.com/centos/7.9.2009/isos/x86_64/
https://repo.huaweicloud.com/java/jdk/11+28/jdk-11_linux-x64_bin.tar.gz
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.6.1-windows-x86_64.zip
https://artifacts.elastic.co/downloads/kibana/kibana-7.6.1-windows-x86_64.zip
https://artifacts.elastic.co/downloads/logstash/logstash-7.6.1.tar.gz

2.上传elasticsearch

上传到JDK、elasticsearch、kibana、logstash 到 /home/elk/7.6.1目录下

3.准备运行环境

yum install vim
#安装vim编辑器

cd /home/elk/7.6.1
tar -xzf elasticsearch-7.6.1-linux-x86_64.tar.gz 
tar -xzf kibana-7.6.1-linux-x86_64.tar.gz 
tar -xzf logstash-7.6.1.tar.gz
#解压文件

mv elasticsearch-7.6.1 /usr/local
mv kibana-7.6.1-linux-x86_64 /usr/local/
mv logstash-7.6.1 /usr/local/
#把安装包移动到 /usr/local目录下

mkdir /home/elk/data
mkdir /home/elk/logs
#存放日志和数据
mkdir /home/elk/7.6.1
#存放安装包

二、安装elasticsearch

1.修改elasticsearch配置

cd /usr/local/elasticsearch-7.6.1/config
vim elasticsearch.yml
#编辑配置文件


cluster.name: elasticsearch
node.name: node-1
path.data: /home/elk/data
path.logs: /home/elk/logs
network.host: 0.0.0.0
http.port: 9200
cluster.initial_master_nodes: ["node-1"]
#改成以上配置

2.启动elasticsearch

cd ../bin 
 ./elasticsearch

报错1

can not run elasticsearch as root
解决
出于系统安全考虑设置的条件，ElasticSearch可以接收用户输入的脚本并且执行，所以不能使用root账户启动

[root@localhost bin]useradd elk
[root@localhost bin]chown elk:elk /home/elk -R
[root@localhost bin]# su elk
[elk@localhost bin]$ ./elasticsearch

报错2

Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /usr/local/elasticsearch-7.6.1/config/elasticsearch.keystore
Likely root cause: java.nio.file.AccessDeniedException: /usr/local/elasticsearch-7.6.1/config/elasticsearch.keystore
解决
修改elasticsearch.keystore文件权限

[root@localhost bin]# cd ../config/
[root@localhost config]# ll
total 40
-rw-rw----. 1 root root   199 Mar 23 19:02 elasticsearch.keystore
-rw-rw----. 1 elk  elk   3011 Mar 23 18:59 elasticsearch.yml
-rw-rw----. 1 elk  elk   2301 Feb 29  2020 jvm.options
-rw-rw----. 1 elk  elk  17545 Feb 29  2020 log4j2.properties
-rw-rw----. 1 elk  elk    473 Feb 29  2020 role_mapping.yml
-rw-rw----. 1 elk  elk    197 Feb 29  2020 roles.yml
-rw-rw----. 1 elk  elk      0 Feb 29  2020 users
-rw-rw----. 1 elk  elk      0 Feb 29  2020 users_roles
[root@localhost config]# chown elk:elk elasticsearch.keystore
[root@localhost config]# cd ../bin
[root@localhost bin]# su elk

报错3

max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]

解决

vim /etc/security/limits.conf

在文件的末尾加上

* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096

报错4

max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解决
重启失效

sysctl -w vm.max_map_count=262144

永久生效

vm.max_map_count=655360   
echo "vm.max_map_count=262144" > /etc/sysctl.conf
sysctl -p

报错5

启动成功但是依旧不能访问
解决
需要关闭防火墙和selinux

三、安装kibana-7.6.1-linux-x86_64

1.修改配置

[elk@localhost local]$ cd kibana-7.6.1-linux-x86_64/
[elk@localhost kibana-7.6.1-linux-x86_64]$ ll
total 1772
drwxrwxr-x.    2 elk elk      64 Mar 23 18:55 bin
drwxrwxr-x.    5 elk elk      43 Mar 23 18:55 built_assets
drwxrwxr-x.    2 elk elk      38 Mar 23 18:55 config
drwxrwxr-x.    2 elk elk       6 Feb 29  2020 data
-rw-rw-r--.    1 elk elk   13675 Feb 29  2020 LICENSE.txt
drwxrwxr-x.    6 elk elk     108 Mar 23 18:55 node
drwxrwxr-x. 1286 elk elk   36864 Mar 23 18:55 node_modules
-rw-rw-r--.    1 elk elk 1728134 Feb 29  2020 NOTICE.txt
drwxrwxr-x.    3 elk elk      55 Mar 23 18:55 optimize
-rw-rw-r--.    1 elk elk     738 Feb 29  2020 package.json
drwxrwxr-x.    2 elk elk       6 Feb 29  2020 plugins
-rw-rw-r--.    1 elk elk    4057 Feb 29  2020 README.txt
drwxrwxr-x.   11 elk elk     160 Mar 23 18:55 src
drwxrwxr-x.    2 elk elk     183 Mar 23 18:55 webpackShims
drwxrwxr-x.    5 elk elk     129 Mar 23 18:55 x-pack
[elk@localhost kibana-7.6.1-linux-x86_64]$ cd config/
[elk@localhost config]$ vim kibana.yml
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://localhost:9200"]

2.启动kibana

[elk@localhost bin]$ su root
[root@localhost bin]# ./kibana
Kibana should not be run as root.  Use --allow-root to continue.
[root@localhost bin]# ./kibana --allow-root

四、安装 logstash-7.6.1

1.上传JDK11

2.准备环境

[root@server home]# cd /home/elk/7.6.1/
[root@server 7.6.1]# tar -xzf jdk-11_linux-x64_bin.tar.gz

3.在/etc/profile文件末尾加入以下配置,然后保存/etc/profile
vim /etc/profile

export JAVA_HOME=/home/elk/7.6.1/jdk-11
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin

source  /etc/profile
java -version

cd /usr/local/logstash-7.6.1/bin
 ./logstash -e 'input {stdin{}} output {stdout{}}'