1.logstash下载安装
(图片来自:https://www.elastic.co/cn/downloads/logstash)
(图片来自:https://www.elastic.co/guide/en/logstash/current/configuration.html)
其余配置文件的说明,如下:
(图片来自:https://www.elastic.co/guide/en/logstash/current/config-setting-files.html)
配置收集tomcat日志,如下:
代码语言:javascript复制input {
beats {
port => "5044"
}
file {
path => ".../localhost_access_log*.log"
type => "tomcat-access-log-ceshi"
start_position => "beginning"
stat_interval => "2"
}
}
output {
elasticsearch {
hosts => ["***.***.***.***:9200"]
index => "logstash-tomcat-access-log-ceshi-%{ YYYY.MM.dd}"
}
stdout { codec => rubydebug }
}
详细:ELK 架构之 Logstash 和 Filebeat 安装配置
2.filebeat连接logstash使用elasticsearch记日志的过程
(ME:filebeat和logstash有什么区别和联系?)
代码语言:javascript复制[root@node1 ~]# vi /etc/logstash/conf.d/logstash-filebeat-syslog.conf
input {
beats {
port => 10515
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
elasticsearch {
hosts => [ "node1:9200" ]
index => "%{[@metadata][beat]}-%{ YYYY.MM.dd}"
}
}
(代码来自:https://www.cnblogs.com/xishuai/p/elk-logstash-filebeat.html)
(ME:这里的grok是什么用法?)
(ME:为什么filebeat不把日志直接存储到elasticsearch,而是发给logstash服务器?)
这里对过滤logstash的过滤的职责描述的更清晰一点
Reference
https://blog.csdn.net/z1017915048/article/details/102971692
https://www.cnblogs.com/Applogize/p/13545754.html
https://www.cnblogs.com/yanshicheng/articles/9431335.html