Java 日志转发 syslog

2024-02-06 01:04:30 浏览数 (2)

syslog服务器配置

配置rsyslog配置文件/etc/rsyslog.conf

rsyslog服务端接收

作为日志聚合服务，接受客户端发送过来的日志

开启UDP/TCP接收：

代码语言：properties复制
# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514

配置日志接收规则：

代码语言：properties复制
$template RemoteLogs,"/data/logs/%fromhost-ip%/%$YEAR%-%$MONTH%-%$DAY%.log"
:fromhost-ip, !isequal, "127.0.0.1" ?RemoteLogs
#& ~

rsyslog客户端发送

rsyslog也可以作为客户端向远程的rsyslog服务端转发日志

开启转发：

代码语言：properties复制
# UDP转发
*.* @server_ip:514

# TCP转发
*.* @@server_ip:514

重启rsyslog服务

当rsyslog服务端或客户端配置完成后，重启服务使配置生效

代码语言：bash复制
systemctl restart rsyslog
systemctl status rsyslog

配置支持TLS的rsyslog

安装依赖包

配置支持TLS的rsyslog需要安装以下依赖包

代码语言：bash复制
yum install rsyslog-gnutls
yum install rsyslog
yum install gnutls-utils
yum install gnutls
生成密钥与证书
代码语言：bash复制
# 生成根证书
certtool --generate-privkey --outfile ca-key.pem
certtool --generate-self-signed --load-privkey ca-key.pem --outfile ca.pem

# 生成服务端密钥与证书
certtool --generate-privkey --outfile rslserver-key.pem --bits 2048
certtool --generate-request --load-privkey rslserver-key.pem --outfile request.pem
certtool --generate-certificate --load-request request.pem --outfile rslserver-cert.pem --load-ca-certificate ca.pem --load-ca-privkey ca-key.pem

# 生成客户端密钥与证书
certtool --generate-privkey --outfile rslclient-key.pem --bits 2048
certtool --generate-request --load-privkey rslclient-key.pem --outfile request.pem
certtool --generate-certificate --load-request request.pem --outfile rslclient-cert.pem --load-ca-certificate ca.pem --load-ca-privkey ca-key.pem

注意：输入证书信息时名字不要留空，不然后面Java中会报java.security.cert.CertificateParsingException: X.509 Certificate is incomplete: subject field is empty的错误

配置服务端接收TLS连接
代码语言：properties复制
$DefaultNetstreamDriver gtls
$DefaultNetstreamDriverCAFile /root/tls/ca.pem
$DefaultNetstreamDriverCertFile /root/tls/rslserver-cert.pem
$DefaultNetstreamDriverKeyFile /root/tls/rslserver-key.pem

# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514

$InputTCPServerStreamDriverAuthMode anon
$InputTCPServerStreamDriverMode 1
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverMode 1
$MaxOpenFiles 2048

配置客户端启用TLS连接

代码语言：properties复制
$DefaultNetstreamDriver gtls

$DefaultNetstreamDriverCAFile /root/tls/private/ca.pem
$DefaultNetstreamDriverCertFile /root/tls/private/rslclient-cert.pem
$DefaultNetstreamDriverKeyFile /root/tls/private/rslclient-key.pem

$ActionSendStreamDriverPermittedPeer your.server.com # that should be your rsyslog server
$ActionSendStreamDriverMode 1 # run driver in TLS-only mode
$ActionSendStreamDriverAuthMode x509/name

注意：如果证书报错的话，可以改为$ActionSendStreamDriverAuthMode anon，即不需认证证书。

Log4j转发日志

Log4j 1.X转发syslog

这里Log4j的版本是1.2.17，配置log4j配置文件开启syslog转发

代码语言：plain复制
log4j.rootLogger=INFO, console, fout, SYSLOG

log4j.appender.SYSLOG=org.apache.log4j.net.SyslogAppender
log4j.appender.SYSLOG.threshold=INFO
log4j.appender.SYSLOG.syslogHost={SYSLOG_HOST}
log4j.appender.SYSLOG.facility=LOCAL4
log4j.appender.SYSLOG.layout=org.apache.log4j.PatternLayout
log4j.appender.SYSLOG.layout.conversionPattern=%d{HH:mm:ss,SSS} %-5p %-60c %x - %m%n

需要在代码中导入系统变量

代码语言：java复制
System.setProperty("SYSLOG_HOST", "127.0.0.1");

Log4j2中转发syslog

maven依赖

代码语言：xml复制
<!--Log4j 2 dependencies-->
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-api</artifactId>
    <version>${log4j2-version}</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>${log4j2-version}</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-slf4j-impl</artifactId>
    <version>${log4j2-version}</version>
</dependency>

配置模版

代码语言：xml复制
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" monitorInterval="5">
    <Properties>
        <Property name="console-layout">%d{HH:mm:ss} %-5level %logger{36} - %msg%n</Property>
    </Properties>
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="${console-layout}"/>
        </Console>
        <Syslog name="udp-syslog" host="localhost" port="514" protocol="UDP">
            <PatternLayout pattern="${console-layout}"/>
        </Syslog>
        <Syslog name="tcp-syslog" host="localhost" port="514" protocol="TCP">
            <PatternLayout pattern="${console-layout}"/>
        </Syslog>
    </Appenders>
    <Loggers>
        <Root level="error">
            <AppenderRef ref="Console"/>
            <AppenderRef ref="tcp-syslog"/>
        </Root>
    </Loggers>
</Configuration>

使用TLS转发日志

代码语言：xml复制
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" monitorInterval="5">
    <Properties>
        <Property name="console-layout">%d{HH:mm:ss} %-5level %logger{36} - %msg%n</Property>
    </Properties>
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="${console-layout}"/>
        </Console>
        <Syslog name="tls-syslog" host="localhost" port="6514">
            <SSL>
                <TrustStore location="truststore.jks"      password="KEYSTORE_PASSWORD"/>
            </SSL>
        </Syslog>
        <Syslog name="mtls-syslog" host="localhost" port="6514">
            <SSL>
                <KeyStore   location="log4j2-keystore.jks" password="KEYSTORE_PASSWORD"/>
                <TrustStore location="truststore.jks"      password="KEYSTORE_PASSWORD"/>
            </SSL>
        </Syslog>
    </Appenders>
    <Loggers>
        <Root level="error">
            <AppenderRef ref="Console"/>
            <AppenderRef ref="tls-syslog"/>
        </Root>
    </Loggers>
</Configuration>

其中，如果是单向TLS，只需要提供Rsyslog服务端的公钥证书，存入TrustStore中即可，配置中只需要指定TrustStore。如果需要双向TLS，则需要生成客户端的私钥与证书，存入KeyStore中。

读取证书与私钥

通过以下代码将base64编码的证书或私钥读入程序中

代码语言：java复制
private static X509Certificate base64toCert(String base64String) throws CertificateException {
    byte[] decoded = Base64.getDecoder().decode(base64String);
    CertificateFactory cf = CertificateFactory.getInstance("X.509");
    return (X509Certificate) cf.generateCertificate(new ByteArrayInputStream(decoded));
}

private static RSAPrivateKey base64toPrivateKey(String base64String) throws NoSuchAlgorithmException, InvalidKeySpecException {
    byte[] decoded = Base64.getDecoder().decode(base64String);
    KeyFactory keyFactory = KeyFactory.getInstance("RSA");
    PKCS8EncodedKeySpec keySpec = new PKCS8EncodedKeySpec(decoded);
    return (RSAPrivateKey) keyFactory.generatePrivate(keySpec);
}

注意：私钥如果不是pkcs8格式的，上面代码无法读取，可以通过以下命令转换私钥格式

代码语言：bash复制
certtool --to-p8 --load-privkey rslclient-key.pem --outfile rslclient-key-8.pem

将证书和私钥存入KeyStore

代码语言：java复制
KeyStore keyStore = KeyStore.getInstance("jks");
String password = "yourpassword";
X509Certificate caCert = base64toCert(caBase64String);

// create new key store
keyStore.load(null, password.toCharArray());

// store certificate
keyStore.setCertificateEntry("cert name", caCert);

// store private key
X509Certificate[] certChain = new X509Certificate[2];
certChain[0] = cert;
certChain[1] = caCert;
Key privateKey = base64toPrivateKey(keyBase64String);
keyStore.setKeyEntry("key name", privateKey, password.toCharArray() , certChain);

// save to file
FileOutputStream fos = new FileOutputStream(trustStoreFileName);
keyStore.store(fos, password.toCharArray());

动态修改Log4j2配置

在配置文件中加上monitorInterval属性

代码语言：xml复制
<Configuration monitorInterval="5" status="WARN">
</Configuration>

在程序中指定配置文件的位置，这样就会定期拉取最新的配置文件。注意，需要在所有的logger启动之前执行。

代码语言：java复制
// set log4j2 configuration file location
LoggerContext context = (LoggerContext) LogManager.getContext(false);
File log4j2XmlFile = new File("log4j2.xml");
context.setConfigLocation(log4j2XmlFile.toURI());

接下来只需要修改"log4j2.xml"文件即可

修改XML文件

用Apache Commons Configuration来修改log4j2.xml配置文件

代码语言：java复制
// 读取默认配置
File log4j2XmlFile = new File("log4j2.xml");
Parameters params = new Parameters();
FileBasedConfigurationBuilder<FileBasedConfiguration> builder = new FileBasedConfigurationBuilder<FileBasedConfiguration>(XMLConfiguration.class)
        .configure(params.fileBased()
                .setFile(log4j2XmlFile));
Configuration config = builder.getConfiguration();

// 修改配置
String configNode = "Appenders.Syslog(1)"; // Appreders下有多个Syslog，所以需要指定第几个Syslog
config.addProperty(configNode   ".PatternLayout.[@pattern]", "${remote-layout}") // 为节点添加属性

// 保存配置
builder.save();

maven依赖

代码语言：xml复制
<!--Apache Commons Configuration dependency-->
<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-configuration2</artifactId>
    <version>${apache.commons.configuration-version}</version>
</dependency>
<dependency>
    <groupId>commons-beanutils</groupId>
    <artifactId>commons-beanutils</artifactId>
    <version>1.9.4</version>
</dependency>
拷贝默认文件

可以用Apache Commons IO中的FileUtils.copyFile来拷贝文件

代码语言：java复制
File originLog4j2XmlFile = new File("log4j2-origin.xml");
File log4j2XmlFile = new File("log4j2.xml");
FileUtils.copyFile(originLog4j2XmlFile, log4j2XmlFile);

maven依赖

代码语言：xml复制
<!--Apache Commons IO dependency-->
<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.8.0</version>
</dependency>

References

  • How to Configure rsyslog Server to Accept Logs via SSL/TLS
  • Log4j2 - SyslogAppender
  • Log4j 2 API Overview
  • Log4j Programmatic Configuration
  • Java KeyStore API
  • How to read .pem file to get private and public key
  • Apache Commons Configuration
log4j syslog

0 人点赞