网易开源的camellia-redis-proxy的浅尝

2024-09-12 14:10:34 浏览数 (3)

项目地址:https://github.com/netease-im/camellia

camellia-redis-proxy 是网易云信开发的服务器基础组件。功能很强大,具体功能和特性可以参见上面的github链接。

这里只是简单演示下它的几种常用用法:

配置和启动

代码语言:txt复制
下载解压到/root/目录下

切换到配置文件所在的目录
$ cd ~/camellia-redis-proxy-1.2.29/BOOT-INF/classes

下面是加载了多个插件的配置文件(这里演示使用的是单节点的redis)

$ cat application.yml | egrep -v '^#|^$' 整理后的内容如下:

代码语言:txt复制
server:
  port: 6380
spring:
  application:
    name: camellia-redis-proxy-server
camellia-redis-proxy:
  console-port: 16379
  password: pass123
  proxy-protocol-enable: true
  monitor-enable: true  #monitor enable/disable configure
  monitor-interval-seconds: 60 #monitor data refresh interval seconds
  netty:
    reader-idle-time-seconds: 600
    writer-idle-time-seconds: 0
    all-idle-time-seconds: 0
  plugins: #plugin list
    - monitorPlugin
    - bigKeyPlugin
    - hotKeyPlugin
    - hotKeyCachePlugin
    - troubleTrickKeysPlugin
    - commandDisablePlugin
    - ipCheckerPlugin
  transpond:
    type: local #local、remote、custom
    local:
      type: simple #simple、complex
      resource: redis://@127.0.0.1:6379

$ cat camellia-redis-proxy.properties

代码语言:txt复制
#you can dynamic enable/disable
monitor.enable=true

#you can dynamic update plugin list
proxy.plugin.list=monitorPlugin,hotKeyCachePlugin,hotKeyPlugin,bigKeyPlugin,commandDisablePlugin,troubleTrickKeysPlugin

#########################################################

command.task.queue.capacity=32768

#########################################################
#开关
hot.key.monitor.enable=true
#热key监控LRU计数器的容量,一般不需要配置
hot.key.monitor.cache.max.capacity=100000
#热key监控统计的时间窗口,默认1000ms
hot.key.monitor.counter.check.millis=1000
#热key监控统计在时间窗口内超过多少阈值,判定为热key,默认500
hot.key.monitor.counter.check.threshold=1
#单个周期内最多上报多少个热key,默认32(取top)
hot.key.monitor.max.hot.key.count=32

##监控数据默认通过/monitor进行对外暴露(默认60s刷新一次数据),如果需要实时推送,可以设置callback(实现HotKeyMonitorCallback接口即可)
###默认的callback不做任何处理
hot.key.monitor.callback.className=com.netease.nim.camellia.redis.proxy.plugin.hotkey.DummyHotKeyMonitorCallback

#########################################################
#开关
big.key.monitor.enable=true

#阈值
##默认2M
big.key.monitor.string.threshold=2097152
##默认5000
big.key.monitor.hash.threshold=5000
big.key.monitor.set.threshold=5000
big.key.monitor.zset.threshold=5000
big.key.monitor.list.threshold=5000

##监控数据默认通过/monitor进行对外暴露(默认60s刷新一次数据),如果需要实时推送,可以设置callback(实现BigKeyMonitorCallback接口即可)
###默认的callback不做任何处理
big.key.monitor.callback.className=com.netease.nim.camellia.redis.proxy.plugin.bigkey.DummyBigKeyMonitorCallback

#########################################################

#哪些key需要热key缓存功能,默认实现是PrefixMatchHotKeyCacheKeyChecker,可以基于key的前缀去配置,你可以自定义实现(实现HotKeyCacheKeyChecker接口即可)
hot.key.cache.key.checker.className=com.netease.nim.camellia.redis.proxy.plugin.hotkeycache.PrefixMatchHotKeyCacheKeyChecker

#使用PrefixMatchHotKeyCacheKeyChecker时的前缀配置方法,如果要配置所有key都启用热key缓存功能,设置空字符串即可,默认所有key都不生效
hot.key.cache.key.prefix=["dao_c", "kkk"]

##热key缓存相关的配置
#热key缓存功能的开关,默认true
hot.key.cache.enable=true
#用于判断是否是热key的LRU计数器的容量
hot.key.cache.counter.capacity=100000
#用于判断是否是热key的LRU计数器的时间窗口,默认1000ms
hot.key.cache.counter.check.millis=1000
#判定为热key的阈值,默认100
hot.key.cache.check.threshold=100
#是否缓存null的value,默认true
hot.key.cache.null=true
#热key缓存的时长,默认10s,过期一半的时候会穿透一个GET请求到后端
hot.key.cache.expire.millis=10000
#最多多少个缓存的热key,默认1000
hot.key.cache.max.capacity=1000


##监控数据默认通过/monitor进行对外暴露(默认60s刷新一次数据),如果需要实时推送,可以设置callback(实现HotKeyCacheStatsCallback接口即可)
###默认的callback不做任何处理
hot.key.cache.stats.callback.className=com.netease.nim.camellia.redis.proxy.plugin.hotkeycache.DummyHotKeyCacheStatsCallback
#热key缓存命中情况实时推送的间隔,默认10s
hot.key.cache.stats.callback.interval.seconds=10



#########################################################
#慢查询监控的阈值,默认2000ms
slow.command.threshold.millis=2000
##慢查询监控数据默认通过/monitor进行对外暴露(默认60s刷新一次数据),如果需要实时推送,可以设置callback(实现SlowCommandMonitorCallback接口即可)
slow.command.monitor.callback.className=com.netease.nim.camellia.redis.proxy.plugin.monitor.DummySlowCommandMonitorCallback

#其他监控数据(如请求数、rt等,统一通过/monitor接口对外暴露)
#特别的,对于rt的监控,有一个子开关,默认开启,如果关闭,则只统计tps,不统计rt
command.spend.time.monitor.enable=true

#########################################################

#配置
#表示:针对key1和key2的ZREVRANGEBYSCORE方法,针对key3和key4的GET方法,会被拦截(直接返回错误信息)
trouble.trick.keys=ZREVRANGEBYSCORE:["key1","key2"];GET:["key3","key4"]
# default.trouble.trick.keys=ZRANGE:["key1","key2"];SMEMBERS:["key3","key4"]
#配置(租户级别)
#表示:bid=2/bgroup=default路由配置下,针对key1和key2的ZRANGE方法,针对key3和key4的SMEMBERS方法,会被拦截(直接返回错误信息)
#2.default.trouble.trick.keys=ZRANGE:["key1","key2"];SMEMBERS:["key3","key4"]

#########################################################

#被屏蔽的命令列表(忽略大小写)
disabled.commands=EVAL,KEYS

#########################################################

#黑名单示例(支持ip,也支持网段,逗号分隔):
ip.check.mode=1
ip.black.list=192.168.3.12

#白名单示例(支持ip,也支持网段,逗号分隔):
#ip.check.mode=2
#ip.white.list=2.2.2.2,5.5.5.5,3.3.3.0/24,6.6.0.0/16

#########################################################

# 是否关闭空闲连接
reader.idle.client.connection.force.close.enable=false

#########################################################

启动

代码语言:txt复制
$ cd /root/camellia-redis-proxy-1.2.29
$ sh start.sh 

```
2024-09-10 14:27:51,544 INFO  main  c.n.n.c.r.p.h.CamelliaRedisProxyHttpServer:38 - CamelliaRedisProxyServer with http disabled, skip start
2024-09-10 14:27:51,544 INFO  main  c.n.n.c.r.p.n.CamelliaRedisProxyServer:178 - CamelliaRedisProxyServer start at port: 6380
2024-09-10 14:27:51,551 INFO  main  c.n.n.c.r.p.n.CamelliaRedisProxyServer:207 - CamelliaRedisProxyServer start success, version = 1.2.29
2024-09-10 14:27:51,553 INFO  main  c.n.n.c.h.c.CamelliaHttpConsoleServer:34 - console service init, uri.set = [/status, /reload, /prometheus, /info, /metrics, /custom, /shutdownUpstreamClient, /monitor, /offline, /online, /detect, /check]
2024-09-10 14:27:51,555 INFO  main  c.n.n.c.h.c.CamelliaHttpConsoleServer:55 - Console Server start listen at port 16379
2024-09-10 14:27:51,641 INFO  main  o.s.j.e.a.AnnotationMBeanExporter:433 - Registering beans for JMX exposure on startup
2024-09-10 14:27:51,652 INFO  main  c.n.n.c.r.p.b.Application:59 - Started Application in 1.784 seconds (JVM running for 2.418)
```

可以看到,自带了[/status, /reload, /prometheus, /info, /metrics, /custom, /shutdownUpstreamClient, /monitor, /offline, /online, /detect, /check] 这些http控制接口。
每次改动camellia-redis-proxy.properties ,只要执行 curl -s 127.0.0.1:16379/reload 接口即可生效。

插件演示

bigKeyPlugin

代码语言:txt复制
​	启动压测: redis-benchmark -n 1000000 -r 10000 -c 200 -t get -q -p 6380 -a pass123

	稍等片刻,就可以在控制台看到类似下面的日志(我这里是把hotkey的阈值改小了,不然不容易得到信息):
	2024-09-10 15:11:37,450 INFO  camellia-callback-8-1  c.r.p.stats:102 - ====hot.key.stats====
	2024-09-10 15:11:37,450 INFO  camellia-callback-8-1  c.r.p.stats:105 - bid=default,bgroup=default,key=key:000000001848,times=3,max=10,avg=7.666666666666667,checkMillis=1000,checkThreshold=1
	2024-09-10 15:11:37,450 INFO  camellia-callback-8-1  c.r.p.stats:105 - bid=default,bgroup=default,key=key:000000001968,times=1,max=9,avg=9.0,checkMillis=1000,checkThreshold=1
	2024-09-10 15:11:37,450 INFO  camellia-callback-8-1  c.r.p.stats:105 - bid=default,bgroup=default,key=key:000000000999,times=1,max=12,avg=12.0,checkMillis=1000,checkThreshold=1
	2024-09-10 15:11:37,454 INFO  camellia-callback-8-1  c.r.p.stats:105 - bid=default,bgroup=default,key=key:000000001967,times=2,max=9,avg=8.0,checkMillis=1000,checkThreshold=1
	2024-09-10 15:11:37,478 INFO  camellia-callback-8-1  c.r.p.stats:105 - bid=default,bgroup=default,key=key:000000003529,times=1,max=15,avg=15.0,checkMillis=1000,checkThreshold=1
	2024-09-10 15:11:37,478 INFO  camellia-callback-8-1  c.r.p.stats:105 - bid=default,bgroup=default,key=key:000000001908,times=1,max=9,avg=9.0,checkMillis=1000,checkThreshold=1
	2024-09-10 15:11:37,478 INFO  camellia-callback-8-1  c.r.p.stats:109 - ====hot.key.cache.stats====
	2024-09-10 15:11:37,478 INFO  camellia-callback-8-1  c.r.p.stats:117 - ====slow.command.stats====
	2024-09-10 15:11:37,478 INFO  camellia-callback-8-1  c.r.p.stats:125 - ====upstream.fail.stats====
	2024-09-10 15:11:37,478 INFO  camellia-callback-8-1  c.r.p.stats:132 - ====kv.cache.stats====
	2024-09-10 15:11:37,478 INFO  camellia-callback-8-1  c.r.p.stats:141 - ====kv.executor.stats====
	2024-09-10 15:11:37,478 INFO  camellia-callback-8-1  c.r.p.stats:147 - ====kv.gc.stats====
	2024-09-10 15:11:37,478 INFO  camellia-callback-8-1  c.r.p.stats:153 - ====kv.write.buffer.stats====
	2024-09-10 15:11:37,478 INFO  camellia-callback-8-1  c.r.p.stats:161 - ====kv.storage.stats====
	2024-09-10 15:11:37,478 INFO  camellia-callback-8-1  c.r.p.stats:171 - <<<<<<<END<<<<<<<

troubleTrickKeysPlugin

代码语言:txt复制
	127.0.0.1:6380> get key1
	(nil)
	127.0.0.1:6380> set key1 sss
	OK
	127.0.0.1:6380> get key1
	"sss"
	127.0.0.1:6380> get key3
	(error) ERR trouble trick key fast fail

commandDisablePlugin

代码语言:txt复制
	127.0.0.1:6380> eval a
	(error) ERR command 'eval' is disabled in proxy
	127.0.0.1:6380> keys *
	(error) ERR command 'keys' is disabled in proxy

commandDisablePlugin

代码语言:txt复制
	127.0.0.1:6380> eval a
	(error) ERR command 'eval' is disabled in proxy
	127.0.0.1:6380> keys *
	(error) ERR command 'keys' is disabled in proxy

ipCheckerPlugin

代码语言:txt复制
	[root@xxxx]~# redis-cli -p 6380 -a pass123 -h 192.168.3.14
	Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
	AUTH failed: ip forbidden

其它插件

代码语言:txt复制
用于控制客户端的请求tps
	https://github.com/netease-im/camellia/blob/master/docs/camellia-redis-proxy/plugin/rate-limit.md
	
支持动态设置限流阈值
	https://github.com/netease-im/camellia/blob/master/docs/camellia-redis-proxy/plugin/dynamic-rate-limit.md

将查询结果缓存
	https://github.com/netease-im/camellia/blob/master/docs/camellia-redis-proxy/plugin/hot-key-cache.md

IP黑名单
	https://github.com/netease-im/camellia/blob/master/docs/camellia-redis-proxy/plugin/ip-checker.md

监控

https://github.com/netease-im/camellia/blob/master/docs/camellia-redis-proxy/prometheus/prometheus-grafana.md

官方的压测报告

https://github.com/netease-im/camellia/blob/master/docs/camellia-redis-proxy/performance/performance.md

redis-shake数据迁移注意事项

https://github.com/netease-im/camellia/blob/master/docs/camellia-redis-proxy/other/redis-shake.md

0 人点赞