Redis内存碎片的产生与清理

2022-08-19 11:13:12 浏览数 (1)

前言

在做redis内存清理时我们会关注redis的实时内存占用,即通过info memory命令查看内存使用情况:

我们比较常关注的是used_memory以及used_memory_rss,这两项分别是redis中的数据占用的内存以及redis向操作系统申请的所有内存,可以看到这里这两项差距是很大的,我们也可以通过另一项更直观的观察这两项的差距,即mem_fragmentation_ratio

mem_fragmentation_ratio = used_memory_rss / used_memory

Redis申请了这么多内存是在干啥呢???非常的疑惑

需要了解的几个词

  • used_memory: Amount of memory (in bytes) used by Redis
  • used_memory_rss: Memory allocated by the operating system
  • mem_fragmentation_ratio: Ratio of memory allocated by the operating system to memory requested by Redis

为什么会产生内存碎片?

主要有两个原因:

  • redis自己实现的内存分配器:在redis中新建key-value值时,redis需要向操作系统申请内存,一般的进程在不需要使用申请的内存后,会直接释放掉、归还内存;但redis不一样,redis在使用完内存后并不会直接归还内存,而是放在redis自己实现的内存分配器中管理,这样就不需要每次都向操作系统申请内存了,实现了高性能(但这样其它应用可就不高兴了,自私的Redis)
  • value的更新:redis的每个key-value对初始化的内存大小是最适合的,当这个value改变的并且原来内存大小不适用的时候,就需要重新分配内存了,重新分配之后,就会有一部分内存redis无法正常回收,一直占用着

内存碎片率的意义

From 文档:

The operating system is responsible for allocating physical memory to each process. The operating system’s virtual memory manager handles the actual mapping, mediated by a memory allocator. What does this mean? If your Redis instance has a memory footprint of 1GB, the memory allocator will first attempt to find a contiguous memory segment to store the data. If no contiguous segment is found, the allocator must divide the process’s data across segments, leading to increased memory overhead. Tracking fragmentation ratio is important for understanding your Redis instance’s performance. A fragmentation ratio greater than 1 indicates fragmentation is occurring. A ratio in excess of 1.5 indicates excessive fragmentation, with your Redis instance consuming 150% of the physical memory it requested. A fragmentation ratio below 1 tells you that Redis needs more memory than is available on your system, which leads to swapping. Swapping to disk will cause significant increases in latency (see used memory). Ideally, the operating system would allocate a contiguous segment in physical memory, with a fragmentation ratio equal to 1 or slightly greater.

  • 大于1小于1.5:正常值,有一些内存碎片,但也可以提高性能,可以接受
  • 大于1.5:说明内存碎片率比较大,需要考虑是否要进行内存碎片清理,要引起重视
  • 小于1:内存不够redis用了,已经开始使用swap机制交换内存,也就是使用硬盘了(swap可以在设置中禁用),需要考虑扩容redis

如何清理内存碎片?

From 文档:

If your server is suffering from a fragmentation ratio above 1.5, restarting your Redis instance will allow the operating system to recover memory previously unusable due to fragmentation. In this case, an alert as a notification is probably sufficient. If, however, your Redis server has a fragmentation ratio below 1, you may want to alert as a page so that you can quickly increase available memory or reduce memory usage. Starting in Redis 4, a new active defragmentation feature is available when Redis is configured to use the included copy of jemalloc. This tool can be configured to kick in when fragmentation reaches a certain level and begin to copy values into contiguous memory regions and release the old copies, reducing fragmentation while the server is running.

Redis版本4.0以下

重启redis,自动归还所有内存,简单粗暴

Redis版本4.0以上

可以开启自动内存碎片清理:

代码语言:javascript复制
127.0.0.1:6379[6]> config set activedefrag yes
OK

自动内存清理的一些相关配置如下:

代码语言:javascript复制
# Enabled active defragmentation
# 碎片整理总开关
# activedefrag yes

# Minimum amount of fragmentation waste to start active defrag
# 内存碎片达到多少的时候开启整理
active-defrag-ignore-bytes 100mb

# Minimum percentage of fragmentation to start active defrag
# 碎片率达到百分之多少开启整理
active-defrag-threshold-lower 10

# Maximum percentage of fragmentation at which we use maximum effort
# 碎片率小余多少百分比开启整理
active-defrag-threshold-upper 100

当然,在面对一些复杂的场景时我们希望能根据自己设计的策略来进行内存碎片清理,redis也提供了手动内存碎片清理的命令:

代码语言:javascript复制
127.0.0.1:6379> memory purge
OK

总结

占有欲很高的的redis总是会留下已经不用的内存,这在生产环境中必然是不能接受的,所以内存碎片的清理非常重要

0 人点赞