学习redis的途中,碰上了redis.conf,突发奇想,想着来进行一波翻译输出。 源码之前,了无秘密。
文章目录- 慢删除
- 原文
- 译文
- 线程I/O
- 原文
- 译文
- 追加持久化(只附加模式)
- 原文
- 译文
- LUA脚本
- 原文
- 译文
- 原文
- 译文
- 原文
- 译文
- 原文
- 译文
- 原文
- 译文
慢删除
原文
代码语言:javascript复制############################# LAZY FREEING ####################################
# Redis has two primitives to delete keys. One is called DEL and is a blocking
# deletion of the object. It means that the server stops processing new commands
# in order to reclaim all the memory associated with an object in a synchronous
# way. If the key deleted is associated with a small object, the time needed
# in order to execute the DEL command is very small and comparable to most other
# O(1) or O(log_N) commands in Redis. However if the key is associated with an
# aggregated value containing millions of elements, the server can block for
# a long time (even seconds) in order to complete the operation.
#
# For the above reasons Redis also offers non blocking deletion primitives
# such as UNLINK (non blocking DEL) and the ASYNC option of FLUSHALL and
# FLUSHDB commands, in order to reclaim memory in background. Those commands
# are executed in constant time. Another thread will incrementally free the
# object in the background as fast as possible.
#
# DEL, UNLINK and ASYNC option of FLUSHALL and FLUSHDB are user-controlled.
# It's up to the design of the application to understand when it is a good
# idea to use one or the other. However the Redis server sometimes has to
# delete keys or flush the whole database as a side effect of other operations.
# Specifically Redis deletes objects independently of a user call in the
# following scenarios:
#
# 1) On eviction, because of the maxmemory and maxmemory policy configurations,
# in order to make room for new data, without going over the specified
# memory limit.
# 2) Because of expire: when a key with an associated time to live (see the
# EXPIRE command) must be deleted from memory.
# 3) Because of a side effect of a command that stores data on a key that may
# already exist. For example the RENAME command may delete the old key
# content when it is replaced with another one. Similarly SUNIONSTORE
# or SORT with STORE option may delete existing keys. The SET command
# itself removes any old content of the specified key in order to replace
# it with the specified string.
# 4) During replication, when a replica performs a full resynchronization with
# its master, the content of the whole database is removed in order to
# load the RDB file just transferred.
#
# In all the above cases the default is to delete objects in a blocking way,
# like if DEL was called. However you can configure each case specifically
# in order to instead release memory in a non-blocking way like if UNLINK
# was called, using the following configuration directives.
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
# It is also possible, for the case when to replace the user code DEL calls
# with UNLINK calls is not easy, to modify the default behavior of the DEL
# command to act exactly like UNLINK, using the following configuration
# directive:
lazyfree-lazy-user-del no
译文
Redis有两个原语来删除键。
一个是DEL,它是对象的阻塞删除。这意味着服务器停止处理新命令,以同步方式收回与一个对象关联的所有内存。 如果删除的键与一个小对象相关联,那么执行DEL命令所需的时间非常小,可以与Redis中的大多数O(1)或O(log_N)命令相媲美。 但是,如果键与包含数百万个元素的聚合值相关联,服务器可能会阻塞很长时间(甚至几秒钟)以完成操作。
基于上述原因,Redis还提供了非阻塞删除原语,如UNLINK(非阻塞DEL)和异步选项的FLUSHALL和FLUSHDB命令,以便在后台回收内存。这些命令在固定的时间内执行。另一个线程将在后台以尽可能快的速度递增地释放对象。
FLUSHALL 和 FLUSHDB 的删除选项(DEL, UNLINK 和 ASYNC)是有使用者选定的。
何时使用哪一种会比较好,这取决于应用程序的设计。 然而,有时由于其他操作的副作用,redis不得不删除或刷新整个数据库。
具体来说,在以下场景,redis将自行删除数据对象:
代码语言:javascript复制1)在回收时,由于maxmemory和maxmemory策略配置,为了给新数据腾出空间,而不超过指定的内存限制。
2)因为过期:当必须从内存中删除一个具有关联生存时间的键(参见expire命令)时。
3)由于一个命令的副作用,该命令将数据存储在一个可能已经存在的键上。例如,RENAME命令可能会在用另一个键替换旧键内容时删除它。类似的,SUNIONSTORE或SORT with STORE选项可以删除现有的键。SET命令本身删除指定键的任何旧内容,以便将其替换为指定的字符串。
4)在复制过程中,当一个副本与它的主服务器执行完全的重新同步时,整个数据库的内容被删除,以便加载刚刚传输的RDB文件。
在上述所有情况下,默认情况是以阻塞的方式删除对象,比如调用DEL。但是,您可以具体配置每种情况,以便以非阻塞的方式释放内存,比如在UNLINK被调用时,使用以下配置指令:
代码语言:javascript复制lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
也有可能,当用UNLINK调用替换用户代码DEL调用不那么顺利的情况下,可以修改DEL命令的默认行为,使其完全像UNLINK一样,使用以下配置指令:
代码语言:javascript复制lazyfree-lazy-user-del no
线程I/O
原文
代码语言:javascript复制################################ THREADED I/O #################################
# Redis is mostly single threaded, however there are certain threaded
# operations such as UNLINK, slow I/O accesses and other things that are
# performed on side threads.
#
# Now it is also possible to handle Redis clients socket reads and writes
# in different I/O threads. Since especially writing is so slow, normally
# Redis users use pipelining in order to speedup the Redis performances per
# core, and spawn multiple instances in order to scale more. Using I/O
# threads it is possible to easily speedup two times Redis without resorting
# to pipelining nor sharding of the instance.
#
# By default threading is disabled, we suggest enabling it only in machines
# that have at least 4 or more cores, leaving at least one spare core.
# Using more than 8 threads is unlikely to help much. We also recommend using
# threaded I/O only if you actually have performance problems, with Redis
# instances being able to use a quite big percentage of CPU time, otherwise
# there is no point in using this feature.
#
# So for instance if you have a four cores boxes, try to use 2 or 3 I/O
# threads, if you have a 8 cores, try to use 6 threads. In order to
# enable I/O threads use the following configuration directive:
#
# io-threads 4
#
# Setting io-threads to 1 will just use the main thread as usually.
# When I/O threads are enabled, we only use threads for writes, that is
# to thread the write(2) syscall and transfer the client buffers to the
# socket. However it is also possible to enable threading of reads and
# protocol parsing using the following configuration directive, by setting
# it to yes:
#
# io-threads-do-reads no
#
# Usually threading reads doesn't help much.
#
# NOTE 1: This configuration directive cannot be changed at runtime via
# CONFIG SET. Aso this feature currently does not work when SSL is
# enabled.
#
# NOTE 2: If you want to test the Redis speedup using redis-benchmark, make
# sure you also run the benchmark itself in threaded mode, using the
# --threads option to match the number of Redis theads, otherwise you'll not
# be able to notice the improvements.
译文
Redis大部分是单线程的,但是也有一些线程操作,如解除链接,缓慢的I/O访问和一些其他事情是在侧线程上执行的。 现在它也可以在不同的I/O线程下去处理Redis客户端套接字读写。特别是“写”过程是如此的慢吞吞,通常Redis用户使用管道通信来提高每个核的Redis性能,并衍生多个实例来扩大规模。 使用I/O线程,它可以很容易地加速两倍的Redis没有诉诸于管道或切分的实例。
默认情况下,线程是禁用的,我们建议仅在拥有至少4个或更多核的机器上启用它,至少留下一个备用核。 使用超过8个线程多半没什么用。我们还建议只有在你确实有性能问题的时候才使用线程I/O,因为Redis实例会占用相当大百分比的CPU,所以没事儿开些线程出来玩没必要。 例如,如果你有4核,可以尝试使用2或3个I/O线程,如果你有8核,可以尝试使用6个线程。
以下配置指令开启线程之旅:# io-threads 4
将io-threads设置为1只会像往常一样使用主线程。当启用I/O线程时,我们只使用线程进行写操作,因为对线程来说,write(2)系统调用和传输客户端缓冲区到套接字,然而,也有可能启用线程读取和协议解析使用以下配置指令,通过将它设置为yes:# io-threads-do-reads no
通常情况下线程拿去读没什么意义。
注意1:这个配置指令不能在运行时通过配置设置来改变。当SSL被启用时,这个特性不能用。 注2:如果你想用Redis -benchmark测试Redis加速,请确保你也在线程模式下运行基准测试,使用——threads选项来匹配Redis头的数量,否则你将无法注意到改进。
追加持久化(只附加模式)
原文
代码语言:javascript复制############################## APPEND ONLY MODE ###############################
# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.
appendonly no
# The name of the append only file (default: "appendonly.aof")
appendfilename "appendonly.aof"
# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".
# appendfsync always
appendfsync everysec
# appendfsync no
# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.
no-appendfsync-on-rewrite no
# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes
# When rewriting the AOF file, Redis is able to use an RDB preamble in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
# [RDB file][AOF tail]
#
# When loading Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, and continues loading the AOF
# tail.
aof-use-rdb-preamble yes
译文
默认情况下,Redis异步转储数据集到磁盘上。这种模式在许多应用程序中已经足够好了,但Redis进程的问题或断电可能会导致几分钟的写丢失(取决于配置的保存点)。
Append Only文件是另一种持久性模式,它提供了更好的持久性。例如使用默认数据fsync策略配置文件中(见后),Redis可以在一个意外事件中失去一秒的写操作,比如服务器断电,或者如果Redis进程本身发生了什么问题,一个写操作,但操作系统仍然正常运行。
AOF和RDB持久性可以同时启用而没有问题。如果在启动时启用了AOF, Redis将加载AOF,这是一个更好的持久性保障。 详情见:http://redis.io/topics/persistence
append only 文件名:“appendonly.aof”
fsync()调用告诉操作系统在磁盘上实际写入数据,而不是等待输出缓冲区中的更多数据。一些操作系统会真正刷新磁盘上的数据,而另一些操作系统会尝试尽快这么做。
Redis支持以下三种不同的模式:
代码语言:javascript复制no: 不fsync,只要让操作系统在需要的时候刷新数据即可。尽快。
always: fsync在每次写入append only日志后执行。慢点,安全第一。
everysec:一秒执行一次fsync,折中。
默认值是“每秒钟”,因为这通常是速度和数据安全之间的折中。如果你能处理好这个“不”字,让操作系统刷新输出缓冲区时,为了更好的表现(但是如果你可以忍受一些数据丢失的),那你可以选“不”。或相反,使用“总是”非常缓慢但比everysec更安全一点。
更多细节参见:http://antirez.com/post/redis-persistence-demystified.html
当AOF fsync策略设置为always或everysec,并且后台保存进程(后台保存或AOF日志后台重写)对磁盘执行大量I/O操作时,在一些Linux配置中,Redis可能会在fsync()调用上阻塞太长时间。注意,这个问题目前还没有解决办法,因为即使在不同的线程中执行fsync,也会阻塞我们的同步write(2)调用。
为了减轻这个问题,可以使用以下选项来防止在执行BGSAVE或BGREWRITEAOF时在主进程中调用fsync()。 这意味着,当另一个子对象正在保存,Redis的持久性与“appendfsync none”相同。实际上,这意味着在最坏的情况下(使用默认的Linux设置)可能会丢失最多30秒的日志。
如果有延迟问题,请将此选项转换为“yes”。否则,从耐久性的角度来看,这是最安全的选择。
自动重写仅追加的文件: 当AOF日志大小增长指定的百分比时,Redis会隐式调用BGREWRITEAOF,自动重写日志文件。
它的工作模式是:Redis会记住最近一次重写后的AOF文件的大小(如果重启后没有重写,则使用启动时的AOF文件的大小)。
如果当前大小大于指定的百分比,就会触发重写。您还需要指定要重写的AOF文件的最小大小,这有助于避免重写AOF文件,即使达到百分比增长,但它仍然很小。 指定一个百分比为零,以禁用自动AOF重写功能。
在redis启动期间,可能会发现aof文件在结尾处被截断 ,这可能会发生在Redis运行的系统崩溃时,特别是ext4文件系统在没有data=ordered选项的情况下挂载(然而,这不会发生在Redis本身崩溃或中止但操作系统仍然正常工作时)。
发生这种情况时,redis可以退出并返回错误,或者加载尽可能多的数据。 以下选项控制此行为: 如果AOF -load-truncated被设置为yes,出现AOF文件加载被截断时Redis会发送一个日志来提醒用户。否则,如果将该选项设置为no,服务器将以错误终止并拒绝启动。当选项设置为no时,用户需要在重启服务器之前使用“redis-check-aof”程序修复AOF文件。
注意,如果发现AOF文件在中间损坏,服务器仍然会退出并出现错误。这个选项只适用于当Redis将尝试读取更多的数据从AOF文件,但没有足够的字节时生效。
当重写AOF文件,Redis能够使用一个RDB序言将AOF文件更快的重写和恢复。当这个选项被打开时,重写的AOF文件由两个不同的节组成:[RDB file][AOF tail]
加载Redis时,会识别出AOF文件以“Redis”字符串开头并加载带前缀的RDB文件,然后继续加载AOF尾部。
LUA脚本
原文
代码语言:javascript复制################################ LUA SCRIPTING ###############################
# Max execution time of a Lua script in milliseconds.
#
# If the maximum execution time is reached Redis will log that a script is
# still in execution after the maximum allowed time and will start to
# reply to queries with an error.
#
# When a long running script exceeds the maximum execution time only the
# SCRIPT KILL and SHUTDOWN NOSAVE commands are available. The first can be
# used to stop a script that did not yet called write commands. The second
# is the only way to shut down the server in the case a write command was
# already issued by the script but the user doesn't want to wait for the natural
# termination of the script.
#
# Set it to 0 or a negative value for unlimited execution without warnings.
lua-time-limit 5000
译文
Lua脚本的最长执行时间(毫秒)。
如果达到最大执行时间,Redis将记录在允许的最长时间之后脚本仍在执行中,并将开始以错误的方式答复查询。
当长时间运行的脚本超过最大执行时间时,只有脚本KILL和SHUTDOWN NOSAVE命令可用。第一个命令可用于停止尚未调用write命令的脚本。第二种方法是在脚本已经发出write命令但用户不想等待脚本自然终止的情况下关闭服务器的唯一方法。
将其设置为0或负值,以无限制地执行而不发出警告。