在 K8s 中，使用 helm 部署 redis 主从模式，一主可以有多从，可指定任意数量的从节点，扩容缩容都很方便。
# 1.基于helm安装
- 安装helm不在详述，见其他文章
# 2.创建 redis-cluster 目录
代码语言：javascript复制[root@master linux-amd64]# mkdir -p /opt/charts/redis-cluster
[root@master linux-amd64]# cd /opt/charts/redis-cluster/
[root@master redis-cluster]# ll
total 0# 3.下载最新stalbe/redis-ha
代码语言：javascript复制[root@master redis-cluster]# helm repo add stable http://mirror.azure.cn/kubernetes/charts/
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config
"stable" has been added to your repositories
[root@master redis-cluster]# helm repo update
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈Happy Helming!⎈
[root@master redis-cluster]# helm search repo redis
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config
NAME CHART VERSION APP VERSION DESCRIPTION
apphub/prometheus-redis-exporter 3.2.2 1.3.4 Prometheus exporter for Redis metrics
apphub/redis 10.5.3 5.0.7 Open source, advanced key-value store. It is of...
apphub/redis-cache 0.5.0 4.0.12-alpine A pure in-memory redis cache, using statefulset...
apphub/redis-ha 4.3.3 5.0.6 Highly available Kubernetes implementation of R...
apphub/redis-operator 1.0.0 Redis Operator provides high availability redis...
apphub/redispapa 0.0.1 0.0.1 利用redis的info信息对redis的使用情况进行监控的一...
bitnami/redis 12.7.0 6.0.10 Open source, advanced key-value store. It is of...
bitnami/redis-cluster 4.3.1 6.0.10 Open source, advanced key-value store. It is of...
stable/prometheus-redis-exporter 3.5.1 1.3.4 DEPRECATED Prometheus exporter for Redis metrics
stable/redis 10.5.7 5.0.7 DEPRECATED Open source, advanced key-value stor...
stable/redis-ha 4.4.4 5.0.6 DEPRECATED - Highly available Kubernetes implem...
stable/sensu 0.2.5 0.28 DEPRECATED Sensu monitoring framework backed by...
apphub/codis 3.2 3.2 A Helm chart for Codis
[root@master redis-cluster]# helm pull stable/redis-ha
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config
[root@master redis-cluster]# ll
total 20
-rw-r--r--. 1 root root 17486 Nov 9 09:54 redis-ha-4.4.4.tgz# 4.解压 charts，复制 values.yaml设置
代码语言：javascript复制[root@master redis-cluster]# tar zxvf redis-ha-*.tgz
redis-ha/Chart.yaml
tar: redis-ha/Chart.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/values.yaml
tar: redis-ha/values.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/NOTES.txt
tar: redis-ha/templates/NOTES.txt: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/_configs.tpl
tar: redis-ha/templates/_configs.tpl: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/_helpers.tpl
tar: redis-ha/templates/_helpers.tpl: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-auth-secret.yaml
tar: redis-ha/templates/redis-auth-secret.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-ha-announce-service.yaml
tar: redis-ha/templates/redis-ha-announce-service.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-ha-configmap.yaml
tar: redis-ha/templates/redis-ha-configmap.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-ha-exporter-script-configmap.yaml
tar: redis-ha/templates/redis-ha-exporter-script-configmap.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-ha-pdb.yaml
tar: redis-ha/templates/redis-ha-pdb.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-ha-role.yaml
tar: redis-ha/templates/redis-ha-role.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-ha-rolebinding.yaml
tar: redis-ha/templates/redis-ha-rolebinding.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-ha-service.yaml
tar: redis-ha/templates/redis-ha-service.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-ha-serviceaccount.yaml
tar: redis-ha/templates/redis-ha-serviceaccount.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-ha-servicemonitor.yaml
tar: redis-ha/templates/redis-ha-servicemonitor.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-ha-statefulset.yaml
tar: redis-ha/templates/redis-ha-statefulset.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-haproxy-deployment.yaml
tar: redis-ha/templates/redis-haproxy-deployment.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-haproxy-service.yaml
tar: redis-ha/templates/redis-haproxy-service.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-haproxy-serviceaccount.yaml
tar: redis-ha/templates/redis-haproxy-serviceaccount.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/redis-haproxy-servicemonitor.yaml
tar: redis-ha/templates/redis-haproxy-servicemonitor.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/tests/test-redis-ha-configmap.yaml
tar: redis-ha/templates/tests/test-redis-ha-configmap.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/templates/tests/test-redis-ha-pod.yaml
tar: redis-ha/templates/tests/test-redis-ha-pod.yaml: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/OWNERS
tar: redis-ha/OWNERS: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/README.md
tar: redis-ha/README.md: implausibly old time stamp 1970-01-01 08:00:00
redis-ha/ci/haproxy-enabled-values.yaml
tar: redis-ha/ci/haproxy-enabled-values.yaml: implausibly old time stamp 1970-01-01 08:00:00
[root@master redis-cluster]# cp redis-ha/values.yaml .
[root@master redis-cluster]# ll
total 32
drwxr-xr-x. 4 root root 101 Nov 9 09:54 redis-ha
-rw-r--r--. 1 root root 17486 Nov 9 09:54 redis-ha-4.4.4.tgz
-rwxr-xr-x. 1 root root 11632 Nov 9 09:55 values.yaml
[root@master redis-cluster]# # 5.创建 start.sh 脚本记录启动命令
代码语言：javascript复制[root@master redis-cluster]# vim start.sh
[root@master redis-cluster]# cat start.sh
#!/bin/sh
set -x
cd /opt/charts/redis-cluster
helm install redis --create-namespace --namespace dependency -f ./values.yaml ./redis-ha
[root@master redis-cluster]# ll
total 36
drwxr-xr-x. 4 root root 101 Nov 9 09:54 redis-ha
-rw-r--r--. 1 root root 17486 Nov 9 09:54 redis-ha-4.4.4.tgz
-rw-r--r--. 1 root root 138 Nov 9 10:03 start.sh
-rwxr-xr-x. 1 root root 11632 Nov 9 09:55 values.yaml
[root@master redis-cluster]# pwd
/opt/charts/redis-cluster
[root@master redis-cluster]# # 6.修改当前目录的 values.yaml 配置
代码语言：javascript复制#举例values.yaml 配置如下，没有启用PV
#cat values.yaml
image:
repository: redis
tag: 5.0.6-alpine
replicas: 3
## Redis specific configuration options
redis:
port: 6379
masterGroupName: "mymaster" # must match ^[\w-\.] $) and can be templated
config:
## For all available options see http://download.redis.io/redis-stable/redis.conf
min-replicas-to-write: 1
min-replicas-max-lag: 5 # Value in seconds
maxmemory: "4g" # Max memory to use for each redis instance. Default is unlimited.
maxmemory-policy: "allkeys-lru" # Max memory policy to use for each redis instance. Default is volatile-lru.
repl-diskless-sync: "yes"
rdbcompression: "yes"
rdbchecksum: "yes"
resources:
requests:
memory: 200Mi
cpu: 100m
limits:
memory: 4000Mi
## Sentinel specific configuration options
sentinel:
port: 26379
quorum: 1
resources:
requests:
memory: 200Mi
cpu: 100m
limits:
memory: 200Mi
hardAntiAffinity: true
## Configures redis with AUTH (requirepass & masterauth conf params)
auth: false
persistentVolume:
enabled: false
hostPath:
path: "/opt/charts/redis-cluster/data/{{ .Release.Name }}"# 7.安装
代码语言：javascript复制[root@master redis-cluster]# ./start.sh
cd /opt/charts/redis-cluster
helm install redis --create-namespace --namespace dependency -f ./values.yaml ./redis-ha
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config
NAME: redis
LAST DEPLOYED: Mon Nov 9 11:45:19 2020
NAMESPACE: dependency
STATUS: deployed
REVISION: 1
NOTES:
Redis can be accessed via port 6379 and Sentinel can be accessed via port 26379 on the following DNS name from within your cluster:
redis-redis-ha.dependency.svc.cluster.local
To connect to your Redis server:
1. Run a Redis pod that you can use as a client:
kubectl exec -it redis-redis-ha-server-0 sh -n dependency
2. Connect using the Redis CLI:
redis-cli -h redis-redis-ha.dependency.svc.cluster.local# 8.资源验证
代码语言：javascript复制[root@master redis-cluster]# helm ls -A
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
redis dependency 1 2020-11-09 11:45:19.443314165 0800 CST deployed redis-ha-4.4.4 5.0.6
[root@master redis-cluster]# kubectl get pod,svc -n dependency
NAME READY STATUS RESTARTS AGE
pod/redis-redis-ha-server-0 2/2 Running 0 8m46s
pod/redis-redis-ha-server-1 2/2 Running 0 8m39s
pod/redis-redis-ha-server-2 2/2 Running 0 8m32s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/redis-redis-ha ClusterIP None <none> 6379/TCP,26379/TCP 8m46s
service/redis-redis-ha-announce-0 ClusterIP 10.97.172.100 <none> 6379/TCP,26379/TCP 8m46s
service/redis-redis-ha-announce-1 ClusterIP 10.106.149.95 <none> 6379/TCP,26379/TCP 8m46s
service/redis-redis-ha-announce-2 ClusterIP 10.102.240.219 <none> 6379/TCP,26379/TCP 8m46s# 9.redis主从验证
代码语言：javascript复制[root@master redis-cluster]# kubectl exec -it redis-redis-ha-server-0 sh -n dependency
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl kubectl exec [POD] -- [COMMAND] instead.
Defaulting container name to redis.
Use 'kubectl describe pod/redis-redis-ha-server-0 -n dependency' to see all of the containers in this pod.
/data $ redis-cli -h redis-redis-ha.dependency.svc.cluster.local
redis-redis-ha.dependency.svc.cluster.local:6379> ping
PONG
redis-redis-ha.dependency.svc.cluster.local:6379> info Replication
# Replication
role:master
connected_slaves:2
min_slaves_good_slaves:2
slave0:ip=10.106.149.95,port=6379,state=online,offset=78332,lag=1
slave1:ip=10.102.240.219,port=6379,state=online,offset=78473,lag=1
master_replid:fe82815c06cbfa0d9e975481f45a3e12eda31fb6
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:78473
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:78473
redis-redis-ha.dependency.svc.cluster.local:6379> - 可以看到目前是一主二从 redis-ha-server-0为主 redis-ha-server-1和redis-ha-server-2为两个从
- 扩容演示
replicas: 4
hardAntiAffinity: false[root@master redis-cluster]# helm upgrade redis -n dependency -f ./values.yaml ./redis-ha
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config
WARNING: This chart is deprecated
Release "redis" has been upgraded. Happy Helming!
NAME: redis
LAST DEPLOYED: Mon Nov 9 12:25:11 2020
NAMESPACE: dependency
STATUS: deployed
REVISION: 2
NOTES:
Redis can be accessed via port 6379 and Sentinel can be accessed via port 26379 on the following DNS name from within your cluster:
redis-redis-ha.dependency.svc.cluster.local
To connect to your Redis server:
1. Run a Redis pod that you can use as a client:
kubectl exec -it redis-redis-ha-server-0 sh -n dependency
2. Connect using the Redis CLI:
redis-cli -h redis-redis-ha.dependency.svc.cluster.local
[root@master redis-cluster]# kubectl get pod -n dependency
NAME READY STATUS RESTARTS AGE
redis-redis-ha-server-0 2/2 Running 0 40h
redis-redis-ha-server-1 2/2 Running 0 40h
redis-redis-ha-server-2 2/2 Running 0 40h
redis-redis-ha-server-3 0/2 Pending 0 22s
[root@master redis-cluster]# kubectl get pod -n dependency
NAME READY STATUS RESTARTS AGE
redis-redis-ha-server-0 2/2 Running 0 40h
redis-redis-ha-server-1 2/2 Running 0 40h
redis-redis-ha-server-2 2/2 Running 0 40h
redis-redis-ha-server-3 2/2 Running 0 9m12s- 查看每个节点上的数据储存
[root@node redis]# ll
总用量 4
drwxr-xr-x. 2 summer root 45 2月 5 16:56 conf
-rw-r--r--. 1 summer root 197 2月 5 17:13 dump.rdb
[root@node redis]# pwd
/opt/charts/redis-cluster/data/redis
[root@node redis]# # 10.对外访问
- service.yml
apiVersion: v1
kind: Service
metadata:
name: redis-access-service
namespace: dependency
labels:
app: redis-ha
spec:
type: NodePort
ports:
- name: redis-port
protocol: "TCP"
port: 6379
targetPort: 6379
nodePort: 30179
selector:
app: redis-ha- svc
[root@master redis-cluster]# kubectl get svc -n dependency
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
redis-access-service NodePort 10.111.248.8 <none> 6379:30179/TCP 3m26s
redis-redis-ha ClusterIP None <none> 6379/TCP,26379/TCP 46m
redis-redis-ha-announce-0 ClusterIP 10.97.172.100 <none> 6379/TCP,26379/TCP 46m
redis-redis-ha-announce-1 ClusterIP 10.106.149.95 <none> 6379/TCP,26379/TCP 46m
redis-redis-ha-announce-2 ClusterIP 10.102.240.219 <none> 6379/TCP,26379/TCP 46m
[root@master redis-cluster]# ^C
