自动化部署 tendis集群和redis集群初始化集群步骤一样,创建出3主3从的节点后,再创建一个初始化的job,对集群进行初始化配置。(redis集群创建修改镜像即可)
创建svc
以下是svc的yaml
kind: Service apiVersion: v1 metadata: name: tendis-cluster-auto-9n44 namespace: tendis labels: app: tendis-cluster-auto annotations: kubesphere.io/alias-name: tendis-cluster-auto kubesphere.io/serviceType: statefulservice spec: ports: - name: http-0 protocol: TCP port: 30032 targetPort: 51002 selector: app: tendis-cluster-auto clusterIP: None clusterIPs: - None type: ClusterIP sessionAffinity: None ipFamilies: - IPv4 ipFamilyPolicy: SingleStack internalTrafficPolicy: Cluster |
---|
创建6副本的statefulset
通过pvc模板自动创建对应的pvc挂载,可自行修改
kind: StatefulSet apiVersion: apps/v1 metadata: name: tendis-cluster-auto namespace: tendis labels: app: tendis-cluster-auto spec: replicas: 6 selector: matchLabels: app: tendis-cluster-auto template: metadata: creationTimestamp: null labels: app: tendis-cluster-auto annotations: kubesphere.io/imagepullsecrets: '{}' logging.kubesphere.io/logsidecar-config: '{}' spec: volumes: - name: host-time hostPath: path: /etc/localtime type: '' containers: - name: container-s140dj image: 'tencentdbforkv/tendisplus:latest' ports: - name: http-0 containerPort: 51002 protocol: TCP env: - name: CLUSTER value:yes - name: REDIS_PASSWORD value:test12345 resources: {} volumeMounts: - name: host-time mountPath: /etc/localtime - name: tendis-pvc mountPath: /data1/tendis/51002 terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent restartPolicy: Always terminationGracePeriodSeconds: 30 dnsPolicy: ClusterFirst serviceAccountName: default serviceAccount: default securityContext: {} schedulerName: default-scheduler volumeClaimTemplates: - kind: PersistentVolumeClaim apiVersion: v1 metadata: name: tendis-pvc namespace: tendis creationTimestamp: null spec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi storageClassName: default volumeMode: Filesystem status: phase: Pending serviceName: tendis-cluster-auto-9n44 podManagementPolicy: OrderedReady updateStrategy: type: RollingUpdate rollingUpdate: partition: 0 revisionHistoryLimit: 10 |
---|
自制初始化镜像
docker pull tencentdbforkv/build:latest
这个镜像的系统是Centos8
在这个镜像的基础上更新yum源,然后安装dig命令,commit 提交成一个有dig命令的镜像。
修改build.sh 脚本
通过docker run -itd tencentdbforkv/build:latest sleep 36000s 然后进入容器
该脚本需传入4个环境变量参数
PODNAME,PASSWORD,NAMESPACE,SVCNAME,用于
拼接k8s的DNS域名解析地址${PODNAME}-$i.${SVCNAME}.${NAMESPACE}.svc.cluster.local
就能通过DNS域名去解析到各个节点去执行初始化操作。
然后替换里面的build.sh脚本为下面的脚本
#!/bin/bash set -ue sleep 10 PORT=51002 #PODNAME=$1 SIZE=3 #PASSWORD=$2 #NAMESPACE=$3 #SVCNAME=$4 slot_num=$((16384 / ${SIZE})) for (( i = 0; i < 6; i )); do declare "ip$i"=”$(dig short ${PODNAME}-$i.${SVCNAME}.${NAMESPACE}.svc.cluster.local)“ done for (( i = 0; i < 6; i )); do ip_var="ip$i" ./redis-cli -h ${ip0} -p ${PORT} -a ${PASSWORD} cluster meet ${!ip_var} ${PORT} done echo "cluster meet done......" sleep 10 ./redis-cli -h ${ip0} -p ${PORT} -a ${PASSWORD} cluster nodes for (( i = 0; i < 3; i )); do uuid[i]=$(./redis-cli -h ${ip0} -p ${PORT} -a ${PASSWORD} cluster nodes | grep "ip$i" | awk '{print $1}') done for (( i = 3; i < 6; i )); do ./redis-cli -h "ip$i" -p ${PORT} -a ${PASSWORD} cluster replicate "${uuid[$((i-3))]}" done ./redis-cli -h ${ip0} -p ${PORT} -a ${PASSWORD} cluster addslots {0..5461} ./redis-cli -h ${ip1} -p ${PORT} -a ${PASSWORD} cluster addslots {5462..10922} ./redis-cli -h ${ip2} -p ${PORT} -a ${PASSWORD} cluster addslots {10923..16383} echo "cluster build done.." sleep 5 ./redis-cli -h ${IP}.10 -p ${PORT} -a ${PASSWORD} cluster nodes |
---|
保存脚本然后commit为最新的镜像
创建job初始化集群,并传入4个环境变量
kind: Job apiVersion: batch/v1 metadata: name: tendisbuild namespace: tendis labels: app: tendisbuild parallelism: 1 completions: 1 activeDeadlineSeconds: 60 backoffLimit: 2 selector: matchLabels: controller-uid: b2e57703-21fd-4011-8505-85353c6075f2 template: metadata: creationTimestamp: null labels: app: tendisbuild controller-uid: b2e57703-21fd-4011-8505-85353c6075f2 job-name: tendisbuild spec: containers: - name: container-j8xf02 image: 'harbor/build-tedis:v5' command: - /bin/bash - '-c' args: - cd /data1/build && sh build.sh ports: - name: http-0 containerPort: 20003 protocol: TCP env: - name: PASSWORD value: test12345 - name: DNS value: tendis-cluster-auto-9n44 - name: PODNAME value: tendis-cluster-auto - name: NAMESPACE value: tendis resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent restartPolicy: OnFailure terminationGracePeriodSeconds: 30 dnsPolicy: ClusterFirst serviceAccountName: default serviceAccount: default securityContext: {} schedulerName: default-scheduler completionMode: NonIndexed suspend: false |
---|
查看自动初始化后的集群信息
编辑