目前TKE的StorageClas 支持qcloud-cbs(云硬盘)类型的,还没有支持到cfs(nfs文件存储
),本教程通过创建相关配置文件达到StorageClass分配nfs动态卷。
官方文档可参考https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client
一、申请cfs卷
文件系统需要与云主机同在基础网络或一个私有网络下才能直接访问,请选择需要访问文件系统的云主机所在网络
二、测试挂载CFS
创建一个pv测试能否挂载上nfs
pv.yaml
代码语言:txt复制apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv-1
spec:
capacity:
storage: 4Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
nfs:
path: /
server: 10.0.0.13
已经成功挂载上,说明pv与nfs之间正常通信,接下来开始创建sts
代码语言:txt复制$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
nfs-pv-1 4Gi RWO Recycle Available 2m
三、部署NFS-Client
1.创建rabc
rabc.yaml
代码语言:txt复制kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-provisioner-runner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: run-nfs-provisioner
subjects:
- kind: ServiceAccount
name: nfs-provisioner
namespace: default
roleRef:
kind: ClusterRole
name: nfs-provisioner-runner
apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-provisioner
rules:
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-provisioner
subjects:
- kind: ServiceAccount
name: nfs-provisioner
namespace: default
roleRef:
kind: Role
name: leader-locking-nfs-provisioner
apiGroup: rbac.authorization.k8s.io
2.创建serviceaccount
serviceaccount.yaml
代码语言:txt复制apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-provisioner
3.创建deployment
deployment.yaml
代码语言:txt复制kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: nfs-provisioner
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: nfs-provisioner
spec:
serviceAccount: nfs-provisioner
containers:
- name: nfs-provisioner
image: quay.io/external_storage/nfs-client-provisioner:latest
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: example.com/nfs
- name: NFS_SERVER
value: <CFS IP地址>
- name: NFS_PATH
value: <CFS 挂载点>
volumes:
- name: nfs-client-root
nfs:
server: <CFS IP地址>
path: <CFS 挂载点>
PROVISIONER_NAME 可以修改,具体请根据个人需求修改。
4.创建storageclass
代码语言:txt复制kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: nfs
provisioner: example.com/nfs
5.创建测试claim
接下来要创建测试的claim,以检测StorageClass能否正常工作:
编写并创建test-claim.yaml如下,注意storageClassName应确保与上面创建的StorageClass名称一致。
代码语言:txt复制kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: test-claim1
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Mi
storageClassName: nfs
创建后,用kubectl get pvc查看,观察新创建的PVC能够自动绑定PV。
代码语言:txt复制$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
test-claim1 Bound pvc-093be42e-6a22-11e9-80c6-2e3a7665ed19 1Mi RWX nfs 2h
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-093be42e-6a22-11e9-80c6-2e3a7665ed19 1Mi RWX Delete Bound default/test-claim1 nfs 2h
在控制台中也可以选择已创建的test-claim1 pvc
6.创建StatefulSet案例
statefulset.yaml
代码语言:javascript复制apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: web
spec:
serviceName: "nginx1"
replicas: 2
volumeClaimTemplates:
- metadata:
name: test
annotations:
volume.beta.kubernetes.io/storage-class: "nfs"
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
template:
metadata:
labels:
app: nginx1
spec:
serviceAccount: nfs-provisioner
containers:
- name: nginx1
image: nginx
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: "/persistentvolumes"
name: test
最终可以看到
代码语言:javascript复制$ kubectl get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
nfs-provisioner 1 1 1 1 2h
nginx 1 1 1 1 3d
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
nfs-provisioner-56d4db85bb-gltp6 1/1 Running 0 3m
web-0 1/1 Running 0 2h
web-1 1/1 Running 0 2h
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
test-claim1 Bound pvc-093be42e-6a22-11e9-80c6-2e3a7665ed19 1Mi RWX nfs 2h
test-web-0 Bound pvc-95bac7cc-6a22-11e9-80c6-2e3a7665ed19 1Gi RWO nfs 2h
test-web-1 Bound pvc-9ac782e8-6a22-11e9-80c6-2e3a7665ed19 1Gi RWO nfs 2h
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
nfs-pv-1 4Gi RWO Recycle Available 1h
pvc-093be42e-6a22-11e9-80c6-2e3a7665ed19 1Mi RWX Delete Bound default/test-claim1 nfs 2h
pvc-95bac7cc-6a22-11e9-80c6-2e3a7665ed19 1Gi RWO Delete Bound default/test-web-0 nfs 2h
pvc-9ac782e8-6a22-11e9-80c6-2e3a7665ed19 1Gi RWO Delete Bound default/test-web-1 nfs 2h
官方文档可参考https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client