1、什么是K8s的弹性伸缩?
答:Hpa(全称叫做Horizontal Pod Autoscaler),Horizontal Pod Autoscaler的操作对象是Replication Controller、ReplicaSet或者Deployment对应的Pod(k8s中可以控制Pod的是rc、rs、deployment),根据观察到的CPU使用量与用户的阈值进行比对,做出是否需要增加或者减少实例数量的决策。controller目前使用heapSter来检测CPU使用量,检测周期默认是30秒。
2、K8s的弹性伸缩的工作原理?
答:Horizontal Pod Autoscaler的工作原理,主要是监控一个Pod,监控这个Pod的资源CPU使用率,一旦达到了设置的阈值,就做策略来决定它是否需要增加,做策略的时候还需要一个周期,比如,持续五分钟都发现CPU使用率高,就抓紧增加Pod的数量来减轻它的压力。当然也有一个策略,就是持续五分钟之后,压力一直都很低,那么会减少Pod的数量。这就是k8s的弹性伸缩的工作原理,主要是监控CPU的使用率,然后来决定是否增加或者减少Pod的数量。
3、K8s的弹性伸缩的实践,为了演示效果,这里对rc进行cpu资源的进行限制,方便压力测试效果。
代码语言:javascript复制 1 [root@k8s-master ~]# cd k8s/
2 [root@k8s-master k8s]# ls
3 book-master.war dashboard dashboard.zip deploy health heapster namespace pod rc skydns skydns.zip svc tomcat_demo tomcat_demo.zip
4 [root@k8s-master k8s]# mkdir hpa
5 [root@k8s-master k8s]# cd hpa/
6 [root@k8s-master hpa]# ls
7 [root@k8s-master hpa]# cp ../rc/nginx_rc.yaml .
8 [root@k8s-master hpa]# ls
9 nginx_rc.yaml
10 [root@k8s-master hpa]# vim nginx_rc.yaml
配置内容,如下所示:
代码语言:javascript复制 1 apiVersion: v1
2 kind: Pod
3 metadata:
4 name: myweb
5 labels:
6 app: web
7 env: myweb
8 spec:
9 containers:
10 - name: myweb
11 image: 192.168.110.133:5000/nginx:1.13
12 ports:
13 - containerPort: 80
14 resources:
15 # 最大可以使用的资源,100m的cpu时间片,50Mi的内存。
16 limits:
17 cpu: 100m
18 memory: 50Mi
19 # requests代表资源Pod需求的资源,100m的cpu时间片,50Mi的内存。
20 requests:
21 cpu: 100m
22 memory: 50Mi
开始创建这个RC,如下所示:
代码语言:javascript复制1 [root@k8s-master hpa]# kubectl create -f nginx_rc.yaml
2 replicationcontroller "myweb" created
3 [root@k8s-master hpa]#
查看初始的数量是两个,如下所示:
代码语言:javascript复制 1 [root@k8s-master hpa]# kubectl get all
2 NAME DESIRED CURRENT READY AGE
3 rc/myweb 2 2 2 31s
4
5 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
6 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d
7
8 NAME READY STATUS RESTARTS AGE
9 po/myweb-c0rs7 1/1 Running 0 31s
10 po/myweb-jkqc7 1/1 Running 0 31s
11 [root@k8s-master hpa]# kubectl get all -o wide
12 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR
13 rc/myweb 2 2 2 35s myweb 192.168.110.133:5000/nginx:1.13 app=myweb
14
15 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
16 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d <none>
17
18 NAME READY STATUS RESTARTS AGE IP NODE
19 po/myweb-c0rs7 1/1 Running 0 35s 172.16.16.4 k8s-node3
20 po/myweb-jkqc7 1/1 Running 0 35s 172.16.94.3 k8s-node2
21 [root@k8s-master hpa]#
4、开始创建hpa,可以通过配置文件或者命令进行声明,如下所示:
代码语言:javascript复制1 kubectl autoscale 资源类型(rc、deployment) 资源的名称 最大的pod数量 最小的pod数量 pod设定的阈值,cpu达到%多少使用率的时候就会触发hpa。
pod设定的阈值,cpu达到%多少使用率的时候就会触发hpa,这里测试的时候设置的比较低,生产环境可以设置60%、70%这些,较高些。
代码语言:javascript复制1 [root@k8s-master hpa]# kubectl autoscale replicationcontroller myweb --max=8 --min=1 --cpu-percent=5
2 replicationcontroller "myweb" autoscaled
3 [root@k8s-master hpa]#
此时,查看启动的资源,如下所示:
代码语言:javascript复制 1 [root@k8s-master hpa]# kubectl get all
2 NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
3 hpa/myweb ReplicationController/myweb 5% <waiting> 1 8 53s
4
5 NAME DESIRED CURRENT READY AGE
6 rc/myweb 2 2 2 7m
7
8 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
9 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d
10
11 NAME READY STATUS RESTARTS AGE
12 po/myweb-c0rs7 1/1 Running 0 7m
13 po/myweb-jkqc7 1/1 Running 0 7m
14 [root@k8s-master hpa]# kubectl get all -o wide
15 NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
16 hpa/myweb ReplicationController/myweb 5% <waiting> 1 8 56s
17
18 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR
19 rc/myweb 2 2 2 7m myweb 192.168.110.133:5000/nginx:1.13 app=myweb
20
21 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
22 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d <none>
23
24 NAME READY STATUS RESTARTS AGE IP NODE
25 po/myweb-c0rs7 1/1 Running 0 7m 172.16.16.4 k8s-node3
26 po/myweb-jkqc7 1/1 Running 0 7m 172.16.94.3 k8s-node2
27 [root@k8s-master hpa]#
这里可以看到由hpa控制rc,rc来控制pod的数量,现在开始进行压力测试,这里使用的ab的命令,首先查询到这个pod的ip地址,如下所示:
代码语言:javascript复制 1 [root@k8s-master hpa]# kubectl get all -o wide
2 NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
3 hpa/myweb ReplicationController/myweb 5% <waiting> 1 8 3m
4
5 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR
6 rc/myweb 2 2 2 10m myweb 192.168.110.133:5000/nginx:1.13 app=myweb
7
8 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
9 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d <none>
10
11 NAME READY STATUS RESTARTS AGE IP NODE
12 po/myweb-c0rs7 1/1 Running 0 10m 172.16.16.4 k8s-node3
13 po/myweb-jkqc7 1/1 Running 0 10m 172.16.94.3 k8s-node2
14 [root@k8s-master hpa]#
首先安装一下httpd,安装了这个就包含了ab的命令了,如下所示:
代码语言:javascript复制 1 [root@k8s-master ~]# yum install httpd-tools.x86_64 -y
2 Loaded plugins: fastestmirror, langpacks, product-id, search-disabled-repos, subscription-manager
3
4 This system is not registered with an entitlement server. You can use subscription-manager to register.
5
6 Determining fastest mirrors
7 * base: mirrors.tuna.tsinghua.edu.cn
8 * extras: mirrors.bfsu.edu.cn
9 * updates: mirrors.bfsu.edu.cn
10 base | 3.6 kB 00:00:00
11 extras | 2.9 kB 00:00:00
12 updates | 2.9 kB 00:00:00
13 updates/7/x86_64/primary_db | 2.9 MB 00:00:03
14 Resolving Dependencies
15 --> Running transaction check
16 ---> Package httpd-tools.x86_64 0:2.4.6-93.el7.centos will be installed
17 --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-tools-2.4.6-93.el7.centos.x86_64
18 --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-tools-2.4.6-93.el7.centos.x86_64
19 --> Running transaction check
20 ---> Package apr.x86_64 0:1.4.8-5.el7 will be installed
21 ---> Package apr-util.x86_64 0:1.5.2-6.el7 will be installed
22 --> Finished Dependency Resolution
23
24 Dependencies Resolved
25
26 =================================================================================================================================================================================================================
27 Package Arch Version Repository Size
28 =================================================================================================================================================================================================================
29 Installing:
30 httpd-tools x86_64 2.4.6-93.el7.centos base 92 k
31 Installing for dependencies:
32 apr x86_64 1.4.8-5.el7 base 103 k
33 apr-util x86_64 1.5.2-6.el7 base 92 k
34
35 Transaction Summary
36 =================================================================================================================================================================================================================
37 Install 1 Package ( 2 Dependent packages)
38
39 Total download size: 288 k
40 Installed size: 584 k
41 Downloading packages:
42 (1/3): httpd-tools-2.4.6-93.el7.centos.x86_64.rpm | 92 kB 00:00:00
43 (2/3): apr-util-1.5.2-6.el7.x86_64.rpm | 92 kB 00:00:00
44 (3/3): apr-1.4.8-5.el7.x86_64.rpm | 103 kB 00:00:00
45 -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
46 Total 273 kB/s | 288 kB 00:00:01
47 Running transaction check
48 Running transaction test
49 Transaction test succeeded
50 Running transaction
51 Installing : apr-1.4.8-5.el7.x86_64 1/3
52 Installing : apr-util-1.5.2-6.el7.x86_64 2/3
53 Installing : httpd-tools-2.4.6-93.el7.centos.x86_64 3/3
54 Verifying : apr-1.4.8-5.el7.x86_64 1/3
55 Verifying : httpd-tools-2.4.6-93.el7.centos.x86_64 2/3
56 Verifying : apr-util-1.5.2-6.el7.x86_64 3/3
57
58 Installed:
59 httpd-tools.x86_64 0:2.4.6-93.el7.centos
60
61 Dependency Installed:
62 apr.x86_64 0:1.4.8-5.el7 apr-util.x86_64 0:1.5.2-6.el7
63
64 Complete!
65 [root@k8s-master ~]#
压力测试,总共发起500000次请求,每次发起30个请求,如下所示:
代码语言:javascript复制1 [root@k8s-master ~]# ab -n 500000 -c 30 http://172.16.94.3/index.html/
Kubernetes dashboard的界面,现在,如下所示:
由于我的是启动了两个Pod,这里我也同时压力测试两个Pod,第二个Pod的压力测试如下所示:
代码语言:javascript复制1 [root@k8s-master hpa]# ab -n 500000 -c 30 http://172.16.16.4/index.html/
2 This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
3 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
4 Licensed to The Apache Software Foundation, http://www.apache.org/
5
6 Benchmarking 172.16.16.4 (be patient)
此时的CPU使用情况,如下所示:
可以使用命令查看hpa调度的情况,如下所示:
注意:我后来观察了一下下面的信息,发现没有进行Pod的伸缩的报错原因,原来是下面的错误,如下所示:
代码语言:javascript复制1 FailedGetMetrics unable to get metrics for resource cpu: failed to get heapster service: the server could not find the requested resource (get services http:heapster:)
具体查看详情信息,如下所示:
代码语言:javascript复制 1 [root@k8s-master rc]# kubectl describe hpa myweb
2 Name: myweb
3 Namespace: default
4 Labels: <none>
5 Annotations: <none>
6 CreationTimestamp: Sun, 28 Jun 2020 19:47:19 0800
7 Reference: ReplicationController/myweb
8 Target CPU utilization: 5%
9 Current CPU utilization: <unset>
10 Min replicas: 1
11 Max replicas: 8
12 ReplicationController pods: 2 current / 2 desired
13 Events:
14 FirstSeen LastSeen Count From SubObjectPath Type Reason Message
15 --------- -------- ----- ---- ------------- -------- ------ -------
16 16m 14m 8 {horizontal-pod-autoscaler } Normal MetricsNotAvailableYet unable to get metrics for resource cpu: failed to get heapster service: the server could not find the requested resource (get services http:heapster:)
17 13m 17s 28 {horizontal-pod-autoscaler } Warning FailedGetMetrics unable to get metrics for resource cpu: failed to get heapster service: the server could not find the requested resource (get services http:heapster:)
18 [root@k8s-master rc]#
我对两个Pod同时压力测试都没有伸缩,没有给我扩容Pod,更别提缩减Pod。
我这里一直测试不出,这里将值调整的更低些,看看效果,如何,先将rc,hpa删除掉哈,然后再创建即可,如下所示:
代码语言:javascript复制 1 [root@k8s-master ~]# kubectl get all -o wide
2 NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
3 hpa/myweb ReplicationController/myweb 5% <waiting> 1 8 38m
4
5 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR
6 rc/myweb 2 2 2 45m myweb 192.168.110.133:5000/nginx:1.13 app=myweb
7
8 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
9 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d <none>
10
11 NAME READY STATUS RESTARTS AGE IP NODE
12 po/myweb-c0rs7 1/1 Running 0 45m 172.16.16.4 k8s-node3
13 po/myweb-jkqc7 1/1 Running 0 45m 172.16.94.3 k8s-node2
14 [root@k8s-master ~]# kubectl get rc -o wide
15 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR
16 myweb 2 2 2 45m myweb 192.168.110.133:5000/nginx:1.13 app=myweb
17 [root@k8s-master ~]# kubectl delete rc myweb
18 replicationcontroller "myweb" deleted
19 [root@k8s-master ~]# kubectl get rc -o wide
20 No resources found.
21 [root@k8s-master ~]# kubectl get all -o wide
22 NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
23 hpa/myweb ReplicationController/myweb 5% <waiting> 1 8 38m
24
25 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
26 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d <none>
27 [root@k8s-master ~]# kubectl get hpa -o wide
28 NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE
29 myweb ReplicationController/myweb 5% <waiting> 1 8 39m
30 [root@k8s-master ~]# kubectl delete hpa myweb
31 horizontalpodautoscaler "myweb" deleted
32 [root@k8s-master ~]# kubectl get hpa -o wide
33 No resources found.
34 [root@k8s-master ~]# kubectl get all -o wide
35 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
36 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d <none>
37 [root@k8s-master ~]#
测不出来,k8s的hpa伸缩和缩减Pod的现象,不管修改内存、CPU参数,还是请求参数,先放到这里吧,以后有机会再来看看。这里没有完成伸缩的具体原因是因为下面这段话。
metrics-server,从 Kubernetes 1.8开始,资源使用指标,例如容器 CPU 和内存使用率,可通过 Metrics API 在 Kubernetes 中获得。此 API 不存储指标值,因此想要获取某个指定节点10分钟前的资源使用量是不可能的。在此之前,kubernetes对容器的监控是通过hepater来完成的。之所以Metrics-server会替换掉hepater成为新的k8s 监控组件,在于其使得kubernetes在监控方面与其他功能保持了一致,不再像是一个割裂开的功能,比如风格统一的监控指标api,kubectl top命令等。这里顺便提一下另外一个组件cadvisor,该监控组件本身并非kubernetes内置,但kubelete内置了部分cadvisor功能,因此kubelete可以获取每个节点的容器监控信息。
5、查看一下我的k8s的版本,如下所示:
代码语言:javascript复制1 [root@k8s-master ~]# kubectl version
2 Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"269f928217957e7126dc87e6adfa82242bfe5b1e", GitTreeState:"clean", BuildDate:"2017-07-03T15:31:10Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
3 Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"269f928217957e7126dc87e6adfa82242bfe5b1e", GitTreeState:"clean", BuildDate:"2017-07-03T15:31:10Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
4 [root@k8s-master ~]#
由于heapster停止更新,这里也使用metrics-server来进行监控了。
代码语言:javascript复制 1 [root@k8s-master ~]# cd k8s/
2 [root@k8s-master k8s]# ls
3 book-master.war dashboard dashboard.zip deploy health heapster hpa namespace pod rc skydns skydns.zip svc tomcat_demo tomcat_demo.zip
4 [root@k8s-master k8s]# mkdir metrics
5 [root@k8s-master k8s]# cd metrics/
6 [root@k8s-master metrics]# ls
7 [root@k8s-master metrics]# wget -c https://github.com/kubernetes-sigs/metrics-server/archive/v0.3.6.zip
8 --2020-06-29 10:24:47-- https://github.com/kubernetes-sigs/metrics-server/archive/v0.3.6.zip
9 Resolving github.com (github.com)... 13.250.177.223
10 Connecting to github.com (github.com)|13.250.177.223|:443... connected.
11 HTTP request sent, awaiting response... 302 Found
12 Location: https://codeload.github.com/kubernetes-sigs/metrics-server/zip/v0.3.6 [following]
13 --2020-06-29 10:24:49-- https://codeload.github.com/kubernetes-sigs/metrics-server/zip/v0.3.6
14 Resolving codeload.github.com (codeload.github.com)... 54.251.140.56
15 Connecting to codeload.github.com (codeload.github.com)|54.251.140.56|:443... connected.
16 HTTP request sent, awaiting response... 200 OK
17 Length: unspecified [application/zip]
18 Saving to: ‘v0.3.6.zip’
19
20 [ <=> ] 9,374,036 183KB/s in 69s
21
22 2020-06-29 10:26:01 (132 KB/s) - ‘v0.3.6.zip’ saved [9374036]
23
24 [root@k8s-master metrics]#
查看下载的zip包并进行解压缩操作,如下所示:
代码语言:javascript复制1 [root@k8s-master metrics]# unzip v0.3.6.zip
2
3 [root@k8s-master metrics]# ls
4 metrics-server-0.3.6 v0.3.6.zip
5 [root@k8s-master metrics]# ll
6 total 9156
7 drwxr-xr-x 8 root root 335 Oct 14 2019 metrics-server-0.3.6
8 -rw-r--r-- 1 root root 9374036 Jun 29 10:26 v0.3.6.zip
9 [root@k8s-master metrics]#
查看自己的jdk版本,并进入到指定的目录,如下所示:
代码语言:javascript复制 1 [root@k8s-master metrics]# ls
2 metrics-server-0.3.6 v0.3.6.zip
3 [root@k8s-master metrics]# cd metrics-server-0.3.6/deploy/1.
4 1.7/ 1.8 /
5 [root@k8s-master metrics]# cd metrics-server-0.3.6/deploy/1.
6 1.7/ 1.8 /
7 [root@k8s-master metrics]# cd metrics-server-0.3.6/deploy/1.8 /
8 [root@k8s-master 1.8 ]# ls
9 aggregated-metrics-reader.yaml auth-delegator.yaml auth-reader.yaml metrics-apiservice.yaml metrics-server-deployment.yaml metrics-server-service.yaml resource-reader.yaml
10 [root@k8s-master 1.8 ]# java -version
11 openjdk version "1.8.0_181"
12 OpenJDK Runtime Environment (build 1.8.0_181-b13)
13 OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)
14 [root@k8s-master 1.8 ]#
现在开始修改镜像地址,如下所示:
首先将需要的镜像下载下来,并上传到自己的私有仓库里面,如下所示:
代码语言:javascript复制 1 [root@k8s-master ~]# docker pull docker.io/htcfive/metrics-server-amd64
2 Using default tag: latest
3 Trying to pull repository docker.io/htcfive/metrics-server-amd64 ...
4 sha256:0122b32b24dcb04ac5131cecdefd8abce0c8a3359605dd17b394acc9fc49de1d: Pulling from docker.io/htcfive/metrics-server-amd64
5 e8d8785a314f: Pull complete
6 b2f4b24bed0d: Pull complete
7 Digest: sha256:0122b32b24dcb04ac5131cecdefd8abce0c8a3359605dd17b394acc9fc49de1d
8 Status: Downloaded newer image for docker.io/htcfive/metrics-server-amd64:latest
9 [root@k8s-master ~]# docker images docker.io/htcfive/metrics-server-amd64
10 REPOSITORY TAG IMAGE ID CREATED SIZE
11 docker.io/htcfive/metrics-server-amd64 latest 77da73af4258 6 months ago 39.9 MB
12 [root@k8s-master ~]# docker tag docker.io/htcfive/metrics-server-amd64:latest 192.168.110.133:5000/docker.io/htcfive/metrics-server-amd64:latest
13 [root@k8s-master ~]# docker push 192.168.110.133:5000/docker.io/htcfive/metrics-server-amd64:latest
14 The push refers to a repository [192.168.110.133:5000/docker.io/htcfive/metrics-server-amd64]
15 7bf3709d22bb: Pushed
16 932da5156413: Pushed
17 latest: digest: sha256:0122b32b24dcb04ac5131cecdefd8abce0c8a3359605dd17b394acc9fc49de1d size: 738
18 [root@k8s-master ~]#
修改镜像地址,metrics-server默认使用node的主机名,但是coredns里面没有物理机主机名的解析,部署的时候添加一个参数--kubelet-preferred-address-types=InternalIP,Hostname,InternalDNS,ExternalDNS,ExternalIP。这里直接通过InternalIP进行访问,忽略客户端证书kubelet-insecure-tls。
代码语言:javascript复制 1 [root@k8s-master 1.8 ]# cat metrics-server-deployment.yaml
2 ---
3 apiVersion: v1
4 kind: ServiceAccount
5 metadata:
6 name: metrics-server
7 namespace: kube-system
8 ---
9 apiVersion: apps/v1
10 kind: Deployment
11 metadata:
12 name: metrics-server
13 namespace: kube-system
14 labels:
15 k8s-app: metrics-server
16 spec:
17 selector:
18 matchLabels:
19 k8s-app: metrics-server
20 template:
21 metadata:
22 name: metrics-server
23 labels:
24 k8s-app: metrics-server
25 spec:
26 serviceAccountName: metrics-server
27 volumes:
28 # mount in tmp so we can safely use from-scratch images and/or read-only containers
29 - name: tmp-dir
30 emptyDir: {}
31 containers:
32 - name: metrics-server
33 # image: k8s.gcr.io/metrics-server-amd64:v0.3.6
34 image: 192.168.110.133:5000/docker.io/htcfive/metrics-server-amd64:latest
35 # imagePullPolicy: Always
36 imagePullPolicy: IfNotPresent
37 command:
38 - /metrics-server
39 - --kubelet-preferred-address-types=InternalIP
40 - --kubelet-insecure-tls
41 volumeMounts:
42 - name: tmp-dir
43 mountPath: /tmp
创建metrics-server报错了,先放置吧,搜了一堆也没有解决,后续再来看看吧。
代码语言:javascript复制 1 [root@k8s-master 1.8 ]# kubectl create -f .
2 Error from server (BadRequest): error when creating "aggregated-metrics-reader.yaml": ClusterRole in version "v1" cannot be handled as a ClusterRole: no kind "ClusterRole" is registered for version "rbac.authorization.k8s.io/v1"
3 Error from server (BadRequest): error when creating "auth-delegator.yaml": ClusterRoleBinding in version "v1beta1" cannot be handled as a ClusterRoleBinding: no kind "ClusterRoleBinding" is registered for version "rbac.authorization.k8s.io/v1beta1"
4 Error from server (BadRequest): error when creating "auth-reader.yaml": RoleBinding in version "v1beta1" cannot be handled as a RoleBinding: no kind "RoleBinding" is registered for version "rbac.authorization.k8s.io/v1beta1"
5 Error from server (AlreadyExists): error when creating "metrics-server-deployment.yaml": serviceaccounts "metrics-server" already exists
6 Error from server (AlreadyExists): error when creating "metrics-server-service.yaml": services "metrics-server" already exists
7 Error from server (BadRequest): error when creating "resource-reader.yaml": ClusterRole in version "v1" cannot be handled as a ClusterRole: no kind "ClusterRole" is registered for version "rbac.authorization.k8s.io/v1"
8 Error from server (BadRequest): error when creating "resource-reader.yaml": ClusterRoleBinding in version "v1" cannot be handled as a ClusterRoleBinding: no kind "ClusterRoleBinding" is registered for version "rbac.authorization.k8s.io/v1"
9 [unable to recognize "metrics-apiservice.yaml": no matches for apiregistration.k8s.io/, Kind=APIService, unable to recognize "metrics-server-deployment.yaml": no matches for apps/, Kind=Deployment]
10 [root@k8s-master 1.8 ]#