上一篇写了如何搭建helm ,TKE搭建helm搭建
由于目前TKE已经集成了helm,用户只需在控制台点击安装便会下发tiller
、swift
已省去了搭建的过程,非常方便,但是目前共有的chart不一定能满足所有人的需求,所以本篇以拓展应用为主,讲述如何在TKE集群中通过集成的helm搭建应用
接下来进入主题,如何用helm搭建rancher
Rancher是什么Rancher是一个开源的企业级容器管理平台。通过Rancher,企业再也不必自己使用一系列的开源软件去从头搭建容器服务平台
官方文档:https://www.cnrancher.com/docs/rancher/v2.x/cn/installation/server-tags/
一、下载helm客户端至节点
由于控制台的安装只下发了tiller、swift,还需手动下载helm client
代码语言:txt复制$ curl -O https://storage.googleapis.com/kubernetes-helm/helm-v2.10.0-linux-amd64.tar.gz
$ tar xzvf helm-v2.10.0-linux-amd64.tar.gz
$ sudo cp linux-amd64/helm /usr/local/bin/helm
注意,这里默认的是v2.10.0版本,client与server的版本务必对齐不然会失败。
代码语言:txt复制$ helm version
Client: &version.Version{SemVer:"v2.10.0", GitCommit:"9ad53aac42165a5fadc6c87be0dea6b115f93090", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.10.0", GitCommit:"9ad53aac42165a5fadc6c87be0dea6b115f93090", GitTreeState:"clean"}
二、初始化helm
当tiller已存在的时候需加上--client-only
代码语言:txt复制You might need to run
helm init
(orhelm init --client-only
if tiller is already installed)
$ helm init --client-only
Creating /root/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /root/.helm.
Not installing Tiller due to 'client-only' flag having been set
Happy Helming!
三、添加rancher仓库
类型 | 添加仓库命令 | 仓库描述 |
---|---|---|
rancher-latest | helm repo add rancher-latest https://releases.rancher.com/server-charts/latest | Rancher server最新版Helm charts仓库,建议此仓库版本用于测试环境。 |
rancher-stable | helm repo add rancher-stable https://releases.rancher.com/server-charts/stable | Rancher server稳定版Helm charts仓库,此仓库版本推荐用于生产环境。 |
添加stable版本
代码语言:txt复制$ helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
"rancher-stable" has been added to your repositories
查看repo
代码语言:txt复制$ helm repo list
NAME URL
stable https://kubernetes-charts.storage.googleapis.com
local http://127.0.0.1:8879/charts
rancher-stable https://releases.rancher.com/server-charts/stable
四、安装cert-manager
代码语言:txt复制helm install stable/cert-manager
--name cert-manager
--namespace kube-system
--version v0.5.2
五、安装rancher-stable
代码语言:txt复制helm install rancher-stable/rancher
--name rancher
--namespace cattle-system
--set hostname=rancher.my.org
------------------------------------------------
$ helm install rancher-stable/rancher --name rancher --namespace cattle-system --set hostname=rancher.my.org
NAME: rancher
LAST DEPLOYED: Wed May 8 12:13:27 2019
NAMESPACE: cattle-system
STATUS: DEPLOYED
RESOURCES:
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
rancher ClusterIP 10.3.255.165 <none> 80/TCP 0s
==> v1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
rancher 3 0 0 0 0s
==> v1beta1/Ingress
NAME HOSTS ADDRESS PORTS AGE
rancher rancher.my.org 80, 443 0s
==> v1alpha1/Issuer
NAME AGE
rancher 0s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
rancher-7bc85fd5ff-98mv6 0/1 ContainerCreating 0 0s
rancher-7bc85fd5ff-m87m2 0/1 ContainerCreating 0 0s
rancher-7bc85fd5ff-s54fx 0/1 ContainerCreating 0 0s
==> v1/ServiceAccount
NAME SECRETS AGE
rancher 1 0s
==> v1/ClusterRoleBinding
NAME AGE
rancher 0s
NOTES:
Rancher Server has been installed.
NOTE: Rancher may take several minutes to fully initialize. Please standby while Certificates are being issued and Ingress comes up.
Check out our docs at https://rancher.com/docs/rancher/v2.x/en/
Browse to https://rancher.my.org
Happy Containering!
输入以上指令后会有详细信息过程的反馈,从反馈中可以看到rancher的chart创建了Service、Deployment、Ingress、Issuer、ServiceAccount、ClusterRoleBinding
通过查看pod可以发现 rancher已经以pod形式启动
代码语言:txt复制$kubectl -n cattle-system rollout status deploy/rancher
Waiting for deployment "rancher" rollout to finish: 0 of 3 updated replicas are available...
deployment "rancher" successfully rolled out
$ kubectl get pod -n cattle-system
NAME READY STATUS RESTARTS AGE
rancher-7bc85fd5ff-98mv6 1/1 Running 1 4m
rancher-7bc85fd5ff-m87m2 1/1 Running 1 4m
rancher-7bc85fd5ff-s54fx 1/1 Running 1 4m
创建完毕后需要做一些修改,符合TKE特性才能正常访问
六、修改配置
a. 修改rancher service type
rancher chart包创建的service type类型默认是clusterIP,需要修改为nodeport形式,否则ingress将无法转发请求至service
选择更新访问方式,选择主机端口访问,主机端口若不填写将从30000-32767中随机挑选一个。
b. 修改ingress
来到ingress的界面中可以看到ingress的 vip 一直在创建中
通过查看事件发现有warning事件:Error during sync: Secret tls-rancher-ingress has no qcloud cert id
说明这个证书没找到,需要将rancher的证书添加至腾讯云中
c. 添加证书至腾讯云中
在rancher v2.x版本后默认是配置了ssl证书的,在第四步中创建的cert-manager就是给rancher颁发证书
Rancher Server is designed to be secure by default and requires SSL/TLS configuration.
获取证书
代码语言:txt复制$ kubectl get secret tls-rancher-ingress -n cattle-system -o yaml
apiVersion: v1
data:
tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURIakNDQWdhZ0F3SUJBZ0lSQU8rQUhOcFgvR0FPbEt1LzdBVmtWeTB3RFFZSktvWklodmNOQVFFTEJRQXcKS0RFU01CQUdBMVVFQ2hNSmRHaGxMWEpoYm1Ob01SSXdFQVlEVlFRREV3bGpZWFIwYkdVdFkyRXdIaGNOTVRrdwpOVEE0TURReE5ETTFXaGNOTWpBd05UQTNNRFF4TkRNMVdqQXdNUlV3RXdZRFZRUUtFd3hqWlhKMExXMWhibUZuClpYSXhGekFWQmdOVkJBTVREbkpoYm1Ob1pYSXViWGt1YjNKbk1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0MKQVE4QU1JSUJDZ0tDQVFFQXJWVjJaUzFSVWFaMG9XWDNORnVRUVRvZFUzTExqL3dqN3JDdXZTMEtWeGlIMWp3TQoxd1VwYmlYU0VZSzRhNXVYRUlPcWZ2UVhYcVhMSTFGWU8wQUswYzkxdmZCUDdUWE13MXRrc01nRXRmUmNaMkt1CllleE5HajJEdDYrbUEvVk9FcVZsc2x1SDd5c0FPYllyYXQ0RmdVNmtWOUpQeGtzVzNmRys5ekVzZmIvSGh6TXAKVHFJVkh1OE1PbVhubjVHeG9qZDdHTGRTTlRyUmllVUh0Y2d4M0tmVW1TVEFHaFAvZjhrL2RMT0ZHRjlVd2J2ZApqa2xsUzBnV3VFSk12cklYamU1ek53Ylc2Q2Z4QjJuQ3VEMHlSbDRXcmlYZmN1T1hPV2ZtbjQrVmpsM2ptdmRvCkhRVjh2UXVNcXhKdm5xTkFuckN5Q3RTWTQyU3lZZnFSRXk4QkxRSURBUUFCb3pzd09UQU9CZ05WSFE4QkFmOEUKQkFNQ0JhQXdEQVlEVlIwVEFRSC9CQUl3QURBWkJnTlZIUkVFRWpBUWdnNXlZVzVqYUdWeUxtMTVMbTl5WnpBTgpCZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFveVpTVWM5bGRTcktybEFCQy9DNzA4bXczcVJaWEp3RmZ3K1NtTDkwCmRzTHR0ZU5OWnhNUDZvWDFISHcyYml4Yy9wbE4xRVJhYisxbWJ1NUJ0RW1zY1d0ZFJZbDc3dDJUcHh1VkxhY0wKRHlMcFhOUXRPc1NHN25mM2tMeU5kdjlhNnc0SXpTcUhCUnZCWGdHRktVSDdVWG1aM0hLWEFxaHE3N0RoTC9VRgpxYkVWQm02SUVyT3R2cjB0T3N5WEpOdHNGelhEVFFTVGNCcEszSkQ5akg5Q25nbHdjRHk1a0NXd2JwQVh4bGhhCmJiMTJSK0w1WCtNdVNMbEpFVUcybitJakZHS08xSG9OTGlPZ1pDdFArbm1BTXpRYTR3YTc0MDNRcEhRd3ptTEEKMm1Hbmx0bXFuYW5CcnBXcGREa2puZkxRYk9wZzNqLzFOV3M2TGxzRk02V1VnUT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0KLS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM3akNDQWRhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFvTVJJd0VBWURWUVFLRXdsMGFHVXQKY21GdVkyZ3hFakFRQmdOVkJBTVRDV05oZEhSc1pTMWpZVEFlRncweE9UQTFNRGd3TkRFek5UQmFGdzB5T1RBMQpNRFV3TkRFek5UQmFNQ2d4RWpBUUJnTlZCQW9UQ1hSb1pTMXlZVzVqYURFU01CQUdBMVVFQXhNSlkyRjBkR3hsCkxXTmhNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTFyYkdUL1grRXZ4TWtxaEUKYTQxYnVQdE5YL3pmTFZiUndZYTFVVmp3QXJPc1ZNTEJWMjg5Q1c0VGR6UHJLdUVBMTA1NUdKZlRKTEJ2aThqYgpEdCtrbTcvYTk1K2Y5Vm5aaDZaT2VFY0tiaHdnei9lRDJuS2VZVUlPN0JhSEV1VktjQnBsd3RjUUFKVWRUZTl6CjFRVkFGU2l4aThMeW5KSDlSWTRtZWhDcmh1N2szMlA1OFFUQWxrNVA3aFE5ZGYrTkx1UWhaVjhPa3VxQTNRb0kKTGtpQVhoRkVrRWhDTVpLcy8rU2JkRlZxeHRIbXRFSjJtc2I2M2oxdjltcld6WkhEbmhQS3BoNkJ5aW0ralBiNQpJUFltSUJnRkZRRGlqVVljN0RYaG1ZQjgwbXZlbmQ3eDFHamFWUzgxSllueWoyQlRQQlg5cFMzN2JsMTgyd2FBClFFczhSd0lEQVFBQm95TXdJVEFPQmdOVkhROEJBZjhFQkFNQ0FxUXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QU4KQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBMWIxVFVYUjNKNDNuU0p1azdOY3lDdTVRQXdIR3lkTWFqeTVwMGZRbgpwUE80d0tEaW1VQnQ2RHRVTGpwQUJML25GYWhCY3lpdEdoNDB1SXpIdHZiRFdkczZ5cFd5S2ZPY2Nub25mTzA1CnBxVDlzS0I4L1F2aldDWXlHODQ1akRpUFdmeUdpMHRySXRLMC9UaStDbENXaTlNYjg0UlpYT2VBbjR3SHVaTjQKU3pCWGJ2MUJIelVrNHZRelhVVjE1YXJRRU1CcmUvQkkzSkVIT3kxRFlaYmk2MHFtekN1RzhheVl2eXA2VEFXOApwWXdsSklZZlI3ZHBjdUJqWnpvNmlEb2NxZFlQZkhrVklIcHZ4am9EZGpXYjcwbEkwYW9Ra3RPeExoc2oreE9aCmVnWTdFOFJsWDNkdmlRRW51VU5EdGhvSEw2USs3Y0ZqVWtTSk9OdUhpMzUzR1E9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcFFJQkFBS0NBUUVBclZWMlpTMVJVYVowb1dYM05GdVFRVG9kVTNMTGovd2o3ckN1dlMwS1Z4aUgxandNCjF3VXBiaVhTRVlLNGE1dVhFSU9xZnZRWFhxWExJMUZZTzBBSzBjOTF2ZkJQN1RYTXcxdGtzTWdFdGZSY1oyS3UKWWV4TkdqMkR0NittQS9WT0VxVmxzbHVIN3lzQU9iWXJhdDRGZ1U2a1Y5SlB4a3NXM2ZHKzl6RXNmYi9IaHpNcApUcUlWSHU4TU9tWG5uNUd4b2pkN0dMZFNOVHJSaWVVSHRjZ3gzS2ZVbVNUQUdoUC9mOGsvZExPRkdGOVV3YnZkCmprbGxTMGdXdUVKTXZySVhqZTV6TndiVzZDZnhCMm5DdUQweVJsNFdyaVhmY3VPWE9XZm1uNCtWamwzam12ZG8KSFFWOHZRdU1xeEp2bnFOQW5yQ3lDdFNZNDJTeVlmcVJFeThCTFFJREFRQUJBb0lCQUdiaU4yaE93ZG5wSjRrMQpjTW9QSDB6WUpkZ0daM1dMZi9pSnZ1NWk4YVJJMXpmK1pOQ3NLNjlKbTZyeElHNGU2RWZSSHo1RHZDdGQxWG1kCkt2WGdTWXZIekdkMVlMdTRGMVpOeFRKNVJ2S0puTjl3MzNrWVBadTVyNWswRlNKOFZGOFU5WXdzY0Q4eUV1d3UKaVE3aHdmckNweUhKbjdoZmthOGpjdXBlc2IzaFFZZTArbkJYNHNTd3RvMFJLWThLaDUxaFNucVl2ekRLa2l1bApUenpCb0VuK0tlQ0o5RzlkRG5CQjZtcVRhUlpMUGYwL2NuYUtWQ0kvUThYMUx1OVFQVG9iWkNtcEoraEZzalVsCjZlR2hsUGMrK25FcGN4c3hyeUprMENuV0lZWDU3WGRyaG1HbFEzSVNLNHJlayt5VkVYa2lBLyt6SWlnelIzTkYKcWs1bm93RUNnWUVBMGxIdVpGbFlzSkR5TDhxdWRXM2dzUHhQRzJKR0Y1bEdINnRLT0VmSGN1R1pTMHZMV0NLMwpCYlZaaU81MXByUStobUVnUzY1OTFLWU4xbUVNam9ORGNnQXJlZ1ZDVTg0VDRlcXl5OGxpc0l5emV3UDFUdHM1CkcvcEtHSnNycVMvSVVNVC83eFJQVXR3c3hlQmR2cUlpZWZnT0JlUVJYVldxTnplNHZBOHhsOTBDZ1lFQTB2c00KeW1ZRVJhV0dWRnM5REllamJhRXdsTThJcjFoNTM5RUdaMGRham1wa0NZalE2UVlSaFFKZlVtbE10N1hJWWQzego0M0JjWWE0R0VHV1RsUHExQ0lRbU9kYklsUDRqWFB4alhjSFgyM1ZBUDVVMTAreTlWeUV2dmFjT2xkNVB2V3RFCjlXRm1Mek5BNEFlVVBlNFk2aVc3Q253cHdJVWpMSENXV1FlT29aRUNnWUVBcjBuajB6VDRDeWl6QVF2K3ZSWlgKeDRweDhCUEc4VklvdEtwUlVIamFqUy9qVUhNYVJobWd5cHdyeEVsaU93OU1jVGt2N29PK2tlWFg4STJ3Sm5MVwpXb1F6dUIzNyt0YnRrVVBId3JyVWl1cXBoK1J6cGtabDRhQ0g4dHdZMW9oaGZwZkI2eVBWUE8wT1Q5bEFxR3AvCks0S0hSWGVnMVZaWmFHNERoSkZ4TnUwQ2dZRUFwdXpTZGVDNVNKRnA4cHd0ZFJoc0NYVWJCdUczTVdTREtMNHMKbnRJcHA3d3RZZzNKbkZTY3ordk1sMXVRdzFra0xhcDZLRnpoK3JiQlBYL2dnR0xMa08xZGdTMmFqYWQxTzBERgpMWTBNeUNuQXA0M2FoTTR1RUIzRnMvdlU0bVorR0pRaWZ6S01zcE9PcExIYWJCVlJ5UkFKS0lhTVVqSUVYa2pGCjY3SzYrc0VDZ1lFQWkyMEhTWGwzK2p1UG11OXBWNDBmbDdtbllLUEkwUVZBUDNqdThwdWg2cWN2SmJCUFBHMVkKRzNmeHRDcnlXU1BEbGxGbFhtMlN1ZkpVQUI3QkdRZjBsWjJGZTg4V3hVRHQ2Y1NYbXhwL3RGemhpMmFqTGNkbQpEd3dWYlZJaHhSYVIyVThUNTZsclNXaE1kSFpYWjVKVXp6NStnTnNkT21mZWFpQkJLdURPUHQ4PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
kind: Secret
metadata:
annotations:
certmanager.k8s.io/alt-names: rancher.my.org
certmanager.k8s.io/common-name: rancher.my.org
certmanager.k8s.io/issuer-kind: Issuer
certmanager.k8s.io/issuer-name: rancher
creationTimestamp: 2019-05-08T04:14:35Z
labels:
certmanager.k8s.io/certificate-name: tls-rancher-ingress
name: tls-rancher-ingress
namespace: cattle-system
resourceVersion: "8671"
selfLink: /api/v1/namespaces/cattle-system/secrets/tls-rancher-ingress
uid: c9a726b7-7147-11e9-ade3-5254009b5518
type: kubernetes.io/tls
将data中的tls.crt 、tls.key用base解码至正常格式,添加进证书中
在ingress界面点击更新转发配置,勾选https:443
将会弹出服务器证书的选择框,此时我们点击现在创建,进入到证书管理界面
解码
将data中的tls.crt 、tls.key用base64解码至正常格式,添加进证书中
填写证书信息
修改ingress配置
回到ingress界面,选择上一步创建的证书,配置转发规则,由于是测试的搭建,所以域名这里也不填写了。
访问rancher
在修改完ingress后,vip已创建出来
通过访问vip 111.230.121.109,可以看到浏览器会提示不是安全的链接,因为这里没有用域名访问且没有对域名等进行配置,在一开始install rancher时 hostname=rancher.my.org,所以后续如果有域名需求的,在一开始install时就要把域名等信息配置好。
继续访问就可以看到进入rancher的界面了,首次访问需要给admin设置密码
设置你的URL,这里是测试所以不配置域名了。
进去后就可以看到rancher的界面了,底下还可以选择中文界面。
通过rancher安装prometheus grafana
点击集群中,点击启用监控并查看实时监控指标
保存配置后,将会创建相关的pod,由于选择了持久化存储,也会创建对应的pvc。
代码语言:txt复制$ kubectl get pod --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
cattle-prometheus exporter-kube-state-cluster-monitoring-565c8854d7-56x8m 1/1 Running 0 2m
cattle-prometheus exporter-node-cluster-monitoring-2x9hn 1/1 Running 0 2m
cattle-prometheus exporter-node-cluster-monitoring-d5p9d 1/1 Running 0 2m
cattle-prometheus exporter-node-cluster-monitoring-kf8q2 1/1 Running 0 2m
cattle-prometheus exporter-node-cluster-monitoring-kqspd 1/1 Running 0 2m
cattle-prometheus exporter-node-cluster-monitoring-lrqnw 1/1 Running 0 2m
cattle-prometheus exporter-node-cluster-monitoring-x2ncf 1/1 Running 0 2m
cattle-prometheus grafana-cluster-monitoring-65c9c7c69f-g2mhb 2/2 Running 0 2m
cattle-prometheus prometheus-cluster-monitoring-0 5/5 Running 1 1m
cattle-prometheus prometheus-operator-monitoring-operator-6d8c95d9c6-vgnp8 1/1 Running 0 2m
cattle-system cattle-cluster-agent-7b5b9765b4-x6nk2 1/1 Running 0 2m
cattle-system cattle-node-agent-hjwrg 1/1 Running 0 2m
cattle-system cattle-node-agent-hs7xw 1/1 Running 0 2m
cattle-system cattle-node-agent-lbh6k 1/1 Running 0 2m
cattle-system rancher-7bc85fd5ff-9gkpp 1/1 Running 0 27m
cattle-system rancher-7bc85fd5ff-d2mms 1/1 Running 1 27m
cattle-system rancher-7bc85fd5ff-mp696 1/1 Running 1 27m
$ kubectl get sc
NAME PROVISIONER AGE
cbs (default) cloud.tencent.com/qcloud-cbs 34m
$ kubectl get pvc -n cattle-prometheus
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
grafana-cluster-monitoring Bound pvc-9f243bf7-716a-11e9-9473-525400d5a795 10Gi RWO cbs 3m
prometheus-cluster-monitoring-db-prometheus-cluster-monitoring-0 Bound pvc-abe75969-716a-11e9-82b2-525400489be9 50Gi RWO cbs 3m
此时再回到集群界面可以看到界面上多了grafana的图标,点进去可以跳转到grafana
界面
常见问题:
1.创建prometheus 时,选择了持久化数据,但是广州二区cbs盘售罄了导致pvc创建失败
解决方式:
- 换个资源的区域
- 换个持久化方式,默认选的是cbs storageclasses,可以通过创建nfs storageclasses来将数据持久化:TKE创建StorageClass配置Provisioner为nfs
2.ingress不能选择后端服务,因为rancher的service type类型没有修改为nodeport,ingress若要转发至service不能为clusterip type。
3.install rancher-stable/rancher报错
需要先安装cert-manager,请看第四步骤
代码语言:txt复制helm install rancher-stable/rancher
> --name rancher
> --namespace cattle-system
> --set hostname=rancher.my.org
Error: validation failed: unable to recognize "": no matches for kind "Issuer" in version "certmanager.k8s.io/v1alpha1"
参考链接
https://www.cnrancher.com/docs/rancher/v2.x/cn/overview/
https://rancher.com/docs/rancher/v2.x/en/installation/ha/helm-rancher/