背景介绍：
去年从腾讯出来，换了家企业，涉及到一些安装运维工作，其中大数据AI产品需要落地，在大厂腾都有现成的可以用，来到这都没有K8S集群，后面需要借助k8s落地，就需要从头开始从K8S集群搭建开始做起，然后公司的业务涉及到海内外还分了开发、测试、预发布等多套K8S集群。采用kubeadm安装集群非常方便，只不过安装前还需要做些前置的系统配置，如果构建多个节点的集群会涉及到一些重复操作， 这时候最好编写成一个脚本快速的安装部署，这里整理个安装脚本。
基础安装脚本，需要在每台机器上执行
代码语言：txt复制#!/bin/bash
echo "1、修改DNS配置如有需要"
chattr -i /etc/resolv.conf
cat <<EOF >> /etc/resolv.conf
nameserver 10.139.0.246
nameserver 10.139.0.247
EOF
echo "清理一些无用的yum repos"
mkdir -p /etc/yum.repos.d/back && mv /etc/yum.repos.d/ambari* /etc/yum.repos.d/hdp* /etc/yum.repos.d/back
yum clean all
### 关闭swap分区
echo "swapoff ## 临时生效"
swapoff -a
## 永久生效可以编辑/etc/fstab文件 注释掉swap那行
echo "关闭防火墙"
systemctl stop firewalld
systemctl disable --now firewalld
echo "关闭sselinux"
setenforce 0
sed -i 's/enforcing/disabled/' /etc/selinux/config
# 内核开启IPv4转发需要开启下面的模块
modprobe br_netfilter
cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
echo "设置docker的目录软连接 /var/lib/docker把docker存储设置到数据盘上"
mkdir -p /data1/docker
ln -s /data1/docker /var/lib
echo "安装docker 依赖 "
yum install yum-utils device-mapper-persistent-data lvm2 -y
### 新增 Docker 仓库。
yum-config-manager
--add-repo
https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum clean all
## 安装 Docker CE.
yum update -y && yum install -y
containerd.io-1.2.13
docker-ce-19.03.11
docker-ce-cli-19.03.11
## 创建 /etc/docker 目录。
mkdir /etc/docker
cat > /etc/docker/daemon.json <<EOF
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
],
"registry-mirrors": [ "https://1nj0zren.mirror.aliyuncs.com", "https://docker.mirrors.ustc.edu.cn", "http://f1361db2.m.daocloud.io", "https://registry.docker-cn.com" ]
}
EOF
sleep 1
systemctl daemon-reload
systemctl restart docker
sudo systemctl enable docker
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
exclude=kube*
EOF
yum install -y kubelet-1.19.3 kubeadm-1.19.3 kubectl-1.19.3 --disableexcludes=kubernetes
sleep 1
kubeadm version
systemctl enable kubelet
systemctl daemon-reload
systemctl restart kubelet- master节点操作
- master节点使用kubeadm命令初始化集群 生成集群初始化YAML配置文件 kubeadm config print init-defaults > kubeadm.yaml
- 按需要修改yaml配置
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
# 此处需要修改为master主节点 IP
advertiseAddress: 10.132.24.99
bindPort: 6443
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: kubernetes-master
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
---
apiServer:
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
# 多主节点需要修改这里：修改为主节点ip
controlPlaneEndpoint: "10.136.17.12:6443"
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
# 国内不能访问 Google，修改为阿里云
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
# 修改版本号
kubernetesVersion: v1.19.0
networking:
dnsDomain: cluster.local
# 配置成 Calico 的默认网段
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.1.0.0/16
scheduler: {}
---
# 开启 IPVS 模式
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
featureGates:
SupportIPVSProxyMode: true
mode: ipvs- 初始化集群 kubeadm init --config kubeadm.yaml --upload-certs // upload-certs用来同时生成master join语句
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of the control-plane node running the following command on each as root:
kubeadm join 10.136.17.12:6443 --token abcdef.0123456789abcdef
--discovery-token-ca-cert-hash sha256:2fbacdf6a9473d5da1d98900f73cd4e2396521772b12ac99017d6ae756d8c3cc
--control-plane --certificate-key f0725584c26c192xxxxx4dc5804a101ebc5d7b40257837eea0676d1972cca
Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 10.136.17.12:6443 --token abcdef.0123456789abcdef
--discovery-token-ca-cert-hash sha256:2fbacdf6a9473d5da1d98900f7xxxsssss6ae756d8c3cc - 拷贝kube/config文件 这样在当前节点可以执行 kubectl指令
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config- 安装网络插件
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml- 添加node节点到集群中kubeadm join 10.136.17.12:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:2fbacdf6a9473d5da1d98900f7xxxsssss6ae756d8c3cc
- 如需添加master节点执行以下操作
kubeadm join 10.136.17.12:6443 --token abcdef.0123456789abcdef
--discovery-token-ca-cert-hash sha256:2fbacdf6a9473d5da1d98900f73cd4e2396521772b12ac99017d6ae756d8c3cc
--control-plane --certificate-key f0725584c26c192xxxxx4dc5804a101ebc5d7b40257837eea0676d1972cca集群清理脚本
这个脚本就是用来清理k8s集群的，在所有的节点都执行清理操作，
代码语言：txt复制#!/bin/bash
kubeadm reset -f
ifconfig cni0 down && ip link delete cni0
ifconfig flannel.1 down && ip link delete flannel.1
rm -rf /var/lib/cni/总结
其实这种方式还是存在一些问题的比如说网络、k8s版本、linux内核配置。在一些公司业务规模比较小、数据量表小的情况下，普通的k8s集群就可以满足需求，不需要在网络插件和相关服务上做性能优化，直接就可以部署着用了
代码语言：txt复制
