使用 k3s 搭建 cilium + istio 实验环境

以下步骤运行在 oracle cloud ARM Ampere A1 实例上，实例配置为：4C24G，操作系统为：ubuntu 22.04 LTS

一、安装 k3s

使用以下命令启动 k3s 集群，不安装 cni， ~~同时设置 cluster cidr 和 service cidr 防止和本机地址冲突(~~ –cluster-cidr=‘10.42.0.0/16’ –service-cidr=‘10.253.0.0/16’ )

1 2 3 4 5 6	curl -sfL -sfL https://get.k3s.io \| INSTALL_K3S_EXEC="--flannel-backend=none --disable-network-policy --disable-kube-proxy --prefer-bundled-bin --disable=traefik,servicelb,local-storage" sh -s -

二、安装 Cilium

1. 操作系统准备

禁用外部路由管理

一些发行版需要配置不管理外部路由，例如 Ubuntu 22.04，详细信息请查看GitHub issue 18706，编辑 /etc/systemd/networkd.conf 添加以下设置：

1 2 3	[Network] ManageForeignRoutes=no ManageForeignRoutingPolicyRules=no

重启systemd-networkd服务

1 2	systemctl daemon-reload systemctl restart systemd-networkd

2. 安装 Cilium CLI

1 2 3 4 5 6 7 8 9 10

CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt) CLI_ARCH=amd64 if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum} sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum} #查看当前安装版本 cilium version --client

3. 安装 Cilium

同时安装 Hubble & Hubble UI

为确保 Cilium 不干扰 Istio，必须使用 –config bpf-lb-sock-hostns-only=true cilium CLI 标志或 socketLB.hostNamespaceOnly=true 来部署 Cilium。同时 ipam.operator.clusterPoolIPv4PodCIDRList 指定 k3s 默认的 pod CIDR。

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml API_SERVER_IP='10.0.0.214' # 服务器 ip 地址 API_SERVER_PORT=6443 cilium install --set hubble.relay.enabled=true --set hubble.ui.enabled=true --set operator.replicas=1 --set kubeProxyReplacement=strict --set ipam.operator.clusterPoolIPv4PodCIDRList="10.42.0.0/16" --set socketLB.hostNamespaceOnly=true --set k8sServiceHost=${API_SERVER_IP} --set k8sServicePort=${API_SERVER_PORT} --set tunnel=disabled --set autoDirectNodeRoutes=true --set ipv4NativeRoutingCIDR="10.42.0.0/16" --set bpf.masquerade=true --version 1.14.1

NATIVE ROUTING：

1 2 3	--set tunnel=disabled --set autoDirectNodeRoutes=true --set ipv4NativeRoutingCIDR="10.42.0.0/16"

IP 地址伪装(Masquerading):

1	--set bpf.masquerade=true

4. 验证 Cilium 安装

运行以下命令验证 cilium 时候正确安装

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

# cilium status --wait /¯¯ /¯¯__/¯¯ Cilium: OK __/¯¯__/ Operator: OK /¯¯__/¯¯ Envoy DaemonSet: disabled (using embedded mode) __/¯¯__/ Hubble Relay: OK __/ ClusterMesh: disabled DaemonSet cilium Desired: 1, Ready: 1/1, Available: 1/1 Deployment hubble-relay Desired: 1, Ready: 1/1, Available: 1/1 Deployment hubble-ui Desired: 1, Ready: 1/1, Available: 1/1 Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1 Containers: cilium-operator Running: 1 cilium Running: 1 hubble-relay Running: 1 hubble-ui Running: 1 Cluster Pods: 6/6 managed by Cilium Helm chart version: 1.14.0 Image versions cilium quay.io/cilium/cilium:v1.14.0@sha256:5a94b561f4651fcfd85970a50bc78b201cfbd6e2ab1a03848eab25a82832653a: 1 hubble-relay quay.io/cilium/hubble-relay:v1.14.0@sha256:bfe6ef86a1c0f1c3e8b105735aa31db64bcea97dd4732db6d0448c55a3c8e70c: 1 hubble-ui quay.io/cilium/hubble-ui:v0.12.0@sha256:1c876cfa1d5e35bc91e1025c9314f922041592a88b03313c22c1f97a5d2ba88f: 1 hubble-ui quay.io/cilium/hubble-ui-backend:v0.12.0@sha256:8a79a1aad4fc9c2aa2b3e4379af0af872a89fcec9d99e117188190671c66fc2e: 1 cilium-operator quay.io/cilium/operator-generic:v1.14.0@sha256:3014d4bcb8352f0ddef90fa3b5eb1bbf179b91024813a90a0066eb4517ba93c9: 1

运行以下命令验证集群网络的连通性

1 2 3 4 5 6	$ cilium connectivity test ℹ️ Single-node environment detected, enabling single-node connectivity test ℹ️ Monitor aggregation detected, will skip some flow validation steps ........ ✅ All 42 tests (184 actions) successful, 13 tests skipped, 0 scenarios skipped.

三、安装 Istio

1. 下载 Istio

使用以下脚本地址下载 Istio

1	curl -L https://istio.io/downloadIstio \| sh -

转到 Istio 包目录。例如，如果包是istio-1.18.2：

1	$ cd istio-1.18.2

安装目录包含：

samples/目录下的示例应用程序
bin/目录下的[istioctl](https://istio.io/latest/zh/docs/reference/commands/istioctl)客户端二进制文件。

将istioctl客户端添加到路径：

1	cp bin/istioctl /usr/local/bin/

2. 使用预设文件部署 istio

采用demo配置组合, 选择它是因为它包含了一组专为测试准备的功能集合，另外还有用于生产或性能测试的配置组合。

1 2 3 4 5 6 7 8	# export KUBECONFIG=/etc/rancher/k3s/k3s.yaml # istioctl install --set profile=demo -y ✔ Istio core installed ✔ Istiod installed ✔ Egress gateways installed ✔ Ingress gateways installed ✔ Installation complete Making this installation the default for injection and validation

命名空间添加标签，指示 Istio 在部署应用的时候，自动注入 Envoy 边车代理

1 2	$ kubectl label namespace default istio-injection=enabled namespace/default labeled

四、卸载

当 k3s 环境出现问题，或者需要重建，需要使用以下方法：

1 2 3 4 5 6 7 8 9 10	ip link delete cilium_host ip link delete cilium_net ip link delete cilium_vxlan iptables-save \| grep -iv cilium \| iptables-restore ip6tables-save \| grep -iv cilium \| ip6tables-restore /usr/local/bin/k3s-killall.sh /usr/local/bin/k3s-uninstall.sh

至此完成 cilium Istio 实验环境搭建，后续将进行 cilium 和 Istio 相关的实验。

istio set 部署操作系统配置

0 人点赞