mac 上学习k8s系列(25)iptables ipvs

2022-08-02 19:34:37 浏览数 (1)

iptables有4 个表,优先级从高到低分别是raw,mangle,nat,filter,默认是filter

filter 表(过滤规则表)

控制数据包是否允许进出及转发,可以控制的链路有 INPUT、FORWARD 和 OUTPUT。

nat 表(地址转换规则表)

控制数据包中地址转换,可以控制的链路有 PREROUTING、INPUT、OUTPUT 和 POSTROUTING。

mangle(修改数据标记位规则表)

修改数据包中的原数据,可以控制的链路有 PREROUTING、INPUT、OUTPUT、FORWARD 和 POSTROUTING。

raw(跟踪数据表规则表)

控制 nat 表中连接追踪机制的启用状况,可以控制的链路有 PREROUTING、OUTPUT。

5 个链

INPUT(入站数据过滤)

外部主机发送数据包给防火墙本机,数据将会经过 PREROUTING 链与 INPUT 链

OUTPUT(出站数据过滤)

如果是防火墙本机发送数据包到外部主机,数据将会经过 OUTPUT 链与 POSTROUTING 链

FORWARD(转发数据过滤)

如果防火墙作为路由负责转发数据,则数据将经过 PREROUTING 链、FORWARD 链以及 POSTROUTING 链。

PREROUTING(路由前过滤)

POSTROUTING(路由后过滤)

基本语法

代码语言:javascript复制
iptables [-t table] COMMAND [chain] CRETIRIA -j ACTION

-t:指定需要维护的防火墙规则表 filter、nat、mangle或raw。在不使用 -t 时则默认使用 filter 表。

COMMAND:子命令,定义对规则的管理。

代码语言:javascript复制
-A  添加防火墙规则
-D  删除防火墙规则
-I  插入防火墙规则
-F  清空防火墙规则
-L  列出添加防火墙规则
-R  替换防火墙规则
-Z  清空防火墙数据表统计信息
-P  设置链默认规则

chain:指明链表。

CRETIRIA:匹配参数。

代码语言:javascript复制
[!]-p  匹配协议,! 表示取反
[!]-s  匹配源地址
[!]-d  匹配目标地址
[!]-i  匹配入站网卡接口
[!]-o  匹配出站网卡接口
[!]--sport  匹配源端口
[!]--dport  匹配目标端口
[!]--src-range  匹配源地址范围
[!]--dst-range  匹配目标地址范围
[!]--limit  四配数据表速率
[!]--mac-source  匹配源MAC地址
[!]--sports  匹配源端口
[!]--dports  匹配目标端口
[!]--stste  匹配状态(INVALID、ESTABLISHED、NEW、RELATED)
[!]--string  匹配应用层字串

ACTION:触发动作。

代码语言:javascript复制
ACCEPT  允许数据包通过
DROP  丢弃数据包
REJECT  拒绝数据包通过
LOG  将数据包信息记录 syslog 曰志
DNAT  目标地址转换
SNAT  源地址转换
MASQUERADE  地址欺骗
REDIRECT  重定向

内核会按照顺序依次检查 iptables 防火墙规则,如果发现有匹配的规则目录,则立刻执行相关动作,停止继续向下查找规则目录

如果所有的防火墙规则都未能匹配成功,则按照默认策略处理。

使用 -A 选项添加防火墙规则会将该规则追加到整个链的最后,而使用 -I 选项添加的防火墙规则则会默认插入到链中作为第一条规则。

k8s node节点的iptables是由kube-proxy生成的,kube-proxy只修改了filter和nat表,它对iptables的链进行了扩充,自定义了KUBE-SERVICES,KUBE-NODEPORTS,KUBE-POSTROUTING,KUBE-MARK-MASQ和KUBE-MARK-DROP五个链,并主要通过为 KUBE-SERVICES链(附着在PREROUTING和OUTPUT)增加rule来配制traffic routing 规则

代码语言:javascript复制
// the services chain
kubeServicesChain utiliptables.Chain = "KUBE-SERVICES"
 
// the external services chain
kubeExternalServicesChain utiliptables.Chain = "KUBE-EXTERNAL-SERVICES"
 
// the nodeports chain
kubeNodePortsChain utiliptables.Chain = "KUBE-NODEPORTS"
 
// the kubernetes postrouting chain
kubePostroutingChain utiliptables.Chain = "KUBE-POSTROUTING"
 
// the mark-for-masquerade chain
KubeMarkMasqChain utiliptables.Chain = "KUBE-MARK-MASQ"     /*对于未能匹配到跳转规则的traffic set mark 0x8000,有此标记的数据包会在filter表drop掉*/
 
// the mark-for-drop chain
KubeMarkDropChain utiliptables.Chain = "KUBE-MARK-DROP"    /*对于符合条件的包 set mark 0x4000, 有此标记的数据包会在KUBE-POSTROUTING chain中统一做MASQUERADE*/
 
// the kubernetes forward chain
kubeForwardChain utiliptables.Chain = "KUBE-FORWARD"

KUBE_SVC和KUBE-SEP:

Kube-proxy接着对每个服务创建“KUBE-SVC-”链,并在nat表中将KUBE-SERVICES链中每个目标地址是service的数据包导入这个“KUBE-SVC-”链,如果endpoint尚未创建,KUBE-SVC-链中没有规则,任何incomming packets在规则匹配失败后会被KUBE-MARK-DROP。在iptables的filter中有如下处理,如果KUBE-SVC处理失败会通过KUBE_FIREWALL丢弃

KUBE-SEP表示的是KUBE-SVC对应的endpoint,当接收到的 serviceInfo中包含endpoint信息时,为endpoint创建跳转规则

1,clusterIP 访问方式

代码语言:javascript复制
PREROUTING   KUBE-SERVICE   KUBE-SVC-XXX   KUBE-SEP-XXX

2,nodePort 方式

非本机访问

代码语言:javascript复制
PREROUTING   KUBE-SERVICE  KUBE-NODEPORTS  KUBE-SVC-XXX   KUBE-SEP-XXX

本机访问

代码语言:javascript复制
OUTPUT   KUBE-SERVICE  KUBE-NODEPORTS  KUBE-SVC-XXX   KUBE-SEP-XXX

kube-proxy 在 ipvs 模式下自定义了八条链,分别为 KUBE-SERVICES、KUBE-FIREWALL、KUBE-POSTROUTING、KUBE-MARK-MASQ、KUBE-NODE-PORT、KUBE-MARK-DROP、KUBE-FORWARD、KUBE-LOAD-BALANCER

1,clusterIP 访问方式

代码语言:javascript复制
PREROUTING   KUBE-SERVICE KUBE-CLUSTER-IP  INPUT KUBE-FIREWALL POSTROUTING

2,nodePort 方式

代码语言:javascript复制
PREROUTING   KUBE-SERVICE KUBE-NODE-PORT INPUT KUBE-FIREWALL POSTROUTING

ipvs 作用于INPUT脸,和iptables的区别是

代码语言:javascript复制
底层数据结构:iptables 使用链表,ipvs 使用哈希表
负载均衡算法:iptables 只支持随机、轮询两种负载均衡算法而 ipvs 支持的多达 8 种;
操作工具:iptables 需要使用 iptables 命令行工作来定义规则,ipvs 需要使用 ipvsadm 来定义规则。

我们在mac m1上如何查看iptables呢?

首先我们知道iptables是linux的组件,在mac上对应的防火墙组件bpf,命令结构是不一样的。但是,docker for mac是工作在xhyve上的虚拟机linux kit里的,所以我们进入linuxkit就可以使用iptables

代码语言:javascript复制
docker run -it --privileged --pid=host alpine:latest nsenter -t 1 -m -u -n -i sh

我们从四个表的优先级依次查看对应的规则

代码语言:javascript复制
# iptables -t raw -nvL
Chain PREROUTING (policy ACCEPT 169M packets, 114G bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 93M packets, 34G bytes)
 pkts bytes target     prot opt in     out     source               destination
代码语言:javascript复制
# iptables -t mangle -nvL
Chain PREROUTING (policy ACCEPT 168M packets, 114G bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain INPUT (policy ACCEPT 167M packets, 113G bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 1538K packets, 775M bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 93M packets, 34G bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 94M packets, 34G bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain KUBE-KUBELET-CANARY (0 references)
 pkts bytes target     prot opt in     out     source               destination

Chain KUBE-PROXY-CANARY (0 references)
 pkts bytes target     prot opt in     out     source               destination
代码语言:javascript复制
/ # iptables -t nat -nvL
Chain PREROUTING (policy ACCEPT 194 packets, 11981 bytes)
 pkts bytes target     prot opt in     out     source               destination
23979 1651K KUBE-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service portals */
12998  833K desktop    all  --  *      *       0.0.0.0/0            0.0.0.0/0
    4  5023 DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 27 packets, 1620 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 7134 packets, 436K bytes)
 pkts bytes target     prot opt in     out     source               destination
 471K   28M KUBE-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service portals */
 306K   18M DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 7154 packets, 438K bytes)
 pkts bytes target     prot opt in     out     source               destination
 493K   30M KUBE-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes postrouting rules */
  101  6060 MASQUERADE  all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ADDRTYPE match src-type LOCAL
 9599  576K MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0
    1    40 MASQUERADE  all  --  *      br-421a9e16d12b  0.0.0.0/0            0.0.0.0/0            ADDRTYPE match src-type LOCAL
    0     0 MASQUERADE  all  --  *      !br-421a9e16d12b  172.18.0.0/16        0.0.0.0/0
    1    60 CNI-0d314787d804206fd0db3f98  all  --  *      *       10.1.0.214           0.0.0.0/0            /* name: "default" id: "d56bc6d2512b0f78cebc9ad1ff17c1fc725ad61a952db34a7980e2d0d783d108" */
    0     0 CNI-48f8ec86c3b0a658588868a6  all  --  *      *       10.1.0.215           0.0.0.0/0            /* name: "default" id: "af82f2fdcea925dc1710fe6d0c9c276509f8e6cd7a794cd3743ea843b677b7f5" */
 1036 73821 CNI-91b2cd657fb8b0fedb8c77ad  all  --  *      *       10.1.0.216           0.0.0.0/0            /* name: "default" id: "c657423898aef5e5327acd484ef00d00242175702ac40cca490a6f25a1242bfe" */
    0     0 CNI-fab4888526a9bec71834f286  all  --  *      *       10.1.0.217           0.0.0.0/0            /* name: "default" id: "778a2bc9fb4e6afba59cc04cefd72e1b0d88124b7d6e6b007197811ac3eec346" */
 1028 73250 CNI-137db40c6a5fbc453fbbcfce  all  --  *      *       10.1.0.218           0.0.0.0/0            /* name: "default" id: "ec757b45cd049459142a9347627310fdba91799d5250309e7f2e5dac5cfd13b0" */
   31  1860 CNI-6563068de8207c093cbc345d  all  --  *      *       10.1.0.220           0.0.0.0/0            /* name: "default" id: "293e15775fa73def3d7470b91c3feb05bb7438638ddf20743efd8d9d4e895de3" */
    0     0 CNI-40508a9bf71b10c255802fb8  all  --  *      *       10.1.0.221           0.0.0.0/0            /* name: "default" id: "02c1e14ac96f8e807f79c6d18609e6345746f37d8e269d2cfaa3b9e4ed6aca2d" */
    0     0 CNI-e4d1efb036294232e4444b5f  all  --  *      *       10.1.0.222           0.0.0.0/0            /* name: "default" id: "71b423acc07836a756a2d34d31db93d5319d1e7fd75d206a6926631003eda89d" */
   18  1236 CNI-e3e4ea5629d0c5627784ddd4  all  --  *      *       10.1.0.223           0.0.0.0/0            /* name: "default" id: "f474c66ddaa93f99db2ec563dc6d77f69c863b25ed00b7cde35eb5c3b058cc8a" */
10623  805K CNI-e93ed372dfdce36419bee3f1  all  --  *      *       10.1.0.224           0.0.0.0/0            /* name: "default" id: "6856a2970e865ce1463eee892de5a5f14be32c2495f4a030cb3fb55034e3282f" */
    0     0 MASQUERADE  tcp  --  *      *       172.17.0.2           172.17.0.2           tcp dpt:443
    0     0 MASQUERADE  tcp  --  *      *       172.17.0.2           172.17.0.2           tcp dpt:80

Chain CNI-0d314787d804206fd0db3f98 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            10.1.0.0/16          /* name: "default" id: "d56bc6d2512b0f78cebc9ad1ff17c1fc725ad61a952db34a7980e2d0d783d108" */
    1    60 MASQUERADE  all  --  *      *       0.0.0.0/0           !224.0.0.0/4          /* name: "default" id: "d56bc6d2512b0f78cebc9ad1ff17c1fc725ad61a952db34a7980e2d0d783d108" */

Chain CNI-137db40c6a5fbc453fbbcfce (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            10.1.0.0/16          /* name: "default" id: "ec757b45cd049459142a9347627310fdba91799d5250309e7f2e5dac5cfd13b0" */
 1028 73250 MASQUERADE  all  --  *      *       0.0.0.0/0           !224.0.0.0/4          /* name: "default" id: "ec757b45cd049459142a9347627310fdba91799d5250309e7f2e5dac5cfd13b0" */

Chain CNI-40508a9bf71b10c255802fb8 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            10.1.0.0/16          /* name: "default" id: "02c1e14ac96f8e807f79c6d18609e6345746f37d8e269d2cfaa3b9e4ed6aca2d" */
    0     0 MASQUERADE  all  --  *      *       0.0.0.0/0           !224.0.0.0/4          /* name: "default" id: "02c1e14ac96f8e807f79c6d18609e6345746f37d8e269d2cfaa3b9e4ed6aca2d" */

Chain CNI-48f8ec86c3b0a658588868a6 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            10.1.0.0/16          /* name: "default" id: "af82f2fdcea925dc1710fe6d0c9c276509f8e6cd7a794cd3743ea843b677b7f5" */
    0     0 MASQUERADE  all  --  *      *       0.0.0.0/0           !224.0.0.0/4          /* name: "default" id: "af82f2fdcea925dc1710fe6d0c9c276509f8e6cd7a794cd3743ea843b677b7f5" */

Chain CNI-6563068de8207c093cbc345d (1 references)
 pkts bytes target     prot opt in     out     source               destination
   30  1800 ACCEPT     all  --  *      *       0.0.0.0/0            10.1.0.0/16          /* name: "default" id: "293e15775fa73def3d7470b91c3feb05bb7438638ddf20743efd8d9d4e895de3" */
    1    60 MASQUERADE  all  --  *      *       0.0.0.0/0           !224.0.0.0/4          /* name: "default" id: "293e15775fa73def3d7470b91c3feb05bb7438638ddf20743efd8d9d4e895de3" */

Chain CNI-91b2cd657fb8b0fedb8c77ad (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            10.1.0.0/16          /* name: "default" id: "c657423898aef5e5327acd484ef00d00242175702ac40cca490a6f25a1242bfe" */
 1036 73821 MASQUERADE  all  --  *      *       0.0.0.0/0           !224.0.0.0/4          /* name: "default" id: "c657423898aef5e5327acd484ef00d00242175702ac40cca490a6f25a1242bfe" */

Chain CNI-e3e4ea5629d0c5627784ddd4 (1 references)
 pkts bytes target     prot opt in     out     source               destination
   16  1112 ACCEPT     all  --  *      *       0.0.0.0/0            10.1.0.0/16          /* name: "default" id: "f474c66ddaa93f99db2ec563dc6d77f69c863b25ed00b7cde35eb5c3b058cc8a" */
    2   124 MASQUERADE  all  --  *      *       0.0.0.0/0           !224.0.0.0/4          /* name: "default" id: "f474c66ddaa93f99db2ec563dc6d77f69c863b25ed00b7cde35eb5c3b058cc8a" */

Chain CNI-e4d1efb036294232e4444b5f (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            10.1.0.0/16          /* name: "default" id: "71b423acc07836a756a2d34d31db93d5319d1e7fd75d206a6926631003eda89d" */
    0     0 MASQUERADE  all  --  *      *       0.0.0.0/0           !224.0.0.0/4          /* name: "default" id: "71b423acc07836a756a2d34d31db93d5319d1e7fd75d206a6926631003eda89d" */

Chain CNI-e93ed372dfdce36419bee3f1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
 9382  722K ACCEPT     all  --  *      *       0.0.0.0/0            10.1.0.0/16          /* name: "default" id: "6856a2970e865ce1463eee892de5a5f14be32c2495f4a030cb3fb55034e3282f" */
 1241 83762 MASQUERADE  all  --  *      *       0.0.0.0/0           !224.0.0.0/4          /* name: "default" id: "6856a2970e865ce1463eee892de5a5f14be32c2495f4a030cb3fb55034e3282f" */

Chain CNI-fab4888526a9bec71834f286 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            10.1.0.0/16          /* name: "default" id: "778a2bc9fb4e6afba59cc04cefd72e1b0d88124b7d6e6b007197811ac3eec346" */
    0     0 MASQUERADE  all  --  *      *       0.0.0.0/0           !224.0.0.0/4          /* name: "default" id: "778a2bc9fb4e6afba59cc04cefd72e1b0d88124b7d6e6b007197811ac3eec346" */

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
   23  1380 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:7443 to:172.17.0.2:443
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:7071 to:172.17.0.2:80

Chain KUBE-KUBELET-CANARY (0 references)
 pkts bytes target     prot opt in     out     source               destination

Chain KUBE-MARK-DROP (0 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x8000

Chain KUBE-MARK-MASQ (26 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x4000

Chain KUBE-NODEPORTS (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/minio-service:api */ tcp dpt:30000
    0     0 KUBE-SVC-SXW22BMJJ7T3N2OP  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/minio-service:api */ tcp dpt:30000
    0     0 KUBE-MARK-MASQ  tcp  --  *      *       127.0.0.0/8          0.0.0.0/0            /* default/ingress-nginx-controller:https */ tcp dpt:32342
    0     0 KUBE-XLB-Q7CDIBSFDYNOJNFE  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/ingress-nginx-controller:https */ tcp dpt:32342
    0     0 KUBE-MARK-MASQ  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/etcd-svc-docker-desktop-xzm */ tcp dpt:32389
    0     0 KUBE-SVC-EDBHCP4VQID7F5J2  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/etcd-svc-docker-desktop-xzm */ tcp dpt:32389
    0     0 KUBE-MARK-MASQ  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/redis:tcp */ tcp dpt:30379
    0     0 KUBE-SVC-CKFHGLZY3HDORVFT  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/redis:tcp */ tcp dpt:30379
    0     0 KUBE-MARK-MASQ  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/apple-service */ tcp dpt:30080
    0     0 KUBE-SVC-Y4TE457BRBWMNDKG  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/apple-service */ tcp dpt:30080
    0     0 KUBE-MARK-MASQ  tcp  --  *      *       127.0.0.0/8          0.0.0.0/0            /* default/ingress-nginx-controller:http */ tcp dpt:30701
    0     0 KUBE-XLB-D7TXZ2ONB4DT7BQA  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/ingress-nginx-controller:http */ tcp dpt:30701
    0     0 KUBE-MARK-MASQ  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/minio-service:console */ tcp dpt:30001
    0     0 KUBE-SVC-ED7LY7V3PRCUB6IJ  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/minio-service:console */ tcp dpt:30001

Chain KUBE-POSTROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination
 7476  459K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match ! 0x4000/0x4000
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK xor 0x4000
    0     0 MASQUERADE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service traffic requiring SNAT */ random-fully

Chain KUBE-PROXY-CANARY (0 references)
 pkts bytes target     prot opt in     out     source               destination

Chain KUBE-SEP-3JXLGUMKBPWA3G7K (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.218           0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */ tcp to:10.1.0.218:53

Chain KUBE-SEP-4XKQ3BKJH4UM3LNT (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.220           0.0.0.0/0            /* default/ingress-nginx-controller-admission:https-webhook */
   20  1200 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/ingress-nginx-controller-admission:https-webhook */ tcp to:10.1.0.220:8443

Chain KUBE-SEP-5EF3GOGMU4HLZ57C (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.216           0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */ tcp to:10.1.0.216:53

Chain KUBE-SEP-5VPUENENYDAFRS6E (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.220           0.0.0.0/0            /* default/ingress-nginx-controller:http */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/ingress-nginx-controller:http */ tcp to:10.1.0.220:80

Chain KUBE-SEP-FEVJFI3I56DO4VH7 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.220           0.0.0.0/0            /* default/ingress-nginx-controller:https */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/ingress-nginx-controller:https */ tcp to:10.1.0.220:443

Chain KUBE-SEP-FLOJ2NBWC6RHDXIG (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       192.168.65.4         0.0.0.0/0            /* kube-system/etcd-svc-docker-desktop-xzm */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/etcd-svc-docker-desktop-xzm */ tcp to:192.168.65.4:2379

Chain KUBE-SEP-FYCASJGNWKVYLA7B (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       192.168.65.4         0.0.0.0/0            /* default/kubernetes:https */
   27  1620 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https */ tcp to:192.168.65.4:6443

Chain KUBE-SEP-GXFFDQL3JO5LZDO7 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.218           0.0.0.0/0            /* kube-system/kube-dns:dns */
   61  4638 DNAT       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns */ udp to:10.1.0.218:53

Chain KUBE-SEP-JPYJVE4BPHWKYBDW (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.216           0.0.0.0/0            /* kube-system/kube-dns:dns */
   67  5090 DNAT       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns */ udp to:10.1.0.216:53

Chain KUBE-SEP-KWOQFGYL6KRGKJ23 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.218           0.0.0.0/0            /* kube-system/kube-dns:metrics */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:metrics */ tcp to:10.1.0.218:9153

Chain KUBE-SEP-N6ACUILWO7XJGY5F (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.217           0.0.0.0/0            /* default/apple-service */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/apple-service */ tcp to:10.1.0.217:5678

Chain KUBE-SEP-NGE6IH2X2N4M4XEF (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.224           0.0.0.0/0            /* cattle-system/cattle-cluster-agent:http */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cattle-system/cattle-cluster-agent:http */ tcp to:10.1.0.224:80

Chain KUBE-SEP-NSIQZ33RUVLUUTPS (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.224           0.0.0.0/0            /* cattle-system/cattle-cluster-agent:https-internal */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cattle-system/cattle-cluster-agent:https-internal */ tcp to:10.1.0.224:444

Chain KUBE-SEP-NVZBSVXMHBDHFW7S (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.223           0.0.0.0/0            /* default/minio-service:console */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/minio-service:console */ tcp to:10.1.0.223:9001

Chain KUBE-SEP-OYXC66MH76XMJWAI (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.223           0.0.0.0/0            /* default/minio-service:api */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/minio-service:api */ tcp to:10.1.0.223:9000

Chain KUBE-SEP-QCJUKRQWA7676OUJ (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.216           0.0.0.0/0            /* kube-system/kube-dns:metrics */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:metrics */ tcp to:10.1.0.216:9153

Chain KUBE-SEP-RAT4S652HRQXMAAD (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.1.0.222           0.0.0.0/0            /* default/redis:tcp */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/redis:tcp */ tcp to:10.1.0.222:6379

Chain KUBE-SERVICES (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-SVC-RXZQBFX6IWO22WWW  tcp  --  *      *       0.0.0.0/0            10.101.247.125       /* cattle-system/cattle-cluster-agent:http cluster IP */ tcp dpt:80
    0     0 KUBE-SVC-SXW22BMJJ7T3N2OP  tcp  --  *      *       0.0.0.0/0            10.99.62.36          /* default/minio-service:api cluster IP */ tcp dpt:9000
    0     0 KUBE-SVC-ERIFXISQEP7F7OF4  tcp  --  *      *       0.0.0.0/0            10.96.0.10           /* kube-system/kube-dns:dns-tcp cluster IP */ tcp dpt:53
    0     0 KUBE-SVC-Q7CDIBSFDYNOJNFE  tcp  --  *      *       0.0.0.0/0            10.108.133.209       /* default/ingress-nginx-controller:https cluster IP */ tcp dpt:443
    0     0 KUBE-SVC-DISNXZXWEI7GIGLU  tcp  --  *      *       0.0.0.0/0            10.101.247.125       /* cattle-system/cattle-cluster-agent:https-internal cluster IP */ tcp dpt:443
    0     0 KUBE-SVC-EDBHCP4VQID7F5J2  tcp  --  *      *       0.0.0.0/0            10.111.136.178       /* kube-system/etcd-svc-docker-desktop-xzm cluster IP */ tcp dpt:2379
    0     0 KUBE-SVC-CKFHGLZY3HDORVFT  tcp  --  *      *       0.0.0.0/0            10.105.105.140       /* default/redis:tcp cluster IP */ tcp dpt:6379
  128  9728 KUBE-SVC-TCOU7JCQXEZGVUNU  udp  --  *      *       0.0.0.0/0            10.96.0.10           /* kube-system/kube-dns:dns cluster IP */ udp dpt:53
    0     0 KUBE-SVC-Y4TE457BRBWMNDKG  tcp  --  *      *       0.0.0.0/0            10.105.42.239        /* default/apple-service cluster IP */ tcp dpt:5678
    0     0 KUBE-SVC-D7TXZ2ONB4DT7BQA  tcp  --  *      *       0.0.0.0/0            10.108.133.209       /* default/ingress-nginx-controller:http cluster IP */ tcp dpt:80
   27  1620 KUBE-SVC-NPX46M4PTMTKRN6Y  tcp  --  *      *       0.0.0.0/0            10.96.0.1            /* default/kubernetes:https cluster IP */ tcp dpt:443
   20  1200 KUBE-SVC-XUD33RTORZBRAEIL  tcp  --  *      *       0.0.0.0/0            10.99.126.26         /* default/ingress-nginx-controller-admission:https-webhook cluster IP */ tcp dpt:443
    0     0 KUBE-SVC-ED7LY7V3PRCUB6IJ  tcp  --  *      *       0.0.0.0/0            10.99.62.36          /* default/minio-service:console cluster IP */ tcp dpt:9001
    0     0 KUBE-SVC-JD5MR3NA4I4DYORP  tcp  --  *      *       0.0.0.0/0            10.96.0.10           /* kube-system/kube-dns:metrics cluster IP */ tcp dpt:9153
 4617  277K KUBE-NODEPORTS  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service nodeports; NOTE: this must be the last rule in this chain */ ADDRTYPE match dst-type LOCAL

Chain KUBE-SVC-CKFHGLZY3HDORVFT (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-SEP-RAT4S652HRQXMAAD  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/redis:tcp */

Chain KUBE-SVC-D7TXZ2ONB4DT7BQA (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-SEP-5VPUENENYDAFRS6E  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/ingress-nginx-controller:http */

Chain KUBE-SVC-DISNXZXWEI7GIGLU (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-SEP-NSIQZ33RUVLUUTPS  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cattle-system/cattle-cluster-agent:https-internal */

Chain KUBE-SVC-ED7LY7V3PRCUB6IJ (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-SEP-NVZBSVXMHBDHFW7S  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/minio-service:console */

Chain KUBE-SVC-EDBHCP4VQID7F5J2 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-SEP-FLOJ2NBWC6RHDXIG  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/etcd-svc-docker-desktop-xzm */

Chain KUBE-SVC-ERIFXISQEP7F7OF4 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-SEP-5EF3GOGMU4HLZ57C  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */ statistic mode random probability 0.50000000000
    0     0 KUBE-SEP-3JXLGUMKBPWA3G7K  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */

Chain KUBE-SVC-JD5MR3NA4I4DYORP (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-SEP-QCJUKRQWA7676OUJ  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:metrics */ statistic mode random probability 0.50000000000
    0     0 KUBE-SEP-KWOQFGYL6KRGKJ23  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:metrics */

Chain KUBE-SVC-NPX46M4PTMTKRN6Y (1 references)
 pkts bytes target     prot opt in     out     source               destination
   27  1620 KUBE-SEP-FYCASJGNWKVYLA7B  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https */

Chain KUBE-SVC-Q7CDIBSFDYNOJNFE (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-SEP-FEVJFI3I56DO4VH7  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/ingress-nginx-controller:https */

Chain KUBE-SVC-RXZQBFX6IWO22WWW (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-SEP-NGE6IH2X2N4M4XEF  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cattle-system/cattle-cluster-agent:http */

Chain KUBE-SVC-SXW22BMJJ7T3N2OP (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-SEP-OYXC66MH76XMJWAI  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/minio-service:api */

Chain KUBE-SVC-TCOU7JCQXEZGVUNU (1 references)
 pkts bytes target     prot opt in     out     source               destination
   67  5090 KUBE-SEP-JPYJVE4BPHWKYBDW  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns */ statistic mode random probability 0.50000000000
   61  4638 KUBE-SEP-GXFFDQL3JO5LZDO7  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns */

Chain KUBE-SVC-XUD33RTORZBRAEIL (1 references)
 pkts bytes target     prot opt in     out     source               destination
   20  1200 KUBE-SEP-4XKQ3BKJH4UM3LNT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/ingress-nginx-controller-admission:https-webhook */

Chain KUBE-SVC-Y4TE457BRBWMNDKG (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-SEP-N6ACUILWO7XJGY5F  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/apple-service */

Chain KUBE-XLB-D7TXZ2ONB4DT7BQA (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* masquerade LOCAL traffic for default/ingress-nginx-controller:http LB IP */ ADDRTYPE match src-type LOCAL
    0     0 KUBE-SVC-D7TXZ2ONB4DT7BQA  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* route LOCAL traffic for default/ingress-nginx-controller:http LB IP to service chain */ ADDRTYPE match src-type LOCAL
    0     0 KUBE-SEP-5VPUENENYDAFRS6E  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Balancing rule 0 for default/ingress-nginx-controller:http */

Chain KUBE-XLB-Q7CDIBSFDYNOJNFE (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-MARK-MASQ  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* masquerade LOCAL traffic for default/ingress-nginx-controller:https LB IP */ ADDRTYPE match src-type LOCAL
    0     0 KUBE-SVC-Q7CDIBSFDYNOJNFE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* route LOCAL traffic for default/ingress-nginx-controller:https LB IP to service chain */ ADDRTYPE match src-type LOCAL
    0     0 KUBE-SEP-FEVJFI3I56DO4VH7  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Balancing rule 0 for default/ingress-nginx-controller:https */

Chain desktop (1 references)
 pkts bytes target     prot opt in     out     source               destination
    2   955 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    2   136 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    2   955 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    2   136 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    3   204 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    2   921 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    5  2196 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    6  2768 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    5  1512 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    5  1511 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    3   204 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    2   136 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    2   136 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    2   136 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    1    68 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    3  1023 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    2   136 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
   54  4041 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
   16  1775 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    3   995 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    4  1071 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    3   994 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    3   995 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    3   995 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    3   995 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    0     0 RETURN     tcp  --  *      *       0.0.0.0/0            172.17.0.0/16
    0     0 RETURN     tcp  --  *      *       0.0.0.0/0            172.18.0.0/16
    0     0 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.5
    6   360 RETURN     tcp  --  *      *       0.0.0.0/0            192.168.65.0/24
    0     0 RETURN     tcp  --  *      *       0.0.0.0/0            127.0.0.0/8
   30  1800 RETURN     tcp  --  *      *       0.0.0.0/0            10.1.0.0/16
    5   300 RETURN     tcp  --  *      *       0.0.0.0/0            10.96.0.0/12
代码语言:javascript复制
# iptables -t filter -nvL
Chain INPUT (policy ACCEPT 3386K packets, 2827M bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 IN_WEB     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80
 167M  113G KUBE-NODEPORTS  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes health check service ports */
 307K   18M KUBE-EXTERNAL-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW /* kubernetes externally-visible service portals */
 167M  113G KUBE-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy ACCEPT 175 packets, 12889 bytes)
 pkts bytes target     prot opt in     out     source               destination
1538K  775M KUBE-FORWARD  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes forwarding rules */
22724 1556K KUBE-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW /* kubernetes service portals */
22722 1555K KUBE-EXTERNAL-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW /* kubernetes externally-visible service portals */
29608 8435K DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0
29608 8435K DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0
 5142 6792K ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
11502  673K ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      br-421a9e16d12b  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-421a9e16d12b  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-421a9e16d12b !br-421a9e16d12b  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-421a9e16d12b br-421a9e16d12b  0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT 1567K packets, 526M bytes)
 pkts bytes target     prot opt in     out     source               destination
 484K   30M KUBE-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW /* kubernetes service portals */
  93M   34G KUBE-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp  --  !docker0 docker0  0.0.0.0/0            172.17.0.2           tcp dpt:443
    0     0 ACCEPT     tcp  --  !docker0 docker0  0.0.0.0/0            172.17.0.2           tcp dpt:80

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
11502  673K DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  br-421a9e16d12b !br-421a9e16d12b  0.0.0.0/0            0.0.0.0/0
29608 8435K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  *      br-421a9e16d12b  0.0.0.0/0            0.0.0.0/0
11502  673K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
29608 8435K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain IN_WEB (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain KUBE-EXTERNAL-SERVICES (2 references)
 pkts bytes target     prot opt in     out     source               destination

Chain KUBE-FIREWALL (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
    0     0 DROP       all  --  *      *      !127.0.0.0/8          127.0.0.0/8          /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT

Chain KUBE-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes forwarding rules */ mark match 0x4000/0x4000
20629 9087K ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED

Chain KUBE-KUBELET-CANARY (0 references)
 pkts bytes target     prot opt in     out     source               destination

Chain KUBE-NODEPORTS (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/ingress-nginx-controller:https health check node port */ tcp dpt:30380
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/ingress-nginx-controller:http health check node port */ tcp dpt:30380

Chain KUBE-PROXY-CANARY (0 references)
 pkts bytes target     prot opt in     out     source               destination
Chain KUBE-SERVICES (2 references)
 pkts bytes target     prot opt in     out     source               destination

我们可以看到只有在nat和filter上定义了防火墙规则。具体的,我们可以在nat这个表上看下KUBE-SERVICES这个链

代码语言:javascript复制
/ # iptables -t nat -L KUBE-SERVICES
Chain KUBE-SERVICES (2 references)
target     prot opt source               destination
KUBE-SVC-DISNXZXWEI7GIGLU  tcp  --  anywhere             10.101.247.125       /* cattle-system/cattle-cluster-agent:https-internal cluster IP */ tcp dpt:https
KUBE-SVC-EDBHCP4VQID7F5J2  tcp  --  anywhere             10.111.136.178       /* kube-system/etcd-svc-docker-desktop-xzm cluster IP */ tcp dpt:2379
KUBE-SVC-Q7CDIBSFDYNOJNFE  tcp  --  anywhere             10.108.133.209       /* default/ingress-nginx-controller:https cluster IP */ tcp dpt:https
KUBE-SVC-CKFHGLZY3HDORVFT  tcp  --  anywhere             10.105.105.140       /* default/redis:tcp cluster IP */ tcp dpt:6379
KUBE-SVC-D7TXZ2ONB4DT7BQA  tcp  --  anywhere             10.108.133.209       /* default/ingress-nginx-controller:http cluster IP */ tcp dpt:http
KUBE-SVC-NPX46M4PTMTKRN6Y  tcp  --  anywhere             10.96.0.1            /* default/kubernetes:https cluster IP */ tcp dpt:https
KUBE-SVC-XUD33RTORZBRAEIL  tcp  --  anywhere             10.99.126.26         /* default/ingress-nginx-controller-admission:https-webhook cluster IP */ tcp dpt:https
KUBE-SVC-ED7LY7V3PRCUB6IJ  tcp  --  anywhere             10.99.62.36          /* default/minio-service:console cluster IP */ tcp dpt:9001
KUBE-SVC-JD5MR3NA4I4DYORP  tcp  --  anywhere             10.96.0.10           /* kube-system/kube-dns:metrics cluster IP */ tcp dpt:9153
KUBE-SVC-TCOU7JCQXEZGVUNU  udp  --  anywhere             10.96.0.10           /* kube-system/kube-dns:dns cluster IP */ udp dpt:domain
KUBE-SVC-Y4TE457BRBWMNDKG  tcp  --  anywhere             10.105.42.239        /* default/apple-service cluster IP */ tcp dpt:5678
KUBE-SVC-SXW22BMJJ7T3N2OP  tcp  --  anywhere             10.99.62.36          /* default/minio-service:api cluster IP */ tcp dpt:9000
KUBE-SVC-ERIFXISQEP7F7OF4  tcp  --  anywhere             10.96.0.10           /* kube-system/kube-dns:dns-tcp cluster IP */ tcp dpt:domain
KUBE-SVC-RXZQBFX6IWO22WWW  tcp  --  anywhere             10.101.247.125       /* cattle-system/cattle-cluster-agent:http cluster IP */ tcp dpt:http
KUBE-NODEPORTS  all  --  anywhere             anywhere             /* kubernetes service nodeports; NOTE: this must be the last rule in this chain */ ADDRTYPE match dst-type LOCAL

现在以具体服务为例来进行分析

代码语言:javascript复制
% kubectl get svc
NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                         AGE
apple-service                        NodePort       10.105.42.239    <none>        5678:30080/TCP                  94d

服务是NodePort方式,可以看到在nat表 KUBE-SERVICES 这条链上定义了一条规则,它跳转到了一条和服务相关的具体链KUBE-SVC-Y4TE457BRBWMNDKG

代码语言:javascript复制
/ # iptables -t nat -L KUBE-SERVICES |egrep apple-service
KUBE-SVC-Y4TE457BRBWMNDKG  tcp  --  anywhere             10.105.42.239        /* default/apple-service cluster IP */ tcp dpt:5678
代码语言:javascript复制
/ # iptables -t filter -L KUBE-SERVICES |egrep apple-service

我们看下这条链的具体内容:

代码语言:javascript复制
/ #  iptables -t nat -L KUBE-SVC-Y4TE457BRBWMNDKG
Chain KUBE-SVC-Y4TE457BRBWMNDKG (2 references)
target     prot opt source               destination
KUBE-SEP-N6ACUILWO7XJGY5F  all  --  anywhere             anywhere             /* default/apple-service */

它跳转到了一条endpints关联的规则链KUBE-SEP-N6ACUILWO7XJGY5F

代码语言:javascript复制
/ #  iptables -t nat -L KUBE-SEP-N6ACUILWO7XJGY5F
Chain KUBE-SEP-N6ACUILWO7XJGY5F (1 references)
target     prot opt source               destination
KUBE-MARK-MASQ  all  --  10.1.0.217           anywhere             /* default/apple-service */
DNAT       tcp  --  anywhere             anywhere             /* default/apple-service */ tcp to:10.1.0.217:5678

在这条链里执行力DNAT规则,修改了目标地址和端口到我们的pod地址端口。

另一条规则链,仅仅是做了流量标记

代码语言:javascript复制
/ #  iptables -t nat -L KUBE-MARK-MASQ
Chain KUBE-MARK-MASQ (26 references)
target     prot opt source               destination
MARK       all  --  anywhere             anywhere             MARK or 0x4000

接着我们分析下,系统的kube-dns服务的规则链

代码语言:javascript复制
~ % kubectl get svc -n kube-system
NAME                          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
etcd-svc-docker-desktop-xzm   NodePort    10.111.136.178   <none>        2379:32389/TCP           23d
kube-dns                      ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP   96d

它是ClusterIP模式工作的。

代码语言:javascript复制
/ # iptables -t nat -L KUBE-SERVICES |grep kube-dns
KUBE-SVC-JD5MR3NA4I4DYORP  tcp  --  anywhere             10.96.0.10           /* kube-system/kube-dns:metrics cluster IP */ tcp dpt:9153
KUBE-SVC-TCOU7JCQXEZGVUNU  udp  --  anywhere             10.96.0.10           /* kube-system/kube-dns:dns cluster IP */ udp dpt:domain
KUBE-SVC-ERIFXISQEP7F7OF4  tcp  --  anywhere             10.96.0.10           /* kube-system/kube-dns:dns-tcp cluster IP */ tcp dpt:domain

它对应了三条规则链,分别是监控、udp 和tcp服务

先看下UDP这条链

代码语言:javascript复制
/ # iptables -t nat -L KUBE-SVC-TCOU7JCQXEZGVUNU
Chain KUBE-SVC-TCOU7JCQXEZGVUNU (1 references)
target     prot opt source               destination
KUBE-SEP-JPYJVE4BPHWKYBDW  all  --  anywhere             anywhere             /* kube-system/kube-dns:dns */ statistic mode random probability 0.50000000000
KUBE-SEP-GXFFDQL3JO5LZDO7  all  --  anywhere             anywhere             /* kube-system/kube-dns:dns */

对应了两个endpoints,概率各50%

代码语言:javascript复制
/ # iptables -t nat -L KUBE-SEP-JPYJVE4BPHWKYBDW
Chain KUBE-SEP-JPYJVE4BPHWKYBDW (1 references)
target     prot opt source               destination
KUBE-MARK-MASQ  all  --  10.1.0.216           anywhere             /* kube-system/kube-dns:dns */
DNAT       udp  --  anywhere             anywhere             /* kube-system/kube-dns:dns */ udp to:10.1.0.216:53
代码语言:javascript复制
/ # iptables -t nat -L KUBE-SEP-GXFFDQL3JO5LZDO7
Chain KUBE-SEP-GXFFDQL3JO5LZDO7 (1 references)
target     prot opt source               destination
KUBE-MARK-MASQ  all  --  10.1.0.218           anywhere             /* kube-system/kube-dns:dns */
DNAT       udp  --  anywhere             anywhere             /* kube-system/kube-dns:dns */ udp to:10.1.0.218:53

里面照样是做了DNAT转换和流量标记

0 人点赞