TKE使用自带nginx-ingress组件实现tcp/udp转发

在TKE内将服务直接通过端口暴露到外网，可以使用CLB类型service，或者nginx-ingress方式实现，最近处理问题时遇到用户需要将ingress跟service同时使用nginx-ingress方式暴露，不想额外使用CLB，这里就涉及到通过nginx-ingress组件暴露四层TCP/udp的问题

下面以实际在TKE部署一个websocket服务，通过nginx-ingress的四层转发实现

1. 现在集群内部署一个deployment类型工作负载，同步创建service（service访问类型为仅在集群内访问）

代码语言：javascript复制

apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    k8s-app: websocket-server
    qcloud-app: websocket-server
  name: websocket-server
  namespace: default
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: websocket-server
      qcloud-app: websocket-server
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        k8s-app: websocket-server
        qcloud-app: websocket-server
    spec:
      containers:
      - env:
        - name: PATH
          value: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
        - name: NGINX_VERSION
          value: 1.21.3
        - name: NJS_VERSION
          value: 0.6.2
        - name: PKG_RELEASE
          value: 1~buster
        image: ccr.ccs.tencentyun.com/v_czhecheng/v_czhecheng-demo:webserver
        imagePullPolicy: IfNotPresent
        name: server
        resources:
          limits:
            cpu: 200m
            memory: 256Mi
          requests:
            cpu: 100m
            memory: 128Mi
        securityContext:
          privileged: false
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      imagePullSecrets:
      - name: qcloudregistrykey
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30

---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: websocket-server
    qcloud-app: websocket-server
  name: websocket-server
  namespace: default
spec:
  ports:
  - name: 8888-8888-tcp
    port: 8888
    protocol: TCP
    targetPort: 8888
  selector:
    k8s-app: websocket-server
    qcloud-app: websocket-server
  sessionAffinity: None
  type: ClusterIP

2. 按照TKE官方文档部署好nginx-ingress组件

https://cloud.tencent.com/document/product/457/50503

3. 部署好nginx-ingress后，默认会在kube-system namespace下生成tcp/udp 转发 configmap 配置资源

名称一般是：xxxxxx-ingress-nginx-tcp

修改这个configmap，添加我们需要配置的TCP转发规则

代码语言：javascript复制

apiVersion: v1
kind: ConfigMap
metadata:
  creationTimestamp: "2022-03-06T05:45:08Z"
  labels:
    k8s-app: webserver-ingress-nginx-tcp
    qcloud-app: webserver-ingress-nginx-tcp
  name: webserver-ingress-nginx-tcp
  namespace: kube-system
data:
  8888: "default/websocket-server:8888"     // 将8888端口流量转发到default ns下的websocket-server:8888

4. TKE内的nginx-ingress组件是通过外网CLB类型的service资源提供外网访问的，因此我们同时需要在 nginx-ingress的 service 内配置添加我们暴露的TCP端口

代码语言：javascript复制

apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: webserver-ingress-nginx-controller
    qcloud-app: webserver-ingress-nginx-controller
  name: webserver-ingress-nginx-controller
  namespace: kube-system
spec:
  externalTrafficPolicy: Cluster
  ports:
  - name: http
    nodePort: 31711
    port: 80
    protocol: TCP
    targetPort: http
  - name: https
    nodePort: 31221
    port: 443
    protocol: TCP
    targetPort: https
  - name: 8888-8888-tcp-3yz6ymqt2nw    // 添加下面的配置即可，添加8888端口的转发，nodePort端口会自动生成，无需填写
    nodePort: 31795
    port: 8888
    protocol: TCP
    targetPort: 8888
  selector:
    k8s-app: webserver-ingress-nginx-controller
    qcloud-app: webserver-ingress-nginx-controller
  sessionAffinity: None
  type: LoadBalancer

5. 测试，通过postman访问nginx-ingress的公网地址 port ,看到已经可以正常建立websocket连接

发

容器服务

0 人点赞