在《研发工程师玩转Kubernetes——使用污点(taint)驱逐Pod》中我们提到亲和性(affinity)中的requiredDuringSchedulingIgnoredDuringExecution,它可以定向调度Pod。本节我们将使用相关特性完成定向调度的介绍。
nodeSelector定向调度
我们先模拟Pod被部署在Master Node上的场景。这个时候我们可以使用nodeSelector对Node的Labels做选择。
首先我们查看当前的Node的情况:
查看Node的Labels
代码语言:javascript复制get nodes --show-labels
代码语言:javascript复制NAME STATUS ROLES AGE VERSION LABELS
ubuntuc Ready <none> 16m v1.26.4 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=ubuntuc,kubernetes.io/os=linux,microk8s.io/cluster=true,node.kubernetes.io/microk8s-worker=microk8s-worker
ubuntue Ready <none> 15m v1.26.4 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=ubuntue,kubernetes.io/os=linux,microk8s.io/cluster=true,node.kubernetes.io/microk8s-worker=microk8s-worker
ubuntud Ready <none> 15m v1.26.4 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=ubuntud,kubernetes.io/os=linux,microk8s.io/cluster=true,node.kubernetes.io/microk8s-worker=microk8s-worker
ubuntub Ready <none> 16m v1.26.4 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=ubuntub,kubernetes.io/os=linux,microk8s.io/cluster=true,node.kubernetes.io/microk8s-worker=microk8s-worker
ubuntua Ready <none> 20m v1.27.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=ubuntua,kubernetes.io/os=linux,microk8s.io/cluster=true,node.kubernetes.io/microk8s-controlplane=microk8s-controlplane
可以看到只有Master Node(UbuntuA)的Labels含有node.kubernetes.io/microk8s-controlplane=microk8s-controlplane,我们就用这个去做条件选择。
清单文件指定Label
代码语言:javascript复制# nginx_deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx-container
image: nginx
ports:
- containerPort: 80
nodeSelector:
node.kubernetes.io/microk8s-controlplane: microk8s-controlplane
注意最后两行,即使用nodeSelector让Pod在Labels含有node.kubernetes.io/microk8s-controlplane: microk8s-controlplane的Node上部署。
代码语言:javascript复制NodeSelector map[string]string `json:"nodeSelector,omitempty"`
部署
针对上述文件部署Pod
代码语言:javascript复制kubectl create -f nginx_deployment.yaml
代码语言:javascript复制deployment.apps/nginx-deployment created
kubectl get pods -o wide
代码语言:javascript复制NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deployment-5d5bc8fb96-75c9d 1/1 Running 0 67s 10.1.94.67 ubuntua <none> <none>
可以看到Pod被部署到Master Node上了。
requiredDuringSchedulingIgnoredDuringExecution定向调度
requiredDuringSchedulingIgnoredDuringExecution是亲和性(affinity)下的一种类型,它表示:只有在规则被满足的时候才能执行调度。
我们修改下清单文件,让Pod只能被调度到Master Node(UbuntuA)上。
清单文件指定亲和性
代码语言:javascript复制# nginx_deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node.kubernetes.io/microk8s-controlplane
operator: In
values:
- microk8s-controlplane
containers:
- name: nginx-container
image: nginx
ports:
- containerPort: 80
nodeSelectorTerms只能是一组matchExpressions。
代码语言:javascript复制type NodeAffinity struct {
// If the affinity requirements specified by this field are not met at
// scheduling time, the pod will not be scheduled onto the node.
// If the affinity requirements specified by this field cease to be met
// at some point during pod execution (e.g. due to a node label update),
// the system will try to eventually evict the pod from its node.
RequiredDuringSchedulingRequiredDuringExecution *NodeSelector `json:"requiredDuringSchedulingRequiredDuringExecution,omitempty"`
// If the affinity requirements specified by this field are not met at
// scheduling time, the pod will not be scheduled onto the node.
// If the affinity requirements specified by this field cease to be met
// at some point during pod execution (e.g. due to a node label update),
// the system may or may not try to eventually evict the pod from its node.
RequiredDuringSchedulingIgnoredDuringExecution *NodeSelector `json:"requiredDuringSchedulingIgnoredDuringExecution,omitempty"`
// The scheduler will prefer to schedule pods to nodes that satisfy
// the affinity expressions specified by this field, but it may choose
// a node that violates one or more of the expressions. The node that is
// most preferred is the one with the greatest sum of weights, i.e.
// for each node that meets all of the scheduling requirements (resource
// request, RequiredDuringScheduling affinity expressions, etc.),
// compute a sum by iterating through the elements of this field and adding
// "weight" to the sum if the node matches the corresponding MatchExpressions; the
// node(s) with the highest sum are the most preferred.
PreferredDuringSchedulingIgnoredDuringExecution []PreferredSchedulingTerm `json:"preferredDuringSchedulingIgnoredDuringExecution,omitempty"`
}
// A node selector represents the union of the results of one or more label queries
// over a set of nodes; that is, it represents the OR of the selectors represented
// by the nodeSelectorTerms.
type NodeSelector struct {
// nodeSelectorTerms is a list of node selector terms. The terms are ORed.
NodeSelectorTerms []NodeSelectorTerm `json:"nodeSelectorTerms,omitempty"`
}
// An empty node selector term matches all objects. A null node selector term
// matches no objects.
type NodeSelectorTerm struct {
// matchExpressions is a list of node selector requirements. The requirements are ANDed.
MatchExpressions []NodeSelectorRequirement `json:"matchExpressions,omitempty"`
}
// A node selector requirement is a selector that contains values, a key, and an operator
// that relates the key and values.
type NodeSelectorRequirement struct {
// key is the label key that the selector applies to.
Key string `json:"key" patchStrategy:"merge" patchMergeKey:"key"`
// operator represents a key's relationship to a set of values.
// Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.
Operator NodeSelectorOperator `json:"operator"`
// values is an array of string values. If the operator is In or NotIn,
// the values array must be non-empty. If the operator is Exists or DoesNotExist,
// the values array must be empty. If the operator is Gt or Lt, the values
// array must have a single element, which will be interpreted as an integer.
// This array is replaced during a strategic merge patch.
Values []string `json:"values,omitempty"`
}
// A node selector operator is the set of operators that can be used in
// a node selector requirement.
type NodeSelectorOperator string
const (
NodeSelectorOpIn NodeSelectorOperator = "In"
NodeSelectorOpNotIn NodeSelectorOperator = "NotIn"
NodeSelectorOpExists NodeSelectorOperator = "Exists"
NodeSelectorOpDoesNotExist NodeSelectorOperator = "DoesNotExist"
NodeSelectorOpGt NodeSelectorOperator = "Gt"
NodeSelectorOpLt NodeSelectorOperator = "Lt"
)
观察
代码语言:javascript复制get pod --watch -o wide
代码语言:javascript复制NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deployment-549db84c68-gk66n 0/1 Pending 0 0s <none> <none> <none> <none>
nginx-deployment-549db84c68-gk66n 0/1 Pending 0 0s <none> ubuntua <none> <none>
nginx-deployment-549db84c68-gk66n 0/1 ContainerCreating 0 0s <none> ubuntua <none> <none>
nginx-deployment-549db84c68-gk66n 0/1 ContainerCreating 0 1s <none> ubuntua <none> <none>
nginx-deployment-549db84c68-gk66n 1/1 Running 0 4s 10.1.94.80 ubuntua <none> <none>
可以看到Pod是在Worker Node(UbuntuA)上部署着。
参考资料
- https://kubernetes.io/zh-cn/docs/concepts/scheduling-eviction/assign-pod-node/
- https://github.com/kubernetes/design-proposals-archive/blob/main/scheduling/podaffinity.md