K8S中提供了自定义资源类型和自定义控制器来扩展功能,还提供了动态准入控制,其实就是通过Webhook来实现准入控制,分为两种:验证性质的准入 Webhook (Validating Admission Webhook) 和 修改性质的准入 Webhook (Mutating Admission Webhook)。Admission controllers是在api server上非常有用的拦截器,但是灵活性不够,从k8s1.7开始,引入了Initializers 和 External Admission Webhooks;从k8s1.9开始External Admission Webhooks 被拆分成 MutatingAdmissionWebhook 和 ValidatingAdmissionWebhook两个部分。
Initializers的局限性在于它只能对请求进行修改,不能做认证。下面我们分析下MutatingAdmissionWebhook的代码实现。https://github.com/morvencao/kube-sidecar-injector,我们还是从main函数开始,源码位于cmd/main.go首先完成了配置的初始化加载和认证:
代码语言:javascript复制 dnsNames := []string{
webhookServiceName,
webhookServiceName "." webhookNamespace,
webhookServiceName "." webhookNamespace ".svc",
}
org := "morven.me"
caPEM, certPEM, certKeyPEM, err := generateCert([]string{org}, dnsNames, commonName)
pair, err := tls.X509KeyPair(certPEM.Bytes(), certKeyPEM.Bytes())
sidecarConfig, err := loadConfig(sidecarConfigFile)
然后初始化更新或者创建webhook的配置,接着启动了一个 WebhookServer,本质是是一个https的服务
代码语言:javascript复制err = createOrUpdateMutatingWebhookConfiguration(caPEM, webhookServiceName, webhookNamespace)
whsvr := &WebhookServer{
sidecarConfig: sidecarConfig,
server: &http.Server{
Addr: fmt.Sprintf(":%v", port),
TLSConfig: &tls.Config{Certificates: []tls.Certificate{pair}},
},
}
mux.HandleFunc(webhookInjectPath, whsvr.serve)
下面看下认证的过程,源码位于cmd/cert.go:
代码语言:javascript复制func generateCert(orgs, dnsNames []string, commonName string) (*bytes.Buffer, *bytes.Buffer, *bytes.Buffer, error)
caPrivateKey, err := rsa.GenerateKey(rand.Reader, 4096)
caBytes, err := x509.CreateCertificate(rand.Reader, ca, ca, &caPrivateKey.PublicKey, caPrivateKey)
newPrivateKey, err := rsa.GenerateKey(rand.Reader, 4096)
newCertBytes, err := x509.CreateCertificate(rand.Reader, newCert, ca, &newPrivateKey.PublicKey, caPrivateKey)
加载配置的过程就是打开yaml文件然后unmarshal,源码位于cmd/webhook.go
代码语言:javascript复制func loadConfig(configFile string) (*Config, error)
data, err := ioutil.ReadFile(configFile)
if err := yaml.Unmarshal(data, &cfg);
webhookserver除了包括sidecar的配置外,还内嵌了一个httpserver
代码语言:javascript复制type WebhookServer struct {
sidecarConfig *Config
server *http.Server
}
接着我们重点看下它的serve方法,也是webhook的核心:
代码语言:javascript复制 func (whsvr *WebhookServer) serve(w http.ResponseWriter, r *http.Request)
admissionResponse = whsvr.mutate(&ar)
annotations := map[string]string{admissionWebhookAnnotationStatusKey: "injected"}
patchBytes, err := createPatch(&pod, whsvr.sidecarConfig, annotations)
patch = append(patch, addContainer(pod.Spec.Containers, sidecarConfig.Containers, "/spec/containers")...)
patch = append(patch, patchOperation{
Op: "add",
Path: path,
Value: value,
})
patch = append(patch, addVolume(pod.Spec.Volumes, sidecarConfig.Volumes, "/spec/volumes")...)
patch = append(patch, patchOperation{
Op: "add",
Path: path,
Value: value,
})
patch = append(patch, updateAnnotation(pod.Annotations, annotations)...)
里面主要和原来资源进行比对,然后完成了资源的新建,增加和修改。
代码语言:javascript复制type Config struct {
Containers []corev1.Container `yaml:"containers"`
Volumes []corev1.Volume `yaml:"volumes"`
}
代码语言:javascript复制type patchOperation struct {
Op string `json:"op"`
Path string `json:"path"`
Value interface{} `json:"value,omitempty"`
}
核心内容是根据请求,修改container,volume,annotation,序列化后返回。因此webhook核心也就是修改container,volume和annotation的信息。
最后看下createOrUpdateMutatingWebhookConfiguration的实现:cmd/webhookconfig.go
代码语言:javascript复制config, err := clientcmd.BuildConfigFromFlags("", kubeconfig)
clientset, err := kubernetes.NewForConfig(config)
mutatingWebhookConfigV1Client := clientset.AdmissionregistrationV1()
mutatingWebhookConfig := &admissionregistrationv1.MutatingWebhookConfiguration{
ObjectMeta: metav1.ObjectMeta{
Name: webhookConfigName,
},
Webhooks: []admissionregistrationv1.MutatingWebhook{{
Name: "sidecar-injector.morven.me",
foundWebhookConfig, err := mutatingWebhookConfigV1Client.MutatingWebhookConfigurations().Get(context.TODO(), webhookConfigName, metav1.GetOptions{})
if _, err := mutatingWebhookConfigV1Client.MutatingWebhookConfigurations().Create(context.TODO(), mutatingWebhookConfig, metav1.CreateOptions{});
if _, err := mutatingWebhookConfigV1Client.MutatingWebhookConfigurations().Update(context.TODO(), mutatingWebhookConfig, metav1.UpdateOptions{});
webhookInjectPath = "/inject"
主要是先获取clientsets,然后得到mutatingWebhookConfigV1Client,接着创建mutatingWebhookConfig,接着根据不同场景完成webhook的创建和更新。