docker 套娃在CI中应用解析

2024-06-12 15:33:47 浏览数 (2)

docker in docker 简介

docker里嵌套运行docker，本文讲解其在jenkins和gitlab-runner 中的调用流程

一、用于jenkins

容器化部署jenkins时调用docker命令集成CI功能

  1. [root@ops-demo~]# docker inspect jenkins --format="{{json .Mounts}}"
  2. [{"Type":"bind","Source":"/usr/bin/docker","Destination":"/usr/bin/docker","Mode":"","RW":true,"Propagation":"rprivate"},{"Type":"volume","Name":"jenkins","Source":"/var/lib/docker/volumes/jenkins/_data","Destination":"/var/jenkins_home","Driver":"local","Mode":"z","RW":true,"Propagation":""},{"Type":"bind","Source":"/var/run/docker.sock","Destination":"/var/run/docker.sock","Mode":"","RW":true,"Propagation":"rprivate"}]

调用流程

执行如下命令可以发现jenkins是使用挂载进宿主机的docker.sock 来调用的docker服务，即：在jenkins容器内和宿主机上执行docker命令，效果是一样的，操作的结果，容器内外查看效果一样，要是构建的时候在jenkins容器执行 docker stop jenkins 会发生什么？

  1. [root@ops-demo ~]# docker images | wc -l
  2. 124
  3. [root@dotnet ~]# docker exec -it jenkins bash
  4. jenkins@855107e4687c:/$ docker images | wc -l
  5. 124
  6. jenkins@855107e4687c:/$
  8. # 容器内外的镜像数据一样，说明用的是用一个docker服务

二、用于gitlab-runner

gitlab-runner可以用容器方式和非容器方式运行，本文讲docker套娃，于是采用容器方式运行gitlab-runner。

gitlab-runner的executor有多种，对于docker套娃有两种方式，下文讲第二种调用流程。

  • 1、像上文jenkins一样，直接挂载 /var/run/docker.sock
  • 2、通过在gitlab-runner里新建一个完整的docker服务，这个完整的服务docker官方有提供一个镜像: docker:dind

docker套娃调用流程解析

注意看中文注释

gitlab-runner config.toml 配置样例

  1. [[runners]]
  2. url = "https://gitlab.com/"
  3. token = TOKEN
  4. executor = "docker" #docker 执行器
  5. [runners.docker]
  6. tls_verify = false
  7. image = "docker:24.0.5" # 用于gitalb-runner容器里提供docker client
  8. privileged = true
  9. disable_cache = false
  10. volumes = ["/cache"]
  11. [runners.cache]
  12. [runners.cache.s3]
  13. [runners.cache.gcs]

gitlab Project中的 gitlab-ci.yaml 配置样例

  1. default:
  2. image: docker:24.0.5 #用于提供 docker client
  3. services:
  4. - docker:24.0.5-dind # 用于启动 Docker daemon
  5. before_script:
  6. - docker info
  7. variables:
  8. # When using dind service, you must instruct docker to talk with the
  9. # daemon started inside of the service. The daemon is available with
  10. # a network connection instead of the default /var/run/docker.sock socket. # 注意这句提示
  11. #
  12. # The 'docker' hostname is the alias of the service container as described at
  13. # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services
  14. #
  15. # If you're using GitLab Runner 12.7 or earlier with the Kubernetes executor and Kubernetes 1.6 or earlier,
  16. # the variable must be set to tcp://localhost:2375 because of how the
  17. # Kubernetes executor connects services to the job container
  18. # DOCKER_HOST: tcp://localhost:2375
  19. #
  20. DOCKER_HOST: tcp://docker:2376 #dind启动的Docker daemon所监听的tcp端口
  21. #
  22. # This instructs Docker not to start over TLS.
  23. DOCKER_TLS_CERTDIR: ""
  24. build:
  25. stage: build
  26. script:
  27. - docker build -t my-docker-image .
  28. - docker run my-docker-image /script/to/run/tests

启动docker:24.0.5看看

  1. [root@ops-demo ~]# docker run -it --privileged --name docker-client docker:24.0.5 sh/
  2. # docker pserror during connect: Get "http://docker:2375/v1.24/containers/json": dial tcp: lookup docker on xxx: no such host

可以看到默认请求的host是 docker ，但是端口为什么不是2376呢？再看

  1. [root@ops-demo ~]# docker run -it --privileged --name docker-client docker:24.0.5 sh
  2. / # env
  3. HOSTNAME=c3b5946753f1
  4. SHLVL=3
  5. HOME=/root
  6. DIND_COMMIT=d58df1fc6c866447ce2cd129af10e5b507705624
  7. TERM=xterm
  8. PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
  9. DOCKER_VERSION=24.0.5
  10. DOCKER_TLS_CERTDIR=/certs
  11. DOCKER_HOST=tcp://docker:2375 # 这里通过环境变量设置的，上文gitlab-ci.yaml中自定义为dind默认监听的端口2376
  12. DOCKER_BUILDX_VERSION=0.11.2
  13. DOCKER_COMPOSE_VERSION=2.21.0
  14. PWD=/
  15. / #

启动dind验证下监听的是不是2376端口？

time="2024-05-29T10:16:09.048087250Z" level=info msg="API listen on [::]:2376"

  1. [root@ops-demo~]# docker run --privileged --name dind docker:dind
  2. ...
  3. time="2024-05-29T10:16:08.938258364Z" level=info msg="Docker daemon" commit=8e96db1 containerd-snapshotter=false storage-driver=overlay2 version=26.1.3
  4. time="2024-05-29T10:16:08.939055975Z" level=info msg="Daemon has completed initialization"
  5. time="2024-05-29T10:16:09.047817847Z" level=info msg="API listen on /var/run/docker.sock"
  6. time="2024-05-29T10:16:09.048087250Z" level=info msg="API listen on [::]:2376"

调用流程

经过以上分析，可以捋出gitlab-runner docker套娃的工作流程

  • 1、宿主机docker启动gitlab-runner容器
  • 2、gitlab-runner调用 docker:24.0.5提供的docker client执行各类docker命令，包括启动docker:dind容器
  • 3、然后docker client调用的Docker daemon是docker:dind提供的

gitlab-runner的工作流程

给出一张gitlab-runner的工作流程图方便理解

gitlab 服务 配置 容器 ci

0 人点赞