[TOC]
0x00 前言简述
PS: 在开源世界中,是没有终结的尽头!
描述:GitLab 是一个非常优秀的开源项目,基于Ruby on Rails开发的开源应用程序。它允许用户在自己的服务器上运行类似于 GitHub 的项目管理系统,实现一个自托管私有的Git项目仓库,可通过Web界面进行访问公开的或者私人的项目Gitlab能够浏览源代码,管理缺陷和注释。
官网地址:https://about.gitlab.com/ 帮助文档:https://about.gitlab.com/install/ 组件参考:https://docs.gitlab.com/ce/development/architecture.html
GitLab 采用传统的开源商业模式,他们有两种产品:
- 免费的开源软件,用户可以在自己的服务器上安装,以及类似于 GitHub 的托管服务。
- 免费的社区版
Gitlab CE
和付费企业版Gitlab EE
- 企业版基于社区版但附带针对企业客户的其他功能,它或多或少与 WordPress.org 或 Wordpress.com 提供的服务类似。
Q: Gitlab 优势及应用场景?
- 1.开源免费,搭建简单,维护成本低,适合中小型公司;
- 2.权限管理,能实现代码对部分人可见,确保项目的安全性;
- 3.离线同步,保证我们不在实时依赖网络环境进行代码提交;
Gitlab优点:
- 有开源免费的版本,可以进行私有开发上传与拉取;
- 社区版具有高度可扩展性,可以在单个服务器或群集上支持 25000 个用户
- GitLab 的一些功能包括:Git 仓库管理,代码评论,问题跟踪,活动源和维基
- 它配备了 GitLab CI,用于持续集成和交付
0x01 环境安装
CentOS
Gitlab 基础要求:至少4GB的空闲RAM来运行GitLab
系统:CentOS Linux release 8.0.1905 (Core)
代码语言:javascript复制GitLab 12.9.2 (ac5568eb5d8)
GitLab Shell 12.0.0
GitLab Workhorse v8.25.1
GitLab APIv 4
Ruby 2.6.5p114
Rails 6.0.2
PostgreSQL 10.12
Gitaly Servers
Gitlab 安装方式:
官方推荐使用Omnibus快速安装(采用rpm软件包进行安装部署(国内推荐直接镜像源下载));
代码语言:javascript复制#此处以社区版本为例
curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | sudo bash
sudo EXTERNAL_URL="https://gitlab.example.com" dnf install -y gitlab-ee
采用yum进行安装部署;
安装流程: Step1.采用rpm进行安装社区版gitlab(替换更新源自己选择即可):
代码语言:javascript复制sudo dn8 install -y curl policycoreutils openssh-server wget
#可以使用 wget 的方式把 rpm 包下载下来安装
wget https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el8/gitlab-ce-12.9.2-ce.0.el8.x86_64.rpm
#dnf install gitlab-ce-12.9.2-ce.0.el8.x86_64.rpm
rpm -ivh gitlab-ce-12.9.2-ce.0.el8.x86_64.rpm
Step2.安装完成后显示以下则说明安装成功:
代码语言:javascript复制# Thank you for installing GitLab!
# GitLab was unable to detect a valid hostname for your instance.
# Please configure a URL for your GitLab instance by setting `external_url`
# configuration in /etc/gitlab/gitlab.rb file.
# Then, you can start your GitLab instance by running the following command:
# sudo gitlab-ctl reconfigure
Step3.修改 gitlab 默认配置
代码语言:javascript复制$vim /etc/gitlab/gitlab.rb
# (1) 站点域名配置,修改为你自己的域名或者 IP,是单引号,而且前面的 http 不要改 (并且将该域名加入到hosts中)
external_url 'http://gitlab.weiyigeek.top'
# 邮件配置
# 启动 smtp
gitlab_rails['smtp_enable'] = false
gitlab_rails['smtp_address'] = "smtp.qq.com"
gitlab_rails['smtp_port'] = 587 #SSL
# 账户邮箱密码
gitlab_rails['smtp_user_name'] = "weiyigeek"
gitlab_rails['smtp_password'] = "password"
gitlab_rails['smtp_authentication'] = ":plain"
gitlab_rails['smtp_enable_starttls_auto'] = true
# gitlab_rails['smtp_tls'] =
# 发件人邮箱即显示名称
# PS:没有邮件服务器可以关闭邮件服务功能
gitlab_rails['gitlab_email_enabled'] = true
gitlab_rails['gitlab_email_from'] = "master#weiyigeek.top"
gitlab_rails['gitlab_email_display_name'] = 'GitLab-System-WeiyiGeek'
Step4.完成修改后需要重新加载配置然后重启即可;
代码语言:javascript复制$ gitlab-ctl reconfigure
# [2020-04-15T10:33:44 08:00] WARN: Please install an English UTF-8 locale for Chef to use, falling back to C locale and disabling UTF-8 support. #如果看着不舒服自己设置系统语系即可
# [2020-04-15T10:33:45 08:00] WARN: Please install an English UTF-8 locale for Chef to use, falling back to C locale and disabling UTF-8 support.
# Starting Chef Client, version 14.14.29
# resolving cookbooks for run list: ["gitlab"]
# Synchronizing Cookbooks:
# - redis (0.1.0)
# - package (0.1.0)
# - postgresql (0.1.0)
# - mattermost (0.1.0)
# - consul (0.1.0)
# - letsencrypt (0.1.0)
# - gitlab (0.0.1)
# - runit (4.3.0)
# - monitoring (0.1.0)
# - praefect (0.1.0)
# - gitaly (0.1.0)
# - registry (0.1.0)
# - acme (4.1.1)
# - nginx (0.1.0)
# - crond (0.1.0)
# Running handlers:
# Running handlers complete
# Chef Client finished, 541/1460 resources updated in 03 minutes 14 seconds
# gitlab Reconfigured!
$ gitlab-ctl restart
# [2020-04-15T10:37:51 08:00] WARN: Please install an English UTF-8 locale for Chef to use, falling back to C locale and disabling UTF-8 support.
# ok: run: alertmanager: (pid 21410) 1s
# ok: run: gitaly: (pid 21422) 0s
# ok: run: gitlab-exporter: (pid 21429) 1s
# ok: run: gitlab-workhorse: (pid 21444) 0s
# ok: run: grafana: (pid 21463) 0s
# ok: run: logrotate: (pid 21474) 0s
# ok: run: nginx: (pid 21480) 0s
# ok: run: node-exporter: (pid 21563) 1s
# ok: run: postgres-exporter: (pid 21569) 0s
# ok: run: postgresql: (pid 21580) 0s
# ok: run: prometheus: (pid 21589) 0s
# ok: run: redis: (pid 21601) 0s
# ok: run: redis-exporter: (pid 21708) 0s
# ok: run: sidekiq: (pid 21716) 0s
# ok: run: unicorn: (pid 21727) 0s
Step5.如果 reconfigure 失败,则需要 systemctl enable gitlab-runsvdir && systemctl restart gitlab-runsvdir
重启一下 gitlab-runsvdir 服务
Step6.打开浏览器进行初始化账户设定密码,这个密码为 root 管理员账户的密码。设置完密码之后会自动跳转到登录页面。
代码语言:javascript复制应用:http://gitlab.weiyigeek.top/
账号:root
密码:WeiyiGeek #8位及以上
WeiyiGeek.
Step7.登录成功的界面,是不是有种疯狂的想写代码的冲动;
WeiyiGeek.
Step8.英文可能对英语不好的新手可能不友好,我们进行汉化(注意:高于12.3.5的版本无需汉化,直接在用户设置里面进行设置自定义语言选择简体中文
);
#停止gitlab
gitlab-ctl stop
# 获取当前安装的版本补丁
git clone https://gitlab.com/xhang/gitlab.git
cd gitlab
# 查看全部分支版本
git branch -a
gitlab_version=$(cat /opt/gitlab/embedded/service/gitlab-rails/VERSION)
# 比较汉化标签和原标签,导出patch用的diff文件
#git diff v${gitlab_version} v${gitlab_version}-zh > ../${gitlab_version}-zh.diff
git diff remotes/origin/12-3-stable remotes/origin/12-3-stable-zh > ../${gitlab_version}-zh.diff
#打补丁的时候会提示一些补丁文件不存在,一定要跳过这些文件,不然后面reconfig的时候会报错的。
patch -d /opt/gitlab/embedded/service/gitlab-rails -p1 < ../${gitlab_version}-zh.diff
#重新编译和启动
gitlab-ctl reconfigure
gitlab-ctl start
Docker
描述: 采用采用一个低权限用户进行管理docker以及利用docker部署gitlab服务;
环境说明:
代码语言:javascript复制$lsb_release -a
Description: Ubuntu 20.04.1 LTS
Release: 20.04
Codename: focal
$docker version
Client: Docker Engine - Community
Version: 19.03.13
API version: 1.40
Go version: go1.13.15
Git commit: 4484c46d9d
Built: Wed Sep 16 17:02:52 2020
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.13
API version: 1.40 (minimum version 1.12)
Go version: go1.13.15
Git commit: 4484c46d9d
Built: Wed Sep 16 17:01:20 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.3.7
GitCommit: 8fba4e9a7d01810a393d5d25a3621dc101981175
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit: fec3683
操作流程:
代码语言:javascript复制# 1.新建低权限用户并将该用户加入到docker组
sudo useradd -m -g docker WeiyiGeek-gitlab && sudo passwd WeiyiGeek-gitlab
sudo gpasswd -a WeiyiGeek-gitlab docker
# 2.切换到低权限的WeiyiGeek-gitlab用户拉去最新社区版本的gitlab
WeiyiGeek-gitlab@gitlab-WeiyiGeek:~$ docker pull gitlab/gitlab-ce:13.6.3-ce.0
# Using default tag: latest
# latest: Pulling from gitlab/gitlab-ce
# 4f53fa4d2cf0: Pull complete
# 6af7c939e38e: Pull complete
# 903d0ffd64f6: Pull complete
# 04feeed388b7: Pull complete
# 25d5e5c7360d: Pull complete
# 0cc025692f2b: Pull complete
# eac308723fda: Pull complete
# 2135ce2185ba: Pull complete
# 3d2db784a8b0: Pull complete
# d2942fac7230: Pull complete
# Digest: sha256:b7daf0c109a62e776f5f72b728a01191059a51f43b5df82c53ef997e877a784f
# Status: Downloaded newer image for gitlab/gitlab-ce:latest
# docker.io/gitlab/gitlab-ce:latest
# 3.设置持久化目录并运行gitlab-server
$ mkdir -vp ~/{config,logs,data,backups}
# mkdir: created directory '/home/WeiyiGeek-gitlab/config'
# mkdir: created directory '/home/WeiyiGeek-gitlab/logs'
# mkdir: created directory '/home/WeiyiGeek-gitlab/data'
$ docker run -d -p 443:443 -p 80:80 -p 22:22 --name gitlab-server --restart always -v /home/WeiyiGeek-gitlab/config:/etc/gitlab -v /home/WeiyiGeek-gitlab/config/backups:/var/opt/gitlab/backups -v /home/WeiyiGeek-gitlab/logs:/var/log/gitlab -v /home/WeiyiGeek-gitlab/data:/var/opt/gitlab gitlab/gitlab-ce:13.6.3-ce.0
docker run -d -p 443:443 -p 80:80 -p 22:22 --name gitlab-server --restart always -v /home/weiyigeek/config:/etc/gitlab -v /home/weiyigeek/config/backups:/var/opt/gitlab/backups -v /home/weiyigeek/logs:/var/log/gitlab -v /home/weiyigeek/data:/var/opt/gitlab gitlab/gitlab-ce:13.6.3-ce.0
512c45077bafaf1c617ddd6e43f4b8a9a147ca3c8b9e8e889b14d58f151647fe
# 4.利用docker logs查看gitlab 初始化是否完成
WeiyiGeek-gitlab@gitlab-WeiyiGeek:~$ docker logs -f --tail 100 512c
==> /var/log/gitlab/puma/puma_stdout.log <==
{"timestamp":"2020-10-10T06:14:37.385Z","pid":767,"message":"PumaWorkerKiller: Consuming 3219.65625 mb with master and 4 workers."}
# 5.主机名称配置
sudo tee /etc/hosts <<'EOF'
192.168.10.222 gitlab.weiyigeek.top
EOF
PS: 配置文件地址/etc/WeiyiGeek-gitlab/config
如何配置请参考下面文章,此处不再多做叙述;
# 修改的键值对
sudo egrep -v "^#|^$" /home/WeiyiGeek-gitlab/config/gitlab.rb
[sudo] password for WeiyiGeek:
external_url 'http://gitlab.WeiyiGeek.top'
user['git_user_name'] = "Gitlab"
user['git_user_email'] = "gitlab@WeiyiGeek.top"
gitlab_rails['gitlab_ssh_host'] = 'gitlab.WeiyiGeek.top'
gitlab_rails['gitlab_shell_ssh_port'] = 2222
gitlab_rails['gitlab_shell_git_timeout'] = 800
gitlab_rails['time_zone'] = 'Asia/Shanghai'
gitlab_rails['gitlab_email_enabled'] = true
gitlab_rails['gitlab_email_from'] = 'gitlab@WeiyiGeek.top'
gitlab_rails['gitlab_email_display_name'] = 'Gitlab'
gitlab_rails['gitlab_email_reply_to'] = 'weiyigeek@WeiyiGeek.top'
gitlab_rails['gitlab_email_subject_suffix'] = 'WeiyiGeek-IT'
gitlab_rails['object_store']['enabled'] = false
gitlab_rails['object_store']['connection'] = {}
gitlab_rails['object_store']['storage_options'] = {}
gitlab_rails['object_store']['proxy_download'] = false
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = nil
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = nil
gitlab_rails['object_store']['objects']['lfs']['bucket'] = nil
gitlab_rails['object_store']['objects']['uploads']['bucket'] = nil
gitlab_rails['object_store']['objects']['packages']['bucket'] = nil
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = nil
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = nil
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.exmail.qq.com"
gitlab_rails['smtp_port'] = 465
gitlab_rails['smtp_user_name'] = "gitlab@WeiyiGeek.top"
gitlab_rails['smtp_password'] = "Hm595Yb"
gitlab_rails['smtp_domain'] = "WeiyiGeek.com.top"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_tls'] = true
# 修改后重启gitlab容器即可
$ docker restart gitlab
0x02 操作说明
Gitlab相关操作及说明:
代码语言:javascript复制#查看Gitlab版本
cat /opt/gitlab/embedded/service/gitlab-rails/VERSION
/etc/gitlab/gitlab.rb #gitlab配置文件
/opt/gitlab #gitlab的程序安装目录
/var/opt/gitlab #gitlab目录数据目录(通过gitlab.rb修改的子模块配置文件存放的)
/var/opt/gitlab/git-data #存放仓库数据
/var/opt/gitlab/backups #默认备份目录
#Gitlab的服务构成:
gitlab-shell:用于处理Git命令和修改authorized keys列表
gitlab-workhorse:轻量级的反向代理服务器
logrotate:日志文件管理工具
nginx:静态web服务器
postgresql:数据库
redis:缓存数据库
sidekiq:用于在后台执行队列任务(异步执行)
unicorn:GitLab Rails应用是托管在这个服务器上面的。
gitlab-ctl 命令
基础命令:
代码语言:javascript复制#常用命令
gitlab-ctl reconfigure #生成配置并重新启动服务
gitlab-ctl start #启动所有 gitlab 组件:
gitlab-ctl status #查看当前gitlab所有服务运行状态
gitlab-ctl stop #停止gitlab服务
gitlab-ctl pg-upgrade #升级PostgreSQL最新版本
#单独启动|停止|重启某个服务
gitlab-ctl start nginx #启动nginx组件
gitlab-ctl stop postgresql #停止所有 gitlab postgresql 组件:
gitlab-ctl restart unicorn #重启相关数据连接服务
gitlab-ctl restart sidekiq #重启相关数据连接服务
gitlab-ctl restart gitlab-workhorse # 重启所有 gitlab gitlab-workhorse 组件:
#日志管理
gitlab-ctl tail #查看所有服务的日志
gitlab-ctl tail redis #实时检查redis的日志(延伸某个服务也是一样得)
gitlab-rails 命令
基础示例:
代码语言:javascript复制#gitlab重置密码
gitlab-rails console
> u=User.where(id:1).first #这个是管理员的,也可以用email等
> u.password = 'your_password' #密码有格式限制,我只知道8位以上否则会保存失败
> u.password_confirmation = 'your_password'
> u.save
gitlab-rake 命令
基础示例:
代码语言:javascript复制gitlab-rake gitlab:check SANITIZE=true --trace # 检查gitlab
gitlab-rake db:migrate # 数据库关系升级
gitlab-rake cache:clear # 清理redis缓存
release-cli 命令
描述: release-cli 一个与GitLab发布API交互的CLI工具。
官方文档: https://gitlab.com/gitlab-org/release-cli/-/blob/master/docs/index.md#usage 参考地址: https://docs.gitlab.com/ee/api/releases/
语法基参 & Syntax
代码语言:javascript复制# USAGE
help [global options] command [command options] [arguments...]
# GLOBAL OPTIONS:
--server-url value The base URL of the GitLab instance, including protocol and port, for example https://gitlab.example.com:8080 [$CI_SERVER_URL]
--job-token value Job token used for authenticating with the GitLab Releases API [$CI_JOB_TOKEN]
--project-id value The current project unique ID; used by GitLab CI internally [$CI_PROJECT_ID]
--timeout value HTTP client timeout in Go duration format https://golang.org/pkg/time/#ParseDuration (default: 30s) [$RELEASE_CLI_TIMEOUT]
--private-token value Private token used for authenticating with the GitLab Releases API, requires api scope https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html, overrides job-token [$GITLAB_PRIVATE_TOKEN]
--help, -h Show help (default: false)
--version, -v Print the version (default: false)
基本命令 - COMMANDS
代码语言:javascript复制create: Create a Release using GitLab Releases API https://docs.gitlab.com/ee/api/releases/#create-a-release
USAGE:
help create [command options] [arguments...]
OPTIONS:
--name value # 发布名称例如 app_Version-1.14
--description value # 用来读取描述内容的文件,必须存在于工作目录内;如果它包含任何空格,它将被视为字符串
--tag-name value # 发布版本将从中创建的标记[$CI_COMMIT_TAG] 例如v1.14
--ref value # 它可以是一个提交SHA、另一个标记名或一个分支名 [$CI_COMMIT_SHA]
--assets-links-name value # DEPRECATED
--assets-links-url value # DEPRECATEDe.g. --assets-links-name "asset 1" --assets-links-url "https://example.com/url/1" --assets-links-name "asset 2" --assets-links-url "https://example.com/url/2")
--assets-link value # JSON字符串表示的资产链接用于替代--assets-links-name/url ( e.g. --assets-link='{"name": "Asset1", "url":"https://<domain>/some/location/1", "type": "other", "filepath": "xzy" }'
--milestone value # 与发行版相关的每个里程碑的标题列表 (里程碑必须存在)
--released-at value # 他标明了发行的日期;默认为当前时间;期望ISO 8601格式 (2019-03-15T08:00:00Z)
--help, -h Show help (default: false)
简单示例 - Example
代码语言:javascript复制# 方式1.命令行发布
release-cli --server-url https://gitlab.com --job-token=SOME_JOB_TOKEN --project-id 12345 create help
release-cli --server-url https://gitlab.mydomain.com --private-token "my-private-token" create --name "My Release" --description "This is a new release for my amazing tool"
# 方式2.Gitlab-CI 发布
release-branch:
stage: release
image: registry.gitlab.com/gitlab-org/release-cli
when: manual
# We recommend the use of `rules` to prevent these pipelines
# from running. See the notes section below for details.
rules:
- if: $CI_COMMIT_TAG
when: never
script:
- >
release-cli create --name release-branch-$CI_JOB_ID --description release-branch-$CI_COMMIT_REF_NAME-$CI_JOB_ID
--tag-name job-$CI_JOB_ID --ref $CI_COMMIT_SHA
--assets-link '{"name":"Asset1","url":"https://<domain>/some/location/1","link_type":"other","filepath":"xzy"}'
--assets-link '{"name":"Asset2","url":"https://<domain>/some/location/2"}'
--milestone "v1.0.0" --milestone "v1.0.0-rc"
--released-at "2020-06-30T07:00:00Z"
# 综合实践
$ release-cli create
--name "Release $CI_COMMIT_TAG"
--description "Notes: $EXTRA_DESCRIPTION"
--tag-name $CI_COMMIT_TAG --ref $CI_COMMIT_SHA
--assets-link '{"name":"Asset1","url":"https://<domain>/some/location/1","link_type":"other","filepath":"xzy"}'
--assets-link '{"name":"Asset2","url":"https://<domain>/some/location/2"}' --milestone "m1" --milestone "m2" --released-at "2020-08-20T6:42:00Z"
基础实例:
代码语言:javascript复制# (1) 描述信息
~ $ release-cli --server-url http://gitlab.weiyigeek.top --private-token "fYyHzos-zEPPP8PDsxLa" --project-id 45 create create --name "My Release" --description "This is a new release for my amazing tool"
# (2) 指定版本
~ $ release-cli --server-url http://gitlab.weiyigeek.top --private-token "fYyHzos-zEPPP8PDsxLa" --project-id 45 create --tag-name "v1.15" --name "My Release" --description "This is a new release for my amazing tool"
# INFO[0000] Creating Release...
# cli=release-cli command=create name="My Release" project-id=45 ref= server-url="http://gitlab.weiyigeek.top" tag-name=v1.15 version=0.6.0
# Tag: v1.15
# Name: My Release
# Description: This is a new release for my amazing tool
# Created At: 2021-02-10 14:52:17.08 0800 CST
# Released At: 2021-02-10 14:52:17.08 0800 CST
# See all available releases here: /-/releases
# INFO[0001] release created successfully!
# (3)指定RELEASE的外部URL
~ $ release-cli --server-url http://gitlab.weiyigeek.top --private-token "fYyHzos-zEPPP8PDsxLa" --project-id 45 create --tag-name "v1.14" --name "v1.15" --description "This is a new release for my amazing tool" --assets-link '{"name":
"info-student-rebuid","url":"http://192.168.12.107:30001/job/HelloWorld/19/artifact/target/info-student-rebuild-0.0.2-SNAPSHOT.jar","link_type":"other","filepath":"/binaries/linux-amd64"}'
# INFO[0000] Creating Release...
# cli=release-cli command=create name=v1.15 project-id=45 ref= server-url="http://gitlab.weiyigeek.top" tag-name=v1.14 version=0.6.0
# Tag: v1.14
# Name: v1.15
# Description: This is a new release for my amazing tool
# Created At: 2021-02-10 14:57:07.358 0800 CST
# Released At: 2021-02-10 14:57:07.358 0800 CST
# Asset::Link::Name: info-student-rebuid
# Asset::Link::URL: http://192.168.12.107:30001/job/HelloWorld/19/artifact/target/info-student-rebuild-0.0.2-SNAPSHOT.jar
# See all available releases here: /-/releases
# INFO[0002] release created successfully!
# (4) 指定RELEASE的多个外部URL
~ $ /usr/local/bin/release-cli --server-url http://gitlab.weiyigeek.top --private-token "fYyHzos-zEPPP8PDsxLa" --project-id 45 create --tag-name "v1.12" --name "v1.12" --description "This is a new release for my amazing tool" --assets-link '{"name":
"info-student-rebuid","url":"http://192.168.12.107:30001/job/HelloWorld/19/artifact/target/info-student-rebuild-0.0.2-SNAPSHOT.jar","link_type":"other"}' --assets-link '{"name":"Asset2","url":"http://192.168.12.107:30001/job/HelloWor
ld/19/artifact/target/info-student-rebuild-0.0.1-SNAPSHOT.jar"}'
# INFO[0000] Creating Release
# Tag: v1.12
# Name: v1.12
# Description: This is a new release for my amazing tool
# Created At: 2021-02-10 15:06:23.059 0800 CST
# Released At: 2021-02-10 15:06:23.059 0800 CST
# Asset::Link::Name: Asset2
# Asset::Link::URL: http://192.168.12.107:30001/job/HelloWorld/19/artifact/target/info-student-rebuild-0.0.1-SNAPSHOT.jar
# Asset::Link::Name: info-student-rebuid
# Asset::Link::URL: http://192.168.12.107:30001/job/HelloWorld/19/artifact/target/info-student-rebuild-0.0.2-SNAPSHOT.jar
# See all available releases here: /-/releases
# INFO[0002] release created successfully!
# cli=release-cli command=create name=v1.12 project-id=45 ref= server-url="http://gitlab.weiyigeek.top" tag-name=v1.12 version=0.6.0
补充说明:
1) 发布的版本列表按released_at排序。
代码语言:javascript复制$ curl -s --header "PRIVATE-TOKEN: fYyHzos-zEPPP8PDsxLa" "http://gitlab.weiyigeek.top/api/v4/projects/45/releases" | jq .[].tag_name
"v1.12"
"v1.14"
"v1.15"
# 验证发布版本的 evidence (释放)
# Create Evidence for an existing Release.
$ curl -s --request POST --header "PRIVATE-TOKEN: fYyHzos-zEPPP8PDsxLa" "http://gitlab.weiyigeek.top/api/v4/projects/45/releases/v1.12/evidence"
2) 查看发布的指定版本的release的信息
代码语言:javascript复制$ curl -s --header "PRIVATE-TOKEN: fYyHzos-zEPPP8PDsxLa" "http://gitlab.weiyigeek.top/api/v4/projects/45/releases/v1.12" | jq '.assets.links[] | select (.id == 5 or .id == 4) | .url'
"http://192.168.12.107:30001/job/HelloWorld/19/artifact/target/info-student-rebuild-0.0.1-SNAPSHOT.jar"
"http://192.168.12.107:30001/job/HelloWorld/19/artifact/target/info-student-rebuild-0.0.2-SNAPSHOT.jar"
3) 利用http请求创建一个版本。 描述: 创建发布需要开发人员对项目的访问。
代码语言:javascript复制$ curl --header 'Content-Type: application/json' --header "PRIVATE-TOKEN: fYyHzos-zEPPP8PDsxLa"
--data '{ "name": "New release", "tag_name": "v1.13", "description": "Super nice release", "milestones": ["v1.13", "v1.13-rc"], "assets": { "links": [{ "name": "hoge", "url": "https://google.com", "filepath": "/binaries/linux-amd64", "link_type":"other" }] } }'
--request POST "http://gitlab.weiyigeek.top/api/v4/projects/45/releases"
4) 更新已发布的 release 版本 (基础信息)
代码语言:javascript复制# 修改前
$ curl -s --header "PRIVATE-TOKEN: fYyHzos-zEPPP8PDsxLa" "http://gitlab.weiyigeek.top/api/v4/projects/45/releases/v1.12" | jq ."description"
"This is a new release for my amazing tool"
$ curl -s --header 'Content-Type: application/json' --request PUT --data '{"name": "Release - v1.12", "description": "更新测试", "milestones": ["v1.12"]}' --header "PRIVATE-TOKEN: fYyHzos-zEPPP8PDsxLa" "http://gitlab.weiyigeek.top/api/v4/projects/45/releases/v1.12" | jq .
{
"name": "Release - v1.12",
"tag_name": "v1.12",
"description": "更新测试",
....
}
# 修改后
~$ curl -s --header "PRIVATE-TOKEN: fYyHzos-zEPPP8PDsxLa" "http://gitlab.weiyigeek.top/api/v4/projects/45/releases/v1.12" | jq ."description"
"更新测试"
5) 删除一个发布的版本。
描述: 删除一个版本并不会删除相关的标签, 注意删除一个版本需要对项目的维护者级别的访问
。
~$ curl --request DELETE --header "PRIVATE-TOKEN: fYyHzos-zEPPP8PDsxLa" "http://gitlab.weiyigeek.top/api/v4/projects/45/releases/v1.12"
{"name":"Release - v1.12","tag_name":"v1.12","description":"更新测试","description_html":"u003cp data-sourcepos="1:1-1:12" dir="auto"u003e更新测试u003c/pu003e","created_at":"2021-02-10T15:06:23.059 08:00","released_at":"2021-02-10T15:06:23.059 08:00","author":{"id":11,"name":"weiyigeek","username":"weiyigeek","state":"active","avatar_url":"http://gitlab.weiyigeek.top/uploads/-/system/user/avatar/11/avatar.png","web_url":"http://gitlab.weiyigeek.top/weiyigeek"},"commit":{"id":"334c630d2944461ac271a2d92f49496dcf30f443","short_id":"334c630d","created_at":"2021-02-02T13:34:45.000 08:00","parent_ids":["f1d2cd8126c8f181b66bc562f94ffff0a4460067"],"title":"RELEASE v1.12 jenkins","message":"RELEASE v1.12 jenkinsn","author_name":"WeiyiGeek","author_email":"1564362804@qq.com","authored_date":"2021-02-02T13:34:45.000 08:00","committer_name":"WeiyiGeek","committer_email":"1564362804@qq.com","committed_date":"2021-02-02T13:34:45.000 08:00","web_url":"http://gitlab.weiyigeek.top/weiyigeek/helloworld/-/commit/334c630d2944461ac271a2d92f49496dcf30f443"},"upcoming_release":false,"commit_path":"/weiyigeek/helloworld/-/commit/334c630d2944461ac271a2d92f49496dcf30f443","tag_path":"/weiyigeek/helloworld/-/tags/v1.12","assets":{"count":4,"sources":[{"format":"zip","url":"http://gitlab.weiyigeek.top/weiyigeek/helloworld/-/archive/v1.12/helloworld-v1.12.zip"},{"format":"tar.gz","url":"http://gitlab.weiyigeek.top/weiyigeek/helloworld/-/archive/v1.12/helloworld-v1.12.tar.gz"},{"format":"tar.bz2","url":"http://gitlab.weiyigeek.top/weiyigeek/helloworld/-/archive/v1.12/helloworld-v1.12.tar.bz2"},{"format":"tar","url":"http://gitlab.weiyigeek.top/weiyigeek/helloworld/-/archive/v1.12/helloworld-v1.12.tar"}],"links":[]},"evidences":[],"_links":{"self":"http://gitlab.weiyigeek.top/weiyigeek/helloworld/-/releases/v1.12","edit_url":"http://gitlab.weiyigeek.top/weiyigeek/helloworld/-/releases/v1.12/edit"}}
~$ curl --request DELETE --header "PRIVATE-TOKEN: fYyHzos-zEPPP8PDsxLa" "http://gitlab.weiyigeek.top/api/v4/projects/45/releases/v1.12"
{"message":"403 Forbidden"}
0x03 基础配置
Gitlab初始化设置
0) 用户主题与语言设置
流程:右上角个人头像->settings->Preferences->主题设置|语法高亮主题|自定义语言和区域相关设置
1) 关闭Gravatar头像功能进行网络加速
Setting -> Gravatar enabled(展开Expand) -> not Checked Gravatar enabled
-> Sava Changes
WeiyiGeek.
2) 用户管理与注册相关设置 描述:是否用户自己进行注册以及设置密码策略和注册邮箱域名(白|黑名单)等信息,如果关闭后首页将不显示注册;
WeiyiGeek.
- 创建用户:
Admin Area
-> User -> New User -> 项目限制(创建得数量默认即可) ->Can create group
(企业内部建议取消) -> 创建用户 (用户邮箱将会收到一份注册邮件) - 模仿用户 impersonate: 可以模仿用户登录,不用退出再登录;
WeiyiGeek.
Q: Gitlab 的用户 、组、项目之间的关系;
1.项目:可分配到组 或者 指定用户 2.组: 可与包含指定用户,而拥有该组的用户拥有相同的权限;(推荐创建利用组来配对应的项目设置人员的相应权限) 3.用户: 我们的注册人员包括开发者、运维、主管等
组与项目绑定流程:
代码语言:javascript复制1.先创建组;
2.创建项目并让该项目隶属于该组;
3.创建用户并添加组并分配权限
Q: 如果有新成员需要加入该项目怎么办?
1.先给新成员创建用户组; 2.再将成员加入组,此时该用户就能看到对应的项目; 3.添加公钥,然后使用
git clone
测试是否能拉取代码; 4.最后更新内容,测试能否进行任务提交;
3) 代码推送提醒功能 描述:添加一个项目有代码提交后发邮件给指定的邮箱提醒通知,这个对于代码审核还是有帮助的,至少知道谁什么时候推送新代码了,更新了哪些内容等;
管理员登陆到gitlab界面 Admin area-> Service Templates -> Emails on push
, 勾选Active,Recipients填写本项目更新需要推送知晓的人员,多个用空格或者逗号隔开;
WeiyiGeek.
4) 设置ssh shell的端口 用于处理Git命令和修改authorized keys列表
代码语言:javascript复制gitlab_rails['gitlab_shell_ssh_port'] = 9022
5) 修改gitlab的仓库(repositories)存放目录
描述:安装好后要将仓库(repositories)放在一个大硬盘上,所以需要修改仓库对应的目录。
(注:我是先将所以的repositories删除再作以下操作的,还把 /var/opt/gitlab/git-data/repositories/用户名
目录下所有用户名下的文件全删除,目录删的只剩下空文件夹)
#查看磁盘空间大小(可以根据您自己添加的磁盘进行操作)
$df -h
/dev/mapper/cl-home 24G 954M 24G 4% /home
#新建新仓库目录
$mkdir -p /home/gitlab/git-data
#配置gitlab的数据存储位置为/home目录下
$vi /etc/gitlab/gitlab.rb
git_data_dirs({ "default" => { "path" => "/home/gitlab/git-data" } })
#重新加载配置
$sudo gitlab-ctl reconfigure
#重载后发现我们更改的仓库地址下面有数据存放了
$ls /home/gitlab/git-data/
repositories
6) 创建一个开发组
在Gitlab登录后的主页面上点击Create a group
->输入相关组信息创建即可;
邀请成员加入到组:Members
->Group members
->Invite memeber
-> 选择用户加入到刚创建的组以及角色权限;
WeiyiGeek.
7) 项目创建与初始化
Gitlab的git地址组成与github是一致(别告诉我您不知道): gitlab地址 用户/群组 自定义名字
Gitlab项目的可见类型有三种级别。
- Private project: 该级别是只有项目拥有者或者已经得到授权的人可以访问该项目,或者这些人是该项目组的成员。
- Internal project: 只要有用户名和密码,
可以登陆该项目所在的Gitlab服务器的
,均可访问该项目。 - public projects: 只要知道该项目的具体位置就是路径,都可以访问该项目,
它们默认的使用的是guest权限
。
比如我们在上面的开发组项目中进行创建项目操作流程如下: 1.进入开发组->New Project->设置项目名称->创建项目 2.建议添加SSH密钥到Gitlab中免密码推送与拉取`git@gitlab.weiyigeek.top:newproject/secopsdev.git`:Setting -> SSH;
代码语言:javascript复制$ ls ~/.ssh/
id_ed25519 id_ed25519.pub id_rsa id_rsa.pub known_hosts
WeiyiGeek.
3.项目的初始化:
代码语言:javascript复制#基础设置(前提你需要下载git)
git config --global user.name "WeiyiGeek"
git config --global user.email "weiyigeek@qq.com"
#创建一个新的存储库
git clone git@gitlab.weiyigeek.top:newproject/secopsdev.git
cd secopsdev
touch README.md
git add README.md
git commit -m "add README"
#推送现有文件夹
cd existing_folder
git init
git remote add origin http://gitlab.weiyigeek.top/newproject/secopsdev.git
git add .
git commit -m "Initial commit"
#推动现有的Git存储库
cd existing_repo
git remote rename origin old-origin #将本地分支改名
git remote add origin http://gitlab.weiyigeek.top/newproject/secopsdev.git
4.需要注意默认是无法进行提交的这是由于Master主分支被保护所导致的,如果想直接对主分支进行更改而又不想创建子分支可以采用以下方式,前提是必须采用Maintainers的用户上传Master分支后进行设置
;
Project -> Settings -> Repository -> Brach (分支权限设置
) 或者 删除保护;
WeiyiGeek.
8) 用户项目权限控制 权限管理理解:
- (1).新建用户的时候选择一个普通权限和管理员权限之分即可
- (2).建git库的时候可以关联一个组或者一个成员
- (3).添加到组里面的人员可以设定Guest Reporter Developer Master Owner
- (4).不同git库里面的关联同一个成员可以设置不同权限
- (5).不同git库里面的关联同一个组无法设置不同权限
- (6).一个git库只能关联一个组成员
- (7).一个组成员可以被多个git库关联,且权限一样
- (8).一个成员可以被多个git库或组成员关联,且权限可以不一样
- (9).Reporter以上才有下代码权限
- (10).企业权限控制:
- 开发人员:developer权限
- 代码审核MDE:master权限
项目组&x项目权限管理:
- Project Name -> Memebers-> 用户成员与
Choose a role permission
-> 以及失效日期设置 参考官方文档网站列出了Guest,Reporter,Developer,Maintainer(Master),Owner
对应的权限,比如上面我们设置的Dev默认对保护分支是没有push权限的;
9) 广播信息设置
描述: 广播信息会显示给全部用户,可以用于通知用户系统定期维护、近期计划升级等信息,在用户登陆界面以及用户提交代码的时候会有显示
。
Area Admin -> Messages -> Broadcast Information (设置即可)
WeiyiGeek.
Gitlab成员权限说明
基础概念: 用户具有不同的能力,具体取决于他们在特定组或项目中的访问级别。如果用户同时在组的项目和项目本身中,则使用最高权限级别。在公共和内部项目中,不会强制实施Guest角色, 所有用户都可以创建问题,发表评论,克隆或下载项目代码。当成员离开团队时,将自动取消分配所有分配的问题和合并请求。
Tips :Gitlab用户在组中有五种权限:Guest、Reporter、Developer、Master (在 11.0 版本中已重命名为 Maintainer) - 维护者
、Owner
成员权限说明:
- Guest:可以创建issue、发表评论,不能读写版本库
- Reporter:可以克隆代码,不能提交,QA、PM可以赋予这个权限
- Developer:可以克隆代码、开发、提交、push,RD可以赋予这个权限
- Maintainer:可以创建项目、添加tag、保护分支、添加项目成员、编辑项目,核心RD负责人可以赋予这个权限
- Owner:可以设置项目访问权限 - Visibility Level、删除项目、迁移项目、管理组成员,开发组leader可以赋予这个权限
- Gitlab中的组和项目有三种访问权限:Private、Internal、Public
项目权限说明:
Private:只有组成员才能看到
Internal:只要登录的用户就能看到,开源项目和组设置的是Internal
Public:所有人都能看到
参考地址: https://docs.gitlab.com/ee/user/permissions.html
0x04 基础使用
Gitlab添加发信邮箱
邮件找回密码: 1)因为gitlab一直都在运行着,所以要修改前记录备份一下配置文件;
代码语言:javascript复制cp /etc/gitlab/gitlab.rb{,.bak}
2)修改配置文件:/etc/gitlab/gitlab.rb 这里使用公司的企业邮箱来发邮件;
代码语言:javascript复制$egrep -v "^$|^#" /etc/gitlab/gitlab.rb
external_url 'http://gitlab.weiyigeek.top'
gitlab_rails['time_zone'] = 'Asia/Shanghai'
gitlab_rails['gitlab_email_enabled'] = true
gitlab_rails['gitlab_email_from'] = 'public@weiyigeek.top'
gitlab_rails['gitlab_email_display_name'] = 'Gitlab CE'
gitlab_rails['gitlab_email_reply_to'] = 'gitlab@weiyigeek.top'
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.exmail.qq.com"
gitlab_rails['smtp_port'] = 465
gitlab_rails['smtp_user_name'] = "public@weiyigeek.top"
gitlab_rails['smtp_password'] = "邮箱密码"
gitlab_rails['smtp_domain'] = "weiyigeek.top"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_tls'] = true
user['git_user_email'] = "public@weiyigeek.top"
3)保存修改,更新配置;
代码语言:javascript复制#只是把修改过的配置文件信息,刷新到配置文件中
gitlab-ctl reconfigure
其实上面修改一个配置文件gitlab.rb 它里面的配置会被此命令调用编写成多个配置文件到安装目录下:/var/opt/gitlab/gitlab-rails/etc/
$ls /var/opt/gitlab/gitlab-rails/etc/
database.yml gitlab_shell_secret gitlab.yml resque.yml smtp_settings.rb #也可以直接修改
gitlab_pages_secret gitlab_workhorse_secret rack_attack.rb secrets.yml unicorn.rb
如果直接修改smtp_settings.rb
文件就只要重启一下gitlab-ctl restart
即可。
4)控制台查看配置与测试发信
代码语言:javascript复制$gitlab-rails console
--------------------------------------------------------------------------------
GitLab: 12.9.2 (ac5568eb5d8) FOSS
GitLab Shell: 12.0.0
PostgreSQL: 10.12
--------------------------------------------------------------------------------
Loading production environment (Rails 6.0.2)
irb(main):001:0> ActionMailer::Base.delivery_method
=> :smtp
irb(main):002:0> ActionMailer::Base.smtp_settings
=> {:authentication=>:login, :address=>"smtp.exmail.qq.com", :port=>465, :user_name=>"public@weiyigeek.top", :password=>"Pa****20", :domain=>"weiyigeek.top", :enable_starttls_auto=>true, :tls=>true, :ca_file=>"/opt/gitlab/embedded/ssl/certs/cacert.pem"}
irb(main):003:0> Notify.test_email('291238737@qq.com', 'GitLab测试邮件', 'GitLab 页面信息').deliver_now
Notify#test_email: processed outbound mail in 2.0ms
Delivered mail 5e9708455a809_6b023fce629dd99085b8@Developement.mail (1543.7ms)
Date: Wed, 15 Apr 2020 21:12:37 0800
From: Gitlab CE <public@weiyigeek.top>
Reply-To: Gitlab CE <noreply@gitlab.weiyigeek.top>
完成上面操作后,回到登陆页面,选择Forgot your password? 输入你帐号的邮箱地址即可收到邮件,根据邮件地址修改新密码即可;
WeiyiGeek.sendemail
补充说明:
在某些专有网络主机上配置了一个gitlab仓库,因为默认这些专有网络的主机都是禁用了25端口的,所以这个gitlab.rb的配置就有做相应的改变,使用465协议。
代码语言:javascript复制#只修改以下两行即可,修改完要更新配置
gitlab_rails['smtp_port'] = 25
gitlab_rails['smtp_tls'] = true
Gitlab添加LDAP认证
LDAP配置说明:http://gitlab.weiyigeek.top/help/administration/auth/ldap
Step1.修改gitlab配置文件设置连接LDAP服务器参数即DN于bindPass
代码语言:javascript复制vim /etc/gitlab/gitlab.rb
gitlab_rails['ldap_enabled'] = true
# gitlab_rails['prevent_ldap_sign_in'] = false
#! **remember to close this block with 'EOS' below**
gitlab_rails['ldap_servers'] = YAML.load <<-'EOS'
main: # 'main' is the GitLab 'provider ID' of this LDAP server
label: 'LDAP'
host: '10.10.107.245'
port: 389
timeout: 10
uid: 'uid'
bind_dn: 'cn=admin,dc=WeiyiGeek,dc=com,dc=cn'
password: 'WeiyiGeek'
encryption: 'plain' # "start_tls" or "simple_tls" or "plain"
verify_certificates: false
smartcard_auth: false
active_directory: true
allow_username_or_email_login: true
lowercase_usernames: false
block_auto_created_users: false
base: 'dc=WeiyiGeek,dc=com,dc=cn'
#user_filter: '(&(objectclass=inetOrgPerson)(memberof=cn=gitlab,ou=Group,dc=WeiyiGeek,dc=com,dc=cn))'
attribute:
username: ['uid']
email: ['mail']
name: 'cn'
first_name: ['sn']
EOS
Step2.LDAP建立groupOfUniqueNames属性
得用户组以及绑定用户;
dn: cn=gitlab,ou=Group,dc=WeiyiGeek,dc=com,dc=cn
uniqueMember: uid=gituser,ou=People,dc=WeiyiGeek,dc=com,dc=cn
dn: uid=gituser,ou=People,dc=WeiyiGeek,dc=com,dc=cn
objectClass: posixAccount
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
loginShell: /bin/bash
homeDirectory: /home/gituser
userPassword:: e1NTSEF9UGVyM21xc1dJcnV3K1d2bWRiVmVpd3RWZHVVeVN6Tks=
uid: gituser
cn: gituser
uidNumber: 10000
gidNumber: 10000
sn: gituser
mail: gituser@weiyigeek.top
Step3.登陆测试如果LDAP用户不存在或者filter的条目不存在会报错Could not authenticate you from Ldapmain because "Invalid credentials for jenkuser".
WeiyiGeek.
Step4.登陆成功界面如下:
WeiyiGeek.
Gitlab备份升级迁移恢复
描述:GitLab作为公司项目代码的版本管理系统,数据非常重要所以必须做好备份。
使用Gitlab 备份恢复与迁移也非常简单. 使用一条命令即可创建完整的Gitlab备份:gitlab-rake gitlab:backup:create
使用以上命令默认会在/var/opt/gitlab/backups
目录下创建一个名称类似为1481598919_gitlab_backup.tar
的压缩包, 这个压缩包就是Gitlab整个的完整部分, 其中开头的1481598919是备份创建的日期
Gitlab提供了两种备份:
- 本地备份: 采用gitlab-rake gitlab:backup:create
- 远程备份: 实际是远程机器连接到gitlab服务器上进行脚本的执行;
关键性配置文件备份:
代码语言:javascript复制/etc/gitlab/gitlab.rb 配置文件须备份
/var/opt/gitlab/nginx/conf nginx配置文件
/etc/postfix/main.cfpostfix 邮件配置备份
本地备份
代码语言:javascript复制#1.GitLab备份的默认目录是
/var/opt/gitlab/backups
#2.修改改备份目录
$vim /etc/gitlab/gitlab.rb
gitlab_rails['manage_backup_path'] = false # 如不需远程来手动管理备份则设置为false
gitlab_rails['backup_path'] = '/home/backups'
gitlab_rails['backup_keep_time'] = 604800 # 备份保留7天 2419200
#3.修改配置后重载配置
$gitlab-ctl reconfigure
#4.执行备份命令
$gitlab-rake gitlab:backup:create
# 2020-04-16 09:44:52 0800 -- Dumping database ...
# Dumping PostgreSQL database gitlabhq_production ... [DONE]
# 2020-04-16 09:44:53 0800 -- done
# 2020-04-16 09:44:53 0800 -- Dumping repositories ...
# 2020-04-16 09:44:53 0800 -- done
# 2020-04-16 09:44:53 0800 -- Dumping uploads ...
# 2020-04-16 09:44:53 0800 -- done
# 2020-04-16 09:44:53 0800 -- Dumping builds ...
# 2020-04-16 09:44:53 0800 -- done
# 2020-04-16 09:44:53 0800 -- Dumping artifacts ...
# 2020-04-16 09:44:53 0800 -- done
# 2020-04-16 09:44:53 0800 -- Dumping pages ...
# 2020-04-16 09:44:53 0800 -- done
# 2020-04-16 09:44:53 0800 -- Dumping lfs objects ...
# 2020-04-16 09:44:53 0800 -- done
# 2020-04-16 09:44:53 0800 -- Dumping container registry images ...
# 2020-04-16 09:44:53 0800 -- [DISABLED]
# Creating backup archive: 1587001493_2020_04_16_12.9.2_gitlab_backup.tar ... done
# Uploading backup archive to remote storage ... skipped
#5.查看备份文件,压缩包包括GitLab整个的完整部分,大概格式xxxxxxxx_gitlab_backup.tar
$ls /var/opt/gitlab/backups
1587001493_2020_04_16_12.9.2_gitlab_backup.tar
补充说明:
(1) 通过任务计划crontab 实现自动备份
代码语言:javascript复制# 每天凌晨两点进行一次自动备份;
sudo crontab -e
0 2 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create
#重启服务
systemctl restart crond
(2) 值得注意的是由于gitlab.rb and gitlab-secrets.json
包含敏感数据,采用上面备份的形式不能将它们进行备份,只能通过手动备份
的方式;
Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data
and are not included in this backup. You will need these files to restore a backup.
Please back them up manually.
远程备份 描述:实际上就是在gitlab服务器上添加备份服务器的pub公钥,然后备份服务器访问Gitlab服务器进行拉取,当然您也可以采用rsync进行备份;
代码语言:javascript复制#假设环境如下:
#备份服务器: 192.168.1.2
#GitLab服务器: 192.168.1.250
#1.备份服务器创建authorized_keys文件
mkdir -p /root/.ssh
touch /root/.ssh/authorized_keys
chmod 400 /root/.ssh/authorized_keys
#2.Gitlab服务器生成rsa公钥并将证书上传到备份服务器(实际环境中建议最小权限用户)
$ssh-keygen -t rsa -C "gitlab@weiyigeek.top"
# Enter file in which to save the key (/root/.ssh/id_rsa): 报错路径
# Enter passphrase (empty for no passphrase): 公钥密码
# Enter same passphrase again: 验证密码
# Your public key has been saved in /root/.ssh/id_rsa.pub. #保存路径
# The key fingerprint is:
# SHA256:YahioDs7XZ9No********RS9jWX1NNXo04i4ZHOLY WeiyiGeek@weiyigeek.top
# The key's randomart image is:
# ---[RSA 2048]----
# | . . |
# | . . o o. |
# |. o.oo= . .|
# |.. ..=.E. |
# |o o.. oS= . . |
# |.= . o . . |
# | *.. = |
# |oo= o o . |
# |.o. |
# ----[SHA256]-----
$ls /root/.ssh/
id_rsa id_rsa.pub
$scp /root/.ssh/id_rsa.pub root@192.168.1.2:/root/.ssh/
#3.备份服务器将id_rsa.pub文件内容追加到authorized_keys 文件中,并且修改authorized_keys文件的权限
$cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys
#4.gitlab备份脚本(主要执行权限呼吁)将备份文件拷贝到服务器端
$cat > /home/gitlab/auto_backup_to_remote.sh<<'END'
DATE=`date "%Y_%m_%d"`
BackupIP=192.168.1.2
BackupUser=root
BackupPATH=/var/opt/gitlab/backups
#删除本地7天前得备份
find ${BackupPATH} -type f -mtime 7 -name '*.tar' -exec rm {} ;
#如果不需要备份文件您也可以备份仓房目录
find ${BackupPATH} -name *${DATE}* -exec scp -r {} ${root}@${BackupIP}:/tmp/ ;
END
$chmod x /home/gitlab/auto_backup_to_remote.sh
#5.添加定时计划(每天凌晨1点执行),重启crond服务
cat >> /etc/crontab <<ENDtop
0 1 * * * /home/gitlab/auto_backup_to_remote.sh
END
systemctl restart crond
PS : 在Docker 容器中利用exec命令执行备份命令
代码语言:javascript复制docker exec -it gitlab-server sh -c "/opt/gitlab/bin/gitlab-rake gitlab:backup:create"
Gitlab恢复
比如:上面我们得备份:/var/opt/gitlab/backups/1587001493_2020_04_16_12.9.2_gitlab_backup.tar
;
操作流程如下:
#停止相关数据连接服务(保证数据库没有新的连接不会有写数据情况)
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq
# 指定恢复文件会自动去备份目录找格式`1587001493_2020_04_16_12.9.2`
# 注意不加_gitlab_backup.tar
cd /var/opt/gitlab/backups/ && chmod -R 777 *
gitlab-rake gitlab:backup:restore BACKUP=1587001493_2020_04_16_12.9.2
#启动Gitlab
gitlab-ctl restart
Gitlab迁移
描述: 迁移如同备份与恢复的步骤一样, 只需要将老服务器/var/opt/gitlab/backups目录下的备份文件拷贝到新服务器上的/var/opt/gitlab/backups
即可(如果你没修改过默认备份目录的话).
PS : 但是需要注意的是新服务器上的Gitlab的版本必须与创建备份时的Gitlab版本号相同. 比如新服务器安装的是最新的10.8.3版本的Gitlab那么迁移之前, 最好将老服务器的Gitlab升级为10.8.3,基于最新版本的状态在进行备份。
关键文件:
代码语言:javascript复制/etc/gitlab/gitlab.rb #gitlab 配置文件须迁移,迁移后需要调整数据存放目录
/var/opt/gitlab/nginx/conf #nginx 配置文件目录须迁移
将旧机器上的备份文件同步过来后的操作步骤:
代码语言:javascript复制gitlab-ctl stop unicorn
# ok: down: unicorn: 0s, normally up
gitlab-ctl stop sidekiq
# ok: down: sidekiq: 0s, normally up
chmod 777 /var/opt/gitlab/backups/1528102291_2018_06_04_10.8.3_gitlab_backup.tar
gitlab-rake gitlab:backup:restore BACKUP=1528102291_2018_06_04_10.8.3
Gitlab升级 1.关闭gitlab服务
代码语言:javascript复制gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq
gitlab-ctl stop nginx
2.备份gitlab
代码语言:javascript复制gitlab-rake gitlab:backup:create
3.下载gitlab的RPM包并进行升级
代码语言:javascript复制//直接安装高版本
yum install gitlab-ce-8.12.13-ce.0.el7.x86_64
//报错.
Error executing action `run` on resource 'ruby_block[directory resource: /var/opt/gitlab/git-data/repositories]'
//解决方法:
chmod 2770 /var/opt/gitlab/git-data/repositories
4.启动并查看gitlab版本信息
代码语言:javascript复制gitlab-ctl reconfigure
gitlab-ctl restart
head -1 /opt/gitlab/version-manifest.txt
gitlab-ce 10.8.3
Gitlab更改默认Nginx
描述: 更换gitlab自带Nginx, 使用自行安装的Nginx来管理gitlab服务。
1.编辑gitlab配置文件禁用自带Nignx服务器
代码语言:javascript复制vi /etc/gitlab/gitlab.rb
...
#设置nginx为false,关闭自带Nginx
nginx['enable'] = false
...
2.检查默认nginx配置文件,并迁移至新Nginx服务
代码语言:javascript复制/var/opt/gitlab/nginx/conf/nginx.conf #nginx配置文件,包含gitlab-http.conf文件
/var/opt/gitlab/nginx/conf/gitlab-http.conf #gitlab核心nginx配置文件
3.重启nginx、gitlab服务
代码语言:javascript复制[root@gitlab-ce ~]# gitlab-ctl reconfigure
[root@gitlab-ce ~]# systemctl start nginx
异常解决: 访问报 502 原因是nginx用户无法访问 gitlab 用户的socket文件,此时需要重启gitlab并需要重新授权
代码语言:javascript复制chmod -R o x /var/opt/gitlab/gitlab-rails
0x05 入坑解决
(0) 用户pull与push代码到gitlab常见错误
错误1:The requested URL returned error: 403
- 原因:表示我们没有权限来pull/push相关代码需要修改相关gitlab账号的权限为
Development或者Maintianer
;
错误2:
代码语言:javascript复制To gitlab.weiyigeek.top:newproject/secopsdev.git
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'git@gitlab.weiyigeek.top:newproject/secopsdev.git'
原因:由于上传代码的gitlab用户权限为开发者而默认创建的是私有的项目,默认是对master分支开启了保护机制,需要对其项目的Branch分支设置 Allowed to merge | Allowed to push
为Development Maintiner;
#最终主分支以及dev分支都测试上次成功
remote:
To gitlab.weiyigeek.top:newproject/secopsdev.git
* [new branch] dev -> dev
Branch 'dev' set up to track remote branch 'dev' from 'origin'.
(1) gitlab密码忘记找回重置方法
方式1:通过gitlab-rails进行重置
代码语言:javascript复制$gitlab-rails console
Loading production environment (Rails 6.0.2)
> u=User.where(id:1).first #这个是管理员的,也可以用email等
> u.password = 'your_password' #密码有格式限制,我只知道8位以上否则会保存失败
> u.password_confirmation = 'your_password'
> u.save
方式2:通过初始化设置邮件进行找回管理员或者在本地 postfix 中查看未发送得邮件,此处以root用户的admin@example.com为例,一般默认都没改(只是一种方式实际中没多大意义);
此处假设服务不能链接到外网之中(为了复现下列操作);
代码语言:javascript复制#(1)查看无法发送出去的邮件队列
$postqueue -p
-Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient-------
#因为不存在此地址就无法发送出去,只能停留在邮件的列队中。
908DCCAC62 6871 Wed Apr 15 19:54:32 gitlab@gitlab.weiyigeek.top #(实际就是我们找回密码邮件)
(Host or domain name not found. Name service error for name=example.com type=MX: Host not found, try again)
admin@example.com
#(2)邮件队列存储位置
$postconf -d | grep queue_directory #默认情况下是没有修改的
queue_directory = /var/spool/postfix
#(3)进到目录下,查找上面那封邮件的队列ID号
$find /var/spool/postfix/ -name 908DCCAC62
/var/spool/postfix/defer/9/908DCCAC62
/var/spool/postfix/deferred/9/908DCCAC62
# (4) 利用postfix自带的工具查看邮件的内容,找到修改密码的地
$postcat /var/spool/postfix/deferred/9/908DCCAC62
*** ENVELOPE RECORDS /var/spool/postfix/deferred/9/908DCCAC62 ***
message_size: 6871 274 1 0 6871 0
message_arrival_time: Wed Apr 15 19:54:32 2020
create_time: Wed Apr 15 19:54:32 2020
named_attribute: rewrite_context=local
sender: gitlab@gitlab.weiyigeek.top
named_attribute: dsn_orig_rcpt=rfc822;admin@example.com
original_recipient: admin@example.com
.....
If you did not perform this request, you can safely ignore this email.
Otherwise, click the link below to complete the process:
#找到如下密码修改连接,并将其复制出来到浏览器中重置密码
http://gitlab.weiyigeek.top/users/password/edit?reset_password_token=gd9MyL7FaSt5R2F3_qA_
WeiyiGeek.
(2)GitLab数据库引起的502错误问题及解决方案
描述:打开GitLab的主页地址http:/gitlab.weiyigeek.top/报错502,重启或检查状态发现都正常sudo gitlab-ctl status
,并且检查了端口号并没被占用,检查刷新配置gitlab-ctl reconfigure
发现了错误;
错误信息如下:
代码语言:javascript复制#gitlab-ctl reconfigure
Running handlers:
There was an error running gitlab-ctl reconfigure:
bash[migrate gitlab-rails database] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash" "/tmp/chef-script20190308-65247-12ck9rp" ----
STDOUT: rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:52:in `block (3 levels) in <top (required)>
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:db:configure
(See full trace by running task with --trace)
STDERR:
---- End output of "bash" "/tmp/chef-script20190308-65247-12ck9rp" ----
Ran "bash" "/tmp/chef-script20190308-65247-12ck9rp" returned 1
#gitlab-ctl tail
WeiyiGeek.
(3) gitlab-ce版本升级记录 我们为了保证数据安全,一步步来慢慢升级,使用官方的源可能比较慢,可以使用国内的源(采用Omnibus方式进行升级直接rpm包部署):https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum
升级方法:先升级到当前大版本最后一版(这里是我的建议,一般差不多最后几版就可以了),接下来就是跨大版本的升级,先从大版本的低版升级,再到最后版本升级,以此类推; 因为我中间尝试过直接跨太多个版本升级,出现过异常,其实主要是数据库的数据导入备份问题,如果新版本跟旧版相关太大的话就会报错了。
比如我的升级过程很长,同时也做了简单的测试
代码语言:javascript复制#升级版本
8.3.1-->8.3.7-->8.8.5-->8.17.4-->9.5.6-->10.0.6-->10.8.5-->11.0.3
#停止服务
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq
gitlab-ctl stop nginx
#创建数据备份,防止更新失败(非常重要)
gitlab-rake gitlab:backup:create
#安装升级包
#yum update gitlab-ce # 升级GitLab-ce 版本(不建议采用此种方式)
rpm -Uvh gitlab-7.4.2_omnibus-1.el6.x86_64.rpm
#重新加载配置&重启Gitlab服务
gitlab-ctl reconfigure
gitlab-ctl restart
比如:项目备份的数据的是否有异常,用户信息,响应速度等等,个人认为还是比较靠普的。
其实只要我们之前有升级过就不会现在这样了,一下子从这么低的版本升级上来。一般建议保持在相同的大版本号就好了。太新可能也会有Bug,太旧了也会有很多历史遗留的问题。
升级完成后你会发现多了好几个进程。可能会对系统硬件要求更高了,比如内存会要求更高了。
建议:如果非必要的功能需求,只是简单的要一个代码仓库的话,个人建立使用在11.x.-12.3.x
之间也不错了。
(4) gitlab-rails database初始化失败 环境说明
代码语言:javascript复制Centos7: 3.10.0-327.el7.x86_64
gitlab版本:原来是8.8.5版本现升级到较新的版本;
问题:在同一台Centos7机器上卸载了旧版本的gitlab后,接着又重新安装新版本的gitlab-ce 在安装完后修改配置,初始化配置时出现以下错误:
代码语言:javascript复制Running handlers:
There was an error running gitlab-ctl reconfigure:
bash[migrate gitlab-rails database] (gitlab::database_migrations line 49) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash" "/tmp/chef-script20190628-7065-vx17en" ----
STDOUT: rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:49:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:db:configure
(See full trace by running task with --trace)
STDERR:
---- End output of "bash" "/tmp/chef-script20190628-7065-vx17en" ----
Ran "bash" "/tmp/chef-script20190628-7065-vx17en" returned 1
原因:因为在卸载旧版本的时候没有把旧的数据删除,所以直接安装时,程序检测到配置文件,数据文件时就会跳过没有覆盖,新版本使用旧版本的数据文件[因为版本相差太多了]就会无法识别导致。
解决方法:卸载完旧版本后,记得删除旧的安装数据配置文件,安装目录,数据目录等
,重新安装新版的gitlab-ce即可正常启动。
rm -rf /var/opt/gitlab/ /opt/gitlab/ /etc/gitlab/
(5) Gitlab的数据库postgresql更新帐号信息 问题描述:由于内部的Ldap认证服务器硬盘坏掉了,导致在上面跑的Ldap服务无法对Jenkins和Gitlab平台做集中认证了,导致在Gitlab上的帐号无法登陆到平台上,也就无法提交拉取代码了。
解决思路:
- 方法一、重新配置一个Ldap服务把之前的数据库导进来,迁移到新的机器上然后修改Gitlab的认证服务地址。
- 方法二、直接修改Gitlab的数据库,对帐号的认证方式修改。
此处选择方式2进行操作其流程如下:
Step1.修改gitlab的数据库配置,开启远程访问(默认本机访问)
代码语言:javascript复制#1.修改配置文件添加最后两行
egrep -v "^#|^$" /var/opt/gitlab/postgresql/data/pg_hba.conf
local all all peer map=gitlab
host all all 127.0.0.1/32 trust
host all all 0.0.0.0/0 trust
#2.修改postgresql配置文件将listen_addresses把它改成*号
$vim /var/opt/gitlab/postgresql/data/postgresql.conf
#------------------------------------------------------------------------------
# CONNECTIONS AND AUTHENTICATION
#------------------------------------------------------------------------------
# - Connection Settings -
listen_addresses = '*' # what IP address(es) to listen on;
# comma-separated list of addresses;
# defaults to 'localhost', '*' = all
# (change requires restart)
port = 5432 # (change requires restart)
max_connections = 200 # (change requires restart)
# Note: Increasing max_connections costs ~400 bytes of shared memory per
# connection slot, plus lock space (see max_locks_per_transaction).
#superuser_reserved_connections = 3 # (change requires restart)
unix_socket_directories = '/var/opt/gitlab/postgresql' # (change requires restart)
#3.重启postgresql服务
gitlab-ctl restart postgresql
#4.使用Navicat工具连postgresql接数据库,下边的初始数据库gitlabhq_production和用户名都是默认的,密码为空
连接上后找到对应的identities表修改对应的字段,其实就是把字段中的user_id修改掉,我这里做法就是把用户的user_id改成了负数,这样在Ldap认证时就找不到了,从而走标准认证(这是在不重启Gitlab的情况下进行)。
WeiyiGeek.
补充说明:
- 更新后如果用户还存在登陆不了的问题,这时候可以让用户自己去Web端登陆gitlab,选择忘记密码,通过忘记密码来重新设置一个密码。(前提是你实现了密码找回功能)
- 完成所有操作后记得要把远程的配置注释掉或者删除了,然后重启postgresql服务即可。
(5) Gitlab 配置中关闭 Promethes / grafana 描述: 关闭 Prometheus / Grafana 解决的问题当主机内存较小时可以将其关闭,但是现在服务器内存一般都是够用的; 配置关键项: /etc/gitlab/gitlab.rb
代码语言:javascript复制# Prometheus
prometheus['enable'] = false
# gitlab_monitor['enable'] = false
# Grafana
grafana['enable'] = false
PS : 修改配置文件后一定要重新初始化Gitlab服务并启动其服务执行 gitlab-ctl reconfigure && gitlab-ctl start
命令;
(6) sidekiq_cluster has been deprecated since 13.6 and will be removed in 14.0
代码语言:javascript复制Deprecations:
* sidekiq_cluster['experimental_queue_selector'] has been deprecated since 13.6 and will be removed in 14.0. The experimental_queue_selector option is now called queue_selector.
0x06 补充附录
安装配置脚本
代码语言:javascript复制#!/bin/bash
#Desc: Gitlab代码服务器自动化部署
#Author:WeiyiGeek
#SupportOS:CentOS7 / CentOS8
GITLAB_BASEDOMAIN=weiyigeek.top
GITLAB_VERSION=12.9.2
GITlABOS7=gitlab-ce-${GITLAB_VERSION}-ce.0.el7.x86_64.rpm
GITlABOS8=gitlab-ce-${GITLAB_VERSION}-ce.0.el8.x86_64.rpm
GITLABRUNNER_VERSION=12.9.0-1
GITLABRUNNER_NAME=gitlab-runner-${GITLABRUNNER_VERSION}.x86_64.rpm
CheckOSVersion=$(uname -r | grep -c el8)
## [镜像源设置]
function repoChange(){
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.$(date "%Y%m%d").backup
if [ $CheckOSVersion -eq 1 ];then
# CentOS8 源
curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-8.repo
# 安装 epel 配置包并地址替换为阿里云镜像站地址
dnf install -y https://mirrors.aliyun.com/epel/epel-release-latest-8.noarch.rpm
sed -i 's|^#baseurl=https://download.fedoraproject.org/pub|baseurl=https://mirrors.aliyun.com|' /etc/yum.repos.d/epel*
sed -i 's|^metalink|#metalink|' /etc/yum.repos.d/epel*
dnf clean all
dnf makecache
else
# CentOS7 源
curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
#添加信任 GitLab 里的 GPG 公钥
sudo cat > /etc/yum.repos.d/gitlab-ce.repo <<EOF
[gitlab-ce]
name=Gitlab CE Repository
baseurl=https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/
gpgcheck=0
enabled=1
gpgkey=https://packages.gitlab.com/gpg.key
EOF
yum clean all
yum makecache
fi
}
#[yum 方式安装]
function yumInstall(){
# 查看可用的版本neng'b
yum list gitlab-ce --showduplicates
# 默认安装最新的版本
yum install -y gitlab-ce
# 安装指定版本 12.3.5
# yum install gitlab-ce-12.3.5-ce.0.el7.x86_64.rpm
}
#[rpm 方式安装-推荐方式]
function OmnibusInstall(){
if [ $CheckOSVersion -eq 1 ];then
wget -O $GITlABOS8 https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el8/${GITlABOS8}
rpm -i $GITlABOS8
else
wget -O $GITlABOS7 https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/${GITlABOS7}
rpm -i $GITlABOS7
fi
}
function gitlabSetting(){
sed -i "s#example.com#${GITLAB_BASEDOMAIN}#g" /etc/gitlab/gitlab.rb
echo "127.0.0.1 gitlab.${GITLAB_BASEDOMAIN}" > /etc/hosts
}
function useage(){
echo -e "e[32m# Description: Gitlab 自动化安装部署脚本"
echo -e "usage: $0 [rpm|yum] #指定rpm安装还是yum安装"
echo -e "Author:WeiyiGeeke[0m"
}
#[低于 12.3.x 版本的才进行设置]
function Chinesization(){
#停止gitlab
gitlab-ctl stop
#获取当前安装的版本补丁
git clone https://gitlab.com/xhang/gitlab.git
cd gitlab
gitlab_version=$(cat /opt/gitlab/embedded/service/gitlab-rails/VERSION)
# 生成对应版本补丁文件
git diff remotes/origin/12-3-stable remotes/origin/12-3-stable-zh > ../${gitlab_version}-zh.diff
# 打补丁的时候会提示一些补丁文件不存在,一定要跳过这些文件,不然后面reconfig的时候会报错的。
patch -d /opt/gitlab/embedded/service/gitlab-rails -p1 < ../${gitlab_version}-zh.diff
gitlab-ctl reconfigure
gitlab-ctl restart
}
## [安装配置脚本入口函数]
function main(){
#关闭Sellinux
echo "当前Selinux: $(getenforce)"
setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
echo "设置Selinux: $(getenforce)"
#环境依赖安装
repoChange
sudo yum install -y curl policycoreutils openssh-server wget postfix git htop ncdu net-tools
systemctl enable postfix
systemctl start postfix
#防护墙设置
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo systemctl reload firewalld
#选择安装方式
if [ $1 == "rpm" ];then
OmnibusInstall
elif [ $1 == "yum" ];then
yumInstall
else
usage
fi
gitlabSetting
gitlab-ctl reconfigure
gitlab-ctl start
gitlab-ctl status
}
#[参数验证]
if [ $# -ne 1 ];then
usage
else
main $1
fi