简介
本节介绍如何在 Docker 容器中构建和运行 DAOS 服务。 至少需要 4GB DRAM 和 16GB 磁盘空间。 在 Mac 上，请确保“Preferences/{Disk, Memory}”下的 Docker 设置已进行相应配置
步骤
拉取项目
代码语言：javascript复制git clone https://github.com/ssbandjl/daos.git下载依赖包(缓存cache)
代码语言：javascript复制cd daos; wget http://rz2fg6ogr.hn-bkt.clouddn.com/cache_tgz编译镜像或下载我这边已经编好的镜像
代码语言：javascript复制编译镜像: docker build . -f utils/docker/Dockerfile.el.8 -t daos
# 已编好的镜像: wget http://rz2fg6ogr.hn-bkt.clouddn.com/daos_main.tgz; docker load -i daos_main.tgz查看docker镜像:
[root@node3 daos]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
daos latest 8a64df225def 2 weeks ago 3.16GB
给宿主机配置大页(如8GB):
永久生效
tee<<EOF>>/etc/sysctl.conf
vm.nr_hugepages = 4096
EOF
sysctl -p
启动docker镜像:
docker run -it -d --privileged --cap-add=ALL --name daos_main1 -v /home/xb/project/stor/daos/main/daos:/home/xb/project/stor/daos/main/daos -v /dev:/dev -v /dev/hugepages:/dev/hugepages -v /sys/fs/cgroup:/sys/fs/cgroup:ro daos:latest
进入镜像: docker exec -u root -it daos_main1 bash -c 'cd /home/xb/project/stor/daos/main/daos;exec "${SHELL:-sh}"'
重新编译并安装DAOS:
./build.sh 或
scons --jobs 1 install PREFIX=/opt/daos BUILD_TYPE=debug TARGET_TYPE=debug --build-deps=yes
which dmg
拷贝证书和配置(启动时根据提示修改权限):
cp -r daosCA/certs /etc/daos/参考配置文件: daos_server_docker_main.yml
代码语言：javascript复制# For a single-server system
name: daos_server
access_points: ['172.17.0.2']
provider: ofi sockets
control_log_mask: DEBUG
control_log_file: /tmp/daos_server.log
transport_config:
allow_insecure: false
client_cert_dir: /etc/daos/certs/clients
ca_cert: /etc/daos/certs/daosCA.crt
cert: /etc/daos/certs/server.crt
key: /etc/daos/certs/server.key
engines:
-
targets: 1
first_core: 0
nr_xs_helpers: 0
fabric_iface: eth0
fabric_iface_port: 31416
log_mask: debug
log_file: /tmp/daos_engine.0.log
env_vars:
- FI_LOG_LEVEL=warn
- FI_SOCKETS_MAX_CONN_RETRY=1
- FI_SOCKETS_CONN_TIMEOUT=2000
- D_LOG_MASK=DEBUG
- DD_SUBSYS=all
- DD_MASK=all
- HG_LOG_LEVEL=debug
# Storage definitions
# When scm_class is set to ram, tmpfs will be used to emulate SCM.
# The size of ram is specified by scm_size in GB units.
scm_mount: /mnt/daos # map to -s /mnt/daos
scm_class: ram
scm_size: 4 #内存模拟scm
# When bdev_class is set to file, Linux AIO will be used to emulate NVMe.
# The size of file that will be created is specified by bdev_size in GB units.
# The location of the files that will be created is specified in bdev_list.
bdev_class: file
bdev_size: 16 #文件模拟nvme
bdev_list: [/tmp/daos-bdev]启动集群
代码语言：javascript复制启动集群:
./start.sh
或参考:
echo -e "start"
# source /root/.bashrc
# run_all "pkill daos_agent;pkill daos_server"
source /root/.bashrc
pkill daos_agent
pkill daos_server
rm -f /tmp/daos*.log
rm -f /tmp/.daos_engine.0.log.swp
rm -f /tmp/daos_engine*
umount /mnt/daos
#export FI_LOG_LEVEL=debug
#export FI_LOG_LEVEL=warn
#export HG_LOG_LEVEL=debug
#export HG_LOG_LEVEL=warn
daos_agent &
# echo -e "dmg storage format"
daos_server start &
# count=0
# while true;do
# joined_num=`dmg sys query -v|grep Joined|wc -l`
# if [[ $joined_num -eq 3 ]];then
# break
# fi
# echo -e "wait all rank join, $count times"
# count=$((count 1))
# sleep 1
# done
sleep 5
dmg storage format
创建池,容器, 挂载daos文件系统:
参考文件: daos_create_pool_cont_mount_dfuse
dmg pool create sxb -z 4g; dmg pool list --verbose
daos container create sxb --type POSIX sxb; daos container query sxb sxb --verbose; daos cont get-prop sxb sxb
mkdir -p /tmp/sxb; dfuse --mountpoint=/tmp/sxb --pool=sxb --cont=sxb; df -h
#cd /tmp/sxb最终效果:
代码语言：javascript复制[root@8b90fa1190f1 daos]# dmg pool list --verbose
Label UUID State SvcReps SCM Size SCM Used SCM Imbalance NVME Size NVME Used NVME Imbalance Disabled UpgradeNeeded?
----- ---- ----- ------- -------- -------- ------------- --------- --------- -------------- -------- --------------
sxb 9b4157a8-7bab-493c-aa4d-5145144d92b3 Ready 0 240 MB 213 kB 0% 3.8 GB 52 MB 0% 0/1 None
[root@8b90fa1190f1 daos]# daos container query sxb sxb --verbose; daos cont get-prop sxb sxb
Container UUID : ccc2f4f7-1e46-4709-8f28-09282a9f82ed
Container Label : sxb
Container Type : POSIX
Pool UUID : 9b4157a8-7bab-493c-aa4d-5145144d92b3
Container redundancy factor : 0
Number of open handles : 2
Latest open time : 0x126c7703f0840000 (2023-08-19 08:02:08.06070272 0000 UTC)
Latest close/modify time : 0x126c76d527b80000 (2023-08-19 08:01:55.50212096 0000 UTC)
Number of snapshots : 0
Object Class : UNKNOWN
Dir Object Class : UNKNOWN
File Object Class : UNKNOWN
Chunk Size : 1.0 MiB
Properties for container sxb
Name Value
---- -----
Highest Allocated OID (alloc_oid) 11
Checksum (cksum) off
Checksum Chunk Size (cksum_size) 32 KiB
Compression (compression) off
Deduplication (dedup) off
Dedupe Threshold (dedup_threshold) 4.0 KiB
EC Cell Size (ec_cell_sz) 64 KiB
Performance domain affinity level of EC (ec_pda) 1
Encryption (encryption) off
Global Version (global_version) 3
Group (group) root@
Label (label) sxb
Layout Type (layout_type) POSIX (1)
Layout Version (layout_version) 1
Max Snapshot (max_snapshot) 0
Object Version (obj_version) 1
Owner (owner) root@
Performance domain level (perf_domain) root (255)
Redundancy Factor (rd_fac) rd_fac0
Redundancy Level (rd_lvl) node (2)
Performance domain affinity level of RP (rp_pda) 4294967295
Server Checksumming (srv_cksum) off
Health (status) HEALTHY
Access Control List (acl) A::OWNER@:rwdtTaAo, A:G:GROUP@:rwtT
[root@8b90fa1190f1 daos]# df -h
Filesystem Size Used Avail Use% Mounted on
overlay 50G 25G 26G 50% /
devtmpfs 63G 0 63G 0% /dev
tmpfs 63G 0 63G 0% /dev/shm
/dev/mapper/xxx-var 50G 25G 26G 50% /etc/hosts
tmpfs 63G 0 63G 0% /sys/fs/cgroup
/dev/mapper/xxx-root 120G 56G 65G 47% /home/xb/project/stor/daos/main/daos
tmpfs 4.0G 500M 3.6G 13% /mnt/daos
dfuse 3.8G 51M 3.7G 2% /tmp/sxb
[root@8b90fa1190f1 daos]# 参考
在docker中搭建开发环境: https://docs.daos.io/v2.0/dev/development/#daos-development-in-docker
晓兵
博客: https://logread.cn | https://blog.csdn.net/ssbandjl | https://cloud.tencent.com/developer/user/5060293/articles
weixin: ssbandjl
公众号: 云原生云
