SUSE Ceph 快速部署 – Storage6

2022-07-28 15:26:37 浏览数 (1)

一、安装环境描述

整个环境采用VMware workstation搭建,一共5台虚拟机,所有虚拟机安装SLES15SP1系统,其中一台安装apache作为仓库使用,剩余4台用于搭建Storage6集群。

硬件环境:     笔记本一台,CPU、内存和磁盘空间足够     笔记本上建立一块环回接口用于分布式存储管理网络

软件环境:     笔记本安装 Win10 操作系统    虚拟环境:VMware Workstation 14 Pro                   使用VMnet2和VMnet3 作为分布式存储 public和cluster网络     虚拟机操作系统:SLES15SP1 企业版操作系统

1、网络

代码语言:javascript复制
主机名      public网络       管理网络         集群网络        描述   
smt                       172.200.50.19                   SUSE仓库
admin     192.168.2.39    172.200.50.39   192.168.3.39    管理主机
node001   192.168.2.41    172.200.50.41   192.168.3.41    MON
node002   192.168.2.42    172.200.50.42   192.168.3.42    MON
node003   192.168.2.43    172.200.50.43   192.168.3.43    MON

2、磁盘 每个系统有2块OSD盘,·1块NVME磁盘

代码语言:javascript复制
 1 # lsblk 
 2 NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
 3 sda               8:0    0   20G  0 disk              # 操作系统盘
 4 ├─sda1            8:1    0    1G  0 part /boot
 5 └─sda2            8:2    0   19G  0 part 
 6   ├─vgoo-lvroot 254:0    0   17G  0 lvm  /
 7   └─vgoo-lvswap 254:1    0    2G  0 lvm  [SWAP]
 8 sdb               8:16   0   10G  0 disk              # osd 数据盘
 9 sdc               8:32   0   10G  0 disk              # osd 数据盘
10 nvme0n1         259:0    0   20G  0 disk              # wal db 

二、操作系统初始化安装

1、临时IP地址设置

代码语言:javascript复制
1 ip link set eth0 up
2 ip addr add 172.200.50.50/24 dev eth0

连接上了设置永久地址

代码语言:javascript复制
yast lan list
yast lan edit id=0 ip=192.168.2.40 netmask=255.255.255.0

2、设置bash环境变量和别名

代码语言:javascript复制
# vim /root/.bash_profile 
alias cd..='cd ..'
alias dir='ls -l'
alias egrep='egrep --color=auto'
alias fgrep='fgrep --color=auto'
alias grep='grep --color=auto'
alias l='ls -alF'
alias la='ls -la'
alias ll='ls -l'
alias ls-l='ls -l'

3、配置after.local文件

代码语言:javascript复制
touch /etc/init.d/after.local
chmod 744 /etc/init.d/after.local

 复制该内容进去

代码语言:javascript复制
#! /bin/sh
#
# Copyright (c) 2010 SuSE LINUX Products GmbH, Germany.  All rights reserved.
#
# Author: Werner Fink, 2010
#
# /etc/init.d/after.local
#
# script with local commands to be executed from init after all scripts
# of a runlevel have been executed.
#
# Here you should add things, that should happen directly after
# runlevel has been reached.
#

 4、仓库配置(所有节点和admin)

代码语言:javascript复制
## Pool
zypper ar http://172.200.50.19/repo/SUSE/Products/SLE-Product-SLES/15-SP1/x86_64/product/    SLE-Product-SLES15-SP1-Pool
zypper ar http://172.200.50.19/repo/SUSE/Products/SLE-Module-Basesystem/15-SP1/x86_64/product/ SLE-Module-Basesystem-SLES15-SP1-Pool
zypper ar http://172.200.50.19/repo/SUSE/Products/SLE-Module-Server-Applications/15-SP1/x86_64/product/ SLE-Module-Server-Applications-SLES15-SP1-Pool
zypper ar http://172.200.50.19/repo/SUSE/Products/SLE-Module-Legacy/15-SP1/x86_64/product/  SLE-Module-Legacy-SLES15-SP1-Pool
zypper ar  http://172.200.50.19/repo/SUSE/Products/Storage/6/x86_64/product/ SUSE-Enterprise-Storage-6-Pool

## Update
zypper ar http://172.200.50.19/repo/SUSE/Updates/SLE-Product-SLES/15-SP1/x86_64/update/ SLE-Product-SLES15-SP1-Updates
zypper ar http://172.200.50.19/repo/SUSE/Updates/SLE-Module-Basesystem/15-SP1/x86_64/update/  SLE-Module-Basesystem-SLES15-SP1-Upadates
zypper ar http://172.200.50.19/repo/SUSE/Updates/SLE-Module-Server-Applications/15-SP1/x86_64/update/ SLE-Module-Server-Applications-SLES15-SP1-Upadates
zypper ar http://172.200.50.19/repo/SUSE/Updates/SLE-Module-Legacy/15-SP1/x86_64/update/ SLE-Module-Legacy-SLES15-SP1-Updates
zypper ar http://172.200.50.19/repo/SUSE/Updates/Storage/6/x86_64/update/ SUSE-Enterprise-Storage-6-Updates
代码语言:javascript复制
# zypper lr
#  | Alias                                              | Name                                               
--- ---------------------------------------------------- ----------------------------------------------------
 1 | SLE-Module-Basesystem-SLES15-SP1-Pool              | SLE-Module-Basesystem-SLES15-SP1-Pool              
 2 | SLE-Module-Basesystem-SLES15-SP1-Upadates          | SLE-Module-Basesystem-SLES15-SP1-Upadates          
 3 | SLE-Module-Legacy-SLES15-SP1-Pool                  | SLE-Module-Legacy-SLES15-SP1-Pool                  
 4 | SLE-Module-Legacy-SLES15-SP1-Updates               | SLE-Module-Legacy-SLES15-SP1-Updates               
 5 | SLE-Module-Server-Applications-SLES15-SP1-Pool     | SLE-Module-Server-Applications-SLES15-SP1-Pool     
 6 | SLE-Module-Server-Applications-SLES15-SP1-Upadates | SLE-Module-Server-Applications-SLES15-SP1-Upadates 
 7 | SLE-Product-SLES15-SP1-Pool                        | SLE-Product-SLES15-SP1-Pool                        
 8 | SLE-Product-SLES15-SP1-Updates                     | SLE-Product-SLES15-SP1-Updates                     
 9 | SUSE-Enterprise-Storage-6-Pool                     | SUSE-Enterprise-Storage-6-Pool                     
10 | SUSE-Enterprise-Storage-6-Updates                  | SUSE-Enterprise-Storage-6-Updates                  

5、安装基本软件 (所有节点和admin)

代码语言:javascript复制
zypper in -y -t pattern yast2_basis base
zypper in -y net-tools vim man sudo tuned irqbalance
zypper in -y ethtool rsyslog iputils less supportutils-plugin-ses
zypper in -y net-tools-deprecated tree wget

6、关闭IPV6 (所有节点和admin)

代码语言:javascript复制
# vim /etc/sysctl.conf
net.ipv6.conf.all.disable_ipv6 = 1                 # 关闭 IPV6
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
vm.min_free_kbytes = 2097152                       # 128GB的RAM,系统预留2GB
kernel.pid_max = 4194303                           # 线程数设置最大

执行生效

代码语言:javascript复制
# sysctl -p

7、调整网络优化参数 (所有节点和admin)

代码语言:javascript复制
# tuned-adm profile throughput-performance
# tuned-adm active
# systemctl start tuned.service
# systemctl enable tuned.service

8、编辑hosts文件 (所有节点和admin)

代码语言:javascript复制
# vim /etc/hosts
192.168.2.39    admin.example.com admin
192.168.2.40    node001.example.com node001
192.168.2.41    node002.example.com node002
192.168.2.42    node003.example.com node003

9、执行更新操作系统,并重启 (所有节点和admin)

代码语言:javascript复制
# zypper ref
# zypper -n update
# reboot  

三、安装Storage6集群

1、安装 salt(admin节点)

代码语言:javascript复制
zypper -n in deepsea
systemctl restart salt-master.service
systemctl enable salt-master.service
systemctl status salt-master.service

osd节点 和 admin节点

代码语言:javascript复制
zypper -n in salt-minion
sed -i '17imaster: 192.168.2.39' /etc/salt/minion
systemctl restart salt-minion.service
systemctl enable salt-minion.service
systemctl status salt-minion.service

接受所有请求(admin节点)

代码语言:javascript复制
salt-key
salt-key --accept-all
salt-key
salt '*' test.ping

2、Admin节点,配置NTP服务 如果没有ntp服务器,默认以admin节点为ntp server Admin节点:

代码语言:javascript复制
# vim /etc/chrony.conf
# Sync to local clock        # 添加本地时钟源
server 127.0.0.1             
allow 127.0.0.0/8
allow 192.168.2.0/24
allow 172.200.50.0/24
local stratum 10
代码语言:javascript复制
systemctl restart chronyd.service
systemctl enable chronyd.service
systemctl status chronyd.service
代码语言:javascript复制
# chronyc sources      
210 Number of sources = 1
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^* 127.127.1.0                  12   6    37    23   1461ns[ 3422ns]  /-  166us

# chronyc -n sources -v

3、修改组(admin节点)

代码语言:javascript复制
cp -p /srv/pillar/ceph/deepsea_minions.sls /tmp/
sed -i "4c # deepsea_minions: 'G@deepsea:*'" /srv/pillar/ceph/deepsea_minions.sls
sed -i "6c deepsea_minions: '*'" /srv/pillar/ceph/deepsea_minions.sls

4、远程连接终端监控(admin节点)

该程序监控提供一个详细的,实时的可视化操作行为,当在执行运行salt-run state.orch时,监控执行期间运行了什么

代码语言:javascript复制
# deepsea monitor

5、更新补丁,并收集硬件信息(admin节点)

代码语言:javascript复制
salt-run state.orch ceph.stage.0
salt-run state.orch ceph.stage.1

报错信息可忽略

代码语言:javascript复制
No minions matched the target. No command was sent, no jid was assigned.
No minions matched the target. No command was sent, no jid was assigned.
[ERROR   ] Exception during resolving address: [Errno 2] Host name lookup failure
[ERROR   ] Exception during resolving address: [Errno 2] Host name lookup failure
[WARNING ] /usr/lib/Python3.6/site-packages/salt/grains/core.py:2827: DeprecationWarning: This server_id is
computed nor by Adler32 neither by CRC32. Please use "server_id_use_crc" option and define algorithm youprefer (
default "Adler32"). The server_id will be computed withAdler32 by default.

GitHub

0 人点赞