lustre中服务通过MGS加载的Device和服务的内核模块

2022-08-17 13:10:13 浏览数 (2)

lustre系统后端磁盘初始化脚本

代码语言：javascript复制
// mdt和mgt共享一个磁盘，因为mgt仅仅需要100M空间，单独一个磁盘会浪费空间
[root@CentOS-Lustre-Server ~]$ mkfs.lustre  --reformat  --fsname=perrynfs --mgs --mdt    --index=0   /dev/sdb

// 初始化ost，这里在ost对应的磁盘写入了mgs的地址
[root@CentOS-Lustre-Server ~]$ mkfs.lustre --ost --reformat --fsname perrynfs --index 0 --mgsnode CentOS-Lustre-Server@tcp0 /dev/sdc

// 客户端挂载时候制定了mgs的地址
[root@CentOS-Lustre-Client ~]$ mount -t lustre CentOS-Lustre-Server@tcp0:/perrynfs /mnt/lustre/

lustre服务中每个服务加载的Device

  • lustre不管是客户端还是MDS、MGS、OSS服务 都是通过加载device来组件整个服务，那么针对MDS/OSS/MGS在每个MGS节点上输出每个服务的device的信息
代码语言：javascript复制
//挂载mgt和mdt节点
[root@CentOS-Lustre-Server ~]$ mount -t ldiskfs /dev/sdb /mnt/mgt_mdt


// 查看当前MGS端文件系统的配置信心
[root@CentOS-Lustre-Server ~]$ ls /mnt/mgt_mdt/CONFIGS/
mountdata  nodemap  osd-ldiskfs-client  params  perrynfs-client  perrynfs-MDT0000  perrynfs-OST0000  perrynfs-OST0001  perrynfs-OST0002
  • 查看客户端通过挂载参数解析,初始化MDC连接到MGS中，MGS通过lustre log推送给客户端，客户端根据lustre log来加载自己的device，这些device如下
代码语言：javascript复制
[root@CentOS-Lustre-Server ~]$ lctl --device MGS llog_print perrynfs-client
- { index: 2, event: attach, device: perrynfs-clilov, type: lov, UUID: perrynfs-clilov_UUID }
- { index: 3, event: setup, device: perrynfs-clilov, UUID:  }
- { index: 6, event: attach, device: perrynfs-clilmv, type: lmv, UUID: perrynfs-clilmv_UUID }
- { index: 7, event: setup, device: perrynfs-clilmv, UUID:  }
- { index: 10, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 11, event: attach, device: perrynfs-MDT0000-mdc, type: mdc, UUID: perrynfs-clilmv_UUID }
- { index: 12, event: setup, device: perrynfs-MDT0000-mdc, UUID: perrynfs-MDT0000_UUID, node: 10.211.55.6@tcp }
- { index: 13, event: add_mdc, device: perrynfs-clilmv, mdt: perrynfs-MDT0000_UUID, index: 0, gen: 1, UUID: perrynfs-MDT0000-mdc_UUID }
- { index: 16, event: new_profile, name: perrynfs-client, lov: perrynfs-clilov, lmv: perrynfs-clilmv }
- { index: 19, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 20, event: attach, device: perrynfs-OST0000-osc, type: osc, UUID: perrynfs-clilov_UUID }
- { index: 21, event: setup, device: perrynfs-OST0000-osc, UUID: perrynfs-OST0000_UUID, node: 10.211.55.6@tcp }
- { index: 22, event: add_osc, device: perrynfs-clilov, ost: perrynfs-OST0000_UUID, index: 0, gen: 1 }
- { index: 25, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 26, event: attach, device: perrynfs-OST0001-osc, type: osc, UUID: perrynfs-clilov_UUID }
- { index: 27, event: setup, device: perrynfs-OST0001-osc, UUID: perrynfs-OST0001_UUID, node: 10.211.55.6@tcp }
- { index: 28, event: add_osc, device: perrynfs-clilov, ost: perrynfs-OST0001_UUID, index: 1, gen: 1 }
- { index: 31, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 32, event: attach, device: perrynfs-OST0002-osc, type: osc, UUID: perrynfs-clilov_UUID }
- { index: 33, event: setup, device: perrynfs-OST0002-osc, UUID: perrynfs-OST0002_UUID, node: 10.211.55.6@tcp }
- { index: 34, event: add_osc, device: perrynfs-clilov, ost: perrynfs-OST0002_UUID, index: 2, gen: 1 }
  • OSS端是初始化磁盘把MGS的地址写入磁盘，挂载磁盘时候，参数初始化MGC连接到MGS,通过lustre log来告诉OSS需要加载那些内核模块,从如下信息得知OSS加载了一个deviceobdfilter.
代码语言：javascript复制
[root@CentOS-Lustre-Server ~]$ lctl --device MGS llog_print perrynfs-OST0000
- { index: 2, event: attach, device: perrynfs-OST0000, type: obdfilter, UUID: perrynfs-OST0000_UUID }
- { index: 3, event: setup, device: perrynfs-OST0000, UUID: dev, node: type, options: f }
[root@CentOS-Lustre-Server ~]$
  • MDS端是初始化磁盘把MGS的地址写入磁盘,解析挂载参数初始化MGC连接到MGS,通过lustre log来告诉MDS需要加载那些device
代码语言：javascript复制
[root@CentOS-Lustre-Server ~]$ lctl --device MGS llog_print perrynfs-MDT0000
- { index: 2, event: attach, device: perrynfs-MDT0000-mdtlov, type: lov, UUID: perrynfs-MDT0000-mdtlov_UUID }
- { index: 3, event: setup, device: perrynfs-MDT0000-mdtlov, UUID:  }
- { index: 6, event: attach, device: perrynfs-MDT0000, type: mdt, UUID: perrynfs-MDT0000_UUID }
- { index: 7, event: new_profile, name: perrynfs-MDT0000, lov: perrynfs-MDT0000-mdtlov }
- { index: 8, event: setup, device: perrynfs-MDT0000, UUID: perrynfs-MDT0000_UUID, node: 0, options: perrynfs-MDT0000-mdtlov, failout: f }
- { index: 11, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 12, event: attach, device: perrynfs-OST0000-osc-MDT0000, type: osc, UUID: perrynfs-MDT0000-mdtlov_UUID }
- { index: 13, event: setup, device: perrynfs-OST0000-osc-MDT0000, UUID: perrynfs-OST0000_UUID, node: 10.211.55.6@tcp }
- { index: 14, event: add_osc, device: perrynfs-MDT0000-mdtlov, ost: perrynfs-OST0000_UUID, index: 0, gen: 1 }
- { index: 17, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 18, event: attach, device: perrynfs-OST0001-osc-MDT0000, type: osc, UUID: perrynfs-MDT0000-mdtlov_UUID }
- { index: 19, event: setup, device: perrynfs-OST0001-osc-MDT0000, UUID: perrynfs-OST0001_UUID, node: 10.211.55.6@tcp }
- { index: 20, event: add_osc, device: perrynfs-MDT0000-mdtlov, ost: perrynfs-OST0001_UUID, index: 1, gen: 1 }
- { index: 23, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 24, event: attach, device: perrynfs-OST0002-osc-MDT0000, type: osc, UUID: perrynfs-MDT0000-mdtlov_UUID }
- { index: 25, event: setup, device: perrynfs-OST0002-osc-MDT0000, UUID: perrynfs-OST0002_UUID, node: 10.211.55.6@tcp }
- { index: 26, event: add_osc, device: perrynfs-MDT0000-mdtlov, ost: perrynfs-OST0002_UUID, index: 2, gen: 1 }

lustre每个服务加载的内核模块

device oss 磁盘 后端 客户端

0 人点赞