1.greenplum为master添加mirror
在已经安装的 Greenplum DB ,给 master 节点添加 standby master 节点 。
启用 master 镜像
- 可以在 greenplum 系统初始化的时候使用 gpinitsystem 添加
- 增加一个 standby master 到一个存在的 gp 系统
1.1.使用 gpinitsystem -s 来完成初始化时，添加 standby master
1.2.使用 gpinitstandby 添加
step 1.安装greenplum 软件在 standby 节点
代码语言：javascript复制dnf install open-source-greenplum-db-6.22.0-rhel8-x86_64.rpm -y
step 2.修改目录权限
代码语言：javascript复制chown -R gpadmin /usr/local/greenplum*
chgrp -R gpadmin /usr/local/greenplum*
step 3.使用 gpadmin 用户 登录master 节点， 交换ssh免密 文件
代码语言：javascript复制gpssh-exkeys -f all_hosts
step 4.初始化 standby master 使用gpadmin 用户 登录 greenplum 系统中的 master 节点; 使用gpinitstandby 初始化gp standby 节点
代码语言：javascript复制gpinitstandby -s gp-smdw02
-s 表示指定主机名; standby节点的目录一定要跟 master 节点的目录保持一致,可以使用 “S” 指定目录；
step 5.检查 standby 节点的 状态
代码语言：javascript复制psql -c 'SELECT procpid, state FROM pg_stat_replication;'
状态是streaming 表示正常 同步
2.greenplum为segment添加mirror
- 创建镜像有两种方式，一种是在初始化gp的时候就创 建mirror
- 在初始化之后，再添加mirror
greenplum目前只允许添加mirror，而不允许删除mirror，所以在添加镜像前，最好对gp做一下备份
查看当前segment的状态：
代码语言：javascript复制[gpadmin@gp-mdw ~]$ psql -c 'select dbid,content,role,port,hostname,address from gp_segment_configuration order by dbid;'
dbid | content | role | port | hostname | address
------ --------- ------ ------ ---------- ----------
1 | -1 | p | 5432 | gp-mdw | gp-mdw
2 | 0 | p | 6000 | gp-sdw01 | gp-sdw01
3 | 1 | p | 6001 | gp-sdw01 | gp-sdw01
4 | 2 | p | 6002 | gp-sdw01 | gp-sdw01
5 | 3 | p | 6003 | gp-sdw01 | gp-sdw01
6 | 4 | p | 6004 | gp-sdw01 | gp-sdw01
7 | 5 | p | 6005 | gp-sdw01 | gp-sdw01
8 | 6 | p | 6000 | gp-sdw02 | gp-sdw02
9 | 7 | p | 6001 | gp-sdw02 | gp-sdw02
10 | 8 | p | 6002 | gp-sdw02 | gp-sdw02
11 | 9 | p | 6003 | gp-sdw02 | gp-sdw02
12 | 10 | p | 6004 | gp-sdw02 | gp-sdw02
13 | 11 | p | 6005 | gp-sdw02 | gp-sdw02
14 | 12 | p | 6000 | gp-sdw03 | gp-sdw03
15 | 13 | p | 6001 | gp-sdw03 | gp-sdw03
16 | 14 | p | 6002 | gp-sdw03 | gp-sdw03
17 | 15 | p | 6003 | gp-sdw03 | gp-sdw03
18 | 16 | p | 6004 | gp-sdw03 | gp-sdw03
19 | 17 | p | 6005 | gp-sdw03 | gp-sdw03
20 | -1 | m | 5432 | gp-smdw | gp-smdw
(20 rows)
[gpadmin@gp-mdw ~]$
每个Segment节点增加一个Mirror，Segment增加Mirror也分两种情况，一种是在现有的服务器上添加，一种是在新增的机器添加。
step 1.创建segment 节点资源文件
代码语言：javascript复制cat > seg_hosts <<EOF
gp-sdw01
gp-sdw02
gp-sdw03
EOF
step 2.创建镜像的存储目录
代码语言：javascript复制gpssh -f seg_hosts -e 'mkdir -p /opt/greenplum/data1/primary'
gpssh -f seg_hosts -e 'mkdir -p /opt/greenplum/data1/mirror'
gpssh -f seg_hosts -e 'mkdir -p /opt/greenplum/data2/primary'
gpssh -f seg_hosts -e 'mkdir -p /opt/greenplum/data2/mirror'
step 3.生成增加镜像的配置文件
代码语言：javascript复制gpaddmirrors -o mirror_config_file
需要输入镜像目录的地址
代码语言：javascript复制[gpadmin@gp-mdw ~]$ gpaddmirrors -o mirror_config_file
20220928:17:54:22:062477 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Starting gpaddmirrors with args: -o mirror_config_file
20220928:17:54:22:062477 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.22.0 build commit:4b6c079bc3aed35b2f161c377e208185f9310a69 Open Source'
20220928:17:54:22:062477 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.22.0 build commit:4b6c079bc3aed35b2f161c377e208185f9310a69 Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), 64-bit compiled on Sep 8 2022 22:39:10'
20220928:17:54:22:062477 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Obtaining Segment details from master...
20220928:17:54:23:062477 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Heap checksum setting consistent across cluster
Enter mirror segment data directory location 1 of 6 >
/opt/greenplum/data1/mirror
Enter mirror segment data directory location 2 of 6 >
/opt/greenplum/data1/mirror
Enter mirror segment data directory location 3 of 6 >
/opt/greenplum/data1/mirror
Enter mirror segment data directory location 4 of 6 >
/opt/greenplum/data2/mirror
Enter mirror segment data directory location 5 of 6 >
/opt/greenplum/data2/mirror
Enter mirror segment data directory location 6 of 6 >
/opt/greenplum/data2/mirror
20220928:17:57:08:062477 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Configuration file output to mirror_config_file successfully.
[gpadmin@gp-mdw ~]$
step 4.查看生成的镜像配置文件：
代码语言：javascript复制[gpadmin@gp-mdw ~]$ cat mirror_config_file
0|gp-sdw02|7000|/opt/greenplum/data1/mirror/gpseg0
1|gp-sdw02|7001|/opt/greenplum/data1/mirror/gpseg1
2|gp-sdw02|7002|/opt/greenplum/data1/mirror/gpseg2
3|gp-sdw02|7003|/opt/greenplum/data2/mirror/gpseg3
4|gp-sdw02|7004|/opt/greenplum/data2/mirror/gpseg4
5|gp-sdw02|7005|/opt/greenplum/data2/mirror/gpseg5
6|gp-sdw03|7000|/opt/greenplum/data1/mirror/gpseg6
7|gp-sdw03|7001|/opt/greenplum/data1/mirror/gpseg7
8|gp-sdw03|7002|/opt/greenplum/data1/mirror/gpseg8
9|gp-sdw03|7003|/opt/greenplum/data2/mirror/gpseg9
10|gp-sdw03|7004|/opt/greenplum/data2/mirror/gpseg10
11|gp-sdw03|7005|/opt/greenplum/data2/mirror/gpseg11
12|gp-sdw01|7000|/opt/greenplum/data1/mirror/gpseg12
13|gp-sdw01|7001|/opt/greenplum/data1/mirror/gpseg13
14|gp-sdw01|7002|/opt/greenplum/data1/mirror/gpseg14
15|gp-sdw01|7003|/opt/greenplum/data2/mirror/gpseg15
16|gp-sdw01|7004|/opt/greenplum/data2/mirror/gpseg16
17|gp-sdw01|7005|/opt/greenplum/data2/mirror/gpseg17
[gpadmin@gp-mdw ~]$
step 5.添加镜像
代码语言：javascript复制[gpadmin@gp-mdw ~]$ gpaddmirrors -i mirror_config_file
20220928:18:00:01:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Starting gpaddmirrors with args: -i mirror_config_file
20220928:18:00:01:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.22.0 build commit:4b6c079bc3aed35b2f161c377e208185f9310a69 Open Source'
20220928:18:00:01:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.22.0 build commit:4b6c079bc3aed35b2f161c377e208185f9310a69 Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), 64-bit compiled on Sep 8 2022 22:39:10'
20220928:18:00:01:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Obtaining Segment details from master...
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Heap checksum setting consistent across cluster
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Greenplum Add Mirrors Parameters
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Greenplum master data directory = /opt/greenplum/data/master/gpseg-1
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Greenplum master port = 5432
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Batch size = 16
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Segment batch size = 64
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 1 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data1/primary/gpseg0
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6000
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data1/mirror/gpseg0
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7000
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 2 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data1/primary/gpseg1
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6001
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data1/mirror/gpseg1
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7001
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 3 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data1/primary/gpseg2
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6002
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data1/mirror/gpseg2
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7002
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 4 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data2/primary/gpseg3
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6003
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data2/mirror/gpseg3
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7003
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 5 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data2/primary/gpseg4
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6004
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data2/mirror/gpseg4
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7004
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 6 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data2/primary/gpseg5
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6005
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data2/mirror/gpseg5
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7005
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 7 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data1/primary/gpseg6
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6000
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data1/mirror/gpseg6
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7000
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 8 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data1/primary/gpseg7
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6001
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data1/mirror/gpseg7
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7001
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 9 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data1/primary/gpseg8
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6002
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data1/mirror/gpseg8
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7002
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 10 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data2/primary/gpseg9
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6003
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data2/mirror/gpseg9
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7003
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 11 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data2/primary/gpseg10
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6004
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data2/mirror/gpseg10
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7004
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 12 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw02
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data2/primary/gpseg11
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6005
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data2/mirror/gpseg11
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7005
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 13 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data1/primary/gpseg12
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6000
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data1/mirror/gpseg12
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7000
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 14 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data1/primary/gpseg13
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6001
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data1/mirror/gpseg13
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7001
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 15 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data1/primary/gpseg14
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6002
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data1/mirror/gpseg14
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7002
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 16 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data2/primary/gpseg15
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6003
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data2/mirror/gpseg15
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7003
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 17 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data2/primary/gpseg16
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6004
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data2/mirror/gpseg16
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7004
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror 18 of 18
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance host = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance address = gp-sdw03
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance directory = /opt/greenplum/data2/primary/gpseg17
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Primary instance port = 6005
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance host = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance address = gp-sdw01
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance directory = /opt/greenplum/data2/mirror/gpseg17
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:- Mirror instance port = 7005
20220928:18:00:02:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:---------------------------------------------
Continue with add mirrors procedure Yy|Nn (default=N):
> y
20220928:18:00:04:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Starting to create new pg_hba.conf on primary segments
20220928:18:00:12:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Successfully modified pg_hba.conf on primary segments to allow replication connections
20220928:18:00:12:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-18 segment(s) to add
20220928:18:00:12:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Setting up the required segments for recovery
20220928:18:00:12:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Updating configuration for mirrors
20220928:18:00:13:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Initiating segment recovery. Upon completion, will start the successfully recovered segments
20220928:18:00:13:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-era is 1a0312416dc95f99_220928124605
gp-sdw02 (dbid 21): pg_basebackup: base backup completed
gp-sdw02 (dbid 22): pg_basebackup: base backup completed
gp-sdw02 (dbid 23): pg_basebackup: base backup completed
gp-sdw02 (dbid 24): pg_basebackup: base backup completed
gp-sdw02 (dbid 25): pg_basebackup: base backup completed
gp-sdw02 (dbid 26): pg_basebackup: base backup completed
gp-sdw03 (dbid 27): pg_basebackup: base backup completed
gp-sdw03 (dbid 28): pg_basebackup: base backup completed
gp-sdw03 (dbid 29): pg_basebackup: base backup completed
gp-sdw03 (dbid 30): pg_basebackup: base backup completed
gp-sdw03 (dbid 31): pg_basebackup: base backup completed
gp-sdw03 (dbid 32): pg_basebackup: base backup completed
gp-sdw01 (dbid 33): pg_basebackup: base backup completed
gp-sdw01 (dbid 34): pg_basebackup: base backup completed
gp-sdw01 (dbid 35): pg_basebackup: base backup completed
gp-sdw01 (dbid 36): pg_basebackup: base backup completed
gp-sdw01 (dbid 37): pg_basebackup: base backup completed
gp-sdw01 (dbid 38): pg_basebackup: base backup completed
20220928:18:01:23:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Triggering FTS probe
20220928:18:01:24:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-******************************************************************
20220928:18:01:24:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Mirror segments have been added; data synchronization is in progress.
20220928:18:01:24:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Data synchronization will continue in the background.
20220928:18:01:24:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-Use gpstate -s to check the resynchronization progress.
20220928:18:01:24:062637 gpaddmirrors:gp-mdw:gpadmin-[INFO]:-******************************************************************
[gpadmin@gp-mdw ~]$
step 6.查看镜像的同步状态：
代码语言：javascript复制[gpadmin@gp-mdw ~]$ gpstate -e
20220928:18:03:07:063065 gpstate:gp-mdw:gpadmin-[INFO]:-Starting gpstate with args: -e
20220928:18:03:07:063065 gpstate:gp-mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.22.0 build commit:4b6c079bc3aed35b2f161c377e208185f9310a69 Open Source'
20220928:18:03:07:063065 gpstate:gp-mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.22.0 build commit:4b6c079bc3aed35b2f161c377e208185f9310a69 Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), 64-bit compiled on Sep 8 2022 22:39:10'
20220928:18:03:07:063065 gpstate:gp-mdw:gpadmin-[INFO]:-Obtaining Segment details from master...
20220928:18:03:07:063065 gpstate:gp-mdw:gpadmin-[INFO]:-Gathering data from segments...
20220928:18:03:08:063065 gpstate:gp-mdw:gpadmin-[INFO]:-----------------------------------------------------
20220928:18:03:08:063065 gpstate:gp-mdw:gpadmin-[INFO]:-Segment Mirroring Status Report
20220928:18:03:08:063065 gpstate:gp-mdw:gpadmin-[INFO]:-----------------------------------------------------
20220928:18:03:08:063065 gpstate:gp-mdw:gpadmin-[INFO]:-All segments are running normally
[gpadmin@gp-mdw ~]$ gpstate -f
step 7.查看mirror节点状态
代码语言：javascript复制[gpadmin@gp-mdw ~]$ gpstate -m
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:-Starting gpstate with args: -m
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.22.0 build commit:4b6c079bc3aed35b2f161c377e208185f9310a69 Open Source'
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.22.0 build commit:4b6c079bc3aed35b2f161c377e208185f9310a69 Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), 64-bit compiled on Sep 8 2022 22:39:10'
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:-Obtaining Segment details from master...
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:--------------------------------------------------------------
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:--Current GPDB mirror list and status
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:--Type = Group
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:--------------------------------------------------------------
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- Mirror Datadir Port Status Data Status
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw02 /opt/greenplum/data1/mirror/gpseg0 7000 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw02 /opt/greenplum/data1/mirror/gpseg1 7001 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw02 /opt/greenplum/data1/mirror/gpseg2 7002 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw02 /opt/greenplum/data2/mirror/gpseg3 7003 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw02 /opt/greenplum/data2/mirror/gpseg4 7004 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw02 /opt/greenplum/data2/mirror/gpseg5 7005 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw03 /opt/greenplum/data1/mirror/gpseg6 7000 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw03 /opt/greenplum/data1/mirror/gpseg7 7001 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw03 /opt/greenplum/data1/mirror/gpseg8 7002 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw03 /opt/greenplum/data2/mirror/gpseg9 7003 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw03 /opt/greenplum/data2/mirror/gpseg10 7004 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw03 /opt/greenplum/data2/mirror/gpseg11 7005 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw01 /opt/greenplum/data1/mirror/gpseg12 7000 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw01 /opt/greenplum/data1/mirror/gpseg13 7001 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw01 /opt/greenplum/data1/mirror/gpseg14 7002 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw01 /opt/greenplum/data2/mirror/gpseg15 7003 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw01 /opt/greenplum/data2/mirror/gpseg16 7004 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:- gp-sdw01 /opt/greenplum/data2/mirror/gpseg17 7005 Passive Synchronized
20220928:18:04:19:063190 gpstate:gp-mdw:gpadmin-[INFO]:--------------------------------------------------------------
[gpadmin@gp-mdw ~]$
3.扩容 segment
Gpexpand是Greenplum数据库的扩容工具，可以为集群增加新节点从而可以存储更多的数据，提供更高的计算能力。Greenplum 5及之前，集群扩容需要停机增加新节点，然后对表数据做重分布。因为集群大小已经改变，所以重分布之前要先将所有哈希分布表改成随机分布，然后再按照新的集群大小重新计算哈希值并重新分布。
所以，旧的扩容技术存在如下几个问题：
- 需要停机，集群无法在扩容期间提供服务。
- 数据重分布过程集群性能差。随机分布表因为不能确定数据分布规律，无法对查询做优化。
- 虽然表的重分布可以并行进行，但是扩容过程中因为额外记录了每个表的更新状态，在对表做完重分布后要更新状态表。Greenplum 6版本之前表的更新操作只能串行。所以对大量小表做并行重分布的时候会因为状态表更新的串行化产生瓶颈。
全新设计的gpexpand支持在线扩容，同时对数据重分布过程做了优化，提高了并发度。整个扩容分为两个阶段：
- 在线增加新节点
- 数据重分布
3.1.如何在线增加新节点?
Greenplum通过catalog表gp_segment_configuration维护集群节点信息。所以在线增加节点并不复杂，只要更新这个系统表即可：首先以master数据库目录为模板创建新的子节点，然后更新gp_segment_configuration表。新节点没有用户数据，只有catalog数据，这些数据在master节点上都有。
因为是在线扩容，这里有几个问题：
- 新节点加入后，哈希分布的数据表还是按照旧集群的大小进行哈希分布。对这样的表怎么处理
- 正在运行的事务
- 新事务
- 扩容过程中，catalog更改操作（建表，删表，改表）如何处理
问题1 为了解决这个问题，我们在catalog表gp_distribution_policy里引入了一个新的属性，numsegments。它描述了该表分布在哪些节点上，n表示分布在前n个节点上。同时优化了执行查询的工作进程的数量管理策略，可以指定在哪些节点上启动执行查询工作进程。所以，即使集群添加了新节点，但是未数据重分布的表仍然只分布数据在集群的旧节点上，并且分布策略不变。所以，无论是正在运行的事务，还是新事务，如果numsegments没有改变，则查询执行进程依然只运行在集群的旧节点上，新节点上没有工作进程。
对于新创建的表，则会获取最新集群大小，应用到所有节点上，numsegments值与最新集群大小相同。对该表执行的DML操作，也会应用到所有节点上
问题2 扩容期间如果catalog发生变化，则会导致不一致：catalog的变化只应用到了老节点，和新节点上的catalog数据不一致。
为了保证扩容期间catalog的一致性，Greenplum 6引入了catalog锁，在增加新节点阶段，防止对catalog执行修改操作。新节点加入到集群后，该catalog锁被释放。通常添加新节点操作比较快，所以对catalog操作的影响不大。
对于只存在于master上catalog表，如gp_segment_configuration、pg_statistic等，新加入的数据节点会做适当的清理。
3.2.数据重分布
表的重分布过程就是将表的数据从旧的集群节点，重新分布到新的集群节点上。重分布后更新numsegments，之后的所有事务会按照新的分布来处理了。在重分布过程中会对表加ACCESS_EXCLUSIVE_LOCK锁，所有对表的操作都会被阻塞。完成某张表的数据重分布后，操作该表的SQL将被调度到集群的所有新旧节点上执行。
3.3.重分布的优化
哈希分布计算方法虽然简单，但是在重分布的过程中，由于集群大小发生了变化，需要按照新的集群节点数目重新计算哈希，数据几乎要完全重新分布，移动数据量巨大，导致重分布过程效率较低。假设原来有N个节点，每个节点上分布1/N的数据，扩容后M个节点，每个节点上分布1/M的数据。理想的情况是，每个旧节点移动出1/N – 1/M的数据到新节点上，从而整体移动量是N(1/N – 1/M) = 1 – N/M。
Greenplum 6采用了一种叫做Jump Consistent Hash的一致性哈希算法。 该算法有如下特点：
- 均匀性：通过概率做到均匀分布。
- 稳定性：在相同集群大小下，同一个Tuple每次计算结果相同。
- 单调性：扩容过程中，旧节点之间没有数据迁移。
- 高效性：对于集群大小为N的时候，时间复杂度为Log(N)。
3.4.重分布的并行化
在Greenplum 5上，HEAP表的更新删除操作是最高级别的锁，相当于完全是串行化。做数据重分布的过程中，要在一张HEAP表中记录数据库中每个表的重分布状态，所以在做数据重分布的过程中，虽然做了并行化，但是更新状态表的时候仍然是串行的。即使更新状态表的操作很短暂，对于大量小表的并行重分布依然会成为瓶颈。在Greenplum 6里，因为增加了分布式死锁检测，所以更新删除表的锁级别已降低，支持并发更新删除操作。对于大量小表做并行重分布性能提升明显。
3.5.重分布期间对查询性能的影响
重分布过程中，不可避免的会存在某些表已经重分布完毕、某些表还未做重分布。对于处于相同分布状态的表，在分布列上做JOIN查询，不需要做数据重分布；否则，需要进行数据重分布。
3.6.FAQ
- 如何控制哈希分布算法 通过GUC gp_use_legacy_hashops 可以控制，默认是Jump Hash算法。
- 扩容过程中新旧节点的catalog是一致的吗 catalog始终是一致的，新节点上即使还没有数据，但是catalog也是存在的。
- 扩容过程中对于其他查询的影响
- 因为是在线扩容，期间不会中断正在运行的所有查询，只是在增加新节点的时候因为要锁catalog，所以在此期间不支持DDL，加完新节点立即释放catalog锁。
- 在数据重分布的时候因为加了最高级别的锁，对正在重分布的表的访问会阻塞。
3.7.gpexpand的具体过程：
gpexpand工具分两个阶段执行系统扩展：Segment初始化和表重新分布
- 在初始化阶段，gpexpand用一个输入文件运行，该文件指定新Segment的数据目录、 dbid值和其他特征。用户可以手动创建输入文件，也可以在交互式对话中 按照提示进行操作。
- 在表数据重分布阶段，gpexpand会重分布表的数据，使数据在新旧segment 实例之间平衡
要开始重分布阶段，可以通过运行gpexpand并指定-d（运行时间周期） 或-e（结束时间）选项，或者不指定任何选项。如果客户指定了结束时间或运行周期，工具会在 扩展模式下重分布表，直到达到设定的结束时间或执行周期。如果没指定任何选项，工具会继续处理直到扩展模式的表 全部完成重分布。每张表都会通过ALTER TABLE命令来在所有的节点包括新增加的segment实例 上进行重分布，并设置表的分布策略为其原始策略。如果gpexpand完成所有表的重分布，它会 显示成功信息并退出。
greenplum扩容segment可以分为2类：
- 单机上的seg增加，称之为“纵向扩展”
- 单机不变，通过增加机器，称之为“横向扩展”
扩容可以分为纵向扩容和横向扩容，扩容的先决条件如下：
- 用户作为Greenplum数据库超级用户（gpadmin）登录。
- 新的Segment主机已被根据现有的Segment主机安装和配置。这包括：
- 配置硬件和操作系统
- 安装Greenplum软件
- 创建gpadmin用户帐户
- 交换SSH密钥
- 用户的Segment主机上有足够的磁盘空间来临时保存最大表的副本。
- 重新分布数据时，Greenplum数据库必须以生产模式运行。Greenplum数据库不能是受限模式或 Master模式。不能指定gpstart的选项-R或者-m 启动Greenplum数据库
扩容的基本步骤：
- 创建扩容输入文件：gpexpand -f hosts_file
- 初始化Segment并且创建扩容schema：gpexpand -i input_file，gpexpand会创建一个数据目录、从现有的数据库复制表到新的Segment上并且为扩容方案中的每个表捕捉元数据用于状态跟踪。在这个处理完成后，扩容操作会被提交并且不可撤回。
- 重新分布表数据：gpexpand -d duration
- 移除扩容schema：gpexpand -c
3.8.纵向拓展
step 1.查看原Greenplum集群状态
代码语言：javascript复制SELECT * from gp_segment_configuration ;
首先看看现有的集群的状态：
代码语言：javascript复制gpstate
在每台机器上再增加1个 segment节点 ，来扩容 segment 完成纵向扩容
step 2.生成参数文件 指定一个数据库，在此数据库下生成 schema，再创建一个seg_hosts的文件，即所有数据节点的主机名
代码语言：javascript复制cat seg_hosts
gw-sdw01
gw-sdw02
gw-sdw03
执行命令，生成参数文件
代码语言：javascript复制[gpadmin@lgh1 conf]$ gpexpand -f seg_hosts
20210331:14:16:29:015453 gpexpand:lgh1:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.14.1 build commit:5ef30dd4c9878abadc0124e0761e4b988455a4bd'
20210331:14:16:29:015453 gpexpand:lgh1:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.14.1 build commit:5ef30dd4c9878abadc0124e0761e4b988455a4bd) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Feb 22 2021 18:27:08'
20210331:14:16:29:015453 gpexpand:lgh1:gpadmin-[INFO]:-Querying gpexpand schema for current expansion state
System Expansion is used to add segments to an existing GPDB array.
gpexpand did not detect a System Expansion that is in progress.
Before initiating a System Expansion, you need to provision and burn-in
the new hardware. Please be sure to run gpcheckperf to make sure the
new hardware is working properly.
Please refer to the Admin Guide for more information.
Would you like to initiate a new System Expansion Yy|Nn (default=N):
> y
You must now specify a mirroring strategy for the new hosts. Spread mirroring places
a given hosts mirrored segments each on a separate host. You must be
adding more hosts than the number of segments per host to use this.
Grouped mirroring places all of a given hosts segments on a single
mirrored host. You must be adding at least 2 hosts in order to use this.
What type of mirroring strategy would you like?
spread|grouped (default=grouped): #默认的mirror方式
>
** No hostnames were given that do not already exist in the **
** array. Additional segments will be added existing hosts. **
By default, new hosts are configured with the same number of primary
segments as existing hosts. Optionally, you can increase the number
of segments per host.
For example, if existing hosts have two primary segments, entering a value
of 2 will initialize two additional segments on existing hosts, and four
segments on new hosts. In addition, mirror segments will be added for
these new primary segments if mirroring is enabled.
How many new primary segments per host do you want to add? (default=0):
> 1
Enter new primary data directory 1:
> /apps/data2/primary
Enter new mirror data directory 1:
> /apps/data2/mirror
Generating configuration file...
20210331:14:17:05:015453 gpexpand:lgh1:gpadmin-[INFO]:-Generating input file...
Input configuration file was written to 'gpexpand_inputfile_20210331_141705'.
Please review the file and make sure that it is correct then re-run
with: gpexpand -i gpexpand_inputfile_20210331_141705 #生成的初始化文件
20210331:14:17:05:015453 gpexpand:lgh1:gpadmin-[INFO]:-Exiting...
step 3.查看初始化文件：
代码语言：javascript复制[gpadmin@gp-mdw conf]$ cat gpexpand_inputfile_20210331_141705
lgh3|lgh3|6001|/apps/data2/primary/gpseg2|7|2|p
lgh2|lgh2|7001|/apps/data2/mirror/gpseg2|10|2|m
lgh2|lgh2|6001|/apps/data2/primary/gpseg3|8|3|p
lgh3|lgh3|7001|/apps/data2/mirror/gpseg3|9|3|m
step 4.初始化Segment并且创建扩容schema
代码语言：javascript复制gpexpand -i gpexpand_inputfile_20210331_141705
step 5.使用gpstate验证集群状态 step 6.重分布数据
代码语言：javascript复制gpexpand -d 1:00:00
如果在扩容的时候失败或者出错了，记得回滚：gpexpand -r ，还有就是扩容成功，数据重分布成功后记得使用analyze或者analyzedb进行分析
step 7.移除扩容schema
代码语言：javascript复制gpexpand -c
3.9.横向扩容
step 1.在需要扩容的服务器上安装 greenplum step 2.创建新服务器清单文件
代码语言：javascript复制cat > new_host.list
host3
host4
host5
step 3.执行 SSH 交换：
代码语言：javascript复制gpssh-exkeys -f all_hosts
step 4.使用 gpexpand 工具执行系统扩展，Segment初始化，生成新节点配置文件
代码语言：javascript复制gpexpand -f new_host.list
step 5.执行配置文件，增加新节点，并在postgres数据库中创建扩展模式
代码语言：javascript复制gpexpand -i gpexpand_inputfile_20220608_153152
step 6.表重新分布
代码语言：javascript复制gpexpand -d 60:00:00 -a
运行gpexpand 最长持续60小时，以将表重新分布给新的Segment,要根据实际业务情况，尽量在业务空闲时段进行，还可以根据参数选项增加并发和进行analyze等操作
step 7.重分布情况查看及分布完成后清理
代码语言：javascript复制gpstate -x
数据库查看未重分布表总数：
代码语言：javascript复制psql postgres
select * from gpexpand.expansion_progress;
数据库查看未重分布表明细：
代码语言：javascript复制select * from gpexpand_status_detail where status='NOT STARTED';
step 8.移除扩容schema
代码语言：javascript复制gpexpand -c
将gpexpand schema和下面关于扩容的表清理掉
如果在扩容的时候失败或者出错了，记得回滚：gpexpand -r ，还有就是扩容成功，数据重分布成功后记得使用analyze或者analyzedb进行分析
3.3.增加 mirror
4.升级
step 1.准备安装软件 open-source-greenplum-db-6.22.0-rhel7-x86_64.rpm
step 2.安装 greenplum 软件
代码语言：javascript复制rpm -ivh open-source-greenplum-db-6.22.0-rhel7-x86_64.rpm --prefix=/opt
step 3.修改目录权限
代码语言：javascript复制chown -R gpadmin:gpadmin /opt/greenplum-db-6.22.0
step 3.复制Oracle兼容函数及其他需要使用的插件的库文件到 /opt/greenplum-db-6.22.0/lib/postgresql目录下 还有相应的SQL文件复制到/opt/greenplum-db-6.22.0/share/postgresql/extension目录下面
step 4.修改 gpadmin 用户环境变量
代码语言：javascript复制source /usr/local/greenplum-db/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/opt/greenplum/data/master/gpseg-1
export PGPORT=5432
export PGUSER=gpadmin
export PGDATABASE=postgres
export LD_PRELOAD=/lib64/libz.so.1 ps
step 5.重启数据库
代码语言：javascript复制gpstop -M fast -a
source /home/gpadmin/.bashrc
gpstart -a
step 6.查看数据库版本
代码语言：javascript复制postgres=# select version();
version
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------
PostgreSQL 9.4.26 (Greenplum Database 6.22.0 build commit:4b6c079bc3aed35b2f161c377e208185f9310a69 Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 8.5.0 20210514 (Red Hat 8
.5.0-10), 64-bit compiled on Sep 8 2022 22:39:10
(1 row)
postgres=#
