爱可生开源社区出品,原创内容未经授权不得随意使用,转载请联系小编并注明来源。
本文约 1300 字,预计阅读需要 5 分钟。
1背景
某客户基于节约资源的想法,将日志盘设置的比较小,日志盘大小约为集群内存规格的 1.5 倍,当创建租户时,CPU 和内存都充足的情况下,却存在报错 "LOG_DISK resource not enough",我们尝试复现问题并定位原因。
2环境信息
- 架构:单节点集群
- 版本:OceanBase:4.2.1.4
MySQL [oceanbase]> select svr_ip,status,build_version from __all_server;
-------------- -------- -------------------------------------------------------------------------------------------
| svr_ip | status | build_version |
-------------- -------- -------------------------------------------------------------------------------------------
| 10.186.64.61 | ACTIVE | 4.2.1.4_104010012024030714-c4f3400ad2839e337bc9dab5d1bfe1d01134a1d7(Mar 7 2024 14:32:22) |
-------------- -------- -------------------------------------------------------------------------------------------
1 row in set (0.01 sec)
查看集群可分配的 CPU、内存、日志盘容量。
- MEM_CAPACITY:observer 进程可用的内存大小
- LOG_DISK_CAPACITY:日志盘空间总大小,41.8GB
- LOG_DISK_ASSIGNED:日志盘已分配大小,6GB
- 剩余可分配 CPU 数量:18-2=16C
- 剩余可分配内存大小:24-2=22GB
MySQL [oceanbase]> SELECT SVR_IP, SVR_PORT , ZONE , SQL_PORT , CPU_CAPACITY , CPU_CAPACITY_MAX, CPU_ASSIGNED , CPU_ASSIGNED_MAX ,MEMORY_LIMIT/1024/1024/1024 MEMORY_LIMIT_GB , MEM_CAPACITY/1024/1024/1024 MEM_CAPACITY_GB, MEM_ASSIGNED/1024/1024/1024 MEM_ASSIGNED_GB ,LOG_DISK_CAPACITY/1024/1024/1024 LOG_DISK_CAPACITY_GB,LOG_DISK_ASSIGNED/1024/1024/1024 LOG_DISK_ASSIGNED_GB,LOG_DISK_IN_USE/1024/1024/1024 LOG_DISK_IN_USE_GB FROM GV$OB_SERVERS;
-------------- ---------- ------- ---------- -------------- ------------------ -------------- ------------------ ----------------- ----------------- ----------------- ---------------------- ---------------------- --------------------
| SVR_IP | SVR_PORT | ZONE | SQL_PORT | CPU_CAPACITY | CPU_CAPACITY_MAX | CPU_ASSIGNED | CPU_ASSIGNED_MAX | MEMORY_LIMIT_GB | MEM_CAPACITY_GB | MEM_ASSIGNED_GB | LOG_DISK_CAPACITY_GB | LOG_DISK_ASSIGNED_GB | LOG_DISK_IN_USE_GB |
-------------- ---------- ------- ---------- -------------- ------------------ -------------- ------------------ ----------------- ----------------- ----------------- ---------------------- ---------------------- --------------------
| 10.186.64.61 | 2882 | zone1 | 2881 | 18 | 18 | 2 | 2 | 30.000000000000 | 24.000000000000 | 2.000000000000 | 41.875000000000 | 6.000000000000 | 0.125000000000 |
-------------- ---------- ------- ---------- -------------- ------------------ -------------- ------------------ ----------------- ----------------- ----------------- ---------------------- ---------------------- --------------------
1 row in set (0.00 sec)
可以看到当前集群仅 sys
租户占用了 2C2G 的资源。
MySQL [oceanbase]> SELECT a.tenant_name,a.tenant_id,b.name unit_config,c.name pool_name,b.max_cpu,b.min_cpu,MEMORY_SIZE/1024/1024/1024 as MEMORY_SIZE
-> FROM
-> OCEANBASE.DBA_OB_TENANTS a,
-> OCEANBASE.DBA_OB_UNIT_CONFIGS b,
-> OCEANBASE.DBA_OB_RESOURCE_POOLS c
-> WHERE a.tenant_id=c.tenant_id
-> AND b.unit_config_id = c.unit_config_id
-> ORDER BY a.tenant_id desc;
------------- ----------- ------------------------------- ----------- --------- --------- ----------------
| tenant_name | tenant_id | unit_config | pool_name | max_cpu | min_cpu | MEMORY_SIZE |
------------- ----------- ------------------------------- ----------- --------- --------- ----------------
| sys | 1 | config_sys_zone1_twoctwog_xio | sys_pool | 2 | 2 | 2.000000000000 |
------------- ----------- ------------------------------- ----------- --------- --------- ----------------
1 row in set (0.01 sec)
3报错复现及疑问
创建资源单元
尝试创建 1 个规格为 4C12G 的资源单元。
代码语言:javascript复制MySQL [oceanbase]> CREATE RESOURCE UNIT mem_test_unit MEMORY_SIZE = '12G',MAX_CPU = 4, MIN_CPU = 4;
Query OK, 0 rows affected (0.02 sec)
创建资源池
存在报错:LOG_DISK resource not enough
MySQL [oceanbase]> CREATE RESOURCE POOL pool_evan UNIT='mem_test_unit', UNIT_NUM=1, ZONE_LIST=('zone1');
ERROR 4733 (HY000): zone 'zone1' resource not enough to hold 1 unit. You can check resource info by views: DBA_OB_UNITS, GV$OB_UNITS, GV$OB_SERVERS.
server '"10.186.64.61:2882"' LOG_DISK resource not enough
问题 1
剩余资源为 16C 22GB,为何创建一个 4C 12GB 的资源池会失败?
问题 2
报错有关 LOG_DISK ,日志盘容量与内存规格存在何种关联?
4日志盘大小与租户内存大小的关系
尝试新建规格为 1C1G 的租户,分析租户内存大小与日志盘容量分配的规律。
- 新建规格为 1C1G 的租户。
- LOG_DISK_ASSIGNED_GB 增长至 9G,相较之前增加了 3G。
MySQL [oceanbase]> CREATE RESOURCE UNIT unit_1g MEMORY_SIZE = '1G',MAX_CPU = 1, MIN_CPU = 1;
Query OK, 0 rows affected (0.01 sec)
MySQL [oceanbase]> CREATE RESOURCE POOL pool_1g UNIT='unit_1g', UNIT_NUM=1, ZONE_LIST=('zone1');
Query OK, 0 rows affected (0.02 sec)
MySQL [oceanbase]> CREATE TENANT IF NOT EXISTS tenant_1g
-> PRIMARY_ZONE = 'zone1',
-> RESOURCE_POOL_LIST=('pool_1g')
-> set OB_TCP_INVITED_NODES='%';
Query OK, 0 rows affected (26.05 sec)
MySQL [oceanbase]>
MySQL [oceanbase]> SELECT SVR_IP, SVR_PORT , ZONE , SQL_PORT , CPU_CAPACITY ,CPU_ASSIGNED ,MEMORY_LIMIT/1024/1024/1024 MEMORY_LIMIT_GB , MEM_CAPACITY/1024/1024/1024 MEM_CAPACITY_GB, MEM_ASSIGNED/1024/1024/1024 MEM_ASSIGNED_GB ,LOG_DISK_CAPACITY/1024/1024/1024 LOG_DISK_CAPACITY_GB,LOG_DISK_ASSIGNED/1024/1024/1024 LOG_DISK_ASSIGNED_GB FROM GV$OB_SERVERS;
-------------- ---------- ------- ---------- -------------- -------------- ----------------- ----------------- ----------------- ---------------------- ----------------------
| SVR_IP | SVR_PORT | ZONE | SQL_PORT | CPU_CAPACITY | CPU_ASSIGNED | MEMORY_LIMIT_GB | MEM_CAPACITY_GB | MEM_ASSIGNED_GB | LOG_DISK_CAPACITY_GB | LOG_DISK_ASSIGNED_GB |
-------------- ---------- ------- ---------- -------------- -------------- ----------------- ----------------- ----------------- ---------------------- ----------------------
| 10.186.64.61 | 2882 | zone1 | 2881 | 18 | 3 | 30.000000000000 | 24.000000000000 | 3.000000000000 | 41.875000000000 | 9.000000000000 |
-------------- ---------- ------- ---------- -------------- -------------- ----------------- ----------------- ----------------- ---------------------- ----------------------
1 row in set (0.00 sec)
小结
每为租户分配 1GB 内存,则对应的分配 3GB 日志盘容量。
5代入本文初始环境计算
计算公式
本文初始背景集群剩余内存规格计算:
(LOG_DISK_CAPACITY_GB - LOG_DISK_ASSIGNED_GB) / 3 =(41.875 - 6.00)/ 3 ≈ 11.958 GB
即,集群剩余可用的内存上限为 11.958GB,取整数为 11GB。
6验证
释放资源
删掉租户 tenant_1g
和对应的 RESOURCE POOL 释放资源。
MySQL [oceanbase]> DROP TENANT tenant_1g;
Query OK, 0 rows affected (35.04 sec)
MySQL [oceanbase]> DROP RESOURCE POOL pool_1g;
Query OK, 0 rows affected (0.01 sec)
MySQL [oceanbase]> SELECT SVR_IP, SVR_PORT , ZONE , SQL_PORT , CPU_CAPACITY ,CPU_ASSIGNED ,MEMORY_LIMIT/1024/1024/1024 MEMORY_LIMIT_GB , MEM_CAPACITY/1024/1024/1024 MEM_CAPACITY_GB, MEM_ASSIGNED/1024/1024/1024 MEM_ASSIGNED_GB ,LOG_DISK_CAPACITY/1024/1024/1024 LOG_DISK_CAPACITY_GB,LOG_DISK_ASSIGNED/1024/1024/1024 LOG_DISK_ASSIGNED_GB FROM GV$OB_SERVERS;
-------------- ---------- ------- ---------- -------------- -------------- ----------------- ----------------- ----------------- ---------------------- ----------------------
| SVR_IP | SVR_PORT | ZONE | SQL_PORT | CPU_CAPACITY | CPU_ASSIGNED | MEMORY_LIMIT_GB | MEM_CAPACITY_GB | MEM_ASSIGNED_GB | LOG_DISK_CAPACITY_GB | LOG_DISK_ASSIGNED_GB |
-------------- ---------- ------- ---------- -------------- -------------- ----------------- ----------------- ----------------- ---------------------- ----------------------
| 10.186.64.61 | 2882 | zone1 | 2881 | 18 | 2 | 30.000000000000 | 24.000000000000 | 2.000000000000 | 41.875000000000 | 6.000000000000 |
-------------- ---------- ------- ---------- -------------- -------------- ----------------- ----------------- ----------------- ---------------------- ----------------------
1 row in set (0.00 sec)
重新创建
创建一个规格为 16C11G 的租户。
代码语言:javascript复制MySQL [oceanbase]> CREATE RESOURCE UNIT unit_11g MEMORY_SIZE = '11G',MAX_CPU = 16, MIN_CPU = 16;
Query OK, 0 rows affected (0.01 sec)
MySQL [oceanbase]> CREATE RESOURCE POOL pool_11g UNIT='unit_11g', UNIT_NUM=1, ZONE_LIST=('zone1');
Query OK, 0 rows affected (0.02 sec)
MySQL [oceanbase]> CREATE TENANT IF NOT EXISTS tenant_11g
-> PRIMARY_ZONE = 'zone1',
-> RESOURCE_POOL_LIST=('pool_11g')
-> set OB_TCP_INVITED_NODES='%';
Query OK, 0 rows affected (25.99 sec)
MySQL [oceanbase]> SELECT SVR_IP, SVR_PORT , ZONE , SQL_PORT , CPU_CAPACITY ,CPU_ASSIGNED ,MEMORY_LIMIT/1024/1024/1024 MEMORY_LIMIT_GB , MEM_CAPACITY/1024/1024/1024 MEM_CAPACITY_GB, MEM_ASSIGNED/1024/1024/1024 MEM_ASSIGNED_GB ,LOG_DISK_CAPACITY/1024/1024/1024 LOG_DISK_CAPACITY_GB,LOG_DISK_ASSIGNED/1024/1024/1024 LOG_DISK_ASSIGNED_GB FROM GV$OB_SERVERS;
-------------- ---------- ------- ---------- -------------- -------------- ----------------- ----------------- ----------------- ---------------------- ----------------------
| SVR_IP | SVR_PORT | ZONE | SQL_PORT | CPU_CAPACITY | CPU_ASSIGNED | MEMORY_LIMIT_GB | MEM_CAPACITY_GB | MEM_ASSIGNED_GB | LOG_DISK_CAPACITY_GB | LOG_DISK_ASSIGNED_GB |
-------------- ---------- ------- ---------- -------------- -------------- ----------------- ----------------- ----------------- ---------------------- ----------------------
| 10.186.64.61 | 2882 | zone1 | 2881 | 18 | 18 | 30.000000000000 | 24.000000000000 | 13.000000000000 | 41.875000000000 | 39.000000000000 |
-------------- ---------- ------- ---------- -------------- -------------- ----------------- ----------------- ----------------- ---------------------- ----------------------
1 row in set (0.01 sec)
7疑问解答
下面我们来解答上面提出的两个疑问。
问题 1
剩余资源为 16C 22GB,为何创建一个 4C 12GB 的资源池会失败?
答:
- 由于日志盘规格为 41.875GB,且
sys
租户已占用 6GB 日志盘份额,经前文计算可得:集群剩余可用的内存上限为 11GB。 - 建租户内存规格超过 11GB 会因申请不到对应份额的日志盘容量而引发报错
LOG_DISK resource not enough
。
问题 2
报错有关 LOG_DISK ,日志盘容量与内存规格存在何种关联?
答:LOG_DISK_SIZE:默认值为内存规格值的 3 倍,最小值为 2G。
8建议
日志盘大小尽量设置为内存上限的 3 或 4 倍(生产环境至少是 3 倍),避免因日志盘不足导致集群已有的内存无法进行分配。
参考资料
- 《GV$OB_SERVERS》:https://www.oceanbase.com/docs/common-oceanbase-database-cn-1000000001051773
- 《ALTER RESOURCE UNIT》:https://www.oceanbase.com/docs/common-oceanbase-database-cn-1000000000220301
本文关键字:#OceanBase# #日志盘# #资源池#