0 概述与总结
- hot更新已经有几篇分析了，这里是最后一篇（总结性的，前面的可以忽略）。
- 前面在看update代码时，大部分集中在heap_update上，没有涉及寻找HOT链的逻辑。本篇重点看HOT链是如何使用的。
（总结速查）
（lp=line pointer：页面中等宽数组，每一个指向页面底部的数据区域）
关键步骤总结（no vacuum场景）：
- HOT链的头部元素的lp始终保存，索引始终指向这个lp（即使这个lp指向行更新了也只是把数据删了，保留lp指针）
- HOT链的中间元素都带HEAP_HOT_UPDATED标记，HOT链的最后一个元素只有HEAP_ONLY_TUPLE标记。
- HOT链在更新时有三个关键步骤：
- 1 走索引找到链头lp：table_index_fetch_tuple（下文3.1）
- 2 遍历HOT链确定需要的lp：heap_hot_search_buffer（下文3.2）
- 3 碎片整理，使数据区域更紧凑，会更新lp的指向位置：compactify_tuples（下文3.3）
- 4 使用找到的lp获取页面位置，memcopy数据上去完成update：heap_update（下文3.4）
堆栈：
代码语言：javascript复制ExecModifyTable
ExecProcNode // ExecProcNode = 0x783005 <ExecIndexScan>
| ExecIndexScan
| ExecScan
| ExecScanFetch
| IndexNext
| index_getnext_slot
| tid = index_getnext_tid // 3.1 总是拿到ip_posid = 130
| index_fetch_heap
| table_index_fetch_tuple(ItemPointer tid) // {ip_posid = 130}
| heapam_index_fetch_tuple
| heap_hot_search_buffer // 3.2 遍历HOT链找旧元组
| heap_page_prune_opt // 3.3 碎片整理
| heap_page_prune
| heap_page_prune_execute
| PageRepairFragmentation
| compactify_tuples
ExecUpdate
ExecUpdateAct
table_tuple_update
heapam_tuple_update
heap_update // 3.4 更新1 分析用例
照常先给出分析用例。
代码语言：javascript复制-- 测试表，单页可以放136条数据
drop table testbl;
create table testbl(i int primary key not null, id int not null, info varchar(200) not null);
alter table testbl set (autovacuum_enabled = false);
insert into testbl select generate_series(1,130), (random()*100)::integer, repeat('DUfw',(5)::integer);
select * from testbl limit 10;
i | id | info
---- ---- ----------------------
1 | 57 | DUfwDUfwDUfwDUfwDUfw
2 | 2 | DUfwDUfwDUfwDUfwDUfw
3 | 29 | DUfwDUfwDUfwDUfwDUfw
4 | 37 | DUfwDUfwDUfwDUfwDUfw
5 | 2 | DUfwDUfwDUfwDUfwDUfw
6 | 44 | DUfwDUfwDUfwDUfwDUfw
7 | 53 | DUfwDUfwDUfwDUfwDUfw
8 | 24 | DUfwDUfwDUfwDUfwDUfw
9 | 49 | DUfwDUfwDUfwDUfwDUfw
10 | 17 | DUfwDUfwDUfwDUfwDUfw2 HOT更新实验
2.1 总结：等宽更新
- HOT多次更新后，发现HOT链会复用元组，并不会一直延长。
- HOT根节点（本例中的lp=130）不变，复用后续的节点。
- 复用节点带
HEAP_ONLY_TUPLE | HEAP_HOT_UPDATED标记。
UPDATE | lp=130 | lp=131 | lp=132 |
|---|---|---|---|
第一次 | 原tuple：HEAP_HOT_UPDATED | 新tuple：HEAP_ONLY_TUPLE | |
第二次 | 原tuple：HEAP_ONLY_TUPLE HEAP_HOT_UPDATED | 新tuple：HEAP_ONLY_TUPLE | |
第三次 | 新tuple：HEAP_ONLY_TUPLE | 原tuple：HEAP_ONLY_TUPLE HEAP_HOT_UPDATED | |
第四次 | 原tuple：HEAP_ONLY_TUPLE HEAP_HOT_UPDATED | 新tuple：HEAP_ONLY_TUPLE | |
第五次 | 新tuple：HEAP_ONLY_TUPLE | 原tuple：HEAP_ONLY_TUPLE HEAP_HOT_UPDATED |
数组区域状态（等宽更新，对应上表）
注意uppdate都是使用lp有效的位置，用之前先做碎片整理，把有效的向下移动，填充到删除的地方。然后再insert。
2.1 总结：不等宽更新
数组区域状态（不等宽更新）
注意第四次更新和第五次更新，新数据更宽了，可以明显看到碎片整理的过程：
- 第五次更新时，先把132的数据向下移动到888-967；然后再对132的数据进行更新；更新后132被删除；131被复用，放在了页面的upper指针 数据大小的位置。
实验：更新前xid=8169
代码语言：javascript复制SELECT lp, lp_off, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask FROM heap_page_items(get_raw_page('testbl', 0));
lp | lp_off | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask
----- -------- -------- -------- ---------- --------- ------------- ------------
130 | 912 | 8169 | 0 | 0 | (0,130) | 3 | 2050
============
t_infomask
============
2050(0x802) = HEAP_XMAX_INVALID | HEAP_HASVARWIDTH
============
t_infomask2(11 bits for number of attributes)
============
number_of_ = 3实验：第一次更新xid=8170
代码语言：javascript复制update testbl set info = 'DDDDDDDDDDDDDDDDDDD1' where i = 130;
SELECT lp, lp_off, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask FROM heap_page_items(get_raw_page('testbl', 0));
lp | lp_off | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask
----- -------- -------- -------- ---------- --------- ------------- ------------
130 | 912 | 8169 | 8170 | 0 | (0,131) | 16387 | 258
131 | 856 | 8170 | 0 | 0 | (0,131) | 32771 | 10242
============
t_infomask
============
258(0x102) = HEAP_XMIN_COMMITTED | HEAP_HASVARWIDTH
10242(0x2802) = HEAP_UPDATED | HEAP_XMAX_INVALID | HEAP_HASVARWIDTH
============
t_infomask2
============
16387(0x4003) = HEAP_HOT_UPDATED | 3attributes
32771(0x8003) = HEAP_ONLY_TUPLE | 3attributes实验：第二次更新xid=8171
代码语言：javascript复制update testbl set info = 'DDDDDDDDDDDDDDDDDDD2' where i = 130;
SELECT lp, lp_off, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask FROM heap_page_items(get_raw_page('testbl', 0));
lp | lp_off | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask
----- -------- -------- -------- ---------- --------- ------------- ------------
130 | 131 | | | | | |
131 | 912 | 8170 | 8171 | 0 | (0,132) | 49155 | 8450
132 | 856 | 8171 | 0 | 0 | (0,132) | 32771 | 10242
============
t_infomask
============
8450(0x2102) = HEAP_UPDATED | HEAP_XMIN_COMMITTED | HEAP_HASVARWIDTH
10242(0x2802) = HEAP_UPDATED | HEAP_XMAX_INVALID | HEAP_HASVARWIDTH
============
t_infomask2
============
49155(0xC003) = HEAP_ONLY_TUPLE | HEAP_HOT_UPDATED | 3attributes
32771(0x8003) = HEAP_ONLY_TUPLE | 3attributes实验：第三次更新xid=8172
代码语言：javascript复制update testbl set info = 'DDDDDDDDDDDDDDDDDDD3' where i = 130;
SELECT lp, lp_off, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask FROM heap_page_items(get_raw_page('testbl', 0));
lp | lp_off | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask
----- -------- -------- -------- ---------- --------- ------------- ------------
130 | 132 | | | | | |
131 | 856 | 8172 | 0 | 0 | (0,131) | 32771 | 10242
132 | 912 | 8171 | 8172 | 0 | (0,131) | 49155 | 8450
============
t_infomask
============
10242(0x2802) = HEAP_UPDATED | HEAP_XMAX_INVALID | HEAP_HASVARWIDTH
8450(0x2102) = HEAP_UPDATED | HEAP_XMIN_COMMITTED | HEAP_HASVARWIDTH
============
t_infomask2
============
49155(0xC003) = HEAP_ONLY_TUPLE | HEAP_HOT_UPDATED | 3attributes
32771(0x8003) = HEAP_ONLY_TUPLE | 3attributes实验：第四次更新xid=8173
代码语言：javascript复制update testbl set info = 'DDDDDDDDDDDDDDDDDDD4' where i = 130;
SELECT lp, lp_off, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask FROM heap_page_items(get_raw_page('testbl', 0));
lp | lp_off | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask
----- -------- -------- -------- ---------- --------- ------------- ------------
130 | 131 | | | | | |
131 | 912 | 8172 | 8173 | 0 | (0,132) | 49155 | 8450
132 | 856 | 8173 | 0 | 0 | (0,132) | 32771 | 10242
============
t_infomask
============
8450(0x2102) = HEAP_UPDATED | HEAP_XMIN_COMMITTED | HEAP_HASVARWIDTH
10242(0x2802) = HEAP_UPDATED | HEAP_XMAX_INVALID | HEAP_HASVARWIDTH
============
t_infomask2
============
49155(0xC003) = HEAP_ONLY_TUPLE | HEAP_HOT_UPDATED | 3attributes
32771(0x8003) = HEAP_ONLY_TUPLE | 3attributes实验：第五次更新xid=8174
代码语言：javascript复制update testbl set info = 'DDDDDDDDDDDDDDDDDDD5' where i = 130;
SELECT lp, lp_off, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask FROM heap_page_items(get_raw_page('testbl', 0));
lp | lp_off | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask
----- -------- -------- -------- ---------- --------- ------------- ------------
130 | 132 | | | | | |
131 | 856 | 8174 | 0 | 0 | (0,131) | 32771 | 10242
132 | 912 | 8173 | 8174 | 0 | (0,131) | 49155 | 8450
============
t_infomask
============
10242(0x2802) = HEAP_UPDATED | HEAP_XMAX_INVALID | HEAP_HASVARWIDTH
8450(0x2102) = HEAP_UPDATED | HEAP_XMIN_COMMITTED | HEAP_HASVARWIDTH
============
t_infomask2
============
49155(0xC003) = HEAP_ONLY_TUPLE | HEAP_HOT_UPDATED | 3attributes
32771(0x8003) = HEAP_ONLY_TUPLE | 3attributes3 场景分析：lp=130空、lp=131删除、lp=132有效
- 当前HOT链：130（重定向）—>132（有效）
- 预期发生：碎片整理：132准备转移到131的位置；新增数据到132原来的位置、用131指向新增数据。
3.1 索引扫描
从顶层ExecModifyTable进入索引扫描部分，因为关掉了VAUUM，索引总是返回130：index_getnext_tid
代码语言：javascript复制ExecModifyTable
ExecProcNode // ExecProcNode = 0x783005 <ExecIndexScan>
ExecIndexScan
ExecScan
ExecScanFetch
IndexNext
index_getnext_slot
tid = index_getnext_tid // 3.1 总是拿到ip_posid = 130 <<<<
index_fetch_heap
table_index_fetch_tuple(ItemPointer tid) // {ip_posid = 130}
heapam_index_fetch_tuple
heap_hot_search_buffer // 3.2 遍历HOT链找旧元组
heap_page_prune_opt // 3.3 碎片整理3.2 遍历HOT链找旧元组
再用130去找元组：heap_hot_search_buffer
代码语言：javascript复制bool
heap_hot_search_buffer(
ItemPointer tid, // {ip_blkid = {bi_hi = 0, bi_lo = 0}, ip_posid = 130}
Relation relation, Buffer buffer,
Snapshot snapshot, HeapTuple heapTuple,
bool *all_dead, bool first_call)
{
...
...
/* Scan through possible multiple members of HOT-chain */
for (;;)
{
ItemId lp;
/* check for bogus TID */
if (offnum < FirstOffsetNumber || offnum > PageGetMaxOffsetNumber(dp))
break;【第一轮循环读到HOT链130，找到132】
这里比较重要，offnum现在还是130，但是lp拿出来就直接是132了：
ItemIdData = {lp_off = 132, lp_flags = 2, lp_len = 0}
这里的lp_flags=2表示LP_REDIRECT，重定向到132。
#define LP_UNUSED 0 /* unused (should always have lp_len=0) */
#define LP_NORMAL 1 /* used (should always have lp_len>0) */
#define LP_REDIRECT 2 /* HOT redirect (should have lp_len=0) */
#define LP_DEAD 3 /* dead, may or may not have storage */
【第二轮循环读到HOT链132，找到数据位置】
第二次进入循环后，offnum=132拿到的lp：
ItemIdData = {lp_off = 912, lp_flags = 1, lp_len = 53}（LP_NORMAL）
现在的912就是指向数据区域了。
代码语言：javascript复制 lp = PageGetItemId(dp, offnum);
/* check for unused, dead, or redirected items */
if (!ItemIdIsNormal(lp))
{
/* We should only see a redirect at start of chain */
if (ItemIdIsRedirected(lp) && at_chain_start)
{
/* Follow the redirect */
offnum = ItemIdGetRedirect(lp);使用lp=132拿到offnum=132，继续下一轮循环继续遍历HOT链。
代码语言：javascript复制 at_chain_start = false;
continue;
}
/* else must be end of chain */
break;
}遍历完HOT链出循环，开始读132的数据部分，拼接元组。
代码语言：javascript复制 heapTuple->t_data = (HeapTupleHeader) PageGetItem(dp, lp);
heapTuple->t_len = ItemIdGetLength(lp);
heapTuple->t_tableOid = RelationGetRelid(relation);
ItemPointerSet(&heapTuple->t_self, blkno, offnum);
if (!skip)
{
/* If it's visible per the snapshot, we must return it */
valid = HeapTupleSatisfiesVisibility(heapTuple, snapshot, buffer);
HeapCheckForSerializableConflictOut(valid, relation, heapTuple,
buffer, snapshot);
if (valid)
{
ItemPointerSetOffsetNumber(tid, offnum);
PredicateLockTID(relation, &heapTuple->t_self, snapshot,
HeapTupleHeaderGetXmin(heapTuple->t_data));
if (all_dead)
*all_dead = false;找到132返回。
代码语言：javascript复制 return true;
}
}
skip = false;3.3 碎片整理
将132移动到131的位置上，因为131删掉已经是空洞了。
这里就不展开分析了。记录下函数堆栈。
代码语言：javascript复制ExecModifyTable
ExecProcNode // ExecProcNode = 0x783005 <ExecIndexScan>
ExecIndexScan
ExecScan
ExecScanFetch
IndexNext
index_getnext_slot
tid = index_getnext_tid // 总是拿到ip_posid = 130 <<<<
index_fetch_heap
table_index_fetch_tuple(ItemPointer tid) // {ip_posid = 130}
heapam_index_fetch_tuple
heap_hot_search_buffer // 3.2 遍历HOT链找旧元组
heap_page_prune_opt // 3.3 碎片整理
heap_page_prune
heap_page_prune_execute
PageRepairFragmentation
compactify_tuples3.4 开始更新heap_update
代码语言：javascript复制ExecModifyTable
ExecProcNode // ExecProcNode = 0x783005 <ExecIndexScan>
| ExecIndexScan
| ExecScan
| ExecScanFetch
| IndexNext
| index_getnext_slot
| tid = index_getnext_tid // 总是拿到ip_posid = 130 <<<<
| index_fetch_heap
| table_index_fetch_tuple(ItemPointer tid) // {ip_posid = 130}
| heapam_index_fetch_tuple
| heap_hot_search_buffer // 3.2 遍历HOT链找旧元组
| heap_page_prune_opt // 3.3 碎片整理
| heap_page_prune
| heap_page_prune_execute
| PageRepairFragmentation
| compactify_tuples
ExecUpdate
ExecUpdateAct
table_tuple_update
heapam_tuple_update
heap_update1 拿到数据地址
ItemId = {lp_off = 912, lp_flags = 1, lp_len = 53}
lp = PageGetItemId(page, ItemPointerGetOffsetNumber(otid));2 拼接旧元组：
HeapTupleData = {t_len = 53, t_self = {ip_blkid = {bi_hi = 0, bi_lo = 0}, ip_posid = 132}, t_tableOid = 32946, t_data = 0x2aaaab4ae610}
3 新元组位置未确定：
HeapTupleData = {t_len = 53, t_self = {ip_blkid = {bi_hi = 65535, bi_lo = 65535}, ip_posid = 0}, t_tableOid = 32946, t_data = 0x26fe440}
oldtup.t_tableOid = RelationGetRelid(relation);
oldtup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
oldtup.t_len = ItemIdGetLength(lp);
oldtup.t_self = *otid;
newtup->t_tableOid = RelationGetRelid(relation);4 判断旧元组可见性
result = TM_Ok
result = HeapTupleSatisfiesUpdate(&oldtup, cid, buffer);5 同页更新使用use_hot_update
旧的加：HEAP_HOT_UPDATED 新的加：HEAP_ONLY_TUPLE
代码语言：javascript复制1: *newtup = {t_len = 53, t_self = {ip_blkid = {bi_hi = 65535, bi_lo = 65535}, ip_posid = 0}, t_tableOid = 32946, t_data = 0x26fe440}
3: oldtup = {t_len = 53, t_self = {ip_blkid = {bi_hi = 0, bi_lo = 0}, ip_posid = 132}, t_tableOid = 32946, t_data = 0x2aaaab4ae610}6 insert插入新tuple
RelationPutHeapTuple执行insert操作，需要先找到插入位置。
代码语言：javascript复制RelationPutHeapTuple
PageAddItem核心逻辑（重要）
- 正向遍历itemid，所以，数据区域是反向遍历的。
- 找到空闲的lp131，然后132经过碎片整理下移到131的位置上了，原来132的位置在最上面空出来了，申请这个位置使用。
for (offsetNumber = FirstOffsetNumber;
offsetNumber < limit; /* limit is maxoff 1 */
offsetNumber )
{
itemId = PageGetItemId(phdr, offsetNumber);
/*
* We check for no storage as well, just to be paranoid;
* unused items should never have storage. Assert() that the
* invariant is respected too.
*/
Assert(ItemIdIsUsed(itemId) || !ItemIdHasStorage(itemId));
if (!ItemIdIsUsed(itemId) && !ItemIdHasStorage(itemId))
break;
}
