实战经验：Oracle DG 的归档缺失修复

2021-05-31 15:26:50 浏览数 (1)

客户某天反馈说：”DG库自0221以来就已经不同步了，请核查。“

于是我远程登录进行查看。

故障检查

检查归档同步情况

一、查看数据库的情

代码语言：javascript复制
select database_role,flashback_on,open_mode,current_scn from v$database

DATABASE_ROLE   FLASHBACK_ON      OPEN_MODE         CURRENT_SCN
---------------- ------------------ -------------------- ---------------
PHYSICAL STANDBY NO        READ ONLY WITH APPLY  16657544972059

二、查看归档的最大线程与最大接收的归档情况。

代码语言：javascript复制
select thread#,max(sequence#) from v$archived_log group by thread#;

生产库：
SQL> select thread#,max(sequence#) from v$archived_log group by thread#;
   THREAD# MAX(SEQUENCE#)
---------- --------------
   1     136973
   2     132693
   4     149599
   3     133277
--DG库
SYS@hisnewdb> select thread#,max(sequence#) from v$archived_log group by thread#;

   THREAD# MAX(SEQUENCE#)
---------- --------------
   1     136973
   2     132693
   4     149598
   3     133277
  • 可见4个节点归档是都有会过来的，sequence都能对得上。

三、查是否存在GAP

代码语言：javascript复制
select * from v$archived_gap;

日志应用情况

查看延时的应用情况

代码语言：javascript复制
select name ,value,time_computed from v$dataguard_stats where rownum<33;

NAME         VALUE                  TIME_COMPUTED
-------------------------------- ---------------------------------------------------------------- ------------------------------
transport lag        11 06:41:27                03/04/2021 16:41:20
apply lag        11 06:41:27                03/04/2021 16:41:20
apply finish time      00 04:23:39.868              03/04/2021 16:41:20
estimated startup time     37                  03/04/2021 16:41:20
  • 可看到apply lag的应用已经延时11天6小时了。
  • apply finish time应用最快的恢复时长为4小时。

恢复思路

应用日志

代码语言：javascript复制
alter database recover managed standby databse cancel; --取消应用日志
alter database open read only;  --打开只读库
alter database recover managed standby ; 
alter database recover managed standby disconnect from session; -- 后台应用，建议上面命令，放前台应用。

归档还保留或者GAP较少的情况

1）归档还在主库

方法一：

首先通过备库sql查出相应的 node[thread#] 和归档位置 name：

传输上面文件到备库归档位置 archive log list

备库上注册归档文件alter database register logfile '归档文件绝对路径' 或rman注册日志catalog start with '';

应用日志，查看select * from V$ARCHIVE_GAP;,监视是否还存在其它的GAP出现。如出现，如上面步骤循环操作。

方法二：

  1. 在配置 fal_client={备库的监听} 和 fal_server={主库的监听}
  2. 直接应用日志，由备库寻找日志。

2）归档已在备库

应用日志

归档已经被删除或GAP较多的情况

查看归档所在的位置

alert.log日志：

  • 提供等thread 线程4的序列为148164的归档，获取的序号有148164-148165
  • control_keep_record_keep_time是控制文件的重用记录数据。提示在这个记录天数内没找到归档文件，建议设置更长些天数。以便GAP找到缺失的日志。
    • 默认7天，1-365天范围。
    • 记录的是归档日志，各种备份记录。
    • 不记录数据文件，表空间，redo thread记录。除非被drop，否则不会重用这部分记录
代码语言：javascript复制
started logmerger process
Thu Mar 04 16:19:53 2021
Managed Standby Recovery not using Real Time Apply
Parallel Media Recovery started with 16 slaves
Waiting for all non-current ORLs to be archived...
All non-current ORLs have been archived.
Media Recovery Waiting for thread 4 sequence 148164
Fetching gap sequence in thread 4, gap sequence 148164-148165
Thu Mar 04 16:19:57 2021
Completed:  alter database recover managed standby database disconnect from session
----------
Thu Mar 04 16:21:50 2021
FAL[client]: Failed to request gap sequence
 GAP - thread 4 sequence 148164-148165
 DBID 3828421454 branch 984679630
FAL[client]: All defined FAL servers have been attempted.
------------------------------------------------------------
Check that the CONTROL_FILE_RECORD_KEEP_TIME initialization
parameter is defined to a value that's sufficiently large
enough to maintain adequate log switch information to resolve
archivelog gaps.
------------------------------------------------------------
 Thu Mar 04 16:22:25 2021
RFS[18]: Selected log 29 for thread 4 sequence 149600 dbid -466545842 branch 984679630
Thu Mar 04 16:22:25 2021

1）找到当前的最小SCN

对比数据文件最后检查点的scn，数据文件头部检查点的scn，缺失归档的对应scn（下个日志文件第一个更改号），当前数据库的scn：

代码语言：javascript复制
select thread#,low_sequence#,high_sequence# from v$archive_gap;
 
col datafile_scn for 999999999999999
col DATAFILE_HEADER_SCN for 999999999999999
col current_scn for 999999999999999
col  next_change# for 999999999999999
select ( select min(d.checkpoint_change#) from v$datafile d ) datafile_scn ,
  ( select min(d.checkpoint_change#) from v$datafile_header d where rownum=1) datafile_header_scn,
  (select current_scn from v$database) current_scn,
  (select next_change# from v$archived_log where sequence#=148164 and resetlogs_change# = (select d.resetlogs_change# from v$database d ) and rownum=1 ) next_change#
from dual;

DATAFILE_SCN     DATAFILE_HEADER_SCN      CURRENT_SCN     NEXT_CHANGE#
---------------- ------------------- ---------------- ----------------
  16657544969028      16657544972060   16657544972059
  • 取上面最小的scn作为增量备份的SCN

2）主库做SCN增量备份

停用备库的日志应用

代码语言：javascript复制
alter database recover managed standby database cancel;

rman备份

  1. 切换日志
  2. 切记备份当前控制文件
  3. 增量scn备份
代码语言：javascript复制
run {
allocate channel c1 device type disk; 
allocate channel c2 device type disk; 
allocate channel c3 device type disk; 
allocate channel c4 device type disk; 
allocate channel c5 device type disk; 
allocate channel c6 device type disk; 
CONFIGURE DEVICE TYPE DISK PARALLELISM 6 BACKUP TYPE TO BACKUPSET;
backup as compressed backupset current controlfile for standby format '/home/oracle/backup/backup_ctl_%U.rman';
backup as compressed backupset incremental from scn 16657544969028 database format '/home/oracle/backup/backup_%d_%s_%c_%U_%T.rman' include 
current controlfile for standby filesperset 10 tag 'forsdb_16657544969028_0304';
release  channel c1 ; 
release  channel c2 ; 
release  channel c3 ; 
release  channel c4 ; 
release  channel c5 ; 
release  channel c6 ;

传输备份文件到备库

代码语言：javascript复制
scp -rp /home/oracle/backup/backup host2:/home/oracle

介质恢复备库

  1. 查出控制文件的绝对目录位置，后停备库
  2. 启动到nomount
  3. 恢复控制文件
  4. 启动到mount
  5. 恢复数据文件
  6. 检查rman进展
代码语言：javascript复制
select name from v$controlfile;
shu immediate;
startup nomount;
代码语言：javascript复制
rman target / <<eof
restore standby controlfile from '/home/oracle/backup/backup_ctl_%U.rman';
alter database mount; 
eof
  • 如果没有单独备份standby controlfile，就一个一个文件来测试恢复standby controflie
代码语言：javascript复制
restore standby controlfile to '/oradata/hisnewdb/control01.ctl' from  '/home/oracle/backup/某个文件';
  • 如果文件太多，可以先rman注册文件后，再恢复控制文件。
    • 要找开备库mount状态才能注册
代码语言：javascript复制
rmant target / <<eof
startup mount; 
catalog start with '/home/oracle/backup/';
list backup of controlfile;
restore standby controlfile automatic;
eof
#大概是这样。restore standby controlfile automatic;如果不通，就采用上面list的信息，找到具体含有standby controflile的备份文件，再通过restore standby controfile from '';来恢复 。
代码语言：javascript复制
catalog start with '/home/oracle/backup/';
recover database noredo;

查看rman的恢复进展：

代码语言：javascript复制
set line 9999
select sid,serial#,opname,round(sofar/totalwork*100) completed,trunc(elapsed_seconds/60) elapsed ,trunc(time_remaining/60) remaining,context ,target,sofar,totalwork
from v$session_longops
where opname like 'RMAN%' and opname not like '%aggregate%' and totalwork!=0 and sofar<>totalwork;

应用日志

检查standby redo files是否存在:

代码语言：javascript复制
select * from v$standby_log;

注册standby redolog files

代码语言：javascript复制
-- 添加单个文件：
alter database add standby logfile group {组号} 'standby redo logs files 绝对目录文件';
-- 添加多个standby redologs file
alter database add standby logfile group {组号} ('standby redo logs file 1','logfiles2');

应用日志

代码语言：javascript复制
alter database recover managed standby database cancel ; 
startup mount; 
alter database open read only;
select open_mode,status,protection_level,protection_mode from v$database ;
--前台应用日志
alter database recover managed standby database ; 
-- 8 parallel 后台应用日志
alter database recover managed standby database parallel 8 disconnect from session;

检查应用日志的情况

检查各个线程thread#的最大应用日志的序列，与主库进行对比。

代码语言：javascript复制
select thread#,max(sequence#) from v$archived_log where applied='YES' group by thread#;

墨天轮原文链接：https://www.modb.pro/db/46707（复制到浏览器或者点击“阅读原文”立即查看）

apply thread time 备份 日志

0 人点赞