深入解析pmon

我们在平时处理故障时甚至进行优化时，必须了解一些进程的原理，否则很难定位到根本的原因。在Oracle实例的核心进程中，Pmon进程也是一个极为关键的进程之一，这里我们将逐步揭开pmon进程的神秘面纱！
Pmon是Process Monitor process的简称。

PMON的主要作用：

pmon进程会被定期唤醒，来清理dead process，并释放dead process持有的资源(latch and lock)
回滚dead transaction，前_cleanup_rollback_entries个undo entries
动态注册服务
监控后台进程，如果核心进程crash，pmon负责终止实例
rac服务端负载均衡，PMON进程每3秒会将各自节点的负载及连接数更新到service_register里面

1）通过event 10246和KST跟踪来验证pmon清理dead process以及dead transaction

SYS@LX(lx):1>show parameter event
NAME                                 TYPE       VALUE
------------------------------------ ---------- ------------------------------
_trace_events                        string     10000-10999:255:ALL
event                                string     trace[db_trace] disk=highest,
memory=disable
xml_db_events                        string     enable
SYS@LX(lx):1>alter system set events '10246 trace name context forever, level 1';
System altered.
SQL> delete from test.t where rownum<1000;
999 rows deleted.
SQL> select * from v$lock where type='TX';
ADDR             KADDR                   SID TYPE          ID1        ID2      LMODE    REQUEST      CTIME      BLOCK
---------------- ---------------- ---------- ------ ---------- ---------- ---------- ---------- ---------- ----------
000000008A4C8228 000000008A4C82A0         19 TX         327704       5527          6          0          9          0
SQL> select XIDUSN,XIDSLOT,XIDSQN,XID from v$transaction;
XIDUSN    XIDSLOT     XIDSQN XID
---------- ---------- ---------- ----------------
5         24       5527 0500180097150000
SQL> !kill -9 2868

SYS@LX(lx):1>show parameter event

NAME TYPE VALUE

------------------------------------ ---------- ------------------------------

_trace_events string 10000-10999:255:ALL

event string trace[db_trace] disk=highest,

memory=disable

xml_db_events string enable

SYS@LX(lx):1>alter system set events '10246 trace name context forever, level 1';

System altered.

SQL> delete from test.t where rownum<1000;

999 rows deleted.

SQL> select * from v$lock where type='TX';

ADDR KADDR SID TYPE ID1 ID2 LMODE REQUEST CTIME BLOCK

---------------- ---------------- ---------- ------ ---------- ---------- ---------- ---------- ---------- ----------

000000008A4C8228 000000008A4C82A0 19 TX 327704 5527 6 0 9 0

SQL> select XIDUSN,XIDSLOT,XIDSQN,XID from v$transaction;

XIDUSN XIDSLOT XIDSQN XID

---------- ---------- ---------- ----------------

5 24 5527 0500180097150000

SQL> !kill -9 2868

pmon trace：

*** 2020-11-16 01:05:42.141
marked process 0x918bc910 pid=20 serial=4 ospid = 3224 dead
User session information :
sid: 19 ser: 5
client details:
O/S info: user: oracle, term: pts/2, ospid: 3223
machine: lx program: sqlplus@lx (TNS V1-V3)
application name: sqlplus@lx (TNS V1-V3), hash value=916868158
*** 2020-11-16 01:05:42.142
deleting process 0x918bc910 pid=20 serial=4 priority=0
deleting session 0x91aa3e60 sid=19 serial=5
2020-11-16 01:05:42.142280 :8008BAEC:db_trace:ktur.c@5675:ktugru(): [10444:2:1] next rollback uba: 0x00c0016e.0000.09
...总共100个undo entries
2020-11-16 01:05:42.144516 :8008BB4F:db_trace:ktur.c@5675:ktugru(): [10444:2:1] next rollback uba: 0x00c0016b.0000.0e
ksuprog() called at ktur.c:3160
CLEANUP_ROLLBACK_ENTRIES=100 rollback entries processed
2020-11-16 01:05:42.144551 :8008BB50:db_trace:ksq.c@8542:ksqrcli(): [10704:2:1] ksqrcl: release TX-00050018-00001597 mode=X
2020-11-16 01:05:42.152999 :8008BB51:db_trace:ksl2.c@16103:ksl_update_post_stats(): [10005:2:1] KSL POST SENT postee=13 num=3 loc='ktm.h LINE:614 ID:ktmpsm' id1=0 id2=0 name=   type=0
2020-11-16 01:05:42.153032 :8008BB52:db_trace:ksq.c@8542:ksqrcli(): [10704:2:1] ksqrcl: release TM-00015ab0-00000000 mode=SX
2020-11-16 01:05:42.153050 :8008BB53:db_trace:ksq.c@8828:ksqrcli(): [10704:2:1] ksqrcl: SUCCESS
2020-11-16 01:05:42.153133 :8008BB56:db_trace:ksq.c@8542:ksqrcli(): [10704:2:1] ksqrcl: release AE-00000064-00000000 mode=S
2020-11-16 01:05:42.153147 :8008BB58:db_trace:ksq.c@8828:ksqrcli(): [10704:2:1] ksqrcl: SUCCESS
2020-11-16 01:05:42.153339 :8008BB67:db_trace:ksfd.c@16284:ksfdfods(): [10298:2:1] ksfdfods:fob=0x8f037960 aiopend=0
2020-11-16 01:05:42.153350 :8008BB69:db_trace:kss.c@1418:kssdch(): [10809:2:1] kssdch(0x918bc910 = process, 3) 1 0 exit
deletion of process 918bc910 pid=20 seq=4 successful

*** 2020-11-16 01:05:42.141

marked process 0x918bc910 pid=20 serial=4 ospid = 3224 dead

User session information :

sid: 19 ser: 5

client details:

O/S info: user: oracle, term: pts/2, ospid: 3223

machine: lx program: sqlplus@lx (TNS V1-V3)

application name: sqlplus@lx (TNS V1-V3), hash value=916868158

*** 2020-11-16 01:05:42.142

deleting process 0x918bc910 pid=20 serial=4 priority=0

deleting session 0x91aa3e60 sid=19 serial=5

2020-11-16 01:05:42.142280 :8008BAEC:db_trace:ktur.c@5675:ktugru(): [10444:2:1] next rollback uba: 0x00c0016e.0000.09

...总共100个undo entries

2020-11-16 01:05:42.144516 :8008BB4F:db_trace:ktur.c@5675:ktugru(): [10444:2:1] next rollback uba: 0x00c0016b.0000.0e

ksuprog() called at ktur.c:3160

CLEANUP_ROLLBACK_ENTRIES=100 rollback entries processed

2020-11-16 01:05:42.144551 :8008BB50:db_trace:ksq.c@8542:ksqrcli(): [10704:2:1] ksqrcl: release TX-00050018-00001597 mode=X

2020-11-16 01:05:42.152999 :8008BB51:db_trace:ksl2.c@16103:ksl_update_post_stats(): [10005:2:1] KSL POST SENT postee=13 num=3 loc='ktm.h LINE:614 ID:ktmpsm' id1=0 id2=0 name= type=0

2020-11-16 01:05:42.153032 :8008BB52:db_trace:ksq.c@8542:ksqrcli(): [10704:2:1] ksqrcl: release TM-00015ab0-00000000 mode=SX

2020-11-16 01:05:42.153050 :8008BB53:db_trace:ksq.c@8828:ksqrcli(): [10704:2:1] ksqrcl: SUCCESS

2020-11-16 01:05:42.153133 :8008BB56:db_trace:ksq.c@8542:ksqrcli(): [10704:2:1] ksqrcl: release AE-00000064-00000000 mode=S

2020-11-16 01:05:42.153147 :8008BB58:db_trace:ksq.c@8828:ksqrcli(): [10704:2:1] ksqrcl: SUCCESS

2020-11-16 01:05:42.153339 :8008BB67:db_trace:ksfd.c@16284:ksfdfods(): [10298:2:1] ksfdfods:fob=0x8f037960 aiopend=0

2020-11-16 01:05:42.153350 :8008BB69:db_trace:kss.c@1418:kssdch(): [10809:2:1] kssdch(0x918bc910 = process, 3) 1 0 exit

deletion of process 918bc910 pid=20 seq=4 successful

smon trace：

*** 2020-11-16 01:05:42.153
SMON: system monitor process posted msgflag:0x0008 (-/-/UNDORCV/-/-/-/-)
...
2020-11-16 01:05:42.153251 :8008BB60:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"
2020-11-16 01:05:42.153260 :8008BB61:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS
2020-11-16 01:05:42.153266 :8008BB62:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)
2020-11-16 01:05:42.153288 :8008BB63:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0016b.0000.0d
...
2020-11-16 01:05:42.154273 :8008BBCE:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00168.0000.12
2020-11-16 01:05:42.154283 :8008BBCF:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec
2020-11-16 01:05:42.154289 :8008BBD0:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X
...
2020-11-16 01:05:42.154365 :8008BBDB:db_trace:ktur.c@1727:kturRecoverUndoSegment(): [10444:13:131] UNDO SEG (BEFORE RECOVERY): usn = 5
2020-11-16 01:05:42.154371 :8008BBDC:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"
2020-11-16 01:05:42.154377 :8008BBDD:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS
2020-11-16 01:05:42.154382 :8008BBDE:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)
2020-11-16 01:05:42.154388 :8008BBDF:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00168.0000.11
...
2020-11-16 01:05:42.155271 :8008BC42:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0013d.0000.16
2020-11-16 01:05:42.155280 :8008BC43:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec
2020-11-16 01:05:42.155285 :8008BC44:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X
...
2020-11-16 01:05:42.155356 :8008BC50:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"
2020-11-16 01:05:42.155362 :8008BC51:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS
2020-11-16 01:05:42.155366 :8008BC52:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)
2020-11-16 01:05:42.155372 :8008BC53:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0013d.0000.15
...
2020-11-16 01:05:42.156235 :8008BCB6:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0013a.0000.1d
2020-11-16 01:05:42.156244 :8008BCB7:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec
2020-11-16 01:05:42.156249 :8008BCB8:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X
2020-11-16 01:05:42.156321 :8008BCC4:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"
2020-11-16 01:05:42.156326 :8008BCC5:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS
2020-11-16 01:05:42.156331 :8008BCC6:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)
2020-11-16 01:05:42.156337 :8008BCC7:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0013a.0000.1c
...
2020-11-16 01:05:42.157218 :8008BD2A:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00138.0000.01
2020-11-16 01:05:42.157228 :8008BD2B:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec
2020-11-16 01:05:42.157234 :8008BD2C:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X
2020-11-16 01:05:42.157303 :8008BD38:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"
2020-11-16 01:05:42.157309 :8008BD39:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS
2020-11-16 01:05:42.157313 :8008BD3A:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)
2020-11-16 01:05:42.157319 :8008BD3B:db_trace:ktur.c@5694:ktugru(): [10444:13:131] next rollback uba: 0x00c00137.0000.00
...
2020-11-16 01:05:42.158204 :8008BD9E:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00135.0000.08
2020-11-16 01:05:42.158215 :8008BD9F:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec
2020-11-16 01:05:42.158221 :8008BDA0:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X
2020-11-16 01:05:42.158293 :8008BDAC:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"
2020-11-16 01:05:42.158298 :8008BDAD:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS
2020-11-16 01:05:42.158303 :8008BDAE:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)
2020-11-16 01:05:42.158310 :8008BDAF:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00135.0000.07
...
2020-11-16 01:05:42.159273 :8008BE12:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00132.0000.0d
2020-11-16 01:05:42.159282 :8008BE13:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec
2020-11-16 01:05:42.159287 :8008BE14:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X
...
2020-11-16 01:05:42.159358 :8008BE20:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"
2020-11-16 01:05:42.159364 :8008BE21:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS
2020-11-16 01:05:42.159368 :8008BE22:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)
2020-11-16 01:05:42.159374 :8008BE23:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00132.0000.0c
...
2020-11-16 01:05:42.160339 :8008BE86:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0012f.0000.14
2020-11-16 01:05:42.160348 :8008BE87:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec
2020-11-16 01:05:42.160355 :8008BE88:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X
...
2020-11-16 01:05:42.160427 :8008BE94:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"
2020-11-16 01:05:42.160433 :8008BE95:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS
2020-11-16 01:05:42.160437 :8008BE96:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)
2020-11-16 01:05:42.160443 :8008BE97:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0012f.0000.13
...
2020-11-16 01:05:42.161395 :8008BEFA:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0012c.0000.1d
2020-11-16 01:05:42.161404 :8008BEFB:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec
2020-11-16 01:05:42.161409 :8008BEFC:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X
...
2020-11-16 01:05:42.161478 :8008BF08:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"
2020-11-16 01:05:42.161483 :8008BF09:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS
2020-11-16 01:05:42.161488 :8008BF0A:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)
2020-11-16 01:05:42.161493 :8008BF0B:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0012c.0000.1c
...
2020-11-16 01:05:42.162457 :8008BF6D:db_trace:ktur.c@5694:ktugru(): [10444:13:131] next rollback uba: 0x00000000.0000.00
2020-11-16 01:05:42.162469 :8008BF6E:db_trace:ktur.c@6776:kturend(): [10444:13:131] Fully recovered transaction (5, 24) by (99) rec
2020-11-16 01:05:42.162475 :8008BF6F:db_trace:ktu.c@14251:ktudax(): [10813:13:131] ktudax: dec cnt xid: 5.24.5527 nax 0 nbx 0
2020-11-16 01:05:42.162480 :8008BF70:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X

*** 2020-11-16 01:05:42.153

SMON: system monitor process posted msgflag:0x0008 (-/-/UNDORCV/-/-/-/-)

...

2020-11-16 01:05:42.153251 :8008BB60:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"

2020-11-16 01:05:42.153260 :8008BB61:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS

2020-11-16 01:05:42.153266 :8008BB62:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)

2020-11-16 01:05:42.153288 :8008BB63:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0016b.0000.0d

...

2020-11-16 01:05:42.154273 :8008BBCE:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00168.0000.12

2020-11-16 01:05:42.154283 :8008BBCF:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec

2020-11-16 01:05:42.154289 :8008BBD0:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X

...

2020-11-16 01:05:42.154365 :8008BBDB:db_trace:ktur.c@1727:kturRecoverUndoSegment(): [10444:13:131] UNDO SEG (BEFORE RECOVERY): usn = 5

2020-11-16 01:05:42.154371 :8008BBDC:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"

2020-11-16 01:05:42.154377 :8008BBDD:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS

2020-11-16 01:05:42.154382 :8008BBDE:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)

2020-11-16 01:05:42.154388 :8008BBDF:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00168.0000.11

...

2020-11-16 01:05:42.155271 :8008BC42:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0013d.0000.16

2020-11-16 01:05:42.155280 :8008BC43:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec

2020-11-16 01:05:42.155285 :8008BC44:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X

...

2020-11-16 01:05:42.155356 :8008BC50:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"

2020-11-16 01:05:42.155362 :8008BC51:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS

2020-11-16 01:05:42.155366 :8008BC52:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)

2020-11-16 01:05:42.155372 :8008BC53:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0013d.0000.15

...

2020-11-16 01:05:42.156235 :8008BCB6:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0013a.0000.1d

2020-11-16 01:05:42.156244 :8008BCB7:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec

2020-11-16 01:05:42.156249 :8008BCB8:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X

2020-11-16 01:05:42.156321 :8008BCC4:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"

2020-11-16 01:05:42.156326 :8008BCC5:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS

2020-11-16 01:05:42.156331 :8008BCC6:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)

2020-11-16 01:05:42.156337 :8008BCC7:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0013a.0000.1c

...

2020-11-16 01:05:42.157218 :8008BD2A:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00138.0000.01

2020-11-16 01:05:42.157228 :8008BD2B:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec

2020-11-16 01:05:42.157234 :8008BD2C:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X

2020-11-16 01:05:42.157303 :8008BD38:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"

2020-11-16 01:05:42.157309 :8008BD39:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS

2020-11-16 01:05:42.157313 :8008BD3A:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)

2020-11-16 01:05:42.157319 :8008BD3B:db_trace:ktur.c@5694:ktugru(): [10444:13:131] next rollback uba: 0x00c00137.0000.00

...

2020-11-16 01:05:42.158204 :8008BD9E:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00135.0000.08

2020-11-16 01:05:42.158215 :8008BD9F:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec

2020-11-16 01:05:42.158221 :8008BDA0:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X

2020-11-16 01:05:42.158293 :8008BDAC:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"

2020-11-16 01:05:42.158298 :8008BDAD:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS

2020-11-16 01:05:42.158303 :8008BDAE:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)

2020-11-16 01:05:42.158310 :8008BDAF:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00135.0000.07

...

2020-11-16 01:05:42.159273 :8008BE12:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00132.0000.0d

2020-11-16 01:05:42.159282 :8008BE13:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec

2020-11-16 01:05:42.159287 :8008BE14:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X

...

2020-11-16 01:05:42.159358 :8008BE20:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"

2020-11-16 01:05:42.159364 :8008BE21:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS

2020-11-16 01:05:42.159368 :8008BE22:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)

2020-11-16 01:05:42.159374 :8008BE23:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c00132.0000.0c

...

2020-11-16 01:05:42.160339 :8008BE86:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0012f.0000.14

2020-11-16 01:05:42.160348 :8008BE87:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec

2020-11-16 01:05:42.160355 :8008BE88:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X

...

2020-11-16 01:05:42.160427 :8008BE94:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"

2020-11-16 01:05:42.160433 :8008BE95:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS

2020-11-16 01:05:42.160437 :8008BE96:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)

2020-11-16 01:05:42.160443 :8008BE97:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0012f.0000.13

...

2020-11-16 01:05:42.161395 :8008BEFA:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0012c.0000.1d

2020-11-16 01:05:42.161404 :8008BEFB:db_trace:ktur.c@2359:kturRecoverTxn(): [10444:13:131] Partially recovered transaction (5, 24) by (100) rec

2020-11-16 01:05:42.161409 :8008BEFC:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X

...

2020-11-16 01:05:42.161478 :8008BF08:db_trace:ksq.c@7845:ksqgtlctx(): [10704:13:131] ksqgtl: acquire TX-00050018-00001597 mode=X flags=GLOBAL why="contention"

2020-11-16 01:05:42.161483 :8008BF09:db_trace:ksq.c@8369:ksqgtlctx(): [10704:13:131] ksqgtl: SUCCESS

2020-11-16 01:05:42.161488 :8008BF0A:db_trace:ktur.c@2043:kturRecoverTxn(): [10444:13:131] Recovering transaction (5, 24)

2020-11-16 01:05:42.161493 :8008BF0B:db_trace:ktur.c@5675:ktugru(): [10444:13:131] next rollback uba: 0x00c0012c.0000.1c

...

2020-11-16 01:05:42.162457 :8008BF6D:db_trace:ktur.c@5694:ktugru(): [10444:13:131] next rollback uba: 0x00000000.0000.00

2020-11-16 01:05:42.162469 :8008BF6E:db_trace:ktur.c@6776:kturend(): [10444:13:131] Fully recovered transaction (5, 24) by (99) rec

2020-11-16 01:05:42.162475 :8008BF6F:db_trace:ktu.c@14251:ktudax(): [10813:13:131] ktudax: dec cnt xid: 5.24.5527 nax 0 nbx 0

2020-11-16 01:05:42.162480 :8008BF70:db_trace:ksq.c@8542:ksqrcli(): [10704:13:131] ksqrcl: release TX-00050018-00001597 mode=X

测试总结：

pmon通过轮询方式去检测dead process，轮询间隔为_dead_process_scan_interval，并清理dead process
事务回滚每次回滚_cleanup_rollback_entries个undo entries
pmon会清理dead process持有的资源(lock and latch)
pmon会负责dead transaction的回滚工作，但只负责未超过_cleanup_rollback_entries
undo entries的部分，如果超过则post通知smon(SMON posted for undo segment recovery统计次数会+1），剩下由smon来完成剩余的回滚工作

2）监控后台进程,如果核心进程crash，pmon负责终止实例

Background process SMON found dead
Oracle pid = 13
OS pid (from detached process) = 3012 
OS pid (from process state) = 3012
dtp = 0x60032b10, proc = 0x918b5408
Dump of memory from 0x0000000060032B10 to 0x0000000060032B70
060032B10 000000CC 00000000 918B5408 00000000  [.........T......]
060032B20 00000000 00000000 00000000 4E4F4D53  [............SMON]
060032B30 00000200 00000000 00000BC4 00000000  [................]
060032B40 000FE87E 00000000 8B77B700 00007F79  [~.........w.y...]
060032B50 00000BC4 00000000 000FE87E 00000000  [........~.......]
060032B60 00000001 19A80D4A 00040081 00000000  [....J...........]
Dump of memory from 0x00000000918B5408 to 0x00000000918B64C0
0918B5400                   00000301 00000000          [........]
0918B5410 00000302 00000000 00000000 00000000  [................]
0918B5420 918B5408 00000000 918B5428 00000000  [.T......(T......]
0918B5430 918B5428 00000000 00000000 00000000  [(T..............]
0918B5440 00000000 00000000 00000000 00000000  [................]
Repeat 1 times
0918B5460 0000000E 00000000 90D85918 00000000  [.........Y......]
0918B5470 90F334A8 00000000 914D0EB0 00000000  [.4........M.....]
0918B5480 90F32250 00000000 00000000 00000000  [P"..............]
0918B5490 90F322E8 00000000 90F322E8 00000000  [."......."......]
0918B54A0 90F33488 00000000 00001601 00000010  [.4..............]
0918B54B0 9147A440 00000000 914D0EB0 00000000  [@.G.......M.....]
0918B54C0 0000000D 00000000 00000000 0000000D  [................]
0918B54D0 00000001 009084E4 91EA1C80 00000000  [................]
0918B54E0 00000008 00000000 8FC3B2E0 00000000  [................]
0918B54F0 914D6878 00000000 914D6B70 00000000  [xhM.....pkM.....]
0918B5500 00000002 00000000 00000000 00000000  [................]
0918B5510 90D829D0 00000000 90D85970 00000000  [.)......pY......]
0918B5520 00000000 00000000 00000000 00000000  [................]
Repeat 3 times
0918B5560 00000000 00000000 00060000 00000000  [................]
0918B5570 00000000 00000000 00050000 00000000  [................]
0918B5580 00000000 00000000 001D0000 00000000  [................]
0918B5590 00000000 00000000 00050000 00000000  [................]
Repeat 2 times
0918B55C0 00000000 00000000 000E0000 00000000  [................]
0918B55D0 00000000 00000000 000A0000 00000000  [................]
0918B55E0 00000000 00000000 00060000 00000000  [................]
0918B55F0 00000000 00000000 00000000 00000000  [................]
Repeat 1 times
0918B5610 00000000 00000000 000E0000 00000000  [................]
0918B5620 00000000 00000000 00170000 00000000  [................]
0918B5630 00000000 00000000 00000000 00000000  [................]
0918B5640 00000000 00000000 00040000 00000000  [................]
0918B5650 00000000 00000000 00000000 00000000  [................]
Repeat 5 times
0918B56B0 00000000 00000000 0000012C 00000000  [........,.......]
0918B56C0 00000000 00000000 00000000 00000000  [................]
0918B56D0 918B56D0 00000000 918B56D0 00000000  [.V.......V......]
0918B56E0 00000000 00000000 00000001 00000000  [................]
0918B56F0 918B56F0 00000000 918B56F0 00000000  [.V.......V......]
0918B5700 00000000 00000000 00000000 00000000  [................]
Repeat 3 times
0918B5740 000000E9 0000001A 000000E1 0000000C  [................]
0918B5750 918A9C20 00000000 918B3298 00000000  [ ........2......]
0918B5760 00000003 00000000 00010003 00000000  [................]
0918B5770 00000000 00000000 00000000 00000000  [................]
Repeat 2 times
0918B57A0 00000BC4 00000000 00000000 00000000  [................]
0918B57B0 00000000 00000000 00000000 00000000  [................]
Repeat 2 times
0918B57E0 918B5408 00000000 00000000 00000000  [.T..............]
0918B57F0 00000000 00000000 00000000 00000000  [................]
Repeat 11 times
0918B58B0 918B58B0 00000000 918B58B0 00000000  [.X.......X......]
0918B58C0 00000000 0000018E 00000000 00000000  [................]
0918B58D0 00000000 00000001 00000000 00000000  [................]
0918B58E0 00000000 000D8002 00000011 00000000  [................]
0918B58F0 00000BC4 00000000 000FE87E 00000000  [........~.......]
0918B5900 8B77B700 00007F79 00000BC4 00000000  [..w.y...........]
0918B5910 000FE87E 00000000 00000000 00000000  [~...............]
0918B5920 918B5928 00000000 00000000 00000000  [(Y..............]
0918B5930 00000000 00000000 00000000 00000000  [................]
Repeat 1 times
0918B5950 00000000 00000000 00000038 00000000  [........8.......]
0918B5960 00000006 FFFFFFFF 00000000 00000000  [................]
0918B5970 00000000 00000000 00000000 00000000  [................]
Repeat 12 times
0918B5A40 00000000 00000000 6361726F 0000656C  [........oracle..]
0918B5A50 00000000 00000000 00000000 00000000  [................]
0918B5A60 00000000 00000000 00000006 00000000  [................]
0918B5A70 0000786C 00000000 00000000 00000000  [lx..............]
0918B5A80 00000000 00000000 00000000 00000000  [................]
Repeat 2 times
0918B5AB0 00000002 00000000 00000000 4E4B4E55  [............UNKN]
0918B5AC0 004E574F 00000000 00000000 00000000  [OWN.............]
0918B5AD0 00000000 00000000 00000000 00000000  [................]
0918B5AE0 00000007 00000000 32313033 00000000  [........3012....]
0918B5AF0 00000000 00000000 00000000 00000000  [................]
0918B5B00 00000004 00000000 6361726F 6C40656C  [........oracle@l]
0918B5B10 53282078 294E4F4D 00000000 00000000  [x (SMON)........]
0918B5B20 00000000 00000000 00000000 00000000  [................]
0918B5B30 00000000 00000000 00000010 00000000  [................]
0918B5B40 4E4F4D53 00000000 00000002 00000000  [SMON............]
0918B5B50 00000000 00000000 00000000 00000000  [................]
Repeat 9 times
0918B5BF0 00000000 00000200 00000000 00000000  [................]
0918B5C00 00000000 00000000 00000000 00000000  [................]
0918B5C10 919515E8 00000000 918B6CD0 00000000  [.........l......]
0918B5C20 918B4B60 00000000 00000000 00000000  [`K..............]
0918B5C30 00000000 00000000 91277048 00000000  [........Hp'.....]
0918B5C40 00000000 00000000 00000000 00000000  [................]
Repeat 1 times
0918B5C60 918B5C60 00000000 918B5C60 00000000  [`\......`\......]
0918B5C70 00270000 00000000 00170000 00000000  [..'.............]
0918B5C80 0009C53E 00000000 00111156 00000000  [>.......V.......]
0918B5C90 0030C53E 00000000 000101D8 00000000  [>.0.............]
0918B5CA0 00000000 00000000 00197B98 00000000  [.........{......]
0918B5CB0 000007D8 00000000 000000E0 00000000  [................]
0918B5CC0 000007D8 00000000 00000000 00000000  [................]
0918B5CD0 00000000 00000000 00000000 00000000  [................]
Repeat 2 times
0918B5D00 00000000 00000000 0D000100 084EE000  [..............N.]
0918B5D10 00000000 01000000 00000000 00000000  [................]
0918B5D20 00000000 00000000 00000000 00000000  [................]
Repeat 1 times
0918B5D40 00000000 00000003 00080000 00000000  [................]
0918B5D50 00000000 00000000 00000000 00000000  [................]
Repeat 113 times
0918B6470 918B6470 00000000 918B6470 00000000  [pd......pd......]
0918B6480 00000000 00000000 918B6488 00000000  [.........d......]
0918B6490 918B6488 00000000 00000000 00000000  [.d..............]
0918B64A0 00000000 00000000 00000000 00000000  [................]
Repeat 1 times
kjzduptcctx: Notifying DIAG for crash event
----- Abridged Call Stack Trace -----
ksedsts()+465<-kjzdssdmp()+267<-kjzduptcctx()+232<-kjzdicrshnfy()+63<-ksuitm()+5570<-ksulhdcb()+489<-ksucln()+7265<-ksbrdp()+1045<-opirip()+623<-opidrv()+603<-sou2o()+103<-opimai_real()+250<-ssthrdmain()+265<
-main()+201<-__libc_start_main()+253 
----- End of Abridged Call Stack Trace -----
*** 2020-11-16 01:58:22.507
PMON (ospid: 2988): terminating the instance due to error 474

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

Background process SMON found dead

Oracle pid = 13

OS pid (from detached process) = 3012

OS pid (from process state) = 3012

dtp = 0x60032b10, proc = 0x918b5408

Dump of memory from 0x0000000060032B10 to 0x0000000060032B70

060032B10 000000CC 00000000 918B5408 00000000 [.........T......]

060032B20 00000000 00000000 00000000 4E4F4D53 [............SMON]

060032B30 00000200 00000000 00000BC4 00000000 [................]

060032B40 000FE87E 00000000 8B77B700 00007F79 [~.........w.y...]

060032B50 00000BC4 00000000 000FE87E 00000000 [........~.......]

060032B60 00000001 19A80D4A 00040081 00000000 [....J...........]

Dump of memory from 0x00000000918B5408 to 0x00000000918B64C0

0918B5400 00000301 00000000 [........]

0918B5410 00000302 00000000 00000000 00000000 [................]

0918B5420 918B5408 00000000 918B5428 00000000 [.T......(T......]

0918B5430 918B5428 00000000 00000000 00000000 [(T..............]

0918B5440 00000000 00000000 00000000 00000000 [................]

Repeat 1 times

0918B5460 0000000E 00000000 90D85918 00000000 [.........Y......]

0918B5470 90F334A8 00000000 914D0EB0 00000000 [.4........M.....]

0918B5480 90F32250 00000000 00000000 00000000 [P"..............]

0918B5490 90F322E8 00000000 90F322E8 00000000 [."......."......]

0918B54A0 90F33488 00000000 00001601 00000010 [.4..............]

0918B54B0 9147A440 00000000 914D0EB0 00000000 [@.G.......M.....]

0918B54C0 0000000D 00000000 00000000 0000000D [................]

0918B54D0 00000001 009084E4 91EA1C80 00000000 [................]

0918B54E0 00000008 00000000 8FC3B2E0 00000000 [................]

0918B54F0 914D6878 00000000 914D6B70 00000000 [xhM.....pkM.....]

0918B5500 00000002 00000000 00000000 00000000 [................]

0918B5510 90D829D0 00000000 90D85970 00000000 [.)......pY......]

0918B5520 00000000 00000000 00000000 00000000 [................]

Repeat 3 times

0918B5560 00000000 00000000 00060000 00000000 [................]

0918B5570 00000000 00000000 00050000 00000000 [................]

0918B5580 00000000 00000000 001D0000 00000000 [................]

0918B5590 00000000 00000000 00050000 00000000 [................]

Repeat 2 times

0918B55C0 00000000 00000000 000E0000 00000000 [................]

0918B55D0 00000000 00000000 000A0000 00000000 [................]

0918B55E0 00000000 00000000 00060000 00000000 [................]

0918B55F0 00000000 00000000 00000000 00000000 [................]

Repeat 1 times

0918B5610 00000000 00000000 000E0000 00000000 [................]

0918B5620 00000000 00000000 00170000 00000000 [................]

0918B5630 00000000 00000000 00000000 00000000 [................]

0918B5640 00000000 00000000 00040000 00000000 [................]

0918B5650 00000000 00000000 00000000 00000000 [................]

Repeat 5 times

0918B56B0 00000000 00000000 0000012C 00000000 [........,.......]

0918B56C0 00000000 00000000 00000000 00000000 [................]

0918B56D0 918B56D0 00000000 918B56D0 00000000 [.V.......V......]

0918B56E0 00000000 00000000 00000001 00000000 [................]

0918B56F0 918B56F0 00000000 918B56F0 00000000 [.V.......V......]

0918B5700 00000000 00000000 00000000 00000000 [................]

Repeat 3 times

0918B5740 000000E9 0000001A 000000E1 0000000C [................]

0918B5750 918A9C20 00000000 918B3298 00000000 [ ........2......]

0918B5760 00000003 00000000 00010003 00000000 [................]

0918B5770 00000000 00000000 00000000 00000000 [................]

Repeat 2 times

0918B57A0 00000BC4 00000000 00000000 00000000 [................]

0918B57B0 00000000 00000000 00000000 00000000 [................]

Repeat 2 times

0918B57E0 918B5408 00000000 00000000 00000000 [.T..............]

0918B57F0 00000000 00000000 00000000 00000000 [................]

Repeat 11 times

0918B58B0 918B58B0 00000000 918B58B0 00000000 [.X.......X......]

0918B58C0 00000000 0000018E 00000000 00000000 [................]

0918B58D0 00000000 00000001 00000000 00000000 [................]

0918B58E0 00000000 000D8002 00000011 00000000 [................]

0918B58F0 00000BC4 00000000 000FE87E 00000000 [........~.......]

0918B5900 8B77B700 00007F79 00000BC4 00000000 [..w.y...........]

0918B5910 000FE87E 00000000 00000000 00000000 [~...............]

0918B5920 918B5928 00000000 00000000 00000000 [(Y..............]

0918B5930 00000000 00000000 00000000 00000000 [................]

Repeat 1 times

0918B5950 00000000 00000000 00000038 00000000 [........8.......]

0918B5960 00000006 FFFFFFFF 00000000 00000000 [................]

0918B5970 00000000 00000000 00000000 00000000 [................]

Repeat 12 times

0918B5A40 00000000 00000000 6361726F 0000656C [........oracle..]

0918B5A50 00000000 00000000 00000000 00000000 [................]

0918B5A60 00000000 00000000 00000006 00000000 [................]

0918B5A70 0000786C 00000000 00000000 00000000 [lx..............]

0918B5A80 00000000 00000000 00000000 00000000 [................]

Repeat 2 times

0918B5AB0 00000002 00000000 00000000 4E4B4E55 [............UNKN]

0918B5AC0 004E574F 00000000 00000000 00000000 [OWN.............]

0918B5AD0 00000000 00000000 00000000 00000000 [................]

0918B5AE0 00000007 00000000 32313033 00000000 [........3012....]

0918B5AF0 00000000 00000000 00000000 00000000 [................]

0918B5B00 00000004 00000000 6361726F 6C40656C [........oracle@l]

0918B5B10 53282078 294E4F4D 00000000 00000000 [x (SMON)........]

0918B5B20 00000000 00000000 00000000 00000000 [................]

0918B5B30 00000000 00000000 00000010 00000000 [................]

0918B5B40 4E4F4D53 00000000 00000002 00000000 [SMON............]

0918B5B50 00000000 00000000 00000000 00000000 [................]

Repeat 9 times

0918B5BF0 00000000 00000200 00000000 00000000 [................]

0918B5C00 00000000 00000000 00000000 00000000 [................]

0918B5C10 919515E8 00000000 918B6CD0 00000000 [.........l......]

0918B5C20 918B4B60 00000000 00000000 00000000 [`K..............]

0918B5C30 00000000 00000000 91277048 00000000 [........Hp'.....]

0918B5C40 00000000 00000000 00000000 00000000 [................]

Repeat 1 times

0918B5C60 918B5C60 00000000 918B5C60 00000000 [`\......`\......]

0918B5C70 00270000 00000000 00170000 00000000 [..'.............]

0918B5C80 0009C53E 00000000 00111156 00000000 [>.......V.......]

0918B5C90 0030C53E 00000000 000101D8 00000000 [>.0.............]

0918B5CA0 00000000 00000000 00197B98 00000000 [.........{......]

0918B5CB0 000007D8 00000000 000000E0 00000000 [................]

0918B5CC0 000007D8 00000000 00000000 00000000 [................]

0918B5CD0 00000000 00000000 00000000 00000000 [................]

Repeat 2 times

0918B5D00 00000000 00000000 0D000100 084EE000 [..............N.]

0918B5D10 00000000 01000000 00000000 00000000 [................]

0918B5D20 00000000 00000000 00000000 00000000 [................]

Repeat 1 times

0918B5D40 00000000 00000003 00080000 00000000 [................]

0918B5D50 00000000 00000000 00000000 00000000 [................]

Repeat 113 times

0918B6470 918B6470 00000000 918B6470 00000000 [pd......pd......]

0918B6480 00000000 00000000 918B6488 00000000 [.........d......]

0918B6490 918B6488 00000000 00000000 00000000 [.d..............]

0918B64A0 00000000 00000000 00000000 00000000 [................]

Repeat 1 times

kjzduptcctx: Notifying DIAG for crash event

----- Abridged Call Stack Trace -----

ksedsts()+465<-kjzdssdmp()+267<-kjzduptcctx()+232<-kjzdicrshnfy()+63<-ksuitm()+5570<-ksulhdcb()+489<-ksucln()+7265<-ksbrdp()+1045<-opirip()+623<-opidrv()+603<-sou2o()+103<-opimai_real()+250<-ssthrdmain()+265<

-main()+201<-__libc_start_main()+253

----- End of Abridged Call Stack Trace -----

*** 2020-11-16 01:58:22.507

PMON (ospid: 2988): terminating the instance due to error 474

哪些后台进程crash，实例会down掉？

SQL> SELECT indx,ksuprpnm,TO_CHAR(ksuprflg,'XXXXXXXXXXXXXXXX')
2    FROM x$ksupr
3    WHERE BITAND(ksuprflg,4) = 4 ORDER BY indx
4    /
INDX KSUPRPNM                       TO_CHAR(KSUPRFLG,'XXXXXXXXXXXXXXXX')
---------- ------------------------------ ---------------------------------------------------
2 oracle@lx (PMON)                               E
3 oracle@lx (PSP0)                               6
4 oracle@lx (VKTM)                               6
5 oracle@lx (GEN0)                               6
7 oracle@lx (DBRM)                               6
9 oracle@lx (MMAN)                               6
10 oracle@lx (DBW0)                               6
11 oracle@lx (LGWR)                               6
12 oracle@lx (CKPT)                               6
13 oracle@lx (SMON)                              16
10 rows selected.

SQL> SELECT indx,ksuprpnm,TO_CHAR(ksuprflg,'XXXXXXXXXXXXXXXX')

2 FROM x$ksupr

3 WHERE BITAND(ksuprflg,4) = 4 ORDER BY indx

4 /

INDX KSUPRPNM TO_CHAR(KSUPRFLG,'XXXXXXXXXXXXXXXX')

---------- ------------------------------ ---------------------------------------------------

2 oracle@lx (PMON) E

3 oracle@lx (PSP0) 6

4 oracle@lx (VKTM) 6

5 oracle@lx (GEN0) 6

7 oracle@lx (DBRM) 6

9 oracle@lx (MMAN) 6

10 oracle@lx (DBW0) 6

11 oracle@lx (LGWR) 6

12 oracle@lx (CKPT) 6

13 oracle@lx (SMON) 16

10 rows selected.

3）动态注册服务

可以参考mos文档How to Trace Dynamic Registration from PMON ? [ID 787055.1]

Enable Oracle Net Server tracing, adding following to SQLNET.ORA.

DIAG_ADR_ENABLED=off # Disable ADR if database version 11g TRACE_LEVEL_SERVER = 16 # Enable level 16 trace TRACE_DIRECTORY_SERVER = <DIRECTORY> # Control trace file location

1
2
3

DIAG_ADR_ENABLED=off # Disable ADR if database version 11g
TRACE_LEVEL_SERVER = 16 # Enable level 16 trace
TRACE_DIRECTORY_SERVER = <DIRECTORY> # Control trace file location

Add to the listener.ora file the following and reload the listener via listener control

DIAG_ADR_ENABLED_<listener Name> =off # Disable ADR if version 11g
TRACE_LEVEL_<Listener Name> = 16 # Enable level 16 trace
TRACE_TIMESTAMP_<Listener Name> = ON # Set timestamp in the trace files
TRACE_DIRECTORY_<Listener Name> = <Directory> # Control trace file location

DIAG_ADR_ENABLED_<listener Name> =off # Disable ADR if version 11g

TRACE_LEVEL_<Listener Name> = 16 # Enable level 16 trace

TRACE_TIMESTAMP_<Listener Name> = ON # Set timestamp in the trace files

TRACE_DIRECTORY_<Listener Name> = <Directory> # Control trace file location

Set event for PMON

oradebug setospid <PMON_SPID> oradebug Event 10257 trace name context forever, level 16；

1
2

oradebug setospid <PMON_SPID>
oradebug Event 10257 trace name context forever, level 16；

发表回复取消回复

近期文章

近期评论

深入解析pmon

发表回复 取消回复

近期文章

近期评论

发表回复取消回复