Oracle made big improvements to its diagnosability infrastructure in database version 11g. Every major kernel function and component has been instrumented with the new dbk*/dbg* debug functions and can be traced with ORADEBUG. Before 11g, the previous Oracle diagnostic events infrastructure had much more limited syntax and usage. Very shortly, the new 11g kernel debug & diagnostics infrastructure allows you to be much more precise when tracing & dumping diagnostic info. For example, the below example would dump SQL Trace only when running the SQL with SQLID 32cqz71gd8wy3 below and the pgadep 0 means that only trace when the SQL is executed from top level (directly by application) as opposed to from recursive context (like SQL fired from PL/SQL or data dictionary queries where dep>0 in SQL trace): SQL> alter session set events 'sql_trace[SQL: 32cqz71gd8wy3] {pgadep: exactdepth 0} plan_stat=all_executions,wait=true,bind=true'; Session altered. The plan_stat=all executions above is yet another 11g improvement, which dumps the STAT# lines (execution plan & execution statistics from last execution) at every execution of the cursor as opposed to only when the cursor is closed. For low-level diagnostics, you can even enable tracing/actions only when the current function call stack contains a specific function name (or prefix), like in below case I only trace things happening in the execute phase of the SQL (opiexe stands for Oracle Program Interface Execute): SQL> alter session set events 'sql_trace[SQL: 32cqz71gd8wy3] {pgadep: exactdepth 0} {callstack: fname opiexe} plan_stat=all_executions,wait=true,bind=true';
Session altered. Another example enables kernel tracing (not SQL trace) for all Oracle components which are part of the RDBMS.SQL_Transform component family. The tracing is only done when the SQL statement executed has SQLID 32cqz71gd8wy3. I have enabled tracing to a tracefile (if the components have anything to trace) and additionally I have enabled three additional actions every time any events in SQL_Transform are hit and the SQLID scope is met. Note that this syntax is experimental here, it's not widely used and this particular case below probably dumps a lot of detailed info, which is not useful for everyday regular troubleshooting tasks. I wrote such syntax for illustrating the full power of the new debug infrastructure: SQL> alter session set events 'trace[RDBMS.SQL_Transform] [SQL: 32cqz71gd8wy3] disk=high RDBMS.query_block_dump(1) processstate(1) callstack(1)'; Session altered. Now the optimizer SQL Transformation engine tracing would happen only when executing the SQLID mentioned above and additionally three dumps (query_block_dump,processstate and current call stack) are performed every time this SQLID execution hits events in the SQL_Transform module. Here's the full syntax of what you can use starting from 11g onwards, you can extract all this yourself by using the new command ORADEBUG DOC: There's a new command ORADEBUG DOC for showing all the new syntax that oradebug supports: ORADEBUG DOCSQL> ORADEBUG DOC Internal Documentation ********************** EVENT Help on events (syntax, event list, ...) COMPONENT [<comp_name>] List all components or describe <comp_name> ORADEBUG DOC EVENTThe syntax immediately below shows that it's possible to assign actions (action_list) to events (functions, code locations in Oracle) and make these actions fire only in specific conditions (when event scope (SQL_ID) and filter conditions are matched). SQL> ORADEBUG DOC EVENT Event Help: *********** Formal Event Syntax -------------------- <event_spec> ::= '<event_id> [<event_scope>] [<event_filter_list>] [<event_parameters>] [<action_list>] [off]' <event_id> ::= <event_name | number>[<target_parameters>] <event_scope> ::= [<scope_name>: scope_parameters] <event_filter> ::= {<filter_name>: filter_parameters} <action> ::= <action_name>(action_parameters) <*_parameters> ::= <parameter_name> = <value>[, ] Some Examples ------------- * Set event 10235 level 1: alter session set events '10235'; * Set events SQL_TRACE (a.k.a. 10046) level 1: alter session set events 'sql_trace'; * Turn off event SQL_TRACE: alter session set events 'sql_trace off'; * Set events SQL_TRACE with parameter <plan_stat> set to 'never' and parameter <wait> set to 'true': alter session set events 'sql_trace wait=true, plan_stat=never'; * Trace in-memory the SQL_MONITOR component (the target) and all its sub-components at level high. Get high resolution time for each trace: alter session set events 'trace[sql_mon.*] memory=high, get_time=highres'; * On-disk trace PX servers p000 and p005 for components 'sql_mon' and 'sql_optimizer' (including sub-components) at level highest: alter system set events 'trace[sql_mon | sql_optimizer.*] {process: pname = p000 | p005}'; * Same as above but only when SQL id '7ujay4u33g337' is executed: alter system set events 'trace[sql_mon | sql_optimizer.*] [sql: 7ujay4u33g337] {process: pname = p000 | p005}'; * Execute an action immediatly by using 'immediate' for the event name: alter session set events 'immediate eventdump(system)' * Create an incident labeled 'table_missing' when external error 942 is signaled by process id 14534: alter session set events '942 {process: 14534} incident(table_missing)'; Notes ----- * Implicit parameter level is 1 by default e.g. '10053' is same as '10053 level 1' * Event target (see [<target_parameters>] construct) is only supported by specific events like the TRACE[] event * <event_scope> and/or <event_filter> are constructs that can be used for any event * Same event can be set simultaneously for a different scope or target but not for different filters. * '|' character can be used to select multiple targets, scope or filters. E.g. 'sql_trace [sql: sql_id=g3yc1js3g2689 | sql_id=7ujay4u33g337]' * '=' sign is optional in <*_parameters> E.g. 'sql_trace level 12'; * Like PL/SQL, no need to specify the parameter name for target, scope, filters and action. Resolution is done by position in that case: E.g. 'sql_trace [sql: g3yc1js3g2689 | 7ujay4u33g337]' Help sub-topics --------------- NAME [<event_name>] List all events or describe <event_name> SCOPE [<scope_name>] List all scopes or describe <scope_name> FILTER [<filter_name>] List all filters or describe <filter_name> ACTION [<action_name>] List all actions or describe <action_name> ORADEBUG DOC EVENT NAMESQL> ORADEBUG DOC EVENT NAME Events in library DIAG: ------------------------------ trace[] Main event to control UTS tracing disable_dde_action[] Event used by DDE to disable actions ams_trace[] Event to dump ams performance trace records ams_rowsrc_trace[] Event to dump ams row source tracing sweep_verification Event to enable sweep file verification enable_xml_inc_staging Event to enable xml incident staging format Events in library RDBMS: ------------------------------ alert_text event for textual alerts trace_recursive event to force tracing recursive SQL statements clientid_overwrite event to overwrite client_identifier when client_info is set sql_monitor event to force monitoring SQL statements eventsync_tac Event posted from events syncing tac sql_trace event for sql trace pmon_startup startup of pmon process background_startup startup of background processes db_open_begin start of db open operation Events in library GENERIC: ------------------------------ kg_event[] Support old error number events (use err# for short) Events in library CLIENT: ------------------------------ oci_trace event for oci trace Events in library LIBCELL: ------------------------------ libcell_stat libcell statistics level specification cellclnt_skgxp_trc_ops Controls to trace SKGXP operations cellclnt_high_lat_ops Control to trace High-latency I/O operations Events in library ADVCMP: ------------------------------ arch_comp_level[] arch_comp_level[<ulevel, 1-7>] ccmp_debug columnar compression debug event ccmp_align columnar compression enable alignment ccmp_countstar columnar compression enable count(*) optimization ccmp_dumpunaligned columnar compression dump dbas of unaligned CUs ORADEBUG DOC EVENT NAME <event_name>This syntax allows you to get more information about parameters for some events (especially these ones suffixed with [ ] in above output): SQL> ORADEBUG DOC EVENT NAME sql_trace sql_trace: event for sql trace Usage ------- sql_trace wait < false | true >, bind < false | true >, plan_stat < never | first_execution | all_executions >, level <ub4> SQL> ORADEBUG DOC EVENT NAME trace trace: Main event to control UTS tracing Usage ------- trace [ component <string> ] disk < default | lowest | low | medium | high | highest | disable >, memory < default | lowest | low | medium | high | highest | disable >, get_time < disable | default | seq | highres | seq_highres >, get_stack < disable | default | force >, operation <string>, function <string>, file <string>, line <ub4> SQL> ORADEBUG DOC EVENT NAME kg_event kg_event: Support old error number events (use err# for short) Usage ------- kg_event [ ] level <ub4>, lifetime <ub4>, armcount <ub4>, traceinc <ub4>, forever <ub4> ORADEBUG DOC EVENT SCOPESQL> ORADEBUG DOC EVENT SCOPE Event scopes in library RDBMS: ------------------------------ SQL[] sql scope for RDBMS ORADEBUG DOC EVENT SCOPE SQLThe scope specifier allows you to fire events and trace only when Oracle is executing a specific SQL_ID: SQL> ORADEBUG DOC EVENT SCOPE SQL SQL: sql scope for RDBMS Usage ------- [SQL: sql_id <string> ] ORADEBUG DOC EVENT FILTERThe event filter allows you to fire event actions only when certain event filter conditions are met, like only when a specific (background) process hits the event, when a specific error condition is raised or when a specific function is in the call stack: SQL> ORADEBUG DOC EVENT FILTER Event filters in library DIAG: ------------------------------ occurence filter to implement counting for event checks callstack filter to only fire an event when a function is on the stack tag filter to only fire an event when a tag is set Event filters in library RDBMS: ------------------------------ process filter to set events only for a specific process pgadep filter to only fire an event when the pgadep matches a given value or falls within a range Event filters in library GENERIC: ------------------------------ errarg filter to set error events only for a specific error argument You can get more documentation for each item listed in the above output: SQL> ORADEBUG DOC EVENT FILTER occurence occurence: filter to implement counting for event checks Usage ------- {occurence: start_after <ub4>, end_after <ub4> } SQL> ORADEBUG DOC EVENT FILTER callstack callstack: filter to only fire an event when a function is on the stack Usage ------- {callstack: fname <string>, fprefix <string>, maxdepth <ub4> } SQL> ORADEBUG DOC EVENT FILTER tag tag: filter to only fire an event when a tag is set Usage ------- {tag: tname <string> } SQL> ORADEBUG DOC EVENT FILTER process process: filter to set events only for a specific process Usage ------- {process: ospid <string>, orapid <ub4>, pname <string> } SQL> ORADEBUG DOC EVENT FILTER pgadep pgadep: filter to only fire an event when the pgadep matches a given value or falls within a range Usage ------- {pgadep: exactDepth <ub4>, lessThan <ub4>, greaterThan <ub4> } SQL> ORADEBUG DOC EVENT FILTER errarg errarg: filter to set error events only for a specific error argument Usage ------- {errarg: arg1 <string>, arg2 <string>, arg3 <string>, arg4 <string>, arg5 <string>, arg6 <string>, arg7 <string>, arg8 <string> } ORADEBUG DOC EVENT ACTIONFinally, the action keyword defines what to do when the event has been hit: SQL> ORADEBUG DOC EVENT ACTION Actions in library DIAG: --------------------------- dumpFrameContext - Dump Frame Context contents dumpBuckets kgsfdmp dumpDiagCtx dumpDbgecPopLoc dumpDbgecMarks dumpGenralConfiguration dumpADRLockTable act1 action1 action2 UTDumpGC dbgvci_action_signal_crash Actions in library RDBMS: --------------------------- incident - Create an Incident sqlmon_dump - SQL Monitor Dump SGA Action flashfreeze oradebug - debug process using ORADEBUG debugger - debug process using System Debugger debug - alias for 'debugger' - debug process using System Debugger crash - crash the instance eventdump - list events that are set in the group kdlut_bucketdump_action kzxt_dump_action dumpKernelDiagState HMCHECK (async) DATA_BLOCK_INTEGRITY_CHECK (async) CF_BLOCK_INTEGRITY_CHECK (async) DB_STRUCTURE_INTEGRITY_CHECK (async) REDO_INTEGRITY_CHECK (async) TRANSACTION_INTEGRITY_CHECK (async) SQL_TESTCASE_REC (async) SQL_TESTCASE_REC_DATA (async) ORA_12751_DUMP sqladv_dump_dumpctx ORA_4030_DUMP - dump summary of PGA memory usage, largest allocations kcfis_action - kcfis actions exadata_dump_modvers - Exadata dump module versions QUERY_BLOCK_DUMP - Debug action for dumping a qbcdef tree ASM_MOUNT_FAIL_CHECK (async) ASM_ALLOC_FAIL_CHECK (async) ASM_ADD_DISK_CHECK (async) ASM_FILE_BUSY_CHECK (async) TRACE_BUFFER_ON - Allocate trace output buffer for ksdwrf() TRACE_BUFFER_OFF - Flush and deallocate trace output buffer for ksdwrf() LATCHES - Dump Latches XS_SESSION_STATE - Dump XS session state PROCESSSTATE - Dump process state SYSTEMSTATE - Dump system state INSTANTIATIONSTATE - Dump instantiation state CONTEXTAREA - Dump cursor context area HEAPDUMP - Dump memory heap (1-PGA, 2-SGA, 4-UGA, +1024-Content) POKE_LENGTH - Set length before poking value POKE_VALUE - Poke a value into memory POKE_VALUE0 - Poke 0 value into memory GLOBAL_AREA - Dump fixed global area(s) (1=PGA/2=SGA/3=UGA, add +8 for pointer content) REALFREEDUMP - Dump PGA real free memory allocator state FLUSH_JAVA_POOL - Flush Java pool PGA_DETAIL_GET - Ask process to publish PGA detail info (level is pid) PGA_DETAIL_DUMP - Dump PGA detail information for process (level is pid) PGA_DETAIL_CANCEL - Free PGA detail request (level is pid) PGA_SUMMARY - Summary of PGA memory usage, largest allocations MODIFIED_PARAMETERS - Dump parameters modifed by session (level unused) ERRORSTACK - Dump state (ksedmp). Use INCIDENT action to create incident CALLSTACK - Dump call stack (level > 1 to dump args) RECORD_CALLSTACK - Record or dump call stack, level = #frames (level += 1000000 go to trc) BG_MESSAGES - Dump routine for background messages ENQUEUES - Dump enqueues (level >=2 adds resources, >= 3 adds locks) KSTDUMPCURPROC - Dump current process trace buffer (1 for all events) KSTDUMPALLPROCS - Dump all processes trace buffers (1 for all events) KSTDUMPALLPROCS_CLUSTER - Dump all processes (cluster wide) trace buffers (1 for all events) KSKDUMPTRACE - Dumping KSK KST tracing (no level) DBSCHEDULER - Dump ressource manager state LDAP_USER_DUMP - Dump LDAP user mode LDAP_KERNEL_DUMP - Dump LDAP kernel mode DUMP_ALL_OBJSTATS - Dump database objects statistics DUMPGLOBALDATA - Rolling migration DUMP GLOBAL DATA HANGANALYZE - Hang analyze HANGANALYZE_PROC - Hang analyze current process HANGANALYZE_GLOBAL - Hang analyze system GES_STATE - Dump DML state OCR - OCR client side tracing CSS - CSS client side tracing CRS - CRS client side tracing SYSTEMSTATE_GLOBAL - Perform cluster wide system state dump (via DIAG) DUMP_ALL_COMP_GRANULE_ADDRS - MMAN dump all granule addresses of all components (no level) DUMP_ALL_COMP_GRANULES - MMAN dump all granules of all components (1 for partial list) DUMP_ALL_REQS - MMAN dump all pending memory requests to alert log DUMP_TRANSFER_OPS - MMAN dump transfer and resize operations history DUMP_ADV_SNAPSHOTS - MMAN dump all snapshots of advisories (level unused) CONTROLF - DuMP control file info FLUSH_CACHE - Flush buffer cache without shuting down the instance BUFFERS - Dump all buffers in the buffer cache at level l SET_TSN_P1 - Set tablespace # for buffer dump (level = ts# + 1) BUFFER - Dump all buffers for full relative dba <level> at lvl 10 BC_SANITY_CHECK - Run buffer cache sanity check (level = 0xFF for full) SET_NBLOCKS - Set number of blocks for range reuse checks CHECK_ROREUSE_SANITY - Check range/object reuse sanity (level = ts#) DUMP_PINNED_BUFFER_HISTORY - kcb Dump pinned buffers history (level = # buffers) REDOLOGS - Dump all online logs according to the level LOGHIST - Dump the log history (1: dump earliest/latest entries, >1: dump most recent 2**level entries) REDOHDR - Dump redo log headers LOCKS - Dump every lock element to the trace file GC_ELEMENTS - Dump every lock element to the trace file FILE_HDRS - Dump database file headers KRB_TRACE - Set krb trace options FBINC - Dump flashback logs of the current incarnation and all its ancestors. FBHDR - Dump all the flashback logfile headers FLASHBACK_GEN - Dump flashback generation state KTPR_DEBUG - Parallel txn recovery (1: cleanup check, 2: dump ptr reco ctx, 3: dump recent smon runs) DUMP_TEMP - Dump temp space management state (no level) DROP_SEGMENTS - Drop unused temporary segments TREEDUMP - Dump an index tree rooted at dba BLOCKDBA (<level>) KDLIDMP - Dump 11glob inodes states (level = what to dump) ROW_CACHE - Dump all cache objects LIBRARY_CACHE - Dump the library cache (level > 65535 => level = obj @) CURSORDUMP - Dump session cursors CURSOR_STATS - Dump all statistics information for cursors SHARED_SERVER_STATE - Dump shared server state LISTENER_REGISTRATION - Dump listener registration state JAVAINFO - Dump Oracle Java VM KXFPCLEARSTATS - Clear all Parallel Query messaging statistics KXFPDUMPTRACE - Dump Parallel Query in-memory traces KXFXSLAVESTATE - Dump PX slave state (1: uga; 2: current cursor state; 3: all cursors) KXFXCURSORSTATE - Dump PX slave cursor state WORKAREATAB_DUMP - Dump SQL Memory Manager workarea table OBJECT_CACHE - Dump the object cache SAVEPOINTS - Dump savepoints RULESETDUMP - Dump rule set FAILOVER - Set condition failover immediate OLAP_DUMP - Dump OLAP state AWR_FLUSH_TABLE_ON - Enable flush of table id <level> (ids in X$KEWRTB) AWR_FLUSH_TABLE_OFF - Disable flush of table id <level> (ids in X$KEWRTB) ASHDUMP - Dump ASH data (level = # of minutes) HM_FW_TRACE - DIAG health monitor set tracing level IR_FW_TRACE - DIAG intelligent repair set/clear trace HEAPDUMP_ADDR - Heap dump by address routine (level > 1 dump content) POKE_ADDRESS - Poke specified address (level = value) CURSORTRACE - Trace cursor by hash value (hash value is address) RULESETDUMP_ADDR - Dump rule set by address Actions in library GENERIC: --------------------------- xdb_dump_buckets dumpKGERing - Dump contents of KGE ring buffer Actions in library CLIENT: --------------------------- kpuActionDefault - dump OCI data kpuActionSignalCrash - crash and produce a core dump (if supported and possible) kpudpaActionDpapi - DataPump dump action You can get more details about some actions by running the doc command for the library.action: SQL> ORADEBUG DOC EVENT ACTION RDBMS.query_block_dump QUERY_BLOCK_DUMP - Debug action for dumping a qbcdef tree Usage ------- QUERY_BLOCK_DUMP( level <ub4>) ORADEBUG DOC COMPONENTYou can trace and set actions for Oracle kernel components (if you don't know exact function name which you want to trace). The following command would apply the actions query_block_dump, processstate dump and call stack dump only when executing code in RDBMS.SQL_Transform module (the query transformation engine) and only when the SQL ID executed is 32cqz71gd8wy3. Disk=high allows tracing to disk (if that component is able to trace anything). SQL> alter session set events 'trace[RDBMS.SQL_Transform] [SQL: 32cqz71gd8wy3] disk=high RDBMS.query_block_dump(1) processstate(1) callstack(1)'; Session altered. You can check what sub-components are there under the SQL_Transform component (and you can trace each of these individually if you like): SQL> ORADEBUG DOC COMPONENT RDBMS.SQL_Transform SQL_Transform SQL Transformation (kkq, vop, nso) SQL_MVRW SQL Materialized View Rewrite SQL_VMerge SQL View Merging (kkqvm) SQL_Virtual SQL Virtual Column (qksvc, kkfi) All (registered) components in Oracle kernel can be listed like this: SQL> ORADEBUG DOC COMPONENT |