SELECT sql_id, executions, elapsed_time/1000000 elapsed_sec, elapsed_time/1000/NULLIF(executions,0) ms_per_exec, cpu_time/1000000 cpu_sec, buffer_gets, disk_reads, rows_processed FROM v$sql WHERE sql_id = '0bujgc94rg3fj';
And find whether it is executing right now:
SELECT s.sid, s.serial#, s.username, s.status, s.event, s.wait_class, s.seconds_in_wait, s.sql_id, s.module, s.action, s.program, s.machine FROM v$session s WHERE s.sql_id = '0bujgc94rg3fj';What is this SQL waiting on when it becomes slow?
That is where ASH/AWR becomes important.
Run this first:
SELECT sql_id, plan_hash_value, executions, elapsed_time/1000000 elapsed_sec, cpu_time/1000000 cpu_sec, buffer_gets, disk_reads, rows_processed FROM v$sql WHERE sql_id = '0bujgc94rg3fj';Then check whether there are multiple child cursors:
SELECT child_number, plan_hash_value, executions, elapsed_time/1000000 elapsed_sec, cpu_time/1000000 cpu_sec, buffer_gets, disk_reads, loads, invalidations, parse_calls FROM v$sql WHERE sql_id = '0bujgc94rg3fj' ORDER BY child_number;More importantly, use ASH:
SELECT event, wait_class, session_state, COUNT(*) samples FROM v$active_session_history WHERE sql_id = '0bujgc94rg3fj' GROUP BY event, wait_class, session_state ORDER BY samples DESC;If Diagnostic Pack/AWR is available, check historical ASH:
SELECT event, wait_class, session_state, COUNT(*) samples FROM dba_hist_active_sess_history WHERE sql_id = '0bujgc94rg3fj' GROUP BY event, wait_class, session_state ORDER BY samples DESC;Also check SQL performance over time:
SELECT sn.begin_interval_time, ss.plan_hash_value, ss.executions_delta, ROUND(ss.elapsed_time_delta/1000000,2) elapsed_sec, ROUND( ss.elapsed_time_delta / NULLIF(ss.executions_delta,0) / 1000, 2 ) ms_per_exec, ss.buffer_gets_delta, ss.disk_reads_delta FROM dba_hist_sqlstat ss, dba_hist_snapshot sn WHERE ss.snap_id = sn.snap_id AND ss.dbid = sn.dbid AND ss.instance_number = sn.instance_number AND ss.sql_id = '0bujgc94rg3fj' ORDER BY sn.begin_interval_time DESC;
++++++++++++++SELECT NVL(sql_id,'NO_SQL_ID') sql_id, NVL(event,'ON CPU') event, wait_class, session_state, COUNT(*) samples FROM dba_hist_active_sess_history WHERE top_level_sql_id = '0bujgc94rg3fj' GROUP BY sql_id, event, wait_class, session_state ORDER BY samples DESC;This is probably the most valuable query at this point.
It may reveal something like:
TOP LEVEL WF_EVENT.LISTEN | +--> SQL A against WF_EVENT_SUBSCRIPTIONS | +--> SQL B against WF_EVENTS | +--> SQL C against AQ queue | +--> SQL D ...Then we can identify which one is burning the CPU and logical reads.
Also run this version
This will give us the internal SQL IDs ranked by activity:
SELECT sql_id, COUNT(*) samples, SUM(CASE WHEN session_state = 'ON CPU' THEN 1 ELSE 0 END) cpu_samples, SUM(CASE WHEN session_state = 'WAITING' THEN 1 ELSE 0 END) wait_samples FROM dba_hist_active_sess_history WHERE top_level_sql_id = '0bujgc94rg3fj' GROUP BY sql_id ORDER BY samples DESC;Send me that result.
One more thing I noticed in your AWR screenshot
You have identical
BEGIN_INTERVAL_TIMEvalues appearing more than once, for example around 04-Aug.That could be because the database has multiple instances. Your query currently doesn't display
INSTANCE_NUMBER.If this is RAC, we absolutely need to know which instance is experiencing the problem.
Modify the AWR query to:
SELECT sn.begin_interval_time, ss.instance_number, ss.plan_hash_value, ss.executions_delta, ROUND( ss.elapsed_time_delta / 1000000, 2 ) elapsed_sec, ROUND( ss.elapsed_time_delta / NULLIF(ss.executions_delta,0) / 1000, 2 ) ms_per_exec, ss.buffer_gets_delta, ROUND( ss.buffer_gets_delta / NULLIF(ss.executions_delta,0) ) buffer_gets_per_exec, ss.disk_reads_delta FROM dba_hist_sqlstat ss, dba_hist_snapshot sn WHERE ss.snap_id = sn.snap_id AND ss.dbid = sn.dbid AND ss.instance_number = sn.instance_number AND ss.sql_id = '0bujgc94rg3fj' ORDER BY sn.begin_interval_time DESC, ss.instance_number;++++++++++++++++++
Find the exact execution-plan operation where ASH is accumulating
SELECT sql_plan_line_id, sql_plan_operation, sql_plan_options, event, session_state, COUNT(*) samples FROM dba_hist_active_sess_history WHERE top_level_sql_id = '0bujgc94rg3fj' AND sql_id = 'b2jckdvl94knx' GROUP BY sql_plan_line_id, sql_plan_operation, sql_plan_options, event, session_state ORDER BY samples DESC;Find which Oracle object is generating the reads
Run this one as well:
SELECT ash.current_obj#, obj.owner, obj.object_name, obj.object_type, ash.event, COUNT(*) samples FROM dba_hist_active_sess_history ash LEFT JOIN dba_objects obj ON obj.object_id = ash.current_obj# WHERE ash.top_level_sql_id = '0bujgc94rg3fj' AND ash.sql_id = 'b2jckdvl94knx' AND ash.current_obj# > 0 GROUP BY ash.current_obj#, obj.owner, obj.object_name, obj.object_type, ash.event ORDER BY samples DESC;sql_id, child_number, plan_hash_value, executions, ROUND(elapsed_time/1000000,2) elapsed_sec, ROUND(cpu_time/1000000,2) cpu_sec, buffer_gets, disk_reads, rows_processed, sql_fulltext FROM v$sql WHERE sql_id = 'b2jckdvl94knx';If it is no longer in shared pool:
SELECT sql_id, DBMS_LOB.SUBSTR(sql_text,4000,1) sql_text FROM dba_hist_sqltext WHERE sql_id = 'b2jckdvl94knx';The next query should therefore combine everything into one result.
SELECT sql_id, SUM(executions) executions, ROUND(SUM(elapsed_time)/1000000,2) elapsed_sec, ROUND(SUM(cpu_time)/1000000,2) cpu_sec, SUM(buffer_gets) buffer_gets, SUM(disk_reads) disk_reads, ROUND( SUM(elapsed_time) / NULLIF(SUM(executions),0) / 1000, 2 ) ms_per_exec, ROUND( SUM(buffer_gets) / NULLIF(SUM(executions),0), 2 ) buffer_gets_per_exec, ROUND( SUM(disk_reads) / NULLIF(SUM(executions),0), 2 ) disk_reads_per_exec FROM gv$sql WHERE sql_id IN ( '8nzx90zdhgfgc', '63v1yg88zt6gs', 'd119xbzfdqyar', 'cyly9yv9y91hb' ) GROUP BY sql_id ORDER BY elapsed_sec DESC;That result is much more meaningful.
Also determine why there are multiple rows
Run:
SELECT inst_id, sql_id, child_number, plan_hash_value, executions, ROUND(elapsed_time/1000000,2) elapsed_sec, ROUND(cpu_time/1000000,2) cpu_sec, buffer_gets, disk_reads FROM gv$sql WHERE sql_id IN ( '8nzx90zdhgfgc', '63v1yg88zt6gs', 'd119xbzfdqyar', 'cyly9yv9y91hb' ) ORDER BY sql_id, inst_id, child_number;Now get the SQL text
This is the most important next piece:
SELECT DISTINCT sql_id, DBMS_LOB.SUBSTR(sql_fulltext,4000,1) sql_text FROM gv$sql WHERE sql_id IN ( '8nzx90zdhgfgc', '63v1yg88zt6gs', 'd119xbzfdqyar', 'cyly9yv9y91hb' );SELECT event, wait_class, session_state, COUNT(*) samples FROM dba_hist_active_sess_history WHERE sql_id = '8nzx90zdhgfgc' GROUP BY event, wait_class, session_state ORDER BY samples DESC;Then:
SELECT ash.current_obj#, obj.owner, obj.object_name, obj.object_type, ash.event, COUNT(*) samples FROM dba_hist_active_sess_history ash LEFT JOIN dba_objects obj ON obj.object_id = ash.current_obj# WHERE ash.sql_id = '8nzx90zdhgfgc' AND ash.current_obj# > 0 GROUP BY ash.current_obj#, obj.owner, obj.object_name, obj.object_type, ash.event ORDER BY samples DESC;
No comments:
Post a Comment