Sunday, August 10, 2025

Your Go-To SQL Queries for Monitoring Oracle EBS Concurrent Requests

Your Go-To SQL Queries for Monitoring Oracle EBS Concurrent Requests 🧑‍💻

Need to see what’s happening in your Oracle E-Business Suite instance right now? Instead of clicking through endless forms, you can use these simple, read-only SQL queries to get a real-time snapshot of your concurrent requests. These scripts work on EBS 12.1 and 12.2 and can be run as the APPS user or any user with select grants on the FND tables.

Let's dive in!


0) Quick Reference: Phase & Status Codes

Before we start, here’s a handy key for decoding the phase_code and status_code columns you'll see in the queries.

  • phase_code: P=Pending, R=Running, C=Completed
  • status_code: R=Running, T=Terminating, X=Terminated, C=Normal, E=Error, G=Warning, D=Cancelled, W=Wait, H=On Hold

1) What is running right now?

This is the most fundamental query. It shows all currently executing concurrent requests, ordered by how long they've been running.


SELECT
  r.request_id,
  p.concurrent_program_name              AS prog_short,
  p.user_concurrent_program_name         AS program,
  u.user_name                            AS requested_by,
  r.actual_start_date,
  ROUND( (SYSDATE - r.actual_start_date)*24*60, 1 ) AS mins_running,
  r.phase_code,
  r.status_code,
  r.argument_text
FROM apps.fnd_concurrent_requests r
JOIN apps.fnd_concurrent_programs_vl p
  ON p.concurrent_program_id = r.concurrent_program_id
 AND p.application_id        = r.program_application_id
JOIN apps.fnd_user u
  ON u.user_id = r.requested_by
WHERE r.phase_code = 'R'        -- Running now
ORDER BY mins_running DESC, r.request_id;

2) Running requests with manager/node details

This query extends the previous one to show which concurrent manager, on which server node, is responsible for running the request.


SELECT
  r.request_id,
  p.user_concurrent_program_name AS program,
  q.user_concurrent_queue_name   AS manager,
  q.concurrent_queue_name        AS manager_code,
  q.target_node                  AS node,
  cp.concurrent_process_id,
  cp.os_process_id,
  r.actual_start_date,
  ROUND((SYSDATE - r.actual_start_date)*24*60, 1) AS mins_running
FROM apps.fnd_concurrent_requests r
JOIN apps.fnd_concurrent_programs_vl p
  ON p.concurrent_program_id = r.concurrent_program_id
 AND p.application_id        = r.program_application_id
LEFT JOIN apps.fnd_concurrent_processes cp
  ON cp.concurrent_process_id = r.controlling_manager
LEFT JOIN apps.fnd_concurrent_queues_vl q
  ON q.concurrent_queue_id    = cp.concurrent_queue_id
 AND q.application_id         = cp.queue_application_id
WHERE r.phase_code = 'R'
ORDER BY mins_running DESC;

3) See the Database session, waits, and SQL_ID

This is crucial for performance tuning. It links a running request directly to its database session (SID), wait events, and active SQL_ID.


SELECT
  r.request_id,
  p.user_concurrent_program_name AS program,
  s.inst_id,
  s.sid, s.serial#,
  s.username                     AS db_user,
  s.status                       AS sess_status,
  s.sql_id,
  s.event                        AS wait_event,
  s.seconds_in_wait,
  r.actual_start_date,
  ROUND((SYSDATE - r.actual_start_date)*24*60, 1) AS mins_running
FROM apps.fnd_concurrent_requests r
JOIN apps.fnd_concurrent_programs_vl p
  ON p.concurrent_program_id = r.concurrent_program_id
 AND p.application_id        = r.program_application_id
JOIN gv$session s
  ON s.audsid = r.oracle_session_id
WHERE r.phase_code = 'R'
ORDER BY mins_running DESC;

Note: If a request doesn't show up here, it might be executing code within the manager itself and not running a specific SQL statement at this exact moment.


4) Get the full SQL text for a running request

When you have the SQL_ID from the query above, you can use this to fetch the complete SQL text. Use this one sparingly as it can be a heavy query.


SELECT
  r.request_id,
  p.user_concurrent_program_name AS program,
  s.inst_id, s.sid, s.serial#, s.sql_id,
  q.sql_text
FROM apps.fnd_concurrent_requests r
JOIN apps.fnd_concurrent_programs_vl p
  ON p.concurrent_program_id = r.concurrent_program_id
 AND p.application_id        = r.program_application_id
JOIN gv$session s
  ON s.audsid = r.oracle_session_id
JOIN gv$sql q
  ON q.sql_id = s.sql_id
 AND q.inst_id = s.inst_id
WHERE r.phase_code = 'R';

5) Find concurrency hot-spots

Is a specific report being run by many users at once? This query identifies programs that have multiple instances running simultaneously.


SELECT
  p.user_concurrent_program_name AS program,
  COUNT(*) AS running_count,
  MIN(r.actual_start_date) AS oldest_start,
  MAX(r.actual_start_date) AS newest_start
FROM apps.fnd_concurrent_requests r
JOIN apps.fnd_concurrent_programs_vl p
  ON p.concurrent_program_id = r.concurrent_program_id
 AND p.application_id        = r.program_application_id
WHERE r.phase_code = 'R'
GROUP BY p.user_concurrent_program_name
HAVING COUNT(*) > 1
ORDER BY running_count DESC, oldest_start;

6) Check the current manager load

This query provides a quick summary of how many requests each concurrent manager is currently handling.


SELECT
  q.user_concurrent_queue_name AS manager,
  q.target_node                AS node,
  COUNT(*)                     AS running_requests
FROM apps.fnd_concurrent_requests r
JOIN apps.fnd_concurrent_processes cp
  ON cp.concurrent_process_id = r.controlling_manager
JOIN apps.fnd_concurrent_queues_vl q
  ON q.concurrent_queue_id    = cp.concurrent_queue_id
 AND q.application_id         = cp.queue_application_id
WHERE r.phase_code = 'R'
GROUP BY q.user_concurrent_queue_name, q.target_node
ORDER BY running_requests DESC, manager;

7) Find long-running requests

Use this script to find all jobs that have been running longer than a specific threshold (e.g., more than 15 minutes).


-- Set :mins_threshold to your desired value, e.g., 15
SELECT
  r.request_id,
  p.user_concurrent_program_name AS program,
  u.user_name                     AS requested_by,
  r.actual_start_date,
  ROUND((SYSDATE - r.actual_start_date)*24*60, 1) AS mins_running
FROM apps.fnd_concurrent_requests r
JOIN apps.fnd_concurrent_programs_vl p
  ON p.concurrent_program_id = r.concurrent_program_id
 AND p.application_id        = r.program_application_id
JOIN apps.fnd_user u
  ON u.user_id = r.requested_by
WHERE r.phase_code = 'R'
  AND (SYSDATE - r.actual_start_date) * 24 * 60 >= :mins_threshold
ORDER BY mins_running DESC;

8) See what's in the pending queue

This shows you all the requests that are waiting to run, whether they are scheduled for the future or on hold.


SELECT
  r.request_id,
  p.user_concurrent_program_name AS program,
  u.user_name                     AS requested_by,
  r.requested_start_date,
  r.phase_code,
  r.status_code,
  r.hold_flag
FROM apps.fnd_concurrent_requests r
JOIN apps.fnd_concurrent_programs_vl p
  ON p.concurrent_program_id = r.concurrent_program_id
 AND p.application_id        = r.program_application_id
JOIN apps.fnd_user u
  ON u.user_id = r.requested_by
WHERE r.phase_code = 'P'   -- Pending
ORDER BY NVL(r.requested_start_date, SYSDATE), r.request_id;

9) Get log and output file locations

Quickly find the exact log and output file names and server locations for any running request.


SELECT
  r.request_id,
  p.user_concurrent_program_name AS program,
  r.logfile_name,
  r.logfile_node_name,
  r.outfile_name,
  r.outfile_node_name
FROM apps.fnd_concurrent_requests r
JOIN apps.fnd_concurrent_programs_vl p
  ON p.concurrent_program_id = r.concurrent_program_id
 AND p.application_id        = r.program_application_id
WHERE r.phase_code = 'R'
ORDER BY r.request_id;

10) See running children of a request set

When a request set is running, use this query to see the status of all its child requests.


-- Replace :parent_request_id with the parent request ID
SELECT
  r.parent_request_id,
  r.request_id,
  p.user_concurrent_program_name AS program,
  r.phase_code,
  r.status_code,
  r.actual_start_date,
  ROUND((SYSDATE - r.actual_start_date)*24*60, 1) AS mins_running
FROM apps.fnd_concurrent_requests r
JOIN apps.fnd_concurrent_programs_vl p
  ON p.concurrent_program_id = r.concurrent_program_id
 AND p.application_id        = r.program_application_id
WHERE (r.parent_request_id = :parent_request_id OR r.request_id = :parent_request_id)
  AND r.phase_code IN ('R','P')  -- running or pending
ORDER BY r.parent_request_id, r.request_id;

11) Filter by a specific program name

Quickly find all running instances of a specific program, searching by either its short name or its user-facing display name.


-- Bind either :prog_short (e.g., 'XX_REPORT') or :prog_name
SELECT
  r.request_id,
  p.concurrent_program_name      AS prog_short,
  p.user_concurrent_program_name AS program,
  r.phase_code,
  r.status_code,
  r.actual_start_date,
  ROUND((SYSDATE - r.actual_start_date)*24*60, 1) AS mins_running
FROM apps.fnd_concurrent_requests r
JOIN apps.fnd_concurrent_programs_vl p
  ON p.concurrent_program_id = r.concurrent_program_id
 AND p.application_id        = r.program_application_id
WHERE r.phase_code = 'R'
  AND (p.concurrent_program_name = :prog_short OR p.user_concurrent_program_name = :prog_name)
ORDER BY mins_running DESC;

12) See all long-running jobs started today

This is a handy end-of-day check to see which jobs kicked off today are still running.


SELECT
  r.request_id,
  p.user_concurrent_program_name AS program,
  r.actual_start_date,
  ROUND((SYSDATE - r.actual_start_date)*24*60, 1) AS mins_running
FROM apps.fnd_concurrent_requests r
JOIN apps.fnd_concurrent_programs_vl p
  ON p.concurrent_program_id = r.concurrent_program_id
 AND p.application_id        = r.program_application_id
WHERE r.phase_code = 'R'
  AND r.actual_start_date >= TRUNC(SYSDATE)
ORDER BY mins_running DESC;

✅ Key Takeaways & Gotchas

  • The most reliable flag for an executing job is phase_code='R'.
  • The join from r.oracle_session_id to gv$session.audsid is the definitive way to link a request to its database session.
  • If you are on a RAC database, using GV$ views (like gv$session) instead of V$ is critical to see sessions on all nodes.
  • For a higher-level view, you can also query APPS.FND_CONC_REQ_SUMMARY_V, but the queries above give you the raw, detailed data for deep-dive analysis.

Saturday, August 9, 2025

Your Guide to Locking in a Good Execution Plan 🚀

Taming the Oracle Optimizer: Your Guide to Locking in a Good Execution Plan 🚀

Ever had a critical SQL query that ran perfectly fast yesterday, but is crawling today? You haven't changed the code, so what gives? The culprit is often the Oracle Optimizer changing its mind about the execution plan—the internal "road map" it uses to fetch your data.

When performance is unpredictable, you need to take control. SQL Plan Management (SPM) is Oracle's built-in feature that lets you find a "golden" execution plan and tell the database to use it every time. This guide will walk you through how to capture and enforce a good plan using SQL Plan Baselines.


## Before You Start: Quick Checks & Privileges

Before diving in, make sure your environment is ready.

  1. Confirm SPM is enabled: The optimizer_use_sql_plan_baselines parameter must be set to TRUE. Run this check:

    SQL
    SHOW PARAMETER optimizer_use_sql_plan_baselines;
    
  2. Ensure you have the right privileges: You'll need specific permissions to manage baselines and query performance data. Your DBA can grant you these:

    • ADMINISTER SQL MANAGEMENT OBJECT: Required for using the DBMS_SPM package.

    • SELECT_CATALOG_ROLE: Required for querying the Automatic Workload Repository (AWR) views like dba_hist_sqlstat.


## Step 1: Identify the SQL and the "Good" Plan

First, you need to find the specific query and the high-performing execution plan you want to stabilize. Every query has a unique SQL_ID, and each of its execution plans has a PLAN_HASH_VALUE (PHV).

  • If the good plan ran recently, you can find it in the cursor cache:

    SQL
    -- Find a recent plan in the cursor cache
    SELECT
        sql_id,
        plan_hash_value,
        parsing_schema_name,
        executions
    FROM
        gv$sqlarea
    WHERE
        sql_id = '&SQL_ID';
    
  • If the good plan is older, you'll need to look in the AWR history:

    SQL
    -- Find a historical plan in AWR
    SELECT
        snap_id,
        sql_id,
        plan_hash_value,
        elapsed_time_delta,
        executions_delta
    FROM
        dba_hist_sqlstat
    WHERE
        sql_id = '&SQL_ID'
    ORDER BY
        elapsed_time_delta DESC; -- Find the fastest executions
    

Once you have the SQL_ID and the PLAN_HASH_VALUE of your desired plan, you're ready to create a baseline.


## Step 2: Create the SQL Plan Baseline

You can load a baseline from either the live cursor cache or the historical AWR data.

### Option A: From the Cursor Cache (The Quickest Method)

Use this method if the good plan is still in memory. It's the fastest way to create a baseline.

SQL
DECLARE
  l_loaded NUMBER;
BEGIN
  l_loaded := DBMS_SPM.LOAD_PLANS_FROM_CURSOR_CACHE(
                sql_id          => '&SQL_ID',
                plan_hash_value => &PLAN_HASH_VALUE,
                enabled         => 'YES'
              );

  DBMS_OUTPUT.PUT_LINE('Baselines loaded: ' || l_loaded);
END;
/

### Option B: From AWR (When the Plan is Not in Cache)

If the plan is no longer in the cache, you can pull it from AWR. First, identify a snapshot window when the good plan was running.

SQL
-- 1. Find the begin and end snapshot IDs
SELECT
    MIN(snap_id) begin_snap,
    MAX(snap_id) end_snap
FROM
    dba_hist_snapshot
WHERE
    end_interval_time BETWEEN TO_DATE('&START_TIME', 'YYYY-MM-DD HH24:MI')
                      AND     TO_DATE('&END_TIME', 'YYYY-MM-DD HH24:MI');

-- 2. Load the baseline from the AWR snapshot window
DECLARE
  l_loaded NUMBER;
BEGIN
  l_loaded := DBMS_SPM.LOAD_PLANS_FROM_AWR(
                begin_snap      => &BEGIN_SNAP,
                end_snap        => &END_SNAP,
                sql_id          => '&SQL_ID',
                plan_hash_value => &PLAN_HASH_VALUE,
                enabled         => 'YES'
              );

  DBMS_OUTPUT.PUT_LINE('Baselines loaded: ' || l_loaded);
END;
/

## Step 3: Verify the Baseline and (Optionally) "Fix" It

After loading the plan, verify that the baseline was created. You'll need the SQL_HANDLE for future actions.

SQL
SELECT
    sql_handle,
    plan_name,
    enabled,
    accepted,
    fixed,
    created
FROM
    dba_sql_plan_baselines
WHERE
    sql_text LIKE '%<unique fragment of the SQL>%';

By default, a new baseline is ENABLED and ACCEPTED. This means the optimizer will consider it. If you want to force the optimizer to only use this plan, you can set it to FIXED.

Best Practice: Avoid fixing a plan immediately. Let it run as ENABLED and ACCEPTED first. Only fix it once you are absolutely certain this plan is the best choice under all conditions.

SQL
-- Optionally "fix" the plan to pin it
DECLARE
  l_out PLS_INTEGER;
BEGIN
  l_out := DBMS_SPM.ALTER_SQL_PLAN_BASELINE(
             sql_handle      => '&SQL_HANDLE',
             plan_name       => '&PLAN_NAME',
             attribute_name  => 'fixed',
             attribute_value => 'YES'
           );
END;
/

## Step 4: Test That the Baseline Is Used ✅

Now for the final check! Run your query again and inspect the execution plan details.

SQL
-- Show the executed plan and check the notes
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL, NULL, 'BASIC +NOTE'));

In the output, you should see a "Note" section confirming that your baseline was used:

Note
-----
   - SQL plan baseline "PLAN_NAME_HERE" used for this statement

You can also query gv$sql to see which baseline is attached to your session's cursor.

SQL
SELECT sql_id, sql_plan_baseline, plan_hash_value FROM gv$sql WHERE sql_id = '&SQL_ID';

## Step 5 (Optional): Migrate Baselines Between Databases

If you've identified and tested a great plan in your UAT or test environment, you can easily migrate it to production using staging tables.

  1. In the source database, create a staging table and pack the baseline into it.

    SQL
    BEGIN
      -- Create the staging table
      DBMS_SPM.CREATE_STGTAB_BASELINE(table_name => 'SPM_STAGE', schema_name => 'APPS');
    
      -- Pack the desired baseline into the table
      DBMS_SPM.PACK_STGTAB_BASELINE(
        table_name   => 'SPM_STAGE',
        schema_name  => 'APPS',
        sql_handle   => '&SQL_HANDLE'
      );
    END;
    /
    
  2. Move the SPM_STAGE table to the target database (using Data Pump or another method).

  3. In the target database, unpack the baseline from the staging table.

    SQL
    BEGIN
      DBMS_SPM.UNPACK_STGTAB_BASELINE(
        table_name   => 'SPM_STAGE',
        schema_name  => 'APPS'
      );
    END;
    /
    

## Step 6: Maintenance - How to Back Out

If a baseline is no longer needed or is causing issues, you can either disable it (keeping it for reference) or drop it completely.

  • Disable a baseline:

    SQL
    DECLARE
      n PLS_INTEGER;
    BEGIN
      n := DBMS_SPM.ALTER_SQL_PLAN_BASELINE(
             sql_handle      => '&SQL_HANDLE',
             plan_name       => '&PLAN_NAME',
             attribute_name  => 'enabled',
             attribute_value => 'NO'
           );
    END;
    /
    
  • Drop a baseline permanently:

    SQL
    DECLARE
      n PLS_INTEGER;
    BEGIN
      n := DBMS_SPM.DROP_SQL_PLAN_BASELINE(
             sql_handle => '&SQL_HANDLE',
             plan_name  => '&PLAN_NAME'
           );
    END;
    /
    

## An Alternative: When to Use a SQL Profile

Sometimes, an application generates SQL with varying literals (e.g., WHERE id = 101 vs. WHERE id = 205). SPM requires an exact text match, so it may not work here. In these cases, a SQL Profile is a better choice.

A SQL Profile doesn't lock a plan; instead, it attaches a set of hints to a query to "nudge" the optimizer toward the right plan shape.

  1. Get the Outline Hints for the good plan:

    SQL
    SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR('&SQL_ID', NULL, 'ADVANCED'));
    -- Copy the hints from the "Outline Data" section
    
  2. Import a SQL Profile with those hints:

    SQL
    DECLARE
      h SYS.SQLPROF_ATTR;
    BEGIN
      h := SYS.SQLPROF_ATTR(
        'USE_HASH_AGGREGATION(@SEL$1)',
        'LEADING(@"SEL$1" "T1"@"SEL$1" "T2"@"SEL$1")',
        'INDEX_RS_ASC("T1"@"SEL$1" "T1_IDX")'
        -- Paste all other outline hints here
      );
    
      DBMS_SQLTUNE.IMPORT_SQL_PROFILE(
        sql_text     => q'[ PASTE THE EXACT SQL TEXT HERE ]',
        profile      => h,
        name         => 'PROF_FIX_MY_QUERY',
        force_match  => TRUE, -- Set TRUE to match queries with different literals
        replace      => TRUE
      );
    END;
    /
    

While flexible, SQL Profiles offer less deterministic control than a fixed SPM baseline. Prefer SPM for strict plan stability.


## Practical Tips & Common Gotchas

  • Exact Text Match: SPM is picky about SQL text. If your application uses literals instead of bind variables, consider setting CURSOR_SHARING to FORCE or using a SQL Profile with force_match => TRUE.

  • Statistics Drift: A baseline forces a plan's shape (join order, access methods), but the optimizer's row count estimates can still change if statistics become stale. Keep your stats fresh!

  • Bind Peeking: A plan that's great for one set of bind variables might be terrible for another. If a single fixed plan isn't safe, avoid fixing it and explore adaptive features.

  • Troubleshooting: If your baseline isn't being used, double-check that optimizer_use_sql_plan_baselines is TRUE, the SQL text matches perfectly, and you don't have conflicting baselines.

Thursday, August 7, 2025

How to Find Old Execution Plans for Any SQL_ID

 

Oracle's Time Machine: How to Find Old Execution Plans for Any SQL_ID

By a Performance Tuning Enthusiast in Hyderabad | August 7, 2025

Here in Hyderabad, as the evening draws in, there's a familiar story playing out in data centers across the city. A critical report that flew through the system yesterday is crawling today. Users are complaining, tickets are being raised, and all eyes are on the DBA. What went wrong? The prime suspect is almost always the same: a change in the query's execution plan.

But how can you prove it? To solve this mystery, you need a time machine. You need to see not only the plan the query is using now but also the plan it was using before the trouble started. Luckily, Oracle Database 19c provides the tools to do just that. All you need is the query’s unique identifier: the SQL_ID.

Let's dive into how you can become a database detective and uncover a query's past.

First, The Key Concepts: SQL_ID and PLAN_HASH_VALUE

Think of it like this:

  • SQL_ID: This is like the license plate for your query text. No matter how many times the query runs, if the text is identical, it will have the same SQL_ID.

  • PLAN_HASH_VALUE: This is the unique identifier for a specific execution plan, or the route the query takes to get the data. A single SQL_ID can have multiple PLAN_HASH_VALUEs over time if the optimizer decides on a different route.

Our mission is to find the different routes (plans) a single car (SQL_ID) has taken.


Method 1: The Crime Scene Investigator 🔍 (Finding Recent Plans)

Scenario: The performance issue is happening right now, or happened very recently. The evidence is still fresh and likely sitting in the database's memory (the "shared pool").

This is the quickest way to get an answer.

Step 1: Get the SQL_ID

If you don't have it already, you can grab it from V$SQL using a snippet of the query text.

SQL
-- Find the license plate for your query
SELECT sql_id, child_number, sql_text
FROM v$sql
WHERE sql_text LIKE '%some_unique_part_of_your_query%';

Step 2: Display the Cached Plan

Now, use the magical DBMS_XPLAN.DISPLAY_CURSOR function. It reads the plan directly from the cache.

SQL
-- Show me the current route!
SELECT *
FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR('your_sql_id', NULL, 'TYPICAL'));
  • Just replace 'your_sql_id' with the one you found.

  • Providing NULL for the second parameter tells Oracle to show you all the plans for that SQL_ID currently in memory.

This will give you a formatted execution plan, showing you exactly how Oracle is running the query right now.


Method 2: The Historian ⏳ (Digging into the Past with AWR)

Scenario: The problem started yesterday, or last week. The evidence is long gone from the live memory cache. We need to go to the archives.

For this, we turn to the Automatic Workload Repository (AWR). This is Oracle's built-in performance data warehouse.

Important Note: Using AWR features requires an Oracle Diagnostic Pack license. Make sure your organization is licensed before using these queries in production.

Step 1: Find the Historical SQL_ID

Similar to before, but this time we search the AWR history.

SQL
SELECT sql_id, sql_text
FROM dba_hist_sqltext
WHERE sql_text LIKE '%some_unique_part_of_your_query%';

Step 2: Uncover All Historical Plans

We'll use a different function, DBMS_XPLAN.DISPLAY_AWR, which is specifically designed to pull plans from the AWR.

SQL
-- Open the archives for this SQL_ID
SELECT *
FROM TABLE(DBMS_XPLAN.DISPLAY_AWR('your_sql_id', NULL, NULL, 'TYPICAL'));

This command will display every distinct execution plan (PLAN_HASH_VALUE) that AWR has ever captured for your SQL_ID. You might see the "good" plan from last week and the "bad" plan from today side-by-side!

Step 3: Pinpoint When the Plan Changed

This is the masterstroke. How do you know which plan was active and when? By joining the AWR's SQL statistics with its snapshot information, you can build a timeline.

SQL
-- Show me a timeline of every plan used by this query
SELECT
    s.snap_id,
    s.end_interval_time AS last_seen,
    st.plan_hash_value,
    st.executions_delta AS executions_in_snapshot
FROM
    dba_hist_sqlstat st
JOIN
    dba_hist_snapshot s ON st.snap_id = s.snap_id
WHERE
    st.sql_id = 'your_sql_id'
ORDER BY
    s.end_interval_time DESC;

The result of this query is gold. The LAST_SEEN column tells you the timestamp of the last time a specific PLAN_HASH_VALUE was observed. By looking at the top rows, you can see the most recent plan, and by scrolling down, you can see what the plan was before and pinpoint the exact time the plan flipped.

Your Performance Tuning Workflow

  1. Problem Now? Use DBMS_XPLAN.DISPLAY_CURSOR for an instant look.

  2. Problem in the Past? Use DBMS_XPLAN.DISPLAY_AWR to see all historical plans.

  3. Need to Know When? Run the final AWR history query to build a timeline of plan changes.

Armed with this knowledge, you are no longer just guessing. You can definitively say, "This query started performing poorly at 10:00 AM yesterday because the plan changed from 3541068894 to 1048557912." That's not just debugging; that's database forensics.

Happy tuning!

Tuesday, August 5, 2025

Troubleshooting Oracle EBS Workflow Mailer Errors

Expert Guide: Troubleshooting Workflow Mailer Errors in Oracle EBS

The Oracle E-Business Suite Workflow Mailer is a linchpin for business process automation, but a silent failure can quickly halt critical notifications. Diagnosing these issues requires a methodical and expert approach, often in preparation for opening a service request with Oracle Support. This post provides a comprehensive guide to performing a deep-dive analysis of Workflow Mailer problems.

A Comprehensive Troubleshooting Guide

Follow these expert steps to diagnose and resolve Workflow Mailer issues, moving from simple checks to in-depth analysis using Oracle's own tools and log files.

  1. Test the Mailer in OAM:

    • Log in as SYSADMIN and navigate to the Workflow Manager.

    • Select the Notification Mailer and click View Details.

    • Click the Test Mailer icon. Do not use SYSADMIN as the recipient. Instead, use the user role from a known failed notification to determine if the issue is user-specific.

    • This simple test can confirm basic connectivity and configuration.

  2. Use the wfmlrdbg.sql Diagnostic Script:

    • This is an invaluable tool provided by Oracle for deep-diving into a specific notification.

    • Log into the EBS application tier.

    • Source the environment file.

    • Execute the script: $FND_TOP/sql/wfmlrdbg.sql.

    • Provide the NOTIFICATION_ID of a failed notification.

    • The script will generate a detailed log of the notification's journey, including the full XML payload that the mailer attempted to process. Analyze this output carefully for any errors.

  3. Analyze Concurrent Log Files:

    • The most detailed error information is often found in the Workflow Mailer log files on the concurrent tier.

    • Navigate to the log directory: $APPLCSF/$APPLLOG/.

    • First, confirm the presence of the log files with a simple ls:

      ls -l FNDCPGSC*.txt
      
    • Next, use grep to filter for specific error patterns. The output is redirected to new files for easy review.

      grep -i "ERROR:" FNDCPGSC*.txt > mailer_errors.log
      grep -i "EXCEPTION:" FNDCPGSC*.txt > mailer_exceptions.log
      grep -i "UNEXPECTED:" FNDCPGSC*.txt > mailer_unexpected.log
      
    • Review these generated files for clues about connection issues, malformed messages, or other systemic problems. If the output files are empty, it means grep found no matches. You can then try a broader search or focus on a different log file.

  4. Verify Workflow Mailer Service Status:

    • Ensure the Workflow Notification Mailer service is up and running correctly.

    • You can check this in Oracle Applications Manager (OAM) or with a SQL query:

      SELECT component_status FROM fnd_svc_components WHERE component_name = 'Workflow Notification Mailer';
      
    • The status should be RUNNING.

By following these expert-level diagnostic steps, you can efficiently move from problem identification to resolution, or gather all the necessary information to present a detailed and actionable case to Oracle Support.

Title: How to Find the Tables Behind a SQL_ID in Oracle

 Title: How to Find the Tables Behind a SQL_ID in Oracle

As a database administrator or developer, you've likely encountered a situation where you need to troubleshoot a slow-running query. You have the SQL_ID from a performance report or a monitoring tool, but the SQL text itself is long, complex, or a mystery. The first step to understanding the query's behavior is to figure out which tables it's actually accessing.

Fortunately, in an Oracle database, this is a straightforward process. The execution plan—the set of steps the database uses to execute a SQL statement—holds all the answers. By examining the plan, we can quickly identify the tables, views, and indexes involved.

Let's dive into the two primary methods for doing this.

Method 1: Finding Tables for Active or Recent Queries

If the SQL statement is currently running or was executed very recently, its execution plan will still be in the shared pool. We can access this information using the V$SQL_PLAN view.

The V$ views are dynamic performance views, which means they show the current state of the database. V$SQL_PLAN specifically stores the execution plans for all SQL_IDs in the shared pool.

Here's the query you'll use:

SQL
SELECT
    t.object_owner,
    t.object_name,
    t.operation,
    t.options,
    s.sql_text
FROM
    v$sql_plan t,
    v$sql s
WHERE
    t.sql_id = s.sql_id
    AND t.sql_id = '&sql_id'
ORDER BY
    t.id;

How it works:

  • We join V$SQL_PLAN (t) with V$SQL (s) on sql_id.

  • V$SQL_PLAN gives us the execution plan details, including OBJECT_OWNER and OBJECT_NAME for each step.

  • V$SQL gives us the actual SQL_TEXT for context.

  • We filter the results by providing our specific SQL_ID.

  • The ORDER BY t.id ensures the plan steps are displayed in the correct sequence.

When you run this query, you'll be prompted to enter the sql_id. The output will be a list of operations, and any line with an OPERATION like 'TABLE ACCESS', 'INDEX FULL SCAN', or 'INDEX RANGE SCAN' will show you the corresponding table or index name under OBJECT_NAME.

Pro-Tip: If the query is on a view, the execution plan will typically show the underlying base tables, giving you a complete picture.

Method 2: Finding Tables for Historical Queries

What if the query was executed days or even weeks ago? The SQL_ID is no longer in the shared pool, so V$SQL_PLAN won't help. This is where the Automatic Workload Repository (AWR) comes in.

Oracle's AWR automatically collects, processes, and maintains performance statistics, including execution plans for popular or resource-intensive queries. This data is stored in historical views, which are prefixed with DBA_HIST_.

The view we're interested in is DBA_HIST_SQL_PLAN.

Here's the query to find historical tables for a SQL_ID:

SQL
SELECT
    t.object_owner,
    t.object_name,
    t.operation,
    t.options
FROM
    dba_hist_sql_plan t
WHERE
    t.sql_id = '&sql_id'
ORDER BY
    t.plan_hash_value,
    t.id;

How it works:

  • This query is very similar to the previous one, but we're now querying DBA_HIST_SQL_PLAN.

  • The plan_hash_value is included in the ORDER BY clause because a single SQL_ID can have multiple execution plans over time. This helps to group the steps for each plan together.

Permissions are Key! To run these queries, you'll need the necessary permissions. Typically, users with roles like SELECT_CATALOG_ROLE or DBA will have access to these views. If you encounter an "insufficient privileges" error, you'll need to contact your DBA to grant you the required permissions.

By using these simple but powerful queries, you can quickly demystify any SQL_ID and get a clear understanding of the tables and objects it's interacting with, putting you on the right path to performance tuning.

Happy troubleshooting!