Monday, December 5, 2011

Oracle E-Business Suite 11i and Database

Oracle E-Business Suite 11i and Database FAQ [ID 285267.1]


Modified 21-AUG-2009 Type FAQ Status PUBLISHED

Oracle E-Business Suite 11i and Database FAQ

( Last Updated: November 21st 2006)

This document is intended for use by customers interested in installing and configuring E-Business Suite 11i . The Technical References section contains pointers to the complete installation and configuration instructions for some of these components

Users are strongly advised to make backups of their environments before executing any of the procedures noted, and to test their environments before executing these procedures in production environments. Some of the procedures in this document may have significant effects on Oracle E-Business Suite Release 11i environments and should be executed only by skilled Oracle E-Business Suite database or systems administrators.

This document is frequently updated. Please obtain the latest version from MetaLink before proceeding further.

1

Supported Versions

2

Technical Architecture

3

Install

4

Concurrent Processing

5

Partitioning

6

Disaster Recovery

7

Real Application Clusters (RAC)

8

Oracle Global Single Instance

9

Technical References

Supported Versions

1. Is 10g certified with E-Business Suite?
Oracle Applications 11i (version 11.5.9 and version 11.5.10) is now certified for use with the Oracle Database 10g Release 2 (minimum version 10.2.0.2) running on Linux x86 and other platforms. Requisite patches and other instructions have been published via MetaLink Note# 362203.1 (Interoperability Notes - Oracle Applications 11i with Oracle Database 10g Release 2). For specific platform versions supported, please check Certify.

Oracle Applications 11i (version 11.5.9 and version 11.5.10) is certified for use with the Oracle Database 10g Release 1 (
minimum version 10.1.0.4) running on Linux x86 and other platforms. Requisite patches and other instructions have been published via MetaLink Note# 282038.1 (Interoperability Notes - Oracle Applications 11i with Oracle Database 10g Release 1). For specific platform versions supported, please check Certify.

2. Will there be additional database patches required on top of 10g ?
Yes.
All the required patches for Oracle Database 10g Release1 are documented in
MetaLink Note# 282038.1
All the required patches for Oracle Database 10g Release2 are documented in MetaLink Note# 362203.1

3. What are the supported versions for 9ir2 Database ( 9.2.0.x ) and E-Business Suite ?
Currently 9.2.0.8 is certified on E-Business Suite 11.5.7 and above. You can refer
Metalink Note# 216550.1 for more details..

4. Are the Real Application Clusters ( RAC ) and Automatic Storage Management (ASM) features also certified with 10g ?
Yes. If 10g certification is announced on a platform, RAC and ASM are also certified on that platform

5. My customer would like to start planning for 10g Real Application Clusters ( RAC ) and Automated Storage Management (ASM) upgrade immediately, are there any development programs to assist him?
Yes, for customers planning on using 10g RAC or 10g ASM they can avail of the 10g APPSRAP program. This program is available on invitation for all customers. Customers interested in this program can direct their Account Manger or Service Delivery Manager to the Oracle Internal AppsRAP website:
http://appsrap.us.oracle.com and contact the Program Manager, Vamsi Mudumba ( vamsi.mudumba@oracle.com )

6. What are the supported versions for 9i RAC and E-Business Suite ?
Currently 9.2.0.8 RAC is certified on E-Business Suite 11.5.7 and above. For the latest information on certified combinations ,platform information please go to
Certify on MetaLink

7. Which Application Modules in the E-Business Suite support Real Application Clusters ( RAC ) ?
All E-Business Suite modules work when deployed in a RAC enabled database platform. For specific Application specific best practices, look at the Section on
Application Specific Best Practices

Technical Architecture

1. Is it possible to deploy E-Business Suite 11i on an Itanium Database Server ?
Yes. More details regarding supported Itanium platforms and migration process are documented in Metalink Note# 304489.1

2. Is the use of BIG-IP Load Balancer across multiple mid-tiers supported ?
Hardware load balancers are a supported configuration for Application architecture. Now, there is autoconfig support too for this architecture. For more information, see Oracle
MetaLink Note 217368.1 , Advanced Configurations and Topologies for Enterprise Deployments of E-Business Suite 11i.

3. Can Oracle Web Cache be used with Oracle iStore 11i implementations?
Oracle Web Cache is certified for use with Oracle iStore 11i. See
MetaLink Note# 186981.1 for a more detailed discussion on this topic

4. Is MTS (Multi Threaded Server) supported for E-Business Suite 11i customers ?
MTS can be used with E-Business Suite 11i However, it is only useful on 32 bit Windows. It is not recommended for other platforms as the overall performance is worse with MTS, and the shared pool footprint explodes. The dispatcher also increases overall data server CPU with 11i workloads.

5. Are extended clusters with RAC supported for 11i ?
Yes. A few customers are currently live with this architecture ( eg: Canon Europa in Netherlands )

6. Is the use of separate Operating Systems supported for Midtier and Database ?
Yes. This is called a split configuration for 11i. For more details on what combinations are certified look at
Certify on Metalink

Install

1. Can an OUI based global inventory be recreated ?
If the new XML format inventory is being used, just re-register to the global inventory. If that's not the case then there is no easy way to recover it. It will be a manual effort specific to each scenario and will result in data loss which might not be critical always.

2. What's the impact of setting the column SESSION_COOKIE_NAME in the ICX_PARAMETERS table in a RAC environment ?
The Self–Service infrastructure (ICX), by default, uses the database instance name as part of the cookie name if the cookie name has not been explicitly set by the administrator. In the RAC environment, the database instance name can change based on which instance the users were connected. Hence, you should explicitly set the cookie name (SESSION_COOKIE_NAME) to a constant value in the ICX_PARAMETERS table. Update the ICX_PARAMETERS table from the Apps schema using the following example.

SQL> update ICX_PARAMETERS
set SESSION_COOKIE_NAME=’VISRAC1’;

3. What's the impact of setting the profile option Applications Database Id in a RAC environment ?
Applications Database Id needs to be set in RAC based environments in order to avoid a cookie mismatch. Since RAC environments will have multiple rows in gv$instance, the profile must be set to avoid mismatches when building the cookie. The difference between setting the column SESSION_COOKIE_NAME in ICX_PARAMETERS table, and Applications Database Id is that SESSION_COOKIE_NAME provides control to override the default cookie naming at the site level, while Applications Database Id, can be implemented at either the site, responsibility, or user levels. It's best to set both of them to the same value at the site level.

Concurrent Processing

1. How should SFM processes be defined in a PCP environment with RAC?
Define a primary and secondary node for all the SFM managers just like the rest of the managers. SFM must run corresponding to every database node that serves fulfillment requests ( similar to the transaction managers). Please look at
MetaLink Note# 240818.1 for more information

2. Can multiple transaction managers be started on multiple servers serving a difference instance without using PCP ?
No. PCP should be used for all such configurations.

3. Is there a requirement for a NAS device ( like NetApp) to store all the output and log files when using PCP ?
No. PCP can be used with files that are stored on local servers. You will have to setup FNDFS appropriately so that files on one node can be accessed by requests from another node. It is also recommended to set the init.ora parameter max_commit_propagation_delay= 0. For Compaq Tru64 clusters only, please set max_commit_propagation_delay= 1.

4: Does FNDFS support load balancing and failover ?
Yes, if load balancing or failover support is required for FNDFS, then the file system will have to be identical on all the hosts.

5. Should I separate my concurrent processing tier from the database tier ?
Historically, the accepted best practice for concurrent manager deployment has been to run the Concurrent Manager processes on the same server(s) as the database itself, to eliminate network traffic between the managers and their dedicated server processes. However, with the availability of fast LANs between the middle tier servers and database servers, best practice now is to configure the managers to run on separate middle tier servers. This gives you maximum flexibility on choosing hardware platforms (the database tier can be a different platform than the middle tiers) and allows the database servers to be fully available for database processing, with no cycles needed for batch programs.

6. Are virtual hostnames currently supported with Concurrent Processing ?
No. There is no support for virtual hosts with CP

7. We run a huge number of concurrent requests in our environment. What are general tips for better performance ?
1) Check the corresponding concurrent manager parameters and tune them - process, sleep, cache
2) Check the process and session parameters in init.ora or spfile for the database instance
3) Check to see if there are any fnd tables that are frequently locking and increase the freelist parameter ( if not using Automatic Segment Space Management, ASSM )
4) Run the purge concurrent request job

8. How do Java concurrent programs work ? Can I direct load for Java concurrent programs to a specific node ?
Java Concurrent Programs and Self-service applications use the connect string specified in DBC file. Currently there is no way to create a separate dbc file for Java Concurrent programs and constrain where they run in a RAC environment if the CP tier and Web tier are shared.

Partitioning

1. Is Partitioning supported with E-Business Suite ?
Yes. Partitioning of Applications standard or custom tables is supported. Oracle Applications utilizes partitioning in the standard product in many modules out-of-the-box.

2. Are there documented best practices for partitioning E-Business Suite objects ?
Currently there are no documented best practices on this topic. Customers intending to partition should understand the data access paths, growth rates, access patterns and modules that the object is used in.

Disaster Recovery

1. Is Logical Standby supported with E-Business Suite 11i ?
No. Logical standby is not supported with E-Business Suite 11i.

2. Is Physical Standby with Real Application Clusters ( RAC) supported with E-Business Suite ?
Yes. This is a supported configuration. Best practices are document in Metalink Note# 341437.1

Real Application Clusters ( RAC )

Troubleshooting

1. How do I check the IP address / NIC being used for the inter-node traffic ?
To determine the IP address / NIC being used for RAC related traffic, run the following commands in SQL*Plus:

SQL> oradebug setmypid
SQL> oradebug ipc

The information written to a trace file in the user_dump_dest will indicate the IP address

2. How can interconnect traffic be monitored ?
1) From the hourly Statspack reports, monitor these statistics ( generated every second ) from both instances: 'global cache cr blocks served' and 'global cache current blocks served'. This should give the total number of blocks sent across the interconnect per second during each interval. It would be useful to understand how the interconnect traffic varies over time and what the system is doing when these numbers are high.
2) Run Statspack at level 7 (or higher) to get segment statistics. This should show which segments that contribute to the interconnect traffic. You can also check gv$segment_statistics and gv$cache_transfer.
3) Monitor gv$session_wait to see which sessions are involved in RAC related waits. This might give us an idea about which users or modules that are involved in generating the interconnect traffic. It will also give us the specific blocks.
4) Use OS utilities to monitor the interconnect traffic as well. Look for variations over time.
5) Keep an eye on the statistic 'global cache blocks lost' which might indicate problems with the interconnect.
6) Watch for any network errors reported by the OS monitoring utilities.

Database Load Balancing

1. Do I need to do functional partitioning for RAC ?
No.. But you can if you need to push your database servers close to their maximum capacity, and need the small percentage of improved performance provided by distributing the load along functional boundaries. This however does entail more maintenance during failover.

2. How can one achieve functional partitioning for forms modules ?
Set the profile option Database Instance to a specific instance either at the application, responsibility or user level. The internal name for this profile option is INSTANCE_PATH. For more details refer to MetaLink Note# 294652.1 E-Business Suite 11i on RAC : Configuring Database Load balancing & Failover

3. How can one achieve functional partitioning for self-service modules ?
The current method of achieving load partitioning is to redirect self-service users to a specific midtier ( using a hardware load balancer ) and setting the apps_jdbc_url in the dbc file to be instance specific. Please look at MetaLink Note# 244366.1 for more information.

Cloning

1. Is Cloning supported for a 11i RAC environment ?
Yes. Cloning from RAC to RAC and cloning from RAC to non-RAC systems is supported using Rapid Clone. For a detailed list of steps and best practices, refer to the Advanced Cloning Options Section of MetaLink Note# 230672.1 Cloning Oracle Applications Release 11i with Rapid Clone

2. Is Cloning supported from Raw devices to a cooked file system if an environment needs to be cloned from RAC system to Non-RAC ?
No. This specific scenario is not currently supported with the Rapid Clone infrastructure and is considered a consulting solution. RMAN can be leveraged for the migration from Raw devices to a cooked file system

Application Specific Best Practices

1. What is the recommended value of max_commit_propagation_delay in init.ora for E-Business Suite on RAC ?
It is recommended to set set the init.ora parameter max_commit_propagation_delay= 0. For Compaq Tru64 clusters only, please set max_commit_propagation_delay= 1

2. Can I use Advanced Planning and Scheduling ( APS ) on a separate database that is a RAC cluster ?
Yes, you can. Merging APS into OLTP database and isolating the load to a separate RAC instance is supported. Refer
MetaLink Note# 279156.1 and MetaLink Note# 286729.1 for more details.

3. Can I run Email Center in a RAC environment ?
Yes, see
MetaLink Note# 272266.1 for RAC related specific instructions.

4. Can I run Oracle Financial Services Applications (OFSA) in a RAC environment ?
Yes, you can. Refer
MetaLink Note# 280294.1 for RAC related best practices.

5. Can I run ABM ( Activity Based Management ) in a RAC environment ?
Yes. ABM is supported in a RAC environment. Refer
MetaLink Note# 303542.1 for RAC related best practices.

6. Is patching in a 11i RAC environment difference from a single instance ?
No all best practices that pertain to a single instance environment can be applied for a RAC environment.


Oracle Global Single Instance ( GSI )

Configuration

1.What is the configuration of Oracle's GSI Production ?

Database Server:
Hardware: 4 node RAC, 4 Sun F25K , 36 X 1.2GHz CPUs, 144G of Memory
Storage: EMC Disk (6.4 TB)

SGA:
25.0 G total
Buffer Cache 10.0 G
Shared Pool 14.0 G

Middle Tier:
54 Linux middle tier machines
Dell 2650
2 x 3.00 GHz 6G Memory ( with hyperthreading )
NetApps Filer Cluster

Hardware Load Balancing:
BigIP

Concurrent Manager

1. What are the issues to be considered during the planning stage for concurrent manager ?
a. Are you going to have a shared code tree?
b.Are you going to have a shared log/out file system?
c. How many middle tier hosts and how are you going to distribute them across the instances?
d. Do you have programs that use UTL_FILE_DIR, this is instance specific
e. How should failover be setup in the environment ?

2. What strategies are used for managing concurrent queues in production ?

To obtain best throughput for the 100,000 requests run each day, there is a queue framework setup for "critical" workload (approx. 10% of the requests), and "non-critical" workload (approx. 90+% of the requests).

Capacity is always given first to the "critical queues", which should always process with the shortest pending times. The "non-critical" workload is managed by assigning the programs to a queue based upon their expected run-times. This framework prevents long-running requests from clogging up all the queues, which causes very short, fast requests to be stuck up in the queue for a very long time

GSI uses a few workshifts to reduce capacity given to non-critical concurrent workload during the highest on-line peaks of the global day, but all queues have some slots 24 hours / day.

Critical queues: Business determines which programs are assigned to these queues; there is a set of of queues for each module (OM, AR, AP/GL, PA, Payroll/HMRS); programs are assigned to the appropriate critical queue using specialization rules to include the specific program name. The critical queue for the application is directed to the database node appropriate to the application (e.g., OM Critical queue runs on node 1)

Non-critical queues: There is a set of 4 queues created for each database node. Programs are assigned to the appropriate queue based upon
a) the average elapsed time of program, and
b) the application identifier for the program.

Programs are manually assigned to appropriate queue by "including" the program's “request type” associated with each elapsed time threshold.

o Immediate Queues (request type='IMMEDIATE'): intended for programs whose elapsed time is always < 5 minutes

o Priority Queues (request type='PRIORITY'): intended to include programs whose avg. elapsed time is < 10 minutes (note: the Priority Queues also include the request type for the Immediate Queues)

o Standard Queues (request type='STANDARD'): intended to include programs whose avg. elapsed > 10 minutes and < 10 hours, but can also run Immediate and Priority programs; since this queue is the "default" queue, specialization rules for this queue are "EXCLUDE" only (i.e., exclude Long-running programs)
Long-running, previously called “overnight” (request type='LONGRUN') Queues: intended to include programs whose avg. elapsed > 10 hours, but also includes the request type for Immediate, Priority, and Standard)

3. How are the concurrent queues distributed across multiple instances ?
The concurrent programs are grouped by application with each application group directed to a database instance. This load direction is accomplished by defining a set of queues for each database instance. The instance a concurrent manager connects to is controlled by the environment setup on the associated concurrent processing node. Each set of managers has a set of specialization rules that "include" the request types for the application groups which are associated with the instance the manager assigned to

Storage

1. Are you using standard size devices?
Yes.

2. How many sizes do you maintain? Why ?
Three sizes are maintained:1Gb, 4Gb and 13Gb. We have found that these three sizes are optimal for the growth that we are seeing. 1Gb is for small tablespaces that don’t grow much – e.g. system. 4Gb datafiles are used for all everything else apart from the LOB tablespaces and the large tables/indexes that were moved to dedicated tablespace

3. Are there any standards used for naming datafiles and maintaining them ?
The volumes bear the same names as the datafiles so it is quite a clean configuration. We actually refer to the datafiles themselves via links. E.g. filename in dba_datafiles refers to a link, which then points to the datafile. Experience has proven that this approach is very flexible if we need to perform reorganizations and when creating copies (also known as cloning). We use the same approach with our cooked and raw datafile environments

4. How much free space do you maintain?
Tablespace are maintained with at least 5% free space – we get an automatic alert at priority two when there is less than 10% free, at priority 1 when there is less than 5% free.

5. Do you maintain spares, how many, what do you name them?
Spare volumes are created when we have less than 10 left of any size and named extra_size_XXX (XXX being a number). When these are required to be used for new datafiles they are renamed to the datafile name with the lvrename command

6. Are you using volume group striping?
Yes – groups of 8 x 68Gb disks ( software striping ) , stripe width is 1024 MB. In addition, 8-way striping at hardware level is utilized.

7. What was the process of conversion to raw devices ?
Prior to converting to raw we had established the practices of standard sized datafiles. When it came to the conversion we used dd to relocate the data, dbv to verify the copied files and once they had all been relocated the links were repointed (as described above) to point to the new locations. On large systems this can be parallelized to limit the time taken.

8. Were there any issues faced after converting to raw ?
No – there really isn’t any difference between using raw and cooked datafiles aside from the discipline needed to maintain adequate free space, and that is covered by our EM

9. What storage is being used ?
DMX3000

10. Are you using autoextend? What max size? What increment by?
We do not use auto extend as it cannot be used with Raw volumes

11. Do you have multiple dbf’s for the same tablespace? Are all autoextend on?
Yes we have multiple datafiles per tablespace, but we don’t use autoextend

Monitoring & Troubleshooting

1. What tools are used for monitoring space ?
Enterprise Manager

2. What thresholds are set for monitoring
As above – less than 10% free raises a P2 alert, less that 5% free a P1

3. What set of processes or parameters are monitored ?
Examples are various health points of each node of the database (for example, whether the database is up or down, someone holding a lock which is blocking other users, etc.). Additionally, we monitor on one of the nodes events which are common to the entire database (for example, data storage issues etc.). We can also monitor for concurrent manager events such as long-running and long-pending so that we can investigate the causes of these – they may be legitimate but it may imply issues.

4. What tools are used for troubleshooting RAC?
EM is RAC aware and can be used for both monitoring and administrative tasks. Some checks are performed just for a RAC environment – for example tablespaces filling, extents monitoring etc. We also do per-instance monitoring of session_waits, enqueues, library cache pins etc so that we can quickly start investigating if these start to increase above normal running levels.

RAC

1.How were the number of nodes in the RAC cluster calculated ?
For us this was part of a consolidation exercise. As such we calculated the anticipated load of the consolidated environments and specified two servers that could satisfy the demand.

2.How is the interconnect monitored ? What parameters are watched ?
Measureware tool is used. This collects performance data and PerfView, which reports for network traffic. Multi Router Traffic Grapher (MRTG) is used to show each cluster’s interconnect network traffic.

3.How did you implement, phased approach, node at a time? What would you do differently?
Phased approach – a node at a time. This reflects the manner in which the load increased as more and more source environments were migrated into the single instance.

Cloning

1. What sort of cloning mechanism is used ?
We generate a create controlfile script on the source environment. We then create a copy of the source environment from a BCV backup, renaming the volumes if we are doing a raw to cooked datafile conversion. This is usualy from one of our nightly hot backups. We edit the create controlfile script to reflect the new instance name and run it to create the controlfiles, and then apply any archived log files as required to open the instance

Technical References

Metalink Note# 362135.1 Configuring E-Business Suite 11i with 10g Release 2 RAC and 10g ASM

Metalink Note# 312731.1 Configuring E-Business Suite 11i with 10g Release 1RAC and 10g ASM

Metalink Note# 341437.1 Business Continuity for Oracle Applications 11i Using Oracle Real Application Clusters and Physical Standby Database

MetaLink Note# 279956.1 Oracle E-Business Suite Release 11i with 9i RAC: Installation and Configuration

MetaLink Note# 294652.1 E-Business Suite 11i on RAC : Configuring Database Load balancing & Failover

MetaLink Note# 224863.1: Troubleshooting Oracle 9i RAC & Oracle E-Business Suite

MetaLink Note# 233428.1 Sharing an Appl_TOP in Oracle Applications 11i

MetaLink Note# 217368.1 Advanced Configurations and Topologies for Enterprise Deployments of E-Business Suite 11i

1 comment: