Sunday, April 12, 2015

eBS r12 opatch issue on a cloned environment


My eBS r12 production environment is 12.1.X with a 11.2.0.3 database and using an RMAN catalog server for backup. I recently applied the CPUJan2015 to the production server after of course testing the on GOLD,GOLD and UAT no issues were detected.

When I refreshed the GOLD environment with the PROD clone I got a login page asking me to “contact the system administrator (myself)”

And error
ERROR at line 1:
ORA-29516: Aurora assertion failure: Assertion failure at eox.c:359
Uncaught exception Root of all Java exceptions: java.lang.UnsatisfiedLinkError sun.net.PortConfig.getLower0
ORA-06512: at "SYS.DBMS_JAVA", line 793
ORA-06512: at line 2

My first thought was the GUEST account issue and when I run the below SQL I got

SQL>select FND_WEB_SEC.VALIDATE_LOGIN('GUEST','ORACLE') from dual;
FND_WEB_SEC.VALIDATE_LOGIN('GUEST','ORACLE')
--------------------------------------------------------------------------------
N
1 row selected.
Surprised at the output I checked on all other environments and the output was a “Y”
So I embarked on trying to reset the GUEST password using the API format below in vain in.
[oracle@servernode applmgr]$ java oracle.apps.fnd.security.AdminAppServer APPS/"APPS Password" UPDATE GUEST_USER_PWD=GUEST/"Guest User Password" DB_HOST="Host_Name" DB_PORT="PortNumber" DB_NAME="SID"

• Check the log of FNDCPASS for any error for GUEST user if there is no error for GUEST user then do the next step

• Compile JSP
[oracle@servernode appl]$ cd $FND_TOP/patch/115/bin
[oracle@servernode bin]$ ojspCompile.pl --compile --flush

• Set GUEST_USER_PWD password in $FND_SECURE/.dbc to GUEST/ORACLE (in my case. could be different in your case)

• Bounced the Middle Tier Services and Retested the issue.

The problem was still not solved and I was back where I was started. I then realized that on the production environment where I have applied the same patches everything was running fine

Scouring the internet I landed on a blog which said

“Sometimes an application running in the Oracle JVM will fail with a java permissions error having the following format: java.sql.SQLException: ORA-29532:Java call terminated by uncaught Java exception:”

And OJVM was one of the components I had patched so I checked the OraInventory on the source the output showed the patch was well applied and to my surprise the with the output below

[oracle@SINJFMISAR002 19393542]$ opatch lsinventory -all
Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation. All rights reserved.


Oracle Home : /a01/oracle/GOLD/db/tech_st/11.2.0
Central Inventory : /a01/oracle/oraInventory
from : /a01/oracle/GOLD/db/tech_st/11.2.0/oraInst.loc
OPatch version : 11.2.0.3.4
OUI version : 11.2.0.3.0
Log file location : /a01/oracle/GOLD/db/tech_st/11.2.0/cfgtoollogs/opatch/opatch2015-03-29_13-45-48PM_1.log

List of Homes on this system:

Home name= GOLD_TOOLS__a04_applmgr_GOLD_apps_tech_st_10_1_2, Location= "/a04/applmgr/GOLD/apps/tech_st/10.1.2"
Home name= GOLD_WEBOH__a04_applmgr_GOLD_apps_tech_st_10_1_3, Location= "/a04/applmgr/GOLD/apps/tech_st/10.1.3"
Inventory load failed... OPatch cannot load inventory for the given Oracle Home.
Possible causes are:
Oracle Home dir. path does not exist in Central Inventory
Oracle Home is a symbolic link
Oracle Home inventory is corrupted
LsInventorySession failed: OracleHomeInventory gets null oracleHomeInfo

OPatch failed with error code 73

1.1. Re-create Inventory
The inventory needs to be present for the cloned database.

As Oracle
- Shutdown Oracle
. $ORACLE_HOME/$CONTEXT_NAME.env
export TMP=$HOME/tmp
cd $ORACLE_HOME/appsutil/clone
./ouicli.pl

chmod 775 /a01/oracle/oraInventory/ContentsXML
cd /a01/oracle/oraInventory/ContentsXML
chmod 775 *


- As applmgr
- Shutdown application
export TMP=$HOME/tmp

chmod +r 775 /a01/oracle/oraInventory

. $INST_TOP/ora/10.1.2/$CONTEXT_NAME.env
cd $ORACLE_HOME/appsutil/clone
./ouicli.pl

. $INST_TOP/ora/10.1.3/$CONTEXT_NAME.env
cd $ORACLE_HOME/appsutil/clone
./ouicli.pl

Validated by running “opatch lsinventory” for both oracle and applmgr user.
So I decided to repair the inventory in the below chronology

Made changes to the following file:
/a01/oracle/GOLD/db/tech_st/11.2.0/oraInst.loc
Running OUI CLI home cloning from within OracleHomeCloner:
/a01/oracle/GOLD/db/tech_st/11.2.0/oui/bin/runInstaller -clone -silent -waitForCompletion -invPtrLoc /etc/oraInst.loc ORACLE_HOME=/ a01/oracle/GOLD/db/tech_st/11.2.0 ORACLE_BASE=/a01/oracle/GOLD/db/tech_st/11.2.0 ORACLE_HOME_NAME=GOLD_DB__a01_oracle_GOLD_db_tech_st_11_ 2_0 oracle_install_OSDBA=dba oracle_install_OSOPER=dba -noConfig -nowait
Finished OUI CLI cloning for s_db_oh with return code: 0Sat Apr 11 21:43:43 2015
[oracle@Servername clone]$ chmod 775 /a01/oracle/oraInventory/ContentsXML

[oracle@Servername clone]$ cd /a01/oracle/oraInventory/ContentsXML

[oracle@Servername ContentsXML]$ chmod 775 *

[oracle@Servername ContentsXML]$ ls

comps.xml inventory.xml inventory.xml.270315 libs.xml
[oracle@Servername ContentsXML]$ opatch lsinventory

Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation. All rights reserved.
Oracle Home : /a01/oracle/GOLD/db/tech_st/11.2.0
Central Inventory : /a01/oracle/oraInventory
from : /a01/oracle/GOLD/db/tech_st/11.2.0/oraInst.loc
OPatch version : 11.2.0.3.4
OUI version : 11.2.0.3.0
Log file location : /a01/oracle/GOLD/db/tech_st/11.2.0/cfgtoollogs/opatch/opatch2015-04-11_21-52-56PM_1.log

Lsinventory Output file location : /a01/oracle/GOLD/db/tech_st/11.2.0/cfgtoollogs/opatch/lsinv/lsinventory2015-04-11_21-52-56PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (2):

Oracle Database 11g 11.2.0.3.0
Oracle Database 11g Examples 11.2.0.3.0
There are 2 products installed in this Oracle Home.

Interim patches (6) :

Patch 19121548 : applied on Tue Nov 11 11:07:01 CET 2014
Unique Patch ID: 17940820
Patch description: "Database Patch Set Update : 11.2.0.3.12 (19121548)"
Created on 10 Oct 2014, 06:27:11 hrs PST8PDT
Sub-patch 18522512; "Database Patch Set Update : 11.2.0.3.11 (18522512)"
Sub-patch 18031683; "Database Patch Set Update : 11.2.0.3.10 (18031683)"
Sub-patch 17540582; "Database Patch Set Update : 11.2.0.3.9 (17540582)"
Sub-patch 16902043; "Database Patch Set Update : 11.2.0.3.8 (16902043)"
Sub-patch 16619892; "Database Patch Set Update : 11.2.0.3.7 (16619892)"
Sub-patch 16056266; "Database Patch Set Update : 11.2.0.3.6 (16056266)"
Sub-patch 14727310; "Database Patch Set Update : 11.2.0.3.5 (14727310)"
Sub-patch 14275605; "Database Patch Set Update : 11.2.0.3.4 (14275605)"
Sub-patch 13923374; "Database Patch Set Update : 11.2.0.3.3 (13923374)"
Sub-patch 13696216; "Database Patch Set Update : 11.2.0.3.2 (13696216)"
Sub-patch 13343438; "Database Patch Set Update : 11.2.0.3.1 (13343438)"
Bugs fixed:
13593999, 10350832, 19433746, 14138130, 12919564, 14198511, 13561951
1xxxxxxxxx xxxxxx xxxxxxx xxxxxxxxxxxxxxx
12821418, 13502183, 13705338, 14237793, 16794239, 13554409, 15862024
13103913, 13645917, 12772404
Patch 13366268 : applied on Tue Jun 18 09:26:46 CEST 2013
Unique Patch ID: 14394336
Created on 21 Dec 2011, 08:14:06 hrs PST8PDT
Bugs fixed:
13366268
Patch 13258936 : applied on Tue Jun 18 09:26:18 CEST 2013
Unique Patch ID: 14465000
Created on 5 Dec 2011, 02:41:32 hrs PST8PDT
Bugs fixed:
13258936
Patch 12985184 : applied on Tue Jun 18 09:24:33 CEST 2013
Unique Patch ID: 14296756
Created on 30 Nov 2011, 15:18:06 hrs PST8PDT
Bugs fixed:
12985184
Patch 12960302 : applied on Tue Jun 18 09:23:53 CEST 2013
Unique Patch ID: 14254250
Created on 10 Nov 2011, 02:40:18 hrs PST8PDT
Bugs fixed:
12960302
Patch 4247037 : applied on Tue Jun 18 09:21:05 CEST 2013
Unique Patch ID: 14211388
Created on 17 SEP 2009, 09:52:00 hrs US/Pacific
Bugs fixed:
4247037
--------------------------------------------------------------------------------

OPatch succeeded.

I then proceeded to re-apply the CPUJan2015


Since CpuJan15 and later, Oracle JavaVM Component Database PSU is a new Patch Set Update for Database versions 11.1.0.7, 11.2.0.3, 11.2.0.4, 12.1.0.1 and 12.1.0.2. This PSU addresses Oracle JavaVM CPU program security vulnerabilities.

CPU Patch for Application

Follow below procedure to apply CPU patch to e-business suite application.
Copy/FTP both the patches to /u01/One-Off/cpuJan15/ directory.

The patches need to be applied using application owner. applmgr
1.2. Shutdown Application Service.

 Log into unix using any ssh client such as PUTTY
Login> applmgr 
Pwd> ****** 

 Stop the Application services
$> cd $HOME/bin 
$> {ENV}_APPS.stop 

 Check that unix process for applmgr are stopped
$> ps -ef |grep applmgr |egrep -v 'ps|bash' 

Once all the applmgr processes have gone, move to next step.
1.3. Enable Maintenance Mode.

Enable the maintenance mode in application by running adadmin tool

$ adadmin
- Option 5 (Change Maintenance Mode)
- Option 1 (Enable Maintenance Mode)
- Enter
- Option 3 (Return to Main Menu)
- Option 6 (Exit AD Administration)
1.4. Take Pre Invalid List.

Take a snapshot of invalid objects, which will be compared later after patch.

$ cd /u01/One-Off/cpuJan15/
$ sqlplus apps/$APPS_PWD @/home/applmgr/scripts/objinval > objinval_{ENV}.pre







1.5. Apply Patch 19393542.

This patch is opatch and needs to be applied as Oracle user on 11.2.0.3 Home
Shutdown database and listener
Run below command to verify any patch conflict regarding this patch.
$ cd /u01/One-Off/cpuJan15/19393542
$ export PATH=$ORACLE_HOME/OPatch:$PATH
$ opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./

Apply patch using opatch.
$ cd $PATCH_DIRECTORY/19393542
$ opatch apply
Output should look like this
[oracle@Servername 19393542]$ opatch apply
Oracle Interim Patch Installer version 11.2.0.3.4 Copyright (c) 2012, Oracle Corporation. All rights reserved. Oracle Home : /a01/oracle/GOLD/db/tech_st/11.2.0
Central Inventory : /a01/oracle/oraInventory
from : /a01/oracle/GOLD/db/tech_st/11.2.0/oraInst.loc
OPatch version : 11.2.0.3.4
OUI version : 11.2.0.3.0
Log file location : /a01/oracle/GOLD/db/tech_st/11.2.0/cfgtoollogs/opatch/19393542_Mar_12_2015_14_27_57/apply2015-03-12_14-27-57PM_1.log Applying interim patch '19393542' to OH '/a01/oracle/GOLD/db/tech_st/11.2.0' Verifying environment and performing prerequisite checks...
All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/a01/oracle/GOLD/db/tech_st/11.2.0') Is the local system ready for patching? [y|n] y
User Responded with: Y
Backing up files... Patching component oracle.rdbms, 11.2.0.3.0... Verifying the update...
Patch 19393542 successfully applied
Log file location: /a01/oracle/GOLD/db/tech_st/11.2.0/cfgtoollogs/opatch/19393542_Mar_12_2015_14_27_57/apply2015-03-12_14-27-57PM_1.log

OPatch succeeded.
[oracle@Servername 19393542]$

Review the log file under $ORACLE_HOME/cfgtoollogs/opatch directory for any errors.

Bring up database and listener.

1.1. Apply Patch 19873049.
As applmgr.
$ cd /u01/One-Off/cpuJan15/19873049
$ $AD_TOP/bin/admsi.pl -patch_top=/u01/One-Off/cpuJan15/19873049 -appspass=$APPS_PWD

[applmgr@Servername One-off]$ $AD_TOP/bin/admsi.pl -patch_top=/home/applmgr/One-off/19873049 -appspass=$APPS_PWD
Gathering Information..
Logfile for this session is located at admsi.log
Generating installation instructions for patch 19873049..
Updating database....
install_19873049.html generated successfully
install_19873049.txt generated successfully


As oracle

$ cd $ORACLE_HOME/appsutil/admin
$ cp -pr /a04/applmgr/GOLD/apps/apps_st/appl/admin/adgrants.sql .
$ sqlplus /nolog
SQL> connect / as sysdba
SQL> @adgrants.sql APPS
Connected.
------ adgrants.sql started at 2015-03-12 14:57:05 ---
Creating PL/SQL profiler objects.
------- profload.sql started at 2015-03-12 14:57:05 ---
Package created.
Grant succeeded.
Synonym created.
Library created.
Package body created.
Testing for correct installation
SYS.DBMS_PROFILER successfully loaded.
PL/SQL procedure successfully completed.
---- profload.sql completed at 2015-03-12 14:57:07 ---
----proftab.sql started at 2015-03-12 14:57:07 ---
Table dropped.
Table dropped.
Table dropped.
Sequence dropped.
Table created.
Comment created.
Table created.
Comment created.
Table created.
Comment created.
Sequence created.
----- profltab.sql completed at 2015-03-12 14:57:07 ---
Installing Hierarchical Profiler.
Table dropped.
Table dropped.
Table dropped.
Sequence dropped.
Table created.
Comment created.
Table created.
Comment created.
Table created.
Comment created.
Sequence created.
Package created.
Synonym created.
Grant succeeded.
Grant succeeded.
Grant succeeded.
Grant succeeded.
Start of Creating AD_JAR context
PL/SQL procedure successfully completed.
End of Creating AD_JAR context
Creating PL/SQL Package AD_DBMS_METADATA.
Package created.
Package body created.
End of Creating PL/SQL Package AD_DBMS_METADATA.
PL/SQL procedure successfully completed.
Commit complete.

As the application owner --- applmgr

$ adpatch

[applmgr@Servername 19873049]$ $AD_TOP/bin/admsi.pl -patch_top=/u01/One-Off/cpuJan15/19873049 -appspass=$APPS_PWD
Gathering Information..
Logfile for this session is located at admsi.log
Generating installation instructions for patch 19873049..
Updating database....
install_19873049.html generated successfully
install_19873049.txt generated successfully

Review the log file adpatch_19873049.log under $APPL_TOP/admin/{ENV}/log directory for any errors.
*If there there is a failure with ADFIXUSER.sql its because the patch has delivered a new version of adgrants so coppy and re-run it should run successfuly

#######################################################
ATTENTION: All workers either have failed or are waiting:
FAILED: file ADFIXUSER.sql on worker 1.
Perform the adgrants copy and re-run again.and restart the worker
#######################################################


$ $AD_TOP/bin/admsi.pl --done

Gathering Information..
Enter the patch_top : /u01/ebs_patches/FamilyPack/19278976
Enter the APPS user password:
Logfile for this session is located at admsi.log
Updating database for completion..
Done.

[applmgr@Servername 19873049]$ $AD_TOP/bin/admsi.pl --done
Gathering Information..
Enter the patch_top: /u01/One-Off/cpuJan15/19873049
Enter the APPS user password:
Logfile for this session is located at admsi.log
Updating database for completion...Done.
[applmgr@Servename 19873049]$
1.2. Run Autoconfig.
Apply autoconfig step after patch.

$ $INST_TOP/admin/scripts/adautocfg.sh appspass=$APPS_PWD
2. CPU Patch for Database
Follow below procedure to apply CPU patch to all database including RMAN server.
The patch needs to be applied as the database owner Oracle.

2.1. Stop Database Service.
 Log into unix using any ssh client such as PUTTY
Login> oracle 
Pwd> ****** 


 Stop the Application services
$> cd $HOME/bin 
$> {ENV}_DB.stop 

 Check that unix process for oracle are stopped
$> ps -ef |grep oracle |egrep -v 'ps|bash' 
2.2. Verify Patch List.
Run below command to verify current PSU patch list.
$ export PATH=$ORACLE_HOME/OPatch:$PATH
$ opatch lsinventory -bugs_fixed |grep -i "PATCH SET UPDATE"

The output will show as below
19121548 19121548 Tue Nov 11 11:07:01 CET 2014 DATABASE PATCH SET UPDATE 11.2.0.3.12 (INCLUDES CP
18522512 18522512 Tue Nov 11 10:21:00 CET 2014 DATABASE PATCH SET UPDATE 11.2.0.3.11 (INCLUDES CP
18031683 18031683 Tue Jun 24 13:33:56 CEST 2014 DATABASE PATCH SET UPDATE 11.2.0.3.10 (INCLUDES CP
17540582 17540582 Mon Mar 17 13:40:22 CET 2014 DATABASE PATCH SET UPDATE 11.2.0.3.9 (INCLUDES CPU
16902043 16902043 Mon Mar 17 13:40:09 CET 2014 DATABASE PATCH SET UPDATE 11.2.0.3.8 (INCLUDES CPU
16619892 16619892 Mon Sep 30 10:24:39 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.7 (INCLUDES CPU
16056266 16056266 Mon Sep 30 10:24:12 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.6 (INCLUDES CPU
14727310 14727310 Mon Sep 30 10:24:00 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.5 (INCLUDES CPU
14275605 14275605 Mon Sep 30 10:23:10 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.4 (INCLUDES CPU
13923374 13923374 Mon Sep 30 10:23:01 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.3 (INCLUDES
13696216 13696216 Mon Sep 30 10:22:29 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.2 (INCLUDES
13343438 13343438 Mon Sep 30 10:21:22 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.1
2.3. Verify Patch Conflict.
Run below command to verify any patch conflict regarding this patch.

$ cd /u01/One-Off/cpuJan15/20132646/19769496
$ opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./
[oracle@SINJFMISAR003 19769496]$ opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./
Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /a01/oracle/GOLD/db/tech_st/11.2.0
Central Inventory: /a01/oracle/oraInventory
from : /a01/oracle/GOLD/db/tech_st/11.2.0/oraInst.loc
OPatch version : 11.2.0.3.4
OUI version : 11.2.0.3.0
Log file location: /a01/oracle/GOLD/db/tech_st/11.2.0/cfgtoollogs/opatch/opatch2015-03-13_12-24-20PM_1.log
Invoking prereq "checkconflictagainstohwithdetail"
Prereq "checkConflictAgainstOHWithDetail" passed.
OPatch succeeded.

This should give no patch conflict.

Prereq "checkConflictAgainstOHWithDetail" passed.
2.4. How to correct inventory if corrupt
At times the Inventory location can become corrupt or might have changed due to architecture due to different file systems etc. this will cause Opatch to error out with error code 104 and can hamper the database patch application.

To fix a corrupted oracle inventory implicates at times moving it to a different location or editing Contents of the XML and changing the ORACLE_HOME_NAME

[oracle@SINJFMISAR002 bin]$ ./runInstaller -silent -invPtrLoc "/a01/oracle/GOLD/db/tech_st/11.2.0/oraInst.loc" -attachHome ORACLE_HOME="/a01/oracle/GOLD/db/tech_st/11.2.0" ORACLE_HOME_NAME="Ora11g_home1"
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB. Actual 17407 MB Passed
The inventory pointer is located at /a01/oracle/GOLD/db/tech_st/11.2.0/oraInst.loc
The inventory is located at /a01/oracle/oraInventory
'AttachHome' was successful.
2.5. Apply Patch 19769496
Run below command to verify any patch conflict regarding this patch.
$ cd /u01/One-Off/cpuJan15/20132646/19769496
$ opatch apply
Warning:
OPatch found the word "warning" in the stderr of the make command.
Please look at this stderr. You can re-run this make command.
Stderr output:
ins_precomp.mk:19: warning: overriding commands for target `pcscfg.cfg'
/a01/oracle/GOLD/db/tech_st/11.2.0/precomp/lib/env_precomp.mk:2160: warning: ignoring old commands for target `pcscfg.cfg'
/a01/oracle/GOLD/db/tech_st/11.2.0/precomp/lib/ins_precomp.mk:19: warning: overriding commands for target `pcscfg.cfg'
/a01/oracle/GOLD/db/tech_st/11.2.0/precomp/lib/env_precomp.mk:2160: warning: ignoring old commands for target `pcscfg.cfg'

Solution: Ignore - Opatch warning: overriding commands for target xxxx (Doc ID 1448337.1)
2.6. Verify Patch List.
Run below command to verify current PSU patch list.
$ export PATH=$ORACLE_HOME/OPatch:$PATH
$ opatch lsinventory -bugs_fixed |grep -i "PATCH SET UPDATE"

The output will show as below
19769496 19769496 Tue Jan 27 15:51:30 CET 2015 DATABASE PATCH SET UPDATE 11.2.0.3.13 (INCLUDES CP
19121548 19121548 Tue Nov 11 11:07:01 CET 2014 DATABASE PATCH SET UPDATE 11.2.0.3.12 (INCLUDES CP
18522512 18522512 Tue Nov 11 10:21:00 CET 2014 DATABASE PATCH SET UPDATE 11.2.0.3.11 (INCLUDES CP
18031683 18031683 Tue Jun 24 13:33:56 CEST 2014 DATABASE PATCH SET UPDATE 11.2.0.3.10 (INCLUDES CP
17540582 17540582 Mon Mar 17 13:40:22 CET 2014 DATABASE PATCH SET UPDATE 11.2.0.3.9 (INCLUDES CPU
16902043 16902043 Mon Mar 17 13:40:09 CET 2014 DATABASE PATCH SET UPDATE 11.2.0.3.8 (INCLUDES CPU
16619892 16619892 Mon Sep 30 10:24:39 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.7 (INCLUDES CPU
16056266 16056266 Mon Sep 30 10:24:12 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.6 (INCLUDES CPU
14727310 14727310 Mon Sep 30 10:24:00 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.5 (INCLUDES CPU
14275605 14275605 Mon Sep 30 10:23:10 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.4 (INCLUDES CPU
13923374 13923374 Mon Sep 30 10:23:01 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.3 (INCLUDES
13696216 13696216 Mon Sep 30 10:22:29 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.2 (INCLUDES
13343438 13343438 Mon Sep 30 10:21:22 CEST 2013 DATABASE PATCH SET UPDATE 11.2.0.3.1
2.7. Load Modified SQL Files.
Load modified sql files into the database.
$ cd $ORACLE_HOME/rdbms/admin
$ sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> STARTUP
SQL> @catbundle.sql psu apply
SQL> QUIT
Review the log files under $ORACLE_HOME/cfgtoollogs/catbundle
2.8. Verify The Patch.
Load modified sql files into the database.

$ sqlplus /nolog
SQL> connect /as sysdba
Connected.
SQL> set lines 200
col ACTION format a15
col NAMESPACE format a10
col version format a10
col COMMENTS format a25
col BUNDLE_SERIES format a13
col action_time format a30
select * from sys.registry$history;

The output is shown below.

ACTION_TIME ACTION NAMESPACE VERSION ID COMMENTS BUNDLE_SERIES
------------------------------ --------------- ---------- ---------- ---------- ------------
18-JUN-13 12.51.49.442929 PM VIEW INVALIDATE 8289601 view invalidation
18-JUN-13 12.51.52.381437 PM UPGRADE SERVER 11.2.0.3.0 Upgraded from 11.1.0.7.0
18-JUN-13 12.57.57.918181 PM APPLY SERVER 11.2.0.3 0 Patchset 11.2.0.2.0 PSU
23-JAN-08 04.07.02.182324 PM CPU SERVER 10.2.0.3.0 6079591 CPUJul2007
26-NOV-08 11.39.09.067225 PM UPGRADE SERVER 11.1.0.6.0 Upgraded from 10.2.0.3.0
22-JAN-09 04.34.49.320298 AM UPGRADE SERVER 11.1.0.7.0 Upgraded from 11.1.0.6.0
17-MAR-14 01.49.21.350997 PM APPLY SERVER 11.2.0.3 9 PSU 11.2.0.3.9 PSU
24-JUN-14 01.35.47.421582 PM APPLY SERVER 11.2.0.3 10 PSU 11.2.0.3.10 PSU
11-NOV-14 10.24.04.404791 AM APPLY SERVER 11.2.0.3 11 PSU 11.2.0.3.11 PSU
11-NOV-14 11.09.18.638470 AM APPLY SERVER 11.2.0.3 12 PSU 11.2.0.3.12 PSU
27-JAN-15 03.56.56.396273 PM APPLY SERVER 11.2.0.3 13 PSU 11.2.0.3.13 PSU
2.9. Upgrade RMAN Catalog
Upgrade the catalog for rman.

Note: ONLY ONCE DONE IN FOR PROD as only PROD has RMAN:

$ rman catalog username/password@alias
RMAN> UPGRADE CATALOG;
3. OJVM Patch for Database
Follow below procedure to apply OJVM patch to all database including RMAN server.

The patches need to be applied using as the database owner (oracle).

3.1. Shutdown Database
Shutdown the database for applying OJVM patch
3.2. Verify Patch Conflict.
Run below command to verify any patch conflict regarding this patch.

$ cd /u01/One-Off/cpuJan15/20132646/19877443
$ opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./
Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation. All rights reserved.
PREREQ session

Oracle Home : /a01/oracle/GOLD/db/tech_st/11.2.0
Central Inventory : /a01/oracle/oraInventory
from : /a01/oracle/GOLD/db/tech_st/11.2.0/oraInst.loc
OPatch version : 11.2.0.3.4
OUI version : 11.2.0.3.0
Log file location : /a01/oracle/GOLD/db/tech_st/11.2.0/cfgtoollogs/opatch/opatch2015-03-29_16-21-02PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"
Prereq "checkConflictAgainstOHWithDetail" passed.
OPatch succeeded.

This should give no patch conflict.

Prereq "checkConflictAgainstOHWithDetail" passed.
3.3. Apply Patch 19877443

Run below command to verify any patch conflict regarding this patch.

$ cd /u01/One-Off/cpuJan15/20132646/19877443
[oracle@Servername 19877443]$ opatch apply
Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation. All rights reserved.


Oracle Home : /a01/oracle/GOLD/db/tech_st/11.2.0
Central Inventory : /a01/oracle/oraInventory
from : /a01/oracle/GOLD/db/tech_st/11.2.0/oraInst.loc
OPatch version : 11.2.0.3.4
OUI version : 11.2.0.3.0
Log file location : /a01/oracle/GOLD/db/tech_st/11.2.0/cfgtoollogs/opatch/19877443_Mar_29_2015_16_21_50/apply2015-03-29_16-21-50PM_1.log

Applying interim patch '19877443' to OH '/a01/oracle/GOLD/db/tech_st/11.2.0'
Verifying environment and performing prerequisite checks...
Patch 19877443: Optional component(s) missing : [ oracle.sqlj, 11.2.0.3.0 ]
All checks passed.
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: Y

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/a01/oracle/GOLD/db/tech_st/11.2.0')

Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Patching component oracle.javavm.server, 11.2.0.3.0...
Patching component oracle.rdbms, 11.2.0.3.0...
Patching component oracle.rdbms.dbscripts, 11.2.0.3.0...
Patching component oracle.sqlj.companion, 11.2.0.3.0...
Patching component oracle.dbjava.ic, 11.2.0.3.0...
Patching component oracle.dbjava.jdbc, 11.2.0.3.0...
Verifying the update...
--------------------------------------------------------------------------------
***********************************************************************************
***********************************************************************************
** ATTENTION **
** **
** It is critical and essential to complete the post installation (if you applying ** the patch) or post deinstallation (if you are doing rollback of the patch) as
** per Readme to have a working and functional Oracle JavaVM Component. **
** Failure to follow the instructions could lead to inconsistencies in the database. **
***********************************************************************************
***********************************************************************************
--------------------------------------------------------------------------------
Patch 19877443 successfully applied
Log file location: /a01/oracle/GOLD/db/tech_st/11.2.0/cfgtoollogs/opatch/19877443_Mar_29_2015_16_21_50/apply2015-03-29_16-21-50PM_1.log

OPatch succeeded.

3.4. Verify Patch List.
Run below command to verify current PSU patch list.

$ opatch lsinventory -bugs_fixed |grep -i "ORACLE JAVAVM COMPONENT 11.2"

The output will show as below
[oracle@SINJFMISAR002 19877443]$ opatch lsinventory -bugs_fixed |grep -i "ORACLE JAVAVM COMPONENT 11.2"
19852361 19877443 Sun Mar 29 16:22:52 CEST 2015 ORACLE JAVAVM COMPONENT 11.2.0.3.1 DATABASE PSU -
19877443 19877443 Sun Mar 29 16:22:52 CEST 2015 ORACLE JAVAVM COMPONENT 11.2.0.3.2 DATABASE PSU (J

3.5. Run PostInstall SQL File.
Load modified sql files into the database.

$ cd $ORACLE_HOME/sqlpatch/19877443
$ sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> STARTUP
SQL> @postinstall.sql
SQL> QUIT
3.6. Run Compile.
Load modified sql files into the database.

$ cd $ORACLE_HOME/rdbms/admin
$ sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> @utlrp
SQL> QUIT

3.7. Start Database Listener
 Log into unix using any ssh client such as PUTTY
Login> oracle 
Pwd> ****** 
 Start the database listener
$> cd $HOME/bin 
$> lsnrctl start {ENV} 
3.8. Take Post Invalid List.

Take a snapshot of invalid objects, which will be compared later after patch.
$ cd /u01/One-Off/cpuJan15/
$ sqlplus apps/$APPS_PWD @/home/applmgr/scripts/objinval > objinval_{ENV}.post
4. Post Patch Steps.
4.1. Disable Maintenance Mode
Disable the maintenance mode in application by running adamin tool
$ adadmin
- Option 5 (Change Maintenance Mode)
- Option 2 (Disable Maintenance Mode)
- Enter
- Option 3 (Return to Main Menu)
- Option 6 (Exit AD Administration)
Change Maintenance Mode
----------------------------------------
Maintenance Mode is currently: [Enabled].
Maintenance mode should normally be enabled when patching
Oracle Applications and disabled when users are logged on
to the system. See the Oracle Applications Maintenance
Utilities manual for more information about maintenance mode.
Please select an option:
1. Enable Maintenance Mode
2. Disable Maintenance Mode
3. Return to Main Menu
Enter your choice [3] : 2
sqlplus -s &un_apps/***** @/a04/applmgr/GOLD/apps/apps_st/appl/ad/12.0.0/patch/115/sql/adsetmmd.sql DISABLE

Successfully disabled Maintenance Mode.

4.2. Start Application Service.
 Log into unix using any ssh client such as PUTTY
Login> applmgr 
Pwd> ****** 
 Stop the Application services
$> cd $HOME/bin 
$> {ENV}_APPS.start 
 Check that unix process for applmgr are started
$> ps -ef |grep applmgr 
4.3. Validation
Validate application by logging into application and running Active User Report.
5. APPENDIX

Monday, June 11, 2012

CONTENT
OEM GRID CONTROL 11GR1 INSTALLATION ON REDHAT LINUX 5.5 (64-BIT) 3
FILE PARTITIONS 3
MIDDLEWARE ORACLE WEBLOGIC SERVER 11GR1 (10.3.2) INSTALLATION 3
ORACLE DATABASE 11G R2 FOR LINUX 64-BIT INSTALLATION 12
DROP THE OEM REPOSITORY 26
DATABASE PARAMETER CHANGE 27
APPLY LATEST VERSION OPATCH - 27
JAVA SE DEVELOPMENT KIT 6U18 29
ENTERPRISE MANAGER GRID CONTROL 11G INSTALLATION 29

OEM GRID CONTROL 11GR1 INSTALLATION ON REDHAT LINUX 5.5 (64-BIT)
During this installation I will use a HP with 2G RAM and 114G disk space. The swap size was set at 5G and the firewall and SELinux were disabled.

File partitions

• /u01 34G Oracle executables
• /u02 39G Oracle datafiles
• /u03 41G Archives logs


Start the X windows server in my case here Xming server

applprod@gvaoragrid:~ $ export DISPLAY=10.41.10.18:0.0
Middleware Oracle WebLogic Server 11gR1 (10.3.2) Installation
Grid Control 11g Release 1 no longer bundles the middleware components, so we must install the WebLogic Server before we can proceed with the Grid Control installation. The installation is started with the following command, run as the "applprod" user.
gvaoragrid(gshelton):/u03/GRID/Software/GRID2011/Weblogic
$ java -jar wls1032_generic.jar
Click the "Next" button on the welcome screen.

Enter "/u01/GRID/apps/product/Middleware" as the "Middleware Home Directory" and click the "Next" button.

Enter your Oracle Support details, or uncheck the checkbox and click the "Next" button.

If a warning dialog appears, click the "Yes" button, followed by the "Next" button from the previous screen.

Accept the "Typical" installation by clicking the "Next" button.

Accept the default JDK by clicking the "Next" button.

Accept the default "Product Installation Directories" by clicking the "Next" button.

Click the "Next" button on the summary "Installation Summary" screen.

Wait while the installation takes place.


Uncheck the "Run Quickstart" checkbox and click the "Done" button.



Next we must apply the "WDJ7" patch to the installation by running the Smart Update utility.
$applprod@gvaoragrid:/u01/GRID/apps/product/Middleware/utils/bsu $ ./bsu.sh
/u01/app/oracle/Middleware/utils/bsu/bsu.sh
Update the patch client by clicking the "OK" button on the message dialog.



Enter your support crednetials, then click the "Login" button.



These are the Metalink credentilas for downloading the updates



If you have chosen not to be updated with security information you may have to cycle through the warning message several times, but it will eventually continue. On Smart Update screen, click the "Get Patches" tab, select the "WDJ7" patch and click the "Download Selected" button and the "OK" button on the subsequent dialog.



Accept the default options and click okay


Click on the "Manage Patches" tab and click on the "Apply" button next to the "WDJ7" patch. Click "OK" on the subsequent validation dialog.






The patches are now listed under the "Default" tab.



Exit the Smart Update dialog, and the subsequent available maintenance screen if it appears.


Oracle Database 11g R2 for Linux 64-bit Installation
Unlike previous Grid Control installations, the database is no longer bundled, so we must install the database software separately. To accomplish this I followed the Oracle Database 11g Release 2 installation instructions here.

In addition to the packages listed in the DB installation, the following packages must be installed as the "root" user for WebLogic and the Grid Control to install correctly.
# From Enterprise Linux 5 DVD
cd /media/cdrom/Server
rpm -Uvh compat-db-4*
rpm -Uvh compat-libstdc++-296*
rpm -Uvh rng-utils-2*
rpm -Uvh setarch-2*
cd /
eject
applprod@gvaoragrid:/u03/GRID/Software/GRID2011/Oracle_Database_11gR2_for_Linux_x86-64/database $ ./runInstaller

Confiture the Metalink credentials



Choose the option for fresh install of the database



Choose server class


Single instance installation



I choose the advanced install to have all options













Choose the oracle Inventory directory here I put it in the Oracle Owner’s home
/home/applprod/


















If you choose a simple password you might encounter

[ins-30011] The password entered does not conform to the Oracle recommended standards.
Are you sure you want to continue choose Yes as you can harden the passwords later



All the operating systems Oracle recommended rpm’s were validated except the userid so I checked ignore


The install summary looks fine, you can create a response file for later silent install by clicking the save response file button







Creating the datafile and the creating the objects







root@gvaoragrid:/home/applprod/oraInventory # ls
root@gvaoragrid:/home/applprod/oraInventory # ./orainstRoot.sh
Changing permissions of /home/applprod/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /home/applprod/oraInventory to dba.
The execution of the script is complete.
root@gvaoragrid:/home/applprod/oraInventory #


root@gvaoragrid:/u01/GRID/apps/product/11.2.0 # ./root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
ORACLE_OWNER= applprod
ORACLE_HOME= /u01/GRID/apps/product/11.2.0

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y
Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y
Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y
Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Finished product-specific root actions.



Drop the OEM repository
I performed the default installation you will notice the EM was created so I need to deconfigure Enterprise Manager Database Control. Run the following command as the "applprod" user.
$ applprod@gvaoragrid:/u01/GRID/apps/product/11.2.0/bin
$emca -deconfig dbcontrol db -repos drop -SYS_PWD -SYSMAN_PWD
$ emca -deconfig dbcontrol db -repos drop -SYS_PWD grid2011 -SYSMAN_PWD grid2011
applprod@gvaoragrid:/u01/GRID/apps/product/11.2.0/bin $ ./emca -deconfig dbcontrol db -repos drop -SYS_PWD grid2011 -SYSMAN_PWD grid2011

STARTED EMCA at Mar 24, 2011 2:29:20 PM
EM Configuration Assistant, Version 11.2.0.0.2 Production
Copyright (c) 2003, 2005, Oracle. All rights reserved.

Enter the following information:
Database SID: GRID
Listener port number: 1521

Do you wish to continue? [yes(Y)/no(N)]: Y
Mar 24, 2011 2:41:36 PM oracle.sysman.emcp.EMConfig perform
INFO: This operation is being logged at /u01/GRID/apps/product/cfgtoollogs/emca/GRID/emca_2011_03_24_14_29_20.log.
Mar 24, 2011 2:41:36 PM oracle.sysman.emcp.util.DBControlUtil stopOMS
INFO: Stopping Database Control (this may take a while) ...
Mar 24, 2011 2:41:59 PM oracle.sysman.emcp.EMReposConfig invoke
INFO: Dropping the EM repository (this may take a while) ...
Mar 24, 2011 2:45:11 PM oracle.sysman.emcp.EMReposConfig invoke
INFO: Repository successfully dropped
Enterprise Manager configuration completed successfully
FINISHED EMCA at Mar 24, 2011 2:45:14 PM
You have new mail in /var/spool/mail/applprod
Database parameter change

Make the following initialization parameter changes.
sqlplus / AS SYSDBA

ALTER SYSTEM SET log_buffer=10485760 SCOPE=SPFILE;
ALTER SYSTEM SET processes=500 SCOPE=SPFILE;
ALTER SYSTEM SET session_cached_cursors=200 SCOPE=SPFILE;
If you have done a default installation as described here, the UNDO tablespace will be autoextensible. If you have done a custom installation make sure the UNDO tablespace is at least 200M in size.
Alter some table space sizes log on as sysdba

SQL> ALTER DATABASE DATAFILE '/u02/GRID/dbf/GRID/undotbs01.dbf' RESIZE 400M;
SQL> ALTER DATABASE DATAFILE '/u02/GRID/dbf/GRID/system01.dbf' RESIZE 2048M;
SQL> ALTER DATABASE DATAFILE '/u02/GRID/dbf/GRID/sysaux01.dbf' RESIZE 1024M;

Apply Latest version opatch -
Next we must install the latest version of the "opatch" utility.
$ applprod@gvaoragrid: cd /u03/GRID/Software/GRID2011/
$ cp unzip -o p6880880_112000_Linux-x86-64.zip /u01/GRID/apps/product/11.2.0
$ applprod@gvaoragrid: unzip -o p6880880_112000_Linux-x86-64.zip
Sanity Check for the latest Opatch
applprod@gvaoragrid:/u03/GRID/Software/GRID2011 $ opatch lsinventory
Invoking OPatch 11.2.0.1.4
Oracle Interim Patch Installer version 11.2.0.1.4
Copyright (c) 2010, Oracle Corporation. All rights reserved.
Oracle Home : /u01/GRID/apps/product/11.2.0
Central Inventory : /home/applprod/oraInventory
from : /etc/oraInst.loc
OPatch version : 11.2.0.1.4
OUI version : 11.2.0.1.0
OUI location : /u01/GRID/apps/product/11.2.0/oui
Log file location : /u01/GRID/apps/product/11.2.0/cfgtoollogs/opatch/opatch2011-03-24_16-37-49PM.log

Patch history file: /u01/GRID/apps/product/11.2.0/cfgtoollogs/opatch/opatch_history.txt
Lsinventory Output file location : /u01/GRID/apps/product/11.2.0/cfgtoollogs/opatch/lsinv/lsinventory2011-03-24_16-37-49PM.txt

Installed Top-level Products (1):
Oracle Database 11g 11.2.0.1.0
There are 1 products installed in this Oracle Home.
There are no Interim patches installed in this Oracle Home.

OPatch succeeded.

The documentation states that patches for database bugs 9002336 and 9067282 must be applied. Patch 9002336 has a prerequisite of patch 9352237 (a PSU), which includes patch 9067282, so I have only applied patches 9352237 and 9002336, which can be downloaded from My Oracle Support. The database patches include installation instructions, but to summarize, these commands should be run as the "applprod" user.
$ dbshut $ORACLE_HOME
$ export PATH=$PATH:$ORACLE_HOME/OPatch
$ cd /u03/GRID/Software/GRID2011/

$ unzip p9352237_112010_Linux-x86-64.zip
$ cd 9352237
$ opatch apply
$ Verifying the update...
Inventory check OK: Patch ID 9352237 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 9352237 are present in Oracle Home.
********************************************************************************
********************************************************************************
** ATTENTION **
** **
** Please note that the Patch Set Update Installation (PSU Deinstallation) **
** is not complete until all the Post Installation (Post Deinstallation) **
** instructions noted in the Readme accompanying this PSU, have been **
** successfully completed. **
** **
********************************************************************************
********************************************************************************
Execution of 'sh /u03/GRID/Software/GRID2011/9352237/custom/scripts/post -apply 9352237 ':
Return Code = 0
The local system has been patched and can be restarted.
OPatch succeeded.

$ unzip p9002336_112011_Linux-x86-64.zip
$ cd 9002336
$ opatch apply
Patching component oracle.rdbms, 11.2.0.1.0...
Updating archive file "/u01/GRID/apps/product/11.2.0/lib/libserver11.a" with "lib/libserver11.a/kkqtutl.o"
Running make for target ioracle
ApplySession adding interim patch '9002336' to inventory
Verifying the update...
Inventory check OK: Patch ID 9002336 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 9002336 are present in Oracle Home.
The local system has been patched and can be restarted.
OPatch succeeded.


$ dbstart $ORACLE_HOME
The database installation is now complete.


Java SE Development Kit 6u18

Java Installation
Install Java JDK 6u18 using the following commands as the "root" user. Later version of the JDK cause errors when patching WebLogic (unless you are using a proxy), so the version of the JDK is very important.
# chmod u+x jdk-6u18-linux-x64-rpm.bin
# ./jdk-6u18-linux-x64-rpm.bin

Enterprise Manager Grid Control 11g Installation

Unzip the Grid control software into an empty directory.
$ /u03/GRID/Software/GRID2011
$ unzip GridControl_11.1.0.1.0_Linux_x86-64_1of3.zip
$ unzip GridControl_11.1.0.1.0_Linux_x86-64_2of3.zip
$ unzip GridControl_11.1.0.1.0_Linux_x86-64_3of3.zip
Start the installer as the "applprod" user.
applprod@gvaoragrid:/u03/GRID/Software/GRID2011/Oracle_Enterprise_Manager_11g_Grid_ControlRelease1_11.1.0.1.0_64Bits
$ ./runInstaller
Enter your support details, or uncheck the notification checkbox, then click the "Next" button. Click the "Yes" button on any subsequent warning dialogs.



Select the "Download and install updates from My Oracle Support" option and enter your support details, then click the "Next" button.



Wait while the check is performed, then click the "Download Updates" button on the subsequent dialog and wait for the updates to be downloaded.



Accept the default "Apply Software Updates" option by clicking the "Next" button.



Accept the default "Install a new Enterprise Manager system" option by clicking the "Next" button.



Wait while the prerequisite checks are performed. If any fail, correct them before proceeding by clicking the "Next" button.

I check the ignore all so that the highlighted memory check does not stop the installation



Enter "/u01/GRID/apps/product/Middleware/oms11g" as the OMS instance base location, then click the "Next" button.




Enter password for the WebLogic domain and node manager users, then click the "Next" button.





Enter the database connection details and click the "Next" button.



Enter password details for the "SYSMAN" database user and click the "Next" button.



Enter the password for secure agent connections, then click the "Next" button.



Amend the ports as you see fit, then click the "Next" button.



If you are happy with the information on the "Review" screen, click the "Next" button.



Wait while the installation takes place.



Run the specified script, then click the "OK" button.


root@gvaoragrid:/u01/GRID/apps/product/Middleware/oms11g # ./allroot.sh

Starting to execute allroot.sh .........

Starting to execute /u01/GRID/apps/product/Middleware/oms11g/root.sh ......
Running Oracle 11g root.sh script...

The following environment variables are set as:
ORACLE_OWNER= applprod
ORACLE_HOME= /u01/GRID/apps/product/Middleware/oms11g

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)[n]: y
Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)[n]: y
Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)[n]: y
Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Adding entry to /etc/oratab file...
Finished execution of /u01/GRID/apps/product/Middleware/oms11g/root.sh ......


Starting to execute /u01/GRID/apps/product/Middleware/agent11g/root.sh ......
Running Oracle 11g root.sh script...

The following environment variables are set as:
ORACLE_OWNER= applprod
ORACLE_HOME= /u01/GRID/apps/product/Middleware/agent11g

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)[n]: y
Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)[n]: y
Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)[n]: y
Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
Adding entry to /etc/oratab file...
Finished execution of /u01/GRID/apps/product/Middleware/agent11g/root.sh ......
Wait while the configuration steps take place.





Make a note of the URLs on the "Finish" screen, then click the "Close" button.



Connect to the Grid Control console (https://gvaoragrid.temenosgroup.com:7799/em) using the username (SYSMAN) and password defined during the installation, accept the license agreement and you are presented with the console.


Startup/Shutdown
Use the following commands to turn on all components installed by this article.
#!/bin/bash
export OMS_HOME=/u01/GRID/apps/product/Middleware/oms11g
export AGENT_HOME=/u01/GRID/apps/product/Middleware/agent11g

# Start everything
dbstart $ORACLE_HOME

$OMS_HOME/bin/emctl start oms

$AGENT_HOME/bin/emctl start agent
Use the following commands to turn off all components installed by this article.
#!/bin/bash
export OMS_HOME=/u01/GRID/apps/product/Middleware/oms11g
export AGENT_HOME=/u01/GRID/apps/product/Middleware/agent11g

# Stop everything
$OMS_HOME/bin/emctl stop oms -all

$AGENT_HOME/bin/emctl stop agent

dbshut $ORACLE_HOME
Its all about FNDLOAD
What is Data Synchronization ?
Data Synchronization is a process in which some setup data would be synchronized, and this would be more important when you are working in oracle application development/implementation project? The equally important that AOL data Migration takes place necessary to synchronize the data across databases instance during
• Installations (New desc. flex field creations etc)
• Upgrades (Apps upgrade etc)
• Maintenance (Value set changes etc)
Then… What.. FNDLOAD
Yes, that is one and only oracle solution. It can be defined as a concurrent program that can move Oracle Applications data between database and text file representations. Or this can be defined as FNDLOAD can download data from an application entity into an editable text file, which can be uploaded to another database.
How it works
Conversion between database format and text file format is specified by a configuration file. Oracle does provide confirmation file and it is important to just pass the configuration file name and then just call the loader and leave everything .The schematic visual representation of the loader is like:

What can be done?
These are the extensive list which can be done through FNDLOAD
• Concurrent Programs, Executables
• Request Groups, Request Sets
• Profile Options
• Key and Descriptive Flexfields
• Menus and Responsibilities
• Forms and Form Functions
• Attachments
• Messages
• Value Sets and Values
• Lookup Types
• User Responsibilities
• Printer Definitions
• FND Dictionary
• Help Configuration
• Document Sequences
• Concurrent Manager Schedules
Then…what are advantages when using FNDLOAD are :
1. Because downloaded data is stored in a text file, version administration is possible
2. There is nothing to worry to go to purchase because..Investment = 0$
3. No learning curve. this is relief for developer/dba’s
4. Fully supported and recommended by Oracle
5. Capture the migrations in a file and use it during installations, clones etc. to migrate in batch
6. Pin-point when something happened and where (database) easily
7. Your AOL data migration process is now simplified and streamlined – goal attained!!
And these are some Disadvantages also :
1. Applications patching mechanisms use FNDLOAD heavily – possibility of negative impact is not zero
2. UPLOAD_MODE=REPLACE only for menus
3. No validation against migrating database/instance sensitive data
The Syntax
To use FNDLOAD, the following syntax is needed.
FNDLOAD apps/appspwd 0 Y mode configfile datafile entity [parameter1…..]
As you can see, some info is needed.- First you will need to know the Apps password.
• The mode is either DOWNLOAD or UPLOAD.
• The configfile is the file that Fndload needs to download on upload data.
• T he data file is the output file, in which the downloaded data is written
• The entity is the entity you want to download,
• Parameters van is passed to download a certain Alert
Modes of Operation
This is important because it would drive the whole flow, and it always be two mode .. Upload and Download…
Example of download:
FNDLOADapps/pwd 0 Y DOWNLOAD ${FND_TOP}/patch/115/import/afcpprog.lct myfile.ldt \ PROGRAM CONCURRENT_PROGRAM_NAME= concurrent_program_short_name> APPLICATION_SHORT_NAME=
Example of Upload
FNDLOAD apps/pwd 0 Y UPLOAD ${FND_TOP}/patch/115/import/afcpprog.lct myfile.ldt - CUSTOM_MODE=FORCE undocumented parameter
WHAT IS FNDLOADLoad Options:
• Application level OR single entity level download
o (Example) Download all the profile options of Inventory or just the INV: Default Primary UOM
• Entire OR Partial upload of a data file
o (Example) Upload the entire myfile.ldt or just a single enity indicated by - and mode UPLOAD or UPLOAD_PARTIAL
o Entity name required for UPLOAD_PARTIAL mode
• Custom mode force update
o To override Oracle's upload algorithm and update the custom AOL data regardless, use CUSTOM_MODE= FORCE
o UPLOAD_MODE= REPLACE (only for menus)
• Support for NLS uploads
o NLS data uploads for translated columns are supported, use UPLOAD_MODE= NLS
Then ..Where is Config File Located
By default Oracle delivers most of configuration files you can use to download certain entities.
• Configuration files with extension .lct
o On Unix - all the configuration files are in $FND_TOP/patch/115/import directory
o On Unix – Oracle also places the original configuration files in $FND_TOP/admin/import directory
• Data files with extension .ldt
and..FNDLOAD File Structures is
• The config files (.lct) are delivered and maintained by Oracle
• It has entity definitions, parent-child relationships and user input parameters identified by :NAME
o Downloading a parent automatically downloads all children - (Example) Concurrent Program download
Take a note FNDLOAD File Structures
• The data files (.ldt) have both entity definition and the data
• It also shows the version and the location of the config file (.lct) that was used
• Without the config file, data file is useless
• Without the data file, config file is meaningless
..Remember regarding FNDLOAD Files
• Key files: .lct and .ldt
• You must run the FNDLOADas apps user not as applsys or any other, otherwise you will receive Ora-6550
• Both are easily readable, editable and portable
• DO NOT MODIFY Oracle’s .lct files
• Use your favorite editor (I would suggest Textpad or Editplus) to manipulate only the .ldt files but be cautious about data type, length, delimiter placements etc.
• Use the log file outputs or .ldt file contents creatively for quick file comparisons and answer questions faster (Why can’t I access that? What is that profile option name, value and level? What is the value set used for that DFF segment attribute10 etc.)
• Partial string searches (which value set has Priority “something” in its where clause? etc)
Sample Script Code for these Objects :
1 - Printer Styles
FNDLOAD apps/apps@seed115 O Y DOWNLOAD $FND_TOP/patch/115/import/afcppstl.lct file_name.ldt STYLE PRINTER_STYLE_NAME="printer style name"
2 - Lookups
FNDLOAD apps/apps@seed115 O Y DOWNLOAD $FND_TOP/patch/115/import/aflvmlu.lct file_name.ldt FND_LOOKUP_TYPE APPLICATION_SHORT_NAME="prod" LOOKUP_TYPE="lookup name"
3 - Descriptive Flexfield with all of specific Contexts
FNDLOAD apps/apps@seed115 O Y DOWNLOAD $FND_TOP/patch/115/import/afffload.lct file_name.ldt DESC_FLEX P_LEVEL=?COL_ALL:REF_ALL:CTX_ONE:SEG_ALL? APPLICATION_SHORT_NAME="prod" DESCRIPTIVE_FLEXFIELD_NAME="desc flex name" P_CONTEXT_CODE="context name"
4 - Key Flexfield Structures
FNDLOAD apps/apps@seed115 O Y DOWNLOAD $FND_TOP/patch/115/import/afffload.lct file_name.ldt KEY_FLEX P_LEVEL=?COL_ALL:FQL_ALL:SQL_ALL:STR_ONE:WFP_ALL:SHA_ALL:CVR_ALL:SEG_ALL? APPLICATION_SHORT_NAME="prod" ID_FLEX_CODE="key flex code" P_STRUCTURE_CODE="structure name"
5 - Concurrent Programs
FNDLOAD apps/apps@seed115 O Y DOWNLOAD $FND_TOP/patch/115/import/afcpprog.lct file_name.ldt PROGRAM APPLICATION_SHORT_NAME="prod" CONCURRENT_PROGRAM_NAME="concurrent name"
6 - Value Sets
FNDLOAD apps/apps@seed115 O Y DOWNLOAD $FND_TOP/patch/115/import/afffload.lct file_name.ldt VALUE_SET FLEX_VALUE_SET_NAME="value set name"
7 - Value Sets with values
FNDLOAD apps/apps@seed115 O Y DOWNLOAD $FND_TOP/patch/115/import/afffload.lct file_name.ldt VALUE_SET_VALUE FLEX_VALUE_SET_NAME="value set name"
8 - Profile Options
FNDLOAD apps/apps@seed115 O Y DOWNLOAD $FND_TOP/patch/115/import/afscprof.lct file_name.ldt PROFILE PROFILE_NAME="profile option" APPLICATION_SHORT_NAME="prod"
9 - Requset Group
FNDLOAD apps/apps@seed115 O Y DOWNLOAD $FND_TOP/patch/115/import/afcpreqg.lct file_name.ldt REQUEST_GROUP REQUEST_GROUP_NAME="request group" APPLICATION_SHORT_NAME="prod"
10 - Request Sets
FNDLOAD apps/apps@seed115 O Y DOWNLOAD $FND_TOP/patch/115/import/afcprset.lct file_name.ldt REQ_SET APPLICATION_SHORT_NAME="prod" REQUEST_SET_NAME="request set"
11 - Responsibilities
FNDLOAD apps/apps@seed115 O Y DOWNLOAD $FND_TOP/patch/115/import/afscursp.lct file_name.ldt FND_RESPONSIBILITY RESP_KEY="responsibility
12 - Menus
FNDLOAD apps/apps@seed115 O Y DOWNLOAD $FND_TOP/patch/115/import/afsload.lct file_name.ldt MENU MENU_NAME="menu_name"
13 – Forms/Functions
FNDLOAD apps/apps@seed115 0 Y DOWNLOAD $FND_TOP/patch/115/import/affrmcus.lct file_name.ldt FND_FORM_CUSTOM_RULES The Upload syntax for all styles: FNDLOAD apps/apps@seed115 0 Y UPLOAD $FND_TOP/patch/115/import/affrmcus.lct file_name.ldt
14. User/Responsibilities
FNDLOAD apps/apps@seed115 0 Y DOWNLOAD @FND:patch/115/import/afscursp.lct file_name.ldt FND_USER Then UPLOAD FNDLOAD apps/apps@seed115 0 Y UPLOAD [UPLOAD_PARTIAL] @FND:patch/115/import/afscursp.lct file_name.ldt FND_USER []

Hybrid clone with rman

Hybrid database cloning using RMAN and appsTier file copy This document demonstrates how to clone the dbTier using RMAN and refreshing the appsTier using the file copy. The RMAN APPSClone scripts found on the appsTier /home/applprod/APPSClone is not yet mature as it’s prone to some errors some refined tests needs to be carried out. Delete the datafiles from dbtier The following files should be deleted before launching the RMAN refresh applprod@gvaebsdb2:$ • /u02/DEV2/rdo/* • /u03/DEV2/rdo/* • /u04/DEV2/rdo/* • /u02/DEV2/dbf/DEV2/* Launch the RMAN refresh start time for the DB restore 1:36 pm The scripts for the RMAN backup and catalog management are in /home/applprod/DBclone. The DB refresh script will ask for 2 parameter the source PROD and destination DEVx In the example below I refreshing DEV2 using PROD backup. It will run for approximately 6 hours applprod@gvaebsdb2:~/DBClone/bin $ refresh_DB.ksh PROD DEV2 Checking options for restore... ********************************************************* ******** Running Database Restore Process For ******** ******** PROD to DEV2 ******** ********************************************************* Refresh Start Date/Time: 12032011_1339 Step 1: Preparing new init.ora file for restore... Step 2: Preparing Password file for restore... Step 3: Launching RMAN restore process... Step 3.1: Shutting down DB and Listener... Step 3.2: Starting up DB in nomount with RMAN init parameters ... Step 3.3: Launching RMAN command for the restore, For more detail pls tail the logs under /RMAN/logs Step 4: Restarting Database and Listener with old init values... Step 5: Adding Temp tablespace and changing SYSTEM password... Step 6: Launchng the autoconfig on the DB tier... The log file for this session is located at: /u01/DEV2/apps/proddb/11.2.0/appsutil/log/DEV2_gvaebsdb2/12032002/adconfig.log AutoConfig is configuring the Database environment... AutoConfig will consider the custom templates if present. Using ORACLE_HOME location: /u01/DEV2/apps/proddb/11.2.0 Classpath : /u01/DEV2/apps/proddb/11.2.0/jdk/jre/lib/rt.jar:/u01/DEV2/apps/proddb/11.2.0/jdk/lib/dt.jar:/u01/DEV2/apps/proddb/11.2.0/jdk/lib/tools.jar:/u01/DEV2/apps/proddb/11.2.0/jdbc/lib/ojdbc5.jar:/u01/DEV2/apps/proddb/11.2.0/appsutil/java/xmlparserv2.zip:/u01/DEV2/apps/proddb/11.2.0/appsutil/java:/u01/DEV2/apps/proddb/11.2.0/jlib/netcfg.jar:/u01/DEV2/apps/proddb/11.2.0/jlib/ldapjclnt11.jar Using Context file: /u01/DEV2/apps/proddb/11.2.0/appsutil/DEV2_gvaebsdb2.xml Context Value Management will now update the Context file Updating Context file...COMPLETED Attempting upload of Context file and templates to database...COMPLETED Updating rdbms version in Context file to db112 Updating rdbms type in Context file to 64 bits Configuring templates from ORACLE_HOME ... AutoConfig completed successfully. REFRESH FINISHED... Refresh Finish Date/Time: 12032011_2004 applprod@gvaebsdb2:~/DBClone/bin $ Restore_DEV2_13:39:42-12-03-2011.log Check the RMAN logs for errors The RMAN error logs can be found on the mounted partition RMAN/logs gvaebsdb2@applprod:$/RMAN/logs# Recovery Manager: Release 11.2.0.1.0 - Production on Sat Dec 3 13:39:43 2011 Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved. RMAN-06549: connected to auxiliary database: DEV2 (not mounted) RMAN-08030: allocated channel: c1 RMAN-08500: channel c1: SID=1171 device type=DISK RMAN-08030: allocated channel: c2 RMAN-08500: channel c2: SID=5 device type=DISK ……………………. ………………………….. …………………………………… ……………………………………………… RMAN-08161: contents of Memory Script: { Alter clone database open resetlogs;} RMAN-08162: executing Memory Script RMAN-06400: database opened RMAN-03091: Finished Duplicate Db at 12/03/11 20:00:36 RMAN-08031: released channel: c1 RMAN-08031: released channel: c2 RMAN-08031: released channel: c3 RMAN-08031: released channel: c4 RMAN-08031: released channel: c5 RMAN-08031: released channel: c6 RMAN> Recovery Manager complete. Sat Dec 3 20:00:37 CET 2011 Remove the lock file RMAN DUPLICATE FINISH Total Refresh time 6:30 mins Put the database in noarchivelog After the successful database clone the RMAN script puts the newly cloned database in archivelog mode. De-activate the archiving by running the below process. Log on the system as sys applprod@gvaebsdb2:/u01/DEV2/apps/proddb/11.2.0/dbs $ sqlplus '/as sysdba'; SQL*Plus: Release 11.2.0.1.0 Production on Sun Dec 4 13:25:35 2011 Copyright (c) 1982, 2009, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production With the Partitioning, Oracle Label Security, OLAP, Data Mining and Real Application Testing options SQL> create pfile from spfile; File created. Edit the initDEV2.ora located in $ORACLE_HOME/dbs add or remove the below incase you want to archiving or not * log_archive_dest_1='LOCATION=/u03/DEV2/arch/logs/' SQL> shutdown immediate; Database closed. Database dismounted. ORACLE instance shut down. Deactivate archiving SQL> startup mount; ORACLE instance started. Total System Global Area 1068937216 bytes Fixed Size 2220200 bytes Variable Size 444600152 bytes Database Buffers 608174080 bytes Redo Buffers 13942784 bytes Database mounted. SQL> alter database noarchivelog; Database altered. SQL> archive log list; Database log mode No Archive Mode Automatic archival Disabled Archive destination /u01/DEV2/arch/logs Oldest online log sequence 39 Current log sequence 41 SQL> alter database open; Database altered. Run the postclone scripts Run as apps user with the source apps passowrd run on the newly cloned database the following postclone scripts found in /home/dba/dbaScripts • cmclean.sql input DUAL for this script to continue this will clean the fnd type commit at the end as this script as it does not auto commit. • create_fnd_trig.sql SQL> @create_fnd_trig.sql; Trigger created. PL/SQL procedure successfully completed. Commit complete. • tem_postclone_script.sql Either run the tem_postclone_script.sql or create a script all.sql in /home/applprod with the following contents see below ################################## script start here############################# -- Update salary entries with random values between 500'000 and 800'000 UPDATE pay_element_entry_values_f pev SET pev.screen_entry_value = TRUNC(dbms_random.VALUE(500000,800000)) WHERE pev.element_entry_id IN (SELECT pee.element_entry_id FROM pay_element_entries_f pee, pay_element_types_f pet WHERE pev.effective_start_date <= pee.effective_end_date AND pee.effective_start_date <= pev.effective_end_date AND pee.element_type_id = pet.element_type_id AND pet.reporting_name = 'Salary'); COMMIT; -- Update salary proposals with salary entries UPDATE per_pay_proposals ppp SET proposed_salary_n=NVL( (SELECT MAX(pev.screen_entry_value) FROM pay_element_entries_f pee,pay_element_types_f pet,pay_input_values_f piv,pay_element_entry_values_f pev WHERE pee.element_entry_id = pev.element_entry_id AND pev.effective_start_date BETWEEN pee.effective_start_date AND pee.effective_end_date AND pev.input_value_id = piv.input_value_id AND pet.element_type_id = piv.element_type_id AND EXISTS (SELECT 1 FROM per_pay_bases ppb, pay_input_values_f piv2 WHERE ppb.input_value_id=piv2.input_value_id AND piv2.element_type_id=pet.element_type_id) AND ppp.assignment_id=pee.assignment_id AND ppp.change_date=pee.effective_start_date AND ppp.date_to=pee.effective_end_date ), 0); COMMIT; --- To disable messaging system in alerts UPDATE ALR_MESSAGE_SYSTEMS SET enabled_flag='N'; COMMIT; ---- To disable scheduled requests UPDATE fnd_concurrent_requests REQ SET phase_code = 'C',status_code = 'D',completion_text = 'Cancelled by clone'WHERE phase_code IN ('P','I'); commit; -- To disable active alerts UPDATE alr_alerts SET enabled_flag = 'N' WHERE enabled_flag = 'Y'; COMMIT; UPDATE fnd_profile_option_values SET profile_option_value='Temenos E-Business [ DEVx ] Environment clone of ('||to_char(sysdate-1,'dd-Mon-yyyy')||')' WHERE profile_option_id=125; TRUNCATE TABLE applsys.FND_CONCURRENT_REQUESTS; TRUNCATE TABLE applsys.FND_RUN_REQUESTS; TRUNCATE TABLE applsys.FND_CONC_REQUEST_ARGUMENTS; TRUNCATE TABLE applsys.FND_CONCURRENT_PROCESSES; TRUNCATE TABLE applsys.FND_CONC_STAT_LIST; TRUNCATE TABLE applsys.FND_CONC_STAT_SUMMARY; TRUNCATE TABLE applsys.FND_RUN_REQ_PP_ACTIONS; TRUNCATE TABLE applsys.WF_NOTIFICATIONS; UPDATE fnd_profile_option_values f SET f.profile_option_value = 'BLUE' WHERE f.profile_option_id = 5785; ################################script ends here ################################## Run dbTier autoconfig To ensure that the new parameter are taken into consideration re-run the dbTier autoconfig applprod@gvaebsdb2: cd /u01/DEV2/apps/proddb/11.2.0/appsutil/scripts/DEV2_gvaebsdb2 input the source database apps password to continue applprod@gvaebsdb2:/u01/DEV2/apps/proddb/11.2.0/appsutil/scripts/DEV2_gvaebsdb2 $ ./adautocfg.sh Enter the APPS user password: The log file for this session is located at: /u01/DEV2/apps/proddb/11.2.0/appsutil/log/DEV2_gvaebsdb2/12042301/adconfig.log AutoConfig is configuring the Database environment... AutoConfig will consider the custom templates if present. Using ORACLE_HOME location : /u01/DEV2/apps/proddb/11.2.0 Classpath : /u01/DEV2/apps/proddb/11.2.0/jdk/jre/lib/rt.jar:/u01/DEV2/apps/proddb/11.2.0/jdk/lib/dt.jar:/u01/DEV2/apps/proddb/11.2.0/jdk/lib/tools.jar:/u01/DEV2/apps/proddb/11.2.0/jdbc/lib/ojdbc5.jar:/u01/DEV2/apps/proddb/11.2.0/appsutil/java/xmlparserv2.zip:/u01/DEV2/apps/proddb/11.2.0/appsutil/java:/u01/DEV2/apps/proddb/11.2.0/jlib/netcfg.jar:/u01/DEV2/apps/proddb/11.2.0/jlib/ldapjclnt11.jar Using Context file : /u01/DEV2/apps/proddb/11.2.0/appsutil/DEV2_gvaebsdb2.xml Context Value Management will now update the Context file Updating Context file...COMPLETED Attempting upload of Context file and templates to database...COMPLETED Updating rdbms version in Context file to db112 Updating rdbms type in Context file to 64 bits Configuring templates from ORACLE_HOME ... AutoConfig completed successfully. applprod@gvaebsdb2:/u01/DEV2/apps/proddb/11.2.0/appsutil/scripts/DEV2_gvaebsdb2 $ Stop and restart the dbTier Set the dbTier environment applprod@gvaebsdb2:~ $ . db.env applprod@gvaebsdb2:~ $ dbctrl stop applprod@gvaebsdb2:~ $ dbctrl start Change the sys and system password applprod@gvaebsdb2:~ $ sqlplus '/as sysdba'; SQL*Plus: Release 11.2.0.1.0 Production on Mon Dec 5 11:35:51 2011 Copyright (c) 1982, 2009, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production With the Partitioning, Oracle Label Security, OLAP, Data Mining and Real Application Testing options SQL> alter user sys identified by sysdev2; User altered. SQL> alter user system identified by sysdev2; User altered. Recreate the passwordfile Creating the password enables remote connections applprod@gvaebsdb2:/u01/DEV2/apps/proddb/11.2.0/dbs $ orapwd file=orapwDEV2 password=sysdev2 entries=5 force=y Method 1 Configuring the appsTier Copy the appsTier of the source environment in this case the production gva2ebsapps2@applprod:$ cd /u01/PROD/apps/ gva2ebsapps2@applprod:$ cp –R * /tmp FTP the above directories to the same location on the new server from the tmp directory using winscp Gunzip the files applprod@gvaebsapps2:~ $/u01/DEV2/apps/prodcomn $ gunzip *.tgz applprod@gvaebsapps2:~ $/u01/DEV2/apps/prodora $ gunzip *.tgz applprod@gvaebsapps2:~ $/u01/DEV2/apps/prodappl $ gunzip *.tgz Then execute the below command in all the applprod@gvaebsapps2:/u01/DEV2/apps/prodora $ find . -name "*.tar" -exec tar xf {} \; applprod@gvaebsapps2:/u01/DEV2/apps/prodcomn $ find . -name "*.tar" -exec tar xf {} \; applprod@gvaebsapps2:/u01/DEV2/apps/prodappl $ find . -name "*.tar" -exec tar xf {} \; gvaebsapps2@applprod:$ cd /u01/DEV2/apps/ After the application tier software unzipping are complete, the file system configurations need to be updated to reflect the new environment. To do this on the application tiers, log on to the NEW application tier system as the APPLPROD user and execute the following commands. Since your environment scripts are not yet set up, you will need to manually resolve the reference to and : $ cd /clone/bin $ perl adclonectx.pl /admin/.xml Answer the questions appropriately. • The new host name is the server you are on. • Note the database name will stay the same as the production DB, but the database server name will be different. • Do not validate the context, as the process will not be able to connect to the database. • When it asks for the new context path and file name, give it /admin/.xml applprod@gvaebsapps2: cd /u01/DEV2/apps/prodappl/ad/11.5.0/bin applprod@gvaebsapps2: $ perl adclonectx.pl /u01/DEV2/apps/prodappl/admin/PROD_gva2ebsapps2.xml /usr/java/jdk1.6.0_21/bin/java -classpath ../../../fnd/11.5.0/java/3rdparty/stdalone/jdbc12.zip:../../../fnd/11.5.0/java/3rdparty/stdalone/xmlparserv2.zip:/usr/java/jdk1.6.0_21/lib/rt.jar:../java: oracle.apps.ad.context.CloneContext -e /u01/DEV2/apps/prodappl/admin/PROD_gva2ebsapps2.xml -perl "/usr/bin/perl" -perl5lib "/usr/lib/perl5/5.8.5/i386-linux-thread-multi:/usr/lib/perl5/5.8.5:/usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi:/usr/lib/perl5/site_perl/5.8.5:/usr/lib/perl5/site_perl:/usr/lib/perl5/vendor_perl/5.8.5/i386-linux-thread-multi:/usr/lib/perl5/vendor_perl/5.8.5:/usr/lib/perl5/vendor_perl/5.8.3/i386-linux-thread- multi:/usr/lib/perl5/vendor_perl/5.8.3:/usr/lib/perl5/vendor_perl:." -tmpl ../admin/template/adxmlctx.tmp Provide the values required for creation of the new APPL_TOP Context file. Do you want to use a virtual hostname for the target node (y/n) [n] ?:n Target hostname [gva2ebsapps2]:gvaebsapps2 It is recommended that your inputs are validated by the program. However you might choose not to validate your inputs under following circumstances: -If cloning a context on source system for a remote system. -If cloning a context on a machine where the ports are taken and you do not want to shutdown the services at this point. -If cloning a context but the database it needs to connect is not available. Do you want the inputs to be validated (y/n) [n] ?:n Target system database SID [PROD]:DEV2 Username for the applications file system owner [applprod]: Group for the applications file system owner [dba]: Target system database server node [gva2ebsdbprod]:gvaebsdb2 Target system database domain name [temenosgroup.com]: Does the target system have more than one application tier server node (y/n) [y] ?:n Is the target system APPL_TOP divided into multiple mount points (y/n) [n] ?: Target system APPL_TOP mount point [/u01/PROD/apps/prodappl]:/u01/DEV2/apps/prodappl Target system COMMON_TOP directory [/u01/PROD/apps/prodcomn]:/u01/DEV2/apps/prodcomn Target system 8.0.6 ORACLE_HOME directory [/u01/PROD/apps/prodora/8.0.6]:/u01/DEV2/apps/prodora/8.0.6 Target system iAS ORACLE_HOME directory [/u01/PROD/apps/prodora/iAS]:/u01/DEV2/apps/prodora/iAS Do you want to preserve the Display set to gva2ebsapps2:0.0 (y/n) [y] ?:n Target system Display [gvaebsapps2:0.0]: Location of the JDK on the target system [/usr/java/jdk1.6.0_21]: Do you want to preserve the port values from the source system on the target system (y/n) [y] ?:y Complete port information available at /u01/DEV2/apps/prodappl/admin/out/DEV2_gvaebsapps2/portpool.lst New context path and file name [/u01/DEV2/apps/prodappl/admin/DEV2_gvaebsapps2.xml]: Creating the new APPL_TOP Context file from : /u01/DEV2/apps/prodappl/ad/11.5.0/admin/template/adxmlctx.tmp The new APPL_TOP context file has been created : /u01/DEV2/apps/prodappl/admin/DEV2_gvaebsapps2.xml Log file located at /tmp/CloneContext_1205100437.log applprod@gvaebsapps2:/u01/DEV2/apps/prodappl/ad/11.5.0/bin $ When the script is finished and the context file is created, execute the following, again resolving the reference to manually: Using the perl script adclonectx.pl clone the production xml to new context file Edit the newly created XML above and change the following to the appropriate values and ports • 1522temenosgroup.comgvaexchangeebizadmindev2@temenos.comebizadmindev2vezfe1europe.temenosgroup.comhermes.temenosgroup.com7779http://bidev1.temenosgroup.com:9705gvaebsapps2_dev2 applprod@gvaebsapps2:$ perl adconfig.pl contextfile=/u01/DEV2/apps/prodappl/admin/DEV2_gvaebsapp2.xml run=INSTE8 Type the perl command all on one line. Answer the questions when prompted. This creates your environment files on the application tier. It tries to connect to the database, so some portions fail, but the environment scripts should be successfully created. applprod@gvaebsapps2:/u01/DEV2/apps/prodappl/ad/11.5.0/bin $ perl adconfig.pl contextfile=/u01/DEV2/apps/prodappl/admin/DEV2_gvaebsapps2.xml run=INSTE8 Enter the APPS user password: Script execution phase specified: INSTE8 The log file for this session is located at: /u01/DEV2/apps/prodappl/admin/DEV2_gvaebsapps2/log/12051048/adconfig.log AutoConfig is configuring the Applications environment... AutoConfig will consider the custom templates if present. Using APPL_TOP location : /u01/DEV2/apps/prodappl Classpath : /usr/java/jdk1.6.0_21/jre/lib/rt.jar:/usr/java/jdk1.6.0_21/lib/dt.jar:/usr/java/jdk1.6.0_21/lib/tools.jar:/u01/DEV2/apps/prodcomn/java/appsborg2.zip:/u01/DEV2/apps/prodcomn/java Using Context file : /u01/DEV2/apps/prodappl/admin/DEV2_gvaebsapps2.xml Context Value Management will now update the Context file Updating Context file...COMPLETED Execution phase specified at run time: INSTE8 No uploading of Context File and its templates to database. Configuring templates from all of the product tops... Configuring AD_TOP........COMPLETED Configuring FND_TOP.......COMPLETED Configuring ICX_TOP.......COMPLETED Configuring IEO_TOP.......COMPLETED Configuring ABM_TOP.......COMPLETED Configuring ECX_TOP.......COMPLETED Configuring BIS_TOP.......COMPLETED Configuring GL_TOP........COMPLETED Configuring AMS_TOP.......COMPLETED Configuring CCT_TOP.......COMPLETED Configuring WSH_TOP.......COMPLETED Configuring CLN_TOP.......COMPLETED Configuring OKE_TOP.......COMPLETED Configuring OKL_TOP.......COMPLETED Configuring OKS_TOP.......COMPLETED Configuring CSF_TOP.......COMPLETED Configuring XNC_TOP.......COMPLETED Configuring IGS_TOP.......COMPLETED Configuring IBY_TOP.......COMPLETED Configuring PA_TOP........COMPLETED Configuring JTF_TOP.......COMPLETED Configuring MWA_TOP.......COMPLETED Configuring CN_TOP........COMPLETED Configuring CSI_TOP.......COMPLETED Configuring WIP_TOP.......COMPLETED Configuring CSE_TOP.......COMPLETED Configuring EAM_TOP.......COMPLETED Configuring IMT_TOP.......COMPLETED Configuring FTE_TOP.......COMPLETED Configuring ONT_TOP.......COMPLETED Configuring AR_TOP........COMPLETED Configuring AHL_TOP.......COMPLETED Configuring OZF_TOP.......COMPLETED Configuring IES_TOP.......COMPLETED Configuring CSD_TOP.......COMPLETED Configuring IGC_TOP.......COMPLETED AutoConfig completed successfully. Run FNDCPASS Change the apps ,sysadmin password using FNDCPASS applprod@gvaebsapps2:~ $ FNDCPASS apps/appspassword 0 Y system/sysdev2 SYSTEM APPLSYS appsdev2 applprod@gvaebsapps2:~ $ FNDCPASS apps/appspassword 0 Y system/sysdev2 USER SYSADMIN sysadmindev2 Rerun adautocfg.sh applprod@gvaebsapps2:/u01/DEV2/apps/prodcomn/admin/scripts/DEV2_gvaebsapps2 $ ./adautocfg.sh Enter the APPS user password : The log file for this session is located at: /u01/DEV2/apps/prodappl/admin/DEV2_gvaebsapps2/log/12051112/adconfig.log AutoConfig is configuring the Applications environment... AutoConfig will consider the custom templates if present. Using APPL_TOP location : /u01/DEV2/apps/prodappl Classpath : /usr/java/jdk1.6.0_21/jre/lib/rt.jar:/usr/java/jdk1.6.0_21/lib/dt.jar:/usr/java/jdk1.6.0_21/lib/tools.jar:/u01/DEV2/apps/prodcomn/java/appsborg2.zip:/u01/DEV2/apps/prodcomn/java Using Context file : /u01/DEV2/apps/prodappl/admin/DEV2_gvaebsapps2.xml Context Value Management will now update the Context file Updating Context file...COMPLETED Attempting upload of Context file and templates to database... Configuring templates from all of the product tops... Configuring AD_TOP........COMPLETED Configuring FND_TOP.......COMPLETED Configuring ICX_TOP.......COMPLETED Configuring IEO_TOP.......COMPLETED Configuring ABM_TOP.......COMPLETED Configuring ECX_TOP.......COMPLETED Configuring BIS_TOP.......COMPLETED Configuring GL_TOP........COMPLETED Configuring AMS_TOP.......COMPLETED Configuring CCT_TOP.......COMPLETED Configuring WSH_TOP.......COMPLETED Configuring CLN_TOP.......COMPLETED Configuring OKE_TOP.......COMPLETED Configuring OKL_TOP.......COMPLETED Configuring OKS_TOP.......COMPLETED Configuring CSF_TOP.......COMPLETED Configuring XNC_TOP.......COMPLETED Configuring IGS_TOP.......COMPLETED Configuring IBY_TOP.......COMPLETED Configuring PA_TOP........COMPLETED Configuring JTF_TOP.......COMPLETED Configuring MWA_TOP.......COMPLETED Configuring CN_TOP........COMPLETED Configuring CSI_TOP.......COMPLETED Configuring WIP_TOP.......COMPLETED Configuring CSE_TOP.......COMPLETED Configuring EAM_TOP.......COMPLETED Configuring IMT_TOP.......COMPLETED Configuring FTE_TOP.......COMPLETED Configuring ONT_TOP.......COMPLETED Configuring AR_TOP........COMPLETED Configuring AHL_TOP.......COMPLETED Configuring OZF_TOP.......COMPLETED Configuring IES_TOP.......COMPLETED Configuring CSD_TOP.......COMPLETED Configuring IGC_TOP.......COMPLETED AutoConfig completed successfully. Start the application Set the application environment applprod@gvaebsapps2:~ $ . apps.env applprod@gvaebsapps2:~ $ appsctrl start You should Change the application profile Color UPDATE fnd_profile_option_values f SET f.profile_option_value = '%' WHERE f.profile_option_id = 5785; Where % is in 1. Teal 2. Titanium (DON’T USE exclusively for production) 3. Red 4. Khaki 5. Blue 6. Olive 7. Purple Use the different colors for the diffrernet DEV server from DEV1 to DEV5 Sitename UPDATE fnd_profile_option_values SET profile_option_value= 'Temenos Development DEVx a clone of ('||to_char(sysdate-1,'dd-Mon-yyyy')||')' WHERE profile_option_id=125; Please see section Run the postclone scripts above

Friday, April 8, 2011

Fixing a corrupt or a cloned database 11g Oracle Inventory

DBA’s working with Oracle especially ERP applications are familiar with cloning environment on demand. The issue becomes tricky when you need to apply a database patch using Opatch for testing before moving it to production.
The Oracle Inventory is a critical part of oracle installation and created using OUI. This is useful when you what to find out what patches have been applied to the database, mostly during upgrades and patch applications.

When you run the command opatch lsinventory from the Opatch directory in the $ORACLE_HOME in a cloned environment you might see the output like below

applprod@gvaebsdb3:/u01/DEV3/apps/proddb/11.2.0/OPatch $ opatch lsinventory
Invoking OPatch 11.1.0.6.6
Oracle Interim Patch Installer version 11.1.0.6.6
Copyright (c) 2009, Oracle Corporation. All rights reserved
Oracle Home : /u01/DEV4/apps/proddb/11.2.0
Central Inventory : n/a
from :
OPatch version : 11.1.0.6.6
OUI version : 11.2.0.1.0
OUI location : /u01/DEV3/apps/proddb/11.2.0/oui
Log file location : n/a
Patch history file: /u01/DEV3/apps/proddb/11.2.0/cfgtoollogs/opatch/opatch_history.txt
OPatch cannot find a valid oraInst.loc file to locate Central Inventory.
OPatch failed with error code 104
applprod@gvaebsdb3:/u01/DEV3/apps/proddb/11.2.0/OPatch

or
Now check the status of the inventory
applprod@gvaebsdb3:/u01/DEV3/apps/proddb/11.2.0/OPatch $ opatch lsinventory
Invoking OPatch 11.1.0.6.6
Oracle Interim Patch Installer version 11.1.0.6.6
Copyright (c) 2009, Oracle Corporation. All rights reserved
Oracle Home : /u01/DEV4/apps/proddb/11.2.0
Central Inventory : n/a
Central Inventory : /home/applprod/oraInventory
from : /etc/oraInst.loc
OPatch version : 11.1.0.6.6
OUI version : 11.2.0.1.0
OUI location : /u01/DEV3/apps/proddb/11.2.0/oui
Log file location : /u01/DEV3/apps/proddb/11.2.0//cfgtoollogs/opatch/opatch-2011_Apr_09_40-20-30-CET_Tue.log

LsInventory Session failed: OPatch failed to locate Central Inventory.
Possible causes are:
The Central Inventory is corrupted
The oraInst.loc file specified is not valid.

OPatch failed with error code 72

At times the Inventory location can become corrupt or might have changed due to architecture due to different file systems etc. this will cause Opatch to error out with error code 104 and can hamper the database patch application.

To fix a corrupted oracle inventory implicates at times moving it to a different location or editing ContentsXML and changing the ORACLE_HOME_NAME

applprod@gvaebsdb3:~ /home/applprod/oraInventory/ContentsXML



Then go the $ORACLE_HOME/oui/bin

Launch the repair as shown below .Note you should attached the correct ORACLE_HOME and ORACLE_HOME_NAME can be anything

applprod@gvaebsdb3:/u01/DEV3/apps/proddb/11.2.0/oui/bin $ ./runInstaller -silent -invPtrLoc "/etc/oraInst.loc" -attachHome ORACLE_HOME="/u01/DEV3/apps/proddb/11.2.0" ORACLE_HOME_NAME="Ora11g_home1"
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB. Actual 7954 MB Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2011-04-08_11-32-32AM. Please wait ...applprod@gvaebsdb3:/u01/DEV3/apps/proddb/11.2.0/oui/bin $ The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /home/applprod/oraInventory
'AttachHome' was successful.
applprod@gvaebsdb3:/u01/DEV3/apps/proddb/11.2.0/oui/bin