Wednesday, 10 June 2009

huge ons.log size in Oracle 10g AS Infra

Introduction:
We had an issue of huge size of ons.log file under Oracle 10g AS R2 infra tier, and it was making the 1.4 GB ons.log files and in days it was filling more then 20GB file system space.

Troubleshooting:
When we checked the ons.log file, it was having the following errors, and these errors were keep coming in the ons.log files.

--------------------
09/06/08 12:56:48 [4] Local connection 0,127.0.0.1,6101 missing form factor09/06/08 12:56:48 [4] Local connection 0,127.0.0.1,6101 missing form factor09/06/08 12:56:48 [4] Local connection 0,127.0.0.1,6101 missing form factor09/06/08 12:56:48 [4] Local connection 0,127.0.0.1,6101 missing form factor09/06/08 12:56:48 [4] Local connection 0,127.0.0.1,6101 missing form factor09/06/08 12:56:48 [4] Local connection 0,127.0.0.1,6101 missing form factor09/06/08 12:56:48 [4] Local connection 0,127.0.0.1,6101 missing form factor
--------------------

In this test machien we had the following Oracle products installed.

1- Oracle 10g R2 Database
2- Oracle 10g AS R2 (infra and middle tier)
3- Oracle 10g AS R3 (SOA)

We raised a SR with Oracle support, and they came up that there is port conflict between one of the ons of above Oracle Products.

Resolution:

We checed the opmn/conf files from all the Oracle products, and found the port conflict between Oracle 10g R2 Database ons and Oracle 10g AS R2 infra ons.

There were following options to be applied.

1- The TNS listener should be configured so it would not let ons subscribe to it. (this option is for Oracle 10g R2 Database listener.) Following settings were recommended by Oracle support.

Following needs to be added in the listener.ora file.

SUBSCRIBE_FOR_NODE_DOWN_EVENT_LISTENER=OFF

2- rename the ons.config file for the Oracle 10g R2 Database, so that ons should not be started and try to be subscribed to listener.

3- the last but not the least option would be to change the conflicted ports in the ons.config file of Oracle 10g R2 Database.

we changed the ports in ons.config file, but first we made sure that new ports should not be used in the system, we did this by checking netstat -rn command on Solaris box. After resetting the ports we stopped the ons, listener and restarted the ons and listener and every thing was fine.

1 comment:

Ryan Kao said...

Thank you! I used the first solution and it worked perfectly. Awesome!