How and why the Oracle database should be monitored in RSA Identity Governance & Lifecycle

[#1]

Article Number

000031637

Applies To

RSA Product Set: RSA Identity Governance & Lifecycle 
RSA Version/Condition: All
 

Issue

The purpose of this RSA Knowledge Base Article is to explain the importance of monitoring the Oracle database used for the RSA Identity Governance & Lifecycle application. Failure to monitor the database results in undetected errors that can lead to catastrophic system failures. These failures can be avoided by following the recommendations in this RSA Knowledge Base Article.
 

Resolution

Reasons to monitor your RSA Identity Governance & Lifecycle Oracle database:

Data corruption. 

Data corruption can be logical or physical. Not much can be done to avoid logical corruptions before they happen but physical corruptions can be avoided by monitoring the physical server that hosts the database.

Space issues.

Out of space issues can occur if the database is not properly monitored for increasing table sizes, tablespace sizes and disk space. Running out of space can cause data corruption and/or a complete standstill of the database.

Oracle errors.

Like any software product, the RSA Identity Governance & Lifecycle database might run into an Oracle issue that may or may not affect your database health.

Operating system errors.

Operating system errors can alert you to problems before they happen and should be investigated, not ignored.

How to Monitor the Oracle database:

The following tools and documentation can assist with monitoring the Oracle database. 

Aveksa Statistics Report (ASR).

See RSA Knowledge Base Article How to create an Aveksa Statistics Report (ASR) in RSA Identity Governance & Lifecycle for information on overall monitoring of your Oracle database and RSA Identity Governance & Lifecycle application. 

RSA Knowledge Base Article 000038168.

See RSA Knowledge Base Article 000038168 -- How to manage database growth in RSA Identity Governance & Lifecycle for information on managing Oracle database growth to avoid space and corruption issues. 

Oracle Enterprise Manager Database Control (OEM).

The Hardware Appliance and the Software Appliance with a local database are installed with the Oracle Enterprise Manager (OEM) Database Console (DBConsole). DBConsole is a lightweight monitoring tool that comes with every Oracle database and monitors a single database. Some organizations with multiple Oracle databases, may have the  more extensive Enterprise Manager Cloud Control monitoring tool which can monitor multiple databases. Check with your Database Administrator team to see which monitoring tool is available for your RSA Identity Governance & Lifecycle database. If your organization has the Enterprise Manager Cloud Control monitoring tool, the RSA Identity Governance & Lifecycle database could potentially be integrated with the monitoring of other Oracle databases in your organization.
 

The OEM database console is automatically started with RSA Identity Governance & Lifecycle, and is accessed with the following URL:

https://<acm-server>:1158/em/console

The following is an example of OEM output. Note the items in yellow are places where you would see errors and alerts show up.
 

Image description
 

You can configure OEM to send email notifications of issues. Refer to the Oracle documentation on Setting Up Direct Alert Notification for information on configuring email notifications.

Oracle Database Alert Log.

The Oracle database alert log is:

/u01/app/oracle/diag/rdbms/avdb/AVDB/trace/alert_AVDB.log

The database alert log will show any errors that happen on the database, and each one should be carefully examined to determine if they can be ignored or measures should be taken to correct the issue. ANY ERROR MESSAGE in the database alert log should be examined for its importance.

Operating System Log.

The Operating System log for Hardware and Software Appliances is:

/var/log/messages

The Operating System log indicates hardware or operating system issues. Issues in this log file should be carefully examined and mediated if necessary.  Special attention should be paid to any disk-related error messages. Disks are smart and can relocate bad blocks, however, a growing number of disk errors may well indicate a hardware issue requiring a disk swap/replacement. If a growing number of disk errors are observed, further diagnostics should be performed to confirm if the disk errors are hardware issues. If so, the disk will need to be swapped as soon as possible.
 

Conclusion:

Any error in the Oracle database alert log or the Operating System log should be detected in real time and reacted upon to prevent a potentially serious outage in RSA Identity Governance & Lifecycle.