000030081 - RSA Identity Governance & Lifecycle AFX connectors remain in a Deployed state; Could not locate data file kahadb/db-XXX.log

The $AFX_HOME/activemq/data/activemq.log has an error similar to this:

ERROR | Looking for key 284 but not found in fileMap: {307=db-307.log number = 307 , length = 1951190} | org.apache.kahadb.journal.Journal | main 2014-12-22 11:09:33,857 | ERROR | Failed to start ActiveMQ JMS Message Broker. Reason: java.io.IOException: Could not locate data file /home/oracle/AFX/activemq/data/kahadb/db-284.log

A restart of AFX, RSA Identity Governance & Lifecycle, and even a system reboot does not fix the problem.

The key message in the home/<afxuser>/AFX/activemq/data/activemq.log is this:

Could not locate data file /home/oracle/AFX/activemq/data/kahadb/db-284.log 

KahaDB is a file-based persistence database used by ActiveMQ. In this case, ActiveMQ is unable to find the persistence database (db-XXX.log). This means the KahaDB database is corrupt.

We have seen this file become corrupted on a system that ran out of disk space while AFX was actively being used.

 

1. Login as user oracle and clean up disk space. Please see  RSA Knowledge Base Article 000027944 -- How to check local file system disk space usage for RSA Identity Governance & Lifecycle and/or contact RSA Identity Governance & Lifecycle Support for assistance on disk space cleanup.
2. Navigate to $AFX_HOME/activemq/data/kahadb/.
3. Stop AFX as the afxuser.
4. Touch the db-XXX.log, where XXX is the number of the file that is missing; in this case db-284.log.
5. List the directory contents to make sure the file permissions for the file are correct.
6. Start AFX as the afxuser.

$ cd $AFX_HOME/activemq/data/kahadb/
$ afx stop
$ touch db-XXX.log
$ ls -la
-rw-rw-r-- 1 oracle oinstall 0 May 13 2019 db-XXX.log
$ afx start

7. If this does not resolve your issue, recreate the persistence database as follows. Perform these steps as the afxuser.

$ cd $AFX_HOME/activemq/data
$ mv kahadb kahadb.old
$ afx start