AFX Connectors remain in a Deployed state, mmc-console fails to start and 'Unable to initialize query handler' acccessing the SearchIndex in RSA Identity Governance & Lifecycle

RSA Product Set: RSA Identity Governance & Lifecycle 
RSA Version/Condition: 7.0.x, 7.1.x, 7.2.x
 

The AFX Server is in a Running state but the RSA Identity Governance & Lifecycle AFX Connectors are stuck in a Deployed state and do not progress to a Running state. For example, (AFX > Connectors):

The AFX command line tool (afx status) shows the AFX Server as active (even though it is not fully functional as indicated by errors in the AFX log files.)

The $AFX_HOME/esb/logs/esb.AFX-MAIN.log has a Failed to get flows for server error which indicates that mmc-console cannot be reached.
 

2019-11-08 04:39:31.397 [ERROR] com.aveksa.afx.server.manager.MMCRequestManagerImpl:136 - Unable to get status for all Connectors from MMC
com.aveksa.afx.server.manager.MMCException: Failed to get flows for server. No associated server Id. Check Request Manager Initialization

The $AFX_HOME/mmc-console/logs/catalina.out file which logs information about the deployment of the mmc-console shows that the deployment of the mmc-console failed:
 

Nov 08, 2019 9:21:57 AM org.apache.catalina.core.StandardContext startInternal
SEVERE: One or more listeners failed to start. Full details will be found in the appropriate container log file

The $AFX_HOME/mmc-console/logs/mmc-console-app.log shows the failure is Unable to initialize the query handler when attempting to access the SearchIndex.

07 Nov 2019 14:14:51 [localhost-startStop-1] ERROR org.apache.jackrabbit.core.RepositoryImpl - Failed to initialize workspace 'default'
javax.jcr.RepositoryException: Unable to initialize query handler: org.apache.jackrabbit.core.query.lucene.SearchIndex@474b01a7

The AFX Connectors fail to deploy because mmc-console (part of the AFX Server) has failed to start. The mmc-console collects connector information in order for the connectors to run. If mmc-console fails to start, then the connector information is never collected and the connectors cannot move from Deployed to Running.

Mmc-console has failed to start because the indexes for the internal database used by the mmc-console are corrupted as indicated by the SearchIndex errors.

This issue may occur upon AFX startup if the previously running AFX instance was not shutdown cleanly.

 

Stop the AFX instance and delete (rename) the two directories that hold the index files. When AFX is restarted, the indexes will be recreated.

As the AFX user, login to the server and perform the following steps:
 

mv /home/oracle/AFX/mmc-console/mmc-data/workspaces/default/index /home/oracle/AFX/mmc-console/mmc-data/workspaces/default/index_mmddyyyy
mv /home/oracle/AFX/mmc-console/mmc-data/repository/index /home/oracle/AFX/mmc-console/mmc-data/repository/index_mmddyyyy

 

Alternatively you may redeploy AFX as described in RSA Knowledge Base Article 000037993 -- How to download and install the AFX Server Archive in RSA Identity Governance & Lifecycle.