AFX Server is in a 'Not running' State in the user interface but 'afx status' indicates AFX is running in RSA Identity Governance & Lifecycle
4 years ago
Originally Published: 2018-06-20
Article Number
000042151
Applies To
RSA Product Set: RSA Identity Governance & Lifecycle
RSA Version/Condition: 7.0.2, 7.1.0
 
Issue
A once-working AFX server goes into a state of Not running and all the AFX connectors are in a Stopped state.
 
User-added image

Additional Symptoms

  • The output from the afx status command indicates that AFX is running.
  • No changes have been made to the system.
  • There may be connection refused errors in the $AFX_HOME/esb/logs/esb.AFX-INIT.log but the other AFX logs and the aveksaServer.log have no errors.
  • The following message is seen repeatedly in the $AVEKSA_HOME/wildfly/standalone/log/aveksaServer.log file:
[com.aveksa.afx.server.service.AFXServerAgentServiceProvider] getServerAgentUptimeDate: 
Removing the uptime date for server agent 1 - prior uptime date: null, prior last polled date: null, 
last polled date: 2018-06-12 19:08:56.0, current date: 2018-06- 12 20:25:35.0, polling interval 5000

Please refer to RSA Knowledge Base Article 000030327 -- Artifacts to gather in RSA Identity Governance & Lifecycle to find the location of the aveksaServer.log file for your specific deployment if you are on a WildFly cluster or a non-WildFly platform. The aveksaServer.log may also be downloaded from the RSA Identity Governance & Lifecycle user interface (Admin > System > Server Nodes tab > under Logs.)
 
Cause
There is a locking problem with the T_AV_AFX_SERVER_AGENT table which is an internal database table that logs information about the AFX agent that runs the AFX server.
 
Resolution
This issue is resolved in RSA Identity Governance & Lifecycle 7.1.0 P03.

This patch updates the persistence layer which prevents deadlocks when updating the T_AV_AFX_SERVER_AGENT table (and other tables).

Unfortunately due to the complexity of the fix for this issue, this issue cannot be back-ported to 7.0.2.
 
Workaround
To workaround this problem, the database, the RSA Identity Governance & Lifecycle application, and AFX need to be restarted. As the oracle user, execute the following commands in the order as shown::
 
afx stop
acm stop
acm stopdb
acm startdb
acm start
afx start