AFX Server intermittently shuts down in RSA Identity Governance & Lifecycle
2 years ago
Originally Published: 2019-10-07
Article Number
000042481
Applies To
RSA Product Set: RSA Identity Governance & Lifecycle
RSA Version/Condition: 7.1.0
 
Issue
The RSA Identity Governance & Lifecycle  AFX Server periodically stops running every few days. Once restarted, it runs for a few days, then stops again.

The aveksaServer.log shows the following warnings and errors:
  
12/12/2018 04:00:00.015 WARN (ThreadPool-5) [com.aveksa.server.scheduler.SchedulerServiceProvider] ScheduledTask: Heartbeat_TASK could not be persisted 
12/12/2018 04:04:29.887 WARN (EmailSenderServiceProvider) [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] SQL Error: 17002, SQLState: 08006 
12/12/2018 04:04:29.887 ERROR (EmailSenderServiceProvider) [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] IO Error: The Network Adapter could not establish the connection DSRA0010E: SQL State = 08006, Error Code = 17,002 
12/12/2018 04:00:00.010 WARN (ThreadPool-5) [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] SQL Error: 17002, SQLState: 08006 
12/12/2018 04:00:00.010 ERROR (ThreadPool-5) [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] IO Error: The Network Adapter could not establish the connection DSRA0010E: SQL State = 08006, Error Code = 17,002 
12/12/2018 04:00:00.010 ERROR (ThreadPool-5) [com.aveksa.server.node.HeartbeatServiceProvider] Unable to update Server Node status 
com.aveksa.server.system.SystemServiceException: com.aveksa.server.db.PersistenceException: n/a 


Please refer to RSA Knowledge Base Article 00000327 -- Artifacts to gather in RSA Identity Governance & Lifecycle to find the location of the aveksaServer.log file for your specific deployment.


The home/afxuser/AFX/esb/logs/esb.AFX-MAIN.log file shows the following errors:
  
2018-12-11 21:33:12.656 [WARN] com.aveksa.afx.server.component.ResponseTransformerComponent:159 - Unable to process primary response from ACM (http status 401):2018-12-11 21:33:12.656 [WARN] com.aveksa.afx.server.component.ResponseTransformerComponent:159 - Unable to process primary response from ACM (http status 401):Error 401: SRVE0295E: Error reported: 401
2018-12-11 21:34:13.080 [ERROR] org.mule.exception.DefaultMessagingExceptionStrategy:337 - ********************************************************************************Message               : Failed to route event via endpoint: DefaultOutboundEndpoint{endpointUri=https://<server-name>:8444/aveksa/afx/settings-test, connector=HttpsConnector{  name=afxRequestHttpsConnector  lifecycle=start  this=11e9cd82  numberOfConcurrentTransactedReceivers=4  createMultipleTransactedReceivers=true  connected=true  supportedProtocols=[https]  serviceOverrides=    session.handler=org.mule.session.NullSessionHandler},  name='endpoint.https.<server-name>.8444.aveksa.afx.settings.test', mep=REQUEST_RESPONSE, properties={exceptionOnMessageError=true, http.method=POST, followRedirects=false}, transactionConfig=Transaction{factory=null, action=INDIFFERENT, timeout=0}, deleteUnacceptedMessages=false, initialState=started, responseTimeout=60000, endpointEncoding=UTF-8, disableTransportTransformer=false}. Message payload is of type: PostMethodCode                  : MULE_ERROR--2--------------------------------------------------------------------------------Exception stack is:1. Read timed out (java.net.SocketTimeoutException)  java.net.SocketInputStream:-2 (null)2. Failed to route event via endpoint: DefaultOutboundEndpoint{endpointUri=https://<server-name>:8444/aveksa/afx/settings-test, connector=HttpsConnector{  name=afxRequestHttpsConnector  lifecycle=start  this=11e9cd82  numberOfConcurrentTransactedReceivers=4  createMultipleTransactedReceivers=true  connected=true  supportedProtocols=[https]  serviceOverrides=    session.handler=org.mule.session.NullSessionHandler},  name='endpoint.https.<server-name>.8444.aveksa.afx.settings.test', mep=REQUEST_RESPONSE, properties={exceptionOnMessageError=true, http.method=POST, followRedirects=false}, transactionConfig=Transaction{factory=null, action=INDIFFERENT, timeout=0}, deleteUnacceptedMessages=false, initialState=started, responseTimeout=60000, endpointEncoding=UTF-8, disableTransportTransformer=false}. Message payload is of type: PostMethod (org.mule.api.transport.DispatchException)  org.mule.transport.http.HttpClientMessageDispatcher:151 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/transport/DispatchException.html)--------------------------------------------------------------------------------Root Exception stack trace:java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171)    + 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)********************************************************************************
 
Cause
Whenever AFX was abruptly unable to connect to the database even for a short time, the heartbeat requests were suspended which would cause the AFX Server status to change to a Not running state and remain in that state when database connectivity was restored. 
 
Resolution
This issue is resolved in the following RSA Identity Governance & Lifecycle versions and patch levels: 
  • RSA Identity Governance & Lifecycle 7.1.0 P07
  • RSA Identity Governance & Lifecycle 7.1.1 P01
  • RSA Identity Governance & Lifecycle 7.2
Workaround
Restart the AFX Server. Execute the below restart command as the afxuser:
  
afx restart
 

Related Articles

Trending Articles

Don't see what you're looking for?