000029259 - AFX Server remains in a Not running State, afx status shows 'timed out waiting for AFX applications to start' and mule_ee.log has a 'Could not build a validated path' error in RSA Identity Governance & Lifecycle

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support on Jan 7, 2020
Version 15Show Document
  • View in full screen mode

Article Content

Article Number000029259
Applies ToRSA Product Set: RSA Identity Governance & Lifecycle 
RSA Version/Condition: 6.9.1, 7.0.x, 7.1.x

 
IssueThe AFX Server in RSA Identity Governance & Lifecycle is in a Not running State in the user interface (AFX > Servers).
 
User-added image
 

When logged into the application server as the afx user, the afx status command shows the startup timed out and the AFX Server never fully starts.
 

$ afx status
● afx_server.service - Afx Server
   Loaded: loaded (/etc/systemd/system/afx_server.service; enabled; vendor preset: disabled)
   Active: active (exited) since Mon 2020-01-06 12:30:28 EST; 11s ago
  Process: 19999 ExecStop=/etc/init.d/afx_server stop (code=exited, status=0/SUCCESS)
  Process: 20643 ExecStart=/etc/init.d/afx_server start (code=exited, status=0/SUCCESS)
Main PID: 20643 (code=exited, status=0/SUCCESS)

Jan 06 12:29:18 acm-711 afx_server[20643]: Waiting for AFX applications to start...
Jan 06 12:29:28 acm-711 afx_server[20643]: Waiting for AFX applications to start...
Jan 06 12:29:38 acm-711 afx_server[20643]: Waiting for AFX applications to start...
Jan 06 12:29:48 acm-711 afx_server[20643]: Waiting for AFX applications to start...
Jan 06 12:29:58 acm-711 afx_server[20643]: Waiting for AFX applications to start...
Jan 06 12:30:08 acm-711 afx_server[20643]: Waiting for AFX applications to start...
Jan 06 12:30:18 acm-711 afx_server[20643]: Waiting for AFX applications to start...
Jan 06 12:30:28 acm-711 afx_server[20643]: WARNING!! Timed out waiting for AFX applications to start.
Please check AFX application log files for detailed status information.

Jan 06 12:30:28 acm-711 afx_server[20643]: done
Jan 06 12:30:28 acm-711 systemd[1]: Started Afx Server.

 

When starting AFX, the following errors are logged to the AFX log files:
 


In the $AFX_HOME/esb/logs/mule_ee.log:



++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Failed to deploy artifact '10_AFX-INIT', see below       +
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
org.mule.module.launcher.DeploymentInitException: CertPathBuilderException: Could not build a validated path.
...
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed:
java.security.cert.CertPathBuilderException: Could not build a validated path.
...
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: java.security.cert.CertPathBuilderException:
Could not build a validated path.
 ... 
Caused by: java.security.cert.CertPathBuilderException: Could not build a validated path.
        at com.rsa.cryptoj.o.qb.engineBuild(Unknown Source)


and

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Failed to deploy artifact '15_AFX-MAIN', see below       +
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
org.mule.module.launcher.DeploymentInitException: IllegalArgumentException: Could not resolve placeholder
'afx.server.activemq.password' in string value "${afx.server.activemq.password}"




In the $AFX_HOME/esb/logs/esb.AFX-INIT.log:
 

2020-01-06 12:27:24.425 [ERROR] com.aveksa.afx.server.init.SubmitInitializationRequestComponent:162 -
Unable to establish secure (SSL) connection with RSA Identity Governance and Lifecycle server.
2020-01-06 12:27:24.425 [ERROR] com.aveksa.afx.server.init.SubmitInitializationRequestComponent:171 -
SSL certificates for RSA Identity Governance and Lifecycle server and AFX were not issued by the same
RSA Identity Governance and Lifecycle Certificate Authority(CA). You may encounter this problem if the
RSA Identity Governance and Lifecycle certificate store has been changed, but either the
RSA Identity Governance and Lifecycle server OR AFX installation hasn't been updated with the respective
keystore containing new certificate and CA entries.
Please update both the
RSA Identity Governance and Lifecycle server and AFX installations with latest respective keystore available for download in the
RSA Identity Governance and Lifecycle application.

2020-01-06 12:27:24.426 [ERROR] com.aveksa.afx.server.init.ServerInitializationComponent:79 -
Server initialization failed! Please correct the issue and restart AFX.

and

Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException:
PKIX path building failed: java.security.cert.CertPathBuilderException: Could not build a validated path.


In the $AFX_HOME/esb/logs/esb.AFX-MAIN.log:



java.lang.IllegalArgumentException: Could not resolve placeholder 'afx.server.activemq.password'
in string value "${afx.server.activemq.password}"

CauseThe below error in the $AFX_HOME/esb/logs/esb.AFX_INIT.log indicates the problem cause:
 

SSL certificates for RSA Identity Governance and Lifecycle server and AFX were not issued by the same
RSA Identity Governance and Lifecycle Certificate Authority(CA). You may encounter this problem if the
RSA Identity Governance and Lifecycle certificate store has been changed, but either the
RSA Identity Governance and Lifecycle server OR AFX installation hasn't been updated with the respective
keystore containing new certificate and CA entries. Please update both the
RSA Identity Governance and Lifecycle server and AFX installations with latest respective keystore available for download in the
RSA Identity Governance and Lifecycle application.


Please see RSA Knowledge Base Article 000038314 -- How to update the root (server) and client certificates in RSA Identity Governance & Lifecycle for possible root causes for this error.
Resolution
The process to resolve this error is to generate a new root (server) certificate and a new client certificate for each AFX server and remote agent, redeploy all certificates,and restart the RSA Identity Governance & Lifecycle  application, AFX application, and remote agents. 

This process is described in detail in RSA Knowledge Base Article 000038314 -- How to update the root (server) and client certificates in RSA Identity Governance & Lifecycle.


 

Attachments

    Outcomes