000030163 - AFX Connectors remain in a Deployed state and Port 8585 already in use error in RSA Identity Governance & Lifecycle

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support on Nov 21, 2019
Version 6Show Document
  • View in full screen mode

Article Content

Article Number000030163
Applies ToRSA Product Set: Identity Governance & Lifecycle 
RSA Version/Condition: 6.8.x, 6.9.x, 7.0.x, 7.1.x
 
IssueThe AFX Server is in a Running state but all previously working RSA Identity Governance & Lifecycle AFX Connectors are stuck in a Deployed state and do not progress to a Running state. For example,
 
User-added image


The $AFX_HOME/mmc-console/logs/catalina.out has the following errors in the log file:
 

Oct 15, 2019 9:58:10 AM org.apache.coyote.AbstractProtocol init
SEVERE: Failed to initialize end point associated with ProtocolHandler ["http-bio-8585"]
java.net.BindException: Address already in use (Bind failed) <null>:8585

        at org.apache.tomcat.util.net.JIoEndpoint.bind(JIoEndpoint.java:413)
        at org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:665)
        at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:452)
        at org.apache.coyote.http11.AbstractHttp11JsseProtocol.init(AbstractHttp11JsseProtocol.java:119)
        at org.apache.catalina.connector.Connector.initInternal(Connector.java:978)
        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
        at org.apache.catalina.core.StandardService.initInternal(StandardService.java:560)
        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
        at org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:840)
        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
        at org.apache.catalina.startup.Catalina.load(Catalina.java:642)
        at org.apache.catalina.startup.Catalina.load(Catalina.java:667)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:253)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:427)
Caused by: java.net.BindException: Address already in use (Bind failed)
        at java.net.PlainSocketImpl.socketBind(Native Method)
        at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
        at java.net.ServerSocket.bind(ServerSocket.java:375)
        at java.net.ServerSocket.<init>(ServerSocket.java:237)
        at java.net.ServerSocket.<init>(ServerSocket.java:181)
        at org.apache.tomcat.util.net.DefaultServerSocketFactory.createSocket(DefaultServerSocketFactory.java:49)
        at org.apache.tomcat.util.net.JIoEndpoint.bind(JIoEndpoint.java:400)
        ... 17 more

Oct 15, 2019 9:58:10 AM org.apache.catalina.core.StandardService initInternal
SEVERE: Failed to initialize connector [Connector[HTTP/1.1-8585]]
org.apache.catalina.LifecycleException: Failed to initialize component [Connector[HTTP/1.1-8585]]

        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
        at org.apache.catalina.core.StandardService.initInternal(StandardService.java:560)
        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
        at org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:840)
        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
        at org.apache.catalina.startup.Catalina.load(Catalina.java:642)
        at org.apache.catalina.startup.Catalina.load(Catalina.java:667)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:253)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:427)
Caused by: org.apache.catalina.LifecycleException: Protocol handler initialization failed
        at org.apache.catalina.connector.Connector.initInternal(Connector.java:980)
        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
        ... 12 more
Caused by: java.net.BindException: Address already in use (Bind failed) <null>:8585
        at org.apache.tomcat.util.net.JIoEndpoint.bind(JIoEndpoint.java:413)
        at org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:665)
        at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:452)
        at org.apache.coyote.http11.AbstractHttp11JsseProtocol.init(AbstractHttp11JsseProtocol.java:119)
        at org.apache.catalina.connector.Connector.initInternal(Connector.java:978)
        ... 13 more
Caused by: java.net.BindException: Address already in use (Bind failed)
        at java.net.PlainSocketImpl.socketBind(Native Method)
        at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
        at java.net.ServerSocket.bind(ServerSocket.java:375)
        at java.net.ServerSocket.<init>(ServerSocket.java:237)
        at java.net.ServerSocket.<init>(ServerSocket.java:181)
        at org.apache.tomcat.util.net.DefaultServerSocketFactory.createSocket(DefaultServerSocketFactory.java:49)
        at org.apache.tomcat.util.net.JIoEndpoint.bind(JIoEndpoint.java:400)
        ... 17 more



The $AFX_HOME/esb/logs/esb.AFX-MAIN.log has multiple errors such as:

 



2019-10-15 10:00:20.607
[WARN] com.aveksa.afx.server.manager.MMCRequestManagerImpl:95 -
Unable get/setup the server id from the MMC request
2019-10-15 10:00:20.608
[WARN] com.aveksa.afx.server.manager.MMCRequestManagerImpl:96 -
Retrying MMC connection and server id setup...


2019-10-15 10:00:25.609
[WARN] com.aveksa.afx.server.manager.MMCRequestManagerImpl:99 -
Connection request retry attempt #1 of 12

2019-10-15 10:00:25.612
[INFO] org.apache.commons.httpclient.HttpMethodDirector:193 -
I/O exception (java.net.ConnectException) caught when processing request:
Connection refused (Connection refused) ...


2019-10-15 10:01:20.724
[WARN] com.aveksa.afx.server.manager.MMCRequestManagerImpl:101 -
Unable get/setup the server id from the MMC request
2019-10-15 10:01:20.725
[ERROR] com.aveksa.afx.server.manager.MMCRequestManagerImpl:102 -
Server Id setup try count exceeded


2019-10-15 10:01:20.725
[ERROR] com.aveksa.afx.server.manager.MMCRequestManagerImpl:103 -
Please verify that the MMC console is running and Host & Port in the URL are correct.

2019-10-15 10:01:20.726
[ERROR] com.aveksa.afx.server.manager.MMCRequestManagerImpl:82 -
Unable to initialize the MMC Request Manager com.aveksa.afx.server.manager.MMCException:
Unable to retrieve and setup the Afx server Id information ...


2019-10-15 10:01:20.916
[ERROR] com.aveksa.afx.server.manager.MMCRequestManagerImpl:136 -
Unable to get status for all Connectors from MMC
com.aveksa.afx.server.manager.MMCException:
Failed to get flows for server.  No associated server Id.
Check Request Manager Initialization


 


CauseThe third party Mule Management Console (MMC) used by AFX has failed to start because the port it uses, 8585, is in use by another process. The key error is:
 

java.net.BindException: Address already in use (Bind failed) <null>:8585


The MMC collects connector information in order for the connectors to run. If the MMC application fails to start, then the connector information is never collected and the connectors cannot move from Deployed to Running.
 
ResolutionMake sure port 8585 is open for use. 
  1. To check what application is using port 8585, as the root user, execute the following:


netstat tulpn | grep 8585


  1. Configure the application identified in step 1 to use a different port.

We've seen this situation on Weblogic deployments when the NodeManager is configured to listen on port 8585. Configure NodeManager to list on a different port such as 8586. Please contact your Weblogic Administrator for information on how to modify the NodeManager configuration.

Attachments

    Outcomes