000021825 - Socket leakage for RSA ClearTrust Agent for ClearTrust ACM on port 5608 when SSL enabled

Document created by RSA Customer Support Employee on Jun 16, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 2Show Document
  • View in full screen mode

Article Content

Article Number000021825
Applies ToRSA ClearTrust Agent for ClearTrust ACM
Sun Solaris 2.8
IssueSocket leakage for RSA ClearTrust Agent for ClearTrust ACM on port 5608 when SSL enabled
RSA ClearTrust Agent for ClearTrust ACM fails to respond after running for a period of time
Over time, the machine on which the ACM is installed accumulates sockets to [dispatcher machine]:5608 that are in a CLOSE_WAIT state; for the Apache processes, the number of file descriptors increases at the same rate. The number of sockets/file descriptors increases by 1 at the same rate as the setting cleartrust.agent.auth_server_pool_refresh.
CauseRSA ClearTrust Agents for ClearTrust ACM (which are essentially RSA ClearTrust Agents 3.5 for Apache) prior to hot fix level 3.5.0.16 did not initiate disconnection with a FIN packet, and did not respond correctly to the dispatcher initiating disconnection with a FIN packet (though the agent does respond with an ACK). Expected behavior is that the agent responds to the dispatcher's FIN with an ACK and a FIN of its own; the dispatcher would ACK the agent's FIN, and the socket would move into a TIME_WAIT state. Because the agent did not respond with a FIN, the socket would hang in CLOSE_WAIT.

Sockets in a CLOSE_WAIT state do not timeout, and there's no way to reclaim the resource, so the leakage eventually leads to socket or file descriptor exhaustion. At this point, the performance of the ACM degrades severely and requires restarting to clear the leaked sockets.
ResolutionTo correct this issue, apply hot fix 3.5.0.16 or later to RSA ClearTrust Agent for ClearTrust ACM. Contact RSA Security Customer Support to request this hot fix, or the latest fix level (which is cumulative, and includes fixes from all previous fix levels).

This hot fix causes the agent to initiate disconnection with a FIN packet. When the dispatcher responds with an ACK and a FIN, the Agent ACK's the dispatcher's FIN and moves to a TIME_WAIT state, which remains open in that state for 2*MSL (Maximum Segment Lifetime, the TCP/IP stack's configured value for the maximum time a packet should exist on the network--usually between 30 and 120 seconds). When 2*MSL seconds has elapsed, the TCP/IP stack automatically closes the socket.
Legacy Article IDa25188

Attachments

    Outcomes