000031917 - RSA Access Manager servers are opening connections to the datastore but not closing any connections

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 2Show Document
  • View in full screen mode

Article Content

Article Number000031917
Applies ToRSA Product Set: ClearTrust
RSA Product/Service Type: Access Manager (AxM)
RSA Version/Condition: 6.2 SP3
Platform: Windows
O/S Version: Server 2012 R2
IssueThe RSA Access Manager servers are opening connections to the datastore but not closing any connections, eventually running out of connections.
In the aserver.log, the error "Data Store Error" will be followed by either "Unable to send data to receiver" or "This stream has been closed due to Socket Timeout."
Examples:
 
sequence_number=29,date=2015-10-28 10:24:07:642 EDT,messageID=9,user=62testuser,client_ip_address=192.168.221.128,client_port=63224,browser_ip_address=192.168.221.128,result_code=101,user=62testuser,user_dn=null,result_action=Data Store Error
sequence_number=30,date=2015-10-28 10:24:07:646 EDT,messageID=-2,internal_error,description='Unable to send data to receiver.',details='java.io.IOException: Unable to send data to receiver.'
sequence_number=45,date=2015-10-28 10:26:22:924 EDT,messageID=9,user=62testuser,client_ip_address=192.168.221.128,client_port=63299,browser_ip_address=192.168.221.128,result_code=101,user=62testuser,user_dn=null,result_action=Data Store Error
sequence_number=46,date=2015-10-28 10:26:22:924 EDT,messageID=-2,internal_error,description='This stream has been closed due to Socket Timeout.',details='java.io.IOException: This stream has been closed due to Socket Timeout.'
CauseA defect was introduced in Access Manager Server Version 6.2 SP3.
This defect occurs during the watcher thread when idle connections get validated.
The watcher thread leases the connection to see if valid, but does not return the connection back to the "free" connections in the pool.
The next request will have no free connections to use and a new connection will get opened.
This new connection eventually gets validated during the watcher thread and subsequently does not get returned to the free connections in the pool.
This continues until the aserver reaches the Max connections configured to the data store (default 100).
Once reaching max connections, the aserver will no longer open new connection and since all leased connection are not getting freed, this will cause a catastrophic failure of the aserver.
ResolutionA hotfix has been released to remedy this defect, which is 6.2.3.04.
Please open a case with RSA Customer Support and quote this article number to request the latest patch that includes this fix.

Attachments

    Outcomes