000015824 - RSA Access Manger is unable to open new sockets

Document created by RSA Customer Support Employee on Oct 22, 2018
Version 1Show Document
  • View in full screen mode

Article Content

Article Number000015824
Applies ToRSA Product Set: Access Manager
RSA Product/Service Type: Access Manager
RSA Version/Condition: 6.1, 6.2
 
IssueRSA Access Manager is unable to open new sockets.
Error in aserver out.

Error connecting to the dispatcher at:server.com:5607


Too many open files


Retry in 10 seconds



Error in dispatcher.log
sequence_number=567151,remote_client=dispatcher1,2011-08-30 10:20:21:188 EDT,messageID=2001,event_type=Ping Failure,event_description=Ping failure for server,server=AuthServerDetails( Addr = aserver1, Port = 5615, Exp host = 192.168.2.10, Location Class = class1, Failure Cnt = 3 


When attempting to start the server instance the following error is generated:

Starting RSA Access Manager 6.1.4 Servers: /server-61/bin/aserver.sh: fork: retry: Resource temporarily unavailable


/server-61/bin/aserver.sh: fork: retry: Resource temporarily unavailable



When attempting to start the server instance the aserver.out shows the following exception:

20:36:30:863 [*] [main] - Resolved hostname to following IP(s) : 192.168.0.10


java.lang.OutOfMemoryError: unable to create new native thread


        at java.lang.Thread.start0(Native Method)


        at java.lang.Thread.start(Unknown Source)


        at netscape.ldap.LDAPConnThread.connect(Unknown Source)

CauseThe dispacher.log message indicates that the Manager failed to update its status with the dispatcher at the scheduled interval.  If the failure persists the aserver will be marked as being down. The error in the aserver.out indicates that the aserver was up and was responsive but was unable to open a TCP socket to the dispatcher to send the status update.   The failure was due to a resource problem on the physical machine hosting the aserver process.
ResolutionRun netstat or lsof -a and determine the number of sockets in use on the machine.  Audit the sockets opened by Access Manager and determine if they are logical based on the number of clients of the aserver.  Typically the aserver will require several sockets per agent.  Some agents and monitoring programs may require substantially more sockets.  Determine the actual number of sockets in use during peak loads and set the operating system value for "maximum file handles" to a reasonable value based on your requirements.
The UNIX command ulimit -n will list the current number of file handles allocated to the process.  The ulimit can be set per user, so the command should be executed using the account used to launch the aserver.  The ulimit command can also be used to set the values for the maximum number of file handles.  On UNIX each TCP socket requires a file handle.  File handles are also required for processes and log files.  The default setting for max file handles on some UNIX systems may be 1024 which is insufficient for an RSA Access Manager Aserver.  Set this limit to 2048 or higher depending on the number of actual sockets in use.


If the ulimit cannot be set high enough, the hard limits may need to be raised by editing the following in the  /etc/security /limits.conf file. 

 


* soft nofile 256000


* hard nofile 256000
Legacy Article IDa56144

Attachments

    Outcomes