000034136 - RSA Access Manager Authorization Server (aserver) maintains a bad connection when Oracle is restarted

Document created by RSA Customer Support Employee on Mar 17, 2018
Version 1Show Document
  • View in full screen mode

Article Content

Article Number000034136
Applies ToRSA Product Set: Access Manager
RSA Version/Condition: 6.1 SP4, 6.2, 6.2 SP1, 6.2 SP2, 6.2 SP3
Datastore: SQL

 
IssueAfter a restart of SQL (Oracle) users experienced Access Denied messages and the RSA Access Manager aserver was maintaining bad connections to SQL.

Example of errors from the aserver.log file:

sequence_number=1541,2015-03-19 10:19:49:355 GMT-05:00,messageID=-2,internal_error,description='An error occurred in connecting to the database server.
[SQL Error : ORA-01033: ORACLE initialization or shutdown in progress
Code: (1033) State: (72000)]',details='sirrus.da.exception.DataStoreException: An error occurred in connecting to the database server.
[SQL Error : ORA-01033: ORACLE initialization or shutdown in progress
Code: (1033) State: (72000)]'



sequence_number=1543,2015-03-19 10:20:04:357 GMT-05:00,messageID=-2,internal_error,description='Unable to send data to receiver.',details='java.io.IOException: Unable to send data to receiver.'


Example of errors from aserver out file.

sirrus.da.exception.DataStoreException: SQLException : Code:17002;TextIo exception: Connection reset by peer: socket write error
at sirrus.da.sql.util.SQLEntityHelper.getUser(SQLEntityHelper.java:473)
at sirrus.da.sql.util.SQLConnectionImpl.get(SQLConnectionImpl.java:458)
at sirrus.da.sql.auth.factory.SQLEntityFactory.getUserByName_aroundBody2(SQLEntityFactory.java:136)
at sirrus.da.sql.auth.factory.SQLEntityFactory$AjcClosure3.run(SQLEntityFactory.java:1)



sirrus.da.exception.DataStoreException: SQLException : Code:17410;TextNo more data to read from socket
    at sirrus.da.sql.util.SQLResourceHelper.getWebServer(SQLResourceHelper.java:1555)
    at sirrus.da.sql.util.SQLConnectionImpl.get(SQLConnectionImpl.java:440)
    at sirrus.da.sql.auth.factory.SQLWebServerFactory.getWebServerByName(SQLWebServerFactory.java:111)
    at sirrus.da.auth.cache.factory.CachingWebServerFactory.getWebServerByName(CachingWebServerFactory.java:131)
    at sirrus.da.auth.WebServer.getByName(WebServer.java:63)
    at sirrus.authserver.AuthorizationAPI.addApplicationData(AuthorizationAPI.java:3593)
    at sirrus.authserver.AuthorizationAPI.isUrlProtected(AuthorizationAPI.java:3550)
    at sirrus.authserver.AuthorizationAPI.doCheckResource(AuthorizationAPI.java:1599)
    at sirrus.authserver.AuthorizationAPI.checkResourceStatus(AuthorizationAPI.java:554)
    at sirrus.authserver.DebugAuthorizationAPI.checkResourceStatus(DebugAuthorizationAPI.java:110)
    at sirrus.authserver.TCPServerAPIAdaptor.checkResourceStatus(TCPServerAPIAdaptor.java:69)
Cause

With these defaults, the aserver on startup opens and keeps 10 connections open to oracle:




Reclaimconnections: true
Idletimeout: 300000 (5 minutes)
maxidleconnections : 10
keepalive : true

 

If the Oracle CT instance is restarted, these connections become bad, but are still kept in the connection pool until the watcher thread is run. (keepalive validation)



If the connections get leased and returned, that restarts the clock on them being idle.



In an environment under light load where the connections get removed until reduced to the minimum connections maintained. (determined by .maxidleconnections:)

These minimum connections only get removed if they have been idle for 5 minutes.

Because it is the watcher thread that closes these minimum connections that have been idle for 5 minutes, these bad connections never get closed if a user keeps trying to logon.
This is because the watcher thread never runs on them, as they are getting leased every time a user tries to logon.

ResolutionThis can be resolved by restarting the aserver or by patching to version 6.2 Service Pack 3 Hotfix 04 or later.



 

Attachments

    Outcomes