System attempted to find user “SYSTEM” across identity sources error occurs in RSA Authentication Manager 8.x

[#1]

Article Number

000032456

Applies To

RSA Product Set: SecurID
RSA Product/Service Type: Authentication Manager
RSA Version/Condition: 8.1 or later
 

Issue

The system logs has the error every 60 seconds:
 

System attempted to find user “SYSTEM”across identity sources

Cause

An administrative user has left the company and was deleted from the Active Directory identity source. 

The expanded error shows the deleted admin user account name as a value for Argument 1:
 

Date & Time:                   2018-07-02 11:29:55.102
Log Level:                     ERROR
Description:                   System attempted to find user “SYSTEM” across identity sources
Activity Result Key:           Failure
Result:                        System could not find the user across Identity Sources
Administrator User ID:         SYSTEM
Administrator First Name:      N/A
Administrator Last Name:       N/A
Administrator Security Domain:         N/A
Administrator Identity Source Name:    N/A
Activity Key:                  Find user across Identity Sources
Activity Result Key:           Failure
Instance Name:                 {AM_instance_hostname}
Client IP:                     N/A
Server Node IP:                n.n.n.n
Component Key:                 sa.ims.admin.impl.PrincipalMoveAcrossISTrackerImpl
Argument 1:                    {UserID}
Argument 2:                    N/A
Argument 3:                    N/A
Argument 4:                    N/A
Argument 5:                    N/A
Argument 6:                    N/A
    Exception:         com.rsa.common.DataNotFoundException: Unable to resolve principal,

at com.rsa.ims.admin.iscleanup.impl.IdentitySourceCleanupControllerImpl.trustedResolveAndRepairPrincipal(IdentitySourceCleanupControllerImpl.java:465),
at com.rsa.ims.admin.iscleanup.impl.IdentitySourceCleanupControllerImpl.resolveAndRepairPrincipal(IdentitySourceCleanupControllerImpl.java:427),
at com.rsa.ims.admin.impl.PrincipalMoveAcrossISTrackerImpl$1.run(PrincipalMoveAcrossISTrackerImpl.java:218),
at com.rsa.ims.security.spi.SimpleSecurityContextImpl.doAs(SimpleSecurityContextImpl.java:113),
at com.rsa.security.SecurityContext.doAs(SecurityContext.java:439),
at com.rsa.ims.admin.impl.PrincipalMoveAcrossISTrackerImpl.handlePrincipalMove(PrincipalMoveAcrossISTrackerImpl.java:223),
at com.rsa.ims.admin.impl.PrincipalMoveAcrossISTrackerImpl.trackPrincipalMovesAcrossIS(PrincipalMoveAcrossISTrackerImpl.java:173),
at com.rsa.ims.admin.impl.PrincipalAdministrationImpl.resolveAndRepairPrincipal(PrincipalAdministrationImpl.java:5647),
at com.rsa.ims.admin.impl.PrincipalAdministrationImpl.loadRegisteredPrincipal(PrincipalAdministrationImpl.java:5447),
at com.rsa.ims.admin.impl.PrincipalAdministrationImpl.trustedLookup(PrincipalAdministrationImpl.java:5924),
at com.rsa.ims.admin.impl.PrincipalAdministrationImpl$4.run(PrincipalAdministrationImpl.java:1936),
at com.rsa.ims.admin.impl.PrincipalAdministrationImpl$4.run(PrincipalAdministrationImpl.java:1),
at com.rsa.ims.security.spi.SimpleSecurityContextImpl.doAs(SimpleSecurityContextImpl.java:113),
at com.rsa.security.SecurityContext.doAs(SecurityContext.java:439),
at com.rsa.security.SecurityContext.doAsSystem(SecurityContext.java:474),
at com.rsa.ims.admin.impl.PrincipalAdministrationImpl.lookup(PrincipalAdministrationImpl.java:1933),
at com.rsa.ims.admin.impl.PrincipalAdministrationImpl.lookup(PrincipalAdministrationImpl.java:1904),
at sun.reflect.GeneratedMethodAccessor147.invoke(Unknown Source),
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25),
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309),
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:196),
at com.sun.proxy.$Proxy108.lookup(Unknown Source),
at com.rsa.ims.admin.impl.AdminRoleAdministrationImpl.getPrincipalsWithAdminRole(AdminRoleAdministrationImpl.java:566),
at sun.reflect.GeneratedMethodAccessor208.invoke(Unknown Source),
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25),
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309),
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:196),
at com.sun.proxy.$Proxy117.getPrincipalsWithAdminRole(Unknown Source),
at com.rsa.ims.criticalnotification.impl.EmailNotificationHandler$1.run(EmailNotificationHandler.java:176),
at com.rsa.ims.security.spi.SimpleSecurityContextImpl.doAs(SimpleSecurityContextImpl.java:113),
at com.rsa.security.SecurityContext.doAs(SecurityContext.java:439),
at com.rsa.security.SecurityContext.doAsSystem(SecurityContext.java:474),
at com.rsa.ims.criticalnotification.impl.EmailNotificationHandler.updateSuperAdminEmailList(EmailNotificationHandler.java:173),
at
...
...
...

Resolution

The first step would be to clean up unresolvable users to remove any user meta data from the Authentication Manager database after the removal of the administrative user.  Use the procedure for cleaning up unresolvable users manually.

 

Unchecking the Grace Period option is important for this procedure.

Should the cleanup not resolve the issue then the suggestion would be to perform a flush of all data objects on all of the primary and replica instances in the Authentication Manager deployment followed by a stop and start of the Authentication Manager primary and replica services at the command line, as documented on page 192 of the RSA Authentication Manager 8.4 Administrator's Guide.
 

When restarting services always start with the primary instance, leaving the replica instance(s) authenticating users and ensure the primary has started before stopping then starting the replica instance(s).

Where you have changed console certificates, check they have not expired as the Authentication Manager services will not start where there are expired console certificates. Where there are expired console certificates then please refer to  instructions on how to Replace an Expired Console Certificate.

• To stop the Authentication Manager services at the command line use /opt/rsa/am/server/rsaserv stop all:

rsaadmin@am84p:~> /opt/rsa/am/server/rsaserv stop all
Stopping RSA RADIUS Server: ***
RSA RADIUS Server                                          [SHUTDOWN]
Stopping RSA Runtime Server: *****
RSA Runtime Server                                         [SHUTDOWN]
Stopping RSA Console Server: ***
RSA Console Server                                         [SHUTDOWN]
Stopping RSA Database Server: **
RSA Database Server                                        [SHUTDOWN]
Stopping RSA RADIUS Server Operations Console: **
RSA RADIUS Server Operations Console                       [SHUTDOWN]
Stopping RSA Administration Server with Operations Console: **
RSA Administration Server with Operations Console          [SHUTDOWN]
rsaadmin@am84p:~>
 

To start the Authentication Manager services at the command line use /opt/rsa/am/server/rsaserv start all:

rsaadmin@am84p:~> /opt/rsa/am/server/rsaserv start all
Starting RSA Database Server:
Starting RSA Administration Server with Operations Console: *************************************
RSA Administration Server with Operations Console          [RUNNING]
Starting RSA RADIUS Server Operations Console: \ RSA Database Server                                        [RUNNING]                                       *********************
RSA RADIUS Server Operations Console                       [RUNNING]
Starting RSA Runtime Server: **************************************
RSA Runtime Server                                         [RUNNING]
Starting RSA RADIUS Server: *
RSA RADIUS Server                                          [RUNNING]
Starting RSA Console Server: *******************************************
RSA Console Server                                         [RUNNING]
rsaadmin@am84p:~>
 

Please run through the procedure for cleaning up unresolvable users manually again after the stop and start of the primary and all replica instances in the Authentication Manager deployment. Check for the administrative user in the list where unresolvable users were found.

Where the administrative user was found and cleaned up then check the real-time system activity to confirm the message is no longer being reported. Refer to Real-Time Monitoring Using Activity Monitors for information on real-time activity monitors.

If this issue still persists please contact RSA Customer Support and open a support case.