000032379 - RSA SecurID STIG hardening can prevent the RSA Authentication Agent for Microsoft Active Directory Federation Services (ADFS) from authenticating

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 4Show Document
  • View in full screen mode

Article Content

Article Number000032379
Applies ToRSA Product Set: SecurID
RSA Product/Service Type:  RSA Authentication Agent for AD FS
RSA Version/Condition: 1.0.1
Platform: Windows

Security Technical Implementation Guide (STIG) type hardening or implementing other Group Policy GPOs can prevent the RSA Authentication Agent for Microsoft AD FS from authenticating with the some of following error messages:

[AuthSession.SubmitPasscode] Exception occurred:

Authentication, including test authentication from the AD FS agent's RSA Control Center, will fail and nothing will appear in the real time authentication activity logs or authentication reports for the primary or replica servers.
The AD FS agent's trace.log will have the following messages where the Lock Request (ACM_REQ_LOCK_NAME), which comes just before the authentication request, is not sent

[5728] 14:18:15.801 File:acmgt.c Line:524 # bMgtCallback() [0x197B5990] AUTH_DONE 
[5448] 14:18:15.801 File:newsd_api.c Line:162 # Leaving SD_Close() return: 0 
[5728] 14:18:15.801 File:acutil.c Line:361 # DeleteFromUserList() [0x197B5990] 1/1 [mreynolds] 
[5728] 14:18:15.801 File:acutil.c Line:410 # DeleteFromUserList() [0x197B5990] version=5, type=
ACM_REQ_LOCK_NAME(91), 1 tsec

The ADFS agent's Control Center log (RSAControlCenter.log) will have the following messages:

2016-01-15 15:03:12.172 4036.5 [I] [AuthSession.Authenticate] Authentication sequence started for user:admin
2016-01-15 15:03:12.172 4036.5 [V] [AuthSession.SubmitPasscode] Enter
2016-01-15 15:03:12.173 4036.5 [V] [AuthAPIServiceChannel..ctor] Enter
2016-01-15 15:03:12.173 4036.5 [V] [AuthAPIServiceChannel..ctor] Return
2016-01-15 15:03:12.173 4036.5 [V] [AuthAPIServiceChannel.SD_Lock] Enter
2016-01-15 15:03:12.197 4036.5 [V] [AuthAPIServiceChannel.SD_Lock] Return
2016-01-15 15:03:12.197 4036.5 [I] [AuthSession.SubmitPasscode] SD_Lock returned 0
2016-01-15 15:03:12.197 4036.5 [V] [AuthAPIServiceChannel.Dispose] Enter
2016-01-15 15:03:12.198 4036.5 [V] [AuthAPIServiceChannel.Dispose] Return
2016-01-15 15:03:12.198 4036.5 [E] [AuthSession.SubmitPasscode]
Exception occurred:System..
'RSA.Authentication.SecurID.AuthSession' threw an exception. ---> System.Reflection.TargetInvocationException:
Exception has been thrown by the target of an invocation.
-> System.InvalidOperationException: This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms.
   at System.Security.Cryptography.SHA256Managed..ctor()
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, Culture
   at System.Security.Cryptography.CryptoConfig.CreateFromName(String name, Object[] args)
   at System.Security.Cryptography.SHA256.Create(String hashName)
   at RSA.Authentication.SecurID.AuthSession..cctor()
   --- End of inner exception stack trace ---
   at RSA.Authentication.SecurID.AuthSession.GenerateSessionEntropy(Int32 sessionHandle)
   at RSA.Authentication.SecurID.AuthSession.SubmitPasscode(SecureString passcode, Boolean isTestAuthentication)

2016-01-15 15:03:12.200 4036.5 [V] [AuthSession.SubmitPasscode] Return
2016-01-15 15:03:12.200 4036.5 [I] [AuthSession.Authenticate] Authentication sequence ended for user:bemblyadmin
2016-01-15 15:03:12.200 4036.5 [V] [AuthSession.Authenticate] Return
2016-01-15 15:03:12.200 4036.5 [E] [TestAuthentication.PerformAuthentication] Authentication failed.
CauseThe RSA Authentication Agent 1.0.1 for Microsoft AD FS Administrator’s Guide states that the agent needs firewall access through both UDP port 5500 and TCP port 5500, indicating that this agent invokes Authentication Manager Agent API 8.5 calls, such as AgentCryptoJProvider.getJSafeType.  CryptoJ.jar is in the classpath, using that to invoke a non-FIPS version of JSafe functions.
If a GPO or STIG configuration requires FIPS, this cannot work. Current versions of this AD FS agent do not support FIPS compliant encryption algorithms
ResolutionTo resolve this issue, disable the FIPS-compliant encryption requirement before performing the first test authentication.  This can be done through a registry edit or from the group policy editor.

From the registry

  1. Navigate to HKEY_LOCALMACHINE\System\CurrentControlSet\Control\Lsa\FIPSAlgorithmPolicy.
  2. Change the value for Enabled to 0, so the FIPSAlgorithmPolicy is disabled, as in the image below:

User-added image

From the Group Policy editor

  1. Select Start > Run.  
  2. Type gpedit.msc and click OK.
  3. When the Local Group Policy Editor launches, find the tree in the left frame.  Navigate to Local Computer Policy > Computer Configuration > Windows Settings > Security Settings > Local Policies > Security Options.
  4. On the right frame, scroll to System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing.
  5. Right click this option and choose Properties.
  6. Change this value from Enabled to Disabled.
  7. To have the changes  take effect, restart the application, such as Internet Explorer.
Optionally, refer this Trend Micro URL to see how to disable FIPS compliance on the Windows Server.
WorkaroundRefer to Microsoft GPO documentation for details on how to change the GPO setting requirement to use FIPS compliant algorithms for encryption.
NotesAAWIN-2260 - Request for Enhancement to Allow Windows and AD FS agents to work when FIPS compliant encryption algorithms are enabled