Authentication Manager 8.8 update breaks TLS connections; TLS Handshake error no cipher suites in common
3 months ago
Article Number
000073738
Applies To

RSA Product Set: SecurID
RSA Product/Service Type: Authentication Manager
RSA Version/Condition: AM 8.8 and later

Issue

AM 8.8 adds support for TLSv.1.3, but also restricts certain older ciphersuites that are considered less secure, e.g. TLS_RSA_WITH_AES_256_GCM_SHA384 with TLSv1.2.

Products that do not support TLSv.1.3, such as the EOPS Authentication Agent for Windows, AAWin v. 7.4.x, and some other older LDAPS servers, will no longer connect with AM 8.8 when they previously did at ver. AM 8.7 SP2 or earlier.

This work-around Knowledge Base, KB article explains how to enable a ciphersuite in AM to allow these older products to connect. 

Starting with AM 8.8, TLS Ciphersuites no longer controlled by WebLogic in /opt/rsa/am/server/config/config.xml but that control has moved to Java in 
/opt/rsa/am/appserver/jdk/jre/lib/security/java.security

 

Cause

TLS connections begin after a TCP 3-way Handshake establishes a reliable TCP connection, when the initiating side, the Client, sends a TLS Hello packet, which includes the version of TLS the client supports, and a list of ciphersuites that the Client is capable of using.

The Server side is the node that responds to the initiating side, and it's response will include the version of TLS they will use, e.g. TLSv1.3 or TLSv1.2 and which Cipher or ciphersuite. 

When an AAWin v.7.4.x agent attempts to Auto-Register to an AM primary or replica, that AAWin agent is the TLS Client and AM is the TLS server. But when an AM primary or replica seeks to connect to an external LDAP Identity Source, that AM server is the TLS client.

A TCPDump or Wireshark network packet capture of a TLS connection will show the TLS Client Hello. 

TLS_Client_Hello_ciphersuites

 <screen shot - TLS_Client_Hello_ciphersuites.png>

When a TLS connection fails with "no cipher suites in common" you must either update the TLS client to support one of the stricter ciphers suites on the TLS server, or you must configure the TLS server to allow one of the less secure ciphersuites seen in the TLS Hello, e,g, TLS_RSA_WITH_AES_256_GCM_SHA384

Resolution

SSH to Linux on the RSA AM appliance, or gain console access, authenticating with the rsaadmin account

Make a backup of: /opt/rsa/am/appserver/jdk/jre/lib/security/java.security

    cd /opt/rsa/am/appserver/jdk/jre/lib/security/

    cp java.security java.security.orig

java_security_1_ssh

<screenshot - java_security_1_ssh.png>

Edit java.security

      vi java.security

java_security_2_top_of_file

<screenshot - java_security_2_top_of_file.png>

Look for the line starting:

jdk.tls.disabledAlgorithms=

and the following line containing:

     TLS_RSA_WITH_AES_256_GCM_SHA384

java_security_3_jdk.tls.disabledAlgorithms

<screenshot - java_security_3_jdk.tls.disabledAlgorithms.png>

Then delete "TLS_RSA_WITH_AES_256_GCM_SHA384," from that line

java_security_4_insertmode_backspace_over_cipher

<screenshot - java_security_4_insertmode_backspace_over_cipher.png>

java_security_5_backspace.png

<screenshot - java_security_5_backspace.png>

Restart AM services.

  cd /opt/rsa/am/server

  ./rsaserv restart all

rsaserv_restart_all

<screen shot – rsaserv_restart_all.png>

 

Workaround

The resolution above is a work-around solution.

At this time it is unknown if any related security issues would be triggered by this change.

It is also very likely that changes made to java.security would be undone by AM patches.

Use at your own risk. You are enabling a weaker ciphersuite because your TLS connection failed. The correct solution is to update the TLs side so that it can support the latest and most secure ciphersuites.

 

Notes

Before attempting this process, review the article entitled Does RSA support the installation of third party software or applications on the RSA Authentication Manager 8.X (virtual) appliances?  Each customer making this change needs to assess the risk/benefit of this 'unsupported' change to the AM appliance for their organization. This should be viewed as a temporary solution.