Error "No appropriate protocol" in RSA Access Manager 6.2
4 years ago
Originally Published: 2019-05-31
Article Number
000044077
Applies To
RSA Product Set: Access Manager
RSA Version/Condition: 6.2

 
Issue
The aserver standard output shows the following exception:
  
17:11:40:004 [ssl] [DispatcherReg] - Starting the handshake...
Error connecting to the dispatcher at: axm-server:5607
	No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
	Retry in 10 seconds

The dispatcher.log shows the following exception.    
sequence_number=2247,remote_client=169.254.185.199,2019-05-31 16:22:20:49
EDT,messageID=0,event_type=Error,description=Error reading client
input,error=javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)

 
Cause
RSA Access Manager Server is configured to use Java 8 Update 211 (8u211).  This issue may occur when using Java 8 Update 201 (8u201) or later. The issue occurs due to disabled TLS anon protocols in later builds of Java 1.8.   Anon authentication type is disabled by default in current versions of Java.

See Java 8 Release Highlights.
 
Resolution
RSA Access Manager should be configured to use the more secure AUTH authentication types.   See the RSA Access Manager documentation for how to configure AUTH authentication mode. 
Workaround
Follow the steps below to allow TLS anonymous (anon) in Java for RSA Access Manager to use anon:
  1. Edit the java.security file (C:\Program Files\Java\jdk1.8.0_211\jre\lib\security\java.security).
  2. Modify the disabledAlgorithms section.
jdk.tls.disabledAlgorithms=SSLv3, RC4, DES, MD5withRSA, DH keySize < 1024, \
    EC keySize < 224, 3DES_EDE_CBC, anon, NULL
  1. Remove the anon algorithm type.
jdk.tls.disabledAlgorithms=SSLv3, RC4, DES, MD5withRSA, DH keySize < 1024, \
    EC keySize < 224, 3DES_EDE_CBC, NULL

If the SSL TLS handshake completes correctly, it should show it using the TLS_DH_anon TLS protocol.  (enable -DDEBUG=SSL to show SSL debugging.)
 
18:20:01:109 [ssl] [main] - Enabled protocols for socket: 18:20:01:109 [ssl] [main] - TLSv1,TLSv1.1,TLSv1.2, 18:20:01:109 [ssl] [main] - Adding the Handshake Completed Listener... 18:20:01:109 [ssl] [main] - Starting the handshake... 18:20:01:141 [ssl] [MuxStreamReader-0] - SSL Session info for: [Session-3, TLS_DH_anon_WITH_AES_128_CBC_SHA] 18:20:01:141 [ssl] [MuxStreamReader-0] - Cipher TLS_DH_anon_WITH_AES_128_CBC_SHA 18:20:01:141 [ssl] [MuxStreamReader-0] - Create: 1559341201125 18:20:01:141 [ssl] [MuxStreamReader-0] - ID: [B@6242b9a9 18:20:01:141 [ssl] [MuxStreamReader-0] - Last: 1559341201141 18:20:01:141 [ssl] [MuxStreamReader-0] - Context: sun.security.ssl.SSLSessionContextImpl@6c855b91 18:20:01:141 [ssl] [MuxStreamReader-0] - SSLPeerUnverifiedException encountered. 18:20:01:141 [ssl] [MuxStreamReader-0] - javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated 18:20:01:141 [ssl] [MuxStreamReader-0] - 18:20:01:141 [ssl] [HandshakeCompletedNotify-Thread] - SSLSocketFactory: SSL handshake completed. 18:20:01:141 [ssl] [HandshakeCompletedNotify-Thread] - SSL cipher suite: TLS_DH_anon_WITH_AES_128_CBC_SHA 18:20:01:141 [ssl] [HandshakeCompletedNotify-Thread] - SSL socket: class sun.security.ssl.SSLSocketImpl 18:20:01:141 [ssl] [HandshakeCompletedNotify-Thread] - SSL Session info for: [Session-4, TLS_DH_anon_WITH_AES_128_CBC_SHA] 18:20:01:141 [ssl] [HandshakeCompletedNotify-Thread] - Cipher TLS_DH_anon_WITH_AES_128_CBC_SHA
Don't see what you're looking for?