Error "Key negotiation exchange failed. Server response was CANCELLED" with RSA Authentication Agent API 8.5 and later
RSA Product Set: RSA SecurID RSA Product/Service Type: RSA Authentication Agent API RSA Version/Condition: 8.5 and later
When testing the TCP Authentication from RSA Authentication API 8.5 and later, or any other agents that uses these APIs for Authentication in the TCP mode this is the error seen:
com.rsa.authagent.authapi.AuthAgentException: Error in initial AuthnReq/Rsp for serverTime.Error in processing Authn request: connect exception processing key negotiation request: com.rsa.authmgr.commonagent.h: Key negotiation exchange failed. Server response was CANCELLED
com.rsa.authagent.authapi.AuthAgentException: com.rsa.authagent.authapi.AuthAgentException: Error in initial AuthnReq/Rsp for serverTime.Error in processing Authn request: connect exception processing key negotiation request: com.rsa.authmgr.commonagent.h: Key negotiation exchange failed. Server response was CANCELLED at com.rsa.authagent.authapi.AuthSessionFactory.a(AuthSessionFactory.java) at com.rsa.authagent.authapi.AuthSessionFactory.getInstance(AuthSessionFactory.java) at sample.AuthUser.<init>(AuthUser.java:32) at sample.AuthUser.main(AuthUser.java:62)
If you browse to Security Console –> Setup –> System Settings --> Agents, then click on To configure agents using IPV6, click here. This is the certificate the AM uses in the communication with the TCP Agents. If this certificate is not correct then the authentication will fail with the above error.
The certificate found in the above page should be the same as the one we can export after accessing https://AM fully qualified domain name:7002
Using Google Chrome, browse tohttps://AM fully qualified domain name:7002
Click on the lock icon in the browser address bar.
Click on Certificate.
Click the Certification Path tab.
Double-click on the top-level (root, very first) certificate in the list.
Click on Details tab, then Copy to File...
Click Next, then check the Second Option Base-64 encoded X.509 output format (.CER)
Click Next, then click Browse to choose the location and give it any name, such as root then Click Save.
Click Next then Finish, you'll find the exported certificate in the location chosen in Step 8
Browse to Security Console –> Setup –> System Settings --> Agents, then click on To configure agents using IPV6, click here.
Scroll down under Existing Certificate Details, click on the Choose File Option then browse to the certificate we just exported then Click Update.
If the Authentication Manager is on 8.2 SP1 till 8.2 SP1 Patch 4, you can still see the above errors even after you complete the steps in the Resolution. This is because there is a bug on AM 8.2 SP1 where it doesn't communicate with the certificate we export from the 7002 port that it should use, it communicates with the Console Certificate. The environments that will see this are the environments that have replaced the default self-signed certificate for the consoles with another certificate.
Follow the exact steps in the resolution here, but in step 1 rather than browsing to https://AM fully qualified domain name:7002, browse to either the Security Console or the Operations Console of the Authentication Manager and export the certificate from there, then complete the same steps as they are.