000038783 - After an upgrade to 11.4, RSA NetWitness is inaccessible and unable to access page

Document created by RSA Customer Support Employee on May 6, 2020
Version 1Show Document
  • View in full screen mode

Article Content

Article Number000038783
Applies ToRSA Product Set: RSA NetWitness Platform
RSA Product/Service Type: Core Appliance
RSA Version/Condition: 11.4, 11.4.1
IssueFrom sa.log, you can see error "Request to admin-server.any./rsa/process/ready timed out"

at java.base/java.lang.Thread.run(Thread.java:834)
[taskScheduler-5] ERROR com.netwitness.platform.server.common.atmosphere.WebSocketSessionExpiry - Error retrieving idle session timeout settings from admin-server
com.rsa.asoc.launch.api.transport.client.RequestTimeoutException: Request to admin-server.any./rsa/process/ready timed out
at com.rsa.asoc.launch.api.transport.client.AmqpTransportClientHelper.requestTimeoutException(AmqpTransportClientHelper.java:51)
at com.rsa.asoc.launch.api.transport.client.AmqpTransportClientHelper.throwRequestTimeoutException(AmqpTransportClientHelper.java:44)
at com.rsa.asoc.launch.api.transport.client.AmqpTransportClient.doSendAndReceive(AmqpTransportClient.java:115)
at com.rsa.asoc.launch.api.transport.client.AmqpTransportClient.send(AmqpTransportClient.java:69)
at com.rsa.asoc.launch.api.transport.client.TransportClientInvocationHandler.makeRemoteCall(TransportClientInvocationHandler.java:68)
at com.rsa.asoc.launch.api.transport.client.TransportClientInvocationHandler.invoke(TransportClientInvocationHandler.java:50)

From admin-server log, we can see that certificate is untrusted.


[ main] WARN Security|Certificate for CN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx,OU=NetWitness Platform,O=RSA,L=Reston,ST=VA,C=US issued by CN=Puppet CA: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx is not trusted

 


 

CauseDuring the upgrade, the trust certificate in /etc/netwitness/admin-server/ was modified.  The upgrade happened in this case on Feb 10, something tried changing the certificate of keystore.p12 during the time of the update Feb 10 15:56:
 

[root@xxxx admin-server]# ls -alh
total 84K
drwxr-xr-x. 2 netwitness netwitness 4.0K Feb 10 15:56 .
drwxr-xr-x. 24 netwitness netwitness 4.0K Feb 11 17:43 ..
r-------. 1 netwitness netwitness 576 Jan 11 06:47 admin-server.conf
rw-r----. 1 netwitness netwitness 14K Feb 10 15:56 keystore.p12
rw-rr-. 1 root root 13K Jan 6 18:47 keystore.p12.good
rw-rr-. 1 root root 12K Jan 6 18:47 keystore.p12.good.new
rw-rr-. 1 root root 13K Jan 6 18:47 keystore.p12.orig
rw-rr-. 1 netwitness netwitness 986 May 23 2018 lockbox.ss
rw-rr-. 1 netwitness netwitness 0 May 23 2018 lockbox.ss.lock
rw-rr-. 1 netwitness netwitness 240 Feb 10 15:56 modules.yml
rw-rr-. 1 netwitness netwitness 36 May 23 2018 service-id


 
Workaround
  1. Stop rsa-nw-admin-server service:
    systemctl stop rsa-nw-admin-server.service
  2. Change directory to admin-server:
    cd /var/log/netwitness/admin-server
  3. Backup existing keystore file:
    mv keystore.p12  keystore.p12.backup
  4. Replace keystore.p12 with the last known working keystore.p12.good.new:
    cp keystore.p12.good.new  keystore.p12
  5. Set permissions to keystore.p12:
    chmod 640 keystore.p12
    chown netwitness:netwitness keystore.p12
  6. Start rsa-nw-admin-server service:
    systemctl start rsa-nw-admin-server.service
  7. Restart the web application server service jetty:
    systemctl restart jetty
NotesIf the last known backup keystore.p12 is not working you may have to regenerate the certificates. To do so please see article below: 

  

Reissue root CA security certificates on RSA NetWitness Platform 11.x


   https://community.rsa.com/docs/DOC-107280

Attachments

    Outcomes