000037905 - nwbroker.service is unable to start in RSA NetWitness Platform 11.3.x

Document created by RSA Customer Support Employee on Sep 12, 2019
Version 1Show Document
  • View in full screen mode

Article Content

Article Number000037905
Applies ToRSA Product Set: NetWitness Logs & Network
RSA Product/Service Type: Core Appliance
RSA Version/Condition: 11.3.1.0
Platform: CentOS 7
IssueNwBroker service was failing to start as shown below in NW 11.3.x.

# service nwbroker status
Redirecting to /bin/systemctl status nwbroker.service
   nwbroker.service - Netwitness Broker
   Loaded: loaded (/usr/lib/systemd/system/nwbroker.service; enabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Wed 2019-08-21 15:20:36 KST; 24h ago
 Main PID: 259920 (code=exited, status=1/FAILURE)

Aug 21 15:20:35 mss-broker1 systemd[1]: Unit nwbroker.service entered failed state.
Aug 21 15:20:35 mss-broker1 systemd[1]: nwbroker.service failed.
Aug 21 15:20:35 mss-broker1 systemd[1]: nwbroker.service holdoff time over, scheduling restart.
Aug 21 15:20:35 mss-broker1 systemd[1]: start request repeated too quickly for nwbroker.service
Aug 21 15:20:35 mss-broker1 systemd[1]: Failed to start Netwitness Broker.
Aug 21 15:20:35 mss-broker1 systemd[1]: Unit nwbroker.service entered failed state.
Aug 21 15:20:35 mss-broker1 systemd[1]: nwbroker.service failed.
Warning: nwbroker.service changed on disk. Run 'systemctl daemon-reload' to reload units.

# /usr/sbin/NwBroker
(i) 2019-Aug-22 15:49:45 [Engine]  RSA NetWitness Service Copyright 2001-2019, RSA Security Inc. All Rights Reserved.
(i) 2019-Aug-22 15:49:45 [Engine]  Running broker in console
(d) 2019-Aug-22 15:49:45 [Engine]  [broker](7f1318d5d940): Entering ServiceBase::Initialize()
(d) 2019-Aug-22 15:49:45 [Engine]  [broker](7f1318d5d940): ServiceBase::SetStatus(Stopped, Start Pending)
(a) 2019-Aug-22 15:49:45 [Engine]  RSA NetWitness Service, Broker 11.3.1.0 (Jun 14 2019) 64 bit Starting
(F) 2019-Aug-22 15:49:45 [Engine]  Failed to start engine because of exception: Throw in function X509* nw::{anonymous}::getX509FromPEM(const boost::filesystem::path&)
Dynamic exception type: boost::exception_detail::clone_impl<nw::Exception>
std::exception::what: error loading trusted certificate file
[nw::ssl_error_tag*] = error:0E06D06C:configuration file routines:NCONF_get_string:no value error:02001002:system library:fopen:No such file or directory error:2006D080:BIO routines:BIO_new_file:no such file
[boost::errinfo_at_line_*] = 45
[boost::errinfo_file_name_*] = /etc/netwitness/ng/broker/trustpeers/c5al34bl.0
[boost::errinfo_api_function_*] = BIO_new_file

# ls -al /etc/netwitness/ng/broker/trustpeers/ <-- You need to note this information for restoration after pem file regeneration
total 0
drwxr-x---. 2 netwitness netwitness 23 Aug  1 07:14 .
drwxr-x---. 6 netwitness netwitness 90 Aug  1 07:13 ..
lrwxrwxrwx. 1 root       root       67 Aug  1 07:14 fdc2f8fd.0 -> /etc/pki/nw/peer/sa-server/d4edb4d8-3362-4568-991b-ef5d627dea0c.pem
 
CauseFor some reason, the pem certificate file for the sa-server service id was broken or missing in /etc/pki/nw/peer/sa-server.
In this case, nwbroker service is unable to start.
ResolutionYou need to re-generate the pem file for the sa-server service id in this case.

Please follow the steps below.
  1. Get ssl certificate information and save it to file('root.out')
    # openssl s_client -connect localhost:7000 -tls1_2 > root.out
     
  2. Edit it using vi, then extract content and save it as a d4edb4d8-3362-4568-991b-ef5d627dea0c.pem file.
    Note: certificate is the copy of the section:

    -----BEGIN CERTIFICATE-----
    to
    -----END CERTIFICATE-----
     


  3. Locate pem file into /etc/pki/nw/peer/sa-server/ and link it same as before.
    # ls -al /etc/netwitness/ng/broker/trustpeers
    total 0
    drwxr-x---. 2 netwitness netwitness 23 Aug  1 07:14 .
    drwxr-x---. 6 netwitness netwitness 90 Aug  1 07:13 ..
    lrwxrwxrwx. 1 root       root       67 Aug  1 07:14 fdc2f8fd.0 -> /etc/pki/nw/peer/sa-server/d4edb4d8-3362-4568-991b-ef5d627dea0c.pem

    Once completed, you are able to start nwbroker.service without any issue.

     


Attachments

    Outcomes