000037508 - RSA NetWitness Logs and Network 11.3 Chef run fails with "nw_pki_openssl_hashed_cert" error in chef-solo.log

Document created by RSA Customer Support Employee on Jul 10, 2019
Version 1Show Document
  • View in full screen mode

Article Content

Article Number000037508
Applies To
RSA Product Set: NetWitness Logs and Network
   RSA Product/Service Type: Orchestration/Chef
   RSA Version/Condition:  11.3.0.X
   Platform: CentOS
   O/S Version: 7
IssueWhile doing an operation that requires Chef to run, such as an install, upgrade, or using the new certificate reissue command, the process can stop with this kind of error. Depending on the action and the device being acted on, this may result in a device appearing offline or unable to be talked to until resolved. This error can be found on the /var/log/netwitness/config-management/chef-solo.log file.

[2019-05-08T23:28:10-03:00] ERROR: Running exception handlers
[2019-05-08T23:28:10-03:00] ERROR: Exception handlers complete
[2019-05-08T23:28:10-03:00] FATAL: Stacktrace dumped to /var/lib/netwitness/config-management/cache/chef-stacktrace.out
[2019-05-08T23:28:10-03:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2019-05-08T23:28:10-03:00] ERROR: nw_pki_openssl_hashed_cert[nw-appliance /etc/netwitness/ng/appliance/trustpeers -> ["sa-server"]] (nw-appliance::trusts line 19) had an error:
Mixlib::ShellOut::ShellCommandFailed: execute[launch-peer-cert:sa-server] (/var/lib/netwitness/config-management/cache/cookbooks/nw-pki/resources/openssl_hashed_cert.rb line 65) had an error:
Mixlib::ShellOut::ShellCommandFailed: Command execution failed. STDOUT/STDERR suppressed for sensitive resource
[2019-05-08T23:28:10-03:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

If you review the stack trace, you may see something similar to the following.

---- Begin output of security-cli-client --get-certificates-for-service         --service sa-server --output-dir /etc/pki/nw/peer/sa-server -u deploy_admin -k #%Funky!Password -b cfdb9351-01d6-4d5c-9cfe-da36ccadb98c ----
STDERR: security-cli-client: option requires an argument -- 'k'
---- End output of security-cli-client --get-certificates-for-service         --service sa-server --output-dir /etc/pki/nw/peer/sa-server -u deploy_admin -k #%Funky!Password -b cfdb9351-01d6-4d5c-9cfe-da36ccadb98c ----
Ran security-cli-client --get-certificates-for-service --service sa-server --output-dir /etc/pki/nw/peer/sa-server -u deploy_admin -k #%FZ!JF81w -b cfdb9351-01d6-4d5c-9cfe-da36ccadb98c returned 1

CauseThis can occur if you are using a deployment password that contains special characters that are being wrongly interpreted in the bash shell. This should be addressed at some point in a future release yet to be determined at the time of this writing.
ResolutionPlease change the deployment password and restart the upgrade through CLI if the UI is unavailable. Please see this KB article for directions how: