|Applies To||RSA Product Set: NetWitness Platform|
RSA Product/Service Type: Orchestration/Chef
RSA Version/Condition: 11.2,11.3, 11.4
O/S Version: 7
|Issue||While updating/installing a device to version 11.2 or above, the following error can occur and be found in /var/log/netwitness/config-management/chef-solo.log:|
Thus, the install/upgrade fails.
|Cause||The reason can be because the target host is unable to communicate to the Admin Server on port 53 as it is attempting to use the dnsmasq service on the Admin Server to resolve, in this case, 889e5752-6ae3-4286-a944-c182 33f4ccbc. This is the salt minion id of the admin server. You can see this by running "cat /etc/salt/minion" on the Admin Server to compare.|
|Resolution||If possible, configure any firewalls between the target host and the Admin Server host to be able to communicate on port 53.|
If this is not possible, the workaround is to include the minion id in the /etc/host file on the component hosts and starting in the 11.4 release, modify the chef recipe not to overwrite this workaround.
|Workaround||Take the example /etc/hosts file from an Endpoint Hybrid host.|
Edit /etc/hosts and add the node id, just like you saw in the error, next to nw-node-zero, if it exists. Otherwise, please add a line that has <IP address of Admin Server> <UUID of Admin Server>
Or add the entire line if there is no existing line:
Next, run a command that resolves the name to ensure you successfully made this change as well as saving this value to the resolver cache on the device in question. Ping will work but note that ICMP is blocked by default so this command will normally appear unsuccessful. You can also attempt to curl on a port that should be accessible. It does not matter for our purposes. We care if the name resolves to the correct IP address.
If you are on version 11.4+:
An additional step must be performed if you are on 11.4 or above. The chef recipes attempt to overwrite our workaround from above. So, we must modify the chef recipes to work around this issue. First, we must edit a file:
You locate this section of code and comment it out so that it does not remove our workaround.
Change all this by doing adding '#' to the beginning of the lines.
Please note that whenever you upgrade, you have to reapply these changes each time this file is updated, as part of the upgrade.
After completing the above steps, you may attempt to upgrade once more while tailing the /var/log/netwitness/config-management/chef-solo.log and see if you bypass this error.
A Very Special Note about resolv.conf:
In 11.3 and above, we are making the /etc/resolv.conf an immutable file as part of our Chef process. If you are unable to reach the Admin Server on port 53 or your component host uses a different DNS Server from your Admin Server, you must edit the local resolv.conf on the component host. To be able to edit the file to change what DNS Servers you query, you must undo this change.
Once this is done, you can restore your DNS server settings by vi-ing the file. If you are unsure what they were prior to your upgrade, you can check the backup files that the chef creates as it goes through it is upgrade run. They are date-stamped in the file name.
Also, note that the Admin Server is different. The options in /etc/resolv.conf are being overwritten by what is defined in /etc/netwitness/platform/resolv.dnsmasq. If you want to change the Admin Server's DNS Servers, you must modify it there.
|Notes||If this solution does not work for you and you are still experiencing issues after following the steps in this KB article, please open a case with RSA Technical Support quoting this KB article.|