On the Host to be Removed (if it is still available)
1. Either SSH or use a Console (such as iDRAC console) to connect to node you are removing from the trust model
2. Obtain the puppet node_idOn that Node, obtain the node_id
/etc/puppet/scripts/node_id.py
Example Output:
# /etc/puppet/scripts/node_id.py
b93bdb62-b0e4-4141-a03b-dac803da73a7
Other ways of obtaining puppet node_id
echo $(</var/lib/puppet/node_id)
facter -p node_id
3. Stop services
service collectd stop
service mcollective stop
puppet resource service puppet ensure=stopped
service rabbitmq-server stop
4. Clean-up puppet
rm -rf /var/lib/puppet/ssl
rm -f /var/lib/puppet/state/agent_catalog_run.lock
rm -f /var/lib/puppet/client_data/catalog/$(/etc/puppet/scripts/node_id.py).json
rm -f /var/lib/puppet/yaml/facts/$(/etc/puppet/scripts/node_id.py).yaml
rm -f /var/lib/puppet/yaml/node/$(/etc/puppet/scripts/node_id.py).yaml
rm -rf /var/lib/puppet/reports/$(/etc/puppet/scripts/node_id.py)
5. Remove trustpeers
find /etc/netwitness/ng -name 'trustpeers' -type d -exec rm -rf {} \; 2>/dev/null
On the NetWitness Head Unit Server/SA Server which hosts the puppetmaster:6. The following 3 commands are optional however may improve the likelihood of success
service puppetmaster restart
service mcollective restart
service rabbitmq-server restart
7. Run delNode.pySyntax:
/etc/puppet/scripts/delNode.py <node_id>
Example:
/etc/puppet/scripts/delNode.py b93bdb62-b0e4-4141-a03b-dac803da73a7
8. The previous step runs puppet cert clean <node_id>
Check that there are no current requests for this node_id:
ls /var/lib/puppet/ssl/ca/requests -l
ls /var/lib/puppet/ssl/ca/signed -l
9. Remove from puppetmaster inventory.txt
mv /var/lib/puppet/ssl/ca/inventory.txt /var/lib/puppet/ssl/ca/inventory.old
Syntax:
grep -v <node_id> /var/lib/puppet/ssl/ca/inventory.old > /var/lib/puppet/ssl/ca/inventory.txt
Example:
grep -v b93bdb62-b0e4-4141-a03b-dac803da73a7 /var/lib/puppet/ssl/ca/inventory.old > /var/lib/puppet/ssl/ca/inventory.txt
10. Remove references from mongoDB
Syntax:
node_target=<node_id>
applianceId=$(echo 'db.appliance_info.find( { "identity" : "'$node_target'" },{"_id":1} )' | mongo asg | grep -o "ObjectId(.*" | cut -d\" -f2)
echo 'db.appliance_info.remove( { "identity" : "'$node_target'" } )' | mongo asg
echo 'db.service_info.remove( { "applianceId" : "'$applianceId'" } )' | mongo asg
echo 'db.appliance_update.remove( { "identity" : "'$node_target'" } )' | mongo puppet
echo 'db.entitlement.remove( { "identity" : "'$node_target'" } )' | mongo sa
echo 'db.service_info.remove( { "hostId" : "'$node_target'" } )' | mongo les
Example:
node_target='b93bdb62-b0e4-4141-a03b-dac803da73a7'
applianceId=$(echo 'db.appliance_info.find( { "identity" : "'$node_target'" },{"_id":1} )' | mongo asg | grep -o "ObjectId(.*" | cut -d\" -f2)
echo 'db.appliance_info.remove( { "identity" : "'$node_target'" } )' | mongo asg
echo 'db.service_info.remove( { "applianceId" : "'$applianceId'" } )' | mongo asg
echo 'db.appliance_update.remove( { "identity" : "'$node_target'" } )' | mongo puppet
echo 'db.entitlement.remove( { "identity" : "'$node_target'" } )' | mongo sa
echo 'db.service_info.remove( { "hostId" : "'$node_target'" } )' | mongo les
On the Host to be Removed
11. Stop services in upstart
declare -a services_to_stop=( $(find /etc/init \( -name "nw*" -o -name "rsa*" \) -type f -exec basename {} \; | sed 's/.conf$//g' | grep -v -E "(bad|disable|old)" | sort -r) )
for target in "${services_to_stop[@]}"; do $(stop "$target") 2>/dev/null; done
12. Restart/Reboot OS
shutdown -r now
