envision 4.1 migration
The instructions for the upgrade has been documented in the "RSA enVision Migration Guide". We would request you to follow the steps from the guide and at the same time pass on your feedback too if you come across any issues.
1. RCs stopped sending data for various reasons. Partly fixed. The pi_ls_forwarder.exe is still hanging at times.
2. VAM stopped working (can't run asset reports, the Foundstone VAM SQL is bugged) - No current fix
3. Task Triage not working - Alerter will not create new tasks. - No current fix 4. LC stopped moving FTP files to the NAS, we got this fixed last week
This all happened after 4.1 upgrade.
First obstacle: the password change process (or pre-process) was horrendous. I had to used the Windows 2003 Resource Kit to connect in to NAS and change the passwords, then the OS / AD passwords and then the services. After which to key in those passwords into the envision 4.1 installer.
Second obstacle: there is a character limit to the NAS ID passwords (max 14 chars i think, if you are using the windows 2003 resource kit). So make sure both the NAS ID passwords and the installer's GUI is able to accept the same passwords. If the password doesn't fit the complexity, repeat first obstacle all over again, across all your devices.
Third obstacle: the password change doesn't seem to take effect immediately - the 4.1 installer kept failing when verifying the passwords for about an hour or so before getting through.
Fourth obstacle: The installer exited prematurely, citing error in attempting to connect to a NAS IP Address (which was totally wrong, not sure where did it get that IP Address from). Checked the log file, it says the installation failed. checked the D-Serv file system, the folder already changed from 4000 to 4100. So i'm stuck in-between. Some other NIC devices succeeded, some with errors here and there (saw in the log files) but completed without any hickups.
Fifth obstacle: Not sure why, enVision started the NIC device authentication on its own (recall the whitelist and device authentication launched in SP2). Had to re-do the backward compatibility fix (also detailed in SP2 readme).
Sixth obstacle: The clustering (my site runs EA envision) fails for the collectors. Nodes no longer able to be activated in the cluster group (although can start service).
Seventh obstacle: Some device can't even start the installation at all, the installer throws you a: Failed retrieving master password error (or something along the line).
Anyway, waiting for RSA support to take a look tomorrow.
The whole process took perhaps 3-4 hours. Here’s what we did:
1. Read through the entire Migration Guide
2. Took a final backup of the enVision system.
3. Installed latest Signature Content Updates and VAM Updates
4. Confirmed that there were no errors from the updates.
5. Compacted the nic.db file with the command dbcompact.cmd since it was larger than 500MB (the enVision 4.0 online help has instructions on how to do this and what parameters to use). Note that we had to install a patch from RSA support for the dbcompact to not always fail with an error message.
6. Proceeded to install the enVision 4.1 upgrade. (This alone took about 1 hour on our appliance. The backup procedure that it does at the beginning seem to be what took the longest).
7. Kept getting “folder in use” errors at the beginning of the install. Went through the services list and stopped all services for software that we installed on the enVision server. Took three attempts at stopping more and more services until finally this error disappeared and we could proceed.
8. Confirmed the integrity of the completed installation by going through the steps instructed in the migration guide. We also went in each device type and confirmed that the RSA was receiving the logs for those devices, and glanced at the events in Windows Event Viewer.
9. Run a query against the NIC System events in the enVision (filtering by severity levels 0-3) in order to identify any critical errors.
10. Had to reset the nic_sshd and nic_sftp password cache in WinSSHD control panel (take a look at the Windows Application events and look for WinSSHD errors). Restarted WinSSHD.
11. Proceed to install enVision 4.1 patch 4.
12. Run the password update script in E:\nic\4100\node-name\password\update scripts\ to reset the nic_system password (P.89 of the RSA envision Hardware Setup and Maintenance Guide) because for some reason it kept getting locked out by one of the NIC services.
13. Run the command manageweaklogins -r to identify which accounts haven’t had the password hashes converted to SHA256 yet.
14. Update the path from E:\nic\4000\ to E:\nic\4100\ for any 3rd party software or scripts that referenced the old file path.
15. Upgraded/installed VMware Collector Service 1.0 to 1.1. Took 2 attempts to get this to work properly (had to uninstall and reinstall)
16. Upgraded/installed Windows Eventing Collector service 1.0 to 1.1
Thanks j2008!! Your tips helped prepare us for the hurdles we experienced so that we had the dbcompact Sybase patch from Support ahead of time.
One item I'd like to point out because I didn't really understand the implications of the 4.1 install password change:
- If you were still using default passwords for any of the enVision accounts (master, administrator, nic_system, nic_sshd, nic_sftp) prior to the upgrade, even if those passwords meet the minimum requirements, you CANNOT reuse those passwords. You must choose new passwords during the upgrade. However, if you were using non-default passwords for any of the enVision accounts and those passwords met the minimum requirements, you COULD reuse those passwords.
Hope this adds one more tip to ease the next person's upgrade.