RSA Product Set: SecurID
RSA Product/Service Type: Authentication Manager
RSA Version/Condition: 8.x

The current implementation of Authentication Manager has been upgraded numerous times, and at some point the database has become fragmented.

• After 4 or more major upgrades (8.1 > 8.1.1 > 8.2 > 8.2.1 > 8.3 > 8.4 >8.5), the database can become degraded.
• RSA Technical Support has documented a process for refreshing the deployment by bringing in new replicas as the current deployed version, then promoting one to Primary and deleting the original primary and replicas.
• This process not only "refreshes" the database but also allows new instances to be built with the 500 GB disk image from 8.6 onwards, instead of the 100 GB image offered prior to 8.6.
• However, there are circumstances when the database is so fragmented that tasks such as Clean Up Unresolvable Users fail to complete.  In these cases we want to export the database to a new machine and try the process on a fresh platform.  If it is successful then we can make this new machine the Primary and build a new deployment based on it.

Pre-requisites:

• The build image that matches the current production version.  For example, if your production deployment is at 8.8 patch 3, you will need to obtain the build image for 8.8.0.0.0.
• The patch that matches production.  In the previous example, we would need the 8.8.0.3.0 patch from the RSA Community portal.
• The appropriate Authentication Manager 8.x license file.

You can leave current current deployment intact while building a new system that will mirror the configuration in production.  Once this new system is working, you can decommission the existing deployment and give the new deployment the same hostnames and IP addresses of the original server.  When this system is online, you can dismantle the old one once the cutover is complete.

Steps:

1. First deploy the new appliance that will become your new primary, using the build image that matches production.  Obtain the necessary appliance image from https://my.rsa.com or from your cloud provider (refer to the appropriate RSA Authentication_Manager_8.x_Getting_Started Guide following the steps listed in the section entitled "Set Up a Primary Instance."
2. Apply the necessary patch so that the software version of your new primary exactly matches that of your existing primary in production.
3. Make a database from the Operations Console of the production primary by navigating to Maintenance > Backup and Restore > Backup Now). Save the backup to a shared directory, or download to your workstation using an SFTP client.
4. Restore this backup to the new primary. From the Operations Console navigate to Maintenance > Restore From Backup).  Note that this will overwrite any superadmin or Operations Console admin accounts you created in Step 1. Note that you have options to restore all data or jut log data. More information on these options is available in this article on how to Restore  a Backup.
5. Verify connections to your external identity sources, users, token assignments, etc.
6. Run whatever jobs you were doing that were failing on the original primary (such as Clean Up Unresolvable Users).
7. If successful, we will now bless this new primary and use it to replace the current production primary.  Using the same image from Step 1, create one new virtual machine to replace each of your current replicas.  Give them temporary hostnames and IP addresses.  (Note:  if you are using hardware appliances, you will need to do a factory restore on each machine, then run Quick Setup and attach each as a new replica to the new primary).
8. As a precaution, make backups/snapshots of the current primary and all replicas.  Announce a Change Control window of approximately 2 hours.  This will impact administrators, but end users should be able to authenticate without issue.
9.  Shut down the current production primary.  Your replicas will continue to authenticate during the cutover.
10. Change the hostname and IP of your new primary to match that of the old primary. This must be done via the Operations Console interface by navigating to Administration > Network > Appliance Network Settings).
11. Verify connectivity to the new primary.  In the Security Console, navigate to Reporting > Real Time Activity Monitor > Authentication Activity and ensure users are authenticating to the new primary.
12. Shut down all replicas.  The primary will continue to authenticate users.
13. Ensure there are no references to the old replicas on the new primary.
    1. In the Security Console, navigate to Access > Authentication Agents > Manage Existing.  If any of the replicas are listed as RADIUS servers, delete them.
    2. In the Security Console, navigate to RADIUS > RADIUS Servers.  Again, ensure that only the local primary is listed.  There should be no references to the old primary or any replicas.
14. Build and attach replica(s) to the new primary.
    1. From the Operations Console of the new primary, navigate to Deployment Configuration > Instances > Generate Replica Package.
    2. Stage your new replica(s) by following the instructions in Step 1, but this time choose  Set Up a Replica Instance. Provide the replica package from step 14a when prompted. 
    3. Once attached, go to the Operations Console of the replica and navigate to Administration > Network > Appliance Network Settings. Change the hostname and IP address to match one of the original replicas .
    4. Repeat steps B through C for each replica.
15.  In the Security Console of the primary, navigate to Reporting > Real Time Activity Monitor > Authentication Activity. Click Start Monitor to watch authentications as they happen and ensure users are authenticating to the new replicas.
16. Apply the same patches to the replica(s) that you installed on the new primary.
17. Verify your Web Tiers.
    1. Log onto the Operations Console of the new Primary, and navigate to Deployment Configuration > Web Tier Deployments > Manage Existing.
    2. Verify if the web tiers are online.  Follow any instructions if they are not (i.e. "Offline, Reinstall Required")
18. Once all replicas are authenticating, the old primary and replicas can be decommissioned.