How to manually sync a single replica in an RSA Authentication Manager 8.1 deployment with multiple replicas

[#1]

Article Number

000042848

Applies To

RSA Product Set: SecurID
RSA Product/Service Type: Authentication Manager
RSA Version/Condition: 8.1

Issue

This article provides steps to manually sync a single replica in an Authentication Manager 8.1 deployment with multiple replicas.

Cause

Replication can fail between a primary and a replica for the following reasons:

1. Replication is broken between the primary and a replica instance.
2. Possible network issue between the primary and a replica instance.

Resolution

From the Operations Console, navigate to Maintenance > Backup and Restore > Back Up Now to take a backup before continuing.

1. Using the rsaadmin account, logon to the Authentication Manager primary, with an either an SSH session or at the local console.
2. Navigate to /opt/rsa/am/utils.
3. Retrieve the password for the rsa_dba user using the command ./rsautil manage-secrets -a get com.rsa.db.dba.password -u <Operations Console administrator name> -p <Operations Console administrator password>.  

Commands are documented below:

login as: rsaadmin
Using keyboard-interactive authentication.
Password: <enter OS user password>
Last login: Tue Dec 15 14:07:01 2015 from jumphost.vcloud.local
RSA Authentication Manager Installation Directory: /opt/rsa/am
rsaadmin@am81p:~> cd /opt/rsa/am/utils
rsaadmin@am81p:/opt/rsa/am/utils> ./rsautil manage-secrets -a get com.rsa.db.dba.password -u ocadmin -p support!
com.rsa.db.dba.password: rSKD5bGguLGNL9uGvFWnJoxIcHJah2
rsaadmin@am81p:/opt/rsa/am/utils> /opt/rsa/am/pgsql/bin/psql -h localhost -p 7050 -d db -U rsa_dba
Password for user rsa_dba: <enter the com.rsa.db.dba.password captured above>
psql.bin (9.2.4)    
SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
Type "help" for help.

4. Enter the following SQL statement to list the contents of the rsa_rep.IMS_INSTANCE_NODE table:

db=# SELECT * FROM rsa_rep.IMS_INSTANCE_NODE;

id                |           instance_id            |   name    |          host          |      ip       | product_patch |     activation_time     |     last_updated_on
----------------------------------+----------------------------------+-----------+------------------------+---------------+---------------+-------------------------+------------
-------------
 6d779c4bea8b1f0a1ab26476086cfb4f | 075b5175ea8b1f0a08024e4d8ee9036e | replica81 | replica81.rsalocal.com | 10.31.139.234 |               | 2015-09-21 08:19:30.972 | 2015-09-21 08:26:52.6
 151e9003598b1f0a1a90a885b26a6634 | 091ca479598b1f0a080258079e9bd683 | primary81 | primary81.rsalocal.com | 10.31.139.89  |               | 2015-09-18 12:04:40.3   | 2015-09-21 08:28:03.756
(2 rows)

5. Enter the following SQL statement to list the contents of the rsa_rep.IMS_INSTANCE table:

db=# SELECT * FROM RSA_REP.IMS_INSTANCE;

                id                | cpu_count |          description           | is_primary | deployed_state
----------------------------------+-----------+--------------------------------+------------+----------------
 091ca479598b1f0a080258079e9bd683 |         1 | Initial primary instance.      | t          |
 075b5175ea8b1f0a08024e4d8ee9036e |         1 | inserted by attachment process | f          | active
(2 rows)

Note that the instance_id captured in step 4 is the id in step 5.  Also note carefully which row has the listing for is_primary set to false and note the ID number for that instance.

6. Using the ID number for the replica captured in step 5, update IMS_INSTANCE to change the deployed state of the replica.

db=# UPDATE RSA_REP.IMS_INSTANCE SET deployed_state = 'out_of_sync' WHERE is_primary='FALSE' AND id = '075b5175ea8b1f0a08024e4d8ee9036e';
UPDATE 1

7. Now running a SELECT statement on the IMS_INSTANCE table, shows the deployed state of the replica as out_of_sync.

db=# SELECT * FROM RSA_REP.IMS_INSTANCE;
                id                | cpu_count |          description           | is_primary | deployed_state
----------------------------------+-----------+--------------------------------+------------+----------------
 091ca479598b1f0a080258079e9bd683 |         1 | Initial primary instance.      | t          |
 075b5175ea8b1f0a08024e4d8ee9036e |         1 | inserted by attachment process | f          | out_of_sync
(2 rows)

8. Exit SQL with the command \q.
9. Login to the Operations Console of the primary and select Deployment Configuration > Instances > Status Report. The sync option is now available.
10. Click Sync to perform a database sync between the primary and replica instance

Note: Add an entry in /etc/hosts for the corresponding Authentication Manager servers on the primary and all replica servers.