000011770 - Restoring RSA SecurID Appliance 3.0 database taken before SP2 was applied an SP2 appliance

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 4Show Document
  • View in full screen mode

Article Content

Article Number000011770
Applies ToRSA Product Set:  SecurID
RSA Product/Service Type:  RSA SecurID Appliance
RSA Version/Condition:  3.0 SP0 (3.0.0.0), 3.0 SP1 (3.0.1.0), 3.0 SP2 (3.0.2.0)

 


 
Issue
  • A non SP2 system (i. e., 3.0.0.0 or 3.0.1.0) has failed and needs to be factory reset. 
  • A valid backup exists which was generated recently using the GUI and was copied off the system and archived. 
  • The problem is that an RSA SecurID Appliance 2.0 running at SP2 will not allow  a pre-SP2 backup to be restored onto it through the GUI.
  • The backup needs to be loaded using a series of manual steps run at the command line.
ResolutionAn initial step is involved here.  The non-SP2 system cannot do a normal factory reset because the software on the system expires on 31 December 2009.  This means that a series of steps are needed to get the system updated to SP2 at which point data can be restored.  This includes:
  1. Modifying the time on the system (in the BIOS) to be earlier than 31 Dec 2009.
  2. Do a factory reset and create a primary based on the date selected above.
  3. Use the Operations Console GUI to apply the factory-reset SP2 patch.
  4. Do a factory reset and now build a true 3.0.2 primary.
  5. Restore the pre-SP2 data.

In this article we are only dealing with the last step, as the earlier steps are covered in standard documentation and other knowledgebase articles (see references for details).



Preparing for data restore


Before proceeding there are a few things which need to be done:


  1. Confirm that you can connect over SSH into the system at the command line.
  2. The archived file (with a name like am-3.0.0.0-44454C4C-4C00-1043-8030-C3C04F584C31-backup-20101201-155817-EST.tgz) has been copied onto the box into the /tmp folder)
  3. You know the system's master password.
  4. The master password on the pre-SP2 box matches the current SP2 box to which we are restoring data.
  5. You know the password for the emcsrv user
  6. You have already used the Operations Console to configure a RADIUS server.

With all of these done we can proceed with the restore.  The following annotated example shows the steps taken form the initial SSH connection into the box.


  1. Login as emcsrv and sudo to root.
login as:  emcsrv
emcsrv@cs-appliance3-03.na.rsa.net's password: <enter the operating system password>
Last login: Tue Jan 10 13:39:43 2017 from 10.168.152.116
-bash-3.00$ sudo su -
Password: <enter the operating system password>

  1. Check that the archive file has been copied to /tmp.  Obviously the name will be different and there will also be other files here which we can ignore.
  2. Untar the database backup.
[root@cs-appliance3-03 ~]# cd /tmp
[root@cs-appliance3-03 tmp]# ls
am-3.0.0.0-44454C4C-4C00-1043-8030-C3C04F584C31-backup-20101201-155817-EST.tgz
[root@cs-appliance3-03 tmp]# tar xvf am-3.0.0.0-44454C4C-4C00-1043-8030-C3C04F584C31-backup-20101201-155817-EST.tgz
metadata_v1
backup.tgz
sha1

  1.  Now  unpack one of the files we have just untarred:
[root@cs-appliance3-03 tmp]# mkdir restore
[root@cs-appliance3-03 tmp]# cd restore
[root@cs-appliance3-03 restore]# tar xvf ../backup.tgz
boot/
boot/grub
...
<many other files displayed here>
...
boot/grub/grub.conf
var/lib/raa/raadb.backups

  1. After the files are unpacked, exit from root and become the rsaadmin.
[root@cs-appliance3-03 restore]# exit
-bash-3.00$ sudo su - rsaadmin
Password: <enter operating system password>

  1. Navigate to /usr/local/RSASecurity/RSAAuthenticationManager/server and stop all of the Authentication Manager services, then start only the database server and listener.
-bash-3.00$ cd /usr/local/RSASecurity/RSAAuthenticationManager/server
-bash-3.00$ ./rsaam stop all
RSA Authentication Manager:                                [  OK  ]
RSA Authentication Manager Proxy Server:                   [  OK  ]
RSA Authentication Manager Administration Server:          [  OK  ]
RSA Authentication Manager Node Manager:                   [  OK  ]
RSA Authentication Manager Database Server:                [  OK  ]
RSA Authentication Manager Database Listener:              [  OK  ]
RSA Authentication Manager Operations Console:             [  OK  ]
RSA Authentication Manager Radius:                         [  OK  ]
RSA RADIUS Operations Console:                             [  OK  ]
-bash-3.00$ ./rsaam start db
RSA Authentication Manager Database Listener:              [  OK  ]
RSA Authentication Manager Database Server:                [  OK  ]


Removing the primary replication queue


Before restoring the data, we must remove the primary's replication queues from the database.  This step does not delete the primary database, just some information about the replication process.  Do not run the remove-primary command in a deployment of Authentication Manager that has more than one replica.  If the deployment has more than one replica, please contact RSA Support.

  1. Navigate to ../utils.
  2. Run the command ./rsautil setup-replication -a remove-primary.
  3. When asked " Are you sure you want to remove this primary," type y and press Enter.
-bash-3.00$ cd ../utils
-bash-3.00$ ./rsautil setup-replication -a remove-primary
Enter password: <enter master password>
Setup Replication ims-2.0.2-build20091007172001
Copyright (C) 2008 RSA Security Inc. All rights reserved.
 %% Running at: cs-appliance3-03:[km0iv9to] %%
 =======================================
 %       Removing a Primary Site       %
 =======================================
 Type     Instance name           Hostname                DBname   
 -------- ----------------------- ----------------------- ---------
 Primary  cs-appliance3-03.na.rsa.net  cs-appliance3-03.na.rsa.net  km0iv9to 
 Are you sure you want to remove this primary? (Y/N): y
%% Starting configuration
 -- Status: Removing primary site
 -- Status: Dropping schema capture rule at [km0iv9to]
 -- Status: Removing site status schedule job at [km0iv9to]
 -- Status: Removing [IMS] common scripts at [km0iv9to]
 -- Status: Removing [AM] common scripts at [km0iv9to]
 -- Status: Removing [AM] primary scripts at [km0iv9to]
 -- Status: Dropping [IMS] supplemental logging at [km0iv9to]
 -- Status: Dropping [AM] supplemental logging at [km0iv9to]
 -- Status: Removing queues at [km0iv9to]
 Done...

 
Importing data back into Authentication Manager


To import the data back into the server,
  1. Remain in /utils and run the command ./rsautil manage-backups -a import -D -f /tmp/restore/var/backups/staging/BACKUP.dump, replacing the file path and names with the correct ones for your backup.
  2. When prompted, enter the master password.
  3. When prompted about importing the file and overwriting the existing data in the database, type y then press Enter.
-bash-3.00$ ./rsautil manage-backups -a import -D -f /tmp/restore/var/backups/staging/BACKUP.dump
Enter master password: <enter the master password>
Are you sure you want to import the file and overwrite the existing data in the database?  (Y/N): y
Operation started : SUN JAN 16 09:50:51 EST 2011
Importing the user credentials
Importing the database
Flashback is turned on
Rename URL-based config values
SQL*Plus: Release 10.2.0.4.0 - Production on Sun Jan 16 10:02:39 2011
Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production
With the Partitioning, Data Mining and Real Application Testing options
*************************************************************************
10:02:40 : RSA_REP.IMS_PRINCIPAL_DATA is changed
10:02:40 : RSA_REP.PK_IMS_PRINCIPAL_DATA is changed
10:02:40 : RSA_REP.IDX_IMS_PRINC_DATA_IS_KEY is changed
10:02:40 : RSA_REP.IDX_IMS_PRINC_DATA_UID_IS is changed
10:02:40 : RSA_REP.FBI_IMS_PRINC_DATA_UID is changed
10:02:40 : RSA_BATCHREP.IMS_PRINCIPAL_LOGIN_DATE is changed
10:02:40 : RSA_BATCHREP.PK_IMS_PRINCIPAL_LOGIN_DATE is changed
10:02:40 : RSA_BATCHREP.IDX_IMS_PRINCIPAL_LOGIN_DATE is changed
10:02:40 : RSA_REP.AM_TOKEN is changed
10:02:40 : RSA_REP.IDX_AM_TOKEN_PK is changed
10:02:40 : RSA_REP.IDX_AM_TOKEN_PRINCID_FK is changed
10:02:40 : RSA_REP.IDX_AM_TOKEN_SERIAL_NUMBER_UK is changed
10:02:40 : RSA_REP.IDX_AM_TOKEN_TYPE_TOKEN_FK is changed
10:02:40 : RSA_REP.IDX_SEC_DOM_TOKEN_FK is changed
10:02:40 : RSA_REP.IDX_SW_TKN_DEV_TYP_TKN_FK is changed
10:02:40 : RSA_BATCHREP.AM_TOKEN_OOB is changed
10:02:40 : RSA_BATCHREP.IDX_AM_TOKEN_OOB_PK is changed
10:02:40 : RSA_BATCHREP.IDX_AM_TOKEN_OOB_UTC is changed
*
*  The script is executed successfully
.* .*************************************************************************
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production
With the Partitioning, Data Mining and Real Application Testing options
Reset the IMS console meta data
SQL*Plus: Release 10.2.0.4.0 - Production on Sun Jan 16 10:02:56 2011
Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production
With the Partitioning, Data Mining and Real Application Testing options.
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production
With the Partitioning, Data Mining and Real Application Testing options
All of data has been imported, and it is OK to start the system.
Operation completed : SUN JAN 16 10:02:56 EST 2011


Setting the primary replication queue


The next step is to reconfigure the replication queue.
  1. Run the command ./rsautil setup-replication -a set-primary.
  2. When prompted, enter the master password.
-bash-3.00$ ./rsautil setup-replication -a set-primary
Enter password:  <enter the master password>
Setup Replication ims-2.0.2-build20091007172001
 Copyright (C) 2008 RSA Security Inc. All rights reserved.
 %% Running at: cs-appliance3-03:[km0iv9to] %%
 =======================================
 %       Setting up Primary Site       %
 =======================================
 [Primary]                       
 Port      : 2334                 
 DB name   : km0iv9to             
 DB host   : cs-appliance3-03.na.rsa.net
 Instance  : cs-appliance3-03.na.rsa.net
 Site name : cs-appliance3-03.na.rsa.net
 Is this correct (Y/N):y
 %% Starting configuration
 -- Status: Configuring primary -- Status: Setting up queues at [km0iv9to]
 -- Status: Adding [AM] supplemental logging at [km0iv9to]
 -- Status: Adding [IMS] supplemental logging at [km0iv9to]
 -- Status: Executing [AM] primary scripts at [km0iv9to]
 -- Status: Executing [AM] common scripts at [km0iv9to]
 -- Status: Executing [IMS] primary scripts at [km0iv9to]
 -- Status: Executing [IMS] common scripts at [km0iv9to]
 -- Status: Setting up site status schedule job at [km0iv9to]
 -- Status: Adding schema capture rule at [km0iv9to]
 -- Status: Changing capture retention time [km0iv9to]
 -- Registering primary information
 Done...


Restart Authentication Manager services


 At this point we are almost done, so we now restart all of the Authenticaiton Manager services.
  1. Navigate to ../server.
  2. Run ./rsaam start all.
-bash-3.00$ cd ../server
-bash-3.00$ ./rsaam start all
RSA Authentication Manager Database Listener:              [RUNNING]
RSA Authentication Manager Database Server:                [RUNNING]
RSA Authentication Manager Node Manager:                   [  OK  ]
RSA Authentication Manager Administration Server:          [  OK  ]
RSA Authentication Manager Proxy Server:                   [  OK  ]
RSA Authentication Manager:                                [  OK  ]
RSA Authentication Manager Operations Console:/usr/bin/nohup: appending output to `nohup.out'
                                                           [  OK  ]
RSA Authentication Manager Radius:                         [  OK  ]
RSA RADIUS Operations Console:/usr/bin/nohup: appending output to `nohup.out'
-bash-3.00$ exit
logout


Restoring the RADIUS server


From this point on we are restoring RADIUS.  Some systems have not been using RADIUS and so restoring non-existent changes might be a waste of time.  The rule of thumb here is:
  • If you are 100% sure that you have NOT been using RADIUS then you can stop here.
  • If you are 99.999% sure that you do not use RADIUS, make the assumption that you are wrong and carry on following the instructions!
  1. The RADIUS code runs as the root user so we need to switch into the root account.
  2. Navigate to the RADIUS directory.
-bash-3.00$ sudo su - root
Password: <enter operating system password>
[root@cs-appliance3-03~]# cd /usr/local/RSASecurity/RSAAuthenticationManager/radius

  1. This next command is actually all on one line but has wrapped in the code sample below:
[root@cs-appliance3-03 radius]# cp -r --reply=yes /tmp/restore/usr/local/RSASecurity/RSAAuthenticationManager/radius/*   .
`/tmp/restore/usr/local/RSASecurity/RSAAuthenticationManager/radius/3comsw.dct' -> `./3comsw.dct'
`/tmp/restore/usr/local/RSASecurity/RSAAuthenticationManager/radius/aat.dct' -> `./aat.dct'

  1. A long list of files is displayed.
`/tmp/restore/usr/local/RSASecurity/RSAAuthenticationManager/radius/website/sbr/deployer/linux-x86/deployer' -> `./website/sbr/deployer/linux-x86/deployer'
`/tmp/restore/usr/local/RSASecurity/RSAAuthenticationManager/radius/xylan.dct' -> `./xylan.dct'

  1. Having copied the files into place we now need to finish off with some odds and ends to glue the restore RADIUS data back together with its SecurID data.  Navigate to  ../config.
[root@cs-appliance3-03 RSAAuthenticationManager]# cd ../config

  1. This next command gives some people problems.  The command is  ./configUtil.sh configure util-config updateAdmin -R master.password=<master_password> -R superadmin.username=<super_admin_username> -R superadmin.password=<super_admin_username>.  For example, ./configUtil.sh configure util-config updateAdmin -R master.password=M4st3rP4ssw0rd! -R superadmin.username=RSASuperAdmin -R superadmin.password=sup3r4dm!nP4ssw0rd!.  Your values will be different!
  2. Take a look at the SP2 release notes at tracking number 128401.  The problem is that we need to enter the three values at the command line but if they contain any punctuation values which are interpreted by Linux system then the command fails.  Due to the nature of the command, using things like quotes, double-quotes or escaped slashes does not help.  If you have this type of trouble, temporarily change the master password and the password of the superadmin user to something which can be entered at the command line and then change them back when you have finished
[root@cs-appliance3-03 config]# ./configUtil.sh configure util-config updateAdmin -R master.password=M4st3rP4ssw0rd! -R superadmin.username=RSASuperAdmin -R superadmin.password=sup3r4dm!nP4ssw0rd
Action configure
Product util-config
Module updateAdmin
JVM_HOME=/usr/local/RSASecurity/RSAAuthenticationManager/appserver/jdk
Allowing to run configEngine as non-root user
Configuration complete
Exiting...

  1. With the password values pre-set we can finish the job.  Notice that in this example that an error displays.  This one is just cosmetic and is simply that the RADIUS start operation took a bit longer than the script expected, but was able to carry on as it then re-checked and found the task had completed
[root@cs-appliance3-03 config]# ./configUtil.sh configure radius finalize-radius-restore
Action configure
Product radius
Module finalize-radius-restore
JVM_HOME=/usr/local/RSASecurity/RSAAuthenticationManager/appserver/jdk
readSecrets PropDir: /usr/local/RSASecurity/RSAAuthenticationManager/utils/etc
Action: start
Using service/script 'Steel-Belted Radius'/'radiuswrapper.bin'
Starting RADIUS Service...
Checking XUI Connection...
[ERROR] Unexpected issue with XUI call : RADIUS server does not responds within the timeout
[SOLUTION] RADIUS server is not configured properly ; Please re-run the RADIUS server configuration again
com.rsa.authmgr.radius.exception.RadiusSystemException: RADIUS server does not responds within the timeout
Retrying (timer 5612 ms)
Checking XUI Connection...
Done.
RADIUS Server Cert Generation: SUCCESS
RADIUS Server Cert Install: SUCCESS
RemoteCommand: Properties dir: /usr/local/RSASecurity/RSAAuthenticationManager/utils/etc
RemoteCommand: Connecting to Local AM as superadmin
RemoteCommand: Successfully logged in to AM
AM Registration: SUCCESS
Storing Node Secret into '/usr/local/RSASecurity/RSAAuthenticationManager/radius/securid'
Storing sdconf.rec into '/usr/local/RSASecurity/RSAAuthenticationManager/radius/sdconf.rec'
RADIUS Agent Configuration: SUCCESS
RADIUS Registration: SUCCESS
Action: stop
Using service/script 'Steel-Belted Radius'/'radiuswrapper.bin'
Stopping RADIUS Service...
Done.
Action: start
Using service/script 'Steel-Belted Radius'/'radiuswrapper.bin'
Starting RADIUS Service...
Error return value from start operation: -1
Checking XUI Connection...
[ERROR] Unexpected issue with XUI call : RADIUS server does not responds within the timeout
[SOLUTION] RADIUS server is not configured properly ; Please re-run the RADIUS server configuration again
com.rsa.authmgr.radius.exception.RadiusSystemException: RADIUS server does not responds within the timeout
Retrying (timer 5004 ms)
Checking XUI Connection...
Done.
Configuration complete
Exiting...
[root@cs-appliance3-03 config]# exit

  1. Our system is now fully restored.
Notes

See also:


 
Legacy Article IDa53604

Attachments

    Outcomes