Article Content
Article Number | 000038075 |
Applies To | RSA Product Set: SecurID RSA Product/Service Type: Authentication Manager RSA Version/Condition: 8.2, 8.3 |
Issue | When a customer has a large amount of authentication activity logs built up in the database, and then performs an abnormal reboot without services shutdown properly, the archive log feature requires a full scan of the logs, which can last for more than a hour to do with the current index. Primary instance Services failed to start due to Runtime server hanging |
Cause | The following tables are large and causing the system scan during startup to take more than one hour. To confirm this, follow the steps below:
INFO | jvm 1 | main | 2019/09/10 19:11:49 | <Sep 10, 2019 7:11:49 PM UTC> <Notice> <Stdout> <BEA-000000> <ADMIN log table is scanned, scantime = 113775>
SELECT COUNT(*) FROM rsa_logrep.ims_log_system; SELECT COUNT(*) FROM rsa_logrep.ims_log_audit_rt; count --------- 3183978 (1 row) count --------- 3595064 (1 row) --------------------------------------------------
SELECT nspname || '.' || relname AS "relation",pg_size_pretty(pg_total_relation_size(C.oid)) AS "total_size" FROM pg_class C LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace) WHERE nspname NOT IN ('pg_catalog', 'information_schema') AND C.relkind <> 'i' AND nspname !~ '^pg_toast' ORDER BY pg_total_relation_size(C.oid) DESC LIMIT 20; relation | total_size ------------------------------------------------ rsa_logrep.ims_log_system | 5954 MB rsa_logrep.ims_log_audit_rt | 3891 MB rsa_rep.am_file_data | 1892 MB rsa_logrep.ims_log_audit_adm | 261 MB rsa_rep.am_token | 221 MB rsa_rep.am_token_attr_values | 140 MB rsa_rep.ims_principal_d_ext_att_value | 111 MB rsa_rep.ims_principal_data | 103 MB rsa_norep.ims_report_rt_activity | 75 MB rsa_rep.am_principal | 54 MB rsa_norep.ims_report_adm_activity | 47 MB rsa_rep.am_token_oob | 45 MB rsa_rep.ims_principal_login_date | 37 MB rsa_norep.am_report_all_users | 31 MB rsa_norep.am_report_user_token_wc | 28 MB rsa_rep.am_sms_authenticators | 23 MB rsa_norep.ims_report_systemlog | 21 MB rsa_norep.am_sms_report | 14 MB rsa_norep.am_report_sms_custom_attr | 5872 kB rsa_rep_util.am_r2p_primarystatus | 4296 kB (20 rows) |
Resolution | This issue has been reported in defects AM-30882 and AM-35390. It was resolved in RSA Authentication Manager 8.4 patch 5. If you are experiencing the above described symptoms, please do the following:
/opt/rsa/am/server/rsaserv stop all exclude db
DROP INDEX rsa_logrep.idx_ims_log_system_serial; CREATE INDEX idx_ims_log_system_serial ON rsa_logrep.ims_log_system USING btree (SERIAL, SERVER_NODE_IP); DROP INDEX idx_ims_log_audit_rt_serial; CREATE INDEX idx_ims_log_audit_rt_serial ON rsa_logrep.ims_log_audit_rt USING btree (SERIAL, SERVER_NODE_IP);
/opt/rsa/am/server/rsaserv restart all
|
Workaround | To prevent the system from scanning the logs, shut the instance down, boot into single user mode, disable services from starting up so system logs can be reviewed:
ims.logging.system.fullscan_needed=true ims.logging.audit.admin.fullscan_needed=true ims.logging.audit.runtime.fullscan_needed=true ims.logging.system.fullscan_needed.default=false
/opt/rsa/am/server/rsaserv start all
|
Notes | Resolved in 8.4.0.5 |