Reporting Engine service is not running due to reportstatusmanager.h2.db corrupt
2 years ago
Originally Published: 2016-04-01
Article Number
000064543
Applies To
RSA Product Set: Security Analytics
RSA Product/Service Type: SA Reporting Engine
RSA Version/Condition: 10.4.1.X
Issue
Reporting Engine service fails to start due to Status DB corruption. Below are sample errors indicate status DB corrupt.

tail /home/rsasoc/rsa/soc/reporting-engine/logs/reporting-engine.log
2016-03-24 00:43:31,152 [main] INFO  com.rsa.soc.re.contentstore.impl.ReportingContentStoreImpl - Permissions are already set for content store
2016-03-24 00:43:31,153 [main] INFO  com.rsa.soc.re.contentstore.impl.ReportingContentStoreImpl - Permissions are already set for content store for Administrator role
2016-03-24 00:43:32,218 [main] INFO  com.rsa.soc.re.upgrade.Upgrade_RE10_3 - 10.3 Upgrade is already complete.
2016-03-24 00:43:32,222 [main] INFO  com.rsa.soc.re.resultstore.ResultStoreCacheThreadPool - Maximum Thread pool size should greater than 1. Setting maximum thread pool size to 2
2016-03-24 00:43:32,877 [main] INFO  com.rsa.soc.re.status.manager.report.ReportStatusManagerImpl - Updated the previously running reports execution status.
2016-03-24 00:43:32,878 [main] INFO  com.rsa.soc.re.status.manager.report.ReportStatusManagerImpl - Deleting the sub report execution status from the main table and adding them to corresponding table during reboot
2016-03-24 00:43:47,877 [main] WARN  org.hibernate.util.JDBCExceptionReporter - SQL Error: 90030, SQLState: 90030
2016-03-24 00:43:47,877 [main] ERROR org.hibernate.util.JDBCExceptionReporter - File corrupted while reading record: "page[2661016] data leaf table:21 COMPONENT entries:12 parent:2724827 keys:[119053128, 119053129, 119053130, 119053131, 119053132, 119053133, 119053134, 119053135, 119053136, 119053137, 119053138, 119053139] offsets:[1906, 1764, 1673, 1582, 1440, 1298, 1156, 1064, 972, 882, 740, 598]". Possible solution: use the recovery tool; SQL statement:
select alertstatu0_.executionID as executio1_7_1_, alertstatu0_.endTime as endTime7_1_, alertstatu0_.executionStatus as executio3_7_1_, alertstatu0_.startTime as startTime7_1_, alertstatu0_.statusDescription as statusDe5_7_1_, alertstatu0_.alertID as alertID7_1_, alertstatu0_.alertsCount as alertsCo7_7_1_, alertstatu0_.endSessionID as endSessi8_7_1_, alertstatu0_.startSessionID as startSes9_7_1_, componentl1_.AlertStatus_executionID as AlertSta1_7_3_, component2_.id as componen2_3_, component2_.id as id0_0_, component2_.componentStatus as componen3_0_0_, component2_.componentType as componen4_0_0_, component2_.endTime as endTime0_0_, component2_.startTime as startTime0_0_, component2_.statusDescription as statusDe7_0_0_, component2_.COMPONENT_TYPE as COMPONENT1_0_0_ from AlertStatus alertstatu0_ left outer join AlertStatus_Component componentl1_ on alertstatu0_.executionID=componentl1_.AlertStatus_executionID left outer join Component component2_ on componentl1_.componentList_id=component2_.id where alertstatu0_.executionID=? [90030-174]
2016-03-24 00:43:48,006 [RE-CommonThreadPool-0] INFO  com.rsa.soc.re.status.manager.report.ReportStatusManagerImpl - Initialization of the LastExecutedReport Map completed
2016-03-24 00:43:48,523 [StatusDB ShutdownHook] WARN  com.rsa.soc.re.status.manager.ShutdownStatusManager - Closing all connections to status db
Cause
There are certain circumstances Reporting Engine service fails to start due to status DB corruption.
Resolution
Please use below steps to bring Reporting Engine online.

1. Login to SSH of SA server. 
2. Stop RE service using stop rsasoc_re command 
3. Run below commands in sequence to move corrupted database to different location.
mkdir olddb 
cd /home/rsasoc/rsa/soc/reporting-engine/statusdb 
mv reportstatusmanager.h2.db /root/olddb 

4. start RE service using start rsasoc_re command 
5. Login to SA server GUI as admin and verify Reporting Engine service is running.