000026984 - How to optimize lookup_and_add queries in RSA NetWitness Platform Reporting Engine

When using the RSA NetWitness Reporting Engine lookup_and_add queries, this can result in a vast number of subqueries being generated to complete the task.  Because of this, running these queries can be very performance-intensive and take a long time to complete.

The recommendations below may be used to modify the queries to improve their efficiency.

Modify the maximum pending queries on Services.

1. In the NetWitness UI, navigate to Admin > Services
2. Select the Archiver, Broker or Concentrator service that will be the subject of the lookup_and_add queries and click on View > Explore
3. Navigate to sdk > config in the directory tree in the left pane.
4. Change the configuration value of the max.pending.queries to be 1000.  (The default is 100)
5. The change takes effect immediately.
6. Repeat the above steps for all other services against which queries will be executed.

Change the lookup_and_add queries configuration on the Reporting Engine Service.

1. In the NetWitness UI, navigate to Admin > Services
2. Select the Reporting Engine service that will be the subject of a query and click on View > Config
3. In the General tab under System Configuration, change the Max # Concurrent LookupAndAdd queries setting to 10.  (The default is 2)

Change the query level for those NetWitness logins who are executing the lookup_and_add queries.

1. In the NetWitness UI, navigate to Admin > Services
2. Select the Archiver, Broker or Concentrator service that will be the subject of the lookup_and_add queries and click on View > Security
3. Click on the users that will be making the queries and change the Core Query Timeout setting to 60, for a 60-minute timeout on queries.
4. Repeat the above steps for all other Services against which the queries will be executed, or use the Replicate button to push the change to the other Services.

If you are unsure of any of the steps above or experience any issues, contact RSA Support and quote this article ID for further assistance.