000034876 - Working with Value Max when threshold is reached in RSA NetWitness Logs & Packets

Document created by RSA Customer Support Employee on Mar 21, 2017
Version 1Show Document
  • View in full screen mode

Article Content

Article Number000034876
Applies ToRSA Product Set: Security Analytics
RSA Product/Service Type: SA Security Analytics Server
RSA Version/Condition: 10.5.x, 10.6.x
Platform: CentOS
O/S Version: 6
IssueSummary of Problem: Unable to drill on the msg field by using contains or regex although I see the data exists in the session on event reconstruction. This is also true for all other meta keys that are hitting the valueMax set on the index.xml file on the concentrator.
ResolutionThe value max limitation can be frustrating to customers who want to index all possible unique meta. Unfortunately that is not possible in the general case. Meta keys exist that can have arbitrary random data from anywhere on the Internet, and all unique values cannot be indexed.
However, it is possible to work around some of the limitations of value max by using key level indices instead of value indices. Key level indices are not influenced by value max.
It is possible to use the Navigation view on a meta key indexed at the key level. The database uses value level indices in the where clause where possible, but meta database scanning is used to resolve unique values for the values call. This approach works well when the where clause provides an effective filter to limit search scope to a small number of sessions, perhaps less than
10,000 sessions.
In cases where the value max is reached, the users can perform a database scan on their queries to ensure no relevant values were dropped.
This feature is accessible in the Investigator thick client via the right-click menu on the Navigation view.  Although the meta database scan takes a long time, it reassures the customer that they are not missing anything in their reports.
Please see below link to the community:

NotesAdding as a KB article for easy reference by customer.