This topic provides instructions to define a rule to fetch data or events from a NetWitness data source. You can define rules to fetch data or events from a NetWitness data source. The same procedure is used to define a rule to fetch data or events from an Archiver data source.
The Archiver data source can be added in the Services Config View of the Reporting Engine. For more information, see(Optional) Add Archiver as Data Source to Reporting Engine topic in Host and Services Configuration Guide.
Make sure that you:
- Understand which rule type needs to be used in the rule. For more information on rule types, see Rule Types.
- Understand the NWDB rule syntax. For more information, see NWDB Rule Syntax.
- Understand the Rule view components. For more information, see Rule View.
- Understand the Build Rule view components. For more information, see Build Rule View.
- Understand how custom meta keys are created using custom feeds. For more information, see Create Custome Meta Keys using Custom Feed topic in Host and Services Configuration Guid.
Perform the following steps to define a rule to fetch data or events from a NetWitness Data Source:
In the Security Analytics menu, click Administration > Reports.
The Manage tab is displayed.
The Build Rule view tab is displayed.
- In the Rule Type field, NetWitness DB is selected by default.
- In the Name field, enter a name that is used to Identify or label the rule in alerts and reports.
- The Summarize field determines the type of summarization or aggregation for the rule. Based on the type of rule to be defined, you must select one of the following:
- To define a Non-Aggregate rule without any grouping, select: None
To define an Aggregate rule with special aggregation like the collection (sessions/events/packets) related aggregates, select one of the following:
- Event Count
- Packet Count
- Session Size
To define an Aggregate rule with meta values and custom aggregates like sum(), count(), and so on, select: Custom
Choosing 'Custom' in the Summarize field enables you to define aggregate function of your choice in the Select clause. For example, select ip.src, countdistinct(ip.dst), distinct(ip.dst). The supported aggregate functions are:
- sum (<meta>)
For more detailed information about Aggregate and Non-aggregate rule, see NWDB Rule Syntax.
In the Select field, enter a meta or select a meta from the list of available meta types provided in the Meta Library. For more information, see the topic Meta Panel in Build Rule View. The meta name to fetch raw log is raw. raw can only be used in the Select field. It cannot be used in the Where and Then fields. Multiple aggregate functions are supported for Custom aggregate rule in the Select field.
Note: In earlier versions of Security Analytics, only one aggregate function was supported for Custom aggregate rule in the Select clause. From now, multiple aggregate functions are supported in the Select clause. For example, Select: ip.src, username, service, distinct(country.src), sum(payload).
- In the Where field, enter a meta or select a meta from the list of available meta types and use the operators to construct the Where clause for the base query criteria.
The Group By field is a read-only field which gets populated with meta that are defined in the Select clause. For a Non-Aggregate function, this field is not visible. A maximum of six meta are supported in the Group By field.
Note: In earlier versions of Security Analytics, only one meta was supported for Custom aggregate rule in the Group By clause. From now, a maximum of six meta are supported in the Group By clause.
In the Then field, enter the rule actions that manipulate the original result set of a rule in order to make the output in a report more concrete or add additional functionality other than querying data and displaying it, for example, creating a feed from the results. For a complete list of available rule actions, see NWDB Rule Syntax.
Note: When a rule is executed for an Archiver data source, it is recommended not to use query intensive rule actions such as lookup_and_add() and show_whats_new().
In the Order By field, perform the following:
In the Column Name column, enter the name of the columns by which you want to sort the results. By default, the value is empty. The value gets populated based on the value selected in the Summarize field.
- For Summarize 'None', if no Order By is selected, then by default it is ordered by session or collection time.
- For other Summarize values, the default sorting is based on the first 'group by' meta selected when no 'order by' is defined. For Event Count, Packet Count, and Session size, the accepted values are Total and Value.
In the Sort by column, select one of the following ways to sort the results:
- Ascending Order
- Descending Order
In the Session Threshold field, enter the optimization setting to stop scanning the matching sessions for each possible unique value for the selected meta. The threshold is an integer between 0 (default) and 2147483647.
Note: This is applicable to only NWDB Aggregate rules. If the default value is specified, all the matching sessions will be scanned and the accurate value will be returned. A higher session threshold allows accurate counts for a value. However, this causes longer rule execution time. For example, consider you set the Session Threshold as 1000 for ip.src. If there are 5000 matching sessions then for a particular ip.src value which is present in more than 1000 sessions, NWDB stops the scan after 1000 sessions and returns the extrapolated aggregate value. This optimizes the query execution time. If the value is present in less than 1000 sessions, then the actual value is returned.
- In the Limit field, enter the limit to be put on the query while fetching data from the database. If a result set is sorted by event count, packet count, or session size, the limit represents the top (or bottom) N values to be returned. If the result set is not sorted, the first N values are returned.
Note: Unlike parsed meta, raw logs are fetched from decoders. When both raw log and parsed meta are queried in a single rule, due to different retention periods, parsed meta might be available and raw logs missing in the same session. So the result will have parsed meta values and empty raw value for those sessions. For example, for the rule "Select ip.src, ip.dst, service, username, raw", the parsed meta might be populated and the raw meta remains empty for a few sessions.
You can test the correctness of the rule created by clicking Test Rule. For instructions, see Test a Rule.