Decoder: Configure Decoder Rules

Document created by RSA Information Design and Development on Sep 13, 2017Last modified by RSA Information Design and Development on Oct 11, 2017
Version 6Show Document
  • View in full screen mode
 

This topic provides procedures for creating and managing rules for Decoder or Log Decoder traffic capture in the Services Config view > Rules tabs . Services Config View - Rules Tabs provides details about the Rules tab options.

The following figure illustrates commonly used settings on a Decoder. For a quick basic setup with only the required steps, see Decoder and Log Decoder Quick Setup.

Capture rules can add alerts or contextual information to sessions or logs. They can also define which data is filtered out by a Decoder or Log Decoder. Rules are created for specific metadata patterns, which result in predefined actions when matches are found. For example, to keep all traffic that fits certain criteria, but discard all other traffic, you can create a rule to perform the necessary actions. When applied, rules affect both packet capture file importing, as well as live network capture.

Rule and Query Guidelines provides guidelines that all queries and rule conditions in NetWitness Suite Core Services must follow.

By default, no rules are defined when you first install NetWitness Suite. Until rules are specified, the packets are not filtered. You can deploy the latest rules from Live. You can define three types of rules: Network Rules, Application Rules, and Correlation Rules.

  • Network rules are applied at the packet level and are made up of rule sets from Layer 2, Layer 3, and Layer 4. Multiple rules can be applied to the Decoder. Rules can be applied to multiple layers (for example, when a network rule filters out specific ports for a specific IP address). Network rules are only available on packet Decoders.
  • Application rules are applied at the session level. If the first rule listed is not a match, the Decoder then attempts to match the next rule listed, until a match is found.
  • Correlation rules are applied over a configurable sliding time window. When a match is found, the service creates a new super session that identifies other sessions that match the rule, then creates a session list for analysis.

The two most common uses of rules are:

  • To alert, and thereby create a custom alert meta value, when certain conditions are found.
  • To filter out certain types of traffic that do not add value to the analysis of the data.

Groups of capture rules form rule sets, which you can import and export. This feature enables use of multiple rule sets for various scenarios. You can import the exported rule set, in the form of an .nwr file, to other NetWitness Suite services, simplifying the deployment and configuration of multiple services.

Rule Processing

These are the principles governing capture rule processing:

  • Multiple rules can be applied to the Decoder.
  • Capture rules are executed one after the other, in sequence.
  • Rule processing stops when all rules are processed or after a rule configured to stop rule processing is matched.
  • A default rule can be used to either include or exclude all traffic not otherwise selected by a rule. A default rule, if used, must always be placed at the bottom of the rule list. Otherwise, rule processing stops as soon as the default rule is evaluated since, by definition, all traffic is selected by the default rule.
  • When rule processing stops, the session is saved using the configured session options and debug options.

Rule Configuration

Rule and Query Guidelines

All queries and rule conditions in RSA NetWitness Core services must follow these guidelines:

All string literals and time stamps must be quoted. Do not quote numbers, MAC, or IP addresses.

  • extension = 'torrent'
  • time='2015-jan-01 00:00:00'
  • service=80
  • ip.src = 192.168.0.1

Note: The space on the right and the left of an operator is optional. For example, you can type a rule as service=80 or service = 80 .

Rule Examples

The following table shows examples of rule conditions. You can use rule conditions for log retention collections in an Archiver and for application, network, and correlation rules on a Decoder, Log Decoder, or Concentrator. Rule conditions are also used in all WHERE clauses in all Core database queries.

For detailed information on rule syntax in NetWitness Suite, see "WHERE Clauses" in the "Queries" section of the Core Database Tuning Guide .

                             
Rule Name Condition
ComplianceDevices device.group='PCI Devices' || device.group='HIPPA Devices'
HighValueWindows device.group='Windows Compliance'
MediumValueWindows device.type='winevent_nic' && msg.id='security_4624_security'
LowValueWinLogs device.type='winevent_nic' && msg.id='security_4648_security'
LowValueProxyLogs device.class='proxy' && msg.id='antivirus_license_expired'
GeneralWindows device.type='winevent_nic'

Invalid Rules

NetWitness Suite uses a rule parser that strictly defines valid syntax for rules and queries. When a Core service encounters invalid syntax, it writes a warning in the NetWitness Suite logs indicating the error.

Note: NetWitness Suite 11.0 does not support parsing of legacy syntax rules (as NetWitness Suite 10.6 did). After you update to NetWitness Suite 11.0, rules with invalid syntax are highlighted in the user interface, and no rules will be applied until the invalid rules are corrected. The Rule Editor provides additional tooltips. After you fix the rules, the highlights disappear. See Fix Rules with Invalid Syntax.

The /decoder/config/rules/rule.errors and /concentrator/config/rules/rule.errors stats, contain the count of rules with errors. If rule.errors is nonzero, NetWitness Suite generates a Health and Wellness alert to indicate that you need to fix the rules.

General Syntax Guidelines

  • All text values must quote literal values. Example: username = 'user1'
  • Quotes can use single or double quotes; but they must match. (You cannot start with a single quote and finish with a double quote.)
  • If the literal value has a quote, you can escape it (using a backslash) or use a different starting quote character. Both of the following examples are valid: username = "User's" , username = 'User\'s'

The following are valid syntax rules:

  • To use a backslash in a literal string, escape it using an extra backslash: \
  • All time values should use quotes for dates in this form:
    time = 'YYYY-MM-DD HH:MM:SS'
  • All time values that are the number of seconds since EPOCH (Jan 1, 1970), should not be quoted.
    Example: time = 1448034064
  • Everything else is unquoted: IP addresses, MAC addresses, numerics, and so on. Example: service = 80 && ip.src = 192.168.1.1/16

Capture Rule Syntax

Capture rules compare fields to values or to other fields. This is an example of a simple expression with a meta key on the left side of the operator and a value on the right side.

ip.dst=192.168.1.1

The syntax allows a meta key on the right side of the operator in Decoders and Log Decoders for application and network rules. Meta key comparison does not apply in the where clause in queries. This is an example of a simple expression with a meta key on the left side of the operator and a meta key on the right side.

ip.src=ip.dst

Rules that include a meta key comparison support renamed meta keys; if a rule queries a meta key that has been renamed, the rule is parsed for the renamed meta key. For example, if the meta key ip_dst is used in a rule, it is transparently mapped to the renamed meta key: ip.dst. Existing rules that include original keys will trigger alerts that include data for the renamed meta key. .

This is an example of a rule that finds packets having the same ip.src address and ip.dst address on a Decoder, and generates an alert on the Concentrator.

alert=alert.id name=testRule8 rule="ip.src=ip.dst" order=38

This rule would generate an error because eth.src and ip.src are incompatible formats.

rule="eth.src=ip.src" name="testRule99" alert=alert.id

Values can be expressed as discrete values, a range of values, an upper or lower bound, or a combination of these three. You can create a greater than or less than comparison, and test equality or inequality against a range of values or an upper/lower bound.

key 0-5 (a range of values)

key = 0-u is the same as key >= 0 (upper bound, greater than or equal to)

The following table summarizes the operators on meta keys.

                                                                                                         
Left Operand FormatOperatorRight Operand FormatDescription
any= compatible with left operandEquality operator. You can use values or meta keys on the right side of the equality operator.

any

!= compatible with left operandInequality operator. You can use values or meta keys on the right side of the inequality operator.
any < compatible with left operandLess than operator. You can use values or meta keys on the right side of this operator.
any <= compatible with left operandLess than or equal to operator. You can use values or meta keys on the right side of this operator.
any > compatible with left operandGreater than operator. You can use values or meta keys on the right side of this operator.
any >= compatible with left operandGreater than or equal to operator. You can use values or meta keys on the right side of this operator.
textcontains textFind values that contain the right operand. You can use meta keys or values on the right side of this operator.

text

begins

text

Find values that begin with the right operand. You can use meta keys or values on the right side of this operator.
textends textFind values that end with the right operand. You can use meta keys or values on the right side of this operator.

text

length

integer

Find strings of a certain length. You can use meta keys or values on the right side of this operator.

any

count

integer

Find values with a specific number of occurrences within the session. You can use meta keys or values on the right side of this operator.

any

ucount and unique

integer

Finds a number of uniquely occurring values. You can use meta keys or values on the right side of this operator. For example, if the results include instances of a meta key with 5 unique values and 3 of the same value, the ucount is 6.

N/Aexists anyFinds any values for the meta key. You can use meta keys or values on the right side of this operator.
N/A!exists anyFinds any sessions in which the meta key does not occur. You can use meta keys or values on the right side of this operator.

text

regex

text

Finds values matching a regular expression. You can use values on the right side of this operator.

The following table summarizes other syntax elements used in rules.

                                               
Syntax elementDescription

*

Default rule. By using an asterisk (*) as the sole character in a rule, that rule will select all traffic.

u Upper bound of a range a range of times, IP addresses, or numeric formats. For example, to select all TCP ports above 40000, the syntax would be:
tcp.port = 40000-u
l Lower bound of a range of times, IP addresses, or numeric values. For example, to select all TCP ports below 40000, the syntax would be:
tcp.port = l-40000
- (dash)Denotes a range. This is only applicable to time values, IP or MAC addresses, or numeric values. Separate the lower and upper bounds of the range with a dash (-) character. For example, to select TCP ports between 25 and 443, the syntax would be:
tcp.port = 25-443
, (comma)Denotes a list of ranges or values or meta keys. Single values may be used as well as any combination of ranges and upper or lower bounds. Single meta keys may be used in a list. Meta keys and literal values cannot both appear on the right-hand side of an operator. For example, the following is valid syntax:
tcp.port = 1-10,25,110,143-225,40000-u
( ) Grouping operator. An expression can be enclosed in parentheses to create a new logical expression. For example, the following would select traffic on port 80 to/from 192.168.1.1 OR traffic on port 443 to/from 10.10.10.1:
(ip.addr=192.168.1.1 && tcp.port=80) || (ip.addr=10.10.10.1 && tcp.port=443)

~

Logical NOT operator, a negation of an expression.

&& Logical AND operator, a conjunction of two expressions.
|| Logical OR operator, a disjunction of two expressions.

Configure Capture Rules

The Decoder and Log Decoder rules are editable in the Services Config view. While each type of rule (network, application, and correlation) has its own tab; the functions are similar for all types of rules. You can:

  • Add, edit, and delete rules
  • Enable and disable rules
  • Change the execution sequence of rules
  • Import rules from a file
  • Export rules to a file
  • Push rules to another service
  • Revert or apply rule changes
  • Restore one of the last ten rule configurations from a snapshot

To configure rules in the Rules tabs

  1. Go to ADMIN > Services.
  2. In the Services view, select a Decoder service and Actions menu cropped > View > Config.
  3. In the Services Config view, select one of the Rules tabs: Network Rules, App Rules, or Correlation Rules.
    The rules list for the selected rule type is displayed.
    This is an example of the App Rules tab rules list.

Each type of rule has a list with slightly different columns and different parameters. Several basic guidelines apply to all rule management activities:

  • The rules are executed in the sequence they are displayed in the list. To change the execution sequence of rules, drag and drop rules to the appropriate location in the list or use the context menu options to arrange the rules in the list.
  • To select a single row, click the row.
  • To select a group of adjacent rows, click the first, then shift-click the row at the end of the group.
  • To select multiple non-adjacent rows, click the first, then control-click the others.
  • When editing rules in the Rules tab, you must apply the configuration changes in order to activate.
  • Until changes are applied, you can discard edits to the list and revert to the unedited rules.
  • Once rules are applied, you can recover the last ten rules configurations using the History option in the Actions menu.

To add a rule in any Rules tab, do one of the following:

  • Click The add icon.
  • Right-click a rule, and select Insert Above or Insert Below from the context menu.
    The Rule Editor dialog for that type of rule is displayed.

To remove a rule:

  1. From any Rules tab, select the rules to remove from the rules list.
  2. Click The delete icon.
    The selected rules are removed from the list, but still exist on the service.

To edit a rule

  1. From any Rules tab, select the rule to edit.
  2. Click The edit icon or double-click the rule row.
    The Rule Editor dialog for that type of rule is displayed.

To disable a rule:

  1. From any Rules tab, select the rules to disable.
  2. Click The Disable button.
    The status changes to disabled in the rules list, but the rule is still enabled on the service.

To enable a rule:

  1. From any Rules tab, select the rules to enable.
  2. Click The enable button.
    The status changes to enabled in the rules list, but the rule is still disabled on the service.

Import Rules from a File and Export Rules

You can import network, application, and correlation rules to a Decoder from a file that contains rules of the same type. After the rules are imported, you can edit and manage them as you would any other rules.

When you attempt to import a group of rules, NetWitness Suite Administration checks the type of rules imported. If you are successful, a message displays the number of rules imported. If the rule type differs from the active tab type, the rules are not imported. You must re-import the rules under the correct tab or select another file to import.

To import rules to a service:

  1. From any Rules tab, select The Actions drop-down menu> The import button.
    The Import dialog is displayed.
    This is an example of the Import dialog.
  2. Click The add icon.
    A view of the directory structure is displayed.
  3. Choose one or more NetWitness rules (.nwr) files to import, and click Open.
    The file is added to the list in the Import dialog.
    This is an example of the Import dialog.
  4. Click Import.
    The rules are imported into the user interface. Imported rules have a red corner in each edited column.
  5. Edit or reorder the rules if needed.
  6. To save the rules to the service, click Apply.
    The rules for the service are updated with the changes.

To export a rule to a file:

  1. To export a subset of the rules, select the rules to be exported.
  2. Do one of the following:
    • In the toolbar, select The Actions drop-down menu > Export > Selection. (Export > All exports all rules in the rules listeven if you have a subset selected for export.)
    • Right-click the selected rules and select Export Selection.

A prompt for the filename is displayed.

  1. Enter the filename and click Export.
    The .nwr file is downloaded.

Push Rules to Other Services

You can apply (push) rules or selected rules to other services (Decoders or Log Decoders) or service groups. When you push all rules to other services, all rules on the target services are removed and replaced with all of the rules on the source service. 

To push selected rules from this Decoder to other Decoders:

  1. From any Rules tab, select the rules that you want to push to another Decoder.
  2. Do one of the following:
    • Select The actions drop-down menu > Push > Selection.
    • Right-click the selected rules and select Push Selected Rules.
      The Push Selected Rules dialog is displayed.
      This is an example of the Push Selected Rules dialog.
  3. Select a Push Option:
    • Select Replace All to delete all rules on the target services and replace them with the selected rules. This is the default selection.
    • Select Merge to merge the selected rules with the existing rules on the target services.
  4. On the Services tab, select the target services to receive the pushed rules, or select the groups of services from the Groups tab.
  5. Click Push.
    The rules are pushed to the selected services and become effective immediately.

To push all rules from this Decoder to other Decoders:

  1. From any Rules tab, select The Actions drop-down menu > Push > All.
    (Push > All pushes all rules in the rules list even if you have a subset selected to push.) The Push Selected Rules dialog is displayed.
    This is an example of the Push Selected Rules dialog with Replace All selected.
  2. On the Services tab, select the target services to receive the pushed rules, or select the groups of services from the Groups tab.
  3. Click Push.
    All rules from the target services are deleted and replaced with all of the rules from source service. The rules become effective immediately.

Change Execution Order of Rules

Capture rules are applied in the order they are displayed in the rules list. To reorder rules, use either of these methods:

  • Drag and drop the rules in the appropriate location in the rules list.
  • Right-click a rule to display the context menu, and use the Cut and Paste options.

Restore a Rule Snapshot from History

NetWitness Suite keeps the last ten snapshots of rules applied to a service.

To restore a rules snapshot from history:

  1. Select The Actions drop-down menu > History.
    A submenu of snapshots is displayed.
    the History drop-down list
  2. Select the snapshot time from the submenu.
    The rules from the snapshot are loaded into the rules list, replacing the current set. But the current set is still in use on the service.
  3. To apply the rules to the service, click Apply.
    The rules are applied to the service.
You are here
Table of Contents > Configure Decoder Rules

Attachments

    Outcomes