Decoder: Rule and Query Guidelines

Document created by RSA Information Design and Development on Jul 21, 2016Last modified by RSA Information Design and Development on Oct 24, 2016
Version 5Show Document
  • View in full screen mode
  

All queries and rule conditions in RSA Security Analytics Core services must follow these guidelines: 

All string literals and time stamps must be quoted. Do not quote numbers, MAC or IP addresses.

For example:

  • extension = 'torrent'
  • time='2015-jan-01 00:00:00'
  • service=80
  • ip.src = 192.168.0.1

Note: The space on the right and the left of an operator is optional.
For example, you can use service=80 or service = 80.

Rule Examples

The following table shows examples of rule conditions. You can use rule conditions for log retention collections in an Archiver and for application, network, and correlation rules on a Decoder, Log Decoder, or Concentrator. Rule conditions are also used in all Where clauses in all Core database queries.

For detailed information on rule syntax in Security Analytics, see Where Clauses in the Queries topic in the RSA Security Analytics Core Database Tuning Guide.

                                   
Rule NameCondition
ComplianceDevicesdevice.group='PCI Devices' || device.group='HIPPA Devices'
HighValueWindowsdevice.group='Windows Compliance'
MediumValueWindowsdevice.type='winevent_nic' && msg.id='security_4624_security'
LowValueWinLogsdevice.type='winevent_nic' && msg.id='security_4648_security'
LowValueProxyLogsdevice.class='proxy' && msg.id='antivirus_license_expired'
GeneralWindowsdevice.type='winevent_nic'

Strict Mode Configuration for Security Analytics 10.6

Since version 10.2, Security Analytics has been using a modern parser for rules and queries that strictly defines valid syntax. When a Core service encounters deprecated syntax, it writes a warning in the Security Analytics logs about the deprecated syntax. Security Analytics now enforces strict parsing for new application, network, and correlation rules. The previous generation legacy parser, now deprecated, allows syntax that is ambiguous, which can cause unexpected results. While Security Analytics 10.6 continues to support the deprecated syntax, future versions will no longer support it.

After you update to Security Analytics 10.6, rules with deprecated syntax are highlighted in the user interface. The Rule Editor provides additional tooltips. After you fix the rules, the highlights disappear. See Fix Rules with Deprecated Syntax in the Decoder and Log Decoder Configuration Guide.

The /decoder/config/rules/rule.errors and /concentrator/config/rules/rule.errors stats, introduced in 10.6, contain the count of rules with errors. If rule.errors is non-zero, Security Analytics generates a Health and Wellness alert to indicate that you need to fix the rules.

In addition, there is a migration path for queries from external systems. After an update from an earlier version, the system operates in deprecated mode (controlled by /sdk/config/query.parse). In deprecated mode, the service continues to use the legacy parser for all queries that fail strict parsing. The errors will be logged and a message will be streamed back to the client informing them of the strict parsing failure. But the query still executes and returns results just like previous versions. You should monitor the logs and external clients for any reports, dashboards, rules, and so on, that are written in deprecated syntax and fix those problems as they arise.

After you resolve the issues, you can switch all core services (Decoders, Log Decoders, Concentrators, Brokers, and Archivers) to strict mode and monitor them for issues. Strict mode does not use the legacy parser and any parsing violations return errors. You should perform this task before any major upgrade after 10.6, as the legacy parser will be removed in future releases and there will not be an option to operate in deprecated mode.

All new installs operate in strict mode by default. If you plan to add a new appliance to an existing infrastructure running in deprecated mode, in the Explore view (Administration > Services > Select a service and in the Actions menu, select View > Explore), you can switch /sdk/config/query.parse to deprecated mode until the whole stack has moved to strict mode.

In Security Analytics 10.6, all rule validation will always operate in strict mode, in order to eliminate creating syntax problems.

Valid Syntax with the Modern Parser

The following are valid syntax rules using the modern parser:

  • All text types must quote literal values. Example: username = 'user1'
  • Quotes can use single or double quotes, but they must match (you cannot start with a single quote and finish with a double quote)
  • If the literal value has a quote, you can escape it or use a different starting quote character. Both of the following examples are valid (use backslash for the escape character):

    • username = "User’s"
    • username = 'User\'s'
  • To use a backslash in a literal string, escape it using an extra backslash: \\
  • All time types should use quotes for dates in this form: time = 'YYYY-MM-DD HH:MM:SS'
  • All time types that are the number of seconds since EPOCH (Jan 1, 1970), should not be quoted.
    Example: time = 1448034064
  • EVERYTHING else is unquoted: IP Address, Ethernet Addresses, Numerics, and so on.
    Example: service = 80 && ip.src = 192.168.1.1/16

Ambiguous Syntax Examples with the Legacy Parser

Here is an example of ambiguous syntax with the deprecated legacy parser:

select * where alias.host = server-xeon

In the query above, it seems logical that the query author wants to return all meta where alias.host is equal to server-xeon. Unfortunately, that does not happen with the legacy parser. Instead, it parses that query as select * where alias.host = 'server'-'xeon'. So it converts it to a range query (values BETWEEN ‘server’ and ‘xeon’, using the unquoted dash - operator) and since that range will probably return results that have nothing to do with server-xeon, users may assume that there is a problem in the query engine. In 10.6 strict mode, you will get this error:

expecting <quoted_string> here: "server-xeon" 

This immediately tells the user why the query failed to parse correctly.

Here is another example of ambiguous syntax with the deprecated legacy parser:

select * where username=lastname,firstname

In Security Analytics language, you can specify multiple values by separating them with commas (an implicit OR operator). But did the query author mean 'lastname,firstname' or did the author want to search for two values, 'lastname' and 'firstname'? Again, it is ambiguous, but accepted in the legacy parser (which turns it into 'lastname' OR 'firstname'). With the modern parser, you must be completely clear in your intent, either 'lastname,firstname' or 'lastname','firstname'

The following example fails to parse correctly:

select * where username = lastname, firstname

At first glance, you may not notice that there is a space between the comma and firstname. Because there are no quotes, this fails to parse correctly with the legacy parser, but it does not return a parse error! Instead, it actually executes (by discarding firstname completely because of the whitespace issue). What is worse, you would not know that the executed query is not what you submitted unless you determine through some other means that the result set is incomplete. With strict parsing enabled, this returns a parse error, which is what should always happen. At least in prior versions (and deprecated mode), the logs show a message immediately after the audit log for the query: Rule '<rule name>' in deprecated format. Please make sure all text values are surrounded with quotes.

You are here
Table of Contents > References > Services Config View - Rules Tab > Rule and Query Guidelines

Attachments

    Outcomes