Log Parser Customize: Log Parser Rules Tab

Document created by RSA Information Design and Development on Jul 25, 2018Last modified by RSA Information Design and Development on Sep 20, 2018
Version 4Show Document
  • View in full screen mode
 

Note: The information in this topic applies to RSA NetWitness® Platform Version 11.1 and later.

Introduction

This tab contains details about the rules for the default log parser, as well as any other custom rules and log parsers that have been defined.

The default log parser parses logs that do not match any installed log parsers. The information contained in such a log is processed against the default log parser's rules, and metadata is then extracted by those rules and is available for Enrichment, Investigation, Reporting, and Alerting. This provides immediate visibility into logs from custom or unsupported sources.

You can also add or extend a log parser. For example, you may need to parse certain fields differently than in the manner provided by the log parser for a particular event source. You can add rules that change the way meta information is extracted from the logs for the event source.

Finally, you can view and test sample log messages and rules for your log parsers, including the default log parser.

The Log Parser Rules tab displays information about log parsers that use dynamic log parser rules. This includes the following:

  • The default log parser that parses logs that are not associated with a particular log parser
  • Native XML-defined device parsers that have been extended with dynamic log parser rules, and
  • User-created custom device parsers used to parse unsupported custom event sources

This tab contains the following information:

  • You can view the rules for a particular event source type, including the default parser.
  • You can view the Names, Literals, patterns, and meta for each configured log parser.
  • You can add log parsers
  • You can add, edit, and delete custom rules for log parsers

To access this tab, go to CONFIGURE > Log Parser Rules.

Workflow

This workflow shows processes available from the Log Parser Rules view.

What do you want to do?

                            
RoleI want to...Documentation

Administrator

*View log parser rules.

Default Log Parser and Log Parser Rules

Administrator

*Add, edit or delete a log parser rule (version 11.2 and later)

Add or Delete a Log Parser Rule

Administrator

*Add or remove a log parser (version 11.2 and later)

Add or Delete a Log Parser

*You can perform this task here.

Related Topics

Default Log Parser and Log Parser Rules

Quick Look

Note: The list of log parsers is based on the first Log Decoder that is installed or registered by the Orchestration Server. If you have more than one Log Decoder, this tab only lists log parsers that have been configured on the first one.

The Log Parser Rules tab organizes and displays information about the configured log parsers in your system. This tab consists of three panels: Log Parsers list, Details for the selected log parser, and Rules for the selected log parser.

Log Parsers Panel

         

The Log Parsers Panel lists the configured log parsers.

  • Until you add rules to existing XML parsers on your reference Log Decoder, (or add a new, custom log parser) only the default parser is listed here.
  • Select a specific log parser to view its details in the Details and Rules panels.
  • Click Add Parser to open the Add Dynamic Log Parser dialog box.
  • Click Delete to delete a log parser.

    IMPORTANT: Once you deploy a log parser, you can no longer delete it through this interface. The Delete button is not available for deployed parsers. To manually delete a log parser, see Delete a Log Parser Manually.

The Add Dynamic Log Parser dialog box allows you to add a custom log parser.

When you are adding a log parser, the following parameters are available.

                               
FieldDetails
select log parser

Select NEW, or choose an existing log parser.

By choosing an existing log parser, you can add rules to that parser, essentially extending its parsing capabilities.

Note: If you select an existing log parser, the remaining fields are auto-filled based on the values for selected log parser.

device type

Enter a string to define the device type. The name must be between 3 and 30 alphanumeric characters (including underscores), and must not match the name of any existing log parsers.

device display name

Enter the display name for the log parser.

Note: The display name must be 64 characters or fewer, and must not match the name of any other device display name.

device class

Select a device class.

clone dynamic parser rules from

Leave blank to start with no rules, or select one of the existing log parsers to clone its rules.

Details Panel

The details panel shows the three pieces for the selected rule:

  • Tokens: one or more tokens to match in the message. For example, the Any Port rule looks for the following strings to match against: port , port:, port=, and others.
  • Values: the value that follows the token. This is a string that is captured as meta. For example, assume a log contains the following string:

    port 12345

    The Any Port rule has a token that matches "port ". When it encounters that string, it assigns the token value, "12345" to a meta key.

  • Meta: the meta keys to which the value is mapped. For example, the Any Port rule maps the port value to the port meta key.

Essentially, a rule says, "when you are parsing a message, if you match one of my tokens, assign the value that follows the token to the meta key that I want it stored as."

The bottom section of the Details panel contains sample log messages, and how they would be parsed for the selected log parser.

                             
1

Displays the name of the selected log parser, and the buttons for deploying, saving, and discarding changes. This value changes when you select a different parser.

2

Displays the name of the selected rule. This value changes when you select a different rule for this parser.

3

Displays the list of tokens defined for the selected rule.

4

Displays the type and pattern of the value matching for the selected parser. The values here are determined by the type of the selected value. You can also use the Regex option to define a custom regular expression.

5

Displays the NetWitness meta to which the selected rule maps any matched tokens. The values here are determined by the selected Rule.

6

Displays a sample log message, and highlights strings that match tokens in the selected log parser. You can edit this field, and add in your own logs to preview how the selected parser will parse your logs.

Note: The sample section refreshes whenever a rule is changed or updated, as well as when you paste in samples from your logs.

For example, consider the following scenario:

  • The default parser is selected.
  • The Any Domain rule is selected.
  • The Tokens matching list displays all of the tokens that are matched when found in a log message: Domain, Domain Name, domain, ADMIN_DOMAIN, and so on.
  • The Meta list displays the NetWitness meta to which the value for the token is mapped: domain.

So, let's say the sample log message area has the following text:

       

Below are sample log messages:

May 5 2010 15:55:49 switch : %ACE-4-400000: IDS:1000 IP Option Bad Option List by user admin@test.com from 10.100.229.59 to 224.0.0.22 on port 12345.

Apr 29 2010 03:15:34 pvg1-ace02: %ACE-3-251008: Health probe failed for server 218.83.175.75:81, connectivity error: server open timeout (no SYN ACK) domain google.com with mac 06-00-00-00-00-00.

In this case, the Sample Log Message area looks like this:

Note that some strings are highlighted, and that there are two "pairs" of highlight colors:

  • Dark blue and light blue highlighting is applied to the strings that match the currently selected rule.

    • Dark Blue highlighted strings match a token in the selected rule. In this case, domain is the token that is matched for the Any Domain rule.
    • Light Blue highlighted strings are the values that correspond to the tokens in dark blue. For example, google.com is highlighted in light blue, because it corresponds to the domain token.
  • Orange and yellow highlighting is applied to the strings that match rules for the current parser that are not currently selected.

    • Orange highlighted strings match a token in a rule that is not currently selected.
    • Yellow highlighted strings are the values that correspond to the tokens in orange. For example, the user token matches the Username rule (which is not currently selected).

In this example, the domain meta would be assigned a value of google.com for this log message, if it was parsed using the default log parser.

Rules Panel

The Rules panel displays the list of rules used by the selected log parser. When you select a rule, you change the values that are displayed in both the Tokens and Values areas of the panel.

         

Note the highlighted rules:

  • The currently selected rule is highlighted in blue.
  • Other rules that match tokens in the sample log message area are highlighted in orange.

Other notes for the Rules panel:

  • RSA rules (the rules provided out-of-the-box for each log parser) are identified by [RSA] following the rule name.

    You can copy these rules when adding a new log parser, and then change them as needed.

  • The Delete button is only available for custom rules; for RSA rules, it is greyed out.
  • Use the Add Rule button to add a custom rule.

Disable log Parser Rules

You can disable log parser rules, so that none of them are processed by the Log Decoder. You might have your log parsers working as you like, and do not want any extra processing that you do not need.

You disable them from the reference Log Decoder.

  1. Go toADMIN > Services.
  2. In the Administration Services view, select the Decoder and The actions menu > View > Config.

    The Services Config view is displayed with the General tab open.

  3. Under Parsers Configuration, look at the Config Value for PARSERULESCAN.

    If it is Enabled, log parser rules are processed. If it is Disabled, they are not processed.

  4. If the rules are Enabled, click Enabled and select Disabled to disable the log parser rules.

    To save the changes, click Apply.

You are here
Table of Contents > Log Parser Rules Tab

Attachments

    Outcomes