Alerting: Configure an In-Memory Table Using an EPL Query

Document created by RSA Information Design and Development on Jan 30, 2020
Version 1Show Document
  • View in full screen mode
 

Note: It is preferable to use Context Hub List enrichment sources instead of In-Memory Table enrichment sources for rules. Recurring In-Memory Tables are no longer supported; use Content Hub Lists as enrichment sources. For more information, see Configure a Context Hub List as an Enrichment Source.

When you use an In-Memory Table configuration in expert mode, you can create an enrichment source or named window based on an Esper query. This allows you to have more control over the content and create more dynamic content. When you do this, an EPL query constructs the named window to capture interesting states from the event stream.

Workflow

The following shows the workflow for creating a query using a named window:

  1. The event is sent to the Esper Engine.
  2. An EPL query is generated.
  3. An alert is triggered.
  4. The query checks to see if there is a connection between the event and the Named Window.
  5. If there is a connection, the query that populates the Named Window is run and populated.
  6. The content from the Named Window is added to the alert content and sent or displayed (depending on your settings).

Esper Query workflow

Prerequisites

  • The meta used in the EPL statement must exist in the data.
  • You must create well-formed EPL statements.

Procedure

Note: It is preferable to use Context Hub List enrichment sources instead of In-Memory Table enrichment sources for rules.

  1. Go to Configure > ESA Rules.
    The Configure view is displayed with the Rules tab open.
  2. Click the Settings tab.
  3. In the options panel, select Enrichment Sources.
  4. In the Enrichment Sources section, click Add List icon  > In-Memory Table.
    In-Memory Table - Enrich Advanced Query
  5. Select Adhoc.
    By default, Enable is selected. When you add the in-memory table to a rule, alerts will be enriched with data from it.
  6. In the User-Defined Table Name field, type a descriptive name to describe the in-memory table.
  7. If you want to explain what the enrichment adds to an alert, enter information in the Description field.
    This description displays when you view the list of enrichments from the Enrichment Sources view, so it's a good idea to enter a thorough description as a best practice. Doing this allows other users to understand the content of the enrichment without opening it to examine its contents.
  8. Select Expert Mode to define an advanced in-memory table configuration by writing an EPL query.
    The Table Columns are replaced by a Query field.
  9. Select Persist to preserve the in-memory table on disk when the ESA service stops and to re-populate the table when the service restarts.
  10. Enter the EPL query in the Query field. The query should be well-formed, and it's a good idea to test it before entering it in the field.
  11. Click Save.

Example

For example, you created a rule that searches for five failed attempted logins followed by a successful login. When that rule is triggered, you may want the notification to contain information about the last user logged into the system when this successful login occurred. To add this enrichment to the notification, you might choose to create a stream-based in-memory lookup table that is populated from incoming events to maintain a mapping of IP addresses to the last user logged in from that address. To do this, you create an enrichment using a query as your source.

Step 1: Create Your Rule

First, you need to create your correlation rule. In this case, you create failure and success rule conditions, and group by the ip_src.

                               
Rule ConditionDescription
FailuresThis condition searches for five failed logins with a "followed by" connector, meaning that the condition (Failures) must be followed by the next condition (Success).
SuccessThis condition searches for one successful login. 
GroupBy: ip_src, device classThe GroupBy field ensures that all the previous conditions are grouped by the ip_src and device class. This is important to the construction of the rule because the rule attempts to find a case where a user has attempted to log into the same destination account multiple times, and finally logged in successfully. Grouping by device class ensures that the user logged in from the same machine attempted to log into an account multiple times. The rule may give unexpected results if you do not group the results.
Occurs within 5 minutesThe time window for the events to occur is five minutes. If the events occur outside of this time window, the rule does not trigger. 
Event Sequence: StrictThe event sequence is configured for a strict pattern match. This means that the pattern must match exactly as it is specifiedwith no intervening events. 

For the rule conditions, you create the following statements:

  • The "Failures" statement searches for failed login attempts:
    Failures Statement
  • The "Success" statement searches for one successful login:
    Success Statement
  • Combined, you have the following correlation rule:
    Login Failure Followed by Success

Step 2: Create the Enrichment

Now that you have created your rule, you need to create the enrichment to add to the notification output. Follow the steps above to create the enrichment, name it Last_Logon, and add the following query:

create window LastLogon.std:unique(ip_src) as (ip_src string, user_dst string);

insert into LastLogon select ip_src, user_dst from CoreEvent

where ec_activity='Logon' and ec_outcome='Success';

The enrichment should look like the following:

In-Memory Table dialog showing advanced query

 

Step 3: Add the Enrichment to the Rule

Now that you have created your basic rule and your enrichment, you'll need to add the enrichment to the rule and join (or connect) the enrichment to the meta in the rule.

Open the Login_Failure_Followed_by_Success rule for editing.

Enrichment added to rule

                                 
FieldEnterDescription
Output

In-Memory Table

The In Memory Table option creates a Named Window, which can be populated with the EPL query data.
Enrichment SourceLast_Logon (the enrichment you created above).This is the stream-based in-memory lookup table that is populated from incoming events to maintain a mapping of IP addresses to the last user logged in from that address.

ESA Event Stream Meta

ip_src

This is an event stream meta that you can join to the enrichment data you are populating. Essentially, ip_src is the join condition .

Enrichment Source Column Nameip_srcThis is the meta from the enrichment that you can join to the event stream data. It must be the same as join condition from the Event Stream Meta field.

Once you have added the enrichment, you can save the rule.

When the rule is triggered, the ESA runs the query in the enrichment and populates the Named Window with the data. If the data in the Named Window matches the join condition, the data is added to the output you can view in Email, Syslog or Script, depending on how you configured notifications.

You are here
Table of Contents > Configure an In-Memory Table Using an EPL Query

Attachments

    Outcomes