Investigate: Filter Results in the Event Analysis View

Document created by RSA Information Design and Development on Mar 27, 2018Last modified by RSA Information Design and Development on Apr 25, 2019
Version 6Show Document
  • View in full screen mode
 

You can filter events in the Event Analysis view using the query bar to select a service and time range and then querying the service being investigated. Filtering events helps to narrow the focus of an investigation to a smaller, relevant set of events.

Note: This section applies to version 11.1 and later.

Initial View

When you go to INVESTIGATE > Event Analysis to open the Event Analysis view, the service and time range selectors are displayed in the query bar.

  • By default, the first service is automatically selected (unless you previously selected a service and the selected service is in browser cache).
  • If you do not select a time range, the default time range (24 hours) is used.
  • The query builder field is an empty field to the right of the time range selector.

When you go to the Event Analysis view from the Events view or the Navigate view, the service, time range, and any filters that were selected in the Events view or Navigate view are displayed in the query bar. The service, time range, and individual filters can be modified.

If a profile is selected in the Events view when you right-click or double-click an event and go to the Event Analysis view, the filters from the profile (preQuery) are added to the query builder field as an editable filter. The following figures show a preQuery in the Events view, and the same query added as the first filter in the Event Analysis view.
a  preQuery in the Events view

the preQuery added as a filter in the Event Analysis view

Select a Time Range

The Time Range selector limits the events returned in the Event Analysis view to a specific time range. The time range is displayed in the format Start Time - End Time, showing the date, hours, and minutes in your current timezone based on timezone settings configured for your profile. You can choose a time range relative to the current collection time or create a custom time range (Version 11.3 and later). The time and date format is based on preferences set for the Event Analysis view in the User Preferences dialog (select User Profile icon > Profile).

  • By default, the date format is MM/DD/YYYY. You can change the format to DD/MM/YYYY, or YYYY/MM/DD in the User Preferences dialog.
  • Start time and end time are in the format of HH:MM. Although seconds are not displayed, the value for start time always defaults to HH:MM:00 seconds, and the value for end time always defaults to HH:MM:59 seconds. As an example, a time range of 6:45 pm - 7:45 pm is interpreted as 06:45:00 - 07:45:59 pm.
  • The default time range is the 24-hour clock; you can change it to 12-hour periods.
  • The time format can be either database time or wall clock time. When Database Time is selected, the start and end time for a query is based on the time that the event was captured (collection time). When Wall Clock Time is selected, the query is executed using the end time based on the current browser time; the start time is calculated based on that end time and the time range.

To edit the time range, do one of the following:

  1. Click the drop-down arrow inside the Time Range selector and select a time range from the list. Options are in minutes, hours, days, or all data.

     the Time Range drop-down menu

  1. (Version 11.3 and later) Edit the time range directly by clicking the year, month, day, hour, or minute displayed in the query bar. When a value is highlighted, type a new value for either the start or end time. If your time format preferences are set to 12-hour periods, click am or pm to toggle between the two options.

    time Selector Edit

    If the time range is invalid (for example, the start time is later than the end time), a red border appears around the Time Range selector. The the submit query icon button is disabled because the query is no longer possible, and a tool tip shows an error message explaining what you need to change. The following figure shows an invalid time range.

    an error in the Time Selector
    The selected time range is stored in your browser for the service being queried; you can set different time ranges for different services. A tool tip shows the calculated duration of the query. The following figure is an example of the tool tip.
    Time Selector Duration

Create Filters in the Query Bar

The query bar allows you to reduce the number of events to an interesting set by creating filters.

  • Simple filters are in the form <meta key><operator><meta value>.
  • Complex filters have one of these operators, ||,&&, (), REGEX, or LENGTH. (If you open an event from the Events view or Navigate view and the filter has one of these operators, the complex filter is added as a single filter in the Event Analysis query bar.)

In the query bar, each filter becomes an editable field. Filters line up from left to right, representing the sequence in which the filters were created. As more filters are added and exceed the length of a single line, they wrap to another line and the input area expands vertically so that all filters are visible without scrolling to the right.

In Guided Mode, you are guided with suggestions for auto-complete that show only valid meta keys and operators in the drop-down menu. You can type or select from the drop-down menu. In the drop-down menu, operators that take more time to execute are marked by a stopwatch icon. As you create simple filters, the syntax of each filter is validated and invalid filters are marked by a red outline. If you hover the mouse over the filter, a message that explains the error is displayed.

In Version 11.3 and later, complex filters are validated on the server side, which may take additional time. If you submit the query before the server has returned filter validation results, the the submit query icon is replaced by a spinner. When server validation returns, a query with no invalid filters begins execution. If the query contains an invalid filter, execution is terminated and the invalid filter is outlined in red. This is an example of an invalid query.

example of an invalid query after validation

In Free-Form Mode, you can type or paste a complex query. There is no auto-suggestion, and validation is performed on the server side when you submit the query. If an error is found, the query does not execute.

Note: The the submit query icon button has a different label in versions earlier than Version 11.3. It was previously named Query Events.

The the submit query icon button on the right side of the query bar is active as needed to submit a query. When you click the submit query icon, all of the filters are AND'd to generate results and the the submit query icon button becomes inactive. The the submit query icon button becomes active again in these conditions:

  • If you change the service in the query bar or change the column group in the Events panel, a network call for data for reconstructions or the Events panel continues to use the previous service, time range, and metadata filters until you submit the new query. The the submit query icon button becomes active as an indicator that the data in the view is stale.
  • If more than a minute has passed and the original query's time range would no longer generate the same result set, the the submit query icon button becomes active as an indicator that results may be stale. In Version 11.3 and later, a setting in the Event Analysis view preferences determines this behavior by enabling or disabling the Update Relative Time Window Automatically option (see Configure the Event Analysis View.)

Versions 11.2 and 11.3 have more capability than the original 11.1 query bar. The following table summarizes the differences.

                       
VersionFeatures
11.1
  • The query bar guides you as you create and edit each simple <meta key> <operator> <meta value> filter. The query bar supports only simple filters. If you open an event from the Events view or Navigate view and the filter has more than one operator, ||,&&, (), REGEX, or LENGTH, the filter is added, but editing is not supported in the Event Analysis view.
11.2
  • The query bar guides you as you create and edit each simple <meta key> <operator> <meta value> filter (called Guided Mode). The default mode is Guided Mode, which includes auto-suggest and validation options.
  • You can enter a complex query in Free-Form Mode. In Free-Form Mode validation is performed when you run the query.
11.3
  • When you go to the Event Analysis view from the Navigate view or Events view, with a preQuery in effect for a profile, the preQuery that was displayed in the breadcrumb is available in the Event Analysis view query bar as an editable filter.
  • You can create a free-form (complex) filter while in Guided mode. The auto-suggest drop-down lists are augmented with an Advanced Options section that allows for the creation of a free-form filter. Free-Form Mode is still useful for pasting an entire complex query.
  • Visual cues identify the filter that is the focus of some type of action among multiple selected filters.
  • You can cancel a query while it is executing.

  • Detailed status information about the query being executed is available in the query console.

Switch Between Guided Mode and Free-Form Mode

Two options switch between the modes. If you selected Free-Form Mode the last time you logged in, this choice is in effect the next time you log in.

  • When you switch from Guided Mode to Free-Form Mode, filters that you created in Guided Mode are transformed to a text query in the Free-Form field.
  • When you switch from Free-Form Mode to Guided Mode, the query you were typing is added to the query bar as a single uneditable filter.

    Note: Beginning with Version 11.3, a Free-Form filter can be edited but it does not include auto-suggest options.

The following figure is an example of the Event Analysis view with the Guided Mode query builder in use.

example of a query in the Guided Mode query builder

The following figure is an example of the Free-Form query builder in use.

example of the same query in the Free-Form query builder

Work with Multiple Filters

As you work in the query bar, you can see when a filter has focus for editing (a green outline) and which filters are selected (blue background). This is useful because you can have multiple filters selected for right-click actions, but only one can be edited at a time. The figure below shows the green outline marking a filter that has focus and the blue background indicating that two filters are selected.

example of one filter in focus and two filters selected

This figure illustrates the same set of filters with all filters selected (blue background) and one filter that has focus (blue background and green outline).

example of one filter in focus and all three are selected

A right-click action from the drop-down menu applies to all selected filters as shown in this figure.

the right-click options for selected filters

These are a few basic concepts that explain how to work in the query bar:

  • You can select multiple filters, but only one can have focus and the last selected filter is the one with active focus at any point in time.
  • To select a filter and give it focus, click the filter. To deselect the filter and remove focus, click the filter again, press Esc, or click anywhere else on the page.
  • To add a filter, click before or after an existing filter. To create a new filter before or after the filter in focus, press the right or left arrow key.
  • To open a filter for editing, double-click the filter or click it and press Enter. To exit without saving changes and leave the filter in focus, press Esc.
  • To delete a filter, click the filter and press Delete or click X on the filter.

Cancel Execution of a Query

After you click the submit query icon to submit a query, the button changes to the Cancel Query icon (the stop query option). The stop query option remains until all the events are loaded in the Events panel. To cancel the query, click the Cancel Query icon.

If the query is canceled before all results have been returned, the following message is displayed at the end of results in the Events list: "Because the query was canceled, only partial results are displayed."

View Status of a Query

After a query is submitted, you can click the information icon () in the query bar to open the query console. In the query console, you can see which service, time range, and metadata was queried as well as real-time information about the status of the query and the services being queried. The time range displayed in the Query console always shows each date as YYYY-MM-DD. Here is an example range from the Query Console: "2014-09-20 20:57:00"-"2018-11-02 18:57:59".)

The following figure is an example of the information in the Query Console when a query executes successfully and the slowest service is marked by an amber stopwatch.
the query console after a query has executed with no errors or warnings, and the slowest service is marked by an amber stopwatch

While a query is executing, a progress bar indicates the query's completion percentage at the bottom of the console. The status lets you know details about what is happening; for example, you can tell when the query is executing, queued, reading the index file for the queried service, retrieving events, and complete. All statuses and non-fatal messages are displayed as they come in, and the border color of the query bar changes to amber if a non-fatal error occurs.

Icons provide additional information about individual services.

  • An amber stopwatch marks the slowest service.
  • An amber triangle indicates a warning was received.
  • A red triangle shows that an error was received when trying to query the service.

Executing and Reading the Index File to Find Events. The first stage of a query is complete when the queried services have found results. The query console provides a nested hierarchical listing of all the services being queried with indicators showing which are online or offline, and the time in seconds that the service took to find results.

Retrieving Events and Loading in the Events Panel. While the found events are being retrieved and loading in the Events panel, the progress bar shows a visual indicator and text description of what is happening. In the figure below, results were found and are being retrieved.

an example of the query console while retrieving

Request Complete. If there are no errors or warnings when loading is complete, the query console is outlined in blue and the the submit query icon button is disabled as an indicator that the data in the view is fresh. The following figure is an example of the query console for a completed query with no errors or warnings.

example of the status for a completed query with no errors or warnings

Errors and Warnings. A fatal error such as a syntax error in the query, or the queried service being offline, stops execution of the query. A red triangle is displayed in the upper right corner of the query console, and the console is outlined in red to indicate that the query failed. If the queried service is offline, only the queried service with no hierarchy of services is listed in the query console and marked by a red triangle.

A non-fatal error does not prevent a query from executing. The query is executed and events are loaded, but a red triangle is displayed in the upper right corner of the query console, and the console is outlined in red as a warning. The following figure shows the appearance of the query console when the queried service proxies to another service that is offline.

example of the query console with one service offline

A warning does not prevent a query from executing. The query is executed and events are loaded, but an amber triangle is displayed in the upper right corner of the query console, and the console is outlined in amber. The following figure shows the appearance of the query console when there was a warning that results may be truncated on one of the services in the service hierarchy, but the request completed.

the query console showing a completed query with a warning that results may be truncated

Build a Query in Guided Mode

Guided Mode is the easiest way to create a query with features to help analysts enter valid queries. The following figure illustrates the initial Event Analysis view with Guided Mode in effect in the query bar.

the Event Analysis view when initially opened, with Guided Mode in effect

Note: Guided Mode in Version 11.2 supports only simple filters in the form <meta key><operator><meta value>. If the Events view or Navigate view has a filter with more than one operator, not, >, <, <=, >=, ||, &&, (), REGEX, or LENGTH, the filter is added, but editing is not supported in the Event Analysis view. The same is true for a filter created with Free-Form mode in effect and then viewed in Guided Mode. In Version 11.3, you can enter and edit a complex filter in Guided Mode, but Free-Form Mode is still better for pasting an entire complex query.

Keyboard Actions to Use in Guided Mode

In Guided Mode, the query builder allows entry, editing, and deletion of filters using the key strokes without having to use a pointer. Although you can use the pointer, you have the option to keep your fingers on the keyboard. This table identifies the available keyboard actions in Guided Mode when the cursor is located in the query bar; these do not apply to the service selector and time range.

                                                                   
ActionKeyboard Entry
Submit a query.With focus on the query bar and no pending filters, press Enter.
Select the filter to the immediate left if one exists.With no selection in the query bar, press the Left Arrow key.
Select the filter to the immediate right if one exists.With no selection in the query bar, press the Right Arrow key
Insert a new filter to the immediate left of the selected filter.With a filter selected, press the Left Arrow key.
Insert a new filter to the immediate right of the selected filter.With a filter selected, press the Right Arrow key.
Insert a new filter to the immediate left of the selected filter, and open for editing.With a filter selected, press the Shift + Left Arrow keys.
Insert a new filter to the immediate left of the selected filter, and open for editing.With a filter selected, press the Shift + Right Arrow keys.
Select all filters to the right of the current filter.With a filter selected, press the Shift + Down Arrow keys.

Select all filters to the left of the current filter.

With a filter selected, press the Shift + Up Arrow keys.

Edit a selected filter

With a single filter selected, press the Enter key.

Deselect all filters.With a filter selected, press the ESC key.

Delete all selected filters.

With filters selected, choose the right-click > Delete selected filters option, press Delete, or press Backspace.

Update query with only the selected filters.With filters selected, choose the right-click > query with selected filters option.

Open a new tab with the selected filters.

With filters selected, choose the right-click > query with selected filters in a new tab option.

Visual Feedback in Guided Mode

Guided Mode provides visual feedback during query construction. This table identifies and describes the possible feedback.

                                                     
FeedbackIconDescription

Blue Background on a Filter

a filter with blue background

Indicates that a filter is selected.

Green Circle Green circle inbetween filters The cursor has been placed in between two existing filters. Clicking inserts a new filter at this location.
Green Outline on a Filter a filter with a green border Marks the single filter that has focus.

Red Filter Outline

Red outline example

Value-type is not valid for the selected meta key. A tool tip that explains the error is displayed. For example, a string value for a meta key that expects an integer.

Loading Spinner

the loading spinner

The query is currently being processed. The Query Events button is disabled while this occurs.

Query Events button the Query Events button

The Query Events button has three possible states:
Query Events button Ready to submit a query using filters in the Query Builder.
Query Events spinner Waiting for server validation to complete before executing the query.
Query builder submit icon when is Cancel mode The query is executing, click to cancel execution.

Slow Service Icon slow service icon This service took the longest time to load results from the query.
Stopwatch the stopwatch icon The selected meta key/operator combination requires extra time to process. While the query is still executable, a more efficient meta key or operator is recommended.

Add a Simple Filter in Guided Mode

To create a simple filter in the Event Analysis view in Guided Mode:

  1. Go to the Event Analysis view and do one of the following:
    1. (Version 11.2 and later) Select Guided Mode below the query bar and click in the query builder field.
    2. (Version 11.1) Click in the query builder field, or before or after an existing filter. This is an example of the empty query bar in Guided Mode before you begin entering a filter.
      query bar before creating a filter
      If the insertion point is between two filters, a green circle marks the insertion point. If the insertion point is at the end of the query bar, the filter entry field opens with a blinking cursor at the entry point. A drop-down menu lists available meta keys passed from the service being investigated in alphabetical order. Meta keys that require more time to process are marked by a stopwatch icon.
      the meta key drop-down
  1. To select a meta key do one of the following:
    1. If there is only one option in the drop-down menu, press the Enter.
    2. If there are two or more options in the drop-down menu, click on the meta key or use the up/down arrow and press Enter.
    3. Start typing the meta key. As you type the meta key, the list is further updated. To select the meta key, press Enter.
    4. If you want to edit or delete the meta key, press Backspace or Delete.
      As you backspace and delete a character, the meta key drop-down menu is filtered to include meta keys that begin with those characters. To select a meta key, press Enter.
      The meta key is added to the query bar, a list of valid operators for the selected meta key is displayed. Operations that require more time to process are marked by a stopwatch icon. This figure shows the stopwatch icon.
      the operations droop-down list with stopwatches
  2. To select an operator, do one of the following:

    1. If there is only one option in the drop-down menu, press Enter to select it.
    2. If there are two or more options in the drop-down menu, click on the operator or use the up/down arrow and press Enter.
    3. Type the operator and press Enter.
  3. (Optional) Type a value and press Enter.

  4. To create the filter, press Enter. If you click anywhere outside the box before pressing Enter, the filter is not created.
    The new filter is inserted, and the blinking cursor is refocused after the last filter, the meta keys drop-down is displayed. If there is an error in the filter, it is outlined in red. You can hover over the filter to see a tool tip explaining the error. This figure shows a query being created with no errors.
    example of a filters created in Guided Mode with no errors
  5. If the filters have no errors, you are ready to execute the query in the query bar. Click the query events button.
    The results are returned and loaded in the Events panel. The first 50,000 events that match the query begin loading in the Events panel. As the events are loaded, a status bar at the top tracks progress and you can scroll to the bottom of the list to see the completion status.
  6. (Optional in Version 11.3 and later) If you want to see detailed status in the Query Console, click the information icon the Query Console icon, the blue letter i in a blue circle.
  7. (Optional in Version 11.3 and later) If you want to cancel the query before it finishes executing, click the cancel query button.
    The query stops executing and a notification that the query has been canceled is displayed.

Add a Free-Form Filter in Guided Mode (Version 11.3 and Later)

To filter the data displayed in the Event Analysis view using a free-form filter in Guided Mode:

  1. Go to the Event Analysis view, select Guided Mode below the query bar, and click in the query builder field.
    If the insertion point is between two filters, a green circle marks the insertion point. If the insertion point is at the end of the query bar, the filter entry field opens with a blinking cursor at the entry point. A drop-down menu lists available meta keys passed from the service being investigated in alphabetical order. Meta keys that require more time to process are marked by a stopwatch icon.
    the meta key drop-down menu with Advanced Options
  2. Do one of the following:
    1. Place the cursor in the Free-Form Filter field and begin typing the query.
    2. Begin typing the filter as usual beginning with a meta key.
      When no matching meta keys or operators are available in the drop-down menu, the Free-Form Filter option becomes available, and the text you typed is available in the Free-Form Filter field.
      the values drop-down menu with GET typed in
  3. Continue typing the entire expression and press Enter.
    (If you click anywhere outside the box before pressing Enter, the filter is not created.) This figure shows a complex expression created by continuing to type after the value GET.
    the value drop-down menu with a complex query ready to enter
    The new filter is inserted, and the blinking cursor is refocused after the last filter, the meta keys drop-down menu is displayed. If there is an error in the filter, it is outlined in red. You can hover over the filter to see a tool tip explaining the error.
  4. To execute the query, click the query events button. While the query is executing, the the query events button button changes to the cancel query button.
    a query in progress in the Query builder with the Cancel Query button available
  5. If you want to cancel the query before it finishes executing, click the cancel query button.
    If you do not cancel the query, you can click the Info icon to view the status of query execution. When the query is finished executing, the Events panel displays appropriate results for the query.

Edit a Filter in Guided Mode

With a query in the Guided Mode query bar, you can edit a filter. To edit a filter:

  1. Double-click the filter, or click the filter and press Enter.
  2. Edit the filter. When finished editing, press Enter to update the filter.
  3. If you want to execute the query again, click the query events button.
    The Events panel displays results for the updated filter.

Query Using Selected Filters in Guided Mode

With one or more filters in the query bar in Guided Mode, you can refocus the same query to include only selected filters. The results are displayed in the current browser tab or a new browser tab. To update the query using only selected filters:

  1. Begin with a Guided Mode query that includes one or more filters, for example a query has three filters: risk.info exists, direction ='lateral', and threat.category exists.
  2. To open a new tab with the selected filters, select direction = 'lateral', right-click the filter and select Query with selected filters in a new tab in the drop-down menu.
    the Query with selected filters in a new tab option selected
    A new tab opens with the results for the selected filter and the original query is left intact on the previous tab.
  3. To query the selected filters in the same tab, select direction = "lateral" and threat.category exists. Then right-click and select Query with selected filters in the drop-down menu.
    the Query with selected filters option selected in the drop-down menu
    A query with only the selected filters is submitted and all remaining filters are removed.

Delete a Filter in Guided Mode

To delete a filter:

  1. Click X in a filter, click the filter to select it and press Delete, or right-click one or more filters and select Delete selected filters in the drop-down menu.
  2. If you want to execute the query again, click the query events button.
    The selected filter is deleted and the Events List is refreshed.

Create a Query in the Free-Form Mode

Free-form queries are most useful when you have a complex query saved that you want to paste, or if you have one in mind that you want to enter quickly, and you know the meta keys, valid operators, and valid syntax for entering values. The following figure illustrates the initial Event Analysis view with the empty Free-Form query builder field. The first example is Version 11.2 and the second example is Version 11.3.

Free-Form query builder, with the field empty

the empty query builder with Free-Form mode selected

The blinking cursor indicates that you can enter a query. You can enter free text here. As more expressions are added and they cannot be displayed in a single line, they wrap to another line and the input area expands vertically so that all filters are visible without scrolling to the right.

These are some examples of queries that you can enter in Free-Form mode:

To find events with an 8- to 11- character username similar to atreeman-72:
user.all length 8-11 && (user.all regex '^a[a-z]{2}ee[a-z]{3}-[0-9]{2}')

To find events that are either HTTP network events or related to aix or ciscoasa logs:
service=80 || (device.type = 'aix','ciscoasa')

To find all outbound events not going to Canada or the United States:
direction = 'outbound' AND not(country.dst = 'united states' || country.dst = 'canada')

If you have a submitted query in Guided Mode, the query is transformed into text when you click switch to Free-Form mode. This is an example of a query submitted in Guided Mode as two filters, service = 80 and direction = 'outbound', and then viewed in Free-Form mode.
query from Guided Mode

The the query events button button on the right side of the query builder is visible as needed to input a query. The query is applied when you click the query events buttons. At that time the query is validated to show syntax and logic errors.

Operations that require more processing time are not highlighted as they are in Guided Mode, but this table provides a summary of expensive operations for reference.

                                                    
Index MethodNon-Text ValueText ValueRegular OperationsExpensive Operations
By Key  exists, !existseq, !eq
By Key  exists, !existseq, !eq, begins, ends, contains
By Value  exists, !exists, eq, !eqno expensive operators
By Value  exists, !exists, eq, !eq, begins

ends, contains

By Nonespecial case for sessionid exist, !exits, eq, !eq

no expensive operators

You are here
Table of Contents > Analyzing Raw Events and Meta Data in the Event Analysis View > Filter Results in the Event Analysis View

Attachments

    Outcomes