Investigation: Visualize Metadata as Parallel Coordinates

[#1]

This topic tells analysts how to use the parallel coordinates visualization in the Navigate view to focus the investigation on combinations of meta keys and values that may indicate events are abnormal and worth investigation.

The parallel coordinates chart is a way of visualizing the current drill point in Investigation to examine more than two meta keys simultaneously. Visualizing multiple meta keys simultaneously can help in identifying security issues associated with multivariate patterns and comparisons, such as when individual meta keys and values may not be of concern, but combining them together may bring an abnormal pattern or relationship to light. 

Best Practices for Effective Parallel Coordinates Charts

To create effective parallel coordinates charts, follow these recommendations:

• Start from a drill point in the Navigate view rather than attempting to visualize all data. 
• Limit the time range if necessary.
• Choose the smallest useful set of meta keys to display as axes. 
• Specify the sequence of axes to highlight anomalies between the meta values as you follow a line across the chart.
• When you can identify a useful set of meta keys and sequence, create a custom meta group to use for future investigations. For example, you can create a custom meta group for Windows executable file types.
• Import custom meta groups that RSA has distributed through the RSA community.
• Re-use and share custom meta groups by importing and exporting groups as .jsn files.
• It may be useful to create two versions of each custom meta group. One for analysis of meta values and one for creating a parallel coordinates chart focusing on a smaller subset of the same use case.

Note: When importing meta groups meta groups into Security Analytics server, Security Analytics displays an error message if any of the groups are already present in Security Analytics.  To import a group that is a duplicate, you must first delete the existing group. If you want to delete a meta group, it can not be in use by a profile.

To help build better parallel coordinates charts, several optimizations are included in Security Analytics10.5 and above.

• Analysts can specify that only sessions in which all meta keys exist are rendered in the chart.
• The administrator can increase the number of meta values rendered in the Parallel Coordinates Settings in the Administration System view.

RSA Meta Groups for Parallel Coordinates Use Cases

A set of predefined custom meta groups is available through the RSA community as a jsn file: MetaGroups_ootb_w_query.jsn. To get started with some meta groups that RSA has configured to highlight certain activities, you can import this .jsn file in the Manage Meta Groups dialog. Some of the targeted activities that lend themselves well to Parallel Coordinates visualizations are:

• Botnet Beaconing
• Covert Channels
• Email
• Encrypted Sessions
• File Analysis
• Malware Analysis
• Query Files
• Query Hosts
• Query IPs
• Query Mail
• Query Users
• Query Web
• SQL Injection Attacks
• Threat Analysis
• Web Analysis

View a Parallel Coordinates Visualization

From an investigation in the Investigation > Navigate view:

1. If the Visualization panel above the Values panel is closed, select Visualization.
2. In the toolbar, select Use Meta Group > File Analysis.
3. In the Values panel, in the Forensic Fingerprint meta key, click windows_executable and then javascript, so that the breadcrumb reads filetype = 'windows_executable' | filetype = 'javascript'.
   
4. A default visualization for the current drill point is displayed as a timeline.
   
5. In the Visualization panel, select Options.
   The Visualization Options dialog is displayed.
6. In the Visualization drop-down list, select Coordinates and click Apply.
   
   The visualization is loaded. In this example, 249 events are found and 199 unique paths are visualized.
   

Select Meta Keys for a Parallel Coordinates Visualization

With a Parallel Coordinates visualization open, do the following:

1. In the Visualization panel, select Options.
   The Visualization Options dialog is displayed. In the toolbar, click to display the recommended number of axes for a readable visualization. When a recommended count of keys is displayed, the count changes based on the browser size. If you make the browser window larger, the recommended count is increased.
   
2. If you want to change the sequence of the meta keys, drag meta keys up or down to the desired sequence. 
3. If you want to delete any meta keys, click in the selection box, and click .
   The meta keys are removed, but the change has not been applied.
4. If you want to revert to the previous state, click .
   Any meta keys you have deleted are restored and any changes that you made are removed.
5. If you want to select individual meta keys, click , select From Default keys, and in the drop-down list, select the meta keys.
   
   The selected keys are listed.
   
6. If you want to add all the keys in a meta group, you cannot add individual meta keys. Select From Meta Groups, and select a group from the drop-down list.
   
   The selected meta groups are listed in the field.
7. Select the method of adding the keys or groups: Replace the current list of keys, Append to the current list of keys (at the end), or Insert at the beginning of current list of keys.
   
8. To complete the procedure, click Add.
   The Visualization Options dialog is displayed with the meta keys or groups you selected.
9. To display the new visualization chart, click Apply.
   

Optimize a Parallel Coordinates Visualization

1. To optimize the visualization by removing events in which not all meta keys exist, select Options.
   
2. In the Visualization Options dialog, select All Meta Keys Must Exist in an Event. Click Apply.
   The resulting graph is more readable and useful and usually has fewer unique paths.
   
3. If you want to highlight a small set of points to see the path of the line from right to left, click on an axis. The cursor changes to cross hairs, which you can drag to select one or more values. When you let go of the mouse, the lines are highlighted. In the example below, the SSL service type is highlighted by a gray  box.
   
4. If you want to enlarge the visualization, drag the bottom edge of the panel down and drag the right edge of the browser window wider.

Sample Use Case

Below is an example of a parallel coordinates visualization of meta keys representing file metadata in a session. There are three meta keys or axes from left to right: Extensions, Forensic Fingerprint, and Filename with values listed along each axis. Values on the Extension axis show the file extension, and values on the Forensic fingerprint axis are windows executables. Normally the file type matches the expected forensics fingerprint; however, it is abnormal for a gif file type to be combined with the Windows executable fingerprint. The gif file type is selected to highlight the correlations of that file type, x86pe , and two filenames in the third axis so that an analyst can quickly identify the files that merit investigation.

To reach this view:

1. Order by Value and Sort in Ascending order.
2. Apply two filters (file type = 'windows executable' and extension = 'gif') in the Navigate view to limit the amount of data.
3. Configure a parallel coordinates chart by choosing three axes: file extension, forensic fingerprint, and filename.
   

Sample Visualization of a Large Data Set

This example of a parallel coordinates visualization applied to a larger set of data illustrates several messages that help analysts to understand what has been charted.

• To create a chart, Security Analytics begins scanning meta values and returning results. A typical time range could have up to 10,000,000 meta values. When the number of meta values returned reaches the Meta Values Result Limit, the chart is rendered even if Security Analytics has not scanned a number of meta values equal to the Meta Values Scan Limit. 
• There is a fixed limit on the amount of data that can be rendered as a parallel coordinates chart. In Security Analytics 10.4 and prior, the limit is based on the number of axes times data values: 1000 x the number of axes to protect performance, but in Security Analytics 10.5 and above the administrator configures parallel coordinates limits as part of the Investigation settings In the  Administration > System view.

With a larger set of data, the parallel coordinates chart takes longer to process than the smaller set of data and meta keys. To preserve performance, Security Analytics renders the meta values from the Values panel below until the limits set by the Administrator are reached. An informational message tells you: Only a subset of events is displayed.

Of all the data visualized for 249 events, there were only 199 unique parallel coordinates paths. Some events are included though they do not include some of the meta keys; these are labeled DNE because the meta does not exist in the event.

Previous Topic:Use Investigation Profiles to Encapsulate Custom Views

Next Topic:Query Data in the Navigate View

You are here

Table of Contents > Conduct an Investigation > Filter Information in the Navigate View > Visualize Metadata as Parallel Coordinates