Investigate: How NetWitness Investigate Works

Document created by RSA Information Design and Development on Sep 18, 2017Last modified by RSA Information Design and Development on Apr 3, 2018
Version 12Show Document
  • View in full screen mode
 

NetWitness Investigate provides the data analysis capabilities in RSA NetWitness® Suite, so that analysts can analyze packet, log, and endpoint data and identify possible internal or external threats to security and the IP infrastructure.

Note: In Version 11.1, the Hosts and Files views are part of the NetWitness Suite for analysts to investigate endpoint data.

Metadata, Meta Keys, Meta Values, and Meta Entities

RSA NetWitness Suite audits and monitors all traffic on a network. One type of service, a Decoder, ingests, parses, and stores the packets, logs, and endpoint data traversing the network.

The configured parsers and feeds on the Decoder create metadata that analysts can use to investigate the ingested logs and packets. Another type of service, called a Concentrator, indexes and stores the metadata.

The metadata is in the form of a meta key and meta values for the key. For example, ip.src is a meta key, and an IP address that is the source of the traffic is tagged as ip.src. When you view data in Investigate, you see the meta key ip.src and all of the IP addreses (values) that are tagged with that key. Some meta keys are built-in and others may be custom keys defined by the administrator.

Meta entities are available in Version 11.1 and later. A meta entity is an alias that groups together the results from other meta keys. Meta entities organize similar meta keys into a single, easier to use, meta type. Some meta entities are already included by default, and the administrator can create custom meta entities. In Investigate, analysts can use a meta entity in a query, a meta group, a column group, and a profile. Parallel coordinates visualizations do not support meta entities. Administrators can use meta entities to define a query prefix to apply to a user role and a user. The Decoder Configuration Guide provides additional information about creating meta entities and how they can be used in rules.

For example, the default Core database language includes distinct meta keys for IP source and IP destination. One of the built-in meta entities named ip.all represents the combined set of all IP sources and destinations.

Analysts usually query the Concentrator to discover threats. The Concentrator handles queries, only going to the Decoder when a full reconstruction of sessions or raw logs is required. ESA, Malware Analysis, and Reporting Engine also query the Concentrator, where they can quickly get all the pertinent metadata associated with an event and generate information on the event without having to go to each Decoder. In some special cases, analysts may query a Decoder.

 

Note: While a hybrid appliance can perform the Concentrator function, a separate Concentrator appliance is required for any large environment that needs greater bandwidth or events per second (EPS). The Concentrator appliance has storage layout that uses solid state drives for the index, which increases read performance.

Triggers for an Investigation

These are a few examples of triggers for an investigation:

  • You receive intelligence from a third party about a new active directory hack. Starting in the Events view, you use that intelligence to run a search across all of your raw Active Directory log data for the last 24 hours.
  • You are asked by the SOC manager to find any Pokemon Go malware due to its current popularity. Starting in the Navigate view, you craft a query to look for an HTTP session using a specific user agent related to the malware he found on a security blog.
  • An incident responder escalates a ticket that shows some odd indicators related to a host. Starting in the Hosts view, you examine that host to find specific details.
  • You are looking for the next zero day attack and start pivoting through network metadata in the Navigate view to find any abnormal automated sessions leaving the enterprise.
  • You are asked by your SOC manager to find any information related to user jarvis, an employee who was just let go. Starting in the Hosts view, you query against the past week for that username.

Workflow of an Investigation

Analysts can investigate data captured by NetWitness Suite, and deep dive from information on a NetWitness Suite dashboard, a NetWitness Respond incident or alert, a report created by the NetWitness Suite Reporting Engine, or a third-party application. During the course of an investigation, analysts can move seamlessly between the views in Investigation: the Navigate view, the Events view, the Event Analysis view, the Hosts view (Version 11.1 and later), the Files view (Version 11.1 and later), and the Malware Analysis view.

This figure illustrates the NetWitness Investigate submenus.

NetWitness Investigate Submenus

Note: Specific user roles and permissions are required for a user to conduct investigations and malware analysis in NetWitness Suite. If you cannot perform an analysis task or see a view, the administrator may need to adjust the roles and permissions configured for you.

You can access each view from the Investigate submenu and from other Investigate views. You can also go directly into an Investigate view from NetWitness Respond, and go directly from NetWitness Investigate to NetWitness Respond and standalone NetWitness Endpoint. Your use case determines the starting point for your investigation. This table provides general guidance on the starting view for different use cases.

                                   
Go to...Focus
Navigate viewAll meta keys and meta values for logs, endpoints, and packets, grouped by meta key. You can pivot through the data to refine results, then go to the Events view or Event Analysis view, or look up in Malware Analysis or Live. This is the default NetWitness Investigate view. (See Investigating Metadata in the Navigate View.)
Events viewEvents are listed in order by time. You can view the raw event and related metadata, view a reconstruction, and download events and files. You can go to the Event Analysis view. (See Examining Raw Events in the Events View)
Event Analysis viewEvents are listed in the order by time. You can view all meta keys and meta values for logs, endpoints, and packets. You can view the raw event and related meta data, view a reconstruction that offers helpful cues to identify points of interest in a reconstruction. You can go to the Hosts view, pivot to standalone Endpoint, look up in Live, and do external lookups. External lookups allow you to search the internet for meta values that you interacted with, determine passive DNS information related to an IP address, ascertain if a URL is blacklisted, and other third-party context integrations. (See Examining Raw Events and Meta Data in the Event Analysis View
(Version 11.1 and later) Hosts viewHosts on which the NetWitness Endpoint Insights Agents are running are listed. For every host, you can view processes, drivers, DLLs, files (executables), services, and autoruns that are running, and information related to logged-in users. From the Hosts view, you can go to the Navigate and Event Analysis views. (See Investigate Hosts)
(Version 11.1 and later) Files viewUnique files such as PE, Macho, and ELF in your deployment are listed. For each file, you can view details such as, file size, entropy, format, company name, signature, and checksum. From the Files view, you can go to the Navigate and Event Analysis views. (See Investigate Files)
Malware Analysis viewIf you are running a Malware Analysis appliance, you can automatically or manually scan files and see the results of four types of analysis: network, static, community, and sandbox. If a file is malware, you can go to the Hosts view to see which hosts downloaded the file. (See Conducting Malware Analysis

Every situation is unique in terms of the types of information the analyst is attempting to find. Many investigations start in one view, and end in a different view as the analyst learns something and then needs to follow that result to a different line of questioning. This figure shows the high-level workflow of an investigation.

High-Level Investigate Workflow

Focus on Metadata, Query, and Time

Analysts use NetWitness Investigate to hunt for events that drive the incident response workflow and to do strategic analysis after another tool has generated an event. Beginning in the Navigate view, Events view, or Event Analysis view:

  • You start by executing a query on a service for a specific time range, then filter using metadata to a subset of events, reconstruct or analyze an event, and repeat the process to reconstruct or analyze another event.
  • When you encounter an event that bears a closer look, you view the context around the event, and decide whether to create an incident or add the event to an incident. If you decide not to add the event to an incident, you run another query to gain further insight, which starts again at the beginning of the workflow.
  • If you notice suspicious activity or files on a specific host in the network, you can gather additional information about the host and files found on the host in the Hosts and Files view, or in a stand-alone NetWitness Endpoint server.
  • If you find a file or event that potentially contains malware, you can do a Malware Analysis scan of the file or you can open Malware Analysis and start a scan of the service on which the event was seen.

For example, if there is a concern regarding suspicious traffic with foreign countries, the Destination Country meta key reveals all destinations and the frequency of the contact. Drilling into those values yields the specifics of the traffic, such as the IP address of the originator and the recipient. Checking other metadata can expose the nature of attachments exchanged between the two IP addresses.

Focus on Endpoint Analysis

Analysts use the Hosts and Files view to investigate or perform analysis on the hosts or files using various attributes, such as IP address, host name, Mac address, and so on.

  • During an incident triage in the Respond view, review the key information (such as, hostname, filename), and view the context highlights.
  • Pivot to Investigate to open the Navigate view. Select the Endpoint Analysis meta group and review the metadata created.
  • View the metadata in the Event Analysis view to analyze the events. Select the host lookup using the Event Meta panel.
  • In the Hosts view, click on the hostname to view the summary of the endpoint data, snapshots, security configurations, and so on.
  • Perform an on-demand scan to get the most recent information (if required).
  • Search on all snapshots for a specific filename, path, or hash to narrow the search.
  • Review the processes, autoruns, files, libraries, drivers, and system information to investigate further.
  • In the Files view, filter the files using a few indicators (such as, file name, file size, entropy, format, company name, signature, checksum) and pivot to the Navigate view to see if it exists on other hosts in the network.

Focus on NetWitness Respond Incidents and Alerts

An analyst who is working on an incident or an alert in NetWitness Respond can open it in NetWitness Investigate (Navigate view) to do a deeper analysis of the event or alert.

  • The workflow to respond to an incident typically begins in the Respond view, where the analyst who is investigating an incident needs to gather intelligence about the incident in NetWitness Investigate. After you launch an investigation from NetWitness Respond, defined meta keys are queried and the contents of captured packets, logs, and endpoint events is displayed in the Navigate view.
  • If you find events that are relevant to the incident, you can add the events to the incident in Respond. You can also create a new incident in Respond based on one or more events found in Investigate.

NetWitness Investigate Views

This section provides a brief description and example of each main view (Navigate, Events, Event Analysis, Hosts, Files, and Malware Analysis) and introduces views that provide two deep-dive capabilities: additional context for data found and event reconstruction.

Navigate View

The Navigate view provides the capability to drill into and query contents of captured packets, logs, and endpoint events on a Broker, Concentrator, or Decoder (though investigating a Decoder is not typical).

  • When you select a service, the defined meta keys for that service are queried, and values are returned along with the number of events. Clicking on a value at any given level, reveals the results in detail.
  • For certain configured meta keys, such as IP address, or hostname, you can search for additional context information around a value using the Context hub. The additional context may include incidents, alerts, and other sources where the value was mentioned.
  • The Navigate View also provides a sequential visualization of the data in a timeline. Here you can zoom in on a selected time period.

This figure illustrates the Navigate view.

the Navigate view with Context Lookup panel open

Events View

The Events view provides a view of packet, log, and endpoint events in list form so that you can view events sequentially and reconstruct events safely.

  • You can open the Events view for a meta value that you see in the Navigate view.
  • For analysts without sufficient privilege to navigate a service, the Events view is a standalone investigation view in which analysts can access a list of network, log, and endpoint events from a NetWitness Suite Core service without having to drill down through metadata first.
  • The Events view presents event information in three standard forms, a simple grid listing of events, a detailed listing of events, and a log view.
  • You can export events and associated files, and create an incident from an event.

This figure illustrates the Events view.

the Events view

Event Analysis View

The Event Analysis view is an interactive tool to help analysts see the packets, text, or files in an event with visual cues for certain types of information. Depending on the type of reconstruction, for example, packets, text, or files, different information is relevant.

  • When viewing files, you can export files in a zip archive to your local file system.
  • You can download logs from the Text view, and export packets from the Packet view.

This figure is an example of the Event Analysis view.

example of the Event Analysis view

Hosts View

The Investigate > Hosts view lists all hosts with an agent. By default, the hosts are listed based on the last scan time, with the most recently scanned hosts at the top of the list. It provides the capability to drill into the details of the host for investigation.

This figure is an example of the Hosts view.

Hosts View

In this view, you can:

  • Filter and sort hosts to narrow down on the host investigation
  • Export the host attributes to a CSV file
  • Start or stop a scan for the selected hosts
  • Drill-down for host details
  • Pivot to the Navigate or Event Analysis view to investigate on the host
  • Delete hosts

Note: If you have NetWitness Endpoint 4.4.0.2 or later in your deployment, the hosts on which the 4.4.0.2 agent is installed are listed, and can be identified using the agent version. For more information on how you can investigate these hosts, see Investigate NetWitness Endpoint 4.4.0.2 or Later Hosts.

To view details of a host, click on the hostname. The following screen is displayed:

Host Details View

You can:

  • Search on all snapshots (file name, file path, and file SHA-256 checksum are the supported search fields).
  • View multiple snapshots. By default, the data for the latest snapshot is displayed.
  • View additional host information in the following tabs: Overview, Processes, Autoruns, Files, Driver, Libraries, and System Information.
  • Export all categories of endpoint data for the selected host for a specific snapshot in the JSON format.

Files View

The Files view provides a list of unique files found in your deployment and their associated properties. By default, the files are listed based on the first seen time. The following file types, loaded in the memory, are collected during the scan.

  • Portable Executable (PE) (Windows) - These are exe, dll, and sys files. You can view the following properties for each file - checksum, compile details, different sections present in the file, imported libraries, and certificate details (signer, thumbprint, company name).

  • Macho (Mac) - These are app bundles, dylibs, and kernel extensions. You can view the following properties for each file - checksum, different sections present in the file, imported libraries, and certificate details (signer, thumbprint, company name).
  • Executable and Linkable Format (ELF) (Linux) - Each file contains information about checksum, different sections present in the file, and imported libraries. You can view the following properties for each file - checksum, different sections present in the file, and imported libraries.

This figure is an example of the Files view.

Files View

In the Files view, you can:

  • Filter and sort files to narrow down on the investigation
  • Pivot to the Navigate or Event Analysis view to investigate on the file
  • Export the files to a CSV file

Malware Analysis View

The Malware Analysis view provides a means to analyze certain types of file objects (for example, Windows portable executable (PE), PDF, and MS Office) to assess the likelihood that a file is malicious.

  • You can open the Malware Analysis view directly or you can use a context menu action to Scan for Malware from a meta value in a current drill point from the Navigate view.
  • The malware analyst can leverage the multilevel scoring modules to prioritize the massive number of files captured in order to focus analysis efforts on the files that are most likely to be malicious.

This figure illustrates the Malware Analysis view.

example of the Malware Analysis Summary of Events

Contextual Information for an Event

In the Navigate view and the Events view, the Context Lookup panel shows details about elements associated with an event (IP Address, User, Host, Domain, MAC Address, Filename, File hash) in the Context Hub.

  • You can interact with the elements of an event to get further insight including related incidents, alerts, custom lists, Archer assets, active directory details, and NetWitness Endpoint IIOCs.
  • You can click on a data point to return to the Navigate view.

The following figure shows the Context Lookup panel in the Navigate view.

the Context Lookup panel in the Navigate view

Note: For the NetWitness Endpoint, the context lookup is only available for the NetWitness Endpoint 4.4.0.2 or later hosts but not for the NetWitness Endpoint 11.1 hosts.

Event Reconstruction

Three NetWitness Investigate views offer the ability to reconstruct an event: Navigate view, Events view, and Event Analysis view. When you discover an event that merits additional investigation, you can reconstruct an event safely in a form similar to its native form. The rendering of events restricts the use of dynamic or active code that might be contained in the event to limit any adverse outcome to your system or browser. Cache is used to improve performance when viewing previously viewed events. Each analyst has a separate cache of reconstruction data, and you can only access reconstructed events in your own cache.

The Event Reconstruction in the Events view or the Navigate view presents the raw data and the meta keys and meta values for an event in a list form .

  • You can page through the reconstruction to view the next event in this form.
  • Events can be reconstructed using different methods to suit the type of data: meta data, text, hexadecimal, packets, web, mail, files, or the best reconstruction selected automatically.
  • You can export packet capture files, extract files, and export the meta values for the event. This figure is an example of the Event Reconstruction.

the Event Reconstruction view

The Event Analysis view present a interactive event reconstruction, which includes raw data, meta keys and values. Interactive options:

  • Highlight and decode information in headers and payloads.
  • Identify common file signatures.
  • Allow you to search for locations of certain meta keys or values in the reconstruction.
  • You can export events and files.

This figure is an example of a reconstruction in the Event Analysis view.

example of a reconstruction in the Event Analysis view

You are here
Table of Contents > How NetWitness Investigate Works

Attachments

    Outcomes