Investigation Feed

Document created by RSA Information Design and Development on Nov 8, 2016Last modified by RSA Information Design and Development on Nov 15, 2018
Version 150Show Document
  • View in full screen mode
 

The Investigation feed generates metadata in order to assist analysts with threat hunting and content generation such as reports and alerts. This is useful for front line analysts, because it minimizes the time dedicated to mining logs or sessions in support of their findings. The content within the feed is a list of application rules or network Lua parser logic and their associated tags according to the Investigation Model.

Each piece of content has this feed listed as a dependency for deployment through Live and, therefore, no manual deployment is necessary. If you need to manually deploy, or are running a version of RSA NetWitness Platform prior to 10.6.2, see the Deployment section within this document.

The feed is multi-callback, meaning it uses multiple keys to trigger the meta generation. Each application rule or piece of Lua parser logic will output to one of the six hunting keys, or for older content, the meta key called alert.id. On trigger of the content, the feed will fire based on a match to one of these keys and output the additional meta data according to the Investigation Model.

Note: Content that currently registers alert.id values will be updated to instead register directly to the appropriate hunting key instead.

An example output of matches to application rules and Lua parser logic is below. The content writes to one of the six Hunting Keys and the feed generates the Investigation Category and Investigation Context meta.

Feed Structure

The Investigation Feed has the following structure:

  • Index or Trigger Meta Keys

    • Indicators of Compromise
    • Behaviors of Compromise
    • Enablers of Compromise
    • Session Analysis
    • Service Analysis
    • File Analysis
  • Registered Meta Keys:

    • Investigation Category
    • Investigation Context
    • Feed Name

Index/Trigger Meta Keys

These are the keys that trigger the feed callback. All application rules and Lua parser logic developed since the creation of the hunting keys writes to one or more of these six keys.

Hunting Compromise Keys

Compromise keys provide insight and narratives into the varied attributes of an attack. These can be atomic or computed indicators.

  • Indicators of Compromise (ioc): Datatypes used in Threat Indicator Portals, or known signature-type resources should be pushed here. Anything worthy of analysis which denotes high confidence.
  • Behaviors of Compromise (boc): Tactics or techniques employed by malware and/or adversaries. Sometimes this observed behavior could be an anomaly or just poorly written code, or simply administrator activity. Use this key when there is no datatype indicator present but signifies potential cause for concern if high value hosts or parties are involved.
  • Enablers of Compromise (eoc): Reserve usage of this key for activities or policies that may contribute to an incident such as servers running default credentials or shared access amongst administrators.

Hunting Analysis Keys

Analysis keys provide an operational workflow for investigative operations through natural language descriptors. They quickly dissect collections based on features observed in network and security information.

  • Session Analysis (analysis.session): Client-Server communication summations, deviations, conduct and session attributes.
  • Service Analysis (analysis.service): Core application protocols identification. An underlying powerhouse of service-based inspection.
  • File Analysis (analysis.file): A large inspection library that highlights file characteristics and anomalies.

Registered Meta Keys

These are the keys that are generated based on a callback to an indexed meta key.

  • The Investigation Category key (inv.category) pinpoints the purpose of a log's or session's escalation. These investigation categories help dictate one’s analysis approach. There are four Investigation Categories:

    • Threat: Threat monitoring escalations may be assigned to incident response or security teams.
    • Identity: Identity-based content tagged as such assists with rapid analysis in the NetWitness Investigation module, and is useful for determining who is responsible for a certain request, or what is normal for a certain user. For example, a meta value tagged in the identity category could be directly escalated to the Information technology access management personnel responsible for provisioning access.
    • Assurance: Similarly, the Assurance category houses all resources that a risk organization would leverage in their calculations towards potential exposure.
    • Operations: Lastly, the Operations category contains all content that performs session analysis or protocol inspection that is most often utilized in command centers, telecommunications teams and security operations.
  • The Investigation Context key (inv.context) expands on the aforementioned category key, but also describes the literal intent or functional objective of the resource itself. This tactic allows content engineers to organize resources based on natural language descriptors. The model on which these keys are based and the possible values is described in the NetWitness Investigation Model.
  • The Feed Name key (feed.name) will always be populated with the value investigation.

Discontinued and Deprecated Feeds

The Hunting feed has been discontinued, and the Alert ID feeds are deprecated.

Hunting Feed Discontinued

The Hunting feed has been discontinued with the release of the Investigation feed. The Hunting feed generated the same text-based name of the content logic into one of the six Hunting meta keys. Since content now directly writes to one of the six Hunting meta keys, the Hunting feed is no longer necessary.

Note: RSA recommends removal of the Hunting feed from the Log Decoders and Decoders.

To remove the legacy feeds from your Decoders:

  1. Depending on your version:

    • For RSA Security Analytics 10.x: In the Security Analytics menu, select Administration > Services and select a Decoder service.
    • For RSA NetWitness 11.x: From the NetWitness UI, select ADMIN > Services, then select a Decoder service.
  2. In the Actions column, select View > Config.
  3. Select the Feeds tab.
  4. Search for and select the Hunting feed.
  5. Click to remove the selected feed.

Alert ID Feeds Deprecated

Going forward, Content will only write to one of the six Hunting meta keys. Previously, Content wrote to the Alert ID meta key. The value of that meta key was a unique identifier, for example nw132520. The three Alert IDs feeds use the Alert ID meta key to generate a text-based name of the content to one of the meta keys of Risk: Informational, Risk: Suspicious, or Risk: Warning. These keys are deprecated in favor of the Hunting keys, making the Alert ID feeds legacy content.

Note: If you actively use the Risk: Informational, Risk: Suspicious, and Risk: Warning meta keys within your system, RSA recommends you leave the Alert ID feeds in place. Once your content and processes have been updated to use the Hunting and Investigation keys, the feeds should be removed from the Log Decoders and Decoders. Until the Alert IDs feeds are removed, you will get some duplication of meta values across the Risk and Hunting keys (as described in more detail in the following section).

Effects of Continuing Use of Legacy Content

The Alert ID feeds are deprecated, but are still active in Live. They are still in Live for the following reasons:

  • RSA might make future updates to the Alert ID feeds, and
  • Some customers are still using the generated meta from these feeds (the Risk * and alert.id keys).

Note the following duplication of data issues:

  • You will see exact duplicate meta keys and values if you have the Hunting feed and Investigation feeds enabled.
  • You will see some duplication of meta values generated by both the Risk* and Hunting keys. Thus, RSA recommends that customers begin moving from the Risk* keys (Risk: Informational, Risk: Suspicious, Risk: Warning) and Alert ID* feeds (Alert IDs Info, Alert IDs Suspicious, and Alert IDs Warning) in your content to the Hunting keys.

Example of duplicated data:

  • Generated from the Alert ID Info feed

    risk.info = "http post missing content-type"

  • Generated from the Investigation feed:

    analysis.service = "http post missing content-type"

  • Generated from the Hunting feed:

    analysis.service = "http post missing content-type"

Once you have updated your content so that it is not dependent on the Risk keys and Alert ID feeds, you can remove the Alert ID feeds from your system, and thus you will stop receiving duplicate meta values.

 

Deployment

This section discusses:

  • How to add the meta keys used by the feed to the Index file, and
  • How to deploy the feed.

    Note: Manual deployment of the feed should not be required, since it will be downloaded and deployed as a dependency with the content through Live.

About the Meta Keys

To get value from the Investigation feed, the following meta keys are required. The below keys should be added to your index-concentrator-custom.xml.

Note: These keys are now being delivered out of the box in index-concentrator.xml with NetWitness version 10.6.2 and newer. If your installed version is prior to 10.6.2, you must add the keys to index-concentrator-custom.xml.

<key description="Investigation Category" level="IndexValues" name="inv.category" format="Text" valueMax="10000"/>
<key description="Investigation Context" level="IndexValues" name="inv.context" format="Text" valueMax="10000"/>
<key description="Session Analysis" level=IndexValues" name="analysis.session" format="Text" valueMax="10000"/>
<key description="Service Analysis" level=IndexValues" name="analysis.service" format="Text" valueMax="10000"/>
<key description="File Analysis" level="IndexValues" name="analysis.file" format="Text" valueMax="10000"/>
<key description="Indicators of Compromise" level="IndexValues" name="ioc" format="Text" valueMax="10000"/>
<key description="Behaviors of Compromise" level="IndexValues" name="boc" format="Text" valueMax="10000"/>
<key description="Enablers of Compromise" level="IndexValues" name="eoc" format="Text" valueMax="10000"/>

To add keys to index-concentrator-custom.xml:

If your installed version of NetWitness is 10.6.2 or newer, you can skip this procedure.

  1. In the Security Analytics menu, select Administration > Services, and select a Concentrator.
  2. Select View > Config from the Actions menu.
  3. Select the Files tab, then select the index-concentrator-custom.xml file.
  4. Add the following lines:

    <key description="Investigation Category" level="IndexValues" name="inv.category" format="Text" valueMax="10000"/>
    <key description="Investigation Context" level="IndexValues" name="inv.context" format="Text" valueMax="10000"/>
    <key description="Session Analysis" level=IndexValues" name="analysis.session" format="Text" valueMax="10000"/>
    <key description="Service Analysis" level=IndexValues" name="analysis.service" format="Text" valueMax="10000"/>
    <key description="File Analysis" level="IndexValues" name="analysis.file" format="Text" valueMax="10000"/>
    <key description="Indicators of Compromise" level="IndexValues" name="ioc" format="Text" valueMax="10000"/>
    <key description="Behaviors of Compromise" level="IndexValues" name="boc" format="Text" valueMax="10000"/>
    <key description="Enablers of Compromise" level="IndexValues" name="eoc" format="Text" valueMax="10000"/>

  5. Click Apply.

This screen shows the lines being added to the file in NetWitness:

Deploy the Feed

Depending on your version, see:

Previous Topic:RSA Investigation Meta
You are here
Table of Contents > RSA NetWitness Platform Content > Feeds > Investigation Feed

Attachments

    Outcomes