000035708 - How to correct duplicated meta data in RSA NetWitness Logs & Packets

Document created by RSA Customer Support Employee on Nov 8, 2017Last modified by RSA Customer Support Employee on Nov 8, 2017
Version 2Show Document
  • View in full screen mode

Article Content

Article Number000035708
Applies ToRSA Product Set: NetWitness Logs & Packets, Security Analytics
RSA Version/Condition: 10.x, 11.x
Platform: CentOS
O/S Version: 6, 7

Users may see duplicated meta data when they subscribe to the new Investigation Feed.

The Investigation Feed is meant to immediately replace the Hunting Feed, and in the near term, also replace the Alert IDs feeds.

The Hunting Feed generated the same text-based name of the content logic into one of the six Hunting meta keys. Since Content now directly writes to one of the six Hunting meta keys, the Hunting Feed is no longer necessary.

Going forward, Content will only write to one of the six Hunting meta keys. Previously, Content wrote to the Alert ID meta key. The three Alert IDs Feeds use the Alert ID meta key to generate a text-based name of the content to one of the meta keys of Risk: Informational, Risk: Suspicious, or Risk: Warning. These keys are deprecated in favor of the Hunting keys, but are still active in Live.


The Alert ID feeds are still in Live for the following reasons:

  • RSA might make future updates to the Alert ID feeds, and
  • Some customers are still using the generated meta from these feeds (the Risk * and alert.id keys).

Note the following duplication of data issues:

  • You will see exact duplicate meta keys and values if you have the Hunting Feed and Investigation Feeds enabled.
  • You will see some duplication of meta values generated by both the Risk* and Hunting keys. Thus, RSA recommends that customers begin moving from the Risk* keys (Risk: Informational, Risk: Suspicious, Risk: Warning) and Alert ID* Feeds (Alert IDs Info, Alert IDs Suspicious, and Alert IDs Warning) in your content to the Hunting keys.

Example of duplicated data:

  • Generated from the Alert ID Info Feed: risk.info = "http post missing content-type"
  • Generated from the Investigation Feed: analysis.service = "http post missing content-type"
  • Generated from the Hunting Feed: analysis.service = "http post missing content-type"

RSA recommends the immediate removal of the Hunting Feed from the Log Decoders and  Packet Decoders.

If you actively use the Risk: Informational, Risk: Suspicious, and Risk: Warning meta keys within your system, RSA recommends you leave the Alert ID Feeds in place for now.

We recommend that you update your Content and processes to use the Hunting and Investigation keys. Then, you can and should removed the Alert IDs Feeds from your Log Decoders and Decoders. Once you have removed these feeds from your system, you will no longer receive duplicate meta values.