Decoder: Create Custom Meta Keys Using Custom Feed

Document created by RSA Information Design and Development on Sep 13, 2017Last modified by RSA Information Design and Development on Apr 10, 2019
Version 10Show Document
  • View in full screen mode
 

This topic provides information on how to add custom meta keys using a custom feed in the Log Decoder, and highlights the configuration changes to reflect the custom meta keys in the Concentrator, ESA, Archiver, Warehouse Connector, and Reporting Engine schema. You can create custom meta keys to retrieve data, to investigate and analyze the logs and packets. Custom meta keys enable you to add an enrichment context for the log and packet data.

Here is an example of creating a custom meta key in the Log Decoder. In this scenario, an organization wants to track the location of an asset such as a printer. So, a custom meta key source location is introduced, which indicates the location of the asset, for example Printer1, which is located in the 'Fifth Floor A wing'. 

Note: Custom meta keys can be created in the Decoder as well. Select the index-decoder-custom.xml file when you create a custom meta key in the Decoder.

Add a Custom Meta Key in the Log Decoder

To add custom meta keys using custom feed:

  1. Go to ADMIN > Services .
  2. Select a Log Decoder service and click Actions menu> View > Config > Files tab > index-logdecoder-custom.xml.

<Language>
 <?xml version="1.0" encoding="utf-8"?>
 <Language level="IndexNone" defaultAction="Auto">
 <!-- Reserved Meta key for Feed -->
 <Key description="Source Location" level="IndexNone" name="location.src" format="Text"/>
</Language>
name = Name of the key (max is 16 chars)

  1. Restart the Log Decoder service. In the Services view, click Actions menu > Restart.

Deploy a Log Decoder Feed in Live

To deploy the feed in the live environment:

  1. Go to CONFIGURE > Live Content.
  2. Select a group of resources, or a previously created resource package. To select a resource or group of resources:
    1. In the Live Search View, browse Live resources (for example, search for the Log Collector resource Type).
    2. In the Matching Resources panel, select Show Results > Grid.
    3. Select the checkbox to the left of the resources that you want to deploy.
      Matching Resources in the Resources list
    4. In the Matching Resources toolbar, click .
    1. To select a resource package to deploy:
    1. In the Live Search view - Matching Resources toolbar, select Package > Deploy :
      The Package page of the Resource Package Deployment wizard is displayed.
      Resource Package Deploymeny dialog
    2. Click Browse and select a package from your network (for example resourceBundle-FeedsParsersContent.zip).
      ResourceBundle3.PNG
    3. Click Open.
      At this point, whether you are deploying a package or a group of resources, the Deployment Wizard opens, and the Resources page is displayed.
  3. Click Next.
    The Services page is displayed. It has two tabs, Services and Groups, which provide a list of services and service groups that are configured in the Admin > Services view. The columns are a subset of the columns available in the Services view.

    Note: The Live server is "smart" about deploying resources to Services. For example, it does not deploy resources that have a Medium of packets to any Log Decoders. This means that only applicable content resources are deployed to each Service.

  4. Select the services to which you want to deploy the content. You can select any combination of services and service groups.
    Use the Services tab to select individual services, list of services and service groups that are configured in the Admin Services view.
    Use the Groups tab to select groups of services.
  5. Click Next.
    The Review page is displayed.

    Make sure that you have selected correct resources and the services to which you want to deploy them.
  6. Click Deploy.
    The Deploy page is displayed. The Progress bar turns green when you have successfully deployed the resources to the selected services.

    If you try to deploy resources and services that are not compatible, NetWitness Platform displays the Errors and Retry buttons, which you click to review the errors and retry the deployment.

  7. Click Close.

Note: The Source IP should be indexed by selecting the type as 'IP' as the ip.src. and ip.dst are in IPv4 format. 

In this scenario, a custom meta key location.src (location source) is added by indexing the hostname (alias.host). In this example, the printer hostname are populated in meta key 'alias.host'. Select alias.host as callback key, and index type as 'Non IP' in the Feed Wizard as shown below. In the Define Values section, select the custom meta key from the drop down menu.

Add the Custom Meta Key Entry in the Concentrator Custom Index file

To add the custom meta key entry in the Concentrator custom index file:

  1. Go to ADMIN Services > Concentrator.
  2. Click The actions menu > View > Config > Files tab > index-concentrator-custom.xml.
  3. Add the custom meta key entry in the Concentrator index file.

 <Language>
  <?xml version="1.0" encoding="utf-8"?>
  <Language level="IndexNone" defaultAction="Auto">
  <!-- Reserved Meta key for Feed -->
  <Key description="Source Location"  level="IndexValues" name="location.src" format="Text"                 valueMax="10000" defaultAction="Open"/>
 </Language>

  1. To restart the Concentrator service, in the Services view, click The actions menu > Restart.

Note: In case of the Broker, the Broker derives its index from the Concentrator from which it aggregates. So you do not need to create custom meta in the Broker. If you have not indexed the meta key in the Concentrator, the Broker does not display the meta key in Investigate.

Investigate a Custom Meta Key

Note: You have to log out and log back in to the NetWitness Platform User Interface in order to view the custom meta key in Investigate.

To investigate a custom meta key: 

  1. Go to INVESTIGATE.
    The Investigate dialog, which provides services, is displayed.
  2. Select a Concentrator service, and click Navigate

    This is the investigation output.

Here is an example of a report executed on the Concentrator.

This is the Concentrator output.

Additional Procedures

The following procedures must be executed if you have Warehouse Connector, Archiver, Reporting Engine, and ESA configured.

Verify the Custom Meta Keys on ESA 

After you add custom meta keys on the Concentrator, you can verify that your meta keys are updated on ESA.

  1. Go to CONFIGURE > ESA Rules Settings tab.
    ESA Meta Key References
  2. In the Meta Key References, click the Meta Re-Sync (Refresh) icon (Meta Re-Sync (refresh) icon).
  3. Verify that the custom meta keys appear on ESA. If you do not see the meta keys, you may need to restart the Concentrator.

Update the Schema in Archiver

If you want to configure the Archiver, using the new custom meta keys, you need to update the Archiver schema in the Reporting Engine. To update the Archiver schema in Reporting Engine:

  1. Go to ADMIN > Services > Archiver.
  2. Select The actions menu> View > Config > Files > index-archiver-custom.xml.
  3. Add the custom meta key entry in the Archiver index file.

<Language>
 <?xml version="1.0" encoding="utf-8"?> 
 <Language level="IndexNone" defaultAction="Auto">
 <!-- Reserved Meta key for Feed -->
 <Key description="Source Location" level="IndexValues" name="location.src" format="Text"
 valueMax="10000" defaultAction="Open"/>
</Language>

  1. To restart the Archiver service, click The actions menu > Restart.
    The Archiver schema is updated with the custom meta key.

Update the Schema in Warehouse Connector

If you want to configure the Warehouse Connector with custom metadata and use it in a Warehouse Connector report then you need to update the Warehouse Connector schema in the Reporting Engine.

If the Log Decoder or Decoder, where the custom meta key is added, is one of the sources in the Warehouse Connector stream, you need to update the schema in the Warehouse Connector.

To update the Warehouse Connector schema in the Reporting Engine:

  1. Go to ADMIN > Services Warehouse Connector.
  2. Click  The actions menu > View > Config.
    The Services Config view of Warehouse Connector is displayed.
  3. Click the Streams tab.
  4. Select the stream and then click Reload.
    The Warehouse Connector pulls the schema from the downstream devices (Log Decoder/Decoder).
    Warehouse Connector Streams tab

For more information on streams, see "Configure Streams" in the Warehouse Connector Configuration Guide.

Update the Schema in Reporting Engine

To update the schema in Reporting Engine:

  1. Go to ADMIN > Services Reporting Engine.
  2. Click The actions menu > Restart.

Note: Restart the Reporting Engine or wait for thirty minutes for the schema to be updated.

To view the custom meta key:

  1. Navigate to Monitor > Reports > Rules.
  2. In the toolbar, click The add icon.
  3. Select Warehouse DB.
  4. In the Build Rule tab, search for the custom meta from the right panel.
    The custom meta key is displayed.

This is an example of the Build Rule tab.

You are here
Table of Contents > Configure Feeds and Parsers > Create Custom Meta Keys Using Custom Feed

Attachments

    Outcomes