Decoder: Create Custom Meta Keys Using Custom Feed

Document created by RSA Information Design and Development on Sep 13, 2017Last modified by RSA Information Design and Development on Oct 11, 2017
Version 6Show Document
  • View in full screen mode
 

This topic provides information on how to add custom meta keys, using a custom feed in the Log Decoder.

You can create custom meta keys to retrieve data, to investigate and analyze the logs and packets. Custom meta keys enable you to add an enrichment context for the log and packet data. This document highlights the configuration changes to reflect the custom meta keys in the Concentrator, ESA, Archiver, Warehouse Connector, and Reporting Engine schema.

Here is a example of creating the custom meta key in the Log Decoder. In this scenario, an organization wants to track the location of an asset such as a printer. So, a custom meta key source location is introduced, which indicates the location of the asset, for example Printer1, which is located in the 'Fifth Floor A wing'. 

Note: Custom meta keys can be created in the Decoder as well. Select the index-decoder-custom.xml file when you create a custom meta in the Decoder.

Add a Custom Meta Key in the Log Decoder

To add custom meta keys using custom feed:

  1. Go to ADMIN > Services > Log Decoder.
  2. Select a service and click Actions menu> View > Config > Files tab > index-logdecoder-custom.xml.

<Language>
 <?xml version="1.0" encoding="utf-8"?>
 <Language level="IndexNone" defaultAction="Auto">
 <!-- Reserved Meta key for Feed -->
 <Key description="Source Location" level="IndexNone" name="location.src" format="Text"/>
</Language>

  1. Restart the Log Decoder service. In the Services view, click Actions menu > Restart.

Deploy a Log Decoder Feed in Live

To deploy the feed in the live environment:

  1. Go to CONFIGURE > Live Content.
  2. Select a group of resources, or a previously-created resource package. To select a resource or group of resources:
    1. In the Live Search View, browse Live resource (for example, search for the Log Collector resource Type).
    2. In the Matching Resources panel, select Show Results > Grid.
    3. Select the checkbox to the left or the resources that you want to deploy.
      Matching Resources in the Resources list
    4. In the Matching Resources toolbar, click Deploy icon
      .
    1. To select a resource package to deploy:
    1. In the Live Search view - Matching Resources toolbar, select Package > Deploy :
      The Package page of the Resource Package Deployment wizard is displayed.
      Resource Package Deploymeny dialog
    2. Click Browse and select a package from your network (for example resourceBundle-FeedsParsersContent.zip).
      ResourceBundle3.PNG
    3. Click Open.
      At this point, whether you are deploying a package or a group of resources, the Deployment Wizard opens, and the Resources page is displayed.
  3. Click Next.
    The Services page is displayed that has two tabs, Services and Groups, which provide a list of services and service groups that are configured in the Admin > Services View. The columns are a subset of the columns available in the Services View.

    Note: The Live server is "smart" about deploying resources to Services. For example, it does not deploy resources that have a Medium of packets to any Log Decoders. This means that only applicable content resources are deployed to each Service.

  4. Select the services to which you want to deploy the content. You can select any combination of services and service groups.
    Use the Services tab to select individual services, list of services and service groups that are configured in the Admin Services view.
    Use the Groups tab to select groups of services.
    DeploymentServices.png
  5. Click Next.
    The Review page is displayed.
    DeploymentReview.png
    Make sure that you have selected correct resources and the services to which you want to deploy them.
  6. Click Deploy.
    The Deploy page is displayed. The Progress bar turns green when you have successfully deployed the resources to the selected services.
    DeploymentDeploy.png
    If you try to deploy resources and services that are not compatible, NetWitness Suite displays the Errors and Retry buttons, which you click to review the errors and re-attempt the deployment.
    DeployErrorsRetry2.png
  7. Click Close.

Note: The Source IP should be indexed by selecting the type as 'IP' as the ip.src. and ip.dst are in IPv4 format. 

In this scenario, a custom meta key location.src (location source) is added by indexing the hostname (alias.host). In this example, the printer hostname are populated in meta key 'alias.host'. So, select 'alias.host' as callback key, and index type as 'Non IP' in the Feed Wizard as shown below. In the Define Values section, select the custom meta key from the drop down menu.

Add the Custom Meta Key Entry in the Concentrator Custom Index file

To add the custom meta entry in the concentrator custom index file:

  1. Go to ADMIN Services > Concentrator.
  2. Click The actions menu > View > Config > Files tab > index-concentrator-custom.xml.
  3. Add the custom meta key entry in the Concentrator index file.

 <Language>
  <?xml version="1.0" encoding="utf-8"?>
  <Language level="IndexNone" defaultAction="Auto">
  <!-- Reserved Meta key for Feed -->
  <Key description="Source Location"  level="IndexValues" name="location.src" format="Text"                 valueMax="10000" defaultAction="Open"/>
 </Language>

  1. To restart the Concentrator service, in the Services view, click The actions menu > Restart.

Note: In case of the Broker, the Broker derives its index from the Concentrator from which it aggregates. So you do not need to create custom meta in the broker. If you have not indexed the meta key in the concentrator, the broker will not display in the investigation.

Investigate on the Custom Meta Key

Note: You have to log out and log in from the NetWitness Suite User Interface, before you can view the custom meta key in Investigation.

To investigate on the custom meta key: 

  1. Go to INVESTIGATE > Navigate.
  2. Select a Concentrator service, and click Navigate

    This is the investigation output.

Here is an example of a report executed on the concentrator.

This is the Concentrator output.

Additional Procedures

The following procedures must be executed if you have Warehouse Connector, Archiver, Reporting Engine and ESA configured.

Update the Schema in ESA 

Before you update the schema in ESA, the custom meta key should be indexed in the concentrator.

To update the schema ESA rules and to be able to use the new custom meta keys:

  1. Go to ADMIN > Services > ESA- Event Stream Analysis > View > Config.
  2. Edit the Concentrator Datasource.
  3. Click Test Connection.

This is the Edit Service dialog with a successful test connection.

  1. Click Save after the connection is successful.
  2. Click Apply.
  3. Navigate to Alerts > Configure Settings.

This is an example of the Settings tab.

  1. Click the Search tab and search for the name of the custom meta key.
    The custom meta key name and type is displayed.

Example of custom meta key name and type.

Update the Schema in Archiver

If you want to configure the Archiver, using the new custom meta keys, you need to update the Archiver schema in the Reporting Engine. To update the Archiver schema in Reporting Engine:

  1. Go to ADMIN > Services > Archiver.
  2. Select The actions menu> View > Config > Files > index-archiver-custom.xml.
  3. Add the custom meta key entry in the Archiver index file.

<Language>
 <?xml version="1.0" encoding="utf-8"?> 
 <Language level="IndexNone" defaultAction="Auto">
 <!-- Reserved Meta key for Feed -->
 <Key description="Source Location" level="IndexValues" name="location.src" format="Text"
 valueMax="10000" defaultAction="Open"/>
</Language>

  1. To restart the Archiver service, click The actions menu > Restart.
    The Archiver schema is updated with the custom meta key.

Update the Schema in Warehouse Connector

If you want to configure the Warehouse with custom meta and use it in warehouse report then you need to update the Warehouse schema in the Reporting Engine.

If the Log Decoder or Decoder, where the custom meta key is added, is one of the sources in the Warehouse Connector stream, you need to update the schema in the Warehouse Connector.

To update the Warehouse schema in the Reporting Engine:

  1. Go to ADMIN > Services Warehouse Connector.
  2. Click  The actions menu > View > Config > Files tab > index-logdecoder-custom.xml.
  3. Select the stream and click Reload.
    The warehouse connector pulls the schema from the downstream devices (Log Decoder/Decoder).

For more information on streams, see "Configure Streams" in the Warehouse Connector Configuration Guide.

Update the Schema in Reporting Engine

To update the schema in Reporting Engine:

  1. Go to ADMIN > Services Reporting Engine.
  2. Click The actions menu > Restart.

Note: Restart the Reporting Engine or wait for thirty minutes for the schema to be updated.

To view the custom meta key:

  1. Navigate to Reports > Rules.
  2. In the toolbar, click The add icon.
  3. Select Warehouse DB.
  4. In the Build Rule tab, search for the custom meta from the right panel.
    The custom meta key is displayed.

This is an example of the Build Rule tab.

You are here
Table of Contents > Configure Feeds and Parsers > Create Custom Meta Keys Using Custom Feed

Attachments

    Outcomes