Skip navigation
All Places > Products > RSA NetWitness Platform > RSA NetWitness Platform Online Documentation > Documents
Log in to create and rate content, and to follow, bookmark, and share content with other members.

Warehouse Connector: Manage a Stream

Document created by RSA Information Design and Development Employee on Sep 14, 2017Last modified by RSA Information Design and Development Employee on Sep 4, 2020
Version 15Show Document
  • View in full screen mode
 

You can manage a stream using the following procedures:

  • Edit a Stream
  • Reload the Stream
  • Specify meta filters for a Stream
  • Define multi-valued metas

Edit a Stream

You can edit a stream to perform the following:

  • Add data sources to the stream.
  • Delete existing data sources from the stream.

To edit a stream:

  1. Go to (Admin) > Services.
  2. In the Services view, select the added Warehouse Connector service and select   > View > Config.

    The Services Config view of Warehouse Connector is displayed.

    Source and Destination in Service view

  3. On the Streams tab, click .
  4. In the Edit Stream dialog, you can perform the following:

    • On the Available Sources tab, you can select the available data sources to add to the stream and click Save.

      edit stream

    • On the Current Sources tab, you can delete an existing data source from the stream. Select the data source and click .

      edit stream

Reload the Stream

When you reload the stream, the Warehouse Connector updates the schema file for the stream. You must reload the stream when you add a new custom meta to the Log Decoder or Decoder.

To reload the stream:

  1. Go to (Admin) > Services.
  2. In the Services view, select the added Warehouse Connector service and select   > View > Config.

    The Services Config view of Warehouse Connector is displayed.

  3. On the Streams tab, select the stream that you want to reload.
  4. Click Reload.

Specify Meta Filters for a Stream

You need to specify the filter for each stream in the export.session.meta.fields parameter in the Explore view of the Warehouse Connector.

The following table lists the values that you can provide as a filter:

                       
ValuesDescription
*All the collected metas are written to SAW.
*, meta1, meta2All the metas except the defined metas are written to SAW.
For example,
Filter:*,ip.src
All the metas except ip.src is written to SAW.
meta1, meta2, meta3 Only the defined metas are written to SAW.

Note: By default, the following metas are written to Warehouse even if you specify them in the filter:
- ng_source
- unique_id
- time

To specify meta filters for a Stream:

  1. Go to (Admin) > Services.
  2. In the Services view, select a Warehouse Connector services and select   > View > Config.

    The Explore view of the Warehouse Connector service is displayed.

    Warehouse connector explore view

  3. In the options panel, select warehouseconnector > streams > <stream_name> > loader > config.
  4. In the export.session.meta.fields parameter, enter the filter.

    specify meta filter

  5. Restart the stream.

Define Multi-valued Metas

You can also define an existing meta or a custom meta to be treated as multi-valued meta.

To define multi-valued metas:

Caution: Defining an existing meta to be treated as multi-valued may change the data type of the meta and cause the associated reports to fail.

  1. Create a new file with the filename multivalue-users.xml in the /etc/netwitness/ng directory.
  2. Add the following entries:

    <?xml version="1.0" encoding="utf-8"?>

    <NetWitness>
       <MultiValueMetas>
          <Meta>NEWMETANAME</Meta>
       </MultiValueMetas>
    </NetWitness>

    Where NEWMETANAME is the existing meta or a custom meta to be treated as multi-valued meta. 

    Caution: Make sure that you do not add metas that are by default treated as non multi-value.

  3. Restart the stream.

View the current schema

You can view the current schema that is used by warehouse connector for writing in AVRO files.

To view the current schema:

  1. Go to (Admin) > Services.
  2. In the Services view, select a Warehouse Connector services and select   > View > Explore.

    The Explore view of the Warehouse Connector service is displayed.

  3. In the options panel, select warehouseconnector > streams > <stream_name> > config.
  4. Set the value for refresh.schemanode.enabled parameter to true. By default, this value is set to false.

  5. Reload the stream. For more information see, Reload the Stream.

  6. Restart the Warehouse Connector service.
  7. Go to (Admin) > Services.
  8. In the Services view, select a Warehouse Connector services and select   > View > Explore.
    The Explore view of the Warehouse Connector service is displayed.

  9. In the options panel, select warehouseconnector > streams > <stream_name> > schema, to view the current schema.

Note: You must to reload the stream every time, before you want to view the current schema or you can add a scheduler to reload the stream automatically at regular intervals as mentioned in step 10.

  1. (Optional) To reload the stream automatically at regular intervals, follow the below steps.
    1. Go to (Admin) > Services.
    2. In the Services view, select a Warehouse Connector services and select   > View > Explore.
      The Explore view of the Warehouse Connector service is displayed.
    3. In options panel, select warehouseconnector > sys > config > scheduler. Right click and select properties.
    4. In the property drop down select addInter and in the Parameters text box, add "hours=24 pathname=/warehouseconnector/streams/<stream name> msg=reload".

    Note: Standard time format used is hours. You can use seconds or minutes format in lieu of hours.

    1. Then click Send.

Next Topic:Manage a Lockbox
You are here
Table of Contents > Manage a Stream

Attachments

    Outcomes