References - File Collection Configuration Parameters

Document created by RSA Information Design and Development on Jun 25, 2017Last modified by RSA Information Design and Development on Jul 28, 2017
Version 2Show Document
  • View in full screen mode
  

This topic describes the user interface for configuring File Collection.

Use this section when you are looking for descriptions of the File Collection user interface and definitions of the features of the user interface.

To access the File Collection Configuration Parameters:

  1. In the Security Analytics menu, select Administration > Services.
  2. In the Services grid, select a Log Collector service.
  3. Click Actions menu cropped under Actions and select View > Config.
  4. In the Log Collector Event Sources tab, select File/Config from the drop-down menu.

The File/Config view in the Event Sources tab has two panels: Event Categories and Sources.

Event Categories Panel

In the Event Categories panel, you can add or delete the appropriate event source types.

                           
FeatureDescription
Icon-Add.png Displays the Available Event Source Types dialog from which you select the event source type for which you want to define parameters.
Icon_Delete_sm.png Deletes the selected event source types from the Event Categories panel.
Checkbox.png Selects event source types.
NameDisplays the name of the event source types that you have added.

Available Event Sources Types Dialog

The Available Event Source Types dialog displays the list of supported event source types.

                           
FeatureDescription
Checkbox.png Selects the event source type that you want to add.
TypeDisplay the event source types that are available to add.
CancelCloses the dialog without adding an event source type.
OKAdds the selected event source type to the Event Categories panel.

Note: The Available Event Source Types dialog displays the list of supported event source types downloaded from the Generic File Reader Type Specification (GFTS) file.  If you do not see any event source types in this list, you did not load the content available with Log Collector upgrade to this release.

Sources Panel

Use this panel to review, add, modify, and delete event source file directories and their parameters for the event source type you selected in the Event Categories panel.

Toolbar

The following table provides descriptions of the toolbar options.

                               
FeatureDescription
Icon-Add.png

Displays the Add Source dialog in which you define the parameters for a Firewall host.

Icon_Delete_sm.png Deletes the host that you selected.
icon-edit.png

Opens the Edit Source dialog, in which you edit the parameters for the selected event source.

Select multiple event sources and click icon-edit.png to open the Bulk Edit Source dialog in which you can edit the parameters values for the selected event sources.

Refer to the Log Collection Configuration Guide for detailed information on how to import, export, and edit event sources in bulk.

ImportSourceIcon.PNG

Opens the Bulk Add Option dialog in which you can import hosts in bulk from a comma-separated values (CSV) file.

Refer to the Log Collection Configuration Guide for detailed information on how to import, export, and edit event sources in bulk.

ExportSourceIcon.PNG

Creates a .csv file that contains the parameters for the selected hosts.

Refer to the Log Collection Configuration Guide for detailed information on how to import, export, and edit event sources in bulk.

Add or Modify Source Dialog

In this dialog, you add or modify a file directory for the selected event source.

                       
FeatureDescription
Netflow Source ParametersLists the Netflow event source parameters populated with the default values. Enter or modify the appropriate values.
CancelCloses the dialog without adding a file directory or saving the parameter values for the selected file directory.
OKIn the Add Source dialog, adds the file directory and its parameters. In the Edit Source dialog, applies the parameter value changes for the selected file directory.

File Directory Parameters

The following table provides descriptions of the source parameters.

                                                                                                          
NameDescription
Basic
File Directory*

Collection directory (for example, Eur_London100) into which the File event source places its files. Valid value is a character string that is conforms to the following regular expression:

[_a-zA-Z][_a-zA-Z0-9]*


This means that the file directory must start with a letter followed by numbers, letters, and underscores. Do not modify this parameter after you start collecting event data.

After you create the collection, the Log Collector creates the work, save, and error sub-directories under the collection directory.

Address*IP address of the event source. Valid value is an IPv4 address, IPv6 address, or a hostname including a fully-qualified domain name.
File SpecRegular expression. For example, ^.*$ = process everything.
File Encoding

Internationalization file encoding. Enter the File Encoding method, the following strings are examples of valid methods:

  • UTF-8 (default)
  • UCS-16LE
  • UCS-16BE
  • UCS-32LE
  • UCS-32BE
  • SHIFT-JIS
  • EBCDIC-US
EnabledSelect the check box to enable the event source configuration to start collection. The check box is selected by default.
Advanced
Ignore Encoding
Conversion Errors

Select the check box to ignore encoding conversion errors and ignore invalid data. The check box is selected by default.

Caution: This may cause parsing and transformation errors.

File Disk Quota

Determines when to stop saving files regardless of the Save On Error and  Save On Success parameter settings. For example, a value of 10 indicates that when there is less than 10% available disk left, the Log Collector stops saving files to reserve enough space for your estimated normal collection processing.

Caution: Available disk refers to a partition where the base collection directory is mounted. If the Log Decoder server has a 10TB disk size and 2TB is allocated to base collection directory, then setting this value to 10 causes log collection to stop when less than 0.2TB (10% of 2TB) of space is left. It does not mean 10% of 10TB.

Valid value is a number in the 0 to 100 range. 10 is the default.

Sequential Processing

Sequential processing flag:

  • Select the check box (default) to process event source files in collection order.
  • Do not select the checkbox to process event source files in parallel.
Save On ErrorSave on error flag. Check the checkbox to retain the eventsource collection file when the Log Collector it encounters an error. The check box is selected by default.
Save On SuccessSave eventsource collection file after processing flag. Select to save the eventsource collection file after processing it. The check box is not selected by default.
Eventsource SSH Key

SSH public key used to upload files for this event source. Please refer to Generate Key Pair on Event Source and Import Public Key to Log Collector for instructions on generating keys.

Note: If File collection is stopped, Security Analytics does not update the authorized_keys file with the SSH public key that you add or modify in this parameter. You must restart File collection to update the public key.
You can add or modify the value of the public key in this parameter in multiple File event sources without File collection running, but Security Analytics will not update the authorized_keys file until File collection is restarted.

Manage Error Files

By default, the Log Collector uses the File Disk Quota parameter to ensure that the disk does not fill up with error files. If you set this parameter to true, you can specify one of these:

  • Maximum space allotted to error files in the Error Files Size parameter.
  • Maximum number of error files allowed in Error Files Count parameter.

A reduction percent is also specified, which tells the system how much to reduce when the maximum is reached.

Select the check box to manage error files. The check box is not selected by default.

Error Files Size

Only valid if the Manage Error Files and Save On Error parameters are set to true.
Specifies to what extent Security Analytics saves error files. The value that you specify is the maximum total size of all the files in the error directory.

Valid value is a number in 0 to 281474976710655 range. You specify these values in either Kilobytes, Megabytes, or Gigabytes. 100 Megabytes is the default. If you change this parameter, the change does not take effect until you restart collection or restart the Log Collector service.

Error Files Count

Only valid if the Manage Error Files and Save On Error parameters are set to true. Maximum number of error files allowed in the error directory. Valid value is a number in 0 to 65536 range. 65536 is the default.

If you change this parameter, the change does not take effect until you restart collection or restart the Log Collector service.

Error Files Reduction %

Percent amount by size or count of the error files that the Log Collector service removes when the maximum size or count has been reached. The service removes the oldest files first.

Valid value is a number in the 0 to 100 range. 10 is the default.

Manage Saved Files

Select the check box to manage saved files. The check box is not selected by default.
By default, the Log Collector uses the File Disk Quota parameter to ensure that the disk does not fill up with saved files. If check this check box, you can specify one of these:

  • Maximum space allotted to saved files in the Saved Files Size parameter.
  • Maximum number of saved files allowed in Saved Files Count parameter.

A reduction percent is also specified, which tells the system how much to reduce when the maximum is reached.

Saved Files Size

Only valid if the Manage Saved Files and Save On Success parameters are set to true.
Maximum total size of all the files in the save directory. Valid value is a number in the 0 to 281474976710655 range. You specify these values in either Kilobytes, Megabytes, or Gigabytes. 100 Megabytes is the default.

If you change this parameter, the change does not take effect until you restart collection or restart the Log Collector service.

Saved Files Count

Only valid if the Manage Saved Files and Save On Success parameters are set to true. Maximum number of saved files in the save directory. Valid value is a number in 0 to 65536 range. 65536 is the default.

If you change this parameter, the change does not take effect until you restart collection or restart the Log Collector service.

Saved File Reduction %

Percent amount by size or count of the saved files that the Log Collector service removes when the maximum size or count has been reached. The service removes the oldest files first.

Valid value is a number in the 0 to 100 range. 10 is the default.

Debug

Caution: Only enable debugging (set this parameter to On or Verbose) if you have a problem with an event source and you need to investigate this problem. Enabling debugging will adversely affect the performance of the Log Collector.

Enables/disables debug logging for the event source.
Valid values are:

  • Off = (default) disabled
  • On = enabled
  • Verbose = enabled in verbose mode ‐ adds thread information and source context information to the messages.

This parameter is designed to debug and monitor isolated event source collection issues. The debug logging is verbose, so limit the number of event sources to minimize performance impact.

If you change this value, the change takes effect immediately (no restart required).

CancelCloses the dialog without making adding an event source type.
OKAdds the parameters for the event source.

Generate Key Pair on Event Source and Import Public Key to Log Collector

To generate the key pair on the event source and import the public key to Log Collector:

  1. Double-click puttygen.exe in the C:\sasftpagent directory. The PuTTY Key Generator starts.
  2. Select SSH2 RSA as the type of key to generate.
  3. Click Generate and move the mouse in the PuTTY Key Generator window until the key is generated.
  4. Save the private key:

    1. Click Save private key.
    2. Select Yes to not use a passphrase.
    3. Save the file as private.ppk in the C:\sasftpagent directory.
  5. Add the public key to the Log Collector:

    1. Copy the public key into your buffer so that you can paste it into the parameter in Security Analytics as described in step 5b.

      In the following example, the public key is enclosed in a red box.

    2. Paste the public key from your buffer into the Eventsource SSH Key parameter in Security Analytics. For details, see the Configure File Event Sources topic in the RSA Security Analytics Log Collection Guide.

  6. Close the puttygen.

Tasks:

Step 1. Configure File Event Sources in Security Analytics

Step 2. Configure File Event Sources to Send Events to Security Analytics

You are here
Table of Contents > File Collection Protocol Configuration Guide > References - File Collection Configuration Parameters

Attachments

    Outcomes