Archiver: Data Retention Tab - Archiver

Document created by RSA Information Design and Development on Apr 18, 2016
Version 1Show Document
  • View in full screen mode
 
  

This topic describes the Data Retention tab for an Archiver. Administrators use this tab to define the criteria for log retention and storage.

On the Administration > Services > Config view > Data Retention tab of an Archiver, Administrators can define the criteria for log retention and storage. As an Administrator, you can configure hot, warm, and cold storage as well as multiple storage collections with different locations and criteria for retaining logs. For example, you can create a Compliance collection that stores logs for a specific time period as required by government regulations. You can create another collection that stores low value logs in hot storage with a much shorter retention period. The flexibility of these collections enables you to have significantly less overall storage requirements.

Procedures related to this tab are described in Step 3. Configure Archiver Storage and Log Retention.

This tab has the following sections:

  • Total Hot Storage: Enables you to configure the total amount of Hot Tier storage available. You can select or add mount points (paths) for your Hot Tier storage locations. These mount points are attached to fast direct storage, such as Direct-Attached Capacity (DAC) storage and SAN.
  • Total Warm Storage: (Optional) Enables you configure the total amount of Warm Tier storage available. You can select or add mount points for your Warm Tier storage locations. These mount points are attached to secondary storage, such as NAS.
  • Total Cold Storage: (Optional) Enables you to configure the total amount of Cold Tier storage available. You can add a mount point for a Cold Tier storage location to back up your log files. This mount point is attached to offline storage, such as NAS, or temporary storage before archiving to tape. Security Analytics does not manage cold storage.  
  • Collections: Enables you to define individual storage collections for different log types. You can specify the maximum size of the Hot and Warm Storage space, whether to use offline storage (Cold Storage), the number of days to retain the logs in the collection, the data compression, and whether to use a hash algorithm to ensure the data integrity of the files being saved.
  • Retention Rule: Enables you to define rules for each of your log storage collections. You must define at least one rule for each collection. 

To access the Data Retention tab for an Archiver: 

  1. In the Security Analytics menu, select Administration > Services.
  2. Select an Archiver service and ic-actns.png  > View > Config.
  3. In the Services Config view for the service, click the Data Retention tab.
    The Data Retention tab for the Archiver is displayed.
    ArcDrTb.png

Total Hot, Warm, and Cold Storage

The Total Hot Storage section shows the total amount of Hot storage available and the number of hot storage mount points. The Total Hot Storage section shows the total amount of Warm storage available and the number of warm storage mount points. The Total Cold Storage section shows the total amount of Cold storage and the remaining free space available in Cold storage. 

TotalStor.png

Hot, Warm, and Cold Storage Mount Points Dialogs

In the Hot, Warm, and Cold Storage Mount Points dialogs, you can specify the mount points for your storage locations. You can specify portions of this storage to use for your log storage collections.

To access the Hot, Warm, and Cold Storage Mount Points dialogs, click the ic-settings.png icon near the respective section.

ArcHotDb2.png

The following table describes features of the Hot, Warm, and Cold Tier Storage dialogs.

                               
FeatureDescription
ic-add.png Adds a mount point. 
ic-delete.png Removes a mount point. You cannot delete a mount point that is in use unless you delete the associated collections.
ic-checkbox.png Select the mount points that you want to include for the Total Hot, Warm, and Cold Storage. You can only select one mount point for Total Cold Storage. 
Mount PointShows the path to the attached physical storage. For example: /var/netwitness/archiver/database0, which is the location of the hot storage DAC.
Do not add collections or subdirectories to the mount points. Security Analytics will automatically create metadb, packetdb, sessiondb, and index directories for each collection defined on the Archiver:
<storageLocation>/<CollectionName>/metadb
<storageLocation>/<CollectionName>/packetdb
<storageLocation>/<CollectionName>/sessiondb
<storageLocation>/<CollectionName>/index

For example, if your hot storage mount point is /var/netwitness/archiver, then the following directories will be created for each of your collections:
/var/netwitness/archiver/<CollectionName>/metadb
/var/netwitness/archiver/<CollectionName>/packetdb
/var/netwitness/archiver/<CollectionName>/sessiondb
/var/netwitness/archiver/<CollectionName>/index

For Cold Storage, you must include the collection name format specifier %n somewhere in the cold storage mount point path name to avoid filename collisions between collections.
Storage SizeShows the size of the attached storage. The Data Retention tab shows the total amount of storage for your reference.  

Collections

The Collections section lists all of your storage collections along with Total Storage for Hot and Warm Storage.

ArcStorCol.png

The following table describes the features of the Collections section. You can hide some of the columns based on your requirements.

                                                                                 
FeatureDescription
ic-add.png Adds a storage collection. Collection Dialog provides additional details.
ic-delete.png Removes the selected collection. Deleting the collection permanently removes all stored data from the collection, but the empty data directories remain.
ic-edit.png Enables you to edit the selected collection. Collection Dialog provides additional details.
 ic-refresh.pngRefreshes collection information.
ic-checkbox.png Selects a collection. For example, you can select a collection for editing or removal.
CollectionShows the name of your collection, such as Default, Compliance, MediumValue, and LowValue. You can create multiple collections with different criteria for retaining logs. If you do not create any collections, the Default collection is used. 

If a collection has errors, the collection name and the columns with errors appear in red text. 
Usage / Hot StorageShows the current hot storage usage and the maximum hot storage for the collection. When the  size of the logs reach the maximum hot storage amount, the logs are removed or they roll to the next available storage tier (warm or cold). 
Usage / Warm StorageShows the current warm storage usage and the maximum warm storage for the collection. When the  size of the logs reach the maximum warm storage amount, the logs are removed or they roll to available cold storage. 
Cold StorageIndicates whether cold storage is enabled or disabled. A solid colored green circle indicates that cold storage is enabled (). An blank white circle indicates that cold storage is disabled.
RetentionShows the number of days that logs are retained before being removed or optionally moved to cold storage. No Limit indicates that log retention is not restricted by a specified number of days.
For Hot and Warm Storage, size and retention period settings for a collection can override each other based on which criterion (size or time) is satisfied first. 
Velocity (last hour)Shows the number of logs captured over the last hour.
Oldest DateShows the date and time of the last log capture.
DurationShows how may days ago that the last log was captured. For example: 20 days.
CompressionShows the compression type used for the meta and raw data in the collection. 
HashShows whether hash is enabled or disabled. When enabled, the hash algorithm is used to ensure the data integrity of the files being saved. By default, the only data being hashed is raw logs and the hash files are saved in the same directory as data. 
# of RulesShows the number of rules applied to the collection.
Define at least one rule for each collection. A collection without any associated rules shows a zero in red text as a warning: NoRules.png The collection name also appears in red text, which indicates an error in the collection.

Caution: If a collection does not have a rule, no logs will ever go into that collection.

ActionsEnables you to see the rules associated with a collection in the Retention Rule section when you select <actions button> > Select Rules. In the Retention Rule section, you can change the overall priority of the collection rules.
Total Storage Shows the current total hot storage usage and the maximum total hot storage at the bottom of the Usage / Hot Storage column. It also shows the current total warm storage usage and the maximum total warm storage at the bottom of the Usage / Warm Storage column.

Any errors in the collection appear in red text. A dotted underline indicates that a tooltip is available with information about the error.

Collections that have editing disabled (grayed out) also have tooltips that provide information on the problem.

Retention Rules

The Retention Rules section lists all of the retention rules used for your storage collections listed in the order of rule execution. 

ArcRetRule.png

The following table describes the features of the Retention Rule section.

                                                               
FeatureDescription
ic-add.png Adds a retention rule to use in a storage collection. Rule Definition Dialog provides additional details.
ic-delete.png Removes the selected retention rule. In order for your log collections to gather and store log data, you must associate them with at least one retention rule.
ic-edit.png Enables you to edit the selected retention rule. Rule Definition Dialog provides additional details.
 ic-refresh.pngRefreshes retention rule information.
ic-up.png Move Up

Moves the selected retention rule up in the Retention Rule priority list. Retention Rule order is very important. Security Analytics evaluates the the retention rules for all of the collections in numerical order by the number listed in the Order column in the Retention Rule section.

You can also use drag and drop to reorder retention rules. 

ic-down.png Move DownMoves the selected retention rule down in the Retention Rule priority list. Retention Rule order is very important. Security Analytics executes the the retention rules for all of the collections in numerical order by the number listed in the Order column in the Retention Rule section.
ApplySaves the rule order change.
ic-revert.png RevertReverts the rule order change.
ic-checkbox.png Selects or shows a selected retention rule.
OrderShows the order of a rule in the overall list of retention rules.
Rule NameShows the name of rule, such as ComplianceDevices and GeneralWindowsLogs.
ConditionShows the conditions for the rule. These conditions specify the type of logs to include in the collection. 
Rule and Query Guidelines presents the guidelines for all queries and rule conditions in Security Analytics Core services.
CollectionShows Collection name and how many days that the collection is retained. For example: MediumValue (30 Days)

 

 

Topics

You are here: References > Archiver: Data Retention Tab - Archiver

Attachments

    Outcomes