The tabs for a Warehouse Connector in the Services Config view provide a way to manage basic service configuration, configure sources, configure destination, and configure streams.
The following are the tabs in the Warehouse Connector Services Config view:
- Sources and Destinations
- Lockbox Settings
- Appliance Service Configuration
Sources and Destinations
The Sources and Destinations tab contains the following two sections:
- Source Configuration
- Destination Configuration
The Source Configuration section allows you to configure the data sources form which the Warehouse Connector service needs to collect data.
The following is an example of the Source Configuration section.
The Source Configuration section allows you to perform the following.
The Destination Configuration section allows you to configure the destination to which the Warehouse Connector service needs to write the collected data.
The Destination Configuration section allows you to perform the following.
|Add the destination.|
|Delete the destination.|
| Edit the destination. |
Note: You can only edit the SFTP destination type.
The Streams tab allows you to perform all of the tasks related to a stream.
The Streams tab allows you to perform the following.
The Streams tab also lists all the created streams in a tabular format.
|Name||Name of the stream.|
|Sources||The sources associated with the stream.|
|Destination||The destinations associated with the stream.|
|Status||Status of the stream.|
The following parameters are displayed in the Stream Statistics.
|Number of files in the permanent rejected folder||Determines the number of files in the permanent rejected folder (named, permfail) in the Warehouse Connector. The permanent rejected folder contains the files that Warehouse Connector failed to write to the destination.|
|Rejected folder usage in percentage||Determines the disk usage of the rejected folder.|
|Session Rate||Determines the rate at which the session is processed by the Warehouse Connector for the source.|
|Status||Indicates the status of the destination.|
|Status||Indicate the status of the source.|
|Sessions Behind||Determines that number of sessions that needs to be processed by the Warehouse Connector.|
|Last Session read||Determines the last session id processed by the Warehouse Connector.|
Change Stream Configuration
You can change configuration of a stream in runtime. In the Streams tab, click Change Configuration to change the configuration of the selected stream.
You can change the following parameters of the Stream Configuration:
Note: If you change the value of any parameter in stream configuration, make sure that you restart the stream.
After upgrading, if the values of Maximum Message Hold Count, Maximum Message Hold Interval and Maximum Message Hold Size are 3000000, 60 and 128 respectively, ensure that you assign the following values to the streams:
Maximum Message Hold Count - 2400000
Maximum Message Hold Interval - 600
Maximum Message Hold Size - 512
|Aggregate max sessions||Determines the maximum number of sessions in a response for an aggregation request from the Warehouse Connector to the source .|
|Aggregation Interval||Determines the time between the responses from the source.|
|Compress files on disk||Enable to compress files on disk.|
|Export Rollup Interval|| |
Determines the roll-up interval for export files and also the directory structure the Warehouse Connector writes to the destination.
For example, iIf the parameter is set to:
If you change the value of the parameter, ensure that you restart the stream.
Recommended value is hour.
|Maximum Message Hold Count||Determines the maximum number of sessions to store in the memory before processing. |
Note: If you have deployed a Warehouse Connector Virtual Appliance, make sure that you change the default value of the parameter to 800000.
|Maximum Message Hold Interval (Seconds)||Determines the maximum time (in seconds) to hold the sessions in memory before processing.|
|Maximum Message Hold Size||Determines the maximum size for the sessions to store in the memory before processing.|
|Remote Export Path||Determines the remote local mount point for HDFS (nfs://) and the location to export the data.|
|Stream Settings||Page Size|
|Auto Startup||Enable to automatically start the stream whenever the Warehouse connector process is restarted. By default, the parameter is set to off.|
The Lockbox Settings tab allows you can set, change, or refresh the lockbox password of the Warehouse Connector.
The General tab displays the system configuration parameters for the Warehouse Connector service.
When you add a Warehouse Connector service, default values are in effect. RSA designed the default values to accommodate most environments and recommends that you do not edit these values because it may adversely affect performance. The following table describes the System Configuration parameters.
|Compression||Determines the minimum amount of bytes before a message is compressed. If set to zero, messages are not compressed.|
|Port||Determines the port used by the service. |
Note: If you change the port number, ensure that you restart the service.
|SSL||If enabled, all the data transferred in the network will be encrypted using SSL.|
|Stat Update Interval||Determines how often (in milliseconds) statistic nodes are updated in the system.|
|Threads||Determines the number of threads in the thread pool to handle incoming requests.|
Appliance Service Configuration
The Appliance Service Configuration tab displays the appliance configuration settings for Warehouse Connector service. For more information, see Appliance Service Configuration in the Hosts and Services Getting Started Guide.