This topic explains how Concentrators can be teamed into clusters that divide the work of aggregation between multiple appliances.
Note: Clustered aggregation does not apply to Brokers.
Clustered aggregation (also known as "gang") aggregation allows multiple concentrators to efficiently aggregate from a single Decoder. Any number of Concentrators can be grouped together to form an aggregation gang. The Concentrators in the gang divide all the sessions between themselves. Group or clustering aggregation allows multiple Concentrators to efficiently aggregate from a single Decoder in the type of architecture shown below:
Any number of Concentrators can be grouped together to form a cluster aggregation. The Concentrators in the cluster divide all the sessions between themselves.
In a two-concentrator gang, the sessions aggregated by each concentrator might look something like this:
|Concentrator 0||Concentrator 1|
|1 - 9,999||10,000 - 19,999|
|20,000 - 29,999||30,000 - 39,999|
|40,000 - 49,999||50,000 - 59,999|
Group Aggregation Parameters
You can configure the cluster as part of the Concentrator and Archiver configuration in the Security Analytics Services Config view. This table lists the parameters, which you can also configure in the Services Explore view.
|Parameter||Where Parameter is Set||Description|
|Aggregate Max Sessions||/concentrator/config/aggregate.sessions.max||The number of sessions that a Concentrator receives from a Concentrator at any given time. When a Concentrator is part of a gang, it is also used to determine how the Decoder's sessions will be divided among the gang. The Decoder's sessions are divided into pieces of size aggregate.sessions.max, and then the pieces are evenly distributed among the Concentrators. For example, if aggregate.sessions.max is 10,000, then sessions 1-9,999 go to the first Concentrator and sessions 10,000-19,999 go to the second concentrator. All concentrators in the gang must use the same value for aggregate.sessions.max.|
|conGangName||/concentrator/devices/<device>/config/options,gang=<gang name>||The gang name is used to determine the membership of a particular gang. There can be an unlimited number of gangs aggregating from a Decoder. The gang parameter is simply a mechanism that allows the decoder to identify which Concentrators are working together. All members of the gang must have the same gang name. If the gang name parameter is not set, gang aggregation is disabled. The gang name can be any string identifier.|
|Gang Size||/concentrator/devices/<device>/config/options,gangSize=<number of devices in gang>||This parameter sets the size of the gang. All Concentrators must have the same value for gang size. There is no limit on the size of the gang.|
|Gang Member ID||/concentrator/devices/<device>/config/options,gangMember=<id number of concentrator in gang>||This parameter sets the position of the concentrator in the group. For any gang of size N, gang member IDs from 0 to N-1 must be set on each of the gang members. For example, if the gang size is 2, one member gets gang member gang member ID 0, and the other gets gang member ID 1. If the gang size is 3, then the members are assigned IDs 0, 1, and 2.|
|Gang Membership Mode||/concentrator/devices/<device>/config/options,gangMembership=(new|replace)||This parameter determines how this concentrator catches up when aggregation is started for the first time. The default behavior is replace. This means that when this Concentrator starts aggregation, it is intended to replace an existing gang member, or all members of the gang are being initialized at the same time. This means that the Concentrator begins aggregation from the oldest session available on the appliance it is aggregating from. If this parameter is set to new, it means this Concentrator is being added as a new member of an existing group. This Concentrator will not try to aggregate any existing sessions from the device. The other members of the group have already aggregated all the sessions on the device. This Concentrator will only aggregate new sessions as they appear on the device.|
The following table shows sample parameters for a two-member gang with Gang ID foo.
|Parameter||Concentrator 0||Concentrator 1|
|device options||gang=foo gangSize=2|
The following table shows sample parameters for a three-member gang with Gang ID baz.
|Parameter||Concentrator 0||Concentrator 1|
|device options||gang=baz gangSize=3|
When the division of sessions for a cluster needs to change, either to increase or decrease the cluster size, you must toggle to an offline state. While offline, all members need to be updated to have the same gangSize parameter. The replacement appliance aggregates only its division of sessions. Likewise, any cluster member can be data-reset and re-aggregated at any time without affecting other members of the cluster.
A cluster member can be replaced at any time by setting the gangMember parameter for the replacement appliance to be the same as the gangMember parameter for the old appliance. The replacement appliance aggregates only its division of sessions. Likewise, any cluster member can be data-reset and re-aggregated at any time without affecting other members of the cluster.