Log Collection Deployment: Throttle Remote Collector to Local Collector Bandwidth

Document created by RSA Information Design and Development on Sep 11, 2017Last modified by RSA Information Design and Development on Oct 12, 2017
Version 7Show Document
  • View in full screen mode
  

To improve performance, you can throttle the bandwidth to control the rate that the Remote Collector sends event data to Local Collector or between Message Brokers. To do this, you configure the Linux kernel's filtering and IPTable functionality.

This works for both push and pull Remote Collector configurations. The set-shovel-transfer-limit.sh shell script located on the /opt/netwitness/bin automates the configuration of the kernel filter and iptables related to this port.

This topic describes how to throttle Remote Collector to Local Collector bandwidth using the set-shovel-transfer-limit.sh shell script. It contains the following sections:

  • The set-shovel-transfer-limit.sh shell script command line help.

    Note: The filter value that you need to set depends on the rate at which remote log collector is sending events to the Local Collector.

  • An example that sets the Filter to 4096 kilobits per second.

Command Line Help for Set Shovel Transfer Limit Script

Issue the –h command to display help for set-shovel-transfer-limit.sh shell script.

cd /opt/netwitness/bin
./set-shovel-transfer-limit.sh

Usage:

code>set-shovel-transfer-limit.sh -s|-c|-d|[-i interface] [-r rate]

where:

  • -c = clear existing
  • -d = display filter
  • -s = set new values
  • -i = interface is the name of the network interface. Default value is eth0
  • -r = rate is the bandwidth rate. Default value is 256kbps

Bandwidths and rates can be specified in:

  • nolimit: disables throttling
  • kbit: Kilobits per second
  • mbit: Megabits per second
  • kbps: Kilobytes per second
  • mbps: Megabytes per second
  • bps: Bytes per second

Set the Filter to 4096 Kilobits per Second

This example sets the Filter to 4096 kilobits per second.

[root@<hostname> bin]# ./set-shovel-transfer-limit.sh -s -r 4096kbit

RATE=4096kbit
PORTNUMBER=5671
DEVICE_INTERACE=eth0

iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: Saving firewall rules to /etc/sysconfig/iptables:[ OK ]

Current/new values...

iptables -t mangle -n -v -L
Chain PREROUTING (policy ACCEPT 2 packets, 161 bytes)
pkts bytes target  prot opt in  out  source          destination

Chain INPUT (policy ACCEPT 2 packets, 161 bytes)
pkts bytes target prot opt in out   source           destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out   source           destination

Chain OUTPUT (policy ACCEPT 2 packets, 248 bytes)
pkts bytes target prot opt in out   source           destination
     0    0 MARK   tcp -- *    eth0     0.0.0.0/0     0.0.0.0/0     multiport dports 5671 MARK set 0xa
     0    0 MARK   tcp -- *    eth0     0.0.0.0/0      0.0.0.0/0    multiport sports 5671 MARK set 0xa

Chain POSTROUTING (policy ACCEPT 2 packets, 248 bytes)
pkts bytes target prot opt in out   source           destination

tc -s -d class show dev eth0
class htb 1:1 root rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 7
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
lended: 0 borrowed: 0 giants: 0
tokens: 20000 ctokens: 20000

class htb 1:2 parent 1:1 prio 0 quantum 51200 rate 4096Kbit ceil 4096Kbit burst 1599b/8 mpu 0b overhead 0b cburst 1599b/8 mpu 0b overhead 0b level 0
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
lended: 0 borrowed: 0 giants: 0
tokens: 48828 ctokens: 48828

Next Topic:Set up a Lockbox
You are here
Table of Contents > Setup > Add Local and Remote Collectors > Throttle RC to LC Bandwidth

Attachments

    Outcomes