Log Collection Deployment: Throttle Remote Collector to Local Collector Bandwidth

Document created by RSA Information Design and Development on May 9, 2016Last modified by RSA Information Design and Development on May 4, 2017
Version 6Show Document
  • View in full screen mode
  

To improve performance, you can throttle the bandwidth to control the rate that the Remote Collector sends event data to Local Collector or between Message Brokers. To do this, you configure the Linux kernel's filtering and IpTable functionality.

This works for both push and pull Remote Collector configurations. The set-shovel-transfer-limit.sh shell script located on the /opt/netwitness/bin automates the configuration of the kernel filter and iptables related to this port.

Context

After reading this topic, you know how to throttle Remote Collector to Local Collector bandwidth using the set-shovel-transfer-limit.sh shell script by reviewing:

  • The set-shovel-transfer-limit.sh shell script command line help.

Note: The filter value that you need to set depends on the rate at which remote log collector is sending events to the Local Collector.

  • An example that sets the Filter to 4096 kilobits per second.

Return to Procedures

set-shovel-transfer-limit.sh Command Line Help

Issue the –h command to display help for set-shovel-transfer-limit.sh shell script.

cd /opt/netwitness/bin
./set-shovel-transfer-limit.sh

Usage: set-shovel-transfer-limit.sh -s|-c|-d|[-i interface] [-r rate]

where:

-c = clear existing

-d = display filter

-s = set new values

-i = interface is the name of the network interface. default=eth0

-r = rate is the bandwidth rate. default=256kbps
       Bandwidths or rates can be specified in:
          nolimit = disables throttling
          kbit    = Kilobits per second
          mbit    = Megabits per second
          kbps    = Kilobytes per second
          mbps    = Megabytes per second
          bps     = Bytes per second  

Set the Filter to 4096 Kilobits per Second

[root@<hostname> bin]# ./set-shovel-transfer-limit.sh -s -r 4096kbit

RATE=4096kbit
PORTNUMBER=5671
DEVICE_INTERACE=eth0

iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: Saving firewall rules to /etc/sysconfig/iptables:[ OK ]

Current/new values...

iptables -t mangle -n -v -L
Chain PREROUTING (policy ACCEPT 2 packets, 161 bytes)
pkts bytes target  prot opt in  out  source          destination

Chain INPUT (policy ACCEPT 2 packets, 161 bytes)
pkts bytes target prot opt in out   source           destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out   source           destination

Chain OUTPUT (policy ACCEPT 2 packets, 248 bytes)
pkts bytes target prot opt in out   source           destination
     0    0 MARK   tcp -- *    eth0     0.0.0.0/0     0.0.0.0/0     multiport dports 5671 MARK set 0xa
     0    0 MARK   tcp -- *    eth0     0.0.0.0/0      0.0.0.0/0    multiport sports 5671 MARK set 0xa

Chain POSTROUTING (policy ACCEPT 2 packets, 248 bytes)
pkts bytes target prot opt in out   source           destination

tc -s -d class show dev eth0
class htb 1:1 root rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 7
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
lended: 0 borrowed: 0 giants: 0
tokens: 20000 ctokens: 20000

class htb 1:2 parent 1:1 prio 0 quantum 51200 rate 4096Kbit ceil 4096Kbit burst 1599b/8 mpu 0b overhead 0b cburst 1599b/8 mpu 0b overhead 0b level 0
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
lended: 0 borrowed: 0 giants: 0
tokens: 48828 ctokens: 48828

You are here
Table of Contents > Log Collection Deployment Guide > Procedures > Throttle Remote Collector to Local Collector Bandwidth

Attachments

    Outcomes