RSA Admin

Faster drilling by inclusion instead of exclusion

Discussion created by RSA Admin Employee on Oct 15, 2012

In our use of Netwitness we have always strived to make our queries as efficient as possible. It's not at all uncommon to spend a few weeks getting new users to understand that jumping in with a bunch of regex queries is NOT the best way to look for what you want.


One thing I found a few months ago was we could get a significant speed improvement on some queries (upwards of 10x faster) by simply avoiding the use of the "!=" operator. Here's a common query that many are familiar with:


    service = 80 && tcp.dstport != 80


which finds sessions that have been identified as HTTP traffic, but the destination TCP port is not 80. This can be a good starting point for finding suspicious/unapproved traffic. But it can take a long time to run if you have a lot of traffic to go through and/or slower gear.


By simply inversing the logic of what you want, the query can be made faster. What we are looking for is sessions where the traffic is HTTP (service=80) and the TCP destination port is not 80 (tcp.dstport != 80). That last part is the same as saying "all ports from 1 to 65535, except 80". So:


  service = 80 && tcp.dstport = l-79,81-u


will show you the same sessions, but should run significantly faster, since both metadata elements are indexed, and we're not asking the Netwitness gear to check every single session (a necessity for != operation). Note that if your environment happens to have a LOT more HTTP traffic on ports other than 80, this may not make as much of a speed improvement.

This same technique applies to any case where you want to check for traffic on non-standard ports, or for non-standard services on a specific port.



  I've used multiple ranges in the tcp.dstport of the query but there's definitely a dimishing of returns with the more commas you use.