How can we benchmark parser performance
Support has a list of parsers which have performance issue, how they know? Experience?
We're going to develop some custom parsers for customer and customer wants to know the performance impact, any idea how can we check?
- Community Thread
- Forum Thread
- RSA NetWitness
- RSA NetWitness Platform
Best you can do is check the stats for a decoder via NwAdmin or use a monitoring tool such as Nagios, Zabbix, etc. to pull the stats via SNMP. I personally recommend the latter technique because it's easy to get rolling graphs of stats.
You definitely want to keep a close eye on dropped packets from capture as that's your real enemy. But you'd probably want to look at CPU and memory usage as well, to make sure they aren't running away.
Unfortunately, there's no real way to measure performance of an individual parser.
Generally, yes experience. There were a few flex parsers which could perform poorly. SMB-flex was notorious, due to the fact that it exercised everything flex was bad at. Lua parsers generally haven't shown any performance issues - with a couple exceptions that have already been addressed by updates. SMB_lua is significantly faster than SMB-flex was, no one should be afraid of it.
But some parsers sometimes just have to do a lot of work. So they aren't problematic so much as requiring a trade off between performance and visibility.
There isn't a way to know, "this parser is occupying these resources". Best practice is to make the parser do as little work as possible - which admittedly is more art than science.
Quick tips (all of these are covered in more depth in "The Book"):
- Use long, unique tokens. Each time a token is matched, the parser is run. Even if it does a little bit of work to decide that "this match" isn't relevant to the parser then bails, that's more work than the decoder needed to do. Choose tokens that are more likely to match only when relevant.
- Minimize the use of payload:find(). The less find-ing a parser has to do, the better.
- Limit the distance searched by payload:find(). The less payload the parser has to search, the better.
- Use small payload objects, rather than the entire stream.
- Localize variables and functions when it makes sense to do so.
- Avoid string manipulation when possible, especially appending - use table.concat() instead.
As for what to watch for, just as Jeff said,
- Dropped packets - classic sign of an unhappy decoder
- Memory usage
Also be sure to watch your logs. I'd also suggest if possible to have an independent non-production decoder dedicated to content development and testing. Import pcaps instead of capturing live traffic so that you have a known, consistent, repeatable set of test data. Import, make changes, import again - repeat until ready to deploy the content for production use.
Yes, having a dedicated test decoder/concentrator is invaluable, although expensive and may be hard to justify for some. I'd definitely love to see RSA offer some sort of virtual solution for this sort of scenario, where testing of customizations and configurations could be done safely without having to buy very expensive equipment.
Oh and one last thing about benchmarking parsers, definitely read William Motley's parser book! If you're not following that as your parser bible you're bound to stab yourself in the foot someday. :}