000013238 - Unable to see certain expected sessions in RSA NetWitness Investigator

Document created by RSA Customer Support Employee on Jan 10, 2017Last modified by RSA Customer Support Employee on Apr 22, 2017
Version 2Show Document
  • View in full screen mode

Article Content

Article Number000013238
Applies ToRSA Netwitness NextGen
RSA NetWitness Decoder
RSA NetWitness Administrator
RSA NetWitness Investigator
IssueUnable to see certain expected sessions in RSA NetWitness Investigator.
You do not see certain expected sessions in NetWitness Investigator, but Investigator is showing other sessions.
ResolutionPlease follow these steps:

1.  Install tcpdump on your NetWitness Decoder.  Tcpdump is supported on all NetWitness Linux appliances, but is not installed by default.

2.  Does your capture interface or interfaces have a network link, and is it linked at the expected speed?  You can confirm this using the ethtool command, which is installed by default on NetWitness appliances.  Do not use mii-tool, as this will likely display incorrect information.
Example:
ethtool eth0

3.  Using tcpdump, confirm that the traffic in question is in fact being seen by the capture interface. 
Example:
tcpdump -nni any host <ip you're looking for> and port <port you're looking for>


4.  If you are seeing the expected traffic in step 3, continue to the following steps.  If not, you may have a misconfiguration in your network tap or SPAN configuration on your switch.  DO NOT continue to the next steps until you have confirmed that the traffic in question is being seen in tcpdump.

NOTE:  In any of the following steps, when generating test traffic, it can take several minutes for any generated meta to aggregate up to your Concentrator and/or Broker.  You can determine the most recent meta stored in your Concentrator's/Broker's index in Investigator by looking at the timestamp in the upper-right hand corner of the Investigator window after refreshing with F5.

5.  Check your Decoder's capture interface.  Are you capturing on the correct interface or interfaces?  You may need to select adapter ALL in NetWitness Administrator->Decoder Service->Stats Tab->Adapters and Rules Button->Adapter Tab.

6.  Make a copy of your Decoder's BPF filter, if applicable, NetWitness Administrator->Decoder Service->Stats Tab->Adapters and Rules Button->Adapter Tab.  In NetWitness Administrator, remove the BPF filter from the Decoder and stop/start capture.  Generate traffic that you are expecting to see.  Wait several minutes for the meta to aggregate to your Concentrator/Broker and check Investigator for the meta.  If the meta is present, your BPF filter was the problem.  Please correct the BPF filter and re-add it back to your Decoder's configuration, followed by stopping and starting capture.  See the knowledgebase article System-level (BPF) packet filtering best practices and examples for RSA NetWitness decoders for more information on constructing BPF filters.

One other method that can rule out your BPF filter as the problem is to use your filter statement in a tcpdump (enclosed in single-quotes) and pipe the output through grep to locate the traffic.  Piping the output through grep has the effect of isolating the BPF statement while still filtering for your traffic.
Example:
tcpdump -nni any 'not ( udp port 10000 || ip proto 50 || udp port 53 || arp || broadcast || multicast )' | grep 10.10.10.20

7.  Check your Decoder's network rules at NetWitness Administrator->Decoder Service->Stats Tab->Adapters and Rules Button->Net Rules Tab.  Take this opportunity to convert your network rules to BPF filters, which is always recommended due to performance and stability implications.  See the link in step 6 for more information.  Before making any changes, it is recommended that you backup all of your network rules to a file by shift-selecting all of the rules and clicking the Save selected Rules to a file button at the top of the Net Rules tab.  If you are not able to convert them now, temporarily disable your network rules, followed by stopping and starting capture.  If the expected meta now appears in Investigator after initiating test traffic and waiting several minutes, the problem is with your network rules.  You should now be able to isolate which network rule is causing the problem by enabling them one-by-one, restarting Decoder capture, and initiating test traffic. 

8.  If none of the above resolves your problem, the problem could be application rules on the Decoder that are filtering out the traffic in question.  Backup your application rules in NetWitness Administrator->Decoder Service->Stats Tab->Adapters and Rules Button->App Rules Tab.  Click twice on the Session Data column header.  This will place all filtering application rules together at the top.  Before making any changes, it is recommended that you backup all of your application rules to a file by shift-selecting all of the rules and clicking the Save selected Rules to a file button at the top of the App Rules tab.  Temporarily disable your application rules that contain the filter action (they should all be sorted together at the top now), followed by stopping and starting capture.  If the expected meta now appears in Investigator after initiating test traffic and waiting several minutes, the problem is with one of these filter-action application rules.  You should now be able to isolate which application rule is causing the problem by enabling the ones you've just disabled one-by-one, by enabling a single app rule, restarting Decoder capture, and initiating test traffic, reviewing Investigator, and repeating the process until the problematic rule is found.

9.  Is your decoder seeing a high rate of packet loss, as reported in the NetWitness Administrator stats tab?  This could also account for the session possibly not appearing in Investigator.  Refer to the knowledgebase article How to tune RSA NetWitness NextGen decoders for maximum capture performance and stability for tuning Decoder performance.

10.  If none of the above resolves your problem, check your decoder and/or concentrator logs for any relevant errors and/or warnings.
Legacy Article IDa58742

Attachments

    Outcomes