Endpoint: Troubleshooting

Document created by RSA Information Design and Development on Mar 16, 2018Last modified by RSA Information Design and Development on Sep 12, 2018
Version 6Show Document
  • View in full screen mode
 

This section provides information about possible issues when using the RSA NetWitness Endpoint Insights.

Agent Communication Issues

                 
Issue

Agent is unable to communicate with the Endpoint server.

Explanation

This could be due to one of the following reasons:

  • In the agent packager:

    • Server IP is incorrect
    • Port specified is not available for communication with the Endpoint server
  • Endpoint Server or Nginx Server is not running
  • Firewall or IP table rules are blocking the connection between the host and Endpoint Server
  • Agent is inactive or manually deleted from the UI
Resolution
  • Check if the Endpoint Server and Nginx Server are reachable
  • Uninstall the agent, reboot the host, and reinstall the agent
  • Update Firewall or IP table rules, if required
                 
Issue

Agent takes a long time to scan.

Explanation

Sometimes, the NetWitness Endpoint scan takes a long time to complete. This is because of the CPU usage by other antivirus programs (such as Windows Defender, Mcafee, Norton, and so on) that may be installed on the agent machines.

Resolution

It is recommended to whitelist the NWEAgent.exe file in the antivirus Windows Suite.

Packager Issues

                     
Message

Failed to load the client certificate.

Issue

Incorrect certificate password.

Explanation

While generating the agent installer, the certificate password does not match with the one provided while downloading the agent packager from the UI.

Resolution

Specify the correct certificate password.

 

                     
Message

An unexpected error has occurred attempting to retrieve this data.

Issue

When attempting to access the Packager tab, it opens with the message.

Explanation

Endpoint Server might be down or not reachable.

Resolution

Check the status of the Endpoint Server under Admin > Service. If the service is not running, start the Endpoint Server.

Scan Schedule Issues

                     
Message

An unexpected error has occurred attempting to retrieve this data.

Issue

When attempting to access the Scan Schedule tab, it opens with the message.

Explanation

Endpoint Server might be down or not reachable.

Resolution

Check the status of the Endpoint Server under Admin > Service. If the service is not running, start the Endpoint Server.

Health and Wellness Issues

                 
BehaviorEndpoint metadata is not available in the Investigate > Navigate or Event Analysis view.
Issue

The health check of the Meta-Ld-Buffer shows Unhealthy in the Health and Wellness with the following exceptions:

dataprocessor-5] WARN MetaManagement|Meta Forwarding waiting for free buffer in Log decoder

Resolution

Make sure that:

  • Capture is enabled on the Log Decoder
  • Metadata is configured properly

 

                 
Behavior

For the NetWitness Endpoint 4.4.0.2 or later, metadata is not reaching the Endpoint Server.

Issue

The health of the Meta-Ld-Buffer shows Unhealthy in the Health and Wellness with the following exceptions:

dataprocessor-5] WARN MetaManagement|Meta Forwarding waiting for free buffer in Log decoder

Explanation

Make sure that:

  • Certificate is obtained and imported to the NetWitness 4.4.0.2 or later Console Server
  • NetWitness Investigate option is enabled in the NetWitness Endpoint UI
  • Metadata forwarding is configured in the NetWitness 4.4.0.2 or later Console server

 

                     
Behavior

The health check of the Data.Application.Connection-Health for Endpoint Server shows Unhealthy.

Issue

Either Mongo or Endpoint Server service is down.

Explanation

For error details, check the Endpoint Server logs in /var/log/netwitness/endpoint-server/endpoint-server.log.

Resolution

Restart the Mongo or Endpoint Server service.

 

                     
Behavior

The health check of the Endpoint.Health.Overall-Health statistic shows Unhealthy.

Issue

Either Mongo or Endpoint Server service is down.

Explanation

Check the other Endpoint Server health statistics (such as, Data.Application.Connection-Health, Endpoint.Health.Ld-Buffer-Health) to identify which stats shows Unhealthy. If one of them is Unhealthy, the overall health of the Endpoint Server shows Unhealthy.

Resolution

See the resolution for these statistics in the Health and Wellness Issues section.

 

                 
Issue

Agent rejection count is more than the alarm threshold.

Explanation

The agent rejected count is more than a specific limit and your custom policy is triggered. For example, agent rejected count for the last 5 hours is 10 percent of the deployed agents.

Resolution

Check the overall health of the Endpoint Server and the sizing guidelines.

 

                 
Issue

Storage size of the Data application statistic has exceeded the alarm threshold.

Explanation

The storage size of the Data application has exceeded the threshold (for example, 75%), and the custom policy is triggered.

Note: By default, the server automatically deletes the older data when it reaches 80% of the disk space.

Resolution

Check the threshold set in the data retention policy.

 

                 
Issue

The health check of the Data.Application.Connection-Health shows Unhealthy or Fatal.

Explanation

The Mongo service is down.

Resolution

Check if the Mongo service is running and the Endpoint Server logs for error details.

 

                 
Issue

The agent request count shows 0 for a alarm threshold.

Explanation

The agent request count shows 0 for the entire day or week. This could be due to one of the following reasons:

  • In the agent packager:

    • Server IP is incorrect
    • Port specified is not available for communication with the Endpoint server
  • Endpoint Server or Nginx Server is not running
  • Firewall or IP table rules are blocking the connection between the host and Endpoint Server
  • Agent is inactive or manually deleted from the UI
Resolution
  • Check if the Endpoint Server and Nginx Server are reachable
  • Uninstall the agent, reboot the host, and reinstall the agent
  • Update Firewall or IP table rules, if required

Installation Issue

                     
Behavior

NetWitness Platform allows multiple instances of Endpoint Hybrid or Endpoint Log Hybrid to be installed.

Issue

Only one instance of the Endpoint Hybrid or Endpoint Log Hybrid can be used for endpoint data.

Explanation

While the installation of Endpoint Hybrid or Endpoint Log Hybrid is in-progress, you can install another instance and the installation will be successful.

Resolution

You must delete all instances of Endpoint Hybrid or Endpoint Log Hybrid except the one that you want to use for endpoint data.

Finding Inactive Agents Issue

                 
Issue

Agent might be inactive or has not communicated with the Endpoint Server for a long time.

Explanation

A list of inactive agents is available in the Mongo database with the agent ID. Using this information, you can search for further details of the inactive agents.

Resolution

To find inactive agents in your deployment, perform the following:

  1. Open the Endpoint Server log file from /var/log/netwitness/endpoint-server/endpoint-server.log and search for Agent <ID> does not exist string.
  2. Copy the agent ID displayed in the log file.
  3. Search for the agent ID in the NGINX access log file (/var/log/nginx/access.log) to retrieve the following details of an inactive agent:

    • IP Address
    • Date and time that the agent became inactive
    • Location
Previous Topic:Packager Tab
You are here
Table of Contents > Troubleshooting

Attachments

    Outcomes