Log Collection: Troubleshoot

Document created by RSA Information Design and Development on Sep 11, 2017•Last modified by RSA Information Design and Development on Sep 12, 2018

Version 12Show Document

Like • Show 0 Likes0
Comment • 0
View in full screen mode

This topic describes the format and content of Log Collection Troubleshooting. NetWitness Platform informs you of Log Collector problems or potential problems in the following two ways.

Log files.
Health and Wellness Monitoring views.

Log Files

If you have an issue with a particular event source collection protocol, you can review debug logs to investigate this issue. Each event source has a Debug parameter that you can enable (set parameter to On or Verbose) to capture these logs.

Caution: Only enable debugging if you have a problem with this event source and you need to investigate this problem. If you have Debug enabled all the time it will adversely affect the performance of the Log Collector.

Health and Wellness Monitoring

Health and Wellness monitoring makes you aware of potential hardware and software problems in a timely manner so that you can avoid to outages. RSA recommends that you monitor the Log Collector statistical fields to make sure that the service is operating efficiently and is not at or near the maximum values you have configured. You can monitor the following statistics (Stats) described in the Admin > Health & Wellness view.

Sample Troubleshooting Format

RSA NetWitness Platform returns the following types of error messages in the log files for.

Log Messages	timestamp failure (LogCollection) Message-Broker Statistics:... timestamp failure (AMQPClientBaseLogCollection):... timestamp failure (MessageBrokerLogReceiver):...
Possible Cause	The Log Collector cannot reach the Message Broker because the Message Broker: has stopped running has erroneous connection settings
Solutions	<use the="the" systemctl="systemctl" command="command" on="on" console="console" to="to" check="check" status="status" of="of" message="message" broker="broker" shell="shell" console.="console.">returns the following if the message broker is not running:</use> prompt$ systemctl status rabbitmq-server rabbitmq start/running, process 10916 Start the RabbitMQ Message Broker on event-broker node in the Explore view:

Log Messages

timestamp failure (LogCollection) Message-Broker Statistics:...

timestamp failure (AMQPClientBaseLogCollection):...
timestamp failure (MessageBrokerLogReceiver):...

Possible Cause

The Log Collector cannot reach the Message Broker because the Message Broker:

has stopped running
has erroneous connection settings

Solutions

<use the="the" systemctl="systemctl" command="command" on="on" console="console" to="to" check="check" status="status" of="of" message="message" broker="broker" shell="shell" console.="console.">returns the following if the message broker is not running:</use>

prompt$ systemctl status rabbitmq-server

rabbitmq start/running, process 10916

Start the RabbitMQ Message Broker on event-broker node in the Explore view:

Troubleshooting - Windows log Collection using Endpoint Agent

The following topics helps troubleshoot issues you can come across while using windows log collection file on Endpoint Insights Agent.

Windows Log Configuration File Format Explained

Caution: Do not edit the generated configuration file. If any changes are made, the agent does not read the information from the file.

The log configuration file contains information helpful for analyzing event logs. Below is an example:

The generated config file contains the following:

Config name: Name of the configuration file.
Servers: Array of server URLs, describing both their address and protocol to use when forwarding the logs. The agent will attempt to contact them in order.
Filter: Windows Event viewer compatible XML which describes the channels to watch and any event ID exclusion. A standard XML filter to collect from channel Application and System, with one event ID excluded from both would look like this:

<QueryList>

<Query Id="0" Path="Application">

<Select Path="Application">*</Select>
<Select Path="System">*</Select>
<Suppress Path="Application">*[System[(EventID=3366)]]</Suppress>
<Suppress Path="System">*[System[(EventID=3366)]]</Suppress>

</Query>

</QueryList>
Enabled: Allows to disable collection but still send a test log if that is enabled.
TestLogOnLoad: Will send a log message when a configuration is loaded, even if event forwarding is not enabled. This helps Analysts test a configuration before enabling collection. This message is not logged locally in the Windows Event log.

Test Log - How to Read

Test log message is sent whenever an Endpoint Agent with windows log collection file is installed for the first time on an Endpoint Agent or when the log configuration file is updated. On a successful install or updation of windows log collection - There are 3 sections displayed in the test log file.

1	Test log message type, Agent's IP address, Agent's Hostname and time of generation of the test log
2	Configuration provided during the creation of the agent
3	Status and the message associated with it

There are three scenarios.

Successful deployment of a log collection configuration - Test Log message type will be -1 and status will be displayed as success.
Whenever the log collection configuration file is tampered with - The Agent Test message will be displayed as -2 and a message displaying the configuration file has been tampered with is displayed. In case, you want to reapply the changes, regenerate the log collection file.
When the custom channel name is wrong - Status Failure message is displayed. Regenerate the log collection with the correct channel.