000017301 - Log Decoder getting unidentified content received on receiver from 127.0.0.1 in RSA NetWitness Platform

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support on Sep 26, 2019
Version 3Show Document
  • View in full screen mode

Article Content

Article Number000017301
Applies ToRSA Product Set: NetWitness Logs & Network
RSA Product/Service Type: Log Decoder
RSA Version/Condition: 10.6.x, 11.x
IssueLog Decoder getting unidentified content received on receiver from 127.0.0.1 in RSA NetWitness
The Log Decoder /var/log/messages file shows the following warning message:  Unidentified content received on receiver from 127.0.0.1.

The /var/log/messages file on the Log Decoder displays an error message similar to the examples below.




Jul 11 13:03:26 testLogDecoder nw[31771]: [SYSLOG] [warning] Unidentified content from 127.0.0.1 received on receiver: 'Investigate why the MySQL server is AVAILABLE.  Ensure that the MySQL server is running.'



May 15 01:20:43 testLogDecoder nw[7362]: [SYSLOG] [warning] Unidentified content from 127.0.0.1 received on receiver: 'bS8NCkFjY2VwdC1MYW5ndWFnZTogZW4tVVMNClVzZXItQWdlbnQ6IE1vemls'


Cause

When the Log Decoder finds that the message sent to it is not in syslog format (RFC-5424), it logs it as a warning with the malformed event displayed.



The text displayed as a base64 encoded string is likely because the malformed event contains non-printable characters.  The Log Decoder Base64-encodes the entire data, and logs it as a warning so that, while troubleshooting later, a programmer can retrieve the warning, Base64-decode the data and find out the actual log that was sent to Log Decoder.  For additional information on this process and to decode Base64 text, refer to http://www.base64decode.org/ or a similar website.



The following is an example of decoding a Base64 string:  'bS8NCkFjY2VwdC1MYW5ndWFnZTogZW4tVVMNClVzZXItQWdlbnQ6IE1vemls' = 'm/\r\nAccept-Language: en-US\r\nUser-Agent: Mozil'.



This example indicates that the Log Collector transformation is not working correctly for certain device messages with multiple lines.



rsyslogd mis-configured
Resolution

In order to resolve the issue, follow the recommendations below.
Ensure that the Log Collector is deployed with the latest transformation files.  This can be performed by following the steps below.

For 10.6.x:



  1. In the NetWitness UI, navigate to Live -> Search.
  2. Under Resource Types, select "RSA Log Collector" and click the Search button.
  3. Deploy the appropriate Device Type(s) to the Log Collector as needed.
For 11.x:

  1. In the NetWitness UI, navigate to CONFIGURE -> Live Content
  2. Under Resource Types, select "Log Collector" and click the Search button.
  3. Deploy the appropriate Device Type(s) to the Log Collector as needed.

NOTE: For some device types, the transform file is deployed on the Log Collector in the /etc/netwitness/ng/logcollection/content/transform directory.

Run the netstat command to check which services have connections to the TCP port 514 on the Log Decoder appliance, as shown below. 
(Expect to find only NwLogDecoder and NwLogCollector connections.)




netstat -anp |grep ':514 '
tcp        0      0 0.0.0.0:514                0.0.0.0:*                  LISTEN      25545/NwLogDecoder
tcp        0      0 10.3.28.196:46963          10.3.28.196:514            ESTABLISHED 1879/rsyslogd
tcp        0      0 10.3.28.196:514            10.3.27.127:51919          ESTABLISHED 25545/NwLogDecoder
tcp        0      0 10.3.28.196:514            10.3.28.196:46963          ESTABLISHED 25545/NwLogDecoder
tcp        0      0 127.0.0.1:35733            127.0.0.1:514              ESTABLISHED 25630/NwLogCollecto
tcp        0      0 10.3.28.196:514            10.3.27.127:50933          ESTABLISHED 25545/NwLogDecoder
tcp        0      0 127.0.0.1:514              127.0.0.1:35733            ESTABLISHED 25545/NwLogDecoder
tcp        0      0 :::514                     :::*                       LISTEN      25545/NwLogDecoder
udp        0      0 0.0.0.0:514                0.0.0.0:*                              25545/NwLogDecoder
udp        0      0 :::514                     :::*                                   25545/NwLogDecoder




In this example the rsyslogd connection is unexpected.

Check the /etc/rsyslog.conf to ensure that it is configured correctly.  See the lines below, which appear at the end of the rsyslog.conf file.




# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###



*.* @@10.3.28.196:514




NOTE:  When using "@@10.3.28.196:514" (double @@), this configures the rsyslogd to re-send syslog messages received by it to TCP/514.
Normally the rsyslogd re-send would be to a remote syslog server and not the Log Decoder appliance.
The TCP/514 on a Log Decoder appliance is usually used only for communication between the Log Collector service and the Log Decoder service.



To configure rsyslogd to send syslog messages to UDP/514, make the entry in rsyslog.conf as ?@10.3.28.196:514? (single @).



 



Disable the rsyslogd daemon from sending syslog messages to the Log Decoder. This can be done by editing the /etc/rsyslog.conf and commenting out the line that appears similar to the following:




# *.* @@10.3.28.196:514




After making the change and saving the file, restart the rsyslog service with the following commands:
For 10.6.x:
# service rsyslog stop
# service rsyslog start



For 11.x:
# systemctl stop rsyslog.service
# systemctl start rsyslog.service


If you are unsure of any of the steps above or experience any issues, contact RSA Support and quote this article ID for further assistance.

Notes

You may also attempt to capture more details about the multi-line message being passed from the Log Collector to the Log Decoder with a packet capture of syslog traffic, issuing the command below.




tcpdump -i lo -Al 'tcp port 514'


Legacy Article IDa67088

Attachments

    Outcomes