000034655 - How to Extract RDQ files generated in RSA Log Collector/ Virtual Log Collector in Readable Format

Document created by RSA Customer Support Employee on Jan 19, 2017Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 3Show Document
  • View in full screen mode

Article Content

Article Number000034655
Applies To
RSA Product Set: Netwitness
   RSA Product/Service Type: Log Collector
   RSA Version/Condition:10.4 and above 
   Platform: CentOS
TasksMake sure to have nwlogcollectortools rpm installed in Log Collector. You can verify the same using:-
#rpm -qa | grep -i nwlog
If it not installed already then please run below command:-
#yum install nwlogcollectortools
Resolution

NwEventReader


The NwEventReader tool can be used to read events stored in protobuf format, either either as streamed to the disk from the Log Collector (using the NGCP protocol), as stored on disk in persistent format as a result of stopping collection, or as pulled from the Message Broker using the NwAMPQReceiver tool.

Syntax:


--help                                                 - print this help message
--b64Format arg (=0)                        - Base 64 format output
--file arg                                            - File to dump. This may be a file captured from streamed (NGCP) output from the Log Collector, or messages captured via the AMQPReceiver tool.
--printEvents arg (=1)                        - Print events
--verbose arg (=0)                             -  Verbose output
--maxFileSize arg (=4294967295)     - Max File Size
 

Options:


--help
Display help.
--b64Format=<bool>
Interpret the input data as line-separated base-64 encoded event protobuf structures, as stored on disk as the result of collection shutdown by the in memory queue. This option is typically only used by development to analyze the contents of event data persisted between shutdown events.
--file=<path>
The path from which to read events. If the files end in ".ngce" (as created via the NwAMQPReceiver tool), this command will assume the files are extracted from the Message Broker. Otherwise, this tool will interpret the event data as event protobuf data sent over the NGCP protocol (e.g., in the case of Content 3 export from the Log Collector.), unless the --b64Format is set to true (see above).
--printEvents=<bool>
Boolean flag indicating whether to print event data to the console.
--verbose=<bool>
Sets verbose output, which may be useful in some cases.
--maxFileSize=<int>
The maximum size of the file to read (0 denotes unlimited).
For Instance:-
1) Execute below command:- 2)  
#NwEventReader --file /tmp/1479103918423-00000001.ngce --printEvents=1
2) Output would be similar to:-
[root@RLC bin]# NwEventReader --file /tmp/1479103918423-00000001.ngce --printEvents=1

NGCE Version: 1.0
Message Header:
"ngce.compression_algorithm" : "1"


Number of events: 1


======================================================
Event: 0:
Event:
collection_meta:
"lc.lpid" : "odbc.epolicyvirus4_5"
"lc.cid" : "PDMVIVLC"
"lc.msgtype" : "1"
"lc.srcid" : "10.60.151.25"
"lc.ctype" : "odbc"
"lc.ctime" : "1479103800394"
"lc.wuid" : "17562157925649023279"
"lc.esname" : "ePO"
"lc.estype" : "epolicyvirus4_5"
"prefix_tag" : "ePolicy"
"field_delimiter" : "^^"
"lc.wusn" : "0"
"level" : "6"
"message_id" : "1203"


content_meta:
"AutoID" : "270236055"
"ServerID" : "PDMVIEPO"
"ReceivedUTC" : "2016-11-14 06:09:59.347"
"DetectedUTC" : "2016-11-14 05:09:22.000"
"Analyzer" : "VIRUSCAN8800"
"AnalyzerName" : "VirusScan Enterprise"
"AnalyzerVersion" : "8.8"
"AnalyzerHostName" : "BKP-SARATHI"
"AnalyzerIPV4" : "10.60.151.25"
"AnalyzerDATVersion" : "8346.0000"
"AnalyzerEngineVersion" : "5800.7501"
"AnalyzerDetectionMethod" : "(managed) Daily Target Folder Scan for Program Files_Laptop"
"SourceHostName" : "(null)"
"SourceIPV4" : "10.60.151.25"
"TargetHostName" : "BKP-SARATHI"
"TargetIPV4" : "10.60.151.25"
"TargetUserName" : "SYSTEM"
"TargetPort" : "(null)"
"TargetProtocol" : "(null)"
"TargetProcessName" : "(null)"
"TargetFileName" : "(null)"
"ThreatCategory" : "ops.task.end"
"ThreatEventID" : "1203"
"ThreatSeverity" : "6"
"ThreatName" : "none"
"ThreatType" : "none"
"ThreatActionTaken" : "none"
"ThreatHandled" : "1"


raw_message: nil


Summary:
========
Read 1 events in 790 bytes.
compression ratio (u/c): 1.67266


 
Notes 

Attachments

    Outcomes