The following NwConsole commands are useful when interacting with Security Analytics Core services:
- feed: Enables you to create and work with feed files.
- makepcap: Converts Packet database (DB) files to PCAP.
- packets: Retrieves packets or logs from the logged in service.
- hash: Creates or verifies hashes of database files.
The following sections as well as the NwConsole help and topic information (man) pages, provide additional information.
feed command provides several utilities for creating and examining feed files. A feed file contains the definition and data of a single feed in a format that has been precompiled for efficient loading by a Decoder or Log Decoder. For a complete reference on feed definitions, see Feed Definitions File in the Decoder and Log Decoder Configuration Guide.
feed create <definitionfile> [-x <password>]
feed create command generates feed files for each feed defined in a feed definition file. A definition file is an XML document that contains one or more definitions. Each feed definition specifies a data file and the structure of that data file. The resulting feed files will be created in the same directory as the definition file with the same name as the data file, but with the extension changed to .feed (for example, datafile.csv results in datafile.feed). Any existing files with the target name will be overwritten without a prompt.
RSA Security Analytics Console 10.5.0.0.0
Copyright 2001-2015, RSA Security Inc. All Rights Reserved.
Type "help" for a list of commands or "man" for a list of manual pages.
> feed create example-definition.xml
Creating feed Example Feed...
done. 2 entries, 0 invalid records
All feeds complete.
example-definition.xml example-data.feed example-data.csv
Optionally, feed files can be obfuscated using the option
-x followed by a password of at least 16 characters (no spaces). This will be applied to all feeds defined in the definition file. In addition to the feed file, a token file will be generated for each feed file. The token file must be deployed with the corresponding feed file.
feed create example-definition.xml -x 0123456789abcdef
feed stats <feedfile>
feed stats command provides summary information for an existing, un-obfuscated feed file. Specifying an obfuscated feed file will result in an error.
> feed stats example.feed
Example Feed stats:
version : 0
keys count : 1
values count: 2
record count: 2
meta key : ip.src/ip.dst
feed dump <feedfile> <outfile>
feed dump command generates a normalized, key-value pair listing of an un-obfuscated feed file. You can use the resulting file to validate a feed file or assist in determining which records were considered invalid when the feed was created. Specifying an obfuscated feed file will result in an error. If
outfile exists, the command will abort without overwriting the existing file.
feed dump example.feed example-dump.txt
Converting Packet DB Files to PCAP
You can use the
makepcap command to quickly convert any Packet DB file to a generic PCAP file, preserving the capture time order. This command offers many options (see
help makepcap), but is easy to use. All it really needs is the Packet DB directory (via the
source=<pathname> parameter) to get started.
This command converts every Packet DB file into a corresponding PCAP file in the same directory. If the disk is almost full, see the next command.
makepcap source=/var/netwitness/decoder/packetdb dest=/media/usb/sde1
This command writes all of the output PCAPs to the directory at /media/usb/sde1.
makepcap source=/var/netwitness/decoder/packetdb dest=/media/usb/sde1 filenum=4-6
This command only converts the files numbered 4 thru 6 and skips all other files. In other words, it converts the Packet DB files: packet-000000004.nwpdb, packet-000000005.nwpdb, and packet-000000006.nwpdb.
makepcap source=/var/netwitness/decoder/packetdb time1="2015-03-01 14:00:00" time2="2015-03-02 07:30:00" fileType=pcapng
This command only extracts packets with a timestamp between March 1st, 2015 at 2 PM and March 2nd, 2015 before or on 7:30 AM. It writes the file as pcapng in the same directory as the source. All timestamps are UTC.
You can use the
packets command to generate a PCAP or log file based on a list of Session IDs, a time period, or a where clause. This command is very flexible you can use it on any running service that has access to the raw data from a downstream component. Before running the command, you must first
login to a service and then change directory to the appropriate sdk node (for example,
cd /sdk). Unlike the
makepcap command, which only works on the local file system, you use this command for a remote service.
packets where="service=80 && time='2015-03-01 15:00:00'-'2015-03-01 15:10:00'" pathname="/tmp/march-1.pcap"
This command writes 10 minutes of HTTP only packets from March 1st to the file /tmp/march-1.pcap. All times are in UTC.
packets time1="2015-04-01 12:30:00" time2="2015-04-01 12:35:00" pathname=/media/sdd1/packets.pcap.gz
This command writes all packets between the two times to a GZIP compressed file at /media/sdd1/packets.pcap.gz.
packets time1="2015-04-01 12:30:00" time2="2015-04-01 12:35:00" pathname=/media/sdd1/mylogs.log
This command writes all logs between the two times to a plaintext file at /media/sdd1/mylogs.log. Any pathname ending with .log indicates that the format of the output file should be plaintext line-delimited logs.
Verifying Database Hashes
By default, Archiver writes an XML file for every DB file that is written. This XML file ends with the extension .hash and contains a hash of the file along with other pertinent information. You can use the
hash command to verify that the DB file has not been tampered with by reading the hash stored in the XML file and then rehashing the DB file to verify that the hash is valid.
hash op=verify hashfile=/var/netwitness/archiver/database0/alldata/packetdb/packet-000004880.nwpdb.hash
This command verifies that the Packet DB file packet-000004880.nwpdb still matches the hash in the XML file packet-000004880.nwpdb.hash. For proper security, the hash file should be stored somewhere else to prevent the XML file from being tampered with (like write once only media), but the hash command itself does not care where it is stored.