Investigate: Examining ing Raw Events and Meta Data in the Event Analysis View

Document created by RSA Information Design and Development on Sep 18, 2017Last modified by RSA Information Design and Development on Apr 3, 2018
Version 14Show Document
  • View in full screen mode
 

Select the Event Analysis Type

To select the event analysis type for an event, do one of the following:

  1. In the Event Analysis view toolbar, click the analysis type in the toolbar.
  2. In the drop-down menu, select the analysis type:, , File Analysis Text Analysis, Packet Analysis, Email (Version 11.1), or Web (Version 11.1).
    If you chose, File Analysis Text Analysis, or Packet Analysis, the view is refreshed with the Packet Analysis panel, File Analysis panel, or Text Analysis panel open.
    If you chose Email or Web, the Events view email or web recontruction of the single opens in a new tab. This is the same reconstruction of an email or web session used in the Events view.

Note: The Packet Analysis panel is only available for network events.

Open, Close, and Adjust the Size of the Panels in the Event Analysis View

The Event Analysis view opens with the event list on the left, and the Network Details, Log Details, or Endpoint Details panel opens on the right. You can click an event in the event list to view a different reconstruction. Initially, the Network Details, Log Details, or Endpoint Details panel occupies 75% of the window width by default.

 

You can adjust the size ratio of the two panels to improve readability by expanding one of the panels, contracting one of the panels, and closing one of the panels. After closing either panel you can reopen it. The ratio you select persists until you change it or refresh the browser.

  • To reopen the Events panel, click the open events panel icon in the upper right corner.

To optimize your view:

  1. To adjust the size ratio of the two panels, do any of the following:
    1. Click expand panel icon in the tool bar of the panel that you want to expand.
    2. Click the reduce panel icon in the tool bar of the panel that you want to contract.
  2. To close either panel, restoring the open panel to its full width, click the close icon.
    This is an example of the reconstruction displayed using the full width of the browser window.
    Packet Analysis displayed in the full width of the browser window
  3. To reopen the Events panel after closing, click the open events panel icon in the top right corner of the Navigate view.
    The Events panel opens to the last state (25%:75% or 50%:50%).
  4. To reopen the Event Details panel, click an event in the Events panel.

Adjust the Display of Requests and Responses

For Event types that have requests and responses in them, you can make several adjustments.

Note: If the analysis type does not have requests and responses, the option is not selectable. The File Analysis panel is an example of a reconstruction type without requests and responses. A reconstructed log event in the Text View is another example.

To select which side of the conversation to show--Request, Response, or both--click one or both of the direction icons.Request and Response icons. The reconstruction is refreshed with the selected information.

Note: If you do not see any data, you may have deselected both Request and Response. You must select one of the two to see data displayed.

View Event Metadata for an Event

When examining events in the Text Analysis panel, Packet Analysis panel, or File Analysis panel, you can click the show Event Meta panel icon to show the associated metadata in an adjacent panel, the Event Meta panel.

When viewing Text Analysis and the Event Meta panel, hovering over the meta key/meta value pairs reveals a pair of binoculars if the meta value is searchable in the raw text. This is an example of the binoculars icon when hovering over the Directory and / meta key/meta value pair.

binoculars icon in the Event Meta panel

Clicking on the icon triggers a search for the meta key/meta value pair (case-insensitive) in the Text Analysis panel and each instance is highlighted. In the Event Meta panel, the highlighted row has a count of the results and a scroller that you can use to quickly find each result in the Text Analysis panel. You can view each highlighted location of the data that triggered generation of the meta key, going forward to view the next, and back to view the previous.

Only meta keys that have relevant values inside the RAW text are searchable. You can search only one meta key at a time. If the value is currently hidden due to truncation of a text entry with more than 3000 characters, the text entry is expanded to reveal the found meta value.

Clicking on the same meta key/meta value pair or a different meta key:value pair in the Event Meta panel removes the highlighting from the raw text. The highlighting is also removed if you close the Event Meta panel.

To search the raw text for meta values that triggered a meta key:

  1. Open a network event in the Text Analysis panel.
    Text Analysis panel
  2. In the toolbar, click the Open Meta Panel icon to open the Event Meta panel. As you hover over the meta key:value pairs in the list, a binoculars icon identifies values that are searchable in the Text Analysis panel.
  3. To search for the value in the raw text, click a row that has the binoculars icon, indicating it is searchable.
    If no relevant occurrence of the value is in the text, the value that you are searching for is highlighted in the Event Meta panel and nothing is highlighted in the Text Analysis panel.
    Text Analysis panel with event meta data highlighted
    If one or more relevant instances of the value is found in the Text Analysis panel, each occurrence is highlighted. The value that you are searching for is highlighted in the Event Meta panel and the scroller is visible.
    Text Analysis panel with the Scroller visible in the Meta panel
  4. To remove the highlighting, close the Event Meta panel, click the same meta key/meta value pair in the Event Meta panel, or click a different meta key/meta value pair in the Event Meta panel.
    The highlighting is removed from the raw text.

Show or Hide the Event Header

To hide the Event Header in the Packet Analysis panel, Text Analysis panel, or File Analysis panel, providing more vertical space for the data, click the Display Header icon.

Expand Truncated Text Entries in the Text Analysis Panel

A reconstruction of a network event in the Text Analysis panel may include requests and responses of many hundred thousands of characters and scrolling through a long entry of more than 6000 characters that is not of interest can waste time. To improve the experience for analysts, all text entries that have more than 6000 characters are truncated to show only the first 2000 characters. This example shows an entry that has more than 2000 characters and a message in the header indicates the percentage of total characters that is being displayed.

Text Analysis with truncated entries.

You can see that 36% of the characters (the first 2000) are displayed, and click Show Remaining 64% to reveal the rest of the entry.

Text Analysis with truncate entries expanded

If you search for meta data seen in the Event Meta panel while text is truncated in the Text Analysis panel, the truncated text is searched. If the meta data exists inside hidden text, the text entry expands to reveal the text with the found meta data.

Perform URL and Base64 Encoding and Decoding in the Text Analysis Panel

If a network session being reconstructed in the Text Analysis panel contains Base64 or URL encoded strings, you can decode a string to better understand the session. If the session contains decoded strings for Base64 or URL, you can view a string in its encoded form in order to search for additional instances of the encoded text in other sessions.

When viewing any network session that contains encoded text in the Text Analysis panel, you can select a subset of the text within a single Request or Response to view in either encoded or decoded form. Depending on the content loaded on the Decoder, there may be additional metadata outlining that Base64 or URL encoded data is contained within the session.

Below are examples of a hover box that is displaying URL encoding and Base 64 encoded text.

Text Analysis displaying encoded text

Text Analysis displaying decoded text

To perform encoding and decoding in the Text Analysis panel:

  1. In the Event Analysis view, go to the Text Analysis panel of a session that contains encoded or decoded content.
  2. To view some decoded text in encoded form, drag to select the text within a single Request or Response.
    A menu offers options to encode and decode.
    the popup menu for decoding and encoding text
  3. Click Encode Selected Text.
    The encoded text is displayed in a hover box, which remains in place until you click the the close icon, select different text in the Text Analysis panel, close the Events panel, select another event for reconstruction, or switch to a different reconstruction view.
    an encoded URL
    When a longer text is selected, the hover box is scrollable and large enough to fit the entire selected text as well as the decoded text.
  4. If the session contains encoded text that you want to see in decoded form, drag to select the text within a single Request or Response.
    A menu offers options to encode and decode.
  5. Click Decode Selected Text.
    The decoded text is displayed in a hover box, which remains in place until you click the close icon, select different text in the Text Analysis panel, close the Events panel, select another event for reconstruction, or switch to a different reconstruction view.
  6. If you want to copy some text from the text reconstruction do one of the following:
    1. Drag to select some text, right-click, and select Copy Selected Text from the popup menu.
    2. Drag to select some text, then select either Decode Selected Text or Encode Selected Text. Within the popup, select the desired text and type Control-C.
      The selected text is copied to the clipboard and available to paste in a query.
  7. When finished, click the Close icon to close the hover box.

View Decompressed Text in an HTTP Network Session in the Text Analysis Panel

When the content of an HTTP network session is compressed and you are viewing the Text Analysis panel, NetWitness Suite displays decompressed content by default. This helps you to determine if there are any patterns and view the readable characters. You can switch between a compressed and decompressed view of compressed text.

Note: Decompressed text is not available for the Packet Analysis panel, the File Analysis panel, non-HTTP network sessions, and log data.

The toggle for changing between compressed and decompressed text is only displayed in the Text Analysis panel, and is enabled only if there is compressed text content.

  1. Open the Text Analysis panel of an HTTP session that contains compressed content.
    By default the session is reconstructed with the text decompressed, and above the reconstruction, is the Display Compressed Payloads toggle switch.
    a decompressed payload
  2. To view the same text in its compressed form, click the toggle switch.
    The view changes so that the compressed text is no longer readable, and the switch indicates the Display Compressed Packets is on.
    a compressed payload
  3. To return to the view of decompressed text, click the switch again.

Download a Log in the Text Analysis Panel

When viewing a log reconstruction in the Text Analysis panel, you can download a log file in the following formats using options in the Download Log drop-down menu:

  • Raw log (log) using the Download Log option
  • Comma-separated values (CSV) using the Download CSV option
  • Extensible Markup Language (XML) using the Download XML option
  • JavaScript Object Notation (JSON) using the Download JSON option

Note: If you initiate a download and move away from the view while the log is being extracted and before the log starts to download, the log is not downloaded in your browser. A message notifies you that you can find the downloaded log in the job queue.

This is an example of a log reconstruction with the Download Log menu options displayed.

Text Analysis of a log showing the Download Log menu

The downloaded log file contains the log and is named to help identify the service on which the log was collected, the session ID, and the file type.

Note: Long running or historically downloaded files are not downloadable.

This is an example of the filename for a raw log: Concentrator_SID2.log. The exported log file is named using the following convention:

<service-ID or host name>_SID<n>.<filetype>

where:

  • <service-ID or host name> is the name of the service (for example a Concentrator or Broker) where the session was saved.
  • SID<n> is the session ID number.
  • <filetype> identifies the format of the downloaded log. These are the possible log types: raw log, CSV, XML, and JSON. By default the format is a raw log.

Note: Some formats do not have time stamps or the device IP where the event was generated, so a log downloaded in CSV, XML, or JSON format has an extra value called timestamp along with the raw log content. The additional information inside the log is in this form: Log timestamp="1490824512" source="10.4.30.65".

To download the log for a session:

  1. In the Text Analysis panel of a log event, select one of the file formats for the downloaded log.
    -To download the log as a raw log (the default format), click Download Log.
    -To download the log in one of the other formats, click the downward arrow on the Download Log button, and select one of the file formats for the downloaded log.
    Text Analysis with Download Log menu
    The log file is downloaded to your local file system in the format specified.

Download Network Data Files in the Text Analysis Panel or the Packet Analysis Panel

When viewing a reconstructed network event in the Packet Analysis panel or the Text Analysis panel, you can export network data files for further analysis. The download includes events for the current time range and drill point. You can download the data in these forms:

  • The entire event as a packet capture (*.pcap) file using the Download PCAP option.
  • The payload as a *.payload file using the Download All Payloads option.
  • The request payload as a *.payload1 file using the Download Request Payload option.
  • The response payload as a *.payload2 file using the Download Response Payload option.

This is an example of the filename for a PCAP file: C01 - Concentrator_SID1697309.pcap. The exported network data file is named using the following convention:

<service-ID or host name>_SID<n>.<filetype>

where:

  • <service-ID or host name> is the name of the service (for example a Concentrator or Broker) where the session was saved.
  • SID<n> is the session ID number.
  • <filetype> is pcap, payload, payload1, or payload2.

The network data is downloaded directly into your browser if the download is quick. If the download takes longer due to network factors or file size, the file is downloaded in the background and the task is tracked in the Jobs queue. In this case, you can check your jobs in the queue and get the file when the download is complete.

Note: If you initiate a download and move away from the view while the file is being extracted and before the file starts to download, the file is not downloaded in your browser. A message notifies you that you can find the downloaded document in the job queue.

To export an event as a network data file:

  1. Go to the Packet Analysis panel of a network event, and select one of the file formats for the downloaded file.
    -To download the event as a PCAP file (the default format), click Download PCAP.
    -To download the event in one of the other formats, click the downward arrow on the Download PCAP button, and select one of the file formats for the downloaded event data.
    Download PCAP menu in the Packet Analysis panel
    The network data file is downloaded to your local file system in the format specified.

Use the Payload Only Option in the Packet Analysis Panel of a Network Session

When viewing a reconstruction of a network session in the Packet Analysis panel, you can choose to view only the main payload for each packet. By default, packet header and footer bytes are displayed for each packet. You can hide these by clicking the Display Payloads Only toggle switch. If you are viewing only the payload bytes, you can revert to the default setting by setting the Display Payloads Only toggle switch to on. This setting persists until you change it or refresh the browser.

  • With the Display Payloads Only option off, the number of packets, packet header, packet footer, and payload are displayed.
  • With the Display Payloads Only option on, no packet header and footer bytes are displayed. Only the packet content of 16 hexadecimal bytes per line and the corresponding ASCII per line is displayed.
  1. In the Event Analysis view, go to the Packet Analysis panel of a network session.
    By default the session is reconstructed with the packet header, footer, and payload displayed.
    Display Payloads Only off
  2. To change the view to show only the payload for each packet, click the Display Payloads Only toggle switch.
    The view changes to that only the payload is visible and contiguous same-side packets are concatenated together to make the payload more readable and understandable.
    Display Payloads Only in effect

View Highlighted Bytes in the Packet Analysis Panel

When you first open a reconstruction in the Packet Analysis panel, the significant header bytes in each packet are highlighted in blue, and the payload bytes are distinguished using shading to help you understand the contents of the packet. This figure shows the default Packet Analysis with highlighting and byte shading.

Common File Patterns and Shaded Bytes in effect

The Shade Bytes option adds shading to identify the different hexadecimal bytes (00 to FF) using degrees of highlighting. Bytes near the lower range are more transparent, and bytes near 255 are more opaque. Both hexadecimal and ASCII bytes are shaded. This is an example of the shading applied to each hexadecimal byte.

example of shading applied to hexadecimal bytes

The Shade Bytes switch controls the shading of bytes. When you set Shade Bytes on or off, your setting persists until you change it or refresh the browser.

Highlight Common File Types in the Packet Analysis Panel

In the Packet Analysis panel, analysts can show or hide highlighting of certain common file types based on the file signature. When the Common File Patterns feature is turned on, the magic number bytes in the file signature are highlighted in the payload and you can hover over the highlighting to see the potential type of file. In this example, 89 50 4e 47 is highlighted in the hexadecimal payload and PNG is highlighted in the ASCII payload. When you hover over the highlighted bytes, the potential file type associated with the magic number is provided in a hover box.

Common File Patterns and Shaded Bytes in effect

These are the files types and corresponding magic numbers that are highlighted if present in the payload:

                                                                                                  
File TypeHexadecimal SignatureASCII Encoding
DOS Executable / Windows PE4D 5AMZ
Portable Network Graphics (PNG) 89 50 4E 47 0D 0A 1A 0APNG
JPEG FF D8 FFJPEG
JPEG/JFIF4A 46 49 46JFIF
JPEG/Exif45 78 69 66Exif
GIF47 49 46 38 37 61GIF87a
GIF47 49 46 38 39 61GIF89a

Non-portable Executable

5A 4D

ZM

BMP42 4DBM
PDF25 50 44 46%PDF
Old Office Document (doc, xls, ppt, msg, and other)D0 CF 11 E0 A1 B1 1A E1ÐÏ.ࡱ.á
ZIP file formats and formats based on it, such as JAR, ODF, OOXML50 4BPK..
7-Zip File Format (7z)37 7A BC AF 27 1C7z¼¯'
Java Class File, Mach-O Fat BinaryCA FE BA BEÊþº¾
Postscript 25 21 50 53%!PS
Unix/Linux Shell script23 21#!
Executable and Linkable Format (ELF) executables7F 45 4C 46 .ELF

To view common file signatures in the Packet Analysis panel:

  1. Navigate to Packet Analysis panel, and turn on the Common File Patterns option.
    If there is more then one highlight in view, all are shown.
  2. To view the hover box, place the cursor over the highlighting.

Download Files from a Network Event in the File Analysis Panel

When viewing reconstructed network events that contain files in the File Analysis panel, you can select one file, one or more files, or all files to download to your local file system.

Note: If you initiate a download and move away from the view while the file is being extracted and before the file starts to download, the file is not downloaded in your browser. A message notifies you that you can find the downloaded file in the job queue.

When files are selected, the Download Files button becomes active and reflects the number of files selected.

File Analysis with files selected

Clicking the button exports the selected files as a password-protected zip archive. The password to open the exported archive is netwitness. Exporting the files in this form ensures that:

  • The archive is not quarantined by antivirus software.
  • Potentially malicious files are not automatically opened by the default application and executed.

This is an example of the filename for an archive: C01 - Concentrator_SID1697309_FC1.zip. The exported archive is named using the following convention:

<service-ID or host name>_SID<n>_FC<n>.zip

where:

  • <service-ID or host name> is the name of the service (for example a Concentrator or Broker) where the session was saved.
  • SID<n> is the session ID number.
  • FC<n> is the file count or number of files in the archive.

Caution: Caution is advised when unzipping and opening files that are associated with a default application; for example, an Excel spreadsheet may automatically open in Excel before you have a chance to verify it is safe.

To export files in a reconstructed event:

  1. In the Event Analysis view, go to the File Analysis panel of an event that contains files.
    File Analysis with a file selected
  2. Click one or more files that you want to extract, and click Download Files.
    The job is scheduled and when complete the selected file are downloaded, in the form of a password-protected zip archive, to the local file system.
  3. To open the archive on your local file system, enter the following password when prompted: netwitness.

Open an Endpoint Event in the NetWitness Endpoint Application

When viewing an endpoint event in the Text Analysis panel, you can pivot to analyze the same event in NetWitness Endpoint.

Note: Version 4.4 of the NetWitness Endpoint Thick Client must be installed on the same server, the NWE meta keys must exist in the table-map.xml file on the Log Decoder, and the NWE meta keys must exist in the index-concentrator-custom.xml file. The NWE Thick Client is a Windows only application. Complete setup instructions are provided in the NetWitness Endpoint User Guide for Version 4.4.

To open an event in NetWitness Endpoint:

  1. To search for endpoint events, select Query in the Navigate view tool bar.
  2. In the Query dialog, select Advanced, and enter one of the following queries: nwe.callback_id exists or device.type='nwendpoint'
    Endpoint data is displayed in the Values panel.
  3. Right-click an event, and select Event Analysis in the context menu.
    The Event Analysis opens with the selected event displayed in the Text Analysis.
    Endpoint Event open in the Text Analysis
  4. In the Event Header click Pivot to Endpoint.
    A new browser tab with the url ecatui://<id> opens and the NWE Thick Client is launched . If the NetWitness Endpoint Thick Client is not installed, no data is displayed and the following message is displayed: Applicable for hosts with 4.x Endpoint agents installed, please install the NetWitness Endpoint Thick Client.

Act on Meta Values in Event Analysis

In the Event Analysis view you can further investigate meta values in an event by right-clicking certain meta values and using the options in a drop-down menu. Right-click actions are not available on all, for example, the time field does not have a drop-down menu.

Follow these steps to search for meta values and perform actions on them:

  1. In the Event Analysis view, right-click a meta value in the Events List, the Event Meta panel, or the Event Header. Some meta values have a drop-down menu.
    Right click on meta values for further actions
  2. Select one of the following actions:
    Copy: Copies the meta value to the clipboard.
    Refocus Investigation in New tab: Launches the another investigation in a new tab with the focus on the selected meta value.
    Apply Drill in New Tab: Applies the drill and launches it in a new tab to drill the data in Navigate view.
    Apply !EQUALS Drill in New Tab: Applies (!EQUALS) to the meta and launches a new tab, effectively excluding the meta value from the results.
    Hosts Lookup: Looks up the value in the Investigate > Hosts view.
    Endpoint Thick Client Lookup: Analyzes the meta value in the Endpoint Thick Client (for clients which have Endpoint Agent).
    Live Lookup: Looks up a meta value on Live for further analysis.
  3. For an external lookup, hover over a meta value, right-click and select External Lookup. In the submenu select one of the available external lookups:
    Google: Looks up a meta value on Google.com
    SANS IP History: Looks up a meta value on SANS IP History, domain = http://isc.sans.org/ipinfo.html?ip=ipaddress
    CentralOps Whois for IPs and Hostnames: Looks up a meta value on CentralOps Whois for IPs and Hostnames, domain = http://centralops.net/co/DomainDossier.aspx?addr=domain&dom_whois=true&dom_dns=true&net_whois=true
    Robtex IP Search: Looks up a meta value on Robtext IP Search, domain = https://www.robtex.com/cidr/domain.ipaddress
    IPVoid: Looks up a meta value on IPVoid, domain = http://www.ipvoid.com/scan/domain/
    URLVoid: Looks up a meta value on URLVoid, domain = http://www.urlvoid.com/scan/ipaddress/
    ThreatExpert Search: Looks up an IP meta value on ThreatExpert Search, domain = http://www.threatexpert.com/reports.aspx?find=IP address
You are here
Table of Contents > Investigate: Examining ing Raw Events and Meta Data in the Event Analysis View

Attachments

    Outcomes