After a query is submitted in the Events view, the Events panel opens with a list of sequential events. The events listed here meet two conditions:
- They match the submitted query.
- They include a value for one or more meta keys required by the selected column group. If you change the column group while viewing the Events list, the original query with the new column group is resubmitted. Unsubmitted query changes made to service, time range, or filter, are ignored.
How Results Are Loaded and Sorted
There is a configurable limit on the number of events that can be loaded; the default value is 5,000. Administrators can configure the limit as described in the System Configuration Guide. The events begin loading into the Events panel; a progress bar at the top of the list tracks progress while events are loaded. Events with the earliest collection time are loaded first and a row number indicator of the form " EVENTS xxx - xxx" is inserted in the list after every group of 100 events as shown in the following figure.
A spinner is displayed while the events are loaded. If that count is greater than or equal to the threshold, a message under the spinner advises that fact and directs you to the Query Console for more details. As data begins loading, the message is removed and the spinner remains until all events are loaded. When all events have loaded, one of these messages is added to the bottom of the list:
- "All events loaded."
- "Reached the 5,000 event limit. Consider refining your query."
- "Retrieved 4,000 of 5,000 events prior to query cancellation."
A message at the top of the list indicates the total number of events loaded, if the 5,000 event limit has been reached, and the sorting method in effect:
- The message when fewer than 5,000 events are listed is: "xx,xxx Events"
- The message when more than 5,000 events are listed is: "Oldest 10,000 Events (Asc)"
If the number of events that match the query exceeds the limit of 5,000 events, the newest or oldest 5,000 events in the time window are loaded in ascending order. The portion of events loaded is based on the sort order. For example, if 300,000 events match your query and the sort sequence is set to Ascending, the oldest 5,000 events are loaded by default. You can change this by changing the sort order to Descending and the newest 5,000 events are loaded. Ascending sort, which loads the oldest events first is usually the best setting for investigating network events. If you want to view the newest 5,000 events in the time window, you can change the Default Event Sort Order to Descending in the Event Preferences dialog.
The sorting method for the list is configured in the Event Preferences dialog (see Configure the Events View). Any change in the setting goes into effect the next time you submit a query. The Default Event Sort Order from the Events Preferences dialog is saved in the database and persists after logging out and logging back in.
- Unsorted (default for Version 11.4.1): To list events as processed by the Core services. Unsorted is faster because it streams back the events as soon as a match is found versus waiting for all Core services to respond and then displaying them in the chosen order.
- Ascending (default for Version 11.4 and earlier): To list the events with the earliest collection time first. The earliest collection time first is well suited to most investigations. When investigating logs, you may want to change the sort sequence to latest collection time first.
- Descending: To list the events with the latest collection time first. The latest collection time first is often useful for investigating logs.
Actions to Refine the Events List
When the results are loaded in the Events panel, you can take the actions to refine the list:
- Select a column for sorting events (Use Columns and Column Groups in the Events List).
- Select a set of meta keys (column group) that help with a specific type of investigation (Use Columns and Column Groups in the Events List).
- Apply a query profile (Use Query Profiles to Encapsulate Common Areas for Investigation).
- (Version 11.5) Filter events by pivoting through the metadata Drill into Metadata in the Events View (BETA).
Actions to Analyze Events
The rest of this section has procedures for working in the Events view and adjust reconstructions to bring interesting data into focus.
- You can download events and create an incident in Respond.
- Clicking an event in the Events panel opens the Event Details panel to a tab that shows an event reconstruction (text, packet, file, email, and web) or the tab that shows host information for network events that are enriched with endpoint data (Version 11.5).
- The Events panel and the Event Details panel can be open at the same time.
- Within the Packet tab and the Text tab, you can use additional features to adjust the way the reconstruction is displayed and bring interesting data into focus.
Open, Close, and Adjust the Size of the Panels in the Events View
Initially, the Network Event Details, Log Event Details, or Endpoint Event Details panel occupies 75% of the window width by default.
You can adjust the size ratio of the Events panel in relation to the details panels to improve readability by expanding one of the panels, contracting one of the panels, and closing one of the panels. After closing either panel you can reopen it. The ratio you select persists until you change it or refresh the browser.
To optimize your view:
- To adjust the size ratio of the two panels, do any of the following:
- To close either panel, restoring the open panel to its full width, click .
This is an example of the reconstruction displayed using the full width of the browser window.
- To reopen the Events panel after closing, click in the top right corner of the Events view.
The Events panel opens to the last state (25%-75% or 50%-50%).
- To reopen the Event Details panel, click an event in the Events panel.
Select the Analysis Type for an Event
To select the analysis type for an event, with an event open in the Event Details panel, click one of the tabs: Text, File, Host , Packet, Email, or Web.
- If you chose Hosts, the host information from extended endpoint data is displayed.
- If you chose File, Text, Packet, or Email, the reconstruction is displayed.
- If you chose Web, the reconstruction of the single event opens in a new tab. This is the same reconstruction of a session used in the Legacy Events view (see Reconstruct an Event in the Legacy Events View).
Adjust the Display of Requests and Responses
For analysis types that have requests and responses, you can select which side of the conversation to show, Request , Response , or both, click one or both of the direction icons. The reconstruction is refreshed with the selected information.
View Associated Metadata for an Event
When examining events in the Text tab, Packet tab, or Filetab, you can click to show the associated metadata in an adjacent panel, the Event Meta panel. For Version 11.5.1, a set of more than 200 meta key symbols replaces the three indexing method symbols to provide a visual indicator of the purpose of the meta key. The color of the meta key symbol identifies the indexing method using the same colors as before: green, yellow, and red. A tooltip also identifies the indexing method and provides a description of the icon. The icons are defined based on categories outlined in the Unified Data Model (https://community.rsa.com/community/products/netwitness/rsa-content/udm). There is a generic icon for most categories that do not have specific meta keys and a default meta key icon to use when a new custom meta key is added.
You can change the order of the metadata listed in the Event Meta panel to better find what you are looking for. Metadata can be organized by the sequence in which they were generated or alphabetically by meta key. This figure illustrates metadata organized as meta keys sorted alphabetically.
This figure illustrates the same metadata presented in the sequence in which it was generated.
When viewing a text reconstruction and the Event Meta panel, hovering over the meta key/meta value pairs reveals a pair of binoculars if the meta value is searchable in the raw text. This figure is an example of white binoculars on a black background marking a searchable meta key.
Clicking the icon triggers a search for the meta key/meta value pair (case-insensitive) in the Text tab and each instance is highlighted. This is an example of white binoculars on a blue background after clicking a searchable meta key/ meta value combination.
In the Event Meta panel, the highlighted row has a count of the results and up and down arrows that you can use to quickly find each result in the Text tab. You can view each highlighted location of the data that triggered generation of the meta key, going forward to view the next, and back to view the previous.
Only meta keys that have relevant values inside the RAW text are searchable. You can search only one meta key at a time. If the value is currently hidden due to truncation of a text entry with more than 3000 characters, the text entry is expanded to reveal the found meta value.
To search the raw text for meta values that triggered a meta key:
- Open a network event in the Text tab and click to open the Event Meta panel.
- Hover the mouse over the meta key/meta value pairs in the list until you see a binoculars icon next to a meta key.
- To search for the value in the raw text, click a row that has the binoculars icon, indicating it is searchable.
If no relevant occurrence of the value is in the text, the value that you are searching for is highlighted in the Event Meta panel and nothing is highlighted in the Text tab.
If one or more relevant instances of the value is found in the Text tab, each occurrence is highlighted. The value that you are searching for is highlighted in the Event Meta panel and the up/down arrows for scrolling are visible.
- To remove the highlighting, click the same meta key/meta value pair in the Event Meta panel, click a different meta key/meta value pair in the Event Meta panel, or close the Event Meta panel.
The highlighting is removed from the raw text.
Show or Hide the Event Header
Page Through Events in the Packet and Text Tabs
Pagination controls allow more flexibility in paging through a list of packets or text. In the Packet tab, you can select the number of packets to display per page, and your selection is saved across logins to the NetWitness Platform application. When a control is unavailable, the control is dimmed; for example, when you are viewing page 1, the and controls are dimmed.
To use pagination controls:
- With an event open in the Events view, click the current number of packets per page (50, 100, 300, or 500), and select the new number of packets per page from the drop-down menu.
- To page forward or back, use the page control icons:
Click to go to the next page.
Click to go to the last page.
Click to go the previous page.
Click to go to the first page.
- To go to a specific page, type a page number in the page number field .
Expand Truncated Text Entries in the Text Tab
A reconstruction of a network event in the Text tab may include requests and responses of many hundred thousands of characters, and scrolling through a long entry that is not of interest can waste time. To save time, text entries that have more than 6000 characters are truncated to show only the first 2000 characters. This example shows an entry that has more than 2000 characters, and a message in the header indicates the percentage of total characters that is being displayed.
You can see that 46% of the characters (the first 2000) are displayed, and click Show Remaining 54% to reveal the rest of the entry.
If you search for metadata seen in the Event Meta panel while text is truncated in the Text tab, the truncated text is searched. If the metadata exists inside hidden text, the text entry expands to reveal the text with the found metadata.
Perform URL and Base64 Encoding and Decoding in the Text Tab
If a network session being reconstructed in the Text tab contains Base64 or URL encoded strings, you can decode a string to better understand the session. If the session contains decoded strings for Base64 or URL, you can view a string in its encoded form in order to search for additional instances of the encoded text in other sessions.
When viewing any network session that contains encoded text in the Text tab, you can select a subset of the text within a single Request or Response to view in either encoded or decoded form. Depending on the content loaded on the Decoder, there may be additional metadata outlining that Base64 or URL encoded data is contained within the session.
To perform encoding and decoding in the Text tab:
- In the Events view, go to the Text reconstruction of a session that contains encoded or decoded content.
- To view some decoded text in encoded form, drag to select the text within a Request or Response.
A menu offers options to encode and decode.
- Click Encode Selected Text.
The encoded text is displayed in a hover box, which remains in place until you click , select different text in the Text tab, close the Events panel, select another event for reconstruction, or switch to a different reconstruction view.
When a longer text is selected, the hover box is scrollable and large enough to fit the entire selected text and the decoded text.
- If the session contains encoded text that you want to see in decoded form, drag to select the text within a Request or Response.
A menu offers options to encode and decode.
- Click Decode Selected Text.
The decoded text is displayed in a hover box, which remains in place until you click , select different text in the Text tab, close the Events panel, select another event for reconstruction, or switch to a different tab in the Event Details panel.
- If you want to copy some text from the text reconstruction, do one of the following:
- Drag to select some text, right-click, and select Copy Selected Text from the popup menu.
- Drag to select some text, then select either Decode Selected Text or Encode Selected Text. Within the popup, select the desired text and type Control-C.
The selected text is copied to the clipboard and available to paste in a query.
- Drag to select some text, right-click, and select Copy Selected Text from the popup menu.
- When finished, click to close the hover box.
View Decompressed Text in an HTTP Network Session in the Text Tab
When the content of an HTTP network session is compressed and you are viewing the Text tab, NetWitness Platform displays decompressed content by default. This helps you to determine if there are any patterns and view the readable characters. You can switch between a compressed and decompressed view of compressed text.
The toggle for changing between compressed and decompressed text is only displayed in the Text tab, and is enabled only if there is compressed text content.
- Open the Text tab of an HTTP session that contains compressed content.
By default the session is reconstructed with the text decompressed, and above the reconstruction, is the Display Compressed Payloads toggle switch.
- To view the same text in its compressed form, click the toggle switch.
The view changes so that the compressed text is no longer readable, and the switch indicates the Display Compressed Packets is on.
- To return to the view of decompressed text, click the switch again.
View a JSON String in Tree Format in the Text Tab (Version 11.5.1)
You can view a text reconstruction of a log event in easy-to-read JSON format instead of the raw block format using the Render JSON toggle switch. By default, the switch is enabled, and JSON snippets in a log event are detected and displayed in fully expanded tree format. Invalid JSON snippets are presented as raw text. If you change the setting of the switch, your setting persists in local storage.
- In the Events view, open a log event in Text tab.
If the raw log contains JSON strings and the Render JSON switch is enabled, all JSON strings found are rendered in tree format.
- If you prefer to see the log as raw text, click the Render JSON switch.
The log is rendered as a single block of text without the nested indentation. The setting persists until you change it and will be in the same state when you log in again.
Use the Payload Only Option in the Packet Tab
When viewing a reconstruction of a network session in the Packet panel, you can hide the header and footer bytes. The view changes so that only the payload is visible and contiguous same-side packets are concatenated together to make the payload more readable and understandable. This setting persists until you change it or refresh the browser.
- With the Display Payloads Only option off, the number of packets, packet header, packet footer, and payload are displayed.
- With the Display Payloads Only option on, no packet header and footer bytes are displayed. Only the packet content of 16 hexadecimal bytes per line and the corresponding ASCII per line is displayed.
- In the Events view, go to the Packet tab of a network session.
By default the session is reconstructed with the packet header, footer, and payload displayed.
- To change the view to show only the payload for each packet, click the Display Payloads Only toggle switch.
The view changes so that only the payload is visible.
View Highlighted Bytes in the Packet Tab
When you open a reconstruction in the Packet tab, the significant header bytes in each packet are highlighted in blue, and the payload bytes are distinguished using shading to help you understand the contents of the packet. This figure shows the default appearance of the packet reconstruction with highlighting and byte shading.
The Shade Bytes option adds shading to identify the different hexadecimal bytes (00 to FF) using degrees of highlighting. Bytes near the lower range are more transparent, and bytes near 255 are more opaque. Both hexadecimal and ASCII bytes are shaded. This is an example of the shading applied to each hexadecimal byte.
The Shade Bytes switch controls the shading of bytes. When you set Shade Bytes on or off, your setting persists until you change it or refresh the browser.
Highlight Common File Types in the Packet Tab
In the Packet tab, analysts can show or hide highlighting of certain common file types based on the file signature. When the Common File Patterns feature is turned on, the magic number bytes in the file signature are highlighted in the payload and you can hover over the highlighting to see the potential type of file. In this example, 42 4d is highlighted in the hexadecimal payload and BM is highlighted in the ASCII payload. When you hover over the highlighted bytes, the potential file type associated with the magic number is provided in a hover box.
To view common file signatures in the Packet tab:
- With a reconstruction open in the Packet tab, turn on the Common File Patterns option.
If there is more then one highlight in view, all are shown.
- To view the hover box, place the cursor over the highlighting.
These are the file types and corresponding magic numbers that are highlighted if present in the payload:
|File Type||Hexadecimal Signature||ASCII Encoding|
|DOS Executable / Windows PE||4D 5A||MZ|
|Portable Network Graphics (PNG)||89 50 4E 47 0D 0A 1A 0A||PNG|
|JPEG||FF D8 FF||JPEG|
|JPEG/JFIF||4A 46 49 46||JFIF|
|JPEG/Exif||45 78 69 66||Exif|
|GIF||47 49 46 38 37 61||GIF87a|
|GIF||47 49 46 38 39 61||GIF89a|
|25 50 44 46|
|Old Office Document (doc, xls, ppt, msg, and other)||D0 CF 11 E0 A1 B1 1A E1||ÐÏ.à¡±.á|
|ZIP file formats and formats based on it, such as JAR, ODF, OOXML||50 4B||PK..|
|7-Zip File Format (7z)||37 7A BC AF 27 1C||7z¼¯'|
|Java Class File, Mach-O Fat Binary||CA FE BA BE||Êþº¾|
|Postscript||25 21 50 53||%!PS|
|Unix/Linux Shell script||23 21||#!|
|Executable and Linkable Format (ELF) executables||7F 45 4C 46||.ELF|