Sneha Sabnis

NetWitness Log Parser Improvements

Blog Post created by Sneha Sabnis Employee on Jul 7, 2017

The RSA NetWitness Log Parsing team has reviewed the Top 50 Log Parsers that generate the most number of incoming “Unknown Message Defect” support cases.

 

Summary for Top 50 Event Sources

Total

# of Log Parsers Released on Live

Number of Event Sources - IMPROVED & RELEASED

38

38 (as of 21 Jun 2017)

Number of Event Sources - CANNOT BE IMPROVED

12

Not Applicable

Final Count

50

38

These enhancements are part of a strategic initiative to drive improvements to Log Parsers.

Benefits from these improvements result in:

  • Fewer Unknown Messages
  • Improved Device Discovery
  • Better Adaptability to newer versions of an Event Source
  • Reduced Parser Maintenance

 

To take advantage of these improvements, you will need to download the latest versions of the Log Parsers listed below from the NetWitness Live Portal.

 

 

A Look into the Details

50 Log Parsers were identified as the ones that were causing almost 80% of ALL the incoming requests for fixing unknown messages & minor defects. All of them were reviewed in detail and checked for scope of adding any improvements.

 

Design Considerations

The team investigated the reasons behind generation of high volume of incoming unknown message tickets.

 

The factors considered were:

  • Common patterns seen across all unknown message tickets for a particular event source.
  • Certain Log formats that generate Unknown messages
  • Were these logs result of a Configuration Error?
  • Were these logs generated after upgrading to a newer version of the Event Source?
  • Backwards Compatibility Impact of the modifications:
    • If the solution is re-designing the parser in any way (re-writing a few or all message definitions or Header definitions)  the 'after-effect' on the Meta Key Footprint should be nothing to very minimal.

 

Log Parsers were updated/Re-designed only If the Backwards Compatibility impact from above parameters were negligible.

Here is a brief highlight of the design improvements-

 

  • The Log Parsers have been redesigned to identify all events generated by the event source. They have been made future proof as much as possible to parse newer events.
  • Log Parsers which parse logs with structured formats that use tag=value functionality, have been updated to accommodate New/Unknown tags, which significantly reduces the number of unknown messages.
  • These updates were also done to cover as many new changes as possible that may be introduced in Newer versions of the product.

 

Here is the list of 38 Log Parsers that were Improved and released to NetWitness Live –

 

 

Log Parser Name

1

Bit9-Bit9 Security Platform

2

Blue Coat-Blue Coat ProxySG SGOS

3

Check Point-Check Point Security Suite, IPS-1

4

Cisco-Cisco Adaptive Security Appliance

5

Cisco-Cisco IronPort Email Security Appliance

6

Cisco-Cisco IronPort Web Security Appliance (WSA)

7

Cisco-Cisco Secure Access Control Server & Cisco-Cisco Identity Services Engine

8

Cisco-Cisco Secure IDS or IPS

9

Cisco-Cisco Wireless LAN Controller (2100 Series and 4400 Series)

10

F5-F5 Big-IP Application Security Manager

11

FireEye-FireEye Web Malware Protection System

12

Fortinet-Fortinet FortiGate

13

IBM-IBM AIX

14

IBM-IBM DB2 Universal Database

15

IBM-IBM iSeries AS400

16

IBM-IBM ISS SiteProtector

17

IBM-IBM WebSphere

18

Juniper-Juniper Networks SSL VPN

19

Lancope-Lancope StealthWatch

20

McAfee-McAfee Email Gateway (formerly known as CipherTrust IronMail)

21

McAfee-McAfee Network Security Platform (Intrushield)

22

Microsoft-Microsoft Exchange Server

23

Microsoft-Microsoft Internet Information Services

24

Microsoft-Microsoft SQL Server

25

Microsoft-Microsoft Windows using Eventing Collection

26

Microsoft-Microsoft Windows using: Adiscon Event Reporter

27

Microsoft-Microsoft Windows using: Intersect Alliance SNARE

28

Oracle-Oracle Access Manager

29

Oracle-Oracle Database

30

Oracle-Sun Solaris

31

SNORT/SourceFire

32

Trend Micro-Trend Micro Control Manager

33

Tripwire-Tripwire Enterprise

34

UnboundID - UnboundID Identity Data Store

35

Vmware-VMware ESX/ESXi

36

Vmware-VMware vCenter Server

37

Voltage SecureData

38

Websense-Websense Web Security

 

 

Here is the list of 12 Log Parsers that cannot be improved further -

 

 

Log Parser Name

1

Check Point-Check Point IPSO (nokiaipso)

2

Cisco-Cisco Router/Switch

3

Citrix-Citrix NetScaler

4

F5-F5 Big IP (Local Traffic Manager)

5

Infoblox-Infoblox NIOS

6

Juniper-Juniper Networks JUNOS

7

McAfee-McAfee ePolicy Orchestrator

8

McAfee-McAfee Web Gateway

9

Palo Alto Networks-Palo Alto Networks Enterprise Firewall

10

Red Hat Linux (RHEL)

11

RSA Authentication Manager/UCM  (rsaacesrv)

12

Symantec-Symantec Endpoint Protection

 

Most of these contain highly unstructured Log formats. Due to several Backwards Compatibility / Performance impact issues, these couldn't be improved.

 

Please note that these 12 Log Parsers are expected to generate unknown messages. The team depends on the incoming support requests for updating these parsers. Once these requests are received, the team will get them updated as soon as possible.

 

 

The RSA NetWitness Log Parsing team will be closely monitoring any incoming requests for the improved Log Parsers and further improve them as applicable. They will continue to power these improvements to other Supported Log parsers in the library.

Outcomes