08APR2020 - UPDATE: adding a couple notes and example typespecs after some additional experimenting over the past week
- You may find the process easier to simply copy an existing 11.4 typespec in the /var/netwitness/source-server/content/collection/file directory on the Admin Server and modify it for the custom collection source you need
- example using IIS typespec:
- another example using a custom typespec to collect Endpoint v4.4 (A.K.A. legacy ECAT) server logs
The NetWitness 11.4 release included a number of features and enhancements for NetWitness Endpoint, one of which was the ability to collect flat file logs (https://community.rsa.com/docs/DOC-110149#Endpoint_Configuration), with the intent that this collection method would allow organizations to replace existing SFTP agents with the Endpoint Agent.
Flat file collection via the 11.4 Endpoint agent allows for a much easier management compared to the SFTP agent, in addition to the multitude of additional investigative and forensic benefits available with both the free version of the Endpoint agent and the advanced version (NetWitness Endpoint User Guide for NetWitness Platform 11.x - Table of Contents).
The 11.4 release included a number of OOTB, supported Flat File collection sources, with support for additional OOTB, as well as custom, sources planned for future releases. However, because I am both impatient and willing to experiment in my lab where there are zero consequences if I break something, I decided to see whether I could port my existing, custom SFTP-based flat file collections to the new 11.4 Endpoint collection.
The process ended up being quite simple and easy. Assuming you already have your Endpoint Server installed and configured, as well as custom flat file typespecs and parsers that you are using, all you need to do is:
- install an 11.4+ endpoint agent onto the host(s) that have the flat file logs
- ...then copy the custom typespec from the Log Decoder/Log Collector filesystem (/etc/netwitness/ng/logcollection/content/collection/file)
- ...to the Node0/Admin Server filesystem (/var/netwitness/source-server/content/collection/file)
- ...after your typespec is copied (and modifed as necessary), restart the source-server on the Node0/Admin Server
- ...now open the NetWitness UI and navigate to Admin/Endpoint Sources and create a new (or modify an existing) Agent File Logs policy (more details and instructions on that here: Endpoint Config: About Endpoint Sources)
- ...and once you have confirmed Collection via the Endpoint Agent, you can stop the SFTP agent on the log source (https://community.rsa.com/docs/DOC-101743#Replace)
And that's it. Happy logging.