How to create new file reader based device?
I am in need of creating a new file reader device XML (and whatever else is needed). This doesn't seem to be docuemnted anywhere, including within the UDS training guide.
Does anyone know how the parsing works? Since file reader tacks on the extra tag at the beginning of the log entries?
For ANY event source, regardless of log transfer method, there are 4 basic steps:
1) Get the logs into the enVision IPDB
2) Dump them out as a syslog file using lsdata
3) Use the syslog file to build the XML and device files
4) Integrate the XML and device files into your enVision install
It sounds like you may be struggling with part 1. I'm attaching a short powerpoint that explains how setting that step up works.
Once you have steps 1 & 2 complete, you can then do step 3 using the ESI (Event Source Integrator) tool to build the XML, and the parsing then works just like any other event source
One additional thing to keep in mind is that the Filereader itself does not have anything to do with parsing. The Filereader is a collection service - that's all!
I hope that helps.
Thanks for that Matt! But my major issue is regarding the parsing. Do I need to export the newly fed in data in order to see the parsable format?
How can I do that without having it assigned correctly to begin with?
Yes, after reading the data in, you must use lsdata to export the data in a format that the ESI tool or UDS console will use:
lsdata -events syslog -time start now -devices 22.214.171.124 >> mylog.unx
126.96.36.199 is the IP Address of the source device you imported the logs from, even if it is an unknown device type.
And only then will it have the correctly parsable format?
Is this (regarding how to properly parse file reader devices) documented anywhere, preferably in the online help?
Did I miss it?
Data is only parsed at the time a report or query is executed (i.e. when it is being extracted from the IPDB for reporting purposes)
The collector simply writes the raw data to the IPDB.
This is part of the enVision data flow, and is typically explained in the class. I'm not sure if it is in the Help or not.
Well, since this thread came to life again, I'm also trying building an ESI device using filereader as the collection method. I drop an example file in the directory, bounce the reader and the file is read (I assume) and then is deleted from the directory.
I have my event viewer in real time mode but I never see the events come thru.... I'm using string matching on the "future" message id which is a literal like "AddEndUser", "DeleteEndUser", etc. and I see no unknown device or undefined events from the IP.
I've tried using lsdata for undefine and actually all events for a give time period and do not see the new events reaching the IPDB.
I ran the file reader debug, and get the following: "Could not find message ID at field 7 =>" appears to be the issue, but I would think the data would come in as unknown/undefined anyway.
Mon Mar 28 11:41:53 <5> %NIC-5-604101: FileReader, FileReader, -, -, -, -, Detail: 3732: Started File:device=GENERIC_FILEREADER,file name=E:\NIC\3700\xxxxxxx-LC1\FTP_FILES\mydevice_xx.xx.xx.xx\transactions.log,file size=16635,file time=1301328595,start time=-191455971174776832
Mon Mar 28 11:41:53 <7> %NIC-7-604111: FileReader, FileReader, -, -, -, -, Detail: 3732: Debug: Found end of file.
Mon Mar 28 11:41:53 <3> %NIC-3-604105: FileReader, FileReader, -, -, -, -, Detail: 3732: Could not find message ID at field 7 =>
Mon Mar 28 11:41:53 <5> %NIC-5-604102: FileReader, FileReader, -, -, -, -, Detail: 3732: Finished File:device=GENERIC_FILEREADER,file name=E:\NIC\3700\xxxxxxx-LC1\FTP_FILES\mydevice_xx.xx.xx.xx\transactions.log,events sent=60,eps=1.#J,end time=1301330513871
Any other ways to see where the data is going, or how I can get the data out? Any advise would be appreciated.Thanks!
Saw another thread that had the message_id position as null, so set my File Reader Type up like that. Sent in the same file, used lsdata to dumped the last 15 minutes of events and found my log records in syslog format. Still curious as to why they don't show in event viewer??