File Collection tuning for reading many small files
I have a custom File collection device which is transferring about 5-6 files/second of size 30KB at most to the VLC upload folder. However the NetWitness collector service is not reading them faster then 2-3 files/second so the backlog will always grow.
Resources on the VLC are not an issue. The VM is effectively idle.
I know that the issue is not the number of events inside the files but rather the number of files that are creating this issue.
How to tune the service to read the small files faster? Assume that I cannot change the event source to generate bigger files.
Thanks in advance
- Community Thread
- file collection
- Forum Thread
- netwitness for logs
- RSA NetWitness
- RSA NetWitness Platform
I found that the File reader was able to keep up with the rate of the incoming files after I unticked the "Sequential processing" in the File options of the event source.
Any impact created by out-of-order events is negligible for us because we are using event.time for our searches.