|Applies To||RSA Product Set: Security Analytics (10.5.x.x, 10.6.x.x), Netwitness for Logs and Network (11.x)|
RSA Product/Service Type: Log Collector (Can be applied to any host that exhibits the same symptoms but was initially discovered on a Log Decoder/Log Collector)
RSA Version/Condition: 10.5.x.x and higher releases
Platform: CentOS 6, 7
- LogCollection Stopped.
- Running command: df -kh shows that the LogCollector director is fully utilized [100% full].
|Cause||Corrupted "xfs" filesystem within the /var/netwitness/logcollector directory .|
- vi the /etc/fstab file to hash the NFS mapping of the LogCollector directory.
Put # prior the line highlighted with Red to be like:
#/dev/mapper/VolGroup00-lcol /var/netwitness/logcollector xfs nosuid,noatime 1 2
- Reboot the appliance using command: reboot.
- Check which process is accessing that directory using command: lsof
- stop nwlogcollector service.
- Unhash the LogCollector NFS mapping.
- Unmount the directory using command: umount /dev/mapper/VolGroup00-lcol
- Perform filesystem check using command: xfs_check /dev/mapper/VolGroup00-lcol
- Perfom filesystem repair using command: xfs_repair -L /dev/mapper/VolGroup00-lcol
- Re-mount the LogCollector durectory using command: umount -a /dev/mapper/VolGroup00-lcol
- Run command "Start nwlogcollector service" to start logcollection service.
- Issue command: df -kh to check the storage status of the "/var/netwitness/logcollector" directory you will find it dropped down around 30% to reach 70% of utilized storage only.
- Run command tail -f /var/log/messages to track the operations of appliance.
- Perform required health-checks by checking the health &Wellness on the SA UI.