000015463 - Failed to retrieve distinct values for specific field across range N to NNN: 500 Server Error

Document created by RSA Customer Support Employee on Jun 14, 2016Last modified by RSA Customer Support Employee on Apr 21, 2017
Version 2Show Document
  • View in full screen mode

Article Content

Article Number000015463
Applies ToRSA Security Analytics
RSA Security Analytics 10.3.2
RSA Security Analytics Concentrator
Issue

After upgrading to SA 10.3 SP2, when running reports or using the Investigation module in Security Analytics, all meta categories periodically show the "Failed to retrieve distinct values for specific field across range N to NNN: 500 Server Error" message, as shown below.


Cause

The most common cause for this error is that the NwConcentrator process has reached the CentOS open file limit (default=1024) and can't open any more files.


In linux, the open file limit for each process can be individually set and the startup/initialization script for the NwConcentrator process should normally have a higher 'number of files' (nofile) limit override.


If this higher open file limit is not set it can cause "Too many open files" errors which in turn will prevent data being populated in the Investigation module.
Resolution

To fix the "failed to retrieve distinct values" error caused by too many files being open, you can manaully increase the limit following these steps:


 


1.   On the concentrator, edit the file /etc/init/nwconcentrator.conf and look for the line that says:  limit core unlimited unlimited


2.   Underneath the line in Step 1, add the following text:  limit nofile 65536 65536


3.   When finished, the file should appear similar to the following:



start on runlevel [35] and stopped rc


stop on runlevel [!35]


respawn


respawn limit 10 300


console output


kill timeout 60


chdir /var/netwitness/concentrator/metadb


limit core unlimited unlimited


limit nofile 65536 65536


exec /usr/sbin/NwConcentrator --stopwhenready


expect stop



 


4.   Once you have added the text in Step 2, save the nwconcentrator.conf file and restart the nwconcentrator service by issuing the commands below:



CentOS 6:


  1. stop nwconcentrator
  2. status nwconcentrator (make sure it's been fully stopped)
  3. start concentrator

CentOS 5:


  1. monit stop nwconcentrator
  2. monit status nwconcentrator (make sure it's been fully stopped)
  3. monit start nwconcentrator

 


If you still receive the errors after performing these steps, contact RSA Support and quote this article ID for further assistance.

Legacy Article IDa64748

Attachments

    Outcomes