000037654 - How to aggregate ESA events from the current time in the RSA NetWitness Platform (Version 11.3 and above)

Document created by RSA Customer Support Employee on Jul 11, 2019•Last modified by RSA Customer Support Employee on Aug 22, 2019
Version 2Show Document
  • Comment • 0
  • View in full screen mode

Article Content

Article Number000037654
Applies ToRSA Product Set: NetWitness Logs & Network
RSA Product/Service Type: Event Stream Analysis 
RSA Version/Condition: 11.3 and above
Platform: CentOS
O/S Version: EL7
IssueConsider the following scenario: A device that is configured to forward events to ESA has been down for a substantial period of time, and millions of events have accumulated. By the time the situation with the device is rectified, an exorbitant number of events may need to be processed. This can lead to a flood of events sent all at once to the ESA server, which may have debilitating effects on performance. Moreover, the historic alerts in situations of this nature typically contain repetitive information that is not needed.
ResolutionIn situations such as those described, or when other circumstances dictate limiting the events sent by time, follow these steps to aggregate ESA events to the current time:
  1. Within the RSA NetWitness UI, navigate to Administration > Services. From the services list, select ESA Correlation > Actions > View > Explore.
  2. Within the Explore menu of the NetWitness UI, navigate to correlation > stream > metrics. In 11.3, ESA keeps track of source positions in order for it to be able to resume aggregation from the same position after restarting the service. To clear the backlog, the position tracking information will need to be deleted - this information is stored within the positionTrackingEntity collection within the correlation-server mongoDB instance on node zero.
    It will show two values on the second column next to each server instance: [<sessionid>, <UTC time>] 
     
  3. On node zero, log into mongo. 

    mongo admin -u deploy_admin -p <password>

  4. Display the list of databases and choose the correlation-server database.

    > show dbs
    > use correlation-server

  5. Display the list of collections and choose the positionTrackingEntity collection. 

    > show collections
    > db.positionTrackingEntity.find().toArray()


     


  6. Within the displayed list of the positionTrackingEntity collection, it will show the specific server instances that you see within the Explore view in the NetWitness UI mentioned above. 
  7. Narrow down the list of all server instances to the specific server instances that are behind in time. Change the IP address below according to your device.

    db.positionTrackingEntity.find({"_id": {$regex: /10.1.1.1/}}).toArray()

  8. Stop the correlation-server. Within the NetWitness UI, navigate to Administration > Services. From the services list, select ESA Correlation > Actions > View > Stop.
  9. Remove all records of the specific server instances found in the list in Step 7. Change the IP address below according to your device.

    db.positionTrackingEntity.remove({"_id": {$regex: /10.1.1.1/}})

  10. Run Step 7 again to confirm that those specific server instances have been removed.
  11. Start the correlation-server. Within the NetWitness UI, navigate to Administration > Services. From the services list, select ESA Correlation > Actions > View > Start. 
  12. Within the Explore menu of the UI, navigate to correlation > stream > metrics. Check the UTC time value for the server instance in question to confirm that aggregation has resumed much closer to the current time.
NotesThis article applies only to version 11.3 and above of the RSA NetWitness Platform.  For earlier versions, please refer to the article entitled How to aggregate ESA events from the current time in the RSA NetWitness Platform (Version 11.2 and below) for instructions.

Attachments

    Outcomes