David Waugh

Troubleshooting ESA Memory Rule Usage

Blog Post created by David Waugh Employee on Apr 19, 2016

I had a look in my test system today and noticed that my ESA Trial Rules had been disabled.

 

esa Trial Rules Disabled.png

 

Further investigation in health and wellness showed that a particular rule was using much more memory than any others. As this is a test VM system, the virtual memory available is much less than available in a physical appliance. Here we can see that this rule is using 311 MB!

 

Bad ESA Rule.png

 

This was not what I intended for this rule, so looking at the EPL for the rule reveals the following:

 

module UserAgent_Alert;

 

@Hint('reclaim_group_aged=60 minutes,reclaim_group_freq=1')

// The stage setters - the output of these statements is for Esper to retain (internally)

CREATE WINDOW UserAgentWatchList.win:time(600 minutes) (client string);

INSERT INTO UserAgentWatchList SELECT client from Event(client IS NOT NULL AND alert_id IS NOT "Client Ignore");

// The real “alerter”. The annotation, identifies it as the one that ESA needs to watch for.

@RSAAlert

@RSAPersist

@Name('UserAgent Watchlist')

@Description('This alert will trigger on new User Agents but only 1 alert per individual IP address.')

SELECT * FROM Event(client is NOT NULL AND alert_id IS NOT "Client Ignore" ) WHERE client NOT in (SELECT client FROM UserAgentWatchList) ;

 

Here the issue is that I put all new clients into the UserAgentWatchList and not just new ones. This would mean that duplicate values were stored in this named window.

 

The correct rule is the following. In this example I only add a client to the named window UserAgentWatchListLowMem if it is not already in the named window.

 

@Hint('reclaim_group_aged=60 minutes,reclaim_group_freq=1')

// The stage setters - the output of these statements is for Esper to retain (internally)

CREATE WINDOW UserAgentWatchListLowMem.win:time(600 minutes) (client string);

On Event(client IS NOT NULL AND alert_id IS NOT "Client Ignore") as myevent

merge UserAgentWatchListLowMem WatchListClient

where WatchListClient.client=myevent.client

when not matched

then insert select client;

 

// The real “alerter”. The annotation, identifies it as the one that ESA needs to watch for.

@RSAAlert

@RSAPersist

@Name('UserAgent WatchlistLowMem')

@Description('This alert will trigger on new User Agents but only 1 alert per individual IP address.')

SELECT * FROM Event(client is NOT NULL AND alert_id IS NOT "Client Ignore" ) WHERE client NOT in (SELECT client FROM UserAgentWatchListLowMem) ;

 

You can check memory utilisation of your Named Windows using this article.

How do I look in the Named Windows of my ESA Rules

Outcomes