Help me out to get the list of newly integrated devices from RSA SA. I have a VLC in my environment, If in case any devices are integrated to the VLC I should get an alert. Also if this could be brought into report that would also be fine.
Help me out to get the list of newly integrated devices from RSA SA. I have a VLC in my environment, If in case any devices are integrated to the VLC I should get an alert. Also if this could be brought into report that would also be fine.
This fits something that I have been playing with for a bit... see if this works for you.
ESA rule that listens for a 'learning' period of time and adds all the distinct device.ip sources that it sees to a window that is persisted to disk. After the learning period is over, it will alert into NW Respond each new distinct device.ip that it sees along with a small amount of information.
This is the ESA rule:
module whatsNewDeviceIP;
/*
Rule Name: What's New for deviceIP
Author: Eric Partington
Modified: 2018-12-11
version: 3
*/
//Update learning phase to desired number of days
@Name('Named Window - learningWindowDeviceIP')
//@RSAPersist
CREATE VARIABLE integer lPhaseInDaysDeviceIP = 1;
CREATE WINDOW lPhaseDeviceIP.win:length(1) (learningPhase long);
INSERT INTO lPhaseDeviceIP
SELECT current_timestamp.plus(lPhaseInDaysDeviceIP days) as learningPhase FROM PATTERN[Event];
//Window to Store New Data
@Name('Named Window - whatsNewDeviceIP')
//testing this should be disabled to reset the window on a new push of the rule
@RSAPersist(serialization=Serialization.JSON)
CREATE WINDOW whatsNewDeviceIP.win:keepall().std:unique(device_ip) (device_ip string, device_host string, device_type string, lc_cid string, did string, device_class string, device_group string, alias_host string, time long, medium string);
//store in the window
@Name('Insert DeviceIP')
INSERT INTO whatsNewDeviceIP
SELECT device_ip, device_host, device_type, lc_cid, did, device_class, device_group ,cast(alias_host, string) as alias_host, time, medium FROM Event(device_ip IS NOT NULL AND device_ip NOT IN (SELECT device_ip FROM whatsNewDeviceIP));
//compare to client stored in the window
@RSAAlert
SELECT device_ip, device_host, device_type, lc_cid, did, device_class, device_group, cast(alias_host, string) as alias_host, time, medium
FROM Event(device_ip NOT IN (SELECT device_ip FROM whatsNewDeviceIP) AND device_ip IS NOT NULL
AND current_timestamp > (SELECT learningPhase FROM lPhaseDeviceIP))
OUTPUT ALL EVERY 1 hours;
the default learning period here is lPhaseInDaysDeviceIP = 1, which you should change to be more like 5-7 days to see the most common sources logging in your env. after that you would assume that the device is new or just a very very infrequent logging device. Once the window sees it once, it will keep it forever in its memory.
output looks like this (this is from an older version of the rule which didnt have some of the columns in the rule above)
and the json details look like this
The Type field will be set properly with the new rule code and triggered by the medium=32 for logs and medium=1 for packets.
It outputs all the new device ip in the last hour together in one alert.
You can report on the alerts with RE using the alerts table i think but you cant access the actual fields from the alert (which is really annoying)
Thank you Eric. Will give a try with this and get back to you.
I changed the alias_host, string to alias_host, string[]. our meta for alias_host is stored in string.
I am getting this error:-
Syntax error in module. Incorrect syntax near '[' expecting a closing parenthesis ')' but found a left angle bracket '[' at line 3 column 108, please check the select clause [//compare to client stored in the window
@RSAAlert
SELECT device_ip, device_host, device_type, lc_cid, did, device_class, device_group, cast(alias_host, string[]) as alias_host, time, medium
FROM Event(device_ip NOT IN (SELECT device_ip FROM whatsNewDeviceIP) AND device_ip IS NOT NULL
AND current_timestamp > (SELECT learningPhase FROM lPhaseDeviceIP))
OUTPUT ALL EVERY 1 hours]
If I try the same using alias_host, string , my ESA rule deployment is getting failed in enabler
What version of NW are you using?
The alias_host string works for me as I cast the vector to a string to insert it.
What is the error when you try to deploy the version I sent (before you changed the string to string[])?
When I tried to deploy it directly without changing any , My ESA rule deployment is getting failed in enabler.
The Version used is :- 11.1.0.1
What is the error when you try to deploy the rule? Go into the ESA service, locate logs and historical tab, the rule should be listed there by name with the error message. Copy the text and send it back here.
Error :-
Esper deployment of module "New event source reporting to SA" (id=5c13587845ce02f457b526d0(default)) failed. Reason: Deployment failed in module 'whatsNewDeviceIP' in module url '5c13587845ce02f457b526d0' in expression '//store in the window @Name('Insert DeviceIP') INS...(309 chars)' : Error starting statement: Failed to validate select-clause expression 'lc_cid': Property named 'lc_cid' is not valid in any stream [//store in the window
@Name('Insert DeviceIP')
INSERT INTO whatsNewDeviceIP
SELECT device_ip, device_host, device_type, lc_cid, did, device_class, device_group ,cast(alias_host, string) as alias_host, time, medium FROM Event(device_ip IS NOT NULL AND device_ip NOT IN (SELECT device_ip FROM whatsNewDeviceIP))]
That should be the vlc id in that key (lc.cid). check the ESA > Settings section and look for lc_cid, it should be listed as string.
If its not there you can choose to restart ESA to see if it loads up or you can remove all the entries for lc_cid in the rule and deploy it.
Eric
it is there I have confirmed the same
We have confirmed the same.
Esper deployment of module "New event source reporting to SA" (id=5c13587845ce02f457b526d0(default)) failed. Reason: Deployment failed in module 'whatsNewDeviceIP' in module url '5c13587845ce02f457b526d0' in expression '//store in the window @Name('Insert DeviceIP') INS...(313 chars)' : Error starting statement: Event type named 'whatsNewDeviceIP' has already been declared with differing column name or type information: Type by name 'whatsNewDeviceIP' in property 'alias_host' expected class [Ljava.lang.String; but receives class java.lang.String [//store in the window
@Name('Insert DeviceIP')
INSERT INTO whatsNewDeviceIP
SELECT device_ip, device_host, device_type, lc_cid, did, device_class, device_group ,cast(alias_host, string) as alias_host, time, medium FROM Event(device_ip IS NOT NULL AND device_ip NOT IN (SELECT device_ip FROM whatsNewDeviceIP))]
Another error I am receiving after that. Tried to change the name ,even after that the same is not working
Here is the same rule with the lc_cid removed to see if it deploys
module whatsNewDeviceIP;
/*
Rule Name: What's New for deviceIP
Author: Eric Partington
Modified: 2018-12-11
version: 3
*/
//Update learning phase to desired number of days
@Name('Named Window - learningWindowDeviceIP')
//@RSAPersist
CREATE VARIABLE integer lPhaseInDaysDeviceIP = 5;
CREATE WINDOW lPhaseDeviceIP.win:length(1) (learningPhase long);
INSERT INTO lPhaseDeviceIP
SELECT current_timestamp.plus(lPhaseInDaysDeviceIP days) as learningPhase FROM PATTERN[Event];
//Window to Store New Data
@Name('Named Window - whatsNewDeviceIP')
//testing this should be disabled to reset the window on a new push of the rule
@RSAPersist(serialization=Serialization.JSON)
CREATE WINDOW whatsNewDeviceIP.win:keepall().std:unique(device_ip) (device_ip string, device_host string, device_type string, did string, device_class string, device_group string, alias_host string, time long, medium string);
//store in the window
@Name('Insert DeviceIP')
INSERT INTO whatsNewDeviceIP
SELECT device_ip, device_host, device_type, did, device_class, device_group ,cast(alias_host, string) as alias_host, time, medium FROM Event(device_ip IS NOT NULL AND device_ip NOT IN (SELECT device_ip FROM whatsNewDeviceIP));
//compare to client stored in the window
@RSAAlert
SELECT device_ip, device_host, device_type, did, device_class, device_group, cast(alias_host, string) as alias_host, time, medium
FROM Event(device_ip NOT IN (SELECT device_ip FROM whatsNewDeviceIP) AND device_ip IS NOT NULL
AND current_timestamp > (SELECT learningPhase FROM lPhaseDeviceIP))
OUTPUT ALL EVERY 1 hours;
Hi Eric,
This got deployed in our ESA successfully. Waiting for the alerts to pop up.Once its done I will confirm you.
It will wait for 5 days to learn what is logging then alert on anything new after 5 days
Hi Eric , This is working. I could see that, after learning period if some devices that are already integrated and was not reporting starts reporting to SIEM, it also triggers an alert. Is there any way to stop that??
The rule ‘learns' for the period of time you set in the rule and whatever it sees in that period of time is added to the list of known. After that anything new that it has not seen will create an alert.
You can restart the rule with a longer learning period to catch more devices that log infrequently or change the rule logic to not alert on certain types. Other than that the rule is working as designed.
Eric
Hi,
you can create one group in the Admin/Event Sources/Manage called for example 'New Devices' with the following condition: custom1 not equals 'known', after that you need to populate all the present devices with the value 'known' in the field custom1 (you can do that for all devices in one shot).
You can create a ESA rule (or a report as well) looking for the meta device.group='New Devices', that will trigger for each new device; don't forget to enable the suppression for the alert based on device.ip to avoid to be flooded of alerts
Cheers,
Alessio
#newdevice