David Waugh

LUA Parser To Generate Meta based on Time Within a message

Discussion created by David Waugh Employee on Feb 23, 2016
Latest reply on Feb 23, 2016 by David Waugh

I had a customer with an interesting use case.

Basically they were receiving Logs containing time stamps and they wanted to generate an alert based on this timestamp. The logs were from CyberArk and looked as follows:

 

1 2016-02-15T11:17:35Z SERVER1 %CYBERARK: MessageID="109";Version="9.10.0000";Message="Get File Request";Issuer="USER1";Station="127.0.0.1";File="Root\MASTER1 CA_TRL";Safe="Safe1";Location="";Category="";RequestId="2";Reason="Server patching. RFC 123456 [From 15/02/2016 08:00:00 to 15/02/2016 17:00:00 (GMT+1.00) multiple operations]";Severity="Info";GatewayStation="127.0.0.1";TicketID="";PolicyID="Standard";UserName="MASTER1 CA_TRL";LogonDomain="";Address="";CPMStatus="";Port="";Database="";DeviceType="Operating System";ExtraDetails=""

 

When the log was parsed meta would be generated into the following meta keys:

 

event.desc ='Server patching. RFC 123456 [From 15/02/2016 08:00:00 to 15/02/2016 17:00:00 (GMT+1.00) multiple operations]'

 

hdatetime ='2016-02-15T11:17:35Z'

 

What the customer wanted to do was generate an alert when the hdatetime value was greater than the time after the to value in the event description above.

 

In the following parser I always assume that the hdatetime field is in UTC (indicated by Z at the end of the timestamp). I also assume that the time in the Event Description is always GMT+1 = UTC plus one hour.

 

In order to compare times, I convert the time strings to Unix Epoch time. Normally LUA os functions could be used for this, but as these are restricted in Security Analytics, I had to borrow an algorithm from someone else.

 

Here is the LUA Parser

 

local lua_CyberArkTime= nw.createParser("CyberArkTime", "Looks to See if the time in a CyberArk message has passed")

--[[

   

    DESCRIPTION

    Takes a log message from a CyberArk where the Event Description contains a FROM and a TO time such as

  Server patching. RFC 123456 [From 15/02/2016 08:00:00 to 15/02/2016 17:00:00 (GMT+1.00) multiple operations]

    We compare if the TO Time has already passed and if it has write Meta into the Alert Field

 

  We assume that the Time in the Event Description is always GMT+1

  We assume that the Time in the Header is always UTC

 

    AUTHOR

    david.waugh2@rsa.com

    VERSION

       1.0 Inital Parser Request

 

    DEPENDENCIES

 

     Make Sure that in your /etc/netwitness/ng/envision/etc/table-map-custom.xml that      the hdatetime key is set to either None or Transient

        <mapping envisionName="hdatetime" nwName="hdatetime" flags="None"      format="Text"/>

--]]

 

function days_from_epoch(y, m, d)

-- This algorithm came from

-- http://howardhinnant.github.io/date_algorithms.html

--

    if m <= 2 then

      y = y - 1

      m = m + 9

    else

      m = m - 3

    end

    local era = math.floor(y/400)

    local yoe = y - era * 400                                           -- [0, 399]

    local doy = math.modf((153*m + 2)/5) + d-1                          -- [0, 365]

    local doe = yoe * 365 + math.modf(yoe/4) - math.modf(yoe/100) + doy -- [0, 146096]

    return era * 146097 + doe - 719468

end

 

 

function epochtime(year,month,day,hour,minute,second)

  local days=days_from_epoch(year, month, day)

  return (days*86400 + hour*60*60 +minute*60 + second)

end

 

-- Write meta into the following meta key(s)

lua_CyberArkTime:setKeys({

  nwlanguagekey.create("alert",nwtypes.Text),

})

 

 

 

function lua_CyberArkTime:sessionBegin()

  HeaderTime = nil

  EventTime = nil

  devicetype = nil

end

 

 

function lua_CyberArkTime:CheckDevice(index,dtype)

  if dtype == 'cyberark' then

  devicetype = dtype

  end

end

 

 

function lua_CyberArkTime:getEventTime(index,myEventTime)

  if devicetype then

  EventTime=myEventTime

  --nw.logInfo("EventTime ")

  --nw.logInfo(EventTime)

  end

 

end

 

 

function lua_CyberArkTime:getHeaderTime(index,myHeaderTime)

  if (devicetype) then

  HeaderTime=myHeaderTime

  --nw.logInfo("HeaderTime ")

  --nw.logInfo(HeaderTime)

  end

 

end

 

 

function lua_CyberArkTime:sessionEnd()

 

 

  if devicetype then

  --nw.logInfo("SessionEnd:EventTime ")

  --nw.logInfo(EventTime)

  --nw.logInfo("SessionEnd:HeaderTime ")

  --nw.logInfo(HeaderTime)

  if (EventTime) then

       if (HeaderTime) then

 

            ---Do All The Work in Here

            local FromDate,ToDate =string.match(EventTime,"From (%d%d/%d%d/%d%d%d%d %d%d:%d%d:%d%d) to (%d%d/%d%d/%d%d%d%d %d%d:%d%d:%d%d) (.*)")

            local EventDay,EventMonth,EventYear, EventHour,EventMinute,EventSecond=string.match(ToDate,"(%d+)/(%d+)/(%d+) (%d+):(%d+):(%d+)")

  local EventEpochTime=-3600+epochtime(tonumber(EventYear),tonumber(EventMonth),tonumber(EventDay),tonumber(EventHour),tonumber(EventMinute),tonumber(EventSecond))

  --nw.logInfo("EventEpochTime " .. EventEpochTime)

 

  local HeaderYear,HeaderMonth,HeaderDay, HeaderHour,HeaderMinute,HeaderSecond= string.match(HeaderTime,"(%d+)%-(%d+)%-(%d+)T(%d+):(%d+):(%d+)Z")

  local HeaderEpochTime =epochtime(tonumber(HeaderYear),tonumber(HeaderMonth),tonumber(HeaderDay),tonumber(HeaderHour),tonumber(HeaderMinute),tonumber(HeaderSecond))

  --nw.logInfo("HeaderEpochTime " .. HeaderEpochTime)

 

  if HeaderEpochTime > EventEpochTime then

  --nw.logInfo("CyberArk: Time Expired: ")

  nw.createMeta(self.keys["alert"], "CyberArk: Time Expired")

  end

  end

  end

  end

end

 

 

 

 

-- declare what tokens and events we want to match

lua_CyberArkTime:setCallbacks({

    [nwevents.OnSessionBegin] = lua_CyberArkTime.sessionBegin,

    [nwlanguagekey.create("device.type", nwtypes.Text)] = lua_CyberArkTime.CheckDevice,

    [nwlanguagekey.create("event.desc", nwtypes.Text)] = lua_CyberArkTime.getEventTime,

    [nwlanguagekey.create("hdatetime", nwtypes.Text)] = lua_CyberArkTime.getHeaderTime,

    [nwevents.OnSessionEnd] = lua_CyberArkTime.sessionEnd,      

})

Attachments

Outcomes