RSA Admin

Help to make Fortiweb parser

Discussion created by RSA Admin Employee on Nov 26, 2013
Latest reply on Dec 10, 2013 by RSA Admin

Hi everybody !

 

We have to make a parser for the appliance Fortiweb 5.0.2 (Web Application Firewall).

 

I began to build it with ESI but I don't succeed to understand everything, so I would need some help/advice

 

Extract of brut log :

 

Oct 22 19:29:24 [10.200.190.166] date=2013-10-22 time=19:35:12 devname=ap-b1-fortinet-1 log_id=20000002 msg_id=000000124644 type=attack subtype="none" pri=alert device_id=FV-1KC3R13700032 timezone="(GMT+1:00)Brussels,Copenhagen,Madrid,Paris"trigger_policy="" severity_level=Low proto=tcp service=http action=Alert_Deny policy="policy-wildcard-extranet" src=XXX.XX.XXX.XXX src_port=56962 dst=10.200.190.65 dst_port=80 http_method=get http_url="/w00tw00t.at.ISC.SANS.DFind:)" http_host="none" http_agent="none" http_session_id=none msg="HTTP Host Violation"

 

Orange -> header part

Green -> payload part

 

 

For Message_ID, I chose the number of the variable log_id, so in this exemple the Message_ID is 20000002.


Header :


<HEADER

        id1="0001"

        id2="0001"

        content="date=&lt;hdate&gt; time=&lt;htime&gt; devname=&lt;hdevice&gt; log_id=&lt;messageid&gt; msg_id=&lt;hmessageid&gt; type=&lt;htype&gt; subtype=&lt;hsubtype&gt; pri=&lt;hpriority&gt; device_id=&lt;hdeviceid&gt; timezone=&lt;hzone&gt; &lt;!payload&gt;"/>



Message for the extract of brut log above :


<MESSAGE

        level="3"

        parse="1"

        parsedefvalue="1"

        tableid="75"

        id1="HTTP_Host_Violation"

        id2="20000002"

        eventcategory="1204000000"

        content="trigger_policy=&quot;&quot; severity_level=&lt;severity&gt; proto=&lt;protocol&gt; service=&lt;network_service&gt; action=&lt;action&gt; policy=&lt;policyname&gt; src=&lt;saddr&gt; src_port=&lt;sport&gt; dst=&lt;daddr&gt; dst_port=&lt;dport&gt; http_method=&lt;web_method&gt; http_url=&lt;web_query&gt; http_host=&lt;hostid&gt; http_agent=&lt;user_agent&gt;&quot; http_session_id=none msg=&lt;message_body&gt;"/>



This couple header/message works well with ALL the logs which have the same log_id :-)



Now, take this extract of brut logs :


Oct 22 18:05:52 [10.200.190.166] date=2013-10-22 time=18:11:40 devname=ap-b1-fortinet-1 log_id=20000008 msg_id=000000124625 type=attack subtype="none" pri=alert device_id=FV-1KC3R13700032 timezone="(GMT+1:00)Brussels,Copenhagen,Madrid,Paris" trigger_policy="" severity_level=Low proto=tcp service=https action=Alert_Deny policy="policy-wildcard-extranet" src=XX.XXX.XXX.XX src_port=28018 dst=10.200.185.249 dst_port=80 http_method=get http_url="/Admin/redirectmail04.nsf/Login.js" http_host="mailagent.extranet.blabla.com" http_agent="Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Firefox/24.0" http_session_id=AT3BN5IXDY2WQEOAP3CEUNDDENJJITRJ msg="url-deny-mailagent : URL Access Violation"

 

This log have the same fields than the first log, so I can use the existing header right ? Only the log_id change (that's normal, it's another type of attack message)

 

I just have to create a new message and it should work right ?

 

Header (the same than before) :


<HEADER

        id1="0001"

        id2="0001"

        content="date=&lt;hdate&gt; time=&lt;htime&gt; devname=&lt;hdevice&gt; log_id=&lt;messageid&gt; msg_id=&lt;hmessageid&gt; type=&lt;htype&gt; subtype=&lt;hsubtype&gt; pri=&lt;hpriority&gt; device_id=&lt;hdeviceid&gt; timezone=&lt;hzone&gt; &lt;!payload&gt;"/>

 


Message (with the log_id 20000008)  :


<MESSAGE

        level="3"

        parse="1"

        parsedefvalue="1"

        tableid="75"

        id1="URL_Access_Violation"

        id2="20000008"

        eventcategory="1201000000"

        content="trigger_policy=&quot;&quot; severity_level=&lt;severity&gt; proto=&lt;protocol&gt; service=&lt;network_service&gt; action=&lt;action&gt; policy=&lt;policyname&gt; src=&lt;saddr&gt; src_port=&lt;sport&gt; dst=&lt;daddr&gt; dst_port=&lt;dport&gt; http_method=&lt;web_method&gt; http_url=&lt;web_query&gt; http_host=&lt;hostid&gt; http_agent=&lt;user_agent&gt; http_session_id=RT1JSEM1QJ1K9SXOK9ALUS8QBHKPDSGJ msg=&lt;message_body&gt;"/>

 

 

This couple header/message works well with ONLY ONE message which have the same log_id

 

Take this other  line of log :

 

Oct 23 12:00:55 [10.200.190.166] date=2013-10-23 time=12:06:44 devname=ap-b1-fortinet-1 log_id=20000008 msg_id=000000124681 type=attack subtype="none" pri=alert device_id=FV-1KC3R13700032 timezone="(GMT+1:00)Brussels,Copenhagen,Madrid,Paris" trigger_policy="" severity_level=Low proto=tcp service=https action=Alert_Deny policy="policy-wildcard-extranet" src=XX.XXX.XXX.XX src_port=41369 dst=10.200.190.65 dst_port=443 http_method=get http_url="/names.nsf" http_host="mail04.extranet.blabla.com" http_agent="Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.3072" http_session_id=none msg="url-deny-mail0X : URL Access Violation"

 

She's the same right (same log_id) ? But she's not well parsed...

 

Have you an idea of what's going wrong ?


EDIT : Variable session_id was missing in the message definition, after have integrated this variable it works well :-)


OK for the first problem, but I 'm still blocked for the second :

 

 

 

Second problem : For some logs, I can't use only the field log_id as message_id because one log_id means diffferent security message. For exemple the log_id 20000010 means these messages :


 

[Signatures name: WebMail] [main class name: Information Disclosure] [sub class name: Application Availability/Errors]: 080080003
[Signatures name: WebMail] [main class name: Generic Attacks] [sub class name: Email Injection]: 050110001
[Signatures name: WebMail] [main class name: Generic Attacks] [sub class name: HTTP Response Splitting]: 050130001
[Signatures name: WebMail] [main class name: Cross Site Scripting]: 010000107
[Signatures name: WebMail] [main class name: Generic Attacks] [sub class name: Command Injection]: 050050052
[Signatures name: WebMail] [main class name: Generic Attacks] [sub class name: PHP Injection]: 050080033
[Signatures name: WebMail] [main class name: Generic Attacks] [sub class name: HTTP Response Splitting]: 050130001
[Signatures name: WebMail] [main class name: Information Disclosure] [sub class name: Microsoft Office Document Properties Leakage]: 080050002
[Signatures name: WebMail] [main class name: Known Exploits] [sub class name: Struts 2 Vulnerability]: 090390001
[Signatures name: WebMail] [main class name: Information Disclosure] [sub class name: CF Source Code Leakage]: 080060001
[Signatures name: WebMail] [main class name: Generic Attacks] [sub class name: OS Command Injection Attacks]: 050010001

 

So I have to create a Message_id ; I tried log_id & the number at the end of the log, for exemple with this line of log :

 

Oct 23 13:54:57 [10.200.190.166] date=2013-10-23 time=14:00:46 devname=ap-b1-fortinet-1 log_id=20000010 msg_id=000000124685 type=attack subtype="none" pri=alert device_id=FV-1KC3R13700032 timezone="(GMT+1:00)Brussels,Copenhagen,Madrid,Paris" trigger_policy="" severity_level=High proto=tcp service=https action=Alert policy="policy-wildcard-extranet" src=XX.XXX.XXX.XXX src_port=56713 dst=10.200.185.250 dst_port=80 http_method=post http_url="/traveler" http_host="m-services.extranet.blabla.com" http_agent="Lotus Traveler Android 9.0" http_session_id=EHVJZ0KJNI76A8HNPKOVVANLTW3OKFUP msg="[Signatures name: WebMail] [main class name: Cross Site Scripting]: 010000107"

 

Message_id is [20000010][010000107], but with this header, the header of logs is not parsed

 

 

Any help/advice is welcome, thx in advance :-)


Outcomes