David Waugh

Writing a Parser to Detect SPF Fields in Email Messages

Blog Post created by David Waugh Employee on Apr 27, 2016

I had a request from a customer to parse out some messages from a mail conversation.

 

Basically the email contains the following headers:

 

Received-SPF: pass (infra1.csuk.eu.rsa.net: 192.168.123.250 is whitelisted) receiver=infra1.csuk.eu.rsa.net; client-ip=192.168.123.250; helo=ECAT.waugh.local; envelope-from=david.waugh@waugh.local; x-software=spfmilter 2.001 http://www.acme.com/software/spfmilter/ with libspf2-1.2.10;
Received: from ECAT.waugh.local ([192.168.123.250])
  by infra1.csuk.eu.rsa.net (8.13.8/8.13.8) with ESMTP id u3RCCOxW024832
  for <david.waugh2@rsa.com>; Wed, 27 Apr 2016 12:12:24 GMT
x-metascan-quarantine-id: e3c4973b-d836-4f37-a47d-62271c21a5cc
Received: from UKXXWAUGHDL1C ([152.62.229.74]) by ECAT.waugh.local with ESMTP ; Wed, 27 Apr 2016 13:12:23 +0100
From: "10.5 Test" <david.waugh@infra1.esc.ai.pri>
To: <david.waugh2@rsa.com>
References: 
In-Reply-To: 
Subject: RE: This is a test through my mail system
Date: Wed, 27 Apr 2016 13:12:23 +0100
Message-ID: <0bd101d1a07e$0e479230$2ad6b690$@waugh.local>
MIME-Version: 1.0
Content-Type: multipart/alternative;
  boundary="----=_NextPart_000_0BD2_01D1A086.700E4420"
X-Mailer: Microsoft Outlook 14.0
Thread-Index: AdGget+xlnYyEWhiQrysrAxYJ7qXxgAApK1AAAAdpuAAAAfMIA==
Content-Language: en-gb
X-Virus-Scanned: clamav-milter devel-clamav-0.98-dmgxar-126-gfde6749 at infra1
X-Virus-Status: Clean

 

The header of interest here is the one called Received-SPF:

 

I created a parser based on Detecting Sinkholed Domains With The X-Factor Parser

 

On my Packet Decoder I created a parser called SPF.parser in /etc/netwitness/ng/parsers

containing the following:

 

<?xml version="1.0" encoding="utf-8"?>

<parsers xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="parsers.xsd">

      <parser name="SPF Factor" desc="This extracts the SPF string from a Header">

                  <declaration>

                                               <token name="tXfactor" value="Received-SPF:" options="linestart" />

                                                <number name="vPosition" scope="stream" />

                                                <string name="vXfactor" scope="stream" />

                                                <meta name="meta" key="xfactor" format="Text" />

                                </declaration>

 

 

                                <match name="tXfactor">

                                                <find name="vPosition" value="&#x0d;&#x0a;" length="512">

                                                 <read name="vXfactor" length="$vPosition">

                                                  <register name="meta" value="$vXfactor"/>

                                                                </read>

                                                </find>

                                </match>

                </parser>

</parsers>

 

Whenever the Received-SPF header was seen, then the rest of the header was put into the xfactor metakey.

 

spf.png

 

If your SPF fields are from a different mail provider, then you could adjust the parser accordingly.

For example if your messages had the following header:

Authentication-Results: mx.messagelabs.com; spf=pass

 

Then the line in the parser could be changed from:

<token name="tXfactor" value="Received-SPF:" options="linestart" />

To

<token name="tXfactor" value="Authentication-Results: mx.messagelabs.com; spf=" options="linestart" />

 

If you wanted to put the result into a different metakey (for example result) then change

<meta name="meta" key="xfactor" format="Text" />

to

<meta name="meta" key="result" format="Text" />

Attachments

Outcomes