Netwitness: How to write a parser for attachments?

[CrisRhea]

How does Netwitness parse and set "filetype"?  From what I've read, parsers can act on the whole payload, rather than iterating on each "attachment".

For various protocols (e.g., HTTP, FTP, SMTP, ...), whole files can be transmitted and are encapsulated by the protocol.

I'd like to be able to create meta data for encrypted RAR files (these can be determined by a header and flag at specific offsets in the file).

Is there a way to write a parser that can look at each "attachment" (protocol independent) to be able to set this meta data?

--- Cris

[RSAAdmin]

Please consider moving this question as-is (no need to recreate) to the proper forum for maximum visibility.  Questions written to the users' own "Discussions" space don't get the same amount of attention and questions can go unanswered for a long time. 

You can do so by selecting "Move" under ACTIONS along the upper-right.  Then search for and select: "RSA Security Analytics" which would be the most relevant for this question.

[RSAAdmin · Accepted solution]

You can do something like that, where you replace Rar!� with the signature of encrypted RAR

<parser name="detect_rar" desc="finds rar content">
    <declaration>
      <token name="magic" value="&#x52;&#x61;&#x72;&#x21;&#x1a;&#x07;&#x00;" />
      <meta name="meta" key="filetype" format="Text" />   
    </declaration>
    <match name="magic">
      <register name="meta" value="RAR Archive" />   
    </match>
</parser>

or if you prefer Lua

local detectrar=nw.createParser("detectrar", "finds rar content")
detectrar:setKeys( {nwlanguagekey.create('filetype')})
function detectrar:onToken(token, first, last)
   nw.createMeta(self.keys['filetype'], "RAR Archive" )  
end
local callbacksTable = { ["Rar!�"]= detectrar.onToken  }
detectrar:setCallbacks(callbacksTable)

[CrisRhea]

The above appears to look for a certain string *anywhere* in a packet.  How does one specify the positional information within a file? How does one do bitwise operations on a specific byte?

From what I understand, an encrypted RAR file has a "magic number" of 526172211a0700 in the first 14 bytes of the file, then skips 6 bytes, then sets the 128 bit in the next byte.

I've never seen example code/documentation that shows more complex operations.

[RSAAdmin]

You can't restrict the search to "at the beginning of file", and you don't need to. The magic number is very specific and you won't get false positives.

To do the offset part, you simpy forward  the desired number of bytes, read the value and compare it to the flag. The documentation on flex parsers is quite good, check out the admin guide. If you prefer Lua, have a look at my NTLM parser which use offsets too.

Edit:

to skip 6 bytes, add

<move value="6">

to read the next byte

<read name="flag" length="1">

to get the last bit

<and name="flag" value="128">