AnsweredAssumed Answered

Lua Parser Help

Question asked by RSA Admin

on Dec 19, 2014
Latest reply on Dec 30, 2014 by Christopher Ahearn

Like • Show 0 Likes0
Comment • 6

I've been looking at the parser book created by William Motley and I've been trying to use a variation of the sample parser, but I keep getting the following error.

[Parse] Meta processing failed for session with: Unmatched [ or [^ in character class declaration. The error occured while parsing the regular expression: '9]{10}/[0>>>HERE>>>'.

Looking at the book it says you can't do pattern matching with payload:find, but that is exactly what is being done with the example in the book.

Here is the parser example I'm trying to get to work. Keep in mind this is just to test, I want it to return meta for what stored into the endOfLine variable. Obviously I'll be changing this prior to production.

local getHeader = nw.createParser("noHeader", "Test Meta")

getHeader:setKeys( {nwlanguagekey.create('risk.info')})

function getHeader:httpGet(token, first, last)

local payload = nw.getPayload()

local endOfLine = payload:find("\013\010\\013\010", last + 1, last + 100)

if endOfLine then

local header = payload:tostring(last + 1, endOfLine - 1)

if header then

nw.createMeta(self.keys{"risk.info"], header)

end

noHeader:setCallbacks({

["^GET "] = getHeader.httpGet,

})

No one else has this question

Outcomes

RSA Admin

Dec 19, 2014 12:27 PM

Actually that payload:find wouldn't be looking for a pattern - it would be looking for "carriage-return line-feed carriage-return line-feed":

hexadecimal 0x0D 0x0A 0x0D 0x0A

decimal 13 10 13 10

lua string "\013\010\013\010"

payload:find("\013\010\013\010", ...)

You have an extra "\" in your example, which could be causing the error?

Actions

Christopher Ahearn

Dec 19, 2014 12:38 PM

Check that payload:find statement.

local endOfLine = payload:find("\013\010", last + 1, last + 100)

The line you had was looking for 0x0D0A0D0A(\013\010\013\010 in DECIMAL) but doing it with an extra slash in there, which may be causing some issues. The ^GET should match but looking for the \013\010 instead, would give you the following:

GET /projects/osxfuse/postdownload?source=dlp HTTP/1.1

would pull back:

/projects/osxfuse/postdownload?source=dlp HTTP/1.1

Also...the callbacks line for the TOKEN is using an incorrect name for the parser "noHeader"

local getHeader = nw.createParser("noHeader", "Test Meta")

getHeader:setKeys( {nwlanguagekey.create('risk.info')})

function getHeader:httpGet(token, first, last)

local payload = nw.getPayload()

local endOfLine = payload:find("\013\010", last + 1, last + 100)

if endOfLine then

local header = payload:tostring(last + 1, endOfLine - 1)

if header then

nw.createMeta(self.keys{"risk.info"], header)

end

getHeader:setCallbacks({

["^GET "] = getHeader.httpGet,

})

I am including an http_url parser I wrote the other day to show how I did this. I don't recommend using it unless you have a business case to do so, but feel free to use it as a reference.

Attachments

http_url.lua2.7 KB

Actions

RSA Admin

@ Christopher Ahearn on Dec 19, 2014 12:49 PM

I'm using a stand alone version of NW (9.8.5.19) to test before using the parser in SA. I get the same error when using one less \ and I get the same error when using the http_url parser provided by ahearc. So I guess there is an issue with NW maybe.

Any suggestions for a debugger I could use?

Actions

Christopher Ahearn

@ RSA Admin on Dec 19, 2014 1:20 PM

Lets try to take this offline to figure why it may not be working in your system. When we identify, we post the solution and continue the discussion. If you would like to email me directly, I may be able to help. Chris

christopher.ahearn@rsa.com

Actions

RSA Admin

@ RSA Admin on Dec 19, 2014 5:27 PM

Probably unrelated to that error message, but still worthy of note:

The book was written for 10.2.2+. There are some significant differences with lua parsing in 9.8 which may result in a lua parser doing unexpected things, or not loading at all. As well, a lua parser written for 9.8 may not work correctly (or at all) on 10.2.2+.

Actions

Christopher Ahearn

Dec 30, 2014 10:24 AM

I have been working with net_cat offline and it turned out that in some cases, the use of the Investigator thick client did not support some of the added lua functions that are part of the 10.x environment. Keep this in mind if you are using Investigator to help build some of your Lua parsers.

Chris

Actions

6 Replies

RSA Admin Dec 19, 2014 12:27 PM
Mark Correct
Correct Answer
Actually that payload:find wouldn't be looking for a pattern - it would be looking for "carriage-return line-feed carriage-return line-feed":

hexadecimal 0x0D 0x0A 0x0D 0x0A
decimal 13 10 13 10
lua string "\013\010\013\010"

payload:find("\013\010\013\010", ...)

You have an extra "\" in your example, which could be causing the error?
Like • Show 0 Likes0
Actions
Christopher Ahearn Dec 19, 2014 12:38 PM
Mark Correct
Correct Answer
Check that payload:find statement.

local endOfLine = payload:find("\013\010", last + 1, last + 100)

The line you had was looking for 0x0D0A0D0A(\013\010\013\010 in DECIMAL) but doing it with an extra slash in there, which may be causing some issues. The ^GET should match but looking for the \013\010 instead, would give you the following:

GET /projects/osxfuse/postdownload?source=dlp HTTP/1.1

would pull back:

/projects/osxfuse/postdownload?source=dlp HTTP/1.1

Also...the callbacks line for the TOKEN is using an incorrect name for the parser "noHeader"

local getHeader = nw.createParser("noHeader", "Test Meta")
getHeader:setKeys( {nwlanguagekey.create('risk.info')})

function getHeader:httpGet(token, first, last)
    local payload = nw.getPayload()
    local endOfLine = payload:find("\013\010", last + 1, last + 100)
    if endOfLine then
        local header = payload:tostring(last + 1, endOfLine - 1)
        if header then
            nw.createMeta(self.keys{"risk.info"], header)
        end
    end
end
getHeader:setCallbacks({
    ["^GET "] = getHeader.httpGet,
})

I am including an http_url parser I wrote the other day to show how I did this. I don't recommend using it unless you have a business case to do so, but feel free to use it as a reference.
Attachments
- http_url.lua2.7 KB
Like • Show 0 Likes0
Actions
- RSA Admin @ Christopher Ahearn on Dec 19, 2014 12:49 PM
  Mark Correct
  Correct Answer
  I'm using a stand alone version of NW (9.8.5.19) to test before using the parser in SA. I get the same error when using one less \ and I get the same error when using the http_url parser provided by ahearc. So I guess there is an issue with NW maybe.
  
  Any suggestions for a debugger I could use?
  Like • Show 0 Likes0
  Actions
  - Christopher Ahearn @ RSA Admin on Dec 19, 2014 1:20 PM
    Mark Correct
    Correct Answer
    Lets try to take this offline to figure why it may not be working in your system. When we identify, we post the solution and continue the discussion. If you would like to email me directly, I may be able to help. Chris
    
    christopher.ahearn@rsa.com
    Like • Show 0 Likes0
    Actions
  - RSA Admin @ RSA Admin on Dec 19, 2014 5:27 PM
    Mark Correct
    Correct Answer
    Probably unrelated to that error message, but still worthy of note:
    
    The book was written for 10.2.2+. There are some significant differences with lua parsing in 9.8 which may result in a lua parser doing unexpected things, or not loading at all. As well, a lua parser written for 9.8 may not work correctly (or at all) on 10.2.2+.
    Like • Show 0 Likes0
    Actions
Christopher Ahearn Dec 30, 2014 10:24 AM
Mark Correct
Correct Answer
I have been working with net_cat offline and it turned out that in some cases, the use of the Investigator thick client did not support some of the added lua functions that are part of the 10.x environment. Keep this in mind if you are using Investigator to help build some of your Lua parsers.

Chris
Like • Show 0 Likes0
Actions

Lua Parser Help

Outcomes

Attachments

Related Content

Recommended Content