Developing complex Suricata rules with Lua – part 2

In part 1 we showed a Lua program to have Suricata detect PDF documents with obfuscated /JavaScript names. In this second part we provide some tips to streamline the development of such programs.

When it comes to developing Lua programs, Suricata is not the best development environment. The “write code & test”-cycle with Suricata can be quite tedious. One of the reasons is that it takes time. It can take 1 minute or more to start Suricata with the new rule, and have it process a test pcap file. And if there are errors in your Lua script, Suricata will not be of much help to identify and fix these errors.

Inspired by Emerging Threats Lua scripts, we adopted the following development method:

Test the script with a standalone Lua interpreter, and move to Suricata for the final tests.

This is one of the reasons why, in part 1, we put the logic of our test in function PDFCheckName which takes a string as input and is called by the match function. By doing this, we can also call (and test) the function from other functions with a standalone Lua interpreter as shown below:

function Test()
    print()
    print(PDFCheckName("testing !!!", true))
    print()
    print(PDFCheckName("testing /JavaScript and more /J#61vaScript !!!", true))
    print()
    print(PDFCheckName("testing /JavaScript and !!!", true))
    print()
    print(PDFCheckName("testing /J#61vaScript !!!", true))
    print()
end

This Test function calls PDFCheckName with different strings as input. We also added extra print statements to the function (see complete source code below), which are activated by the second argument of function PDFCheckName. This boolean argument, bVerbose, adds verbosity to our function when the argument is true.

We can load the Lua program in a Lua interpreter, and then call function Test. One way to do this is type command “lua -i PDFCheckName.lua”, and then type Test() at the Lua prompt. This can all be scripted in a single command like this:

echo Test() ¦ lua -i PDFCheckName.lua

With the following result:

This “code & run”-cycle is faster than using Suricata, and can be more verbose. Of course, you can also do this with an IDE like Eclipse.

We also added a function TestFile that reads a file (the PDFs we want to test), and then calls PDFCheckName with the content of the PDF file as the argument:

This produces the following output:

Being able to test a PDF file directly is also a big advantage, compared to having to create a PCAP file with a http request downloading the PDF file to test.

Conclusion

By using functions and a standalone Lua interpreter, we can significantly improve the development process of Lua programs for Suricata.

Code

-- 2017/02/20

-- echo Test() | lua53.exe -i test.lua
-- echo TestFile() | lua53.exe -i test.lua javascript.pdf

tBlacklisted = {["/JavaScript"] = true}

function PDFCheckName(sInput, bVerbose)
    if bVerbose then
        print('sInput: ' .. sInput)
    end
    for sMatchedName in sInput:gmatch("/[a-zA-Z0-9_#]+") do
        if bVerbose then
            print('sMatchedName: ' .. sMatchedName)
        end
        if sMatchedName:find("#") then
            local sNormalizedName = sMatchedName:gsub("#[a-fA-F0-9][a-fA-F0-9]", function(hex) return string.char(tonumber(hex:sub(2), 16)) end)
            if bVerbose then
                print('sNormalizedName: ' .. sNormalizedName)
            end
            if tBlacklisted[sNormalizedName] then
                if bVerbose then
                    print('Blacklisted!')
                end
                return 1
            end
        end
    end
    if bVerbose then
        print('Not blacklisted!')
    end
    return 0
end

function init(args)
    return {["http.response_body"] = tostring(true)}
end

function match(args)
    return PDFCheckName(tostring(args["http.response_body"]), false)
end

function Test()
    print()
    print(PDFCheckName("testing !!!", true))
    print()
    print(PDFCheckName("testing /JavaScript and more /J#61vaScript !!!", true))
    print()
    print(PDFCheckName("testing /JavaScript and !!!", true))
    print()
    print(PDFCheckName("testing /J#61vaScript !!!", true))
    print()
end

function TestFile()
    local file = io.open(arg[1])
    print()
    print(PDFCheckName(file:read("*all"), true))
    file:close()
end