For such rulesets, Flint can appear to be unresponsive due to the massive amount of processing required. The benefit of this feature is that it lets you know Flint is still hard at work, and which stage it’s currently working in.
As in all previous versions of Flint, you will be redirected to the overview page following processing. Although the content on this page has not been modified in v.1.1.0, you may notice the url has changed to include the sha of the current firewall. This enables bookmarking the test results of your firewall configurations so you can view them later without having to re-process.
Additionally, the UI for running new reports has been changed slightly. The new report popup has been embedded into the new report page, along with a small set of instructions on how to load rules into Flint.
At the time of this writing, version 1.1.0 is only available through out git repository. Check out http://runplaybook.com/flint for more detailed instructions on how to install Flint via git. For those of you interested in the Flint VM, we should have one for 1.1.0 available by the end of the week.
]]>
You can see it all at: http://runplaybook.com/flint
This release of Flint includes:
We also want to give a shout out to the following users who have helped us find and kill bugs:
This is an excellent little video summary of the competition:
CSAW 2009 - The Judge’s Perspective from NYU-Poly on Vimeo.
The competition portion of the event is loosely modeled after the Defcon Capture the Flag competitions of recent years. This past year, the CSAW-CTF competition brought talent from all around the world together during the National Cyber Security Awareness Month which is celebrated at NYU:Poly as CSAW (Cyber Security Awareness Week)
In 2008 when I was asked by Dan to help out I was very eager. That year, my involvement with CSAW-CTF was limited to just a few small challenges (instead I focused on teaching a Reverse Engineering class at NYU that year which Aaron Portnoy took over for this year). This year (2009) however, I went a little more “head down” and spent a bit more time designing, coding, and building reverse engineering challenges for the CSAW-CTF. This blog post is about those challenges.
Times have changed, and these competitions have evolved and matured much like the rest of our “industry”. They are no longer about who can whip out their graffiti’d laptops, mount their sticker laden Zip drives from linux, and use gcc to compile stuff from their packetstorm, rhino9, rootshell.org, or hack.co.za archives. Times have changed. The games are different now and don’t so much deserve to be the target of smug self-righteous vitriol from jaded old security folks.
This became really obvious to me in 2005 when Kenshoto premiered a new kind of Capture the Flag for Defcon that year. Players that year said that they could really tell that despite Kenshoto’s showmanship (and at times, ostentatiousness), all the members (mostly invisig0th, who is responsible for really spear-heading the whole effort) really loved what they did and wanted to share in the fun of software exploitation and reversing. All the challenges were custom made and well documented. They required source auditing skill, custom shellcode payloads, and reverse engineering skill. During the qualifying rounds (a month or so before Defcon) there was also an interactive scoreboard, MUD and IRC channel for all the players and organizers to use. This drew players in to world of the game a bit more and allowed teams and organizers to “hang out” all in one place and keep up-to-date on scores and rankings.
That year, and in the following years following, some big hitters came out to play: Mark Dowd and John McDonald (who ruxxed the quals round in 2005); Chris Eagle and the Naval Post-Graduate Warfare College; Giovanni Vigna and the UCSB team; the SAIC team, the Novell Team, some great international teams; and number of Fortune 500 technology companies (and government agencies) that wished to remain nameless ;-) Kenshoto and John Viega (a Kenshoto member) even got an article in Popular Science that year.
Having been a member of Kenshoto, I really wanted to bring a bit of that spirit to the NYU:Poly CSAW-CTF. The NYU-Poly competitions had a number of different competition categories which included Web Security, Application Security, Forensics, and Embedded Systems.
Myself, Dino Dai Zovi (who blogged briefly about his CSAW experience and even talked about it on the local news), and Dean DeBeer were all responsible for judging and organizing the “Application Security” parts of the competition. Dino Dai Zovi (an alumnus of Matasano and fellow New Yorker) designed all the Software Exploitation challenges and I designed all the Reverse Engineering challenges.
You can read about the full format of the competition online at the csaw website, but essentially the competition happened in two waves: (1) a qualifying round where anyone could participate, and (2) a final round where all the top teams that qualified and were enrolled at universities within the US would be flown to New York and put up in a fancy hotel to participate in the final round held on the NYU:Poly campuses.
The Scoring System:
First things first. How to keep track of scores? A stupid web app? The honor system? Email based submissions?Due to the sheer number of participants I decided that any kind of manual or qualitative scoring system would be a nightmare. Reverse engineering challenges seemed to lend themselves nicely to “token” based scoring so I decided to go that route and to build an automated scoring system. The scoreboard was written in Python and ran as the default shell for an “open” user account to be shared amongst all participants. Competitors were dropped into the BBS-style scoreboard system when they ssh’d into the box.
They would then authenticate to the scoreboard system itself (which maintained it’s own credentials).
The scoring system internally used a client/server model. The server ran at a higher privilege level than the client and communication between the two was all done using python RMI. This was intended to localize the damage if anyone had a clever way of breaking out of the python interpreter. If they were, it would be harder to directly access the scoring database files directly, because all those operations were performed by the server on behalf of the client who made requests via RMI. The scoring system allowed competitors to check scores, read about flags and challenges, and submit/check their answers.
Other than that there isn’t anything really noteworthy about the scoring system. Source code for that is all available online here. Now let’s get on to the Reversing challenges.
For the qualifying round of the competition I put together 8 challenges. The 9th and final challenge would be completely different than all the others and was saved for the Final Round which took place on the NYU:Poly campuses. Each of the 8 challenges were framed as forensics based malware challenges, each having its own fictional “backstory”. Every challenge was distributed in binary form and it was up to the competitor to extract tokens from the binary by reversing or otherwise modifying execution of the executable. Each binary employed what I hoped to be progressively more difficult anti-debugging and anti-reversing techniques and combinations thereof. Let’s individually review each technique before summarizing how they were combined to form each challenge.
The Tricks and Techniques:
Premature Exit:
Before a critical call is made a call to ExitProcess is made. Thus normal execution of the binary will seem to just cause the program to run and exit. Debugging the process however will reveal that more code exists after the call to ExitProcess.
BeingDebugged (custom):
Traditionally this technique involves having the process make a call to IsDebuggerPresent() to see if it is being debugged. This is the standard way to do this using the Microsoft APIs. However many anti-anti-debugging patches for debuggers such as OllyDBG have modules that will intercept this call by setting a breakpoint on it down in kernel32.dll. So instead of using the Microsoft API I wrote some inline assembly that pulled this boolean value directly out of PEB. This way, the only way to catch that “Being Debugged” was being checked was to set a Break-on-Access in PEB itself, something I don’t think any of the common debugger plugins do. My hope was that this would make the process a bit more manual for the competitors.
String Obfuscation:
Many strings that get statically compiled into binaries sit in the BSS section of the executable. You can use canned tools like unix “strings”, IDA, or even simple shell scripts to pull this stuff out. Even simple string obfuscation like XOR is easily spottable in disassembly. So what I wanted to do was to write a string obfuscater that would “raise the bar” high enough that reversing the scheme itself would be more difficult than doing the challenge itself, so I settled on some simple encryption. To achieve this I was originally going to use RC4 and execute it as a shellcode buffer, so I wrote RC4 in NASM.
The reason I chose to do this as shellcode originally is because I thought it would be easier to polymorph the algorithm as shellcode than it was to polymorph it as compiled C code. I eventually found out a way to achieve this with C code (as you will read below) and abandoned the shellcode idea. Instead I wrote (*ahem* stole) an RC4 implementation written in C that I vainly called sa7ori-encode. I generously borrowed from the infamous cipherpunks mailinglist ARC4 leak to write this code. With the crypto scheme implemented, the challenge then became how to make the algorithm look “different each time”. For this I used a combination of Inlining, Call Obfuscation, “Polymorphic” C Code Blobs, and “Here Strings” (all described a bit more below).
Inlining:
Inlining (for those of you who are un-familar) is a way for a C developer to specify (using the compiler directive __forceinline) that when a function is called that execution is not redirected into it using the traditional “call” instruction.
Instead, with inlining wherever a function would normally have been called, the full body of code for that function is “pasted” into the place where it is referenced. This is extremely inefficient because it makes binaries MUCH larger but for that reason it also makes reversing a bit more painful.
Timing Tricks:
I had always heard about these timing tricks but the most interesting use of them was in Kostya Kortchinsky and Fabrice Desclaux’s EXCELLENT presentation entitled Vanilla Skype where they discussed some of the anti-debugging tricks they encountered while completely reverse engineering Skype. (This, by the way, among my favorite presentations of all time.) The idea is simple, x86 CPUs since the Pentium support an instruction called RDTSC which reads a time value from a timer that started inside the CPU when it was powered on. This can be used as a “stopwatch” to measure the time delta between bits of code.
This is particularly useful if you want to detect if there is a debugger attached. By starting the proverbial “stopwatch”, then throwing an exception that only a debugger can handle, and then checking the “stopwatch” after the exception is handled, a program can essentially time how long the debugger (and thusly the human operating the debugger) took to handle the exception. By setting the threshold for this time impossibly low for a human to react to you can effectively detect if there is a “man in the loop”. The RDTSC instruction is wrapped by the Microsoft API function “GetTickCount()”.
Self-Caught Exceptions (to cause code branching):
This technique involves using registered exception handlers to redirect execution in your application. Instead of just calling a function by name, you register the function you wish to call as an exception handler, and then trigger an exception. If a debugger is attached, it catches the exception before the exception handler (which is actually a function critical to the normal operation of the program) executes and thus the program does not continue properly when the debugger has handled the exception and resumed execution.
Debugger Required Exceptions:
This simple technique caused an exception (division by zero, “call 0”, etc.) that required the reverser to have his debugger attached so he could jump over or NOP out the exception. This is effectively like the premature exit. In the VM supplied to the competitors (which had remote Kernel debugging enabled) this had the unintended consequence of freezing the entire VM if they didn’t have a debugger attached ;-).
Call Obfuscation (AntiDisassembly, kinda):
To obscure the area around a function call I decided to use a combination of inline assembly and a property of C include files. C include files (.h) files essentially just “paste” in the code from the .h wherever it is referenced in the C code. In this case I put inline assembly in the .h.
The inline assembly uses the _emit macro to build a large blob of seemingly random bytes that could not be disassembled properly. Using the __COUNTER__ Visual Studio Macro. Stephen Lawler gave me the idea for this trick, unfortunately it can only be used once per function. You can see an example of how I implemented it here. This code was further modified to make it “polymorphic” which is discussed below.
Polymorphic C code (kinda….well, ok not really :-):
My CS vernacular stinks, so I couldn’t really think of a way to describe what I was doing here. To replace my idea to use polymorphic shellcode as the string obfuscater I instead found (what I think to be) a neat way to do this in C. The main use for this was to make the blob generated by the “Call Obfuscation” trick (see above) pseudo-random and thus a little more difficult to “eyeball” in disassembly. Visual Studio has a very useful feature that allows you to create variables at compile-time with the /D compile option for cl.exe. This will create a variable with a value specified (at compile time) that is then accessible from within code. For example, take the line of C code:
#define MYCTR (__COUNTER_ _ + MYOFFSET)
The variable MYOFFSET is not defined anywhere within the source code. Instead it is actually defined at compile time with this:
cl /nologo /Ob2 /Z7 /DMYOFFSET=251 6.cpp /c
I then modified the “Call Obfuscation” code (from the previous section) to use this trick. This made the anti-disassembling blob “polymorphic” between compiles as long as the seed value for “MYOFFSET” was modified accordingly.
;)
Annotated visual of polymorphs between compiles
In the end it came out something like this which could be used in the code like so: #include “evil.h” to screw up disassembly of the compiled code. An annotated visual screenshot of this process is here.
TLS Execution:
I learned about this Thread Local Storage (TLS) execution trick some years ago in one of Ilfak’s blogposts. It is a way (on Windows) to have a piece of your code execute before the main part of your executable (the entry point) is jumped into. The impact of this is that the code will even execute before a debugger can attach to it even if it starts the target program! Combined with anti-debugging tricks this can be really useful. It requires using an awesome custom linker though.
Use of “Here Strings”:
My name for these probably sucks, but once again language fails me and I don’t know the correct term for this (if they even have one). In languages like Ruby or Python they are called “here strings” or “doc strings”. However, in languages like C even static strings get squirreled away in the BSS and then a pointer to them is used in the code. What I wanted was a way to reference a string buffer DIRECTLY where it was or as a really small relative offset. The reason I wanted this was to change the way the call into the string obfuscation functions looked, and to confuse disassembly. To achieve this I used a combination of inline assembly and _emit bytes. I wrote a small python script to take a string as input.It would generate a random key for my encryption scheme and then generate the ciphertext (based on the plaintext input and the generated key).
;)
It would then generate a bit of inline assembly that represented the string as inline asm _emit bytes that exposed pointers to these buffers in the ebx and eax registers respectively. I could then just cut and paste the output of this Python script into my C code and use the “strings” by referencing pointer variables that referenced labels in the inline assembly. Instead of defining my ciphertext and key buffers like this (as I would normally in C):
I could access them as inline assembly “here strings” like this:
If you dont get what I mean, sample output of this Python script is here.
The Challenges:
Now that you have an understanding of the collection of anti-debugging and anti-reversing tricks I was pulling from, let’s take a look at how I combined and mixed them together to create different reversing challenges. The challenges themselves I thought were pretty neat.
Challenge #1: “JumpIt”
The backstory for for this challenge you can read here. (In fact, each challenge name in this post will link directly to the backstory if you want to read them.) The first of these challenges was intended to be fairly simple. The “key” for the challenge was just a static string that was popped up in a User32 MessageBox(). The trick was that the reverser had to jump over a call to ExitProcess to get the popup. Simple stuff. A guy named LordParody made a video tutorial of it even.
Tricks Employed: Premature Exit
Challenge #2: “BeingDebugged”
This executable checks to see if it is being debugged before and after it causes itself to crash. If the reverser subverts it, the flag is displayed to them.
Tricks Employed: String Obfuscation, Being Debugged (custom), Debugger Required Exceptions, Inlining
Challenge #3: “Hey I know you!”
When it starts, this executable checks to see if a process of a specific name is running, if it isn’t it exits. The reverser must find the name and start a process of this name to be displayed the key. One think you might find interesting about this one is that I did not use EnumProcesses() instead I parse structures directly from ZwQuerySystemInformation. I borrowed this from TheMina. This was a good exercise.
Tricks Employed: String Obfuscation, Here Strings
Challenge #4: “Hey I know you too!”
This is the same as Challenge #3 except with some more tricks thrown in and different keys.
Tricks Employed: String Obfuscation, Inlining, Being Debugged (custom), Here Strings, Polymorphic C Code, Call Obfuscation
Challenge #5: “Are You My Mama?”
This is a DLL that exports several functions. These exports are irrelevant because the DLL (upon being loaded) checks to see if the process loading it matches a name it expects, if not, it kills the process entirely. Reversers must satisfy or subvert this check to get the flag.
Tricks Employed: String Obfuscation, Here Strings, Inlining, Polymorphic C Code, Call Obfuscation
Challenge #6: “Timing Matters”
This executable checks if it is being debugged before and after it rides a sled of software breakpoints. The reverser must satisfy or subvert these checks to get the flag.
Tricks Employed: String Obfuscation, Being Debugged (custom), Timing Tricks, Here Strings, Inlining, Polymorphic C Code, Call Obfuscation
Challenge #7: “Spin Lock”
A DLL contains 5 exported functions (each called “tumblers”). Each function takes a character buffer as input and outputs an obfuscated version of the same buffer.
;)
Exported DLL functions
These “tumblers” must be strung together to decrypt the contents of a packet capture file (which contain the flag). Additionally (like Challenge #5) this DLL will not be loaded by just any executable, it needs to be loaded by an executable of a specific name!
Tricks Employed: Inlining, Polymorphic C Code, Call Obfuscation, challenge specific string obfuscation
Challenge #8: “You have bad BHO!”
This was a Browser Helper Object. For those unfamiliar with these, it is another term for “Internet Explorer Plugin“.This particular one was intended to simulate the basic functionality of a banking trojan, watching which sites you surfed to.
;)
The Browser Helper Object is watching…
To unlock the flag you needed to reverse it to find out which site it wanted you to surf to!
Tricks Employed: String Obfuscation, challenge specific string obfuscation
The Final Challenge: “The Fin”
;)
loading the driverThis final challenge (unbeknownst to the competitors until the day of the competition) was a Windows Kernel Driver. The kernel driver (installed on XP SP2 machine) needed to be reversed to locate the IOCTLs it was listening for. Even a few of the IOCTLs it was listening for resulted in some taunting messages (like custom blue screens of death). Since this challenge as given on-site at NYU:Poly it was evaluated qualitatively.
;)
Solution.
Some kernel reversing materials were provided that demonstrated all the techniques needed. Additionally, during the course of the challenges incremental hints were given.
Tricks Employed: String Obfuscation, Here Strings, Call Obfuscation, Inlining, Polymorphic C Code
The Outcome:
Many of the teams solved many of the first 8 qualifying challenges. Unfortunately, no one was able to solve the Final Challenge entirely. The listing of all the final rankings are here. A number of teams were able to obtain some points for explaining their process and providing some cursory information they obtained while reversing. Alex Radocea (of RPI) eventually solved it a month or so later and messaged me online ;-).
The competition was closed out with a big awards ceremony. (If you chose to watch the video below, the “Application Security” awards begin at 15:25).
CSAW 2009 - Awards Ceremony from NYU-Poly on Vimeo.
There are also photo galleries of the awards ceremony here.
We at Matasano are particularly proud of the team RPISEC (http://rpisec.net/) who ranked 2nd place behind the Carnegie Mellon Team. The RPISEC team had (Alexandru Radocea, Ryan Govostes, and Adam Comella) who are all Matasano interns. (Inquire about the Matasano internship program by emailing careers@matasano.com)
Conclusions:
In conclusion, the whole experience of building these challenge for CSAW was a good one. In hindsight, I wish I had done a better job on making the difficulty progression a bit more even. In other words, I don’t think I mixed and matched the different “Tips and Techniques” (summarized earlier) well enough to achieve an good gradient of difficulty. For example, I spent so much time on writing the string obfuscator/encryptor but I forgot to use it in one or two places that could have really benefitted from it, such as the executable names in Challenge #3. Likewise, I spent a lot of time building the “polymorphic call obfuscator” and did not use it in some of the earlier challenges like Challenge #2. Another huge oversight on my behalf was not obscuring the call to MessageBoxA well. I could’ve used my call obfuscator or made an indirect call, but I simply forgot to add this. A few people recognized this pattern and just jumped to the MessageBoxA call. :-( I guess these are all things you can only learn in retrospect.
Nonetheless, a lesson well learned, all of which I can take into my experience for CSAW-CTF 2010. Feel free to try your hand at any of these challenges. The binaries are posted in github. Everything you need is in there along with the source (which the contestants obviously didn’t have so ignore it if you are doing the challenges ;-). Just download onto a Windows XP SP2 machine. Drop me a line to let me know what you think, or if you needs help. Thanks for reading.
]]>
Back then, developers used workstations — I had a Sun E1 and a NeXTCube. They heated your office, irradiated your head with giant CRT tubes, and had keyboards so big you could take naps under them!
Now it is customary to issue employees a laptop so they can work from bed. This is a great advance. They are small, make your thighs sweaty, ruin your eyesight, and torque your wrists — in bed. A single modern laptop can run the half-dozen virtual machines needed for modern web development using portable standards like CSS and HTML.
This proliferation of machines which move in and out of the office present a challenge to the company’s Hall Monitors, or network security ops as they like to call themselves. If Bob uses the company laptop at home, in bed, then we can’t enforce company web surfing policy at all times. That machine could have anything on it when it plugs back into the office network!
Luckily, the laptop has its own firewall built in, we just need a way for the Hall Monitors to manage the rules. Provided such a mechanism, they could key protect laptops with blacklists of drive-by malware sites, limit incoming connections to known services, limit outgoing connections to specific services. Then the next time Bob is getting cream cheese and sesame seeds all over the keyboard while checking email at the cafe, his host firewall is blocking the SMB attack launched by the unemployed financial analyst turned greasy-haired hacker in the corner. This is the stuff Hall Monitor dreams are made of.
An employee’s laptop is somewhat sacred, so we didn’t want to have
creds for each laptop in some central database. Eric threatened
mutiny at such a prospect anyways, perhaps out of fear we would use
the creds to scan his laptop and find the pirated set of Little House
novels in his ~/.snekrit directory.
Since I only arrive in the office on time on prime Julian dates or when Tom is buying dinner, we could not realiably schedule a push from our dogfood Playbook instance to each laptop. Inspired by NCSA Mosaic and the awesome power of HTTP technology, We opted to write a client that would run under each user’s account and pull rules down from a Playbook server.
As Cory mentioned, Matasano has been an Apple shop since 1925, so it was natural that we added support for OSX’s ipfw first. We could torture our own employees with the early releases, all under the guise of protecting them.
We worked up a list of requirements for the client and server communications.
We wanted the authentication to be automated, not dependent upon user password choice, and verifiable in both directions. The obvious solution here is SSL with certificate-based authentication on both sides. Since Playbook is accessed via an Apache instance, using Phusion Passenger, we can rely on Apache to verify the client certificate.
To manage the client certificates we built our own CA based on the QuickCert Gem. We create a key and certificate for each client.
We need to tell Apache to require a valid client certificate for requests to the pull controller, and to only accept one signed by our Playbook CA. Also, in order to allow us to manage Client Certificates without dealing with revocations, we pass along the certificate in the request headers.
;){kind=link}
<Location /pull>
SSLVerifyClient require
SSLOptions +StrictRequire +StdEnvVars +ExportCertData
RequestHeader set X-Client-Cert %{SSL_CLIENT_CERT}e
SSLVerifyDepth 1
</Location>
We don’t want to require a valid client certificate when the laptop is enrolling, aka downloading, the native client.
<Location /pull/enroll>
SSLVerifyClient none
</Location>
Lastly, we tell Apache were the CA Certificate for our client pull subsystem is located.
<IfDefine production>
SSLCACertificateFile "../../data/client_pull_ca_production/cacert.pem"
</IfDefine>
Our controller needs to make sure the client certificate supplied to
us is the same one we have on file for the host in question. The
HTTP_X_CLIENT_CERT entry in the envrionment table contains the
certificate, so we normalize it and generate an SHA1 checksum. That
is compared with a normalized SHA1 checksum of our certificate on
file.
Here is the relevant method:
def check_cert(fw)
cert = request.cgi.env_table['HTTP_X_CLIENT_CERT']
if cert.blank?
raise "No client certificate provided"
end
if fw.client_pull_certificate.blank?
raise "Firewall does not have a client pull certificate"
end
cc = cert.gsub(/\s/,"")
fc = fw.client_pull_certificate.gsub(/\s/,"")
if SHA1::sha1(cc).to_s != SHA1::sha1(fc).to_s
raise "Client pull certificate does not match."
end
end
Before I came to Matasano, I was a Java hacker at the Center for Connected Learning an Computer-Based Modeling, where my job was working on NetLogo. This involved work on lexers, parsers, interpreters, compilers, and other aspects of programming language design. I liked that alot. In Playbook, we’re interested in the lexing and parsing, and some analysis of a range of firewall languages.
Writing lexers and parsers is, well, boring and rewarding at the same time. It’s a tedious cataloging of the “parts of speech” and grammar rules of a language. If you have good documentation for your target language, it is mind-numbing. With poor or no documentation it is maddening. Pray to whatever gods or old ones you believe in that the language is regular and not just “defined by implementation”. Despite this, it is rewarding when done, because you have a really useful tool for analyzing the target language.
This is precisely the kind of task where a good tool for building your lexers and parsers can save you thousands of dollars in time and prescription sedatives.
Adding IPFW rules support to Playbook took an afternoon thank to Ragel and our own lexer toolkit built on top of it, Ralex. Ragel lets you define reusable finite state-machines — it’s like regular expressions turned inside out with hooks all over. Or, as the projct site says:
Ragel compiles executable finite state machines from regular languages. Ragel targets C, C++, Objective-C, D, Java and Ruby. Ragel state machines can not only recognize byte sequences as regular expression machines do, but can also execute code at arbitrary points in the recognition of a regular language. Code embedding is done using inline operators that do not disrupt the regular language syntax.
Ralex is our own class which includes a set of Ragel state-machines that match tokens common to firewall rule languages, and then some book-keeping code that takes a string and a set of keywords, and gives you a token stream usable as RACC input.
When we want to write a rule tagger and parser for a new rule language, we start with the base Ralex clss:
class RalexIpfw < Ralex
%%{
machine ralex_ipfw;
include ralex_base "ralex_base.rl";
Next then we define any new lexemes needed for this specific rule language. Usualy this is limited to idiosyncratic representations of percentages/bandwidth limits or port/IP ranges.
hexnumber = '0x' @collect_token hexdigit+
@collect_token
>{ start_token(:HEXNUMBER) }
;
probability = '.' >collect_token digit+
@collect_token
>{ start_token(:NUMBER) }
;
ipwithmask = ipaddr ":" @collect_token ipaddr
%{ start_token(:IPWITHMASK)}
;
macwithmask = macaddr "/" @collect_token digit+ @collect_token
>{ start_token(:MACWITHMASK)}
;
portset = ( alnum | '\-' | digit )+ '-' @collect_token ( alnum | '\-' | digit )+
@collect_token
>{ start_token(:PORTSET) }
;
slashcomment = '/' '/' >collect_token >getcomment ;
bandwidth = digit+ @collect_token ('K' | 'M') @collect_token ('bit' | 'Byte') @collect_token '/s' @collect_token
>{ start_token(:BANDWIDTH) }
;
Lastly we identify which state-machines we want to use when parsing a rule:
token = ( id
| ipaddr
| ipnet
| ipwithmask
| macaddr
| macwithmask
| portset
| bandwidth
| hexnumber
| number
| percentage
| bytecount
| ralex_specials
| slashcomment
| hashcomment
| probability
| quote ) %{ finish_token; };
After we have our Ralex class for the new firewall language, we then build two RACC parsers on top of that. One tags significant rule elements, like addresses and ports, which drives our rule indexing engine. The second one does full rule parsing and performs syntax checks and generates a ASTish representation of rules for use in or analysis tools.
Since we have a bunch of these written for other rule languages already, it’s pretty easy to put a new one together. The most time consuming, mind-numbing bit is encoding all the keywords in the rule languages, which we currently have to triple code cause I’ve been to lazy to refactor.
This whole process is driven by a bunch of test macros we have for taking examples rules and deriving tests for the tokenizer, tagger, and parser classes.
it "should handle ip address representations" do
token_test("from 192.2.2.2 to any",
[[:FROM, "from"], [:IPADDR, "192.2.2.2"],
[:TO, "to"], [:ANY, "any"]])
# masklen
token_test("from 192.168.1.0/25 to me",
[[:FROM, "from"], [:IPADDR, "192.168.1.0/25"],
[:TO, "to"], [:ME, "me"]])
....
it "should tag addresses and ports " do
parse_test("add deny ip from 123.45.67.0/24 to my.host.org",
["add", "deny", "ip", "from", IPAddr.new("123.45.67.0/255.255.255.0"),
"to", Hostname.new("my.host.org")])
parse_test("add deny ip from 123.45.67.0/24 to my.host.org 80",
["add", "deny", "ip", "from", IPAddr.new("123.45.67.0/255.255.255.0"),
"to", Hostname.new("my.host.org"), PortNumber.new(80)])
At the end of the afternoon I had my tokenizer, and a tagger. We don’t have a full parser yet, so there are no syntax checks for IPFW in the 3.0 release. After defining a new Firewall and Line (as in rule line) class for IPFW, I had working models, and was ready to start on the controller and the OSX client.
That concludes the server portion of our program, in the next episode I’ll gush a bit about ObjC, and reveal the inner secrets of our OSX client.
]]>And on that note, allow me to introduce Playbook 3.0!
New to Playbook? Here’s the elevator pitch: Playbook is the last product you’d think a vulnerability research team would build. It doesn’t decompile MIPS C code. It can’t spot vulnerabilities in VOIP implementations. It doesn’t harden the Flash runtime. It barely contains any compiler code at all! Instead, it takes the working lives of people who have to deal with firewall rules and tries to make them better.
If you write code, you’re familiar with things like Trac and Github. You check your code into version control, you browse and review it in a web interface, you document it in a wiki, and you file tickets for bugs. Every dev team uses something like it.
Playbook takes the same workflow and bends it to firewall management.
Playbook is a web interface to versioned-controlled firewall configurations, which it can slurp up straight out of running firewalls. It parses and understands firewall rules, detects errors, and (most importantly) indexes rule terms so you can find addresses and networks across large numbers of firewalls. It handles end-user requests for rule changes so you don’t have to pick up the phone to talk to the Sharepoint admin. It provides for peer review and approval. And when rules are staged and ready, it offers push-button deployment so you don’t have log in to 100 remote devices.
Playbook 3.0 does these things better: it has a better UI, it’s faster, the rule editor gracefully handles huge rulesets without dumping you into a big text edit window, and (here’s a big feature we’ll talk about later) it now handles host rules for OS X desktops and laptops.
We’re beta testing 3.0 right now. Interested? Our first three qualified beta testers (you qualify if you have production firewalls) get a permanent free starter license. Sign up here. Questions? Start here with the FAQ.
We’ve been quiet on the blog lately, and here’s one of the reasons. Craig will have more to say about Playbook 3.0 shortly, and then we’ll have another major announcement about products early in January. Stay tuned!
]]>Like it or not, developing an attack plan for a penetration test requires standing up a rudimentary threat model. Not surprisingly, threat modeling often produces discomfort in the stomach and uncertainty in the heart of many testers, but you’ve got to cowboy up and do it.
You need to determine where you’re going to spend your limited testing time, which tools you want to pull out of your arsenal, and how to prioritize your findings after the test is complete. Testing an application while blind to the context of how it could be abused in a real world environment is a sure-fire recipe for disaster and embarrassment.
Start with an application overview sitting peacefully on your desk. A sudden bang, a quick diversion, a cloud of smoke - and a test plan appears in its place. Fear no more, for the guide to ninja threat modeling is here.
You’ve probably heard of threat models before, especially in traditional security development lifecycle circles. In that context, the threat models are generated in the requirements/design phase, which is much earlier in the process. It has spawned not just one, but two, Visio-driven tool sets from Microsoft and countless data-flow diagrams, attack trees, consulting engagements, and perplexed developers. When performed by a skilled and experienced team member, the model can be used to identify architectural weaknesses, guide default application behavior, and outline functional requirements for the product.
However, by the time you’re in the verification phase of the lifecycle, generating a formal threat model starts to yield diminishing returns. We recommend taking a short-cut with a set of assumptions that have served us well over the years.
With these assumptions in place, you could go ahead and do some basic data flow analysis. You should be able identify key entry points in the application where data and functionality cross trust boundaries. If you want to stay in step with Microsoft, feel free to pull out the STRIDE* model and apply the threat types to each entry point, keeping in mind the assumptions m= entioned above. With the threat landscape outlined, you’ll find the STRIDE process will go faster than simply working from a blank data flow diagram without context.
However, ninja threat modeling uses data flow analysis as a clean-up task, rather than the opening salvo. We’re impatient and want to know the answer to the question, “What entry points should we really care about?” right away.
* As required by the SDL powers-that-be, I am obligated to mention STRIDE threat types (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privilege) in all communications relating to threat modeling at least once.
A heat-seeking approach to drive your test plan is just what you need. When we blather on about entry points, trust boundaries, and threat vectors, what we’re really thinking is “Where can I do the most damage in the shortest period of time?”
The ninja threat model relies on two techniques: incident and vulnerability history and the commission of deadly sins.
The first technique examines how the product or similar products have been abused in the past. Don’t just look up old advisories or bug reports - look for incident data. Did a product have a vulnerability that allowed malicious code to spread? Was the vulnerability discovered and abused by a 16-year-old to make lots of friends on the social networking platform of the day?
Examine these abuse cases and make sure you can recreate them against the application you’re testing. At the root of each one, there should be a vulnerability, a threat actor, and an impacted asset. If you’re stuck with dry vulnerability advisories, sometimes the researcher will have a good grasp on the impact of the finding and will document it well. In other cases, you have to watch out for advisories with overreaching statements or extreme speculation.
What’s nice about these types of derived threats is a good amount of the test planning is already done for you and can be lifted wholesale. Watch out for tunnel vision, though - time, platform specifics, or other dependencies may have limited the avenues available to the researcher or intruder. Make sure you’rere looking for the pattern that indicates the flaw is present, not just performing a pattern match on yesterday’s exploit.
The second technique focuses on what developers get wrong most of the time - otherwise known as the “Deadly Sins”. The vulnerabilities introduced by these common flaws are mentioned for a reason: they get abused by threat actors on a regular basis.
Every security outfit has a list of their deadly sins, including Microsoft, SANS, OWASP, and Matasano, because lists are just that cool. We think ours is better for one major reason - our Deadly Sins are features rather than defects. As a result, our work fits into ninja threat modeling much cleaner than a laundry list of vulnerabilities.
It’s an issue of perspective and language. Developers don’t roll out of bed in the morning and say, “I’m going to code up a few SQL Injection vulnerabilities today!” Instead, they say, “I’m going to write the Advanced Search module for the Report Engine today!” When you hear this, your utter lack of faith in your development brethren should kick into overdrive and you should be busily entering “Test for SQL Injection here, here, and here” in your test plan.
We’ve appended our list of Deadly Sins for Web Applications for easy reference. Parts of it also apply to other sorts of applications. When you see these features, get out your red pen, add another page to your test plan, and start outlining how many different ways developers get these things wrong and how to test for it.
After wrapping up the heat-seeking exercises, it’s time to sanity check your work with data flow analysis including the selective application of STRIDE mentioned earlier. Hopefully, the overlap should be extensive. If it isn’t, you may be dealing with something that hasn’t been tested extensively and may bear some interesting fruit. On the other hand, you may have been assigned the most boring application known to mankind. Congratulations!
After following this approach, you should have a good definition of the application’s attack surface and the threat landscape which you will be attempting to recreate. As a result, you should be able to produce a meaningful test plan with a threat model that is defensible and sufficient to guide your engagement. You might even get away with doing one without a single Visio diagram.
1. Security
2. E-mail functionality
3. Thick Clients: Web Applications In Name Only (WINOs)
4. File Upload/Download
5. Templating and Content-Controlled Code
6. Advanced Search
7. Persistent network sockets that accept arbitrary connections
8. Installers
9. Use of plug-ins or third party software to write directly to the DOM
10. Single Sign-On Authentication Hand-offs
]]>
Some of our commentors made the point that this was not C++, but merely C with classes. To those commentors I would like to say ‘welcome to the real world’. Code patterns like this are why security vulnerabilities exist!
This code pattern raises some interesting areas for exploitation since your non-existent class instance(s) are basically an API for writing into arbitrary memory locations in the heap. Bad allocations from the new operator are a known problem, Michael Howard at MSFT has written about this issue before and MSFT did something about it! MSVC produces better code that doesn’t allow the integer overflow in the new operator to occur. Unfortunately g++ users are stuck with this issue for the forseeable future.
]]>A few weeks ago we ran a survey for firewall operators. The results were a huge win for us. But most of our readers aren’t firewall operators. A lot of you are company pentesters, though. Are you part of a company application security team? Fill out a survey, and we’ll send you a poster. It will look marginally better and be significantly bigger than the picture below.
Take the survey here. It’ll take you less than 5 minutes.
The first 100 responses will get posters, and we’ll ship 50 more posters by lottery if we go over 100 responses (we’ll let you know in this post if that happens).
]]>