At SourceBoston 2009 fellow New Yorker Dan Guido did a talk entitled “So You Want to Train an Army of Ninjas”. Dan’s talk was on his experience organizing the CSAW competitions at NYU:Polytechnic. If you are unfamiliar with the annual awesomeness that the ISIS program at NYU:Poly puts together, you can read more about the whole event here. To quote someone on twitter “If all of school was like [the program at NYU:Poly] maybe I wouldn’t have dropped out.”

This is an excellent little video summary of the competition:

CSAW 2009 - The Judge’s Perspective from NYU-Poly on Vimeo.

The competition portion of the event is loosely modeled after the Defcon Capture the Flag competitions of recent years. This past year, the CSAW-CTF competition brought talent from all around the world together during the National Cyber Security Awareness Month which is celebrated at NYU:Poly as CSAW (Cyber Security Awareness Week) 

In 2008 when I was asked by Dan to help out I was very eager. That year, my involvement with CSAW-CTF was limited to just a few small challenges (instead I focused on teaching a Reverse Engineering class at NYU that year which Aaron Portnoy took over for this year). This year (2009) however, I went a little more “head down” and spent a bit more time designing, coding, and building reverse engineering challenges for the CSAW-CTF.  This blog post is about those challenges.

Times have changed, and these competitions have evolved and matured much like the rest of our “industry”.  They are no longer about who can whip out their graffiti’d laptops, mount their sticker laden Zip drives from linux, and use gcc to compile stuff from their packetstorm, rhino9, rootshell.org, or hack.co.za archives. Times have changed. The games are different now and don’t so much deserve to be the target of smug self-righteous vitriol from jaded old security folks.

This became really obvious to me in 2005 when Kenshoto premiered a new kind of Capture the Flag for Defcon that year. Players that year said that they could really tell that despite Kenshoto’s showmanship (and at times, ostentatiousness), all the members (mostly invisig0th, who is responsible for really spear-heading the whole effort) really loved what they did and wanted to share in the fun of software exploitation and reversing.  All the challenges were custom made and well documented. They required source auditing skill, custom shellcode payloads, and reverse engineering skill. During the qualifying rounds (a month or so before Defcon) there was also an interactive scoreboard, MUD and IRC channel for all the players and organizers to use. This drew players in to world of the game a bit more and allowed teams and organizers to “hang out” all in one place and keep up-to-date on scores and rankings. 

That year, and in the following years following, some big hitters came out to play: Mark Dowd and John McDonald (who ruxxed the quals round in 2005); Chris Eagle and the Naval Post-Graduate Warfare College; Giovanni Vigna and the UCSB team; the SAIC team, the Novell Team, some great international teams; and number of Fortune 500 technology companies (and government agencies) that wished to remain nameless ;-) Kenshoto and John Viega (a Kenshoto member) even got an article in Popular Science that year.

Having been a member of Kenshoto, I really wanted to bring a bit of that spirit to the NYU:Poly CSAW-CTF. The NYU-Poly competitions had a number of different competition categories which included Web Security, Application Security, Forensics, and  Embedded Systems. 

Myself, Dino Dai Zovi (who blogged briefly about his CSAW experience and even talked about it on the local news), and Dean DeBeer were all responsible for judging and organizing the “Application Security” parts of the competition. Dino Dai Zovi (an alumnus of Matasano and fellow New Yorker) designed all the Software Exploitation challenges and I designed all the Reverse Engineering challenges.

You can read about the full format of the competition online at the csaw website, but essentially the competition happened in two waves: (1) a qualifying round where anyone could participate, and (2) a final round where all the top teams that qualified and were enrolled at universities within the US would be flown to New York and put up in a fancy hotel to participate in the final round held on the NYU:Poly campuses.

The Scoring System:

First things first. How to keep track of scores? A stupid web app? The honor system? Email based submissions?Due to the sheer number of participants I decided that any kind of manual or qualitative scoring system would be a nightmare. Reverse engineering challenges seemed to lend themselves nicely  to “token” based scoring so I decided to go that route and to build an automated scoring system. The scoreboard was written in Python and ran as the default shell for an “open” user account to be shared amongst all participants. Competitors were dropped into the BBS-style scoreboard system when they ssh’d into the box.They would then authenticate to the scoreboard system itself (which maintained it’s own credentials).

The scoring system internally used a client/server model. The server ran at a higher privilege level than the client and communication between the two was all done using python RMI. This was intended to localize the damage if anyone had a clever way of breaking out of the python interpreter. If they were, it would be harder to directly access the scoring database files directly, because all those operations were performed by the server on behalf of the client who made requests via RMI. The scoring system allowed competitors to check scores, read about flags and challenges, and submit/check their answers.

Other than that there isn’t anything really noteworthy about the scoring system. Source code for that is all available online here. Now let’s get on to the Reversing challenges.

For the qualifying round of the competition I put together 8 challenges. The 9th and final challenge would be completely different than all the others and was saved for the Final Round which took place on the NYU:Poly campuses. Each of the 8 challenges were framed as forensics based malware challenges, each having its own fictional “backstory”. Every challenge was distributed in binary form and it was up to the competitor to extract tokens from the binary by reversing or otherwise modifying execution of the executable. Each binary employed what I hoped to be progressively more difficult anti-debugging and anti-reversing techniques and combinations thereof. Let’s individually review each technique before summarizing how they were combined to form each challenge.

The Tricks and Techniques:

Premature Exit:

Before a critical call is made a call to ExitProcess is made. Thus normal execution of the binary will seem to just cause the program to run and exit. Debugging the process however will reveal that more code exists after the call to ExitProcess.

BeingDebugged (custom):

Traditionally this technique involves having the process make a call to IsDebuggerPresent() to see if it is being debugged.  This is the standard way to do this using the Microsoft APIs. However many anti-anti-debugging patches for debuggers such as OllyDBG have modules that will intercept this call by setting a breakpoint on it down in kernel32.dll. So instead of using the Microsoft API I wrote some inline assembly that pulled this boolean value directly out of PEB. This way, the only way to catch that “Being Debugged” was being checked was to set a Break-on-Access in PEB itself, something I don’t think any of the common debugger plugins do. My hope was that this would make the process a bit more manual for the competitors.

String Obfuscation:

  Many strings that get statically compiled into binaries sit in the BSS section of the executable. You can use canned tools like unix “strings”, IDA, or even simple shell scripts to pull this stuff out. Even simple string obfuscation like XOR is easily spottable in disassembly. So what I wanted to do was to write a string obfuscater that would “raise the bar” high enough that reversing the scheme itself would be more difficult than doing the challenge itself, so I settled on some simple encryption.  To achieve this I was originally going to use RC4 and execute it as a shellcode buffer, so I wrote RC4 in NASM. 

 The reason I chose to do this as shellcode originally is because I thought it would be easier to polymorph the algorithm as shellcode than it was to polymorph it as compiled C code. I eventually found out a way to achieve this with C code (as you will read below) and abandoned the shellcode idea. Instead I wrote (*ahem* stole) an RC4 implementation written in C that I vainly called sa7ori-encode. I generously borrowed from the infamous cipherpunks mailinglist ARC4 leak to write this code. With the crypto scheme implemented, the challenge then became how to make the algorithm look “different each time”. For this I used a combination of Inlining, Call Obfuscation, “Polymorphic” C Code Blobs, and “Here Strings” (all described a bit more below).

Inlining:

  Inlining (for those of you who are un-familar) is a way for a C developer to specify (using the compiler directive __forceinline) that when a function is called that execution is not redirected into it using the traditional “call” instruction. Instead, with inlining wherever a function would normally have been called, the full body of code for that function is “pasted” into the place where it is referenced. This is extremely inefficient because it makes binaries MUCH larger but for that reason it also makes reversing a bit more painful.

Timing Tricks:

   I had always heard about these timing tricks but the most interesting use of them was in Kostya Kortchinsky and Fabrice Desclaux’s EXCELLENT presentation entitled Vanilla Skype where they discussed some of the anti-debugging tricks they encountered while completely reverse engineering Skype. (This, by the way, among my favorite presentations of all time.) The idea is simple, x86 CPUs since the Pentium support an instruction called RDTSC which reads a time value from a timer that started inside the CPU when it was powered on. This can be used as a “stopwatch” to measure the time delta between bits of code.

This is particularly useful if you want to detect if there is a debugger attached. By starting the proverbial “stopwatch”, then throwing an exception that only a debugger can handle, and then checking the “stopwatch” after the exception is handled, a program can essentially time how long the debugger (and thusly the human operating the debugger) took to handle the exception. By setting the threshold for this time impossibly low for a human to react to you can effectively detect if there is a “man in the loop”.  The RDTSC instruction is wrapped by the Microsoft API function “GetTickCount()”.

Self-Caught Exceptions (to cause code branching):

This technique involves using registered exception handlers to redirect execution in your application. Instead of just calling a function by name, you register the function you wish to call as an exception handler, and then trigger an exception. If a debugger is attached, it catches the exception before the exception handler (which is actually a function critical to the normal operation of the program) executes and thus the program does not continue properly when the debugger has handled the exception and resumed execution.

Debugger Required Exceptions:

  This simple technique caused an exception (division by zero, “call 0”, etc.) that required the reverser to have his debugger attached so he could jump over or NOP out the exception. This is effectively like the premature exit. In the VM supplied to the competitors (which had remote Kernel debugging enabled) this had the unintended consequence of freezing the entire VM if they didn’t have a debugger attached ;-).

Call Obfuscation (AntiDisassembly, kinda):

  To obscure the area around a function call  I decided to use a combination of inline assembly and a property of C include files. C include files (.h) files essentially just “paste” in the code from the .h wherever it is referenced in the C code. In this case I put inline assembly in the .h.

The inline assembly uses the _emit macro to build a large blob of seemingly random bytes that could not be disassembled properly. Using the __COUNTER__ Visual Studio Macro. Stephen Lawler gave me the idea for this trick, unfortunately it can only be used once per function. You can see an example of how I implemented it here. This code was further modified to make it “polymorphic” which is discussed below. 

Polymorphic C code (kinda….well, ok not really :-):

 My CS vernacular stinks, so I couldn’t really think of a way to describe what I was doing here. To replace my idea to use polymorphic shellcode as the string obfuscater I instead found (what I think to be) a neat way to do this in C. The main use for this was to make the blob generated by the “Call Obfuscation” trick (see above) pseudo-random and thus a little more difficult to “eyeball” in disassembly.  Visual Studio has a very useful feature that allows you to create  variables at compile-time with the /D compile option for cl.exe. This will create a variable with a value specified (at compile time) that is then accessible from within code. For example, take the line of C code: 

#define MYCTR (__COUNTER_ _ + MYOFFSET)

The variable MYOFFSET is not defined anywhere within the source code. Instead it is actually defined at compile time with this:

cl /nologo /Ob2 /Z7 /DMYOFFSET=251 6.cpp /c

I then modified the “Call Obfuscation” code (from the previous section) to use this trick. This made the anti-disassembling blob “polymorphic” between compiles as long as the seed value for “MYOFFSET” was modified accordingly. 

 Annotated visual of polymorphs between compiles

In the end it came out something like this which could be used in the code like so: #include “evil.h” to screw up disassembly of the compiled code. An annotated visual screenshot of this process is here.

TLS Execution:

I learned about this Thread Local Storage (TLS) execution trick some years ago in one of Ilfak’s blogposts. It is a way (on Windows) to have a piece of your code execute before the main part of your executable (the entry point) is jumped into. The impact of this is that the code will even execute before a debugger can attach to it even if it starts the target program! Combined with anti-debugging tricks this can be really useful. It requires using an awesome custom linker though.

Use of “Here Strings”:

  My name for these probably sucks, but once again language fails me and I don’t know the correct term for this (if they even have one). In languages like Ruby or Python they are called “here strings” or “doc strings”. However, in languages like C even static strings get squirreled away in the BSS and then a pointer to them is used in the code. What I wanted was a way to reference a string buffer DIRECTLY where it was or as a really small relative offset. The reason I wanted this was to change the way the call into the string obfuscation functions looked, and to confuse disassembly. To achieve this I used a combination of inline assembly and _emit bytes. I wrote a small python script to take a string as input.It would generate a random key for my encryption scheme and then generate the ciphertext (based on the plaintext input and the generated key).

It would then generate a bit of inline assembly that represented the string as inline asm _emit bytes that exposed pointers to these buffers in the ebx and eax registers respectively. I could then just cut and paste the output of this Python script into my C code and use the “strings” by referencing pointer variables that referenced labels in the inline assembly. Instead of defining my ciphertext and key buffers like this (as I would normally in C):

I could access them as inline assembly “here strings” like this:

If you dont get what I mean, sample output of this Python script is here.

The Challenges:

Now that you have an understanding of the collection of anti-debugging and anti-reversing tricks I was pulling from, let’s take a look at how I combined and mixed them together to create different reversing challenges. The challenges themselves I thought were pretty neat.

Challenge #1: “JumpIt”

The backstory for for this challenge you can read here. (In fact, each challenge name in this post will link directly to the backstory if you want to read them.) The first of these challenges was intended to be fairly simple. The “key” for the challenge was just a static string that was popped up in a User32 MessageBox(). The trick was that the reverser had to jump over a call to ExitProcess to get the popup. Simple stuff. A guy named LordParody made a video tutorial of it even.

Tricks Employed: Premature Exit

Challenge #2: “BeingDebugged”

This executable checks to see if it is being debugged before and after it causes itself to crash. If the reverser subverts it, the flag is displayed to them.

Tricks Employed: String Obfuscation, Being Debugged (custom), Debugger Required Exceptions, Inlining

Challenge #3: “Hey I know you!”

When it starts, this executable checks to see if a process of a specific name is running, if it isn’t it exits. The reverser must find the name and start a process of this name to be displayed the key. One think you might find interesting about this one is that I did not use EnumProcesses() instead I parse structures directly from ZwQuerySystemInformation. I borrowed this from TheMina. This was a good exercise.

Tricks Employed: String Obfuscation, Here Strings

Challenge #4: “Hey I know you too!”

This is the same as Challenge #3 except with some more tricks thrown in and different keys.

Tricks Employed: String Obfuscation, Inlining, Being Debugged (custom), Here Strings, Polymorphic C Code, Call Obfuscation

Challenge #5: “Are You My Mama?”

This is a DLL that exports several functions. These exports are irrelevant because the DLL (upon being loaded) checks to see if the process loading it matches a name it expects, if not, it kills the process entirely. Reversers must satisfy or subvert this check to get the flag.

Tricks Employed: String Obfuscation, Here Strings, Inlining, Polymorphic C Code, Call Obfuscation

Challenge #6: “Timing Matters”

This executable checks if it is being debugged before and after it rides a sled of software breakpoints. The reverser must satisfy or subvert these checks to get the flag.

Tricks Employed: String Obfuscation, Being Debugged (custom), Timing Tricks, Here Strings, Inlining, Polymorphic C Code, Call Obfuscation

Challenge #7: “Spin Lock”

A DLL contains 5 exported functions (each called “tumblers”). Each function takes a character buffer as input and outputs an obfuscated version of the same buffer.

Exported DLL functions

These “tumblers” must be strung together to decrypt the contents of a packet capture file (which contain the flag). Additionally (like Challenge #5) this DLL will not be loaded by just any executable, it needs to be loaded by an executable of a specific name!

Tricks Employed: Inlining, Polymorphic C Code, Call Obfuscation, challenge specific string obfuscation

Challenge #8: “You have bad BHO!”

This was a Browser Helper Object. For those unfamiliar with these, it is another term for “Internet Explorer Plugin“.This particular one was intended to simulate the basic functionality of a banking trojan, watching which sites you surfed to.

The Browser Helper Object is watching…

To unlock the flag you needed to reverse it to find out which site it wanted you to surf to!

Tricks Employed: String Obfuscation, challenge specific string obfuscation

The Final Challenge: “The Fin”

loading the driverThis final challenge (unbeknownst to the competitors until the day of the competition) was a Windows Kernel Driver. The kernel driver (installed on XP SP2 machine) needed to be reversed to locate the IOCTLs it was listening for. Even a few of the IOCTLs it was listening for resulted in some taunting messages (like custom blue screens of death). Since this challenge as given on-site at NYU:Poly it was evaluated qualitatively.

Solution.

Some kernel reversing materials were provided that demonstrated all the techniques needed. Additionally, during the course of the challenges incremental hints were given.

Tricks Employed: String Obfuscation, Here Strings, Call Obfuscation, Inlining, Polymorphic C Code

The Outcome:

Many of the teams solved many of the first 8 qualifying challenges. Unfortunately, no one was able to solve the Final Challenge entirely. The listing of all the final rankings are here. A number of teams were able to obtain some points for explaining their process and providing some cursory information they obtained while reversing. Alex Radocea (of RPI) eventually solved it a month or so later and messaged me online ;-).

The competition was closed out with a big awards ceremony. (If you chose to watch the video below, the “Application Security” awards begin at 15:25).

CSAW 2009 - Awards Ceremony from NYU-Poly on Vimeo.

There are also photo galleries of the awards ceremony here.

We at Matasano are particularly proud of the team RPISEC (http://rpisec.net/) who ranked 2nd place behind the Carnegie Mellon Team. The RPISEC team had (Alexandru Radocea, Ryan Govostes, and Adam Comella) who are all Matasano interns. (Inquire about the Matasano internship program by emailing careers@matasano.com)

Conclusions:

In conclusion, the whole experience of building these challenge for CSAW was a good one. In hindsight, I wish I had done a better job on making the difficulty progression a bit more even. In other words, I  don’t think I mixed and matched the different “Tips and Techniques” (summarized earlier) well enough to achieve an good gradient of difficulty.  For example, I spent so much time on writing the string obfuscator/encryptor but I forgot to use it in one or two places that could have really benefitted from it, such as the executable names in Challenge #3. Likewise, I spent a lot of time building the “polymorphic call obfuscator” and did not use it in some of the earlier challenges like Challenge #2. Another huge oversight on my behalf was not obscuring the call to MessageBoxA well. I could’ve used my call obfuscator or made an indirect call, but I simply forgot to add this. A few people recognized this pattern and just jumped to the MessageBoxA call. :-( I guess these are all things you can only learn in retrospect.

Nonetheless, a lesson well learned, all of which I can take into my experience for CSAW-CTF 2010.  Feel free to try your hand at any of these challenges. The binaries are posted in github. Everything you need is in there along with the source (which the contestants obviously didn’t have so ignore it if you are doing the challenges ;-). Just download onto a Windows XP SP2 machine. Drop me a line to let me know what you think, or if you needs help. Thanks for reading.