The contest is over! We got some good submissions and as expected a few security researchers found the bug rather quickly. The sample code had numerous defects, however not all were exploitable. Here are our top 3 correct submissions in order they were received:

1st Drew Yao: With the quickest submission and first exploit. He exploited the bug on OSX Snow Leopard
2nd Kevin Easton: Sent in code that produces an exploit file
3rd Evin Robertson: With a great fuzzing technique for initially finding the bug (valgrind a.out /dev/urandom)

Other correct entries:
Joe Damato - Joe did a great write up on the challenge here
Rachel Blum
Anonymous 1
Anonymous 2

In addition to our contest submissions, we promised we would provide an answer to our vulnerability challenge, and here it is. The example code has a lot of issues. Everything from missing NULL pointer checks to a missing free. The bug we were looking for is the size overflow in the argument to our new operator. Our program opens up a binary file and reads some values out of it using a _stream structure. From those first few values we get an integer ‘s->num_of_streams’, which of course we sanity check before using it as an argument to a memory allocator. Unfortunately our sanity check is broken in the following if statement:

    if(s->num_of_streams >= INT_MAX / (int)sizeof(int)) {
        safe_count = MAX_STREAMS;
    } else {
        safe_count = s->num_of_streams;
    }

The problem with this check is that our Object class is not sizeof(int). We assign safe_count the value of ‘s->num_of_streams’ and go on our way. Following this bad check we call the C++ new operator to allocate an array of class instances representing each stream in our file. Unfortunately g++/libstdc++’s new operator allows for overflow to occur. This happens because we are asking for new to allocate ‘safe_count * sizeof(Obj)’. This a known issue, but more on that later.

So now we’ve allocated a smaller number of class instances than ‘safe_count’ specifies. Following the allocation of our class instances a meta data structure is placed on the heap using malloc(), zeroed out and a function pointer is setup. Now we enter a for loop using our attacker controlled safe_count value. This is where our problems begin, our for loop allocates a temporary buffer with each iteration copying a stream from the file into it, a DWORD is read from the stream and the parse_stream() method is called for each class instance we allocated earlier. The parse_stream() method sets the class member variable ‘type’ to the value of the parse_stream() method ‘t’ argument. The location of class member variable ‘type’ should be in the heap because we allocated the class instance using the new operator. Unfortunately this is done for more class instances then we actually allocated earlier. This allows us to overwrite the function pointer in the ‘imd’ structure, by way of the parse_stream() method, and gain code execution. This happens because the parse_stream() method effectively performs a 4-byte overwrite into heap memory that contains the ‘imd’ structure.

The fix here should start at the sanity check of s->num_of_streams. We should declare num_of_streams unsigned. This value should then be validated such that s->num_of_streams is not greater then MAX_STREAMS.

We originally wanted to share a similar real-world code pattern but that bug isn’t officially patched yet and we wanted to get this post out. But thats OK because we can still talk about the core issue.

As our example showed you can easily misuse the new operator. This is a known issue in libstdc and was mentioned as far back as 2002 in this Blackhat presentation. Using new to allocate some storage space may be common but there is another similarly dangerous but slightly less used code pattern here, and thats using new to allocate an array of class instances. Most developers have learned to carefully inspect the size of an allocation before copying user data to it. But allocating too few classes can result in similar memory corruption and have subtle but exploitable consequences when all the right stars are aligned, and thats what this challenge was all about.

In the real world vulnerability we found exploitation was not as straight forward and may not even be possible. But the reason for that is actually an interesting story in itself (and by interesting I mean dry and boring). If the Obj class in our challenge had a constructor things would have been far different. Let’s add a simple constructor to our example:

    Obj(){ length = 0; }

To understand why this changes things you need to dive down and disassemble the code. Long story short, even though we tricked our application into allocating 8 classes, g++ compiles the code in such a way that constructors are called for every class instance we requested, all 357913943 of them! Now if the constructor does something like set some class member variables to NULL then our chances of exploitation are pretty slim. This is because our process will essentially rip through its own heap writing NULL bytes until it hits the end of the heap and crashes. Thats quite a destructive constructor! When a constructor like this isn’t present our fake class instances become an API for writing into arbitrary memory. Thats how you gain code execution in our challenge.

Some of our commentors made the point that this was not C++, but merely C with classes. To those commentors I would like to say ‘welcome to the real world’. Code patterns like this are why security vulnerabilities exist!

This code pattern raises some interesting areas for exploitation since your non-existent class instance(s) are basically an API for writing into arbitrary memory locations in the heap. Bad allocations from the new operator are a known problem, Michael Howard at MSFT has written about this issue before and MSFT did something about it! MSVC produces better code that doesn’t allow the integer overflow in the new operator to occur. Unfortunately g++ users are stuck with this issue for the forseeable future.