Wednesday
Jun202007
Exploring Protocols - Part 1
Jun 20, 2007 at 4:15PM Eric Monti
In the process of doing software security analysis, it is pretty common to encounter unknown network protocols or file formats that are part of the attack surface you're investigating.
Not too long ago, we wrote a post entitled Reversing a ZLib-obfuscated? Network Protocol where we talked about reversing an undocumented protocol to look for security weaknesses. We got several good questions about some of our deductions about the protocol as we picked it apart. I'd like to take the opportunity to talk more about protocol reversing in general and hopefully help explain how that deduction process works while getting some broader coverage on the subject in.
This will be the first of at least 2 blog posts. I'm going to start by discussing building blocks and see where that takes us. In the early phases of talking about this process, I'm not making a distinction between whether a protocol is "unknown" because of lack of documentation or because it's simply "unknown to you/me" because we're unfamiliar with it. Of course an undocumented protocol is going to be tricker to reverse. If there's a point to these initial posts, it's that working with documented protocols helps us understand the undocumented ones.
To illustrate some basic protocol dissection ideas, I'm going to talk about iSCSI. I mostly picked iSCSI since I happen to be working with it at the moment and it makes a pretty good case study.
In this post we'll:
1. Talk a little bit about what iSCSI is and what it's for.
2. Use Wireshark to find a iSCSI PDU and isolate it.
3. Compare the raw PDU to the specification.
4. Talk a bit about how this all relates to protocol reversing.
In a nutshell iSCSI is:
... SCSI over IP. It's designed as a low cost solution for network attached storage. A storage server (say a NAS appliance) exports storage as "targets" on any TCP/IP network to which clients (aka "initiators") connect. Once attached by connecting and logging on, the initiator's OS sees the target as a hard drive and treats it as a block device. Filesystem drivers ride on top of the device as they would any other SCSI device. Besides file access, an initiator can arbitrarily partition and format the target using its allocated space.
Sounds a bit crazy from a security perspective, right? Well, just bear in mind that that iSCSI is not intended as a replacement for CIFS or NFS at all. iSCSI is first and foremost designed as an alternative to more expensive fiber channel NAS solutions by using cheaper gig-ethernet and possibly leveraging a company's existing network infrastructure. The iSCSI spec is also apparently designed to be used over other transports besides TCP/IP.
We're interested in what iSCSI looks like on the wire. This is not undocumented or new territory. Wireshark has iSCSI decoding capabilities way above and beyond the simple dissection tools we're going to get into for iSCSI. We're not going to use those decodes much for this discussion, though. Building our own tools gives us more intimate knowledge than relying on Wireshark will. We also want to have some building blocks for doing things later like fault injection if our exploration leads us that way.
iSCSI's a good case study for protocol exploration since it's not exactly a "common" network protocol, but has pretty decent documentation and specifications available in RFC's. Picking it apart with some guidance helps illustrate some common network protocol concepts and we can double-check things against the actual specification to make sure we're getting them right.
Here's a hexdump of an isolated iSCSI PDU as it appears on the wire:
I isolated this using Wireshark and saved it as a as a file to work with. iSCSI uses TCP/3260 as its transport. The pcap filter for this is "tcp port 3260". Here's how I did that:
Now that we've isolated a sample, the next step is making sense out of the raw PDU. If this were an undocumented protocol, this would be the part where we opened it in a hex editor and started trying to separate chunks into boundaries based on educated guesswork, assisted by good conversion tools. Actually that's just one way. Probably the most basic one.
This involves a lot of educated guesswork and is not always a straightforward process. We're still talking about the guesswork, not doing it (yet).
Here's the basic header syntax of an iSCSI PDU as defined in RFC 3720 (yep there it is... we could stop now, but where's the fun in that)
This type of breakout basically represents how we'd like to be able to understand a network protocol. It's very rare, even at best, that you'll actually figure out what every field is for in an undocumented protocol. Just getting fields broken up so you can make sense out of most of them is what you're usually going after initially. As you start to make sense of other things later, the things you may have originally passed over can gain context.
This RFC explains the various fields pretty well and covers much more than just that. There's more information in there than we are even likely to need. This raises a good point. Before you start "reversing" anything, always make sure it isn't documented somewhere or implemented in something you can pull apart.
Using the spec to guide us, we're going to try to understand this header and see what our captured PDU says. We'll need to write a tool for this.
In the next post, we'll:
1. Write a C dissector to emulate Wireshark decodes.
2. Write a Ruby dissector to approximate the C version.
3. Discuss some pros and cons of each.
4. Discuss some of the general things we can learn and how they can be applied to reversing truly unknown protocols.
Not too long ago, we wrote a post entitled Reversing a ZLib-obfuscated? Network Protocol where we talked about reversing an undocumented protocol to look for security weaknesses. We got several good questions about some of our deductions about the protocol as we picked it apart. I'd like to take the opportunity to talk more about protocol reversing in general and hopefully help explain how that deduction process works while getting some broader coverage on the subject in.
This will be the first of at least 2 blog posts. I'm going to start by discussing building blocks and see where that takes us. In the early phases of talking about this process, I'm not making a distinction between whether a protocol is "unknown" because of lack of documentation or because it's simply "unknown to you/me" because we're unfamiliar with it. Of course an undocumented protocol is going to be tricker to reverse. If there's a point to these initial posts, it's that working with documented protocols helps us understand the undocumented ones.
To illustrate some basic protocol dissection ideas, I'm going to talk about iSCSI. I mostly picked iSCSI since I happen to be working with it at the moment and it makes a pretty good case study.
In this post we'll:
1. Talk a little bit about what iSCSI is and what it's for.
2. Use Wireshark to find a iSCSI PDU and isolate it.
3. Compare the raw PDU to the specification.
4. Talk a bit about how this all relates to protocol reversing.
In a nutshell iSCSI is:
... SCSI over IP. It's designed as a low cost solution for network attached storage. A storage server (say a NAS appliance) exports storage as "targets" on any TCP/IP network to which clients (aka "initiators") connect. Once attached by connecting and logging on, the initiator's OS sees the target as a hard drive and treats it as a block device. Filesystem drivers ride on top of the device as they would any other SCSI device. Besides file access, an initiator can arbitrarily partition and format the target using its allocated space.
Sounds a bit crazy from a security perspective, right? Well, just bear in mind that that iSCSI is not intended as a replacement for CIFS or NFS at all. iSCSI is first and foremost designed as an alternative to more expensive fiber channel NAS solutions by using cheaper gig-ethernet and possibly leveraging a company's existing network infrastructure. The iSCSI spec is also apparently designed to be used over other transports besides TCP/IP.
We're interested in what iSCSI looks like on the wire. This is not undocumented or new territory. Wireshark has iSCSI decoding capabilities way above and beyond the simple dissection tools we're going to get into for iSCSI. We're not going to use those decodes much for this discussion, though. Building our own tools gives us more intimate knowledge than relying on Wireshark will. We also want to have some building blocks for doing things later like fault injection if our exploration leads us that way.
iSCSI's a good case study for protocol exploration since it's not exactly a "common" network protocol, but has pretty decent documentation and specifications available in RFC's. Picking it apart with some guidance helps illustrate some common network protocol concepts and we can double-check things against the actual specification to make sure we're getting them right.
Here's a hexdump of an isolated iSCSI PDU as it appears on the wire:
I isolated this using Wireshark and saved it as a as a file to work with. iSCSI uses TCP/3260 as its transport. The pcap filter for this is "tcp port 3260". Here's how I did that:
Now that we've isolated a sample, the next step is making sense out of the raw PDU. If this were an undocumented protocol, this would be the part where we opened it in a hex editor and started trying to separate chunks into boundaries based on educated guesswork, assisted by good conversion tools. Actually that's just one way. Probably the most basic one.
This involves a lot of educated guesswork and is not always a straightforward process. We're still talking about the guesswork, not doing it (yet).
Here's the basic header syntax of an iSCSI PDU as defined in RFC 3720 (yep there it is... we could stop now, but where's the fun in that)
This type of breakout basically represents how we'd like to be able to understand a network protocol. It's very rare, even at best, that you'll actually figure out what every field is for in an undocumented protocol. Just getting fields broken up so you can make sense out of most of them is what you're usually going after initially. As you start to make sense of other things later, the things you may have originally passed over can gain context.
This RFC explains the various fields pretty well and covers much more than just that. There's more information in there than we are even likely to need. This raises a good point. Before you start "reversing" anything, always make sure it isn't documented somewhere or implemented in something you can pull apart.
Using the spec to guide us, we're going to try to understand this header and see what our captured PDU says. We'll need to write a tool for this.
In the next post, we'll:
1. Write a C dissector to emulate Wireshark decodes.
2. Write a Ruby dissector to approximate the C version.
3. Discuss some pros and cons of each.
4. Discuss some of the general things we can learn and how they can be applied to reversing truly unknown protocols.
12 Comments | in Development, Reversing 
Reader Comments (12)
You may want to look at binpac for writing parsers as well.
http://www.bro-ids.org/wiki/index.php/BinPAC
for python ive previously found construct really helpful when it comes to parsing/building protocols .. http://pyconstruct.wikispaces.com/
BinPAC looks interesting. I wasn't able to find too much in the way of documentation. Anyone have more info?
sandro,
For ruby there's (it sounds like) something similar called bit-struct which I've been using a lot of.
http://redshift.sourceforge.net/bit-struct/
Which is going to be a highlight in the next post. I just need to get a few moments to post it up
I'm eating a ham sandwich, and I love Eric Monti his exploration of my protocols.
http://www.matasano.com/log/973/exploring-protocols-2-writing-some-tools/" rel="nofollow">Exploring protocols 2: Writing some tools is finally up.
wow... that took 4 months... i am sooooo sorry for keeping you waiting so long, punky. I hope you didn't run out of ham.
ed hardy hoodies
cheap christian audigier bikini
ed hardy shirts
cheap christian audigier hoody
women ed hardy clothing
ed hardy hoody uk
ed hardy jacket
ed hardy uk
ed hardy shirts
ed hardy swimwear
ed hardy ugg boots
ed hardy shoes uk
ed hardy hoody uk
buy ed hardy shirts
ed hardy womens shoes
ed hardy handbags uk
ed hardy at uk
ed hardy t shirts uk
ed hardy sunglasses uk
ed hardy womens tiger hoodie
For the first time in recent memory, fendi | fendi handbags | fendi bags luxury-goods makers are cuttinglouis vuitton | louis vuitton handbags | louis vuitton sale | lv prices on designer apparel, designer handbags | discount louis vuitton bags | louis vuitton wallet | louis vuitton speedy | louis vuitton silk scarves | louis vuitton mens shoes | louis vuitton hats | louis vuitton neckties | Louis Vuitton Accessories shoes and handbags in the U.S. market. burberry | omega watches | omega | louis vuitton handbags | gucci handbags | Christian Louboutin | louis vuitton | louis vuitton shoes | vuittonWith even the biggest spenders starting to scrimp, louis vuitton luggage | louis vuitton bags | louis vuitton | louis vuitton handbags | louis vuitton bags | louis vuitton bags | louis vuitton wallets | louis vuitton shoes | burett watches | louis vuitton bags luxury companies from Chanel S.A. nba news | nba playoff | nfl 2011 | nhl 2010 | omega citizen | omega replica | omega use | omega watches guide | prada fendi | replica guide | watches sale news | watches women | baby cute | wedding | travel message | hats guide | movie currentto Versace SpA, Christian Louboutin and Chloe are reversing the industry's maxim that luxury prices only move up. The cuts range from 8% to 10% on most products sold in the U.S.
Looking for luxury watches?replica Movado Watches on sale,fashion TAG Heuer Watches sale online,new Panerai Watches for sale,cheap TAG watches sale with high quality.Many christian louboutin products for discount.christian louboutin shoes for sale,beautiful christian louboutin boots set at cheap price.Choose replica christian louboutin shoes high quality.
Our store provides cheap christian louboutin with high quality,we also discount louis vuitton bags, Louis Vuitton shoes and Louis Vuitton wallets and louis vuitton purses and neverfull louis vuitton and louis vuitton luggage. We also have lots of replica burberry watches,they are high quality and cheap,welcome to buy burberry watches and longines watch. Do you konw which websites have cheap coach handbags on sale ? The replica louis vuitton including replica louis vuitton scarves and louis vuitton accessories are nice and cheap. You know that cheap louis vuitton like louis vuitton handbags and louis vuitton bags and replica louis vuitton watches are more and more popular today.a lot of louis vuitton bags for sale at topbrand4lady.com. Do you like burberry watches or jaeger lecoultre watches or longines watches? Welcome to www.worldswatches.com. or which websites discount coach handbags ? These websites have replica coach handbags like www.coachsdesign.com. There are many pink coach purse and imitation coach purse and coach wallets sale ? The website discount coach wallets including many coach womens wallets, of course,there are replica coach wallets.The replica coach purses cheap and coach purse on sale with high quality.If you are looking for louis vuitton handbags,louis vuitton bags or louis vuitton purses , welcome to our replica louis vuitton store, the louis vuitton purse,louis vuitton bag and louis vuitton handbag and louis vuitton wallet are all high quality. The replica louis vuitton are low price and high quality.If you are looking for louis vuitton handbags,louis vuitton bags or louis vuitton purses, welcome to our replica louis vuitton store, the louis vuitton purse,louis vuitton bag and louis vuitton handbag and louis vuitton wallet are all high quality.
How Omega Watches, one might well wonder, did we manage to louis vuitton come to such a pass? How Omega Watches, in an age that seeks to empower women’s Rolex Watches standing, louis vuitton has the female image become honored mainly in its diminution?Rolex Watches Judeo-Christian tradition has always had its own louis vuitton problems with the flesh, louis vuitton watches of course, lv evocative as it is of the carnal louis vuitton handbags and louis vuitton handbag thus sinful (and louis vuitton, by further implication louis vuitton bag, slothful and uncleanlouis vuitton bags). Christian iconography louis vuitton handbags on sale has tended to be half in love with louis vuitton sale flesh as an object of mortification,louis vuitton as in the crucifixion of Christ, while louis vuitton handbags the Judaic tradition strictly regulates Louis Vuitton Accessories the exposure of flesh for both designer caps men and louis vuitton silk scarves women. Meanwhile, as louis vuitton mens shoes Lebesco demonstrates in “Revolting Bodies,” an louis vuitton neckties ethnographic view of fat shows it to be a louis vuitton watches more fluid construct, one that discount louis vuitton bags is governed by prevailing economic and louis vuitton bag prices cultural interests rather than the designer purses negative ider fixe our contemporary luxury women eyes have been conditioned into nba 2011 seeing it as. (bags guide The stigmatizing term “obese,” from the replica guide Latin obesus, originally meant “having eaten well” until it was reclassified longines guide by 19th-century doctors and health workers miumiu news, just as “fat” was once a gucci sale flattering term used by the longines information Greeks.) In premodern times, louis bags for instance, omega citizen chunkiness in women was generally seen as a fifa 2010 positive signifier, indicating fertility in a fashion ladies time when food supplies were scarce kenneth guide or irregular gucci louis vuitton and human reproductivity omega watches guide was endangered; fatness, it appears,nba playoff 2011 was perceived as a kind of bags miumiu internalized abundance. Similarly, during the watches sale news early part of the 20th century in southern Italy, christianlouboutin miumiu where back-breaking work was necessary for survival,watches women corpulence was valued because it was asiangames 2010 equated with wealth and louisvuitton news beauty. By the 1960s, however, when nba news the vast majority of the population southafrica fifa was adequately fed,sports basketball news fat people began to be marginalized watches guide and pathologized kenneth cole rado. Perhaps most interesting, omega replica in stark contrast to our own culture of endless dieting longines dkny and food fetishism, are certain primitive societies that engage in an actual “fattening process,” like the citizen replica Annang of Nigeria, whose women until very recently were deliberately fattened up in fattening huts before marriage.chanel sale This approach sounds so antithetical to our own “Am I fat?” anxieties (fashion show generally asked only by girls and women who are safely thin) as to verge on handbags information the surreal.
I’m pretty new to your blog, but am enjoying what I have read so
far. This post particularly caught my attention.
your post are very useful for me. i like it.
thanks for sharing
look forward your new posts
air jordan 1
air jordan 2
air jordan 3
air jordan 4
air jordan 5
air jordan 6
air jordan 7
air jordan 8
air jordan 9
air jordan 10
air jordan 11
air jordan 12
air jordan 13
air jordan 14
air jordan 15
air jordan 16
air jordan 17
air jordan 18
air jordan 19
air jordan 20
air jordan 21
air jordan 22
air jordan 23
air jordan 2009
air jordan 2010
jordan 1
jordan 2
jordan 3
jordan 4
jordan 5
jordan 6
jordan 7
jordan 8
jordan 9
jordan 10
jordan 11
jordan 12
jordan 13
jordan 14
jordan 15
jordan 16
jordan 17
jordan 18
jordan 19
jordan 20
jordan 21
jordan 22
jordan 23
jordan 2009
jordan 2010