ADD / XOR / ROL

Short note on static analysis and lobbying

2011-09-09T02:37:00.000-07:00

After getting annoyed with Twitters 140-char limit to make a sane point, I switch back to an old-school medium that allows actual arguments ;)

I was commenting on the dishonesty in lobbying for legally mandated static analysis with "cyber security" as an underlying argument.

I think this is a total bullshit argument.

Static analysis is good at removing a large number of low-hanging fruit when it comes to both reliability and security bugs. But: Static analysis alone fails to significantly increase the resilience of real-world systems against determined attacks (although it may increase the resilience to really casual attacks).

So I think it is good & sane to mandate static analysis for some pieces of code for reliability reasons. I'd rather fly in an airplane with formally verified avionics code.

The reality is though that even the most sophisticated static analysis systems nowadays are not terribly good at dealing with dynamically allocated memory, suffer from grotesque overapproximation as a result of summarizing dynamically allocated memory, and in general don't deal with many common programming patterns very well.

Avionics code without dynamic memory allocation and a team of people getting a static analyzer going profits a lot from static analysis. The browser that I am typing this in profits much less: It uses a gazillion programming idioms that are notoriously hard to analyze. No existing and generally available static analysis will significantly impact the difficulty of finding a remotely exploitable bug in this browser.

So, in short: Using SA can have great benefits in particular scenarios. Verifying an avionics system is a great example. Verifying a microkernel that allows me to safely sandbox my terribly buggy browser is another example. Mandating SA for general software development is insanity, though: The current state of research (let's not even speak about available products) isn't capable of impacting the resiliency (against determined attack) of a browser or document viewer significantly. Let's not kid ourselves: The technology just isn't there, and won't be for another few years.

Wow ...

2011-03-01T13:29:00.000-08:00

The company that produces your favourite security researchers' favourite tools has been acquired by Google. You, dear reader, have every right to be surprised; we ourselves are still recovering from the happy surprise.

"In mathematics, you don't understand things. You just get used to them."

-- John von Neumann

Old Jonny was right, but you might substitute "mathematics" with "life" in the above.

zynamics was never designed to be acquired. To be quite honest, zynamics wasn't designed at all -- it mostly just "happened". We never had a plan outside of "build the tools that we want to have, others will then want to have them too". We never took venture capital, and the only business plan I ever created was a half-baked attempt made with wizard software. It was never updated. The fact that we exceeded the forecasts was mostly due to me being an overly pessimistic planner.

Calling our structure engineering-centric would be an understatement; our everything-to-engineering ratio is roughly 7% (if I may still count myself as an engineer).

As we grew, the problems that we wanted to solve grew too -- at a much faster rate than our resources to solve them. The result was that I spent more and more time running around managing, doing sales, and acquiring resources so that my team could do the technical work that I love and am good at. I wanted the chance to focus on technical issues again.

So at some point in this process we started talking with Google, much to our own surprise. We had not anticipated this -- we are not web-centric, we are far away from their core business: At first glance the acquisition seemed like a strange choice for both sides.

Yes, we have excellent technology and a core of great engineers; we were just surprised about the fact that Google would be interested. It was certainly not an obvious pick.

Then again, Google shares an engineering-centric culture, and has just the resources we're lacking.

"The purpose of computing is insight, not numbers."

-- Richard Hamming

A friend of mine (rightfully) mocked me for trying to perform serious computation on something that strongly resembles a pocket calculator. According to him, there's roughly one and a half computers in this world -- and Google happens to have one of them. I tend to concur.

So, as of today, I can say that the entire zynamics team has joined Google. I am looking forward to tackling problems with the resources that Google can provide.

While there will be some changes, our products are not going away - on the contrary !

"Et si tu me demandes quel est donc ce 'propos' que je poursuis a longeur a mille pages, je repondrai: c'est de faire le recit, et par la-meme la decouverte, de l'aventure interieure qu'a ete et qu'est ma vie."

-- Alexandre Grothendiek

Running zynamics with my team was an exciting experience, but I have no doubt that the future will be every bit as exciting.

Trainings class with SP and me at CSW !

2010-03-02T10:00:00.000-08:00

Hey all,

SP and me will be teaching a trainings class this year at CanSecWest. If you have some background in reverse engineering and want to

become a more efficient reverse engineer
become a more efficient bug hunter
become better at understanding stuff like Acrobat's JScript Engine

this class is for you. We will teach you stuff including but not limited to:

Quickly find where the interesting parts of the executable are: Who is parsing user input ? Who is responsible for the crypto ?
Save time: Identify what open-source libraries are statically linked into the executable. Why audit binary when you can read source ?
Want to understand what Acrobat is doing ? Or most C++ programs nowadays ? Generate UML diagrams from binaries, showing you all the classes and their hierarchy.

Anyhow, follow this link if you are interested. I think it's going to be a blast.

Cheers,
Halvar

Tax evasion and welfare fraud

2010-02-08T14:02:00.000-08:00

Hey all,

now that all the technical stuff is going to the zynamics company blog , I will have some room here for writing about other topics. Beware: Politics might be involved, or just general rants.

Tonight I will write a little bit about tax evasion and welfare fraud. I somehow wound up in a discussion about the topic, and the end result was that I spent 20 minutes doing a bit of research on the topic.

Background: The German government was offered a CD containing data of people that have moved money into swiss bank accounts, presumably to evade taxes. The person offering the CD claims that it contains almost exclusively data of tax evaders, and demands a fee of 2.5 million EU to provide the CD to German authorities.

This situation has spawned a debate about the legality of the situation: Is it "right" for the German government to buy data that was obtained in a presumably illicit fashion ? (I intentionally avoid "illegal" here -- the person that obtained the data might be in breach of contract with his employer, but it is unclear whether he broke any criminal laws)

Clearly, it is a tricky question - but the difficulty of this question is not the topic of this blog post.

Recently, a German politician (who, ironically, was repeatedly involved in corruption affairs, most notably in the CDU-party-donations affair) by the name of "Roland Koch" argued that welfare fraud is a serious problem in Germany, and that 15% of all welware recipients do not actually want to work. He argued for annuling benefits of these 15% in a large conservative newspaper (the FAZ).

So in todays discussion, the question came up: What is actually the "bigger" crime (in terms of financial damage): Tax evasion of welfare fraud ?

It is relatively straightforward to calculate the cost of welfare fraud: Germany spent 21.7 billion EU in 2008 on the "Hartz-4" system. This includes administrative overhead. Assuming that Mr. Kochs claim has merit, and assuming that overhead is also inflated due to fraud, ~3.3 billion EU are lost annually to welfare fraud.

It is much more difficult to calculate the cost of tax evasion. There are many numbers that are difficult to justify, and most appear to be made up arbitrarily. The only halfways reliable number I could find was from this article:

The amount of money generated from tax investigations that followed evasion was ~1.6 billion EU in 2004. Inflation-adjusted to 2008 at 2% inflation, this ends up being ~1.73 billion.

This implies something rather interesting:

Assuming that every third tax evader is caught (which I deem more realistic, just by gut feeling, e.g. without any scientific base), tax evasion is already a much bigger problem than welfare fraud.

The question of course is: What is the actual rate of tax evasion to "getting caught" ?

The new, shiny, reverse-engineering-centric zynamics blog !

2010-01-19T03:15:00.000-08:00

Hey all,

for all those that have almost gotten sick of me posting only rarely on this blog:

We have a shiny new reverse-engineering-centric blog up on http://blog.zynamics.com ! :)

The entire team will post RE-related issues there, so I think it'll be a rather good read :)

Interesting Blog Posts

2009-12-18T15:14:00.000-08:00

Hey all,

so while you, dear reader, are still waiting on me finally fullfilling my promise about blogging more, I have some interesting links for you :)

Vincenzo Iozzo has been blogging about some cool stuff he has been doing using NaviPython, REIL, MonoREIL etc. recently, and you can read about it here:

http://viozzo.wordpress.com/2009/12/11/scripting-with-binnavi-cyclomatic-complexity/
http://viozzo.wordpress.com/2009/12/18/finding-interesting-loops-using-monoreil/

Cheers,
Halvar

Clarification to the previous post

2009-11-15T23:52:00.000-08:00

Hey all,

I thought I need to clarify something about the previous post: I was trying to explain the fact that people reacted harshly to the hint that a new standard is being drafted, without knowing anything about it. So I talked about the historical perspective on the old OIS draft, and what my thoughts about it were, and what I think the reasons are that researchers usually do not bother with these things much.

This was not meant as a reaction to the ISO standard under discussion. Katie Moussouris clarifies a lot of important points here -- and what she writes is completely sensible.

Anyhow, enough of this :-). The upside of the entire discussion: I really like the pun in the above link. Yes, I know that I have a weird sense of humor.

Why are most researchers not a fan of standards on "responsible disclosure"

2009-11-15T12:22:00.000-08:00

I usually try to stay away from the politics of vulnerability disclosure, mostly because I think (to paraphrase Feynman) that politics of vulnerability disclosure are as useful to the vulnerability researcher as ornithology is to birds.

But it seems that the entire discussion is not going away. The intensity of the reactions to k8em0's twitter post might be partially explained by the history of this all. I'll try to refresh what I remember:

A lot of the older vulnerability researchers remember the ghastly OIS attempt at forcing a standard written by a bunch of non-researchers down the throats of the research community. From the outside, it looked mostly like an attempt to kiss up to some vendors that were spending a lot of money on security review during that time.

I might be stepping on some people's toes, but to me it looked like a high-school class where the dimmest students drew up guidelines on how smart students "should" behave, and gave that to the teacher in order to earn brownie points - including clauses like 'not contradicting the teacher'.

Unfortunately, most of the research community prefers to do work instead of discussing with people that have little interesting to say about how the researchers should work. The result of this is that researchers were rarely ever involved in the entire discussion. Not for lack of opportunity, but mostly lack of interest -- if I can actually go and surf, why would I discuss with a bunch of people sitting in an office about the right way to come back to the beach ?

The entire discussion has always been somewhat phony. The entire "responsible/irresponsible" angle is sligthly fraudulent. The way I see it is the following:

It is acceptable for AV companies to charge for signatures, which are in essence "information about malware"
It is acceptable for AV companies to not publish, nor provide, malware to other parties, or to charge for it
It is acceptable for software vendors to charge so I can use their software. It is also acceptable for them to charge more so that I can read their source code.
Why again should a researcher be obliged to provide information to vendors free of charge again ?
If anyone argues it's "responsible" to make everyone safer, I say: I'll give all my bugs to all vendors the same day that all security companies of the world provide free licenses for everyone for their software.

But well. Honestly, I am not sure whether I should post this. I do not really feel like spending too much time discussing this. But perhaps that's part of the problem...

Low blogging frequency

2009-11-10T08:44:00.001-08:00

Hey all,

first of all, I seriously have to apologize for the low frequency of blog posts nowadays. We have been doing a bunch of interesting things at work that I will post about shortly. Amongst the
things on my "to-post" list are:

Rants on our experiences distributing VxClass
A method to perform exact directed graph comparison in O(1) (with some caveats ;) -- we have been sitting on this for a year or three, but were caught up in other things so writing it up fell by the wayside
Automated generation of byte signatures from the VxClass results

Anyhow, expect a higher blogging frequency from this blog in the next weeks. I will restrict my use of twitter for this.

Looking for Memoryze dumps of malware

2009-10-22T08:16:00.000-07:00

Hey all,

I am looking for Memoryze dumps of various pieces of malware -- the more the merrier. Does anyone here have some ?

Cheers,
Halvar

Adventures in choosing what to vote

2009-09-21T11:14:00.000-07:00

accidental posting, not ready yet

Restaurant Review: "Le Surfing", Les Estagnots

2009-09-05T12:24:00.000-07:00

Hey all,

I know it is a bit odd to post restaurant reviews on this RE-centric blog, but ... err ... I'll do it anyhow.

So I am currently on my vacation (yay) in southwestern france -- close to Seignosse/Hossegor. On my search for wifi, I ran into a place called "Le Surfing", which is a small ... well ... surf-bar (?) at the beach "Les Estagnots".

The place is definitely pretty damn relaxed -- it seems that a bunch of folks that want to be at the beach decided the easiest way to finance it is through this bar. When I first walked in, I was greeted with "hey, there's a customer, we need someone behind the bar".

So tonight I decided to eat here. After trying to order the Thai-style beef salad, the waitress convinced me that it's saturday, and on saturdays they have sushi, and I absolutely have to eat their sushi. So I ordered their sushi.

And I didn't regret it. It's rather unorthodox -- e.g. it features rice, raw fish, nori, wasabi, and soy sauce, but I wouldn't describe it as traditional sushi. Not that this is bad: I had a stellar roll that consisted of crab meat, a mild goat cream cheese, cucumer, and bell pepper. There was some tuna, fresh crevettes, an onion-heavy-salad, and various other things.

All in all, it was really good. Seriously.

Open position :-)

2009-08-20T15:47:00.000-07:00

Hey all,

we have an open full-time position in our office in Bochum, Germany -- in essence, a vuln-dev-using-BinDiff-and-BinNavi-position.

What are the tasks in this job ?

Use BinNavi and BinDiff to find bugs in software & write exploits
Test BinNavi/BinDiff features. Test them some more.
Provide feedback to the developers about the tools
Write up your results
(Potentially) train clients in the use of the tools

We're looking for someone that has some experience in RE and vuln-dev.

What we can offer:

A no-bullshit environment
Small team, good mood in the office :-)
Direct influence over the development of the tools
Probably more than that, but it is late and I am tired

If you're interested, please send email to halvar.flake@zynamics.com

Klartext please !

2009-07-24T16:56:00.000-07:00

Hey all,

I won't comment too much here right now (I am kinda busy with work), but imaginably in relation to my last blog post:

http://www.microsoft.com/technet/security/bulletin/ms09-jul-ans.mspx

Now my favourite quote from http://blogs.technet.com/msrc/ :

"While we can’t go into specifics about the issue prior to release, we can say that the Visual Studio bulletin will address an issue that can affect certain types of applications."

I would be tempted to say that "if you can't say anything useful, say nothing at all", but I am often guilty of writing useless things, so I won't.

The above formulation inspired me though:

"While I have no clue what this out-of-band patch is all about, it will address an issue that has to do with a programming mistake."
"While going into the specifics would be dangerous to the safety of the world, the issue that will be addressed is an issue that can affect computers."

Sigh. Newspeak.

There is a German expression called "Klartext reden" (literally translated 'to speak clear text'). If you know a german, grab him, and ask him to explain the meaning of the word to you. I wish there was more of this around.

So my 2 cents of speculation:

MS will patch a bunch of libraries (the ATL ?) in Visual Studio
A "certain type" of application is affected and will need recompiling. The msvidctl thingie was an ActiveX component. Anybody wants to add 2 and 2 together ?

I really need to get back to work though.

Poking around MSVIDCTL.DLL

2009-07-09T13:04:00.000-07:00

Hey all,

I have to admit that I did not follow the msvidctl.dll situation all that closely, except for my short tweet that this bug was apparently reported more than a year ago. Yesterday, my old friend Dennis Elser piqued my interest: He had isolated the bug down to a function called ATL::CComVariant::ReadFromStream. He had determined that the function in question made a rather strange mistake: Instead of passing a pointer to a data buffer to IStream::Read, it took the address of a (small) local variable, and passes this address as output buffer to IStream::Read, along with a length read from the stream previously.

Somebody clearly got confused.

So Dennis and me sat down tonight and did a bit of digging around and tried to clarify what was going on. So we dug in a bit deeper, and ended up with the following understanding of the situation:

If the stuff that is supposed to be deserialized does not contain the proper value in the first 2 bytes, 8 more bytes are read, and SafeArrayCreate is used to create a new array with a 4-byte size obtained from these bytes.
A pointer to the allocated memory is obtained by ways of calling SafeArrayAccessData. This function places a pointer to the memory in question into a memory cell pointed to by it's second argument.
Instead of passing the CONTENTS of the memory cell pointed to by the second argument to IStream::Read, the code in question passes the address of this variable in. This happens to be the re-used memory for the first argument of ReadFromStream, hence it is on the stack. Memory corruption hilarity ensues.

This is a cute little bug. First of all, it is a beautiful example of a single excess "&" in the source code. But what is most amusing about this bug is the centrality of it -- it is, after all, in a method of a class from the ATL.

Everybody loves bugs in libraries. Few things fill my heart with quite as much amusement as bugs in heavily-used, statically-linked libraries. OpenSSL (and libeay) was good for many laughs in the past, zlib was a favourite for a long time, too.

So what we have here is a bug in a component that is used fairly widely, and that has the property of being statically linked (yay templates !).

Now, a quick search of my harddisk turned out that a lot of third-party components (flash) contain the same function -- but in old and non-vulnerable versions (for an extra dash of irony, the function used to be safe before all this SafeArray-stuff was added). Only a small fraction of the files that use the ATL and contain this function appear to contain it in a vulnerable version.

We ended up building a really naive / stupid / false-negative-and-false-positive-prone regexp to scan for the vulnerable basic block. This is of course going to fail if anyone has tweaked their optimization settings etc., but it would still be interesting to find out how many files contain this "trivial" byte string.

So I searched my windows directory for the following regexp pattern:

pattern = "\x8B\x07\x6A\x00\xFF\x75\x2E\x8D\x4D\x2E\x51\x57\xFF\x50\x0C\x53"
r = re.compile( pattern, re.DOTALL )

There were a few files in which this pattern was found (XP):

Found pattern in file c:\Windows\system32\ieframe.dll
Found pattern in file c:\Windows\system32\mstscax.dll
Found pattern in file c:\Windows\system32\msvidctl.dll
Found pattern in file c:\Windows\system32\wmp.dll
Found pattern in file c:\Windows\system32\wmpdxm.dll

Dennis searched for the same pattern on his disk (Vista) and found:

c:\windows\system32\cic.dll
c:\windows\system32\comsnap.dll
c:\windows\system32\comsvcs.dll
c:\windows\system32\ieframe.dll
c:\windows\system32\mmcndmgr.dll
c:\windows\system32\mstscax.dll
c:\windows\system32\MSVidCtl.dll
c:\windows\system32\puiobj.dll
c:\windows\system32\rdpencom.dll
c:\windows\system32\shdocvw.dll
c:\windows\system32\wiaaut.dll
c:\windows\system32\wmp.dll
c:\windows\system32\wmpdxm.dll

Why the difference ? Well, amusingly, shdocvw.dll on my XP machine doesn't have this SafeArray-stuff in it yet -- it is probably compiled using an older, not vulnerable variant of the ATL -- wheras Dennis variant of the same DLL is much newer, compiled with the 'broken' variant of the ATL.

So, where does this leave us ?

The bug is actually much "deeper" than most people realize.
The killbit-fix is clearly insufficient, as there are bound to be many other ways of triggering the issue.
The bug might have weaseled it's way into third-party components, IF anyone outside of Microsoft had access to the broken ATL versions.
If this has happened, MS might have accidentally introduced security vulnerabilities into third-party products.
Depending on the optimization settings applied to the executables, it might require a bit of an effort to find out whether a vulnerable or non-vulnerable version of the code is present.
There might be a lot of recompiling next week.
IF this has gotten into third-party-products, I would bet that only a tiny fraction of software vendors will push out proper/timely updates.

It just seems that spending time to improve BinDiffs ability to find statically linked libraries might have been worth it :-)

Anyhow, I really need to get to sleep -- I have a train to catch in a bit more than 4 hours.

A lot of credit for this post has to go to Dennis Elser -- he did most of the hard work before we sat down.

Las Vegas / Blackhat / DefCon

2009-07-06T02:38:00.001-07:00

Hey all,

the annual Blackhat / DefCon thing is coming up again, and I just wanted to announce here that I will be in Las Vegas from the 24th of July to the 2nd of August. I will be busy teaching classes until the 28th of July, but thereafter I am mostly free (e.g. roaming the conference). If you wish to schedule a demo for any of our products (BinDiff, BinNavi, VxClass) or just generally have a chat about all things related to reverse engineering, vulnerability analysis, malware classification, deobfuscation, math or business, do not hesitate to contact me beforehand :-)

Cheers,
Halvar

Strange Cellphone Behavior

2009-07-01T01:29:00.000-07:00

Hey all,

I know this blog post is a bit weird, but I reckon I'd share this: For some reason that is quite unknown to me, my cellphones have a habit of developing strange behaviors. I used to use a Nokia N73, which developed the following habit:

When in foreign time zones (Japan, Norway, USA) the phone would send more-or-less random old text messages to more-or-less random people from my address book. There would be a merry mix & match between the two, leading to more than one amusing misunderstanding that needed clearing up.

Then, at some point last fall, I switched to the silly shiny Apple telephony device (perhaps people do better QA on their backdoors on that platform). For a few months, the problems went away.
This changed last week -- now, when I send text messages to certain numbers, the phone seems to send a more-or-less random old text messages that has already been sent to the same number along with the message. This is a bit nicer (as it will not mix & match), but still annoying.

So .. uhm ... I am trying to come up with plausible explanations for this behavior. Can anyone offer one ? My total-guess-in-the-dark ideas would be:

Current behavior is caused by international text message routing weirdness -- e.g. text messages I sent a few days ago in the US get duplicated for some reason and re-sent
Both current and N73 behavior is triggered by shoddy QA on lawful intercept systems
Both current and N73 behavior is triggered by shoddy QA on the side of the parties that backdoor my phones

Now, I don't know if anyone else has ever suffered from this, or if there is a perfectly valid and proper explanation, or if there is an easy way to do diagnostics, but:

If you backdoor my phone, fix your software. Kthx.
If you write LI software, fix your software. Kthx.
If there are multiple people backdooring my phone, please test for interoperability between your tools.

So, any other theories on what might be going on ?

Reverse Engineering / Bug hunting trainings in Amsterdam

2009-03-09T07:55:00.000-07:00

Hey all,

I haven't given a reverse engineering trainings class in Amsterdam for a few years, but this year is different :-) -- I will be at BH Amsterdam, and there are still seats open in the trainings class for April 14th and 15th.

What will be done in the course ? Well, for one thing, we'll go bug-hunting in some interesting piece of code. Furthermore, we'll talk quite a bit about C++ and it's effects in the binary. We'll do a fair bit of differential debugging, some more bug-hunting, and a lot of IDA automation. Questions like

given a C++ executable, how do I recover an inheritance diagram of the classes ?
given a big and ugly executable, how do I find the interesting places to focus on ?
how do I make sure IDAPython and NaviPython make my life easier ?

will be treated thoroughly.

So, if you still have some trainings/travel budget left in spite of the crisis, you can find more
details here.

Diffing x86 vs ARM code

2009-03-04T12:01:00.000-08:00

I posted a while ago about the new DiffDeluxe comparison engine, and that we'd release it in Q1 2009. Well, we're almost there, the engine is now in beta. If you are a BinDiff user and wish to give the new engine a try, send mail to info@zynamics.com :-)

I mentioned in my last post on the topic that DiffDeluxe was designed to facilitate symbol porting, and to allow comparisons between executables that are "far away" from each other.

In the last post I wrote about Mozilla JS engine vs. Acrobat EScript.dll. Today I am going to try something slightly crazier: In order to evaluate how well these matching algorithms work, we will be diffing an executable that was compiled for ARM against a very similar executable compiled for x86.

My coworker Vincenzo is a big fan of all things OSX, and he brought up the idea of comparing x86 and ARM versions of the OSX dynamic loader -- namely the disassembly of dyld on the iphone against the disassembly of dyld on OSX.

Now, the first voices are going to yell: "You have names for all functions, BinDiffing is easy then!". Well, true, but we will run DiffDeluxe without taking the names into account, and then just using the names to validate the results.

The two executables have 704 (x86) and 618 (ARM) functions respectively. Without name
matching, we match 345 functions. Inspecting the symbols, we see that we have matched
160 of these functions in full accordance with the symbols. Let's have a look at some of the details:

Cute, eh ? Let's look at some more...

It is almost surprising how far one can get without actually looking at the instruction semantics.

If we take the names into account, matching functions becomes easy, but matching basic blocks properly ends up the difficulty. With name matching enabled, DiffDeluxe matches 3809 basic blocks, out of 7904 respective 5196.

So to summarize: The structural comparison is sufficiently strong to yield some useful results even accross two different CPUs. While there is still (a good amount) of room for improvement, I am quite happy with these results so far :-)

So, if you want to beta, and you already use BinDiff, drop us a line !

Washington DC, Trainings, Demos :-)

2009-02-05T09:41:00.000-08:00

Hey all,

I will be in Washington DC from the 16th to the 20th of February. Amongst other things, I will be teaching a course at Blackhat DC. The economic crisis is clearly hitting -- e.g. there are still seats available. We will also get around to using some of the nice features of BinNavi v2 in class, which I am looking forwards to.

Now, aside from the course: If you are in the DC area and interested in a product demo for BinDiff (and the upcoming DiffDeluxe), BinNavi v2 (including REIL), or the latest VxClass (now available as service and virtual appliance), do not hesitate to drop a line to info@zynamics.com :-)

Correction: Clam does have some unpacking support

2009-01-05T01:49:00.000-08:00

Correction of my last post: It appears that Clam has *some* unpacking support. It is not as comprehensive as some of us would like, but progress is being made :-)

ClamAV and unpackers

2009-01-04T09:41:00.000-08:00

Hey all,

this might be a rather odd question, but given the (unfortunate) fact that ClamAV can't unpack
even the simplest packers, has nobody ever contemplated writing packer-specific unpackers
for ClamAV ?

Cheers,
Halvar

TAOSSA blog post I didn't see but will comment on :-)

2008-12-26T16:06:00.000-08:00

http://taossa.com/index.php/2008/10/13/bugs-vs-flaws/#more-83

I didn't see this post beforehand, and I would like to comment on it (mainly because commenting on his blog post might be the easiest way of getting into a conversation with Mr. McDonald these days ;), but I don't have time right now. Will fix this later this week hopefully.

Sometimes, diffing can remove obfuscation (albeit rarely)

2008-12-26T12:31:00.000-08:00

Hey all,

apologies for the sensationalist title, but I found another amusing example today where the same function was present in two different executables -- in two differently obfuscated forms. Amusingly, DiffDeluxe identified the "common components" between these two functions, effectively removing a lot of the obfuscation.

While this is clearly not a typical case, it nonetheless made me smile.

Merry Christmas everyone !

A good protocol attack ...

2008-11-15T07:56:00.000-08:00

... is like a good joke. This one, while requiring special circumstances to succeed with high probability, was responsible for a lot of laughter on my side.

BinDiff / BinNavi User Forum

2008-11-11T12:56:00.001-08:00

Hey all,

we have re-activated the BinDiff / BinNavi User Forum under

https://zynamics.fogbugz.com/default.asp?BinNavi
https://zynamics.fogbugz.com/default.asp?BinDiff

There is not a whole lot there at the moment, but that should change soon :)

Malicious Office/PDFs

2008-11-11T07:09:00.001-08:00

Hey all,

for some research that I'm doing, I'm looking for a collection of malicious Office/PDF documents. If anyone has such documents (e.g. because he was targeted in an attack, or because he found one somewhere), I'd much appreciate submissions ! :)

BinNavi v2 and PHP !

2008-11-10T04:57:00.001-08:00

Hey all,

we have written about the SQL storage format for BinNavi quite a few times on this blog, and how we'd like to encourage third parties to use it. I am quite happy to say that Stefan Esser of
SektionEins GmbH has built code to export PHP byte code into the database format. The (cute) results can be seen under

http://www.suspekt.org/2008/11/05/php-bytecode-in-binnavi-20/

German ways of expressing optimism

2008-11-08T14:34:00.001-08:00

One of my favourite things when travelling and interacting people from other cultures is observing differences in conversational conventions -- and (most importantly) different forms and perceptions of "conversational humor". Aside from comedic protocol screw-ups (e.g. literally translating an essentially untranslateable expression to another language, earning -- at best -- puzzled looks and -- at worst -- thoroughly offending the conversation partner), it often provides interesting insights into one's own culture and habits.

This weeks special: German forms of expressing optimism.

There are many expressions in German that are horribly difficult to translate.

One of my favourites that could cause confusion is the German custom of wishing people luck by wishing them "Hals- und Beinbruch!" (literally: 'broken neck and broken leg') or 'Kopf- und Bauchschuss' (literally: 'shot in the head and stomach') or (for sailors) 'Mast- und Schotbruch' (literally: 'broken mast and ripped sail') upon parting.
A common reply for this would be "wird schon schiefgehen" (literally: 'I have no doubt it's going to go badly'). Counterintuitively, the semantics of this is optimistic -- e.g. whoever says that things are going to turn out badly indicates by this that he is not worried, and that he actually expects that things will be fine.

In essence, one expresses optimism by claiming that an improbably horrible outcome is a near-certainty.

Even though I try hard to not have an all-too-obvious German accent any more, I do catch myself all the time in using the above pattern, even though it does not translate. I (deservedly) earned puzzled looks today by clumsily attempting to use the following German saying to indicate my optimism about the future:

"Lächle und sei froh, sagten sie mir, denn es könnte schlimmer kommen. Und ich lächelte und war froh, und es kam schlimmer."

This has a certain elegance in German, which is totally lost in my clumsy translation:

"Smile and be happy, they told me, because things could be a lot worse. So I smiled and was happy, and things got a lot worse."

Aside from the clumsiness of the expression when translated, the semantics (e.g. the intention to express optimism) was thoroughly lost -- the effect was a thoroughly puzzled and slightly worried look by my conversation partner. I think it is situations like these where Germans earn their bad reputation for being thoroughly unfunny.

Other things that are good for causing confusion between a native English speaker who interacts with someone from the German-speaking world are differences when it comes to acceptable replies to the question "How are you ?". The usual form of this in German is "Wie gehts ?", essentially "How is it going ?". In the English speaking world, acceptable replies seem to be restricted to "good", "good good", or "great".

Proper replies to the question "How is it going" over here would be:
"Muss." -- literal translation: 'it has to somehow'
"Naja, ganz ok." -- 'well... ok ...'
"Könnte schlechter/besser gehen" -- 'could be worse/better'
"Bergauf" or "Bergab" -- uphill / downhill

If the other party feels inclined to have a longer chat, they could reply with
"Yesterday, we stood on a cliff. Today we have advanced by a significant step."
or "Katastrophe". This is usually followed with a short anecdote or complaint about something work-related. From a social perspective, this does wonders as an ice-breaker.

Whenever I catch myself in such a situation, I realize that no matter how much one travels, and no matter how much time one spends in a different cultural climate, certain components of the social interaction are nigh-impossible to change.

Anyhow, time to go to sleep.

The joys of the Volkswagen Caddy Natural Gas car

2008-10-26T15:07:00.000-07:00

So I do own a car (contrary to what most people expect). About a year ago, I bought a VW Caddy EcoFuel. It runs on natural gas in normal mode and only uses the gasoline tank for starting (and when the natural gas has run out).

Up until 4 weeks or so ago I was pretty happy with it, but one morning, the car refused to start unless I hit the gas heavily while starting. I brought the car to the repair shop that belongs to the same place where I bought the car. After a few days of tinkering, they told me that

The particular car I own doesn't lock the tank when the rest of the car is locked and
Somebody poured an unidentifiable liquid into my tank causing the problems
Because this is not a problem with the car itself, warranty doesn't cover it
Removing the tank and the fuel pump and cleaning everything is going to cost 1200 EU

I am somewhat annoyed by some punk pouring an unidentifiable liquid into my tank and agree to pay the money. I also ask for the shop to retain a sample of the tank contents so I can at least find out what was poured into the tank, and perhaps get money back from my insurance.

They agree. When I come to pick up the car, the guys at the shop for some bizarre reason cannot find the sample. I sit and wait for ~1 hour, and they finally produce an unlabelled can from somewhere. Ok. I ask them to sign a piece of paper certifying that this sample is coming from my tank, and they tell me they will send it to me via regular mail the next day. So far so good.

So two weeks pass, and I call back 3 times for that piece of paper. At the beginning of the third week, I have to take my guinea pigs to the vet in the morning (yes, I don't only own a car, I also have guinea pigs). On my way back from the vet, the natural gas runs out, and the car switches to gasoline mode -- while I am going about 130km/h with a large truck behind me. The only complication: My engine switches off. Awesome.

So I manage to stop the car safely on the side of the autobahn and get towed to the next Volkswagen shop. About 2 hours after I leave my car there, I get a call from the repair guy there, telling me that they can see in the VW database which repairs were done on my car recently, but from what they can tell, these repairs never happened. They call in an expert that is certified to appear in court to take pictures & write a report, and he also confirms: The tank was never removed, the gasoline pump never replaced, and the 1200 EU were apparently charged without any of the stuff ever happening.

Clearly, I am somewhat surprised. To my dismay, I am also told that the actual repairs will cost about 2000 EU, and that there is still unidentified stuff in my tank.

So all in all, I am currently stuck with

1200 EU for repairs that never happened
2000 EU for repairs that are happening now
2 * 300 EU for chemical analysis of the two samples taken
unspecified legal costs (most likely covered by my insurance) to deal with the situation

All in all, I am quite dissatisfied with VW on this front -- IMO they should've warned me that the tank doesn't lock, and they shouldn't have "VW Certified Repair Shops" that appear to attempt to defraud customers. I have trouble imagining that not actually performing the repairs was an "honest mistake" (although I usually live by the motto that "one should not attribute anything to malice that can be attributed to incompetence").

Anyhow, let's see how this plays out. As if I don't have other stuff to do.

For those playing with the printer bug...

2008-10-15T02:51:00.000-07:00

... I can't help but post this small PNG. And since blogger rescales/blurs the picture, here is a link to the "full" one.

My bro's comments on the financial crisis

2008-10-05T13:33:00.000-07:00

My brother wrote an article injecting some reality into the discussion about the banking crisis on Spiegel Online. The german version can be seen here. I'll share a short summary of his arguments here (and he'll complain about my distortions later ;).

Short version: The article describes why the situation is less dire than many pundits claim, and explains logical fallacies in commonly-heard arguments.

In the following, here's a summary of his arguments, in the form of "Myth --> Reality"

The US government is taking on a total of 7000bn in liabilities -- about 5500bn by agreeing to step in for Fannie Mae / Freddie Mac, and about 700bn in papers bought by doing the bailout. This equates to roughly half of US GDP, and since the US is already in debt by about 65% of GDP, this would push the total indebtedness of the US to be clearly past 100% of GDP. As a result, serious doubts would have to be cast on the US governments ability to repay debts and service interest on debt.
Reality: Most of the 5500bn are backed by "proper" mortgages with decent quality. It is unclear whether the US gov will lose money on the Fannie Mae / Freddie Mac deal at all. Even the 700bn in "toxic assets" the US is willing to buy have some underlying value. Realistic expectations at the total loss for the US government in this deal runs in the area of 500bn, which would be less than 3% of GDP -- and therefore not a significant source of problems.
The liquidity that central banks are injecting into the markets should lead to hyperinflation. Reality: The measures to help liquidity in the markets do not increase the money supply in the long run. They are usually short-term credits given to struggling banks for a limited amount of time -- weeks or months. After this time, the creditors have to repay the loans, and the money disappears. At the same time, the willingness by existing banks to lend decreases, thus decreasing the money supply in the economy. The statistics by central banks show that the actual money supply M2 is growing a lot less slowly at the moment in spite of all the liquidity injections. Since the money supply is only growing very slowly at the moment, the inflationary pressures are low.
The banking crisis is responsible for the overall slowdown in the EU's economy, and the German government is thus not responsible for having to adjust their growth estimates downwards sharply.
Reality: Most indicators show that the slowdown started way before the crisis reached it's current urgence. The indicators started pointing down much earlier as a result of the heavy increase in energy costs, the appreciation of the euro (and the resulting loss in competitiveness), and Germany's botched reform of accounting rules for writing down investments in equipment. The banking crisis is just the latest "kick" -- but the three previous ones were all known early (and could've been partially corrected).
This is the mother of all financial crises. This banking crisis is the worst crisis in several generations, up to the 1930's crash. Reality: Dramatic banking crises are more common than we think. Since 1970, the IWF has counted 42 crashes in countries like Argentina, Indonesia, China, Japan, Finland or Norway. In comparison to these crises, the current crisis isn't even very deep or expensive: The Paulson-bailout comes at a cost of 700bn, not even 5% of GDP, and only a fraction of this will be actually lost. According to the IWF, the average banking crisis in a country came at the cost of 13% of GDP for that country's tax payer. The Indonesian crisis even came in at four times this. The big difference to the other crises is that this one has caught on in the world's biggest economy, and as such reaches unknown dimensions in absolute terms.

A few things I forgot to mention :-)

2008-10-01T07:48:00.000-07:00

Hey all,

I forgot to mention a few things in the previous post:

We're going to release BinDiff v2.1 on the 15th of October 2008. This is still the "old" diffing engine, albeit with a number of speed & reliability improvements.
We're going to release BinNavi v2.0 on the 15th of October 2008. The number of new features in this release is huge -- it's really quite significant. You can read about it in detail on SP's blog.
I will post some more information myself in the next days. Just a few mouth-watering keywords: Plugin API to extend Navi from Java/JRuby/Jython/JavaScript, built-in intermediate language, hierarchical tagging / namespaces for structuring large disassemblies, cross-module-graphing, managing multiple address spaces in one project, many user interface improvements, faster IDA->SQL export etc. etc. etc.
The DiffDeluxe engine will be part of the next BinDiff release thereafter, probably no later than February 2008. If you are an existing BinDiff customer and would like to try the DiffDeluxe engine in order to provide us with feedback, do not hesitate to contact us -- it's available for testing now. We're especially interested in finding instances where DiffDeluxe performs worse than BinDiff v2.1. Switching the core diffing engine is a significant change, and I would not want to know of any instances where the new engine is worse than the old one.

Improving Binary Comparison (and it's implication for malware classification)

2008-09-29T00:39:00.000-07:00

I am at Virus Bulletin in Ottawa -- if anyone wants to meet to see our new stuff, please drop mail to info@zynamics.com ! :)

It has been a while since I posted here -- partially because I had a lot of work to finish, partially because, after having finished all this work, I took my first long vacation in a ... very long while.

So I am back, and there are a number of things that I am happy to blog about. First of all, I now have in writing that I am officially an MSc in Mathematics. For those that care about obscure things like extending the euclidian algorithm to the ring of boolean functions, you can check the thesis here:
http://www.zynamics.com/files/Diplomarbeit.Thomas.Dullien.Final.pdf

For those that are less crazy about weird computational algebra: Our team here at zynamics has made good progress on improving the core algorithms behind BinDiff further. Our stated goal was to make BinDiff more useful for symbol porting: If you have an executable and you suspect that it might contain a statically linked library for which you have source access (or which you have analyzed before), we want BinDiff to be able to port the symbols into the executable you have, even if the compiler versions and build environments differ significantly, and even if the versions of the library are not quite the same.

Why is this important ? Let's say you're disassembling some piece of network hardware, and you find an OpenSSL-string somewhere in the disassembled image. Let's say you're disassembling an old PIX image (6.34 perhabs) and see the string

OpenSSL 0.9.5a 1 Apr 2000

This implies that PIX contains OpenSSL, and that the guys at Cisco probably backported any fixes to OpenSSL to the 0.9.5a version. Now, it would be fantastic if we could do the following: Compile OpenSSL 0.9.5a with full symbols on our own machine, and then "pull-in" these symbols into our PIX disassembly.

While this was sometimes possible with the BinDiff v2.0 engine (and v2.1, which is still essentially the same engine), the results were often lacking in both speed and accuracy. A few months back, Soeren and I went back to the drawing board and thought about the next generation of our diffing engine -- with specific focus on the ability to compare executables that are "far from each other", that differ significantly in build environments etc. and that only share small parts of their code. The resulting engine (dubbed "DiffDeluxe" by Soeren) is significantly stronger at this task.

Why did the original BinDiff v2 engine perform poorly ? There are a number of reasons to this, but primarily because of the devastating impact that a "false match" can have on further matches in the diffing process, and due to the fact that in the described scenarios, most of the executable is completely different, and only small portions match. The old engine had a tendency to match a few of the "unrelated components" of each executable, and these initial incorrect matches led to further bad matching down the road.

This doesn't mean the BinDiff v2 engine isn't probably the best all-round diffing engine you can find (I think it is, even if some early builds of the v2.0 suffered from silly performance issues -- those of you that are still plagued by this please contact support@ for a fix !) -- but for this particular problem some old architectural assumptions had to be thrown overboard.

Anyhow, to cut a long story short: While the results generated by DiffDeluxe aren't perfect yet, they are very promising. Let's follow our PIX/OpenSSL scenario:

DiffDeluxe operates with two "fuzzy" values for each function match: "Similarity" and "Confidence". Similarity indiciates how successful the matching algorithm was in matching basic blocks and instructions within the two functions, and confidence indicates how "certain" DiffDeluxe is that this match is a correct one. This is useful to sort the "good" and "bad" matches, and to inspect results before porting comments/names. Anyhow, let's look at some high-confidence matches:

Well, one doesn't need to be a rocket scientist to see that these functions match. But in many situations, the similarity between two functions is not 100% evident: The following is a matched function with only 72% similarity (but 92% confidence):

So what is the overall result ? Out of the 3977 functions which we had in libcrypto.so, we were able to match 1780 in our Pix disassembly -- but with a big caveat: A significant number of these have very low similarity and confidence scores. This isn't surprising: The differences between the compiler used upon compile time of our Pix image (sometime 6 years ago ?) and the compiler we used (gcc 4.1, -O3) is drastic. All in all, we end up with around 250 high-confidence matches -- which is not too bad considering that we don't know how many functions from OpenSSL the Pix code actually contains.

In order to have a more clear idea of how well these algorithms perform, we need an example of which we know that essentially the entire library has been statically linked in. For this, luckily, we have Adobe Reader :-)

With all the Adobe patches coming up, let's imagine we'd like to have a look at the Javascript implementation in Acrobat Reader. It can be found in Escript.api. Now, I always presume that everybody else is as lazy as me, so I can't imagine Adobe wrote their own Javascript implementation. But when Adobe added Javascript to Acrobat Reader, there were few public implementations of Javascript around -- essentially only the engine that is nowadays known as "SpiderMonkey", e.g. the Mozilla Javascript engine. So I compiled SpiderMonkey into "libjs.so" on my Linux machine and disassembled Escript.api. Then I ran DiffDeluxe. The result:

Escript contains about 9100 functions, libjs.so contains about 1900. After running the diff, we get 1542 matches. Let's start verifying how "good" these matches are. As discussed above, DiffDeluxe uses a "similarity" and "confidence" score to rate matches. We get 203 matches with similarity and confidence above 90% -- for these functions, we can more or less blindly assume the matches are correct. If we have any doubts, we can inspect them:

Well, there is little question that this match was accurate.

The interesting question is really: How low can we go similarity- and confidence-wise before the results start deteriorating too badly ? Let's go low -- for similarities below 40%. For example the js_ConcatStrings match.

Manual inspection of the screenshot on the right will show that the code performs equivalent tasks, but that hardly any instructions remain identical.

Proceeding further down the list of matches, it turns out that results start deteriorating once both confidence and similarity drop below 0.3 -- but we have around 950 matches with higher scores, e.g. we have successfully identified 950 functions in Escript.api. While this is signifcantly less than the 1900 functions that we perhabs could have identified, it is still pretty impressive: After all, we do not know which exact version of SpiderMonkey was used to compile Escript.api, and significant changes could have been made to the code.

Clearly, we're a long way from matching 95% -- but we're very close to the 50% barrier, and will work hard to improve the 50% to 75% and beyond :-)

Anyhow, what does all this have to do with automatic classification and correlation of malware ?

I think the drastic differences induced by platform/compiler changes make it pretty clear that statistical measures that do not focus on the structure and semantics of the executable, but on some "simple" measure like instruction frequencies, fail. All the time. Behaviorial methods might have a role to play, but they will not help you one bit if you acquire memory from a compromised machine, and are trivially obfuscated by adding random noisy OS interaction.

I am happy to kill two birds with one stone: By improving the comparison engine, I am making my life easier when I have to disassemble Pix -- and at the same time, I am improving the our malware classification engine. Yay :-)

Anyhow, as mentioned above: I am at the Virus Bulletin conference -- if anyone wishes to have a chat or have our products demo'ed, please do not hesitate to send mail to info@zynamics.com.

2008-07-31T02:20:00.000-07:00

My 100th blog post, and why my blog entries never have titles.

Hey all, this is my 100th blog post. And again, it has no title. This is not due to me feeling too cool to provide one, it's simply a matter of my "create" window in blogger not having a title field. I don't know why.

Anyhow, the real reason for the blog post: As of today, I'm done with my exams. Which makes me very happy, and will hopefully mean I will get around to blogging more often.

2008-07-25T23:20:00.000-07:00

I think everybody should read FX's excellent post.

2008-07-22T14:33:00.000-07:00

A few short notes on what's being reported:

It seems that after my previous speculation, a few unforeseen things happened:

Apparently, my post, while partially incorrect, was somewhere close to the truth
A third party accidentally posted full details on the issue, which corrected my mistakes. Shortly after posting these details, the post was pulled down again, but was archived by search engines (and those that had subscribed to the blog where it was posted).

There have been a number of slightly incorrect press reports which I'd like to clarify:

I posted a partially incorrect, but close, guess on what the DNS issue might be. That is not the same as "publishing a reliable way to poison DNS". It is guessing how it might be done.
I did not pull down any posts from my blog.

I do not think anything I have posted takes away from Dan's superb work on this issue. Some people are of the opinion that I "stole his thunder" for his Blackhat talk, and I disagree strongly: Dan's talk is a full hour on DNS, and all the interesting things within DNS. My post was a vague guess.

Imagine: A world-renowned particle physics expert decides to give a one-hour lecture in your hometown, and on your way there some guy on the street tells you "I think he will talk about (...30 seconds of physics here...)". Would you decide that listening to the physics expert talk is no longer necessary because the guy on the street told you everything ?

Also: Guessing how something is done knowing it can be done is easy. Dan did the hard part: Coming up with a clever attack in a protocol that is relied on everywhere. My guess doesn't come close to comparing to what Dan has done: He spotted something that everyone else missed beforehand. He also handled the entire situation with a lot of endurance, patience, and determination. We disagree on whether people have a right (or even duty) to discuss what the issue might be, but that doesn't mean that I do not have the greatest respect for Dan. And his talk will contain much more of interest than my silly 30 lines.

I think (German news site) Heise summed it up well:
"In fact, all of Dullien's hunches had already been sketched out the day that US-CERT published a vulnerability note on the security hole."

I guessed. I was close, perhabs closer than others, but no cigar.

2008-07-21T01:17:00.000-07:00

On Dan's request for "no speculation please"

I know that Dan asked the public researchers to "not speculate publicly" about the vulnerability, in order to buy people time. This is a commendable goal. I respect Dans viewpoint, but I disagree that this buys anyone time (more on this below). I am fully in agreement with the entire way he handled the vulnerability (e.g. getting the vendors on board, getting the patches made and released, and I understand his decision not to disclose extra information) except the proposed "discussion blackout".

In a strange way, if nobody speculates publicly, we are pulling wool over the eyes of the general public, and ourselves. Consider the following:

Let's assume that the DNS problem is sufficiently complicated that an average person that has _some_ background in security, but little idea of protocols or DNS, would take N days to figure out what is problem is.
So clearly, the assumption behind the "discussion blackout" is that no evil person will figure it out before the end of the N days.

Let's say instead of having an average person with _some_ background in security, we have a particularly bright evil person. Perhaps someone whose income depends on phishing, and who is at the same time bright enough to build a reasonably complicated rootkit. This person is smart, and has a clear financial incentive to figure this out. I'd argue that it would take him N/4 days.

By asking the community not to publicly speculate, we make sure that we have no idea what N actually is. We are not buying anybody time, we are buying people a warm and fuzzy feeling.

It is imaginable that N is something like 4 days. We don't know, because there's no public speculation.

So in that case, we are giving people 29 days of "Thank us for buying you time.", when in fact we have bought them a false perception of having time. The actual time they have is N/4th, and we're just making sure they think that N/4th > 30. Which it might not be. It might be ... 1.

It all reminds me of a strange joke I was told last week. It's a russian joke that makes fun of the former east german government, so it might not be funny to everyone. I apologize up front: I am both german and a mathematician, so by definition the following can't be funny.

"Lenin travels with the train through Russia, and the train grinds to a halt. Engine failure. Lenin sends all workers in the factory that might be responsible to a labor camp.

Stalin travels with the train through Russia a few years later, and the train grinds to a halt. Engine failure. Stalin has all workers in the factory that might be responsible shot.

Honecker (the former head of State of the GDR) travels with the train through Russia. The train grinds to a halt. Engine failure. Honecker has a brilliant idea: "The people that are responsible should be forced to rock the train, so we can sit inside and feel like it is still running." "

It feels like we're all trying to rock the train.

If there was public speculation, we'd at least get a lower boundary on the "real" N, not the N we wish for.

So I will speculate.

The last weeks I was in the middle of preparing for an exam, so I really didn't have time to spend on the DNS flaw. I couldn't help myself though and spent a few minutes every other evening or so reading a DNS-for-dummies-text. I have done pretty much no protocol work in my life, so I have little hope for having gotten close to the truth.

As such, anyone with a clue will probably laugh at my naive ideas. Here's my speculation:

Mallory wants to poison DNS lookups on server ns.polya.com for the domain www.gmx.net. The nameserver
for gmx.net is ns.gmx.net. Mallory's IP is 244.244.244.244.

Mallory begins to send bogus requests for www.ulam00001.com, www.ulam00002.com ... to ns.polya.com.
ns.polya.com doesn't have these requests cached, so it asks a root server "where can I find the .com NS?"
It then receives a referral to the .com NS. It asks the nameserver for .com where to find the nameserver for ulam00001.com, ulam00002.com etc.

Mallory spoofs referrals claiming to come from the .com nameserver to ns.polya.com. In these referrals, it says that the nameserver responsible for ulamYYYYY.com is a server called ns.gmx.net and that this server is located at 244.244.244.244. Also, the time to live of this referral is ... long ...

Now eventually, Mallory will get one such referral spoofed right, e.g. the TXID etc. will be guessed properly.

ns.polya.com will then cache that ns.gmx.net can be found at ... 244.244.244.244. Yay.

The above is almost certainly wrong. Can someone with more insight into DNS tell me why it won't work ?

2008-07-13T09:59:00.000-07:00

*Blogspam*
Advanced Reverse Engineering Trainings Class

We still have a number of seats in our advanced RE class available. The class
will be held on the following three days:

Wednesday the 1st of October
Thursday the 2nd of October
Friday the 3rd of October

The class will be held in Frankfurt(Main) in Germany.
The class is limited to 17 students and will cover a lot of interesting ground. Amongst the things we will be teaching are:

What a C++ compiler does and how to recognize these things in a binary:

How to recover classes and inheritance,

What templates will do in the binary

Using the helping hand of MS RTTI to recover classnames and generate inheritance diagrams from the binary

Getting the most out of the RE-DB SQL schema -- storing disassemblies in a uniform way in a database
Differential debugging and isolation of security-critical features (e.g. "where in the world is the encryption code again ?")
Crafting malicious input to reach target program locations
Working on network infrastructure:

Loading ROM images into IDA: IOS, Netscreen etc.

Generic methods of identifying the base address
Debugging IOS (and other network infrastructure) using BinNavi and the GDB protocol

Using BinDiff to full advantage:

Patch Diffing
Porting comments & names
Porting symbols of statically linked libraries (such as OpenSSL) back into your disassembly

A reverse engineer's guide to static analysis:

The reverse engineering intermediate language REIL
Monotone frameworks, lattices, and fun things to do with them

Lots and lots of fun things to do with Python

The class will be taught by me (Halvar Flake), Ero Carrera, and Felix 'Fx' Lindner.

The class will be held in a small Hotel called "Villa Orange" -- which has about 20 rooms, so usually the entire Hotel consists of reverse engineers.

For more info, visit
http://www.zynamics.com/index.php?page=trainings

Cheers,
Halvar
PS: It might be of interest to some readers that the Oktoberfest is from the 20th of September to the 5th of October this year -- this means you can either attend Octoberfest before or after the trainings class (although we recommend the latter).
*End of Blogspam*

2008-07-13T04:37:00.000-07:00

Hey all,

> Supplemental note to Halvar & everybody else who has said, in effect, "this
> is why SSL was invented" -- there's more to internet security than the route
> from your computer to your online bank. Have you thought about what this
> bug implies for NTLM? Or every virgin OS installation on the planet? Or
> Google's entire business model?

just to clarify: I did not say this bug wasn't relevant, and I don't want my blog post to be construed
in that manner. What I did say was:

The average user always has to assume that his GW is owned, hence nothing changes for him. Specifically: He does not need to worry more than usual. Check SSL certificates, check host fingerprints. Don't use plaintext protocols.
For those providing DNS services, it is clearly preferrable to patch. A DNS system without trivial poisoning is preferrable to one with trivial poisoning.
In living memory, we have survived repeated Bind remote exploits, SSH remote exploits, a good number of OpenSSL remote exploits etc. -- I argue that the following inequality holds:
OpenSSL remote >= OpenSSH remote > Bind remote > easy DNS poisoning
I argue this because the left-hand side usually implies the right-hand side given some time & creativity.

The net has survived much worse.

So I guess summary is: Good find, definitely useful for an attacker, but we have survived much worse without a need for the great-vendor-coordination jazz.

Cheers,
Halvar
PS: I am aware that my sangfroid could be likened to a russian roulette player, that after winning 4 games concludes: "This game clearly isn't dangerous."
PPS: It seems that we will find many more critical issues in DNS over the next weeks - it's the first time in years that a significant quantity of people look at the protocol / implementations.

2008-07-10T10:09:00.000-07:00

All this DNS ...

I am taking a very brief break from my books to write a few thoughts about this entire DNS thing that everybody seems to be writing about. And reading all this, I can't help but feel like the only one in the room that doesn't understand the joke.

So Dan Kaminsky found a serious flaw in the implementation of the DNS protocol, apparently allowing DNS cache poisoning. This is good work.

I fail to understand the seriousness with which this bug is handled though. Anybody who uses the Internet has to assume that his gateway is owned. That is why we have SSL, that is why we have certificates, that is why SSH tells you when the host key changes. DNS can never be trusted - you always have to assume that your ISP's admin runs a broken filesharing server on the same box with BIND.

If it were legitimate to operate under the assumption that your gateway is not owned, you would not need SSH, or SSL. If I could operate under the assumption that my gateway wasn't owned, I could TELNET everywhere, and transmit my credit card details in the clear.

I am not saying that Dan's bug doesn't have utility for an attacker -- it's definitely more comfortable/less time consuming to do DNS poisoning than to own the gateway. But for the user, nothing changes, irrespective of whether the patch was applied or not. The basic assumption is always my gateway is controlled by my opponent.

I personally think we've seen much worse problems than this in living memory. I'd argue that the Debian Debacle was an order of magnitude (or two) worse, and I'd argue that OpenSSH bugs a few years back were worse.

So, let's calm down everybody. And I'd even argue that installing the patches is a lot less time-critical (for the user) than in most other scenarios. If you act under the assumption of "my gateway is owned", this should be no risk to you.

2008-07-02T07:26:00.001-07:00

The security book that I'd like to see written (and which I'd buy)

Good security books are few and far between. But IF someone writes the following book, I'll pre-order it immediately, even if it costs a hundred dollars:

"100 UNIX commands to issue on other people's systems"

Generally, I am horrible at all things *nix, and there are few enough good books around which teach you clever things to do with a shell. Unfortunately, there is no book that teaches people what to do with a shell on someone else's box.

Someone from Matasano told me they'd post their favourite commands if I wrote this blog post - so let's see it ! :)

(I'd like to start this by posting, but honestly -- I wouldn't be asking if I knew anything I'd not be embarrassed about. I mentioned above that I suck at all things *nix)

2008-06-28T10:05:00.000-07:00

The RE-DB database format for storing disassemblies

For those of you that are interested in the disassembly database schema discussed here (amongst other places), there is a mailing list for discussion of it now. More information about the ML:

  http://lists.immunityinc.com/mailman/listinfo/re-db

2008-06-15T21:03:00.000-07:00

Intuition, Experience, and the value of getting Pwned

The following is to be taken mostly proverbially. Names have been changed, primarily to protect my bruised ego.

There are few things that I hate more than looking stupid or incompetent. At the same time I like trying new things (and this rarely happens without falling flat on your face a couple of dozen times). As a result, I usually do not advertise that I do something before I haven't gotten some confidence in at least not being significantly worse than average.

So tonight, I had my first free evening in a few weeks. I decided I'd go follow one of my not-publicly-advertised hobbies. I found a place to go, and thought that I was good enough to play.

I got pwned, and it wasn't pretty.

There are many different ways of competing and losing. Whenever this happens, it happens with a certain "delta" -- the skill gap between you and your opponent(s). Small deltas usually trigger a reaction of "get up, try again" in me.

Tonight, the delta between me and the weakest competitor was such a gulf that - within minutes - it was clear that I should practice a few more years before I contemplate coming back. I will not even describe what the delta between me and the stronger competitors was.

Getting knocked down has one great benefit: After you have been knocked down and realized that there is no sense in getting up quickly, you have a few minutes of extraordinary calm to contemplate the situation - your skill level, your competitors' skill level, the value of experience and intuition.

No matter how much work you put into something, and no matter how much talent you have, intuition and experience have tremendous value. And they are nigh-impossible to teach, and to accumulate quickly.

What is intuition ? What is its relation to experience ?

Intuition is what one bases decisions on when knowledge fails. In any field, there are situations where decisions have to be made with very imperfect and incomplete information. Intuition is what we rely on when we don't know anything.

Intuition is usually based on experience - but whereas one can easily talk about "experiences" (they can be recalled usually), talking about the reasoning behind an intuition is often difficult. If one believes in the theory of two brain hemispheres, intuition lives deeply in the nonverbal part of your brain.

When I teach classes, or do collaborative code audits, or when I do some sorts of math, I end up in situations where I have a "feeling" about how things "should" be. This feeling is both tremendously useful and horribly frustrating for students and coworkers. The difficulty of verbalizing all the bits that feed an intuition makes it difficult to follow.

If someone has sufficient experience in a field, some of the things he does seem like magic. My competitors this evening clearly did things I had never seen before, and did so quite well.

Perhabs a skill can be described as a simple real-valued function.

Your innate talent and your work investment influence the slope, and the value of the function at a particular point tells you your current direct "knowledge" of a field. Intuition must then be something that is based on the accumulated area under the curve.

In many situations, it might be possible to catch up with someone experienced on a particular topic in a limited timeframe - but catching up with the value of your "function" is only half the game. You'll have to outperform someone for quite a while before your accumulated "area" exceeds his.

Anyhow, the one thing that I tell myself to get over this is that I was the youngest man in the room by a gap of about 10 years. So I'd like to tell myself that, given that extra 10 years, I could actually compete.

There's one caveat though: There were several women that were younger than me, and the delta to them was no less than to any of the men.

I apologize for the excessive vagueness of this post.

2008-06-15T08:42:00.001-07:00

Travelling & Dopplr

Btw, how many people that travel a lot are using Dopplr ? It seems like
a somewhat clever idea (as I am stuck in silly hotel rooms a lot and
often wonder wether anyone I know is nearby).

2008-06-12T10:24:00.000-07:00

Zynamics Canada Tour, Complex analysis and my stupidity

Hey all -- I know I've been mostly quiet the last weeks. This was principally due to the combination of lots of work at work (the secretary is on vacation) and me having to take a couple of exams.

I can proudly proclaim that I passed my complex analysis / riemann surfaces exam today. I am not so proud of my performance -- some of the mistakes I made deserve getting my shins kicked. The final grade was pretty ok, I just really hate looking stupid in front of people I deem smart.

Anyhow, on to other news:

It's RECon time, and while I cannot attend due to a number of other obligations :-( our BinNavi lead developer Sebastian is attending. So if anyone that is attending RECon would like to have a demo of BinNavi v1.5 OR discuss the cool new things that BinNavi v2 will bring, make sure to drop info@zynamics.com a mail so that we can schedule something.

2008-04-28T15:26:00.000-07:00

There's a lot of hoopla in German media about the german SIGINT folks having to admit that they trojanized Afghanistan's Ministry of Commerce and Industry.

The entire situation is hilarious, as Mrs. Merkel criticized the chinese for having sponsored hacking sprees into German government institutions last year - I guess she is not overly happy about all this stuff hitting the press now.

The first article is actually quite interesting. It is terribly hard to get any information about InfoSec stuff in Europe (we'd need a Mr. Bamford around here I fear), so the article is really amongst the only data points to be found.

In 2006, Division 2 consisted of 13 specialist departments and a management team (Department 20A), employing about 1,000 people. The departments are known by their German acronyms, like MOFA (mobile and operational telecommunications intelligence gathering), FAKT (cable telecommunications intelligence gathering) and OPUS (operational support and wiretapping technology).

So there are people working on this sort of stuff in Germany after all. I wonder why one never meets any at any security conferences - they either have excellent covers or no budget to travel to any conferences.

Another amusing tidbit:

Perhaps it will never be fully clear why the BND chose this particular ministry and whether other government agencies in Kabul were also affected -- most of the files relating to the case have apparently been destroyed.

I find the regularity with which important files regarding espionage or KSK misbehavior are destroyed or lost a little bit ... peculiar.

There's a bit in the article about emails that have a .de domain ending being automatically discarded by their surveillance tools. Hilarious.

The issue came to light because during the surveillance a German reporter had her email read, too (she was communicating with an Afghan official whose emails were being read). This is a violation of the freedom of the press here in Germany, and normally, the BND should've dealt with this by reporting their breach to the parliamentary subcommittee for intelligence oversight, which they somehow didn't. A whistleblower inside the BND then sent a letter to a bunch of politicians, making the situation public.

It's always hard to make any judgements in cases as these, as the public information is prone to being unreliable, but it is encouraging that a whistleblower had the guts to send a letter out. I am a big fan of the notion that everyone is personally responsible for his democracy.

The topic of intelligence and democracies is always difficult: If one accepts the necessity of intelligence services (which, by their nature, operate in dodgy terrain, and which, due to their requirements for secrecy, are difficult to control democratically), then one has to make sure that parliamentary oversight works well. This implies that the intelligence agencies properly inform the parliamentary committee, and it also implies that the parliamentary committee keeps the information provided confidential.

There seem to be only two ways to construct parliamentary oversight in a democracy: Pre-operation or post-operation. Pre-operation would have the committee approve of any potentially problematic operation ahead of it being performed. If things go spectacularly wrong, the fault is to be blamed on the committee. The problem with this is secrecy: Such a committee is big, and for operational security it seems dangerous to disseminate any information this widely.

This appears to be the reason why most democracies seem to opt for a "post-operation" model: The services have in-house legal experts, and these legal experts judge on the 'legality' of a certain operation. The the operation takes place, and the committee is notified after the fact if something goes spectacularly wrong.

The trouble with this model appears to be that the intelligence service doesn't have much incentive to report any problems: They can always hope the problem goes away by itself. It is the higher-ups in the hierarchy that have to report to the committee, and they are the ones whose heads will roll if things go wrong.

It appears to be an organisational problem: Information is supposed to flow upwards in the organisational hierarchy, but at the same time, the messenger might be shot. This is almost certain to lead to a situation where important information is withheld.

I guess it's any managers nightmare that his "subordinates" (horrible word -- this should mean "the guys doing the work and understanding the issues") in the organisation start feeding him misinformation. Organisations start rotting quickly if the bottom-up flow of information is disrupted. The way things are set up here in Germany seems to encourage such disruptions. And if mid-level management is a failure but blocks this information from upper management, the guys in the trenches have not only the right, but the duty to send a letter to upper management.

I have no clue if there is any country that has these things organized in a better way -- it seems these problems haunt most democracies.

Anyhow, if anyone happens to stumble across the particular software used in this case, I think it would make for a terribly interesting weekend of reverse engineering -- I am terribly nosy to what sort of stuff the tool was capable of :)

Cheers,
Halvar

2008-04-25T05:54:00.000-07:00

Patch obfuscation etc.

So it seems the APEG paper is getting a lot of attention these days, and some of the conclusions that are (IMO falsely) drawn from it are:

patch time to exploit is approaching zero
patches should be obfuscated

Before I go into details, a short summary of the paper:

BinDiff-style algorithms are used to find changes between the patched and unpatched version
The vulnerable locations are identified.
Constraint formulas are generated from the code via three different methods:

Static: A graph of all basic blocks on code paths between the vulnerability and the data input into the application is generated, and a constraint formula is generated from this graph.
Dynamic: An execution trace is taken, and if the vulnerability occurs on a program path that one can already execute. Constraints are generated from this path.
Dynamic/Static: Instead of going from data input to target vulnerability (as in the static approach), one can use an existing path that comes "close" to the vulnerability as starting point from which to proceed with the static approach.

The (very powerful) solver STP is used for solving these constraint systems, generating inputs that exercise a particular code path that triggers the vulnerability.
A number of vulnerabilities are discussed which were successfully triggered using the methods described in the paper
The conclusion is drawn that within minutes of receiving a patch, attackers can use automatically generated exploits to compromise systems.

In essence, the paper implements automated input crafting. The desire to do this has been described before -- Sherri Sparks' talk on "Sidewinder" (using genetic algorithms to generate inputs to exercise a particular path) comes to mind, and many discussions about generating a SAT problem from a particular program path to be fed into a SAT solver (or any other solver for that matter).

What the APEG paper describes is impressive -- using STP is definitely a step forwards, as it appears that STP is a much superior solver to pretty much everything else that's publically available.

It is equally important to keep the limitations of this approach in mind - people are reacting in a panicked manner without necessarily understanding what this can and cannot do.

Possible NP-hardness of the problem. Solving for a particular path is essentially an instance of SAT, and we know that this can be NP-hard. It doesn't have to be, but the paper indicates many formulas STP cannot solve in reasonable time. While this doesn't imply that these formulas are in fact hard to solve, it shows how much this depends on the quality of your solver and the complexity of the formulas that are generated.
The method described in the paper does not generate exploits. It triggers vulnerabilities. Anyone who has worked on even a moderately complex issue in the past knows that there is often a long and painful path between triggering an overflow and making use of it. The paper implies that the results of APEG are immediately available to compromise systems. This is, plainly, not correct. If APEG is successful, the results can be used to cause a crash of a process, and I refuse to call this a "compromise". Shooting a foreign politician is not equal to having your intelligence agency compromise him.
Semantic issues. All vulnerabilities for which this method worked were extremely simple. The actual interesting IGMP overflow Alex Wheeler had discovered, for example, would not be easily dealt with by these methods -- because program state has to be modified for that exploit in a non-trivial way. In essence, a patch can tell you that "this value YY must not exceed XX", but if YY is not direct user data but indirectly calculated through other program events, it is not (yet) possible to automatically set YY.

So in short one could say that APEG will succeed in triggering a vulnerability if the following conditions are met:

The program path between the vulnerability and code that one already knows how to execute is comparatively simple
The generated equation systems are not too complex for the solver
The bug is "linear" in the sense that no complicated manipulation of program state is required to trigger the vulnerability

This is still very impressive stuff, but it reads a lot less dramatic than "one can generate an exploit automatically from an arbitrary patch". All in all, great work, and I do not cease to be amazed by the results that STP has brought to code analysis in general. It confirms that better solvers ==> better code analysis.

What the paper gets wrong IMO are the conclusions about what should be done in the patching process. It argues that because "exploits can be generated automatically, the patching process needs fixing". This is a flawed argument, as ... uhm ... useful exploits can't (yet) be generated automatically. Triggering a vulnerability is not the same as exploiting it, especially under modern operating systems (due to ASLR/DEP/Pax/GrSec).

The paper proposes a number of ways of fixing the problems with the current patching process:

1. Patch obfuscation. The proposal that zombie-like comes back every few years: Let's obfuscate security patches, and all will be good. The problems with this are multifold, and quite scary:

Obfuscated executables make debugging for MS ... uhm ... horrible, unless they can undo it themselves
Obfuscated patches remove an essential liberty for the user: The liberty to have a look at a patch and make sure that the patch isn't in fact a malicious backdoor.
We don't have good obfuscation methods that do not carry a horrible performance impact.
Obfuscation methods have the property that they need to be modified whenever attackers break them automatically. The trouble is: Nobody would know if the attackers have broken them. It is thus safe to assume that after a while, the obfuscation would be broken, but nobody would be aware of it.
Summary: Obfuscation would probably a) impact the user by making his code slower and b) impact the user by disallowing him from verifying that a patch is not malicious and c) create support nightmares for MS because they will have to debug obfuscated code. At the same time, it will not provide long-term security.

2. Patch encryption: Distributing encrypted patches, and then finally distributing the encryption key so all systems update at once. This proposal seems to assume that bandwidth is the limiting factor in patch installation, which, as far as I can tell, it is not. This proposal does less damage than obfuscation though -- instead of creating certain disaster with questionable benefit, this proposal just "does nothing" with questionable benefit.

3. Faster patch distribution. A laudable goal, nothing wrong with this.

Anyhow, long post, short summary: The APEG paper is really good, but it uses confusing terminology (exploit ~= vulnerability trigger) which leads to it's impact on patch distribution being significantly overstated. It's good work, but the sky isn't falling, and we are far away from generating reliable exploits automatically from arbitrary patches. APEG does generate usable vulnerability triggers for vulnerabilities of a certain form. And STP-style solvers are important.

2008-04-25T03:18:00.000-07:00

I have not been blogging nor following the news much in recent months, as I am frantically trying to get all my university work sorted. While I have been unsuccessful at getting everything sorted at the schedule I had set myself, I am making progress, and expect to be more visibly active again in fall.

Today, I found out that my blog entry on the BlueHat blog drew more feedback than I had thought. I am consistently surprised that people read the things that I write.

Reading my blog post again, I find it so terse I feel I have to apologize for it and explain how it ended up this way. It was the last day of Bluehat, and I was very tired. Those that know me know me well know that my sense of humor is difficult at the best of times. I have a great talent of sounding bitter and sarcastic when in fact I am trying to be funny and friendly (this had lead to many unfortunate situations in my life :-). So I sat down and tried to write a funny blog post. I was quite happy with it when it was done.

In an attack of unexpected sanity, I decided that someone else should read over the post, so I asked Nitin, a very smart (and outrageously polite) MS engineer. He read it, and told me (in his usual very polite manner) ... that the post sucked. I have to be eternally thankful to him, because truly, it did. Thanks Nitin !

So I deleted it, and decided that writing down just the core points of the first post. I removed all ill-conceived attempts at humor, which made the post almost readable. It also limited the room for potential misunderstandings.

I would like to clarify a few things that seem to have been misunderstood though:

I did not say "hackers have to" move to greener pastures. I said "hackers will move to greener pastures for a while". This is a very important distinction. In order to clarify this, I will have to draw a bit of a larger arc:

Attackers are, at their heart, opportunists. Attacks go by the old basketball saying about jumpshot technique: "Whoever scores is right". There is no "wrong" way of compromising a system. Success counts, and very little else.

When attackers pick targets, they consider the following dimensions:

Strategic position of the target. I will not go into this (albeit important) point too deeply. Let's just assume that, since we're discussing Vista (a desktop OS), the attacker has made up his mind and wishes to compromise a client machine.
Impact by market share: The more people you can hack, the better. A widely-installed piece of software beats a non-widely installed piece of software in most cases. There's many ways of doing this (Personal estimates, Gartner reports, internet-wide scans etc.).
Wiggle Room: How many ways are there for the attacker to interact with the software ? How much functionality does the software have that operates on potentially attacker-supplied data ? If there are many ways to interact with the application, the odds of being able to turn a bug into a usable attack are greatly increased, and the odds of being able to reach vulnerable code locations are greatly increased. Perhabs the more widely used term is "attack surface", but that term fails to convey the importance of "wiggle room" for exploit reliability. Any interaction with the program is useful.
Estimated quality of code: Finding useful bugs is actually quite time consuming. With some experience, a few glances at the code will give an experienced attacker some sort of "gut feeling" about the overall quality of the code.

From these four points, it is clear why IE and MSRPC got hammered so badly in the past: They pretty much had optimal scores on Impact -- they were everywhere. They provided plenty of "Wiggle Room": IE with client-side scripting (yay!), MSRPC through the sheer number of different RPC calls available. The code quality was favourable to the attacker up until WinXP SP2, too.

MS has put more money into SDL than most other software vendors. This holds true both in absolute and in relative terms. MS is in a very strong position economically, so they can afford things other vendors (who, contrastingly, are exposed to market forces) cannot.

The code quality has improved markedly, decreasing the score on the 4th dimension. Likewise, there has been some reduction in attack surface, decreasing the score on the 3rd dimension. This is enough to convince attackers that their time is better spent on 'weaker' targets. The old chestnut about "you don't have to outrun the bear, you just have to outrun your co-hikers" holds true in security more than anywhere else.

In the end, it is much more attractive to attack Flash (maximum score on all dimensions) or any other browser plugins that are widely used.

I stand by my quote that "Vista is arguably the most secure closed-source OS available on the market".

This doesn't mean it's flawless. It just means it's more secure than previous versions of Windows, and more secure than OS X.

There was a second part to my blog post, where I mentioned that attackers are waiting for MS to become complacent again. I have read that many people inside Microsoft cannot imagine becoming complacent on security again. While I think this is true on the engineering level, it is imaginable that security might be scaled down by management.

The sluggish adoption of Vista by end-users is a clear sign that security does not necessarily sell. People buy features, and they cannot judge the relative security of the system. It is thus imaginable that people concerned with the bottom line decide to emphasize features over security again -- in the end, MS is a business, and the business benefits of investing in making code more secure have yet to materialize.

We'll see how this all plays out :-)

Anyhow, the next BlueHat is coming up. I won't attend this time, but I am certain that it will be an interesting event.

2008-04-02T11:37:00.000-07:00

My valued coworker, SP, has just released his "pet project", Hexer. Hexer is a platform-independent Java-based extendible hex editor and can be downloaded under http://www.zynamics.com/files/Hexer-1_0_0.rar

It's also a good idea to visit his blog where he'll write more about it's features and capabilities.

2008-04-01T16:30:00.001-07:00

Oh, before I forget: Ero & me will be presenting on our work on structural malware classification at RSA next week. If anyone wishes to schedule a meeting/demo of any of our things (VxClass/BinDiff/BinNavi), please do not hesitate to contact info@zynamics.com.

Some small eye candy: The screenshot shows BinNavi with our intermediate representation (REIL) made visible. While REIL is still very beta-ish, it should be a standard (and accessible) part of BinNavi at some point later this year.

Having a good IR which properly models side effects is a really useful thing to have: The guys over at the BitBlazer project in Berkeley have shown some really useful things that can be done using a good IR and a good constraint solver :-). I am positively impressed by several papers they have put out.

I also can't wait to have more of this sort of stuff in BinNavi :-).

2008-04-01T16:24:00.000-07:00

Conspiracy theory of the day:

As everyone, I am following the US primaries, and occasionally discussing with my brother on the implications of the developments for the wider world. My brother is usually good for quite some counter-intuitive insights into things, and described to me a "conspiracy theory" that I find amusing/interesting enough to post here.

Please be aware that the following is non-partisan: I do not really have an idea on whether I'd prefer Mrs Clinton, Mr Obama or Mr McCain in the white house, and this is not a post that is intended to weigh in on either side.

I was a bit puzzled on why Mrs Clinton is still in the primary race even though her mathematical odds on winning the democratic nomination seem slim. The conspiracy theory explaining this is the following:

The true goal now for Mrs Clinton is now 2012, not 2008. If Mr Obama wins the nomination _and_ the presidency, Mrs Clinton will very likely not become president in her lifetime. On the other hand: If she manages to damage Mr Obama bad enough so that Mr McCain enters the white house, she has good cards to win the democratic nomination in 2012, and Mr McCain is unlikely to stay a second term (given his age).

It's an interesting hypothesis. Anyhow, I should really get to sleep.

2008-03-11T12:04:00.000-07:00

A short real-life story on why cryptography breaks:

One of the machines that I am using is a vhost hosted at a german hosting provider called "1und1". Clearly, I am accessing this machine using ssh. So a few weeks ago, to my surprise, my ssh warned me about the host key having changed.

Honored by the thought that someone might take the effort to mount a man-in-the-middle attack for this particular box, my rational brain told me that I should call the tech support of the hosting provider first and ask if any event might've lead to a change in keys.

After a rather lengthy interaction with the tech support (who first tried to brush me off by telling me to "just accept the new key"), I finally got them to tell me that they upgraded the OS and that the key had changed. After about 20 minutes of discussion, I finally got them to read the new key to me over the phone, and all was good.

Then, today, the warning cropped up again. I called tech support, a bit annoyed by these frequent changes. My experience was less than stellar - the advice I received was:

"Just accept the new key"
"The key is likely going to change all the time due to frequent relocations of the vhost so you should always accept it"
"No, there is no way that they can notify me over the phone or in a signed email when the key changes"
"It is highly unlikely that any change that would notify you would be implemented"
"If I am concerned about security, I should really buy an SSL certificate from them" (wtf ??)
"No, it is not possible to read me the key fingerprint over the phone"

The situation got better by the minute. After I told them that last time the helpful support had at least read me the fingerprint over the phone, the support person asked how I could be sure that my telephone call hadn't been man-in-the-middled...

I started becoming slightly agitated at this point. I will speak with them again tomorrow, perhabs I'll be lucky enough to get to 3rd-level-support instead of 2nd level. Hrm. As if "customer service" is a computer game, with increasingly difficult levels.

So. Summary: 1und1 seems to think crypto is useless and we should all use telnet. Excellent :-/

2008-03-07T00:36:00.001-08:00

Hey all,

we have released BinNavi v1.5 last week. Normally, I'd write a lot of stuff here about the new features and all, but this will have to wait for a few days -- I am very tied up with some other work.

With the v1.5 release, we have added disassembly exporters that export from both OllyDbg and ImmunityDbg to our database format -- this means that Navi can now use disassemblies generated from those two debuggers, too. The screenshot above is BinNavi running on Ubuntu with a disassembly exported from the Windows VW into which we are debugging.

Anyhow, the real reason for this post is something completely different: We don't advertise this much on our website, but our tools are available in a sort of 'academic program':

If you are currently enrolled as a full-time-student at a university and have an interesting problem you'd like to use our tools for, you can get a license of our tools (Diff/Navi) for a very moderate amount of money. All you have to do is:

Contact us (info@zynamics.com) with your name/address/university etc.
Explain what project you'd like to work on with our tools
Sign an agreement that you will write a paper about your work (after it's done) that we can put on our website

Oh, and you of course have to do the work then and write the paper :-)
Anyhow, I have to get back to work. Expect more posts from me later this year -- things are very busy for me at the moment.

Cheers,
Halvar

2008-02-12T09:09:00.000-08:00

Hey all,

We will be releasing BinNavi v1.5 next week -- and I can happily say that we will have
many cool improvements that I will blog about next week, once it is out.

Pictures often speak louder than words, so I'll post some of them here:

http://www.zynamics.com/files/navi15.1.png
http://www.zynamics.com/files/navi15.2.png
http://www.zynamics.com/files/navi15.3.png
http://www.zynamics.com/files/tree_lookup.jpg

A more detailed list of new features will be posted next week.

VxClass is making progress as well -- but more on this next week.

If there's anyone interested in our products (BinDiff, BinNavi, VxClass)
in the DC area, I should be free to meet & do a presentation on the products
next week.

Cheers,
Halvar

2008-01-08T17:27:00.000-08:00

Happy new year everyone.

In June 2006 Dave Aitel wrote on Dailydave that "wormable bugs" are getting rarer. I think he is right, but this month's patch tuesday brings us a particularly cute bug.

I have created a small shockwave film and uploaded it to
http://www.zynamics.com/files/ms08001.swf

Enjoy ! :-)

On other news: We'll be posting screenshots of BinNavi v1.5 (due out in February) and the current VxClass version in the next two weeks - they are coming along nicely.

Cheers,
Halvar

2007-10-07T12:11:00.000-07:00

Our trainings class in Frankfurt is over, and I think I can safely say that it was a resounding success. I guess the coolest thing about SABRE is our customers. I hope to see you all again someplace again.

PS: I forgot to distribute the python code from the last day, it will be mailed to all participants on monday.

2007-09-24T03:36:00.000-07:00

Blackhat Japan

After the immigration SNAFU in summer, I am scheduled to give my trainings class at Blackhat Japan this November - so if anyone wants to come, sign up now :-)

Cheers,
Halvar

2007-09-04T06:48:00.000-07:00

BinDiff v2.0 finally released !

This is "blog-spam":

After a long wait, SABRE Security GmbH is proud to announce
the official release of BinDiff v2.0. This biggest improvements are:

Higher comparison speeds
Greater accuracy for functions which change only in the structure of the graph, not in the number of nodes/edges
Much greater accuracy on the instruction level comparison
The arguably prettiest UI of all binary comparison tools around

The many detail improvements are too numerous to mention here.
Check the screenshots:

Contact info@sabre-security.com for an evaluation version !

-- SABRE Security Team

2007-08-04T12:23:00.001-07:00

I am quite famous for botching every marketing effort that we try to undertake at SABRE -- a prime example of my ineptitude is the fact that we released BinNavi v1.2 in ... uh ... January, with a ton of new stuff, and I still hadn't updated the website to show some nice pictures.

Similarly for BinDiff -- v2.0 beta has been used by many customers without a hitch, and is a big improvement on the UI front. So I finally got around to adding some nice pictures today.

Also, for those that are into the entire idea of malware classification, you can see some screenshots of VxClass, our unpacker-and-classifier (Disclosure: Before Spender writes a comment ;) about our unpacker's inability to handle TheMida and similar emulating packers, I will do so myself: We do not handle emulating packers at the moment! We do not reconstruct PEs ! But if you have a cool unpacker you can just upload the unpacked file to our classifier :)

So with this blog post it's confirmed: I am not only a failure at marketing, I am also a failure at attempting to pass off marketing as a regular blog post. Have a good weekend everyone !

2007-08-02T16:11:00.000-07:00

I have reached the intellectual level of the sports spectator in an armchair: Comment first, read and understand later. After the last Blog comment, I actually went to read the slides of Joanna's presentation. To summarize: I find the slides informative and well-thought-out. I found that the empirical bits appear plausible and well-researched. The stuff following slide 90 was very informative. It is one of the most substantial slide decks I have read in recent times.

Some points to take home though: Whoever writes a rootkit puts himself in a defending positions. Defending positions against all known attacks is possible given perfection on the side of the defender. That is bloody hard to achieve. There is no doubt that for any given attack one can think of a counter attack, but it's a difficult game to play that doesn't allow for errors.

I think the core point that we should clarify is that rootkits should not fall into an adversary's hand to be analyzed. Once they are known, they fall into a defending position. Defending positions are not long-term substainable, as software has a hard time automatically adapting to new threats.

Once you accept that the key to a good rootkit is to use methods unknown to the victim, one might also be tempted to draw the conclusion that perhabs the virtualisation stuff is too obvious a place to attempt to hide in. But that is certainly open to discussion.

Enough high-level blah blah. I am so looking forwards to my vacation, it's not funny.
Post veröffentlichen

2007-08-02T15:45:00.000-07:00

So it appears the entire Rutkowska-Matasano thing is not over yet. I probably should not harp on about this in my current mood, but since I am missing out on the fun in Vegas, I'll be an armchair athlete and toss some unqualified comments from the sidelines. Just think of me as the grumpy old man with a big gut and a can of beer yelling at some football players on television that they should quit being lazy and run faster.

First point: The blue chicken defense outlined in the linked article is not a valid defense for a rootkit. The purpose of a rootkit is to hide data on the machine from someone looking for it. If a rootkit de-installs itself to hide from timing attacks, the data it used to hide either has to be removed or is no longer hidden. This defeats the purpose of the rootkit: To hide data and provide access to the compromised machine.

Second point: What would happen if a boxer who claims the ability to defeat anyone in the world would reject any challengers unless they pay 250 million for him to fight ? Could he claim victory by telling the press that he "tried out all his opponents punches, and they don't work, because you can duck them like this and parry them like that" ?
I think not.

I am not saying it's impossible to build a rootkit that goes undetected by Matasano's methods. But given access to the code of a rootkit and sufficient time, it will be possible to build a detection for it. Of course you can then change the rootkit again. And then the other side changes the detection. And this goes on for a few decades.

Could we please move on to more fruitful fields of discussion already ?

2007-07-31T12:06:00.000-07:00

Some people in the comments of my blog have hinted that I should have just "followed the rules" and nothing would have happened. This is incorrect -- I did follow the rules. It is perfectly legal for an independent contractor to be contracted to perform a task in the US, come in, do it, and leave. That is (amongst other things) what the "business" checkbox on the I94W is for.

What landed me in this trouble is that the immigration agent decided that even though I am CEO of a company in Germany and have no employment contract with Blackhat (just a contract as an independent contractor), that the status of "independent contractor" does not apply to me - his interpretation was that I was an "employee" of Blackhat without an H1B visa.

This is not a case of me screwing up my paperwork. This is a case of an immigration agent that did not understand my attempts at explaining that I am not a Blackhat employee, and me not knowing the subtleties of being interviewed by DHS/INS agents.

I hope I will be able to clarify the misunderstanding on Thursday morning at the consulate.
=============================
Small addition to clarify: It is perfectly legitimate to come to the US to hold lectures and trainings of the kind that I am holding at Blackhat. To reiterate: The problem originated solely from a misunderstanding where it was presumed I was an "employee" of a US company, which is not correct.

2007-07-29T12:34:00.001-07:00

Short update: I have managed to schedule a hearing for a regular visa. The first available date was the 24th of August *cough*.

While this is clearly too late for Blackhat, but once you have a "regular" meeting scheduled you can ask to have an "urgent" meeting scheduled, too. Wether I am eligible will become clear when the embassy opens at 7am on monday morning.

The current plan is to call them and explain them why the entire thing might've gone haywire in the first place:

There's a special provision in the german tax code that allows for people with certain qualifications to act as special 'freelancers', essentially giving them a status very similar to one-person-companies ("Freiberufler"). It is not totally trivial to obtain this status - for example, you cannot simply be a 'Freiberuf'-programmer if you write "regular" software.

My agreement with Blackhat and all transactions were taxed in Germany under this status.

Personally, I think the fundamental issue in this tragic comedy is that the US doesn't really have such a special status for freelancers, and that therefore the US customs inspector did not understand that there is a distinction between a "regular Joe" and a "single-person company/Freiberufler". Hence the customs officer assumed that this entire thing must be some devious way to bypass getting an H1B visa for someone that would not normally qualified to get one. The frequent repetition of the question "why is your course not given by an American Citizen ?" points to something like that.

I hope that I can clear up this misunderstanding tomorrow morning, but right now, I am not terribly optimistic.

2007-07-29T04:39:00.000-07:00

I've been denied entry to the US essentially for carrying my trainings material. Wow.

It appears I can't attend Blackhat this year. I was denied entry to the US for carrying trainings materials for the Blackhat trainings, and intending to hold these trainings as a private citizen instead of as a company.

After a 9-hour flight and a 4 1/2 hour interview I was put onto the next 9-hour flight back to Germany. Future trips to the US will be significantly more complicated as I can no longer go to the US on the visa waiver program.

A little background: For the last 7 years, I have attended / presented at the 'Blackhat Briefings', a security conference in the US. Prior to the conference itself, Blackhat conducts a trainings session, and for the past 6 years, I have given two days of trainings at these events. The largest part of the attendees of the trainings are US-Government related folks, mostly working on US National Security in some form. I have trained people from the DoD, DoE, DHS and most other agencies that come to mind.

Each time I came to the US, I told immigration that I was coming to the US to present at a conference and hold a trainings class. I was never stopped before.

This time, I had printed the materials for the trainings class in Germany and put them into my suitcase. Upon arrival in the US, I passed immigration, but was stopped in customs. My suitcase was searched, and I was asked about the trainings materials.
After answering that these are for the trainings I am conducting, an immigration officer was called, and I was put in an interview room.
For the next 4 1/2 hours I was interviewed about who exactly I am, why I am coming to the US, what the nature of my contract with Blackhat is, and why my trainings class is not performed by an American citizien. After 4 hours, it became clear that a decision had been reached that I was to be denied entry to the US, on the ground that since I am a private person conducting the trainings for Blackhat, I was essentially a Blackhat employee and would require an H1B visa to perform two days of trainings in the US.

Now, I am a full-time employee (and CEO) of a German company (startup with 5 people, self-financed), and the only reason why the agreement is between Blackhat and me instead of Blackhat and my company is that I founded the company long after I had started training for Blackhat and we never got around to changing it.

Had there been an agreement between my company and Blackhat, then my entry to the US would've been "German-company-sends-guy-to-US-to-perform-services", and everything would've been fine. The real problem is that the agreement was still between me as a person
and Blackhat.

After the situation became clear (around the 4th hour of being interviewed), I offered that the agreement between Blackhat and my company could be set up more or less instantaneously - as a CEO, I can sign an agreement on behalf of my company, and Blackhat would've signed immediately, too.
This would've spared each party of us a lot of hassle and paperwork. But apparently, since I had just tried to enter as a 'normal citizen' instead as an 'employee of a company', I could now not change my application. They would have to put me on the next flight back to Germany.

Ok, I thought, perhabs I will have to fly back to Germany, set up the agreement, and immediately fly back to the states - that would've still allowed me to hold the trainings and attend the conference, at the cost of crossing the Atlantic three times instead of once. But no such luck: Since I have been denied entry under the visa waiver programme, I can now never use this programme again. Instead I need to wait until the American consulate opens, and then apply for a business visa. I have not been able to determine how long this might take -- estimates from customs officials ranged from "4 days" to "more than 6 weeks".

All this seems pretty crazy to me. From the point that 2 days of trainings constitute work that requires an H1B visa, via the issue that everything could've been avoided if I had been allowed to set up the agreement with Blackhat immediately, to the fact that setting up the agreement once I am back in Germany and flying in again is not sufficient, all reeks of a bureacracy creating work for itself, at the expense of (US-)taxpayer money.

I will now begin the Quixotic quest to get a business visa to the US. Sigh. This sucks.

2007-07-12T09:14:00.001-07:00

The Core guys have published a paper on a very cute heap visualisation tool.

What shall I say ? I like it, and we'll play a lot more chess with memory in the future.

2007-07-07T12:36:00.000-07:00

It seems that this country is spinning out of control. We barely have the economy back on track, and now our interior minister is fighting ghosts with flamethrowers:

This link refers to an interview with him where he proclaims that:

Germany should create the status of 'enemy combatant' and allow interning 'dangerous elements'
The 'targeted killing of suspects' is not in discord with our constitution, but a 'legal problem' that hasn't been 'fully clarified'

I have to admit that while I was critical about the fact that the Bush-Administration skipped due process and a host of other essential liberties in the Guantamo/Black Interrogation Sites affair, I was not all-too-concerned -- after all, after the next election the entire thing would've been rolled back and similar madness made impossible for the next n years. I am quite shocked that our interior minister, in desparate need for some agenda, would like to outdo the Bush Administration exactly at a point in time where these policies should be thoroughly discredited.

Time to write a letter to the representative in the german congress...sigh....

2007-06-13T14:29:00.000-07:00

MS07-031

We're close to finally releasing SABRE BinDiff v2.0, and I've posted a small movie showing how it can be used to analyze MS07-031 here. Enjoy !

2007-04-27T21:35:00.000-07:00

Microsoft seems to consider banning memcpy(). This is an excellent idea - and along with memcpy, malloc() should be banned. While we are at it, the addition and multiplication operators have caused so much grief over the last years, I think it would make total sense to ban them. Oh, and if we ban the memory dereference, I am quite sure we'd be safe.

Banning API calls is not the same as auditing code. Auditing is not supergrep. Sigh.

And "we fuzzed, but it was wrapped in an exception handler" is crazy talk. The debugger gets first notification of any exception, before the exception handler - if you are fuzzing without noting down all the exceptions that occur, you're living in ... uhm ... 2001 ?

But either way: The problem is that people think Vista will be "safe", in absolute terms, which
is false. Vista is "safer", e.g. a number of bugs won't be useful any more. Because of the false perception of Vista being "safe", some people are now disappointed (because of ANI).

Enough ranting. Everybody take a deep breath, relax, and watch the game as OS X gets owned badly for the next two years.

2007-03-23T06:39:00.000-07:00

Can someone explain me why there is so few decent java decompilers out there ? Yes, JAD does a decent job in many cases, but sometimes simple control flow confuses it and the reconstruction is less than accurate. JODE is sometimes better in that regard, but fails on a good number of files, and also does not seem to assign new variable names based on the types of the variables.

With all that Java code on my cellphone, it's slightly annoying that it's so difficult to get a decent decompile. I mean, once I have that I can work in eclipse and refactor the class/variable names until I am happy.

Then again, it seems Java decompilers were all the rage in 1997-2002, and nowadays few people seem to be developing them...

2007-02-21T01:50:00.000-08:00

I will be at Blackhat Federal in Washington DC next week, and since I am not giving a talk, I will have some free time to chat :-)

If anybody in the Washington DC area would like to meet and / or have our products demo'ed, please drop me a mail at halvar.flakeXnospamX@sabre-security.com.

Cheers,
Halvar

2007-02-05T02:25:00.000-08:00

I would like to use this blog to make the MD5Sum and the SHA1sum of a certain file public:

MD5Sum:
5e5ed3b92b2abbcc1adaa18cc0ca6aaf

SHA1sum:
FFECBE21E3EC93A5AC2B94889AD2967881398A9C

Cheers,
Halvar

2007-01-18T07:11:00.000-08:00

One of the most amusing new features of BinNavi in the v1.2 release is the GDB agent. FX (of SABRE Labs fame) worked hard to create a proxy that sits in-between BinNavi GUI and something speaking GDB serial protocol either via a serial line or via TCP.

Now, what is this good for ?

First of all, it allows one to use BinNavi's debugging capabilities on platforms that we do not explicitly support (if a recent GDB version works on it). This means most *NIX variants. Let's say, for some reason, you have a FreeBSD system on which you'd like to debug some piece of software, and BinNavi does not come with a FreeBSD debugger. But GDB runs on FreeBSD - so you just run your target under gdbserver and use the BinNavi GDB agent via TCP to transparently debug the target.

Now, using BinNavi on more-or-less arbitrary *NIX systems is nice, but the real joy lies elsewhere: FX made sure that the debugging proxy does not only speak the GDB protocol as spoken by GDB itself, but also the variants spoken by Cisco IOS and ScreenOS.

This makes reverse engineering embedded systems that speak either regular GDB protocol or one of the supported variants a blast: In the past, we had to proceed as follows:

Get a ROM image from somewhere
Stare at the image to figure out methods to decompress it properly
Once this was achieved, load the image into IDA and use switch()-constructs to determine the proper loading address of the image
Load the image into IDA again, this time at the correct address

Of course, live-debugging was usually out of the question.
With the BinNavi GDB Agent, we can now do the following:

Attach the device to a serial port and set it into GDB mode
Read & dump the memory from the current instruction pointer backwards until the device freezes
Read & dump the memory forwards from the current instruction pointer until the device freezes
Load the result into IDA and export the disassembly into BinNavi
Do live-debugging on the device in question :-)

So, as an exercise, we took a Netscreen-VPN5 we had acquired via Ebay. Unfortunately, it did not come with a support contract, so we could not get software images to disassembly. So we set the device into GDB mode by typing "set gdb enable" in the console, and connected:

C:\BinNavi.v1.2\gdbagent>gdbcmd COM1,9600 NS5XT
Connected via \\.\COM1 (baud=9600 parity=N data=8 stop=1) to Netscreen 5XT Agent
/ PowerPC
[q] quit | [r] Registers | [c] Continue | [R] Reset | [b] Breakpoint
[s] step | [m] Read Memory | [D] Detach | [d] Dump Memory Range

Reading Registers ... done
GPR0 = 1
GPR1 = 350f958
GPR2 = aecce8
GPR3 = ffffffffffffffff
GPR4 = 2e
GPR5 = 0
GPR6 = 0
GPR7 = 0
GPR8 = d55e70
GPR9 = ae0000
GPR10 = d50000
GPR11 = d50000
GPR12 = 40000024
GPR13 = 0
GPR14 = 0
GPR15 = 0
GPR16 = 0
GPR17 = 40140130
GPR18 = 0
GPR19 = 186ac40
GPR20 = 0
GPR21 = 350ff78
GPR22 = 186ac4e
GPR23 = ffffffffffffffff
GPR24 = 0
GPR25 = 0
GPR26 = 0
GPR27 = 0
GPR28 = 186ac40
GPR29 = 0
GPR30 = 186a910
GPR31 = ae5684
(...)
PC = 6826c
MSR = 29230
CR = 40000028
LR = 67c10
CTR = 249b30
XER = 20000002

The program counter is set to 0x6826c, and thus we know: Some code is mapped at 0x6826c. It is a pretty safe bet that all code will be consecutive in memory, sow we will now dump the memory forwards and backwards from this address: We type "d" in the command line and enter the base address and the number of bytes (in hex) we want to dump:

Memory at: 68000
Size: 400000
Filename: 0x68000.0x400000.dmp

The agent now begins to read the memory off the device in chunks of 1024 bytes via 9600 baud serial port - so it is a good idea to go to lunch in the meantime. Once we're back from lunch, we reboot the NS5XT - it will have hung when it ran out of memory to dump. We set it back into debugging mode and dump the memory before offset 0x68000:

Memory at: 40000
Size: 28000
Filename: 0x40000.0x28000.dmp

We stitch the two files together end-to-end, load them into IDA and run a few small scripts to identify function entry points and do some minor fixing of the disassembly (principally switch statements, and some function naming), and export everything into the BinNavi database. We then open it as usual in BinNavi, open the callgraph and start browsing around.

On the left, we see a callgraph view of the device's IKE packet handlers (which we inferred from string references in the disassembly), plus the functions that are directly called by them.

Now, which of these functions would be executed when we run a round of ike-scan against the device ?

Clicking on the red button makes BinNavi talk to the BinNavi GDB agent to set one-time breakpoints on all functions in the graph on the left - due to the serial link, this is not blazingly fast, but after seconds, not minutes, we have breakpoints on all these functions. We then run ike-scan against the device, and click on "stop recording" again. The result is the list of functions from our graph that were executed - highlighted in the following pictures:

Clearly we can do the same on the function flowgraph level in, for example, the function labeled IKE_SA_Handler above. Generally, everything you can do with BinNavi on Win32 executables you can also do with BinNavi on the embedded device now: Record traces, set breakpoints, set Python callbacks on breakpoints, read memory, read registers etc. etc...

The following three screenshots show the function in question being debugged. The first screen shows the path that is executed on running an ike-scan against the device highlighted in red. The second screen shows BinNavi having suspended the execution on the basic block with the red/blue border (the blue border indicates a persistent breakpoint on the basic block, the red border indicates that execution is currently suspended on that block). The third screen just shows the registers and some memory of the device at this point in time.

So to sum things up: With the BinNavi GDB Agent, you can debug anything that speaks the GDB protocol more or less just as if it were a regular windows app (small caveat: You are speaking with most embedded devices via a serial port, oftentimes 9600 baud. You probably do not want to set 60.000 breakpoints at once - aside from the bandwidth consumption, it is common for the gdb server to handle only a limited number of breakpoints. In our tests, setting several hundreds was no problem). Extracting ROM images in a format that is easily disassembled is easy, and full on-device debugging helps a lot with all our favourite tasks:

understanding the code at hand
identifzing which functions are responsible for which features
hunting for security vulnerabilities
constructing input to reach vulnerable locations

Have a good week, I have some more reversing to do :)

Oh, and be sure to check out Ero Carrera's Blog - he will post about the SQL database format used by BinNavi at the end of next week, and show why it's useful and flexible.

2006-12-13T04:30:00.001-08:00

SP's blog has a good post today:

:-)

2006-11-23T00:51:00.000-08:00

Over at the Matasano Blog :)

Matasano 's Blog quoted my post on Office bugs, and Ivan Arce made some excellent points in the comments:

1. 'They are inherently one-shot. You send a bad file, and while the user might try to open it multiple times, there is no way the attacker can try different values for anything in order to get control.”'

IA: OK. good point but…think about scale & diversity. Even in a targeted attack sending a one-shot client-side exploit against N desktop systems will with one hardcoded address will offset the value of ALSR with some probability of success for a given N. The attacker only needs ONE exploit instance to work in order to break into ONE desktop system, after that it is game over. Client-side bugs are one shot against the same system but not necesarrilly so against several systems in parallel.

Very true, I did overlook this. It also explains the use of really low-value phone-home bots as payload: If you're going to attack in such a "wide" manner, you essentially accept detection as long as you can compromise one of the relevant clients. This means that whatever you are sending will be lost, and therefore you won't send anything more sophisticated than a simple bot.

” 2. There can not be much pre-attack reconnaissance. Fingerprinting server versions is usually not terribly difficult (if time consuming), and usually one can narrow down the exact version (and most of the times the patch level) of a target before actually shooting valuable 0day down the wire. With client side bugs, it is a lot more difficult to know the exact version of a piece of software running on the other side - one probably has to get access to at least one document created by the target to get any data at all, and even this will usually be a rough guesstimate.”

IA: Hmmm not sure about this either. I would argue the desktop systems (clients) leak A LOT more information about themselves than servers and, generally, those leaks are much less controlled and/or controllable and easier to elicit than server leaks. After all, as a general principle, client apps are _designed_ to provide information about themselves.

Not to mention that a lot of information about your desktop systems has *already* leaked and is publicly available on the net now (server logs, emails, documents, stray packets, etc.), you just need to know how and where to look for it.

I disagree on this to an extent. My system leaks information about my mail client because I participate in public forums etc, but the majority of corporate users never gain any visibility outside of the internal network. Most people just don't use mailing lists or usenet etc. So it will be comparatively easy to attack some security officer (hey, I know his exact client version), but the CEO's secretary (which might be a lot more interesting as a target, and less likely to notice her computer is compromised) will be more or less "invisible".

2006-11-21T15:56:00.000-08:00

Unbelievable but true

I am decompressing a bit after a few weeks of insane stress and thus I am actually reading blogs. And to my greatest surprise, I ended up reading this one. Now, Oracle security has never interested me ever since I tried to audit it in 2000 and it kept falling over without a fight (or without us really doing anything except sending a few letters to it), but I have to admit that Ms. Davidsons blog has a pretty high entertainment value (at least for me, a morallically degenerate piece of eurotrash full of the afterglow of a once good education system), AND it is refreshing to see someone with a bit of a classical education in IT security (I get picked upon regularly for the fact that I got my Latinum "on the cheap" and know jack shit about old greek - then again, my circle of friends includes a mathematician that claims that he can, by means of listening to a record, tell you in which church in france a certain piece of organ music was played, and hence I am always the loud and stupid one).

Anyhow, given Oracle's horrible code quality, I am very much positively surprised at the quality of Ms. Davidsons blog. And given what most people that have worked with static analysis tools before would describe as a horrible mistake in evaluating tool quality, I would like to mention that mathematics and geometry are part of a classical education. Whoever decided on the right source code analysis tool to use for detecting flaws in Oracle apparently failed that part.

2006-11-21T15:30:00.000-08:00

Client Side Exploits, a lot of Office bugs and Vista

I have ranted before about careless use of 0day by seemingly chinese attackers, and I think I have finally understood why someone would use good and nice bugs in such a careless manner:

The bugs are going to expire soon. Or to continue using Dave Aitel's and my terminology: The fish are starting to smell.

ASLR is entering the mainstream with Vista, and while it won't stop any moderately-skilled-but-determined attacker from compromising a server, it will make client side exploits of MSOffice file format parsing bugs a lot harder.

Client-side bugs suffer from a range of difficulties:

They are inherently one-shot. You send a bad file, and while the user might try to open it multiple times, there is no way the attacker can try different values for anything in order to get control.
There can not be much pre-attack reconnaissance. Fingerprinting server versions is usually not terribly difficult (if time consuming), and usually one can narrow down the exact version (and most of the times the patch level) of a target before actually shooting valuable 0day down the wire. With client side bugs, it is a lot more difficult to know the exact version of a piece of software running on the other side - one probably has to get access to at least one document created by the target to get any data at all, and even this will usually be a rough guesstimate.

As a result of this, client-side bugs in MSOffice are approaching their expiration date. Not quickly, as most customers will not switch to Vista immediately, but they are showing the first brown spots, and will at some point start to smell.

So you're in a situation where you're sitting on heaps of 0day in MSOffice, which, contrary to Vista, was not the biggest (private sector) pentest ever (This sentence contains two inside jokes, and I hope that those who understand them aren't mad at me :-). What do you do with those that are going to be useless under ASLR ? Well, damn, just fire them somewhere, with some really silly phone-home-bots inside. If they bring back information, fine, if not, you have not actually lost much. The phone-home bots are cheap to develop (in contrast to a decent rootkit) and look amateurish enough as to not provoke your ambassador being yelled at.

If you are really lucky, you might actually get your opponent to devote time and resources to countermeasures against MS Office bugs, in the hope they don't realize that work will be taken care of elsewhere. In the meantime, you hone your skills in defeating ASLR through out-of-defined-memory-read-bugs (see some blog post in the next few days).

On a side note, I am terribly happy today. I've had more luck this week than I deserve.

2006-11-20T04:08:00.000-08:00

While we're all talking about the next overflow and think that they have significance in the wider scheme of things, I'll climb on the soapbox for 5 minutes:

We should send peacekeeping troops to Darfour/Sudan. I was strongly opposed to the Iraq war (on the ground that invasion would bring civil war), but I plead my government: Take my taxes and send peacekeeping forces to Sudan. _If_ we have decided that the 'europeans-are-from-venus'-stance is obsolete, we have here a primary example of a conflict where external invasion appears necessary according to almost everybody (except the government in Kartoum).

2006-10-05T16:20:00.000-07:00

While I am blogging about strange hobbies: I used to draw a lot, and still appreciate a few comics. Most importantly, local cult hero Jamiri.

Some examples:
http://www.spiegel.de/netzwelt/netzkultur/0,1518,grossbild-650193-422928,00.html

http://www.spiegel.de/netzwelt/netzkultur/0,1518,grossbild-669475-427889,00.html

2006-10-05T16:08:00.000-07:00

I am known for odd hobbies and interests, and for a long while, I have been very fascinated with all forms of syncretism, specifically carribbean syncretism.

For various private reasons I am exposed to quite a bit of information about social anthropology, and I usually find the descriptions of odd rites in various societies very amusing and enlightening.

For example, any diagram of multi-family cross-cousin-marriage in some african societies just brings out the graph theory nerd in me, and serious scientific texts debating the difference between endo- and exocannibalism (eat your own tribe vs. eat the other tribe) are a fun diversion from reading dry stuff all day.

Yet I was unprepared for reading about the "Cargo Cult" today. And thinking about it, the sheer fact that a cargo cult developed in Melanesia makes me want to laugh and cry at the same time.

Read it. It's worth it.

2006-09-08T02:51:00.000-07:00

Matasano refers to Bleichenbachers' recently published attack. Tremendously short comment:

Anything that does RSA with low exponent is likely attackable. And padding should always be OAEP. ;)

2006-09-08T02:18:00.000-07:00

After all the Brouhaha surrounding the work on Apple wireless drivers, I'd like to pitch my two cents:

Who cares wether this is real or not ? The possibility of breaking NIC drivers (especially in multithreaded kernels) is real, and nobody should be surprised if this happens. Has anyone ever disassembled the pos drivers that come with every cheap electronic USB gadget ? I have my doubts that the QA for NIC drivers is a lot better
It seems we are not the only ones with a similar problem: http://eprint.iacr.org/2006/303.ps

In the above paper, Eric Filiol says he has broken E0, but does not give any description of the analysis - just a (significant) number of keys that lead to very long strings of zero's or to keystreams with a predefined hamming weight.

I am not decided on the paper yet - read it yesterday evening, jetlagged, over half a bottle of wine. This sort of publishing would be very easy for hash functions -- I would believe anyone that he can build secondary pre-images (or even pre-images) from MD5 if he can give me a string of input that hashes to "thequickbrownfox....".

Now, we just need stuff like that for bugs ;-)

2006-08-21T22:32:00.000-07:00

Now with all this noise surrounding the ConsumerReports article where they created 5500 new virus variants, I would really like to get my hands on their sample list to see how VxClass, our malware classification engine, deals with them.

2006-08-11T03:50:00.000-07:00

Just to clarify: PaiMei is really good, the previous post was not supposed to be negative or detrimental -- it's definitely cool stuff.

2006-08-11T03:36:00.000-07:00

From Matasano:

"The results of one trace can be used to filter subsequent traces. This is huge (in fairness: it’s something that other people, notably Halvar [I believe], have been working on)."

I have to admit that our flash movies that we posted last year in September are mind-numbingly boring, but they do show this sort of stuff ;) -- BinNavi was able to record commentable debug traces since day 1.

http://www.sabre-security.com/products/BinNavi/flash_binnavi_debugger.html
http://www.sabre-security.com/products/BinNavi/flash.html

The entire idea of breakpointing on everything and doing differential debugging dates back to at least a Blackhat presentation in Vegas 2002. Fun stuff, and good to see that with PaiMei there is finally a free framework to do this.

I really need to re-do the BinNavi movies in the next weeks, they really do not do our product any justice any more.

To continue shamelessly plugging my product :-):

"Can I have stack traces for each hit? I know they’re somewhat redundant, but I can graph them to visualize control flow (in particular, to identify event and “parse” loops)."

You can in the next release (scheduled for October) where you can attach arbitrary python scripts to breakpoints and thus do anything to memory you want.

"Symbols. Pedram acknowledges this in his presentation. It didn’t slow me down much not to have them, but it feels weird."

If IDA has them, BinNavi has them.

"I need to be able to click on a hit and see the assembly for it (if there’s a way to click on something and have it pop up in IDA, so much the better)."

Right-click->open subfunction in BinNavi ;)

"Yeah, I need this for non-Windows targets. Remote debugging is apparently coming, which will help. I don’t imagine Pedram’s working on SPARC support (X86 and Win32 has eaten its way pretty thoroughly through the code). Also,"

We have Linux/ptrace support and a (very experimental) WinCE/ARM support.

I promise to redo the movies in the next weeks.

Enough of the advertisement crap.

Cheers,
Halvar

2006-07-26T20:53:00.000-07:00

The security world never ceases to amaze me. A few years ago, a few friends of mine would run around security conferences and drunkenly yell "fuzz tester ! fuzz tester !" at people that, well, fuzzed. I found this really hilarious.

What I find amazing though is that fuzzers are now being seriously discussed in whitepapers and even called "artificial intelligence". Folks, can we please NOT do the time warp again ? And can we please start writing about something new ?

On a side note: Since I am a bit of a language nerd, I can't fail to notice that "artificial intelligence" takes a semantically cool twist when mentioned in the same sentence as "yellowcake from africa".

PS: This post is a rant about people that write about fuzzing as a new threat, not about people that write and use fuzzers. Just to clarify :)

2006-07-26T10:44:00.000-07:00

I will have an 8-hour layover in Toronto tomorrow -- anyone up for a coffee ?

2006-07-11T03:32:00.000-07:00

The article at this link is a bit funny, but if it is true that Materazzi made racial slurs against Zidane, then his headbutt was the ONLY proper answer to that.

Racism on the pitch should not be tolerated under any circumstances, and a healthy team would not tolerate racist remarks from any team member.

If Zidane's reaction was a response to racist remarks, then his headbutt is a symbol for a world cup that did not tolerate racism, and that united people from all over the world instead of dividing them.

On a side note, I am very happy for all the Italians :-) and I'd like to thank my Italian neighbours for having invited us to their place to watch the final.

Enough football, now back to work.

2006-07-10T14:17:00.000-07:00

I know that I am going to draw the hate of many people for this post, but I refuse to think less of Zidane for the headbutt against Materazzi. As strange as it sounds, for some reason I am quite convinced that he must have had a good reason for this.

Nobody is mad enough to just headbutt an opponent in the worldcup finals in the last game of a legendary career unless he has a very good reason.

But well.

2006-07-04T08:37:00.000-07:00

Question for the Blogosphere: Does anyone know of a real-life crypto protocol in which Diffie-Hellmann over a finite field is used, and that finite field is NOT a prime field ? To be exact, I am looking for examples of real-life crypto using Diffie-Hellmann over GF(p^m) where m > 1.

2006-07-02T14:30:00.000-07:00

This Ebay posting for a Yacht that was previously owned by China's Minister of Defense might in fact be a bargain -- I would assume one automatically buys not only the yacht but also some state-of-the-art (of the mid-90's) electronics. I am not sure if that is still worth 2m USD, but still.

2006-07-01T06:36:00.000-07:00

I used to read security blogs via http://www.dayioglu.net/planet/ , which now seems down.
It's amusing how quickly I have quit reading blogs since. Funny world.

2006-06-24T00:37:00.000-07:00

On bug disclosure and contact with vendors

After reading HDM's blog entry on interaction with MS on one of the recent bugs, I guess I should drop my 2c's worth of opinion into the bowl regarding bug disclosure:

So sometimes I get the urge to find bugs. Then I go out and sometimes I find bugs. Then I usually feel quite happy and sometimes I even write an exploit. I do all this out of personal enjoyment -- I like bugs. I like having to play carambolage billard to get an exploit to work (meaning having to bounce things off of each other in weird angles to get stuff to work). Now, of course, once I am done I have several options on what to do with a bug.

Report it to the vendor. This would imply the following steps, all of which take up time and effort better spent on doing something interesting:

Send mail to their secure@ address, requesting an encryption key. I think it is amusing that some vendors like to call security researchers irresponsible when the default channel for reporting vulnerabilities is unencrypted. That is about as irresponsible as the researchers talking about vulnerabilities on EFNET.
Get the encryption key. Spend time writing a description. Send the description, possibly with a PoC.
MSRC is a quite skilled bunch, but with almost any other software vendor, a huge back and forth begins now where one has to spend time explaining things to the other side. This involves writing boring things explaining boring concepts etc.

Sell it to somebody who pays for vulnerabilities. While this will imply the same lengthy process as mentioned above, at least one can in theory get paid for it. Personally, I wouldn't sell bugs, but that could have several reasons:

I am old and lame and can't find bugs that are good enough any more
The few bugs that I find are too close to my heart to sell -- each good bug and each good exploit has a story, and I am not so broke that I'd need to sell something that I consider inherently beautiful
I don't know the people buying these things. I don't know what they'd do with it. I wouldn't give my dog to a total stranger either.

Keep it. Perhabs on a shelf, or in a frame. This implies zero effort on my side. It also gives me the joy of being able to look at it on my wall and think fondly of the story that it belonged to.

So in case of 1), after having spent weeks on a bug, I have to spend more time doing something unenjoyable, and get a warm handshake with the words 'thanks for helping secure (the internet/the world/our revenue stream'.
In case 2), I get a warm handshake, some money, and a feeling of guilt for having given my dog to a total stranger.
In case 3), I have something to look at with fond memories and have to invest no time at all into things that I don't find interesting.

What would be your choice ?

2006-06-23T00:00:00.000-07:00

I really enjoyed reading Ilfak's blog post today :-) -- it always makes me happy to see clever abstractions and the results they produce. And I really enjoy original ideas (of which there seems to be a very finite amount in IT :)

2006-06-12T15:41:00.000-07:00

Compression, Statistics and such

In the process of doing the usual stuff that I do when I do not struggle with my studies, I ran into the problem of having a number of streams with a very even distribution of byte values. I know that these bytes are executable code somehow encoded. I have a lot of reason to suspect that they are compressed, not encrypted, but I have not been able to make sense of it yet.

This brought me to the natural question: Do common encryption algorithms have statistical fingerprints that would allow them to be distinguished from one another, more-or-less irrespective of the underlying data ? It is clear that this gets harder as the amount of redundancy decreases.

It was surprising (at least for me) that nobody else has worked on this yet (publically).

Also, it made me regret that due to some time constraints involving some more algebraic courses I was unable to attend the Statistics I and II lectures given at my University by Prof. Dette. Had I attended, I would know better how to make sense of the capabilities that software like R could give me.

Another example of the fundamental law of mathematics: For every n denoting the number of days you have studied mathematics there exists a practical problem that make you wish you had studied 2n days already.

2006-06-05T02:03:00.000-07:00

Some shameless self-promotion: Rolf and me are going to teach a special one-day class on BinDiff 2 at BlackHat Las Vegas this year:

http://www.blackhat.com/html/bh-usa-06/train-bh-us-06-hf-sabre.html

We'll cover applications of BinDiff to malware analysis, detecting Code Theft and GPL violations, and of course the usual patch analysis.

2006-06-03T10:11:00.000-07:00

Extra extra: Google/Sun discover the existence of integer overflows :)

2006-05-28T14:36:00.000-07:00

My prediction for the next two years: Apple, Symantec, McAffee, Oracle etc. will get pounded into the ground by lots of bugs being found and disclosed through security researchers that are looking for easier targets than the current MS codebase. And the abovementioned companies won't have monopoly revenue to throw around and fix the issues.

This is a big opportunity for MS to move into all their markets :-) and sell their products as superior on the security side.

While I am in "evil" mood: The german train system is about to be IPO'ed, and there's a lot of debate going on here about details of the contract. What is most interesting but not being debated:
All real estate owned by the Deutsche Bahn AG (the privatized version of the german train system that is going to be floated) is in the books with it's value upon acquisition -- meaning it's value in 1935. The real estate in possession of the DB is, by today's value, worth several times more than the total money they expect to get out of the IPO.

If I was an investment banker, I'd gang up with a bunch of private equity folks, buy the majority in the DB AG once it is IPO'd, and then sell of the real estate. Other countries (USA, Britain) survive without a decent train system, too, and I wouldn't care as I'd have a Rolls and a driver.

Allright, enough of the devil's advocate mode. It was fun seeing my brother the last weekend,
and we always come up with good ideas ;)

2006-05-23T02:20:00.000-07:00

MSASN1 is hard to read these days -- the code makes heavy use of carry-flag-dependent arithmetic (adc, rlc etc) to check for integer overflows.

2006-05-20T10:13:00.000-07:00

The Vodafone virus dropped by today and brought us some mobile viruses to play with - thanks ! :-)

So cross-platform diffing can be fun -- Rolf ran a diff of Commwarrior.B against Commwarrior.C today, and while B is compiled for standard ARM, C is compiled in 'thumb mode', which is pretty much the same as being compiled for a different CPU (thumb means that all instructions are different).

The amusing result is that even though the compilation is for a different platform, we still get roughly 61% of the functions matched. And the functions, which are clearly the same on the 'structural' (e.g. flowgraph) - level, have completely different instructions, and manual inspection will confirm that these differing instructions end up doing the same.

For those of you that want to verify things manually, click here.