Our trainings class in Frankfurt is over, and I think I can safely say that it was a resounding success. I guess the coolest thing about SABRE is our customers. I hope to see you all again someplace again.

PS: I forgot to distribute the python code from the last day, it will be mailed to all participants on monday.

Blackhat Japan

After the immigration SNAFU in summer, I am scheduled to give my trainings class at Blackhat Japan this November - so if anyone wants to come, sign up now :-)

Cheers,
Halvar

BinDiff v2.0 finally released !

This is "blog-spam":

After a long wait, SABRE Security GmbH is proud to announce
the official release of BinDiff v2.0. This biggest improvements are:

• Higher comparison speeds
• Greater accuracy for functions which change only in the structure of the graph, not in the number of nodes/edges
• Much greater accuracy on the instruction level comparison
• The arguably prettiest UI of all binary comparison tools around
  

The many detail improvements are too numerous to mention here.
Check the screenshots:





Contact info@sabre-security.com for an evaluation version !

-- SABRE Security Team

I am quite famous for botching every marketing effort that we try to undertake at SABRE -- a prime example of my ineptitude is the fact that we released BinNavi v1.2 in ... uh ... January, with a ton of new stuff, and I still hadn't updated the website to show some nice pictures.

Similarly for BinDiff -- v2.0 beta has been used by many customers without a hitch, and is a big improvement on the UI front. So I finally got around to adding some nice pictures today.

Also, for those that are into the entire idea of malware classification, you can see some screenshots of VxClass, our unpacker-and-classifier (Disclosure: Before Spender writes a comment ;) about our unpacker's inability to handle TheMida and similar emulating packers, I will do so myself: We do not handle emulating packers at the moment! We do not reconstruct PEs ! But if you have a cool unpacker you can just upload the unpacked file to our classifier :)

So with this blog post it's confirmed: I am not only a failure at marketing, I am also a failure at attempting to pass off marketing as a regular blog post. Have a good weekend everyone !

I have reached the intellectual level of the sports spectator in an armchair: Comment first, read and understand later. After the last Blog comment, I actually went to read the slides of Joanna's presentation. To summarize: I find the slides informative and well-thought-out. I found that the empirical bits appear plausible and well-researched. The stuff following slide 90 was very informative. It is one of the most substantial slide decks I have read in recent times.

Some points to take home though: Whoever writes a rootkit puts himself in a defending positions. Defending positions against all known attacks is possible given perfection on the side of the defender. That is bloody hard to achieve. There is no doubt that for any given attack one can think of a counter attack, but it's a difficult game to play that doesn't allow for errors.

I think the core point that we should clarify is that rootkits should not fall into an adversary's hand to be analyzed. Once they are known, they fall into a defending position. Defending positions are not long-term substainable, as software has a hard time automatically adapting to new threats.

Once you accept that the key to a good rootkit is to use methods unknown to the victim, one might also be tempted to draw the conclusion that perhabs the virtualisation stuff is too obvious a place to attempt to hide in. But that is certainly open to discussion.

Enough high-level blah blah. I am so looking forwards to my vacation, it's not funny.
Post veröffentlichen

So it appears the entire Rutkowska-Matasano thing is not over yet. I probably should not harp on about this in my current mood, but since I am missing out on the fun in Vegas, I'll be an armchair athlete and toss some unqualified comments from the sidelines. Just think of me as the grumpy old man with a big gut and a can of beer yelling at some football players on television that they should quit being lazy and run faster.

First point: The blue chicken defense outlined in the linked article is not a valid defense for a rootkit. The purpose of a rootkit is to hide data on the machine from someone looking for it. If a rootkit de-installs itself to hide from timing attacks, the data it used to hide either has to be removed or is no longer hidden. This defeats the purpose of the rootkit: To hide data and provide access to the compromised machine.

Second point: What would happen if a boxer who claims the ability to defeat anyone in the world would reject any challengers unless they pay 250 million for him to fight ? Could he claim victory by telling the press that he "tried out all his opponents punches, and they don't work, because you can duck them like this and parry them like that" ?
I think not.

I am not saying it's impossible to build a rootkit that goes undetected by Matasano's methods. But given access to the code of a rootkit and sufficient time, it will be possible to build a detection for it. Of course you can then change the rootkit again. And then the other side changes the detection. And this goes on for a few decades.

Could we please move on to more fruitful fields of discussion already ?

Some people in the comments of my blog have hinted that I should have just "followed the rules" and nothing would have happened. This is incorrect -- I did follow the rules. It is perfectly legal for an independent contractor to be contracted to perform a task in the US, come in, do it, and leave. That is (amongst other things) what the "business" checkbox on the I94W is for.

What landed me in this trouble is that the immigration agent decided that even though I am CEO of a company in Germany and have no employment contract with Blackhat (just a contract as an independent contractor), that the status of "independent contractor" does not apply to me - his interpretation was that I was an "employee" of Blackhat without an H1B visa.

This is not a case of me screwing up my paperwork. This is a case of an immigration agent that did not understand my attempts at explaining that I am not a Blackhat employee, and me not knowing the subtleties of being interviewed by DHS/INS agents.

I hope I will be able to clarify the misunderstanding on Thursday morning at the consulate.
=============================
Small addition to clarify: It is perfectly legitimate to come to the US to hold lectures and trainings of the kind that I am holding at Blackhat. To reiterate: The problem originated solely from a misunderstanding where it was presumed I was an "employee" of a US company, which is not correct.

Short update: I have managed to schedule a hearing for a regular visa. The first available date was the 24th of August *cough*.

While this is clearly too late for Blackhat, but once you have a "regular" meeting scheduled you can ask to have an "urgent" meeting scheduled, too. Wether I am eligible will become clear when the embassy opens at 7am on monday morning.

The current plan is to call them and explain them why the entire thing might've gone haywire in the first place:

There's a special provision in the german tax code that allows for people with certain qualifications to act as special 'freelancers', essentially giving them a status very similar to one-person-companies ("Freiberufler"). It is not totally trivial to obtain this status - for example, you cannot simply be a 'Freiberuf'-programmer if you write "regular" software.

My agreement with Blackhat and all transactions were taxed in Germany under this status.

Personally, I think the fundamental issue in this tragic comedy is that the US doesn't really have such a special status for freelancers, and that therefore the US customs inspector did not understand that there is a distinction between a "regular Joe" and a "single-person company/Freiberufler". Hence the customs officer assumed that this entire thing must be some devious way to bypass getting an H1B visa for someone that would not normally qualified to get one. The frequent repetition of the question "why is your course not given by an American Citizen ?" points to something like that.

I hope that I can clear up this misunderstanding tomorrow morning, but right now, I am not terribly optimistic.

I've been denied entry to the US essentially for carrying my trainings material. Wow.

It appears I can't attend Blackhat this year. I was denied entry to the US for carrying trainings materials for the Blackhat trainings, and intending to hold these trainings as a private citizen instead of as a company.

After a 9-hour flight and a 4 1/2 hour interview I was put onto the next 9-hour flight back to Germany. Future trips to the US will be significantly more complicated as I can no longer go to the US on the visa waiver program.

A little background: For the last 7 years, I have attended / presented at the 'Blackhat Briefings', a security conference in the US. Prior to the conference itself, Blackhat conducts a trainings session, and for the past 6 years, I have given two days of trainings at these events. The largest part of the attendees of the trainings are US-Government related folks, mostly working on US National Security in some form. I have trained people from the DoD, DoE, DHS and most other agencies that come to mind.

Each time I came to the US, I told immigration that I was coming to the US to present at a conference and hold a trainings class. I was never stopped before.

This time, I had printed the materials for the trainings class in Germany and put them into my suitcase. Upon arrival in the US, I passed immigration, but was stopped in customs. My suitcase was searched, and I was asked about the trainings materials.
After answering that these are for the trainings I am conducting, an immigration officer was called, and I was put in an interview room.
For the next 4 1/2 hours I was interviewed about who exactly I am, why I am coming to the US, what the nature of my contract with Blackhat is, and why my trainings class is not performed by an American citizien. After 4 hours, it became clear that a decision had been reached that I was to be denied entry to the US, on the ground that since I am a private person conducting the trainings for Blackhat, I was essentially a Blackhat employee and would require an H1B visa to perform two days of trainings in the US.

Now, I am a full-time employee (and CEO) of a German company (startup with 5 people, self-financed), and the only reason why the agreement is between Blackhat and me instead of Blackhat and my company is that I founded the company long after I had started training for Blackhat and we never got around to changing it.

Had there been an agreement between my company and Blackhat, then my entry to the US would've been "German-company-sends-guy-to-US-to-perform-services", and everything would've been fine. The real problem is that the agreement was still between me as a person
and Blackhat.

After the situation became clear (around the 4th hour of being interviewed), I offered that the agreement between Blackhat and my company could be set up more or less instantaneously - as a CEO, I can sign an agreement on behalf of my company, and Blackhat would've signed immediately, too.
This would've spared each party of us a lot of hassle and paperwork. But apparently, since I had just tried to enter as a 'normal citizen' instead as an 'employee of a company', I could now not change my application. They would have to put me on the next flight back to Germany.

Ok, I thought, perhabs I will have to fly back to Germany, set up the agreement, and immediately fly back to the states - that would've still allowed me to hold the trainings and attend the conference, at the cost of crossing the Atlantic three times instead of once. But no such luck: Since I have been denied entry under the visa waiver programme, I can now never use this programme again. Instead I need to wait until the American consulate opens, and then apply for a business visa. I have not been able to determine how long this might take -- estimates from customs officials ranged from "4 days" to "more than 6 weeks".

All this seems pretty crazy to me. From the point that 2 days of trainings constitute work that requires an H1B visa, via the issue that everything could've been avoided if I had been allowed to set up the agreement with Blackhat immediately, to the fact that setting up the agreement once I am back in Germany and flying in again is not sufficient, all reeks of a bureacracy creating work for itself, at the expense of (US-)taxpayer money.

I will now begin the Quixotic quest to get a business visa to the US. Sigh. This sucks.

The Core guys have published a paper on a very cute heap visualisation tool.

What shall I say ? I like it, and we'll play a lot more chess with memory in the future.

It seems that this country is spinning out of control. We barely have the economy back on track, and now our interior minister is fighting ghosts with flamethrowers:

This link refers to an interview with him where he proclaims that:

• Germany should create the status of 'enemy combatant' and allow interning 'dangerous elements'
• The 'targeted killing of suspects' is not in discord with our constitution, but a 'legal problem' that hasn't been 'fully clarified'

I have to admit that while I was critical about the fact that the Bush-Administration skipped due process and a host of other essential liberties in the Guantamo/Black Interrogation Sites affair, I was not all-too-concerned -- after all, after the next election the entire thing would've been rolled back and similar madness made impossible for the next n years. I am quite shocked that our interior minister, in desparate need for some agenda, would like to outdo the Bush Administration exactly at a point in time where these policies should be thoroughly discredited.

Time to write a letter to the representative in the german congress...sigh....

MS07-031

We're close to finally releasing SABRE BinDiff v2.0, and I've posted a small movie showing how it can be used to analyze MS07-031 here. Enjoy !

Microsoft seems to consider banning memcpy(). This is an excellent idea - and along with memcpy, malloc() should be banned. While we are at it, the addition and multiplication operators have caused so much grief over the last years, I think it would make total sense to ban them. Oh, and if we ban the memory dereference, I am quite sure we'd be safe.

Banning API calls is not the same as auditing code. Auditing is not supergrep. Sigh.

And "we fuzzed, but it was wrapped in an exception handler" is crazy talk. The debugger gets first notification of any exception, before the exception handler - if you are fuzzing without noting down all the exceptions that occur, you're living in ... uhm ... 2001 ?

But either way: The problem is that people think Vista will be "safe", in absolute terms, which
is false. Vista is "safer", e.g. a number of bugs won't be useful any more. Because of the false perception of Vista being "safe", some people are now disappointed (because of ANI).

Enough ranting. Everybody take a deep breath, relax, and watch the game as OS X gets owned badly for the next two years.

Can someone explain me why there is so few decent java decompilers out there ? Yes, JAD does a decent job in many cases, but sometimes simple control flow confuses it and the reconstruction is less than accurate. JODE is sometimes better in that regard, but fails on a good number of files, and also does not seem to assign new variable names based on the types of the variables.

With all that Java code on my cellphone, it's slightly annoying that it's so difficult to get a decent decompile. I mean, once I have that I can work in eclipse and refactor the class/variable names until I am happy.

Then again, it seems Java decompilers were all the rage in 1997-2002, and nowadays few people seem to be developing them...

I will be at Blackhat Federal in Washington DC next week, and since I am not giving a talk, I will have some free time to chat :-)

If anybody in the Washington DC area would like to meet and / or have our products demo'ed, please drop me a mail at halvar.flakeXnospamX@sabre-security.com.

Cheers,
Halvar

I would like to use this blog to make the MD5Sum and the SHA1sum of a certain file public:

MD5Sum:
5e5ed3b92b2abbcc1adaa18cc0ca6aaf

SHA1sum:
FFECBE21E3EC93A5AC2B94889AD2967881398A9C

Cheers,
Halvar

One of the most amusing new features of BinNavi in the v1.2 release is the GDB agent. FX (of SABRE Labs fame) worked hard to create a proxy that sits in-between BinNavi GUI and something speaking GDB serial protocol either via a serial line or via TCP.

Now, what is this good for ?

First of all, it allows one to use BinNavi's debugging capabilities on platforms that we do not explicitly support (if a recent GDB version works on it). This means most *NIX variants. Let's say, for some reason, you have a FreeBSD system on which you'd like to debug some piece of software, and BinNavi does not come with a FreeBSD debugger. But GDB runs on FreeBSD - so you just run your target under gdbserver and use the BinNavi GDB agent via TCP to transparently debug the target.

Now, using BinNavi on more-or-less arbitrary *NIX systems is nice, but the real joy lies elsewhere: FX made sure that the debugging proxy does not only speak the GDB protocol as spoken by GDB itself, but also the variants spoken by Cisco IOS and ScreenOS.

This makes reverse engineering embedded systems that speak either regular GDB protocol or one of the supported variants a blast: In the past, we had to proceed as follows:

1. Get a ROM image from somewhere
2. Stare at the image to figure out methods to decompress it properly
   
3. Once this was achieved, load the image into IDA and use switch()-constructs to determine the proper loading address of the image
4. Load the image into IDA again, this time at the correct address

Of course, live-debugging was usually out of the question.
With the BinNavi GDB Agent, we can now do the following:

1. Attach the device to a serial port and set it into GDB mode
2. Read & dump the memory from the current instruction pointer backwards until the device freezes
3. Read & dump the memory forwards from the current instruction pointer until the device freezes
4. Load the result into IDA and export the disassembly into BinNavi
5. Do live-debugging on the device in question :-)

So, as an exercise, we took a Netscreen-VPN5 we had acquired via Ebay. Unfortunately, it did not come with a support contract, so we could not get software images to disassembly. So we set the device into GDB mode by typing "set gdb enable" in the console, and connected:

C:\BinNavi.v1.2\gdbagent>gdbcmd COM1,9600 NS5XT
Connected via \\.\COM1 (baud=9600 parity=N data=8 stop=1) to Netscreen 5XT Agent
/ PowerPC
[q] quit | [r] Registers | [c] Continue | [R] Reset | [b] Breakpoint
[s] step | [m] Read Memory | [D] Detach | [d] Dump Memory Range

Reading Registers ... done
GPR0 = 1
GPR1 = 350f958
GPR2 = aecce8
GPR3 = ffffffffffffffff
GPR4 = 2e
GPR5 = 0
GPR6 = 0
GPR7 = 0
GPR8 = d55e70
GPR9 = ae0000
GPR10 = d50000
GPR11 = d50000
GPR12 = 40000024
GPR13 = 0
GPR14 = 0
GPR15 = 0
GPR16 = 0
GPR17 = 40140130
GPR18 = 0
GPR19 = 186ac40
GPR20 = 0
GPR21 = 350ff78
GPR22 = 186ac4e
GPR23 = ffffffffffffffff
GPR24 = 0
GPR25 = 0
GPR26 = 0
GPR27 = 0
GPR28 = 186ac40
GPR29 = 0
GPR30 = 186a910
GPR31 = ae5684
(...)
PC = 6826c
MSR = 29230
CR = 40000028
LR = 67c10
CTR = 249b30
XER = 20000002

The program counter is set to 0x6826c, and thus we know: Some code is mapped at 0x6826c. It is a pretty safe bet that all code will be consecutive in memory, sow we will now dump the memory forwards and backwards from this address: We type "d" in the command line and enter the base address and the number of bytes (in hex) we want to dump:

Memory at: 68000
Size: 400000
Filename: 0x68000.0x400000.dmp

The agent now begins to read the memory off the device in chunks of 1024 bytes via 9600 baud serial port - so it is a good idea to go to lunch in the meantime. Once we're back from lunch, we reboot the NS5XT - it will have hung when it ran out of memory to dump. We set it back into debugging mode and dump the memory before offset 0x68000:

Memory at: 40000
Size: 28000
Filename: 0x40000.0x28000.dmp

We stitch the two files together end-to-end, load them into IDA and run a few small scripts to identify function entry points and do some minor fixing of the disassembly (principally switch statements, and some function naming), and export everything into the BinNavi database. We then open it as usual in BinNavi, open the callgraph and start browsing around.

On the left, we see a callgraph view of the device's IKE packet handlers (which we inferred from string references in the disassembly), plus the functions that are directly called by them.

Now, which of these functions would be executed when we run a round of ike-scan against the device ?

Clicking on the red button makes BinNavi talk to the BinNavi GDB agent to set one-time breakpoints on all functions in the graph on the left - due to the serial link, this is not blazingly fast, but after seconds, not minutes, we have breakpoints on all these functions. We then run ike-scan against the device, and click on "stop recording" again. The result is the list of functions from our graph that were executed - highlighted in the following pictures:













Clearly we can do the same on the function flowgraph level in, for example, the function labeled IKE_SA_Handler above. Generally, everything you can do with BinNavi on Win32 executables you can also do with BinNavi on the embedded device now: Record traces, set breakpoints, set Python callbacks on breakpoints, read memory, read registers etc. etc...

The following three screenshots show the function in question being debugged. The first screen shows the path that is executed on running an ike-scan against the device highlighted in red. The second screen shows BinNavi having suspended the execution on the basic block with the red/blue border (the blue border indicates a persistent breakpoint on the basic block, the red border indicates that execution is currently suspended on that block). The third screen just shows the registers and some memory of the device at this point in time.

So to sum things up: With the BinNavi GDB Agent, you can debug anything that speaks the GDB protocol more or less just as if it were a regular windows app (small caveat: You are speaking with most embedded devices via a serial port, oftentimes 9600 baud. You probably do not want to set 60.000 breakpoints at once - aside from the bandwidth consumption, it is common for the gdb server to handle only a limited number of breakpoints. In our tests, setting several hundreds was no problem). Extracting ROM images in a format that is easily disassembled is easy, and full on-device debugging helps a lot with all our favourite tasks:

• understanding the code at hand
• identifzing which functions are responsible for which features
• hunting for security vulnerabilities
• constructing input to reach vulnerable locations
  

Have a good week, I have some more reversing to do :)

Oh, and be sure to check out Ero Carrera's Blog - he will post about the SQL database format used by BinNavi at the end of next week, and show why it's useful and flexible.