Today I discovered my best C++ RE automation technique yet, which is alarmingly simple. Every name and type in this screenshot was applied automatically, no manual work. Details eventually when I finish my research binge.
Today I learned that disabling Hex-Rays "fast structural analysis" can clean up some common patterns where it generates suboptimal control flow structure.
Ghidra's extensibility is jaw-dropping. Today I needed the pcode to model the x86 parity flag, which it doesn't do by default. 30 minutes and a 35-line patch to ia.sinc later, I can proceed. No other tool even comes close to how easy that was. (diff: …)
Research went better than expected. No manual work was involved in creating this screenshot. All type information was automatically generated and automatically applied to a freshly-created database.
Recent results: generic detection of STL code in stripped binaries, with automatic commenting. Still a work in progress -- the internals keep evolving -- but so far, so good.
My full-binary, static type reconstruction toolkit is becoming increasingly robust. Pictured are fully-automated results from RPCRT4/x64, an experiment to see if public PDBs make the problem any easier than the stripped case (answer: not by much). Bootleg private PDBs!
Spent the evening reading Ghidra's control flow structuring subsystem. That is a very unique, well thought-out, and well-designed piece of code. Recommended reading for binary analysis aficionados.
I pushed a prototype of some white-box cryptography stuff for Ghidra (differential computation analysis/correlation power analysis for DES): (example script )
While researching C++/STL reverse engineering, I collected a list every MSVC CRT version I could find on github: It helps you narrow down the compiler version from the disassembly, which lets you see code for a version of the STL close to your binary's.
I had been skeptical of Hex-Rays 7.6's new automatic variable renaming, but no more. I did not manually name any of the variables in these screenshots. Great time-saving feature.
Ghidra's, and IDA 7.5's new, "folders" feature is incredibly useful for large-scale reverse engineering. I've found it especially provides clarity when organizing reconstructed data structures.
This is a great book. I've been recommending it to the students in my SMT class, as it's (by far) the largest compendium of constraint satisfaction problems/solutions that I'm aware of, including tons of unique and obscure ones. Good work, Dennis!
I recently discovered
@ModernVintageG
's channel on YouTube. A lot of stuff about old video game copy protections (arcade, console, PC), emulation, game development, etc. Good production values, too. Great stuff for reverse engineering enthusiasts.
New research is going well. Hex-Rays microcode API plugin vs. obfuscating compiler, from an in-the-wild malware sample. Pictorially, obfuscated and deobfuscated versions:
Today is the 25th anniversary of the first time I ever reverse engineered a piece of software. It's been gratifying and rewarding, but I hope to retire from public life before it's time to celebrate the 30th.
Starting work on a C++ reverse engineering training class. Please point me to malware families that make heavy use of C++. The more the better -- STL (w/customized template parameters), exceptions, placement new, multiple/virtual inheritance, dynamic_cast, C++11/C++17, etc
Outside of my upcoming new C++ class, I'm pondering a class just on Hex-Rays. Potential topics include: all UI/type system features, RE strategies, selected Hex-Rays internals, and the SDK. Is there public interest in that? If so, how interesting to you is the SDK vs. the others?
I uploaded my old OCaml-based binary analysis framework to Github since I'm no longer using or developing it. (Abandoned, no support, has obsolete dependencies, is difficult to build, for educational purposes only, etc.)
Current experiment: replacing the control-flow structuring algorithms in Hex-Rays with my own. (Actually, this is a very small tweak to the existing code -- basically reversing the order of a single list.)
What would you like to see in a C++ reverse engineering training class, beyond: structures/classes, templates/STL, inheritance (incl. multiple/virtual), and polymorphism? (As in, any particular libraries, features, or technologies, e.g. <atomic>/<thread>, lambdas, or COM/Qt/ATL?)
Before reverse engineering modern C++ binaries, I wrongly thought move/forwarding/etc. would eliminate copies. There are more copies than ever. In this function, appending a string onto a vector involves five full copies, and a partial one just for laughs:
Today's experiment: targeting common array access patterns on x64, via graph theory and microcode patching, to coax Hex-Rays into generating more for loops.
Mobius Strip Reverse Engineering is announcing public sessions for its Static Reverse Engineering and SMT-Based Binary Program Analysis training classes in April/May 2023 in Manassas, VA. More details can be found on our website:
@arnaugamez
@mr_phrazer
Don't waste your time attempting to publish real-world deobfuscation research in academic venues. That corner of academia is divorced from reality. None of them have ever reverse engineered or deobfuscated anything. They are not your peers. They cannot evaluate your work.
There's so much math to learn and retain, it's hopeless. I remember the stuff I use regularly, but forget everything else and have to relearn it every time, and I expect it'll always be this way
I just learned that Patrick Cousot himself wrote a book about abstract interpretation, out last month: Seems about as dense as his articles -- no reprieve there, sadly -- but centralized and with redundancy removed, at least. Ordered!
If not for a deadly virus outbreak, I likely would've been having beers with my reverse engineering friends on the Hilton Montreal terrace right now. Lamentable! Stay safe and brilliant until we meet again.
Z3 is such great software. I had to ask nicely, but I just convinced it to solve a pseudo-Boolean optimization instance with 160,000 variables. (It took 35 minutes, but I can live with that.)
Looking for new research topics, hence re-upping my periodic question: Which modern malware families have the most difficult custom (non-commercial) obfuscation? Not FinSpy or Obfuscator-LLVM, as I already broke them. [Feel free to email me if new ones arise in the future.]
.
@offensive_con
is a great conference -- the hard-core technical content we all crave, a great audience (a different mixture than North American conferences), and very professional and friendly organizers. You should definitely go if it aligns with your interests.
@netspooky
For MSVC-compiled C++ binaries, the PE header's linker version is generally accurate and reliable as to which version of the C++ standard library it will contain. Combine that with to get an accurate idea of what the inlined STL functions will look like.
Just signed up for my first Github sponsorship to
@awesomekling
and SerenityOS. I like the project and want to see it/him succeed! Check out his YouTube content if you haven't; I've been coding for 26 years and still find it valuable (and humbling) e.g.
I would like to give a talk related to some pure reverse engineering research I did this year, but haven't been paying attention to industry conferences for a while. Apart from next year's RECON, which venues might accept such a talk?
Just got my first shot of Pfizer. It was quick, painless, and professional. I hope your country soon overflows with supply so you can get yours, too, and we can get back to normal. Stay safe!
Just spent an hour and a half badgering the Ghidra developers. That thing has a flabbergasting amount of great features; I'm sure I haven't even scratched the surface. Looking forward to getting to know it better.
ComRAT v4 is a large, sophisticated backdoor that uses many C++ features: polymorphism, templates, multiple/virtual inheritance, many parts of the STL, etc. This is the result of six weeks of static RE, with no symbols/RTTI. Every developer-written function in the IDB is analyzed
@0xrepnz
Good explanation! Hacker's Delight by Warren is a great book that explains a lot of these low-level tricks . Also you may be interested in my slides entitled "Compiler Optimizations for Reverse Engineers"
It's too bad there aren't more industry publications on Hex-Rays scripting/plugins. The more I use it, the more I enjoy it. It opens up new vistas in day-to-day reverse engineering automation, and the results are more easily reusable than ad hoc IDA scripts
Today is the first time I ever reverse engineered MSVC to figure out how an optimization works. It took about an hour, thanks to the public PDB for c2.dll. Note to self: try that sooner next time.
Public service announcement: almost every reverse engineer who with whom I've discussed it confuses "virtual inheritance" with "virtual functions". They are not the same thing. If that's news to you, read up on virtual inheritance:
Fred from
@tetrane
just gave me an exciting demo of their product REVEN (e.g. ). It seems extremely useful for dynamic analysis (particularly vulnerability analysis). I look forward to playing with it more in the future and I hope they succeed!
Just spent an hour looking at MBA obfuscation for the first time. At first glance it seems pretty trivial to break with some abstract algebra. (This was done automatically, based on black-box dynamic analysis, not using a SAT/SMT solver or any third-party libraries):
Writing Hex-Rays plugins is much more challenging than IDA plugins - with many fewer existing plugins to crib from, both times I wound up reverse engineering hexrays.dll and _ida_hexrays.pyd for examples of how the APIs are used in practice
🫣 Let's be honest: nobody likes to look at C++ code. Except every reverser had to audit a C++ binary at least once in his/her career.
At
#HEXACON2023
,
@RolfRolles
will give a good deal of tools to help you analyze C++ codebases, more inforation here:
Nothing satisfies me more as an instructor than when students spontaneously come up with, and succeed at, new project ideas in class! The 2.5 years of work that went into creating the course were all worth it (as grueling and miserable as they were at times).
I am by no means a vuln hunter, but during day 4 of
@RolfRolles
’ SMT Analysis Training, I wrote a program using Z3’s SMT python api to find a new integer overflow vuln in some firmware. 😎
This blog entry (by two talented authors) breaks new ground by being the first foray into applying equality saturation to deobfuscation. Definitely give it a look if that overlaps with your interests!
Folderization was the missing piece for large-scale reverse engineering. I put the target's 2761 functions into folders, and I don't feel overwhelmed by it anymore; I feel like I understand the entire program now.