Debugging – Low Level Software Analysis

1. Debugging – Low Level Software Analysis Roberto Alexis Farah – Microsoft Corporation rafarah@Microsoft.com - http://blogs.msdn.com/b/debuggingtoolbox/

2. Goal of this presentation… • The goal is NOT to teach you debugging or reverse engineering. • The goal is to: • Demonstrate what debugging is, requirements and the power of debugging. • Show the importance of thinking low level while debugging.

3. What is debugging? • “Debuggingis a methodical process of finding and reducing the number of bugs, or defects, in a computer program or a piece of electronic hardware, thus making it behave as expected. Debugging tends to be harder when various subsystems are tightly coupled, as changes in one may cause bugs to emerge in another.” • http://en.wikipedia.org/wiki/Debugging

4. Debugging & Troubleshooting • Some people consider Debugging and Troubleshooting the same thing. • Others consider: • Debugging: low level software analysis you do using a debugger • Involves going deep into the software internals • Troubleshooting: analysis of logs/traces using other tools • Usually what is done before the need for debugging • Wikipedia: • Troubleshooting: http://en.wikipedia.org/wiki/Troubleshooting • Debugging: http://en.wikipedia.org/wiki/Debugging • “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.” Brian Kernighan – Co-author of the first C Programming Language book

5. Where is debugging used? • To isolate problems like: • High CPU hangs • Low CPU hangs • Crashes • Memory leaks • Performance problems • Application errors • Debugging is used to perform Root Cause Analysis.

6. Think Low Level…

7. How to acquire debugging skills • Knowledge of C/C++ • Java, Windows and .NET are build in C++ • Assembly language / Reverse Engineering • Sometimes you just don’t have access to symbols or source code • Algorithms / Data Structures • Linked List, Hash Tables, Arrays, etc. • .NET and CLR Internals • Important to know the internals, not only the programming language. You want to know how the Managed Heap works • How to use a debugger. Preference for WinDbg • WinDbg has less footprint than Visual Studio, easier to install and more powerful • A true desire to know how things work internally, what is under the hood  • Reference: http://blogs.msdn.com/b/debuggingtoolbox/archive/2007/06/08/recommended-books-how-to-acquire-or-improve-debugging-skills.aspx

8. What do you see here? Minesweeper running on Windows 8

9. Thinking Low Level • Minesweeper very likely maps the board in memory. • Thinking about data structures, the board could naturally be created as an Array. • It would be reasonable to think that Array items would indicate whether there is a bomb or not. • What else do you see?

10. Thinking Low Level • DLLs, Assemblies, COM objects…

11. Thinking Low Level • Since it is a Windows application (not a console app), it has a Message Loop.

12. Thinking Low Level • Native Heaps and Managed Heaps if it is running Managed Code.

13. Thinking Low Level • Registers: A small amount of storage available as part of a CPU or other digital processor. • Faster access than memory.

14. Thinking Low Level • 1 MB of call stack space for each thread.

15. Thinking Low Level • Call stack calling convention…

16. Thinking Low Level • PEB – Process Environment Block – data structure with information about the process • TEB – Thread Environment Block – stored information about currently running thread

17. Thinking Low Level • Windows was designed around Little Endian architecture. • For example, consider the 32-bit number, 0xDEADBEEF stored in Little Endian format:

18. Thinking Low Level • The previous slides are, by no means, a complete list of the internals of an application. • The idea is to show you what is under the hood although not necessarily “visible”. • When debugging there are different ways to visualize the same information.

19. Demo #1 – Seeing the application internals • Modules/Assemblies • Threads • Registers • Native heaps • Managed heaps

20. Demo #2 – Hacking Minesweeper • Let’s cheat on Minesweeper...

21. Demo #2 – Hacking Minesweeper • Summary: • We need to think low level to find a starting point to debug the application in order to save debugging time. • First plan was to find methods dealing with random number generators. • Further research showed us there is a method which displays the bombs. • The method is possibly activate via a combination of keys and we have the method with the hot keys used to cheat. • The goal is not to figure out how to activate the key combinations but rather to force the application to do what we want. • We were lucky to figure out the method that shows bombs was already created by the developers. That was not part of the initial plan we had. • Same trick for Minesweeper on Windows XP: http://blogs.msdn.com/b/debuggingtoolbox/archive/2007/03/28/windbg-script-playing-with-minesweeper.aspx

22. Demo #3 - Hacking Minesweeper • There is a method which does exactly what we want: shows the mines! • The goal is not to find out the combination of keys to activate the cheat. • The goal is to “patch” the application and force it to do what we want. • It’s a more complex approach, but much more fun! 

23. Demo #4 – Automating the debugging • Examples of debugging automation: • DebugDiag scripts • Extensions • Dscript tool (internal – SQL Server team) • WinDbg scripts • WinDbg scripting language is very similar to C programming language.

24. Questions anyone?

25. The End • Want to see the debugging session on how to hack Minesweeper? • Want the script to cheat on Minesweeper? • Want to know more about debugging and troubleshooting tools? • Go to: http://blogs.msdn.com/b/debuggingtoolbox/