Monthly Archives: March 2010

A little bit of history about the debugger

In the last entry i mentioned that the debugger is compatible between 32bit and 64bit (you can debug a 64bit program with the 32bit debugger and vice versa). Why is that and why do we have 3 different debuggers anyway?

Its interesting actually: a lot of these things were not planned to be like that. In fact the whole debugger was not planned at all. It was one of these things that start as a simple idea and just grow into something huge. To understand more about that we have to go all the way back to when the current debugger was written. It was the year 2004, the current PB version was 3.91. The Windows debugger consisted of a small window which only showed the global variables and allowed stepping through the program. PureBasic for Linux was much worse. There was no IDE at all and the debugger did notning more than print the output of Debug statements to the command line.

At that time i decided to write the new IDE. Thats another project that vastly outgrew the original plans even on its initial release. The plan was to write a simple crossplatform IDE with the same set of features as the IDE for Windows at the time. Most of the IDE was working and we planned to include it in the 3.93 release. All it did for debugging on Linux was to make sure the program runs in a terminal window so you can see the debug output. To improve that, Fred gave me the source to the Linux debugger lib (just a collection of printf statements) so i could improve it.

The Windows debugger was just a library that got compiled into the executable which launched a new window in a thread. The idea was to do the same for Linux and just duplicate the feature set of the Windows debugger. Man am i glad that this didn’t work out! If that worked, we would probably be stuck with two very platform specific debuggers with probably a similary limited feature set as we had back then, and maybe no OSX debugger at all. Well, it turns out that Gtk doesn’t play so well with threads, especially when the main program doesn’t even know about the debugger thread.

New plan then: the debugger had to be an external program. Usually debuggers get access to their target programs by reading their memory directly, even patching the target program’s code as needed. Since this approach is quite platform specific we decided against that. After all, if we write a new debugger it should be available for Windows as well. So we still have the design of a special debugger library which gets compiled into the executable even today. This library allows the debugger to connect via pipes to exchange the needed data. This design requires a minimum of platform specific code and it works quite well.

Now we come to the reason for the three debuggers. The console debugger exists because I started this work on Linux with the given minimalistic library that just printed the output. This just grew to be the console debugger. To ease the testing i first implemented a new feature such as array display in console mode to make sure i got all the interaction with the compiler generated tables and stuff like that right and only as a last step i also added a GUI version of that. I still do it the same way today. So the main purpose of the console debugger is the development of the debugger itself. I also use it a lot to debug the IDE because I do not want to load that beast into the gui debugger every time.

I almost didn’t write the standalone gui debugger. After all, why add an extra window when you can access all the features from the IDE. The only reason i did it was to not kill the jaPBe project. Without any way to debug a program other than the IDE, people would have abandoned it fairly quickly. I was tempted though. After all, it would have been an easy way to get rid of the “competition”. 🙂 Looking back, the standalone debugger has proven useful in many more situations than that. For example this way programs can be debugged with elevated privileges on Vista or 7 without the need for the whole IDE to run at that level. Also it has again proven helpful in the development of the IDE and debugger itself. The codebase of the gui debugger is much smaller than that of the IDE so it is much easier to find a bug in there. Since most of the code is shared with the IDE we get that fixed as well.

The three debuggers also help in porting to a new platform like with the x64 port. As soon as the compiler works at least at a minimum, the first order of businesses is to get the console debugger up and running. This is then used to further debug the compiler output. When that works reliably enough the next step is to get the gui debugger and IDE running. Getting the IDE to run is quite the milestone for every port, as it is quite huge and makes use of a lot of different libraries.

Back to the communication between executable and debugger. As i said, it works through pipes. Its a simple package based binary protocol that sends a header structure and then data with a variable length. It hasn’t changed much since the first version although the code for handling the pipes has changed quite a lot. With the 64bit port we faced the question what to do with this communication protocol. Changing all values to 64bit would have been rather simple and the obvious choice. The reason why i choose not to do it was again for development purposes. The 64bit version was still quite unstable and having a stable 32bit debugger there to analyse it was a big help. At one point i was running the 32bit gui debugger to debug the 64bit gui debugger while debugging a 32bit executable. 🙂

So this is why the debuggers are compatible. The communication protocol keeps its 32bit header structure and the debuggers can adapt to the executable’s bit size when working with the contained data. Now with the 4.50 this design pays off once again because you can now actually load the 64bit compiler into the 32bit IDE for compiling and debugging. The final step was to make the PowerPC and x86/x64 communication compatible. This is something i just did recently because its a bit more of a change because the different byte-order has to be taken into account and so far there was just no need for this. Anyway, i did not do all this work just so you don’t have to use Rosetta on the IDE on OSX. Now that the debugger communication is compatible between all platforms, the way is finally open for one of my longer term goals: network debugging across all platforms.

Now with the next version we will have that as well: Network debugging across all platforms and processor types. I recently ran the debugger in a 32bit Linux VM, debugging a PPC executable on the Mac, while itself being debugged by another debugger instance on 64bit Vista. That just rocks! 🙂

To all those not really interested in this kind of stuff: you benefit from this as well. All these steps to new platforms and crossplatform compatibility have made the debugger more stable as a whole too, because many bugs become more apparent when you switch platforms frequently. So everybody wins.