: 4.20 Beta 6 for Windows is out, 4.20 Beta 2 for Linux and 4.20 Beta 1 for MacOS X as well !
Here is the fifth beta release of the forthcoming 4.20 version of PureBasic. It's getting longer as expected, and we would like to spend some time to explain why. Don't flip out, it will be probably the last beta, if everything goes nicely
. Here is the story:
At first, 4.20 was meant to be a 'new command only' release. So no big problems were foreseen, as we won't work on the compiler or debugger. Better, most of the new libraries were already written, as we often do extra interresting work when fixing bugs is becoming too boring (we don't include the new library in the release tree, so the library can maturate pacefully). So good so far, 2 months after the 4.10 release, a new beta hit the ground. And now, things got a bit more complex.
We encounter again a LccWin32 bugs/limitation so we decided it was time to migrate to VisualC++ 2005. This choice was made for serveral reasons:
1) It is the 'official' C/C++ compiler on Windows
2) It is free (we use the excellent 'Express' edition)
3) It produce very good code (much better than lcc - some libs gained 50-100% in speed just by switching the compiler)
4) It has a X64 version as well, so we have a direct port to 64 bits Windows without too much hassle (at least for the libraries)
5) Bug free (at least on the compiler side)
So we started so migration of all the libraries and compiler libraries (SystemBase, StringManager, etc..). Some of them used lcc inline ASM, so needed quite some work to have them to work. VC++ is also more picky and tons of warning appears and needed a correction. Moreover we activated the 64 bits portability check and our console were definitely flooded. Think than when we migrate the whole commandset, it affect tousand of files, all the makefiles needs to be adapted etc. But as you can understand, it is for the best. Several weeks later, we got everything building fine and as you guessed it, the 64 bits compiling was (almost) working as well. It was the base to begin the work for the famous X64 compiler.
We though we were done with this, but a strange VC8 optimisation affected the way PureBasic called string functions (PureBasic assumed than the arguments passed on the stack for the functions would never be modified by a C function (and it is the case with lcc and gcc). But there is no clear infos on this, and VC8 actually use the stack parameters as temp area if needed). So a compiler modification was needed to fix that, after a lot of investiigation (we got random crashes on some functions).
In between, we faced some regressions so we have written much more unitary tests to make the commandset more robust to changes. These are running using the 'PureUnit' tool developped by Timo (Fr34k) which will made its way in the official PB package soon, as it's really a great help to ensure a module behaves as expected, on every plateform.
Now that all the libraries and helpers were compiled with VC8, we couldn't let the compiler itself using Lcc. So we started the migration as well. The gains for the compile time was good (10-30%) but we needed to trick a bit VC8 to use an old VC runtime dll (MSVCRT.dll) instead of the new VC8 one, as it wouldn't run out of the box without shipping the VC8 runtimes. Google helped here
On another topic, CVS is dying and replaced everywhere by SVN. We migrated the libs repository some time ago, but the compiler and internal tools were still using this old aged CVS. The migration wasn't very smooth as the CVS repository was on an NT based computer while the script needed to do the migration were on linux. After some twists, it finally worked. So SVN everywhere. And it's way better, honestly.
Timo wanted to add a profiler to the IDE, so he did. And he did well. So it remainded me than i didn't profiled the compiler since a while and it could be quite fun to do. I tried to find a good profiler on Windows (anyone ?) but only found the costly Quantify (and even, the demo version showed only API call with VC8 executable, dunno if it's on purpose or if it's a bug). So i fired it my linux box and used gprof and the linux compiler. I used the IDE source code as benchmark (60 000 lines). And here, the disaster: 23 millions of stricmp() call - called when looking for a token (like constant, structure, procedure, functions, macro etc.). Seems like my old lookup tables are suffering here. For sure, the residents have almost 8000 constants and the lookup table was way too small. I decided to change that with real hash map. The gain was impressive, now only 500 000 calls to stricmp() are done. The whole processing time of the source code fall down from 3500 ms to 500 ms on my computer. This optimisation is available in the Beta 5, so if you have a big project, you may feel the difference.
We also wanted to tackle most of the important remaining bugs since the last 4.10 version (and even before), so we spend quite some time to fix thus (on the 3 OSes). This led me to rework the 'QuadUnit', which is used to manipulate and execute quad based operation. The one developped for PB 4.00 was good but way too complex and some case were really hard to handle. So i simplified the whole design and recoded several key part. This change is also in Beta 5, so don't hesitate to report bugs if any. At least, all the quad reported bugs are now fixed. It should have no or very few regression as all fixed bugs are part of the compiler unit tests (every compiler bugs fixed get a new entry in the regression test suite).
In fact, we added too much commands for a single version, but hey, we are megalomaniacs. Thus we got plently bugs reports on these (logically as these new libraries were not trivials). Thanks to all of you for the reports and the patience.
As we can't stop, when we need to work on some fresh stuffs, we started the 64 bits version of PureBasic and the MacOS X x86 version. One could think than the MacOS X x86 version would be just a matter a recompiling, but you couldn't be that wrong. OS X x86 ABI is just horrible and need very strick stack alignement everywhere (on 16 bytes boundaries) which the Windows/Linux version doesn't requiers. If the stack is misaligned, it sometimes work, sometimes not, leading to very hard to track bugs. So, the compiler has been reworked to handle stack alignment all over the code. Moreover, FASM is not available on OS X x86, so we decided to migrate all the assembly libraries (linked list, misc etc.) on NASM. Indeed the compiler has been modified to output NASM compatible code instead of FASM. On the x64 side, the work has been even harder as the assembly itself isn't the same. Well, both are coming quite nicely, and the public test will come soon.
The last but not the least, many new commands means a lot of doc to write. This takes a lot of time, as we do it in 3 languages as you know. The beta 5 comes with the doc in 3 langages, for all new commands.
Ok, so you can grab the beta 5 on your account and test it. The beta for Linux and OS X are ready as well, but we will wait a bit before publishing them (to see if there is no major quirks. And btw, if you were using the beta 4, check your 'Temp' directory as a nasty bug didn't remove all the PB temp dirs and it could take some space.
Have fun !
The Fantaisie Software Team