Monthly Archives: August 2008

Automation to the rescue !

PureBasic has grown step by step, like every other long time going project,  adding support for more platforms accross the time. With the forthcoming release (4.30), we have added 2 official distributions: Windows x64 and MacOS x86. This led us to a problem: 4 additional packages to manage (full and demo for each of the platforms). Until now, we handled the packaging phase on demand, with own created tools (on Windows) and scripts on Linux and OS X. This was very time consuming to do all the packaging stuff, as you have to boot the correct OS, update all the SVN repositories and pray to have everything still compiling/working. Then you have to check the resulting package, check if nothing is missing and finally upload it on the remote site. As you can see, there was room for improvement. And we did. Let’s see how we processed:

1) We killed the Windows ‘update’ package. It was a .zip package which contained the delta files against PureBasic version… 2.00. Yeah, it was  8 years ago. This package was useful at time, as it was way smaller, and the internet connections were much slower. Now, the .zip is actually bigger than the regular install shield package, as every single files have changed since the version 2.00, so there is no delta anymore. We modified the install shield package to support updating. Result: 1 package removed.

2) So starting from now, we will have 10 packages (Windows x86/x64, Linux x86, MacOS X PowerPC/x86 – all in both demo and full version). The obvious idea was to use virtualization to have a single OS running all the PureBasic supported platforms. We got an offer from a PureBasic user to lend us a specific computer to use it as a build server: that’s was a perfect timing.  We installed Windows Vista x64 on it, and setup the according virtual machines.

3) The next step was to script everything to do all the packaging phase automatically. Here are (roughly) the actions needed:

- Update all the SVN repositories (we have 5 differents)
- Compiles the compiler, libraries, IDE and all provided tools (DocumentationMaker, LibraryMaker, PureUnit etc.) from the sources
- Run the test units for the libraries and the compiler
- Create the documentation
- Create the full package
- Create the demo package (derived from the full package)
- Upload the packages on

And you can repeat it for every OS. We used ‘bash’ (from cygwin on Windows and the native one on linux/osx) to have a common base for scripting. Windows could natively compiles x64 and x86 version of PureBasic, so this case was sorted. Then we used ssh (plink.exe from the putty package) to connect to the various running virtual machines and launch remotely the appropriate packaging script. The automatic ssh connection is handled with ssh key authentification. After a week of work, we had our automated build server, and believe us, that’s really a time and headache saver ;). Ha, the whole build time for all versions when nothing is built is over 6h. On a Core2 duo – 3Gb. When all is built (ie: no change in the libraries and so), it’s just under one hour to build everything and put it on the web site – which is quite ok.

With this new tool, we will be able to publish synchronized alpha, beta and official releases without having to crawl all over the OS and we hope it will increase the overall PureBasic release quality. It should also help when new platforms will be available (linux/osx x64 anyone ?).

Have fun !