Author Archives: Fred

PureBasic on Apple M1 processors

Last week, I bought a new Mac Mini to be able to port PureBasic on the new Apple M1 chip. First boot looks very familiar, you can’t really tell there is a new processor here. A quick look at the Task Monitor and we can see all the programs running on an ‘Apple’ CPU. Great. All seems smooth so far, a very quiet computer in a small form factor.

After a few minutes toying with the prefs to plugin my PC keyboard and Monitor on a KVM switch, I was ready to start to dev, and downloaded XCode. It was the longest installation I never experienced, at a point I though the Mini was broken. It tooks over 4 (four !) hours to install (after downloading). Seems to be known issue if I believe all the posts found on Reddit. Needed to install homebrew for subversion (I know, I know) and I finally started hacking the compiler.

This promised to be exciting, as the C toolchain isn’t based on gcc but on clang/LLVM. I expected some adjustements to do here and here, but there were actually zero. Everything compiled out of the box and the first running executable was created after half day. Two more days and the whole compiler test suits was working which was amazing ! There is indeed some more work to do to build the full package, as some libraries needs to be tuned (missing headers due to new Cocoa SDK mainly), build scripts adjusted and so on, but it does looks very bright.

This will be the first version of PureBasic shipped without an assembly back-end. It does feel a bit weird for me, but I know it will do the job just fine to build your cool apps on new Macs !

Sneak peak into new CPU support for C back-end

I just finished the x86 (32-bit) support for the C Back-end on Windows and it barely took a week to do. Most of the time was spent on upgrading the build chain (linker, windows static libs, packaging the 32-bit version of gcc etc.), it was very very fast. Next step is to work on a ‘real’ new architecture, probably ARM M1 for OS X, looks interesting !

The IDE is finally working with the C back-end !

This is the milestone we waited to validate the C back-end. It tooks quite a lot of iteration with the alpha testers (and thanks to all of you all btw) to iron major issues, but here we are: the IDE is mostly working when compiled with the C back-end. This is a big deal: the IDE is a very large software of about 130 000 lines of PureBasic code and using a lot of different features of the language. So what’s the functional changes ? Well, nothing. And that’s on purpose. The plan is to be able to switch seemlessly from ASM to C back-end and be able to compile the IDE on new platform, like Raspberry or OS X M1.

The only noticeable change is the compilation time, which jumps from 5 secs on the ASM back-end to a whooping 24 secs on the C back-end (based on GCC) with the debugger on and optimisation disabled. Keep in mind I got a very old CPU (Core i7 860 from 2009) and as discussed previously, just swapping the C compiler could lead to dramatic improvements.

All in all, it’s very good news, and we are looking forward to fix the last remaining showstoppers and creating a beta version for all the OS, including may be some new like Raspberry !

Quick look to PureBasic C back-end performance

PureBasic is currently using raw assembly code generation, which means the code is translated line by line to its corresponding assembly counterpart. This way to generate code has pros and cons: it’s easy to do but we loose the code context, so doing advanced optimizations (like variables to registers, loop unrolling, function inlining, code vectorization, variable removing etc.) are off the table. The only optimization PureBasic can perform is once the full assembly code is generated: a pass to detect patterns which can be optimized and replaced by more efficient one. For example:

          MOV ebx,dword [v_Gadget_ImagePool]
          INC ebx
          MOV dword [v_Gadget_ImagePool],ebx

is transformed to:

          INC    dword [v_Gadget_ImagePool]

PureBasic x86 can detect 16 different patterns and optimize them. It’s not much, but it still improves the final code quality. This kind of optimization is called peephole optimizer. It works on final code generation and can’t do much regarding high level optimization. The good point is the speed of it, the additional pass barely add any time to the compilation time. So you may ask why PureBasic can’t generated more complex optimizations ? Well, because we just don’t have the time (and let be honest, the skills) to do it our-self. It takes a lot of efforts which requires large programmer teams and academic researches.

Entering the C language, the world most used low-level language and probably the one which best optimizing compilers. PureBasic is now able to leverage the power of its optimizations, so let’s take a real world look at it. We are using the 3D example ‘MeshManualParametrics’ found in the PureBasic package. This is a mix of 3D rendering (through OGRE) and 3D calculations done in PureBasic. Here are the results:

PureBasic x64 – assembly back-end : 192 FPS

PureBasic x64 – C Back-end with optimizer enabled (-02) : 298 FPS

That’s basically a 50% increase in frame-rate just by switching the compiler. It’s really a lot. Another interesting point is the speed of the executable of C back-end without optimization (-O0) was 192 FPS as well, like PureBasic assembly back-end. Not that bad for small team compiler !

Sneak peek to C generated code

Next major PureBasic version will feature a brand new C backend, which will allow to support virtually any CPU, current and future. The generated file is a big flat file, with no include directive to allow the fastest compilation speed as possible. In the meantime, we decided to create a C output as clean as possible to allow easy modification or even direct reuse into a C project. Let’s take a look to it by comparing a PureBasic snippet and its generated C code counterpart:

Procedure Test()
  Counter = Counter + 5
  
  If Counter = 5
    MessageRequester("Hello world", "Hello")
  EndIf
  
  ProcedureReturn 6
EndProcedure

Test()

And the generated C code (omitting all boilerplate) :

static unsigned short _S1[]={72,101,108,108,111,32,119,111,114,108,100,0};
static unsigned short _S2[]={72,101,108,108,111,0};

static integer f_test() 
{
  integer r=0;
  integer v_counter=0;
  
  v_counter=(v_counter+5);
  
  if (v_counter==5) 
  {
    integer rr0=PB_MessageRequester(_S1,_S2);
  }

  r=6;
  goto end;
  end:
  return r;
}

int __stdcall WinMain(void *instance, void *prevInstance, void *cmdLine, int cmdShow) 
{
  PB_Instance = GetModuleHandleW(0);
  PB_MemoryBase = HeapCreate(0,4096,0);
  SYS_InitString();
  PB_InitDesktop();
  PB_InitRequester();

  integer rr1=f_test();
  
  SYS_Quit();
}

As we can see, the code is pretty close to PureBasic one, and easy to read. Inline C will be supported, and all the objects like variables, arrays, lists etc. could be accessed by following the generated token name pattern (example: ‘v_’ prefix for variables followed with the PureBasic name in lowercase).

What about compilation speed ? We are using GCC 8.1.0 with no optimization (-O0) on Windows 10 x64 on a first gen Core i7 to perform the tests. We compile a 13.000 lines program, DocMaker, with debugger ON:

  • GCC backend: about 3 seconds to create the executable
  • FASM backend: about 1 second to create the executable

So that’s about 3 times slower but still an OK time to develop. Next test is a big program, the PureBasic IDE, with about 125.000 lines of codes with debugger ON. We also includes Microsoft VisualC++ 2015 in the tests:

  • GCC backend: about 24 seconds to create the executable
  • VC++ backend: about 9 seconds to create the executable
  • FASM backend: about 4 seconds to create the executable

GCC is a lot slower, about 6 times, than the current FASM backend. The VC++ backend is about 2 times slower, which is much better but not that great. All in all, the FASM backend will still be better to use for quick development cycle. Keep in mind than it is very early tests and as we just seen, just switching the C compiler could dramatically increase compile time. That’s it for now, next time we will focus about runtime performance between C and FASM backends !

PureBasic and increased varieties of CPU

Early versions of PureBasic started on Amiga, with the support of the Motorola 680×0 CPU series. PureBasic has been designed from start to generate raw assembly code, and the first working CPU assembly back-end was the one for 680×0. As the audience on Amiga was quickly falling, we decided to move on PC. In 1998, there was only one PC CPU architecture, the x86, and it was decided to add a new assembly back-end for it. It is a very time consuming task and requires a lot of learning to be able to handle a new assembly. 680×0 and x86 architectures are quite opposite but it fitted in the PureBasic architecture and PureBasic for x86 was a reality.

Shortly after, we decided to go really cross-platform and support OS X, which was running on PowerPC processors. A new assembly back-end was needed and we did it again, during almost a full year to get something working. Shortly after, Apple announced they will drop support for PowerPC and choose x86 instead ! That was a huge blow for us, and a lot of time wasted. In the meantime, x86-64 was released and slowly gaining traction and we knew we should support it as well. Unlike the name suggests it, x86 and x86-64 are two different beasts, and a lot of work was needed to make it happen. Once done, we though we could finally focus on functionalities for a while, but computer science is evolving at quick rate and a new CPU architecture appeared and took the world by storm, first on mobiles, then on tiny computers and now on desktop: ARM and ARM-64.

So here we are, to support these new platforms, we face the same issue again: we need new assembly back-ends. As a small company, we can’t dedicate a whole year every time a new assembly emerge, so we need to find a future proof solution. Meanwhile, a fork of the PureBasic compiler has been created to generate high-level language instead of raw assembly. This allowed to create SpiderBasic, a PureBasic-like language which can run on the browser by generating JavaScript code.

When Apple announced their new computers will be running on ARM, we knew we needed to change our way to handle new processors architecture and we decided to leverage the work already done of SpiderBasic and port it back for PureBasic. We experimented a lot with LLVM for quite a time now, but it is an headache to find it on all architectures. Also it’s a kind of mix between assembly code and high-level language so it didn’t fit either PureBasic or SpiderBasic back-end architecture. For the past 8 months, we decided to focus on a new high-level back-end for PureBasic: the C language.

The good news is the whole PureBasic compiler test suit is successfully running on the C back-end, so we are very confident we could have a compiler ready to test quickly. Stay tuned, more details to come soon !

20th Anniversary

It’s already the end of 2017 and no post has appeared this year on the blog. Syed Ibrahim (@TI-994A) brings up the idea and the post to fill the gap ! Have a nice reading and see you in 2018, the 20th year of PureBasic existance (I can’t really believe it) !

A BRIEF HISTORY
1998 to 2018

The coming new year is a very special one for PureBasic, as it marks twenty years since the language first appeared, in 1998. So, it seems fitting that the last blog entry of 2017 be dedicated to this glorious history, and its advancements to date.

PureBasic was first commercially available in 1998, for the Commodore Amiga, and shortly after that, in 2000, for the PC. It uses a predominantly BASIC-like syntax, with some similarities to C and other mainstream languages. Built upon of an assembly language compiler, it has the unique ability to include inline assembly code right alongside its own PureBasic code. Needless to say, this also results in very small compiled binaries, and blazing-fast execution of the compiled executables.

PLATFORM EXPANSIONS
From Amiga, to Windows, to Linux, to MacOS, and Beyond

In 2001, a Linux version was added to the line, and a few years after that, in 2005, a version for Apple’s PowerPC. The PureBasic Suite, as it stands today, was completed in 2008, when another version for Apple’s x86 operating system was added, making it the only non-RAD independent programming language in the world to support all the major desktop platforms.

The language has no real parallel as it is not a rapid application development tool (RAD). It is a true platform-native compiler that produces independent, standalone applications that do not require any additional supporting libraries, frameworks, or virtual machines. The compiled binaries remain very small and tight as no such dependencies are bundled within the final packages either. PureBasic compiled programs would run on the intended platforms right out of the box. Simply copy the compiled executable and drop it into any Windows, MacOS, or Linux PC, and it will run, and run fast.

UNPARALLELED PERFORMANCE
A true, native, multi-platform compiler

Besides old-school powerhouses like Assembly and C, all other popular programming languages today fall victim to burdensome dependencies. Java applications will not run on machines that do not have their corresponding JVMs (Java virtual machines) pre-installed, and .NET programs require compatible versions of the framework in order for their applications to work. Ruby and Python are interpreted scripting languages, and Swift and Objective-C are purpose-built for the MacOS. RAD tools like Xamarin and Xojo have no place in this arena as they tend to bundle massive libraries within the final binaries, resulting in bloated and sometimes slow executables.

This leaves PureBasic as the sole forerunner in its class, as a truly multi-platform compiler, that produces native executable binaries which are small and fast, without requiring any additional resources whatsoever.
Over the years, the language has undergone many modifications, as the developers attempt to keep it updated and relevant with the latest trends and technologies. It has introduced a plethora of tools and libraries to aid the programmer in simplifying development tasks, without sacrificing performance. While some upgrades have come at the price of legacy-compatibility, the end-product leaves nothing to be desired. PureBasic gets better with every new iteration, and the diligence and dedication of the development team to bug-fixes and feature implementations are highly commendable.

PureBasic also conforms to the industry’s strictest policies, with its dual-release model. The compilers are released in beta and stable LTS (long-term support) versions, which allow for legacy developments to continue unabated. Furthermore, every version of PureBasic, from publication year 2000, is still available for download from the site’s unique “museum” page. That’s pure dedication.

The team has recently started development on a web-platform tool, known as SpiderBasic, which is syntax-compatible with PureBasic, and also as easy to use. That will definitely hold many new promises for the future as well.

FEATURE CHART
Platforms, Syntax, Toolkits, Designers, and true 64-bit

– simple, readable, terse syntax
– purely procedural paradigm with true multithreading
– supports both 32-bit and true 64-bit environments
– true cross-platform compilation of unmodified code on Windows, MacOS, Linux
– includes comprehensive libraries usable on all platforms
– includes 2D & 3D game and multimedia engines for all platforms
– implements native UI of each OS without any external libraries
– includes visual form designer that automatically generates UI code
– OS API functions, constants, structures accessible without import
– able to import and utilise external third-party libraries
– able to utilise inline assembly for speed-critical code
– compiles to very small, very fast binary executables
– executables require absolutely no dependencies, frameworks, or VMs
– executables can be run out of the box without any installation
– lifetime license on single purchase that includes all three platforms

LOYALTY LICENSING
A Lifetime of Power Performance

Last, but not least, is the unbelievable cherry on this cake. The entire PureBasic Suite, which includes the compilers for Windows, MacOS, and Linux, costs just €79.00, and comes with LIFETIME upgrades! That’s right. Even the first licensee of the very first PureBasic version in 1998 is still eligible for upgrades to the latest current versions today, absolutely free-of-charge.

Now, that’s one feature that no other commercially licensed language has ever offered.

Thank you, TEAM PureBasic, and specifically its commander-in-chief, Mr Frédéric Laboureur, for your diligence and generosity through the years.

Enter GTK3

Since PureBasic 5.40 LTS, GTK3 is the default GUI toolkit used by PureBasic on Linux. It seems like a no brainer update for our users, but the fact is than GTK3 doesn’t fit well the PureBasic GUI library and we have several bad hacks here and here to make it work at all. We decided from the start in PureBasic to handle pixel perfect sizing of the GUI elements to allow maximum flexibity. For example, we added a Dialog library on top of it, which adds automatic layout depending of the font size, and allow to design complex resizable windows. It’s nice, but not mandatory, and depending of your needs, you will or will not use it. Up to GTK2, it was the same: you could define pixel perfect size and the toolkit would honor it. Now, it’s a different story: the toolkit can decide if the size you gave programmatically is too small or not. It’s a big change, as you could end up with a deformed GUI and more important, the running program doesn’t reflect the code entered by the programmer. You may wonder why such change has been implemented ? My guess is to force use of layout for everything, so the toolkit can support themes more efficiently and without change from the programmer. It may be true, but it’s one of the first toolkit to do so and forcing the programmer hand isn’t always a great thing, especially when it removes flexibility: you can write a layout engine using pixel perfect sizing, but you can’t do pixel perfect sizing with a layout engine. And for some specific apps, running on proprietary OS, it can be an issue as theming is irrelevant.

PureBasic 5.30 is out !

That’s it, PureBasic 5.30 final version is out ! The beta phase was longer than usual and even if it’s holiday and summer time, it’s not a valid excuse ;). So let’s jump a bit in PureBasic development internals, so you can better understand why.

First of all, the IDE is now fully unicode. It looks like a no brainer as in PureBasic it’s more or less a checkbox to enable to have your program working in unicode, right ? Well, not this time. The (new) IDE is around for almost 10 years now (09/21/2005) and it grew out of proportion by including tons of features. We heavily use raw pointers to process string buffers quickly, and when switching to unicode, all the strings went to UCS-2 (16-bit character). The stuff here is Scintilla is working internally in UTF-8, so all our comparisons between PB strings and Scintilla internal
didn’t work anymore as one side was UTF-8 and the other UCS-2. Also, all the Scintilla commands which needed a buffer as parameter needed to be changed as we mostly used a pointer to a PB string for this. We also had API calls here and here which required some adaptation work, and there was no compiling error or runtime warning, so usually a nice crash was awaiting us when missing one and reaching this code section. To be honest, it was quite a bit depressing to see a solid IDE get back to a shacky shape. This looks brighter now, and the unicode switch allowed us to fix long standing IDE issues which is cool. It was a needed update, especially for the internal debugger which can now handle unicode programs. There is will be probably some quicks left but it should be ironed out in future versions.

We also made important internal compiler changes to enable runtime serialization trough the XML and new JSON lib. It doesn’t seem like a big deal from the user point of view, but actually it is. What is runtime serialization ? In a nutshell: you have your nice linked list of complex structured elements (which can contain dynamic map, array etc.) and you want to save it, so when you quit your program, you wont loose these data. To do that, you had mostly only one way until now: walk the list, write all the fields of your structure to either a file or a database. It can be time consumming to write this, and if you add a new field in your structure, you have to update both your load and save routines. With 5.30 serialization functions, you can do that with only one command. The compiler will flag your structure as a runtime one so it gets all needed information (like fieldname, fieldtype etc.) and build special information table to allow full recursive analysing and dumping of the structure data. We made this change generic in the compiler, so any command (even user libs) can use this feature. To allow this, we needed to support ‘shadow parameters’ in PB function call. Basically, when we declare a command parameter as “Runtime Structure” like in InsertJSONStructure(), one hidden parameter is added to the function call to add more information to the structure. The same is done to when a runtime list, map or array is need like in InsertJSONList/Map/Array(). Adding such a core feature in the compiler is always tricky and can creates new hard to track issues (like stack mis-aligment), and that’s why the IDE didn’t started anymore on some Windows x64 computers.

And finally, a word about LTS (Long Time Support). The 5.30 is the first version which is released in parallel than an LTS version. That means than we had to update our build servers and deployment tools to handle 2 concurrents versions (when building and when releasing). The website also needed some update to handle this properly. It’s small things but it does add-up and it in the end, it takes time to get everything right and properly working.

As a side note, all the doc are now updated to the new format in every languages and I would like to thank Andre and Mesa for the huge work done on this. A programming langage without a good doc is not a good programming langage.

I hope you didn’t fell asleep when reading this, enjoy the new version !

LTS

Since PureBasic 5.20, we introduced a new LTS suffix after the version name. Some of you wondered what it means, and what was the purpose of it. Basically, a LTS (Long Term Support) version will be supported for 2 years trough bug fixes release without any command or compiler modification. If you need stability in your development, you can choose to stick to an LTS version and then when the support is over go to the next one. We will provide clear migration path when going from one LTS version to another one. The advantage here is you don’t have to worry about code adpation when developping and still have stabilisation releases to have a rock solid development enviromnent. The disadvantage is you don’t get the new features of new major release until the next LTS version is out. That’s a balance to find and a decision to make, but now the choice is yours !