eScript - beta testers requested!

[srod]

Hi,

I am finally putting the finishing touches to the first beta version of my eScript cross-platform scripting framework and am seeking a couple of erstwhile testers to assist in finding bugs and the like etc. The said testers will be given a free license for the complete source code, valid for the entire lifetime of the product.

For more info on eScript please see the eScript section of the nxSoftware site as well as the relevant nxSoftware blog entries.

As a rough outline; eScript allows client apps to offer up scripting facilities (or just some kind of automation etc.) to their end-users. However, eScript is a lot more than this in that it is, for example, a relatively simple process to create a full interpreted language based upon eScript if so desired.

eScript consists of the following basic elements :

• a stack-based virtual machine (VM)
• an assembler / linker
• an 'execution layer' which executes compiled object code against the VM.

In to this you simply slot any compiler which targets the eScript assembler/linker (unless you want to create your own assembler of course!) eScript of course comes with it's own compiler which I would envisage that the majority of non-suicidal consumers will make use of! This compiler is for a version of the Basic language which I have called eScriptBasic.

Because of this architecture, scripts would appear to run very fast indeed (with the PB debugger turned off). For example, a loop of 1 million repeated additions takes about 1.1 seconds to run on my slow old laptop! This was a little faster than an identical VBScript script (though I did run it in a web page and am not including the time taken to compile the script in eScript). A freely available Basic interpreter I downloaded ran the same code in more than 40 seconds! The compiler does some peep-hole optimising which of course helps and I can add more at any time.

It goes without saying that eScript is very simple to use. Compiling and executing scripts is very painless.

This has been a huge amount of work done mostly in my spare time. Off and on, a couple of years in all including several failed attempts to get this project up and rolling.

Anyone interested, please pm me. I would hopefully be able to send eScript out in the next couple of days or so. Only a couple of testers at this stage and am hoping for someone who can put eScript to work almost immediately in some kind of non-trivial app. I will create a quite substantial test program over the weekend to supplement the various simple test scripts I already have. I think I'll create a good old fashioned console mode basic interpreter (you know, using line numbers and the like!) This should give me a chance to break the entire system as well as providing a nice platform to test little scripts without bothering integrating them into apps etc. Testers would not need to concern themselves with testing ASM scripts; just eScriptBasic scripts.

There are no docs at present, but the entire system is detailed within 4 text files. Testers wouldn't really need to concern themselves with these (except for the one listing the eScriptBasic keywords etc.) however unless curiosity gets the better of them!

Thanks.

[srod]

Have enough souls willing to test eScript now and so am not looking for anyone else at this time.

Thanks.

[Zach]

Hi,

Was wondering when you might expect this project to go Retail, and at what price point it might be?

I'm definitely interested in a scripting language I can just plug-in and use without any hassle, to be used with games.

[srod]

Yes we are on the 2nd beta test version right now and whilst the testing proceeds I am busy putting the final touches to the system (just some improved debugging facilities remain now + the standard runtime to sort out).

As it stands I would say that the system is very useable and really quite powerful and easy to use. This first version will lack support for arrays, structures and classes, but they are coming and even without these things it is still a very complete scripting engine. Even API support is coming (Windows versions at least).

When will it be available; very soon. Hopefully, a couple of weeks at the longest depending on whether I wait on the documentation etc.

**EDIT : sorry, you asked about the pricing... unsure at the moment. The cheapest option will be eScript in shared library form (DLL on Windows; .SO on MacOSX etc.) A shared library license will give access to all versions; Windows + OSX etc. The most expensive option will be for the full eScript source; though I am reluctant to make this available considering the work that has gone into it. The source-code, if made available, will not be cheap.

Price of the shared libs... as reasonable as I can make it. There may or may not be charges for future upgrades; I haven't decided yet, though probably not.

[Shield]

I'd be interested in getting more information about the language's syntax and general structure of scripts.

[srod]

Well, eScript allows for the use of any appropriate compiler; the only stipulation being that it targets the eScript assembler of course.

eScript of course comes with a default compiler for what I have called the eScriptBasic language; a version of Basic which I am comfortable with.

Incidentally, whilst eScript has a compiler (with some peep-hole optimising), the system is still interpretive. Source code gets compiled right down to 'object code' which goes far beyond simple tokens etc. Our object code resembles x86 machine code (though the underlying virtual-machine is primarily stack-based like Java) and compiled code seems to run very fast when executed by the runtime engine.

To get an idea of the syntax; here's a test eScriptBasic script which calculates the length of a string (eScript itself offers a simple run-time function for determining such lengths, but the script served to test various facets of the system) :

Code: Select all

FUNCTION EXPORT LenString(str AS STRING) AS INTEGER
  DIM numChars AS INTEGER
  DIM ptrChar AS CHARACTERPTR
  DIM ptrInt AS INTEGERPTR  ;This is used to get the address of the string's character buffer. @str returns the address of the variable.
  IF str
    ptrInt = @str
    ptrCHar = *ptrInt ;Dereferencing this gets the address of the character buffer.
    IF ptrChar
      WHILE *ptrChar  ;Note the dereferencing operation here to get at the underlying character!
        numChars = numChars + 1
        ptrChar = ptrChar + SizeOf(#CHARACTER)
      WEND
    ENDIF
  ENDIF
  RETURN numChars
ENDFUNCTION

Note the use of c-style pointers (and c-style dereferencing); I prefer these over the use of a BYREF clause etc. With these pointers you can actually access variables/memory within the host application etc.

As for the structure of scripts; note that you can include other source scripts (similar to Purebasic's XIncludeFile etc.) You can also link in (statically) other compiled object files and so you can build a library of such files etc.

You can also use inline ASM if you wish.

I hope this gives you some idea.

[Shield]

Thanks, sounds very interesting.

[Zach]

Sounds really interesting (and a bit above my head)..

I don't mind a DLL version, I suppose. But I am wondering, if there will be any chance of a PB Userlib version? I don't think I would ever need the source code, so depending on how it works and how easy it is (for me personally) to get the hang of and use what I want, as long as the "cheap" version is under $100 USD then I will definitely consider purchasing a license for a DLL version.

I do like the idea of it being compiled/interpreted. I'm assuming we have the option either way? i.e we can load pre-compiled scripts with the library at run-time, or we can load uncompiled scripts to be interpreted?

I think this will be important for me, because of how I intend/hope to do things with any game engines I develop. I definitely would like to give the user the ability to Modify the engine, or certain aspects of a game, by editing the script files (or replacing with their own). But then there are also cases where I would not want to allow users to do that, and choose to distribute pre-compiled scripts; or release Build tools that handle custom content creation and that are capable of comping scripts..

That sort of stuff.

Hypothetical:

Let's say I wish to create a "game engine", similar in flexibility to the Engines that a lot of games come with.. i.e I hard code very few components, like the Rendering, Sound, Input routines and such.. Is it possible to do everything else in eScript, and maintain a high performance level? i.e Construction the game logic itself, Artificial Intelligence Scripts, Event Handlers / Dispatchers, Character Combat & World Mechanics, scripted Events, etc.. That would be really cool from a flexibility standpoint.

Based on your previous description, and talk of program integration (i.e you needed this for your reports suite) it sounds like all this will be possible? Would it be painless to Interface the script with a core program in this manner? (i.e The Script(s) and Core Program they are integrated with can talk to each other painlessly, pass / manipulate each other's data (variables, arrays, tables and such) ).

[srod]

Hi Zach,

please note that it is not a case that you can choose between compiling or interpreting a script - you have to do both!

You first compile a source script down to eScript's version of machine code. eScript will then, when you ask it to, interpret this machine code. You will find that this is how the 'fastest' interpreted languages operate. 'Just in time compilation aside', this is how Java and .Net essentially operate.

As to all of your 'requirements' as listed above; it is a resounding yes to all points.

You can ship scripts for users to edit and compile etc. (It is easy for your application to hide the compilation step. As far as your users need be concerned, scripts get executed immediately etc.) Or you can send out precompiled scripts, i.e. send out compiled object files and your users will be able to execute them via the eScript dll.

I will make 2 points, however; first, this first version of eScript will not have support for arrays and structures. As soon as this first version is released, however, I will be working on arrays and structures. Second, whilst my tests confirm that eScript object code executes very fast (with the PB debuggers turned off), I cannot promise at this stage that they would run fast enough to take over some of the 'heavier' elements of your game engines. At the end of the day, eScript code is still being interpreted and there really are some uses to which you shouldn't be putting an interpreter to. You are probably better qualified than I to know exactly what parts of your engines can be converted to script etc?

It is difficult at this relatively early stage to say just how fast compiled eScript code will execute? Certainly, my initial tests look promising in this regards.

In terms of interaction between the host app and scripts, yes, we have full 2-way integration. Your host app can invoke any scripted function. Scripted functions can then call back into your host app by way of run-time functions. Scripts can also directly access variables in your host application via pointers etc. There are also facilities for setting up script callbacks etc. Later on I will look to add a facility for scripts breaking into multiple threads - that is, a script can create threads of it's own; each of which are executing scripted functions from the same script. I also intend allowing scripts to call directly into DLL's. E.g. to call directly into Windows DLL's such as User32 etc.

As for a price for the DLL version... have no fear, it will be way below $100; though I will not complain if you decide to give $100!

I will not create PB userlibs though. Honestly; I find them a pain in the ar*e!

[Zach]

Thanks for taking the time to answer, Srod.

It really does sound like it will be a good tool everyone will/should use once it has matured! We have badly needed a scripting language with tight PB integration for a long time, since a lot of users seem to prefer like me, to not have to mess with messy wrappers and interfaces and all that stuff some of us know so little about.

I take it what you mean by you can't choose between compiling/interpretting, that the eScript works by accessing the compiled scripts only, at run-time. I think I know what you mean there, (I just have to add a compilation step in any user modification tools I release, as well as my own internal development tools) and it does sound fine to me

[srod]

Hi,

when you ship eScript (whether in DLL form or source form) as part of your app you get full compilation and execution facilities all thrown in.

Perhaps it will help if I show some PB code which I am using to compile and execute one of my test scripts.

Notice with the following code that aside from loading the source file from disk, we do not then perform any additional disk operations. That is, we do not compile and then save the compiled object file to disk etc. before execution. (You are of course free to do this if you wish). Instead we compile it and keep the object code in memory and then execute it from there.

Note also that I have removed some lines from the following code; lines giving detailed error messages etc. if the script fails to compile and so on. I thought they would just get in the way here.

Code: Select all

;Set the following constant to point to the main eScript source folder.
  #eScript_HOME = "c:/Pure basic - 4.5/eScript/Source/"  
  XIncludeFile #eScript_HOME + "eScriptHeaderFile.pbi"

  ;Always check that eScript initialised properly.
    If eScript_INITOK = #False
      MessageRequester("eScript problem!", "eScript failed to initialise." + #CRLF$ + #CRLF$ + "Check that the library files are installed correctly.")
      End
    EndIf

  eScript.eScriptExecutionContext = eScript_NewExecutionContext()
  If eScript
    result = eScript\CompileFromFile("linkObjectFile.eScript")
    If result = #eScript_OKAY
      Debug "Compilation successful!"
      ;We now execute the object file rather than save it for later use.
      ;We call a function which itself calls a function which is imported from the 'strings.eso' object file.
        res.eScriptVariant ;This quantity will hold any return value.
        result = eScript\InvokeExportedFunction("testLink", 0, 0, res)
        If result = #eScript_OKAY
          Debug "Execution completed successfully."
          Debug "The result of invoking the 'testLink' function is '" + eScript_VarGetString(res) + "'"
        Else
          Debug "Execution failed :"
          Debug "=================="
        EndIf      
    Else
      Debug "Compilation failed."
      Debug "==================="
    EndIf
    eScript\Destroy()
  Else
    Debug "Could not create an Execution Context!"
  EndIf

[Zach]

I thought I understood but I guess I didn't, since you made another explanation!

Maybe this can help...

I can either

A)Ship product with my eScript files, which then have to be loaded and compiled during run-time and can be executed at that point.
or
B) Ship product with pre-compiled eScript object files which I have generated ahead of time, and load, then run those files.
or
C) I could do both if I wanted to.


???
I guess that is what I am asking.
(A & B are important choices I want to make depending on the game/application. C is just a miscellaneous assumption of what I can/can't do but is not really relevant to me.)

[srod]

No I don't think I have contradicted myself - though I will check.

You can do any and all of A), B) and C), which if I hadn't made clear then I apologise.

With eScript you first create a script (or scripts if you are linking them together). Of course only a text editor is needed to create a script.

After this you would compile the script. This results in an executable object file being produced which will sit in memory.

From here, save the object file to disk if you wish for later use, or, execute it.

At this point it should be clear how your A), B) and C) requirements are all met.

A) Ship the script source files if you wish, allowing your users to edit the source if appropriate. The PB code I exhibited in my last post shows how easy it is to compile and, if you wish, execute the code.

B) Compile the source scripts yourself if you wish and just ship the compiled object files with your application. Your app can easily load up these pre-compiled files and then execute the code within in a very similar way that I showed with my PB example.

C) Yep, do both A) and B) if you wish.

D) Let your users create their own scripts from scratch if it is appropriate.

I don't want to make things sound more complex than they are because the whole system is really quite easy to use... at least inmo.

Right, if this doesn't answer your questions then I really have gotten the wrong end of the stick.

[Zach]

No, you have answered my question Srod.

All is well! I should be able to do exactly what I want!

Just one final thing. You have internal commands for saving the compiled scripts from memory to the executable object files, and also for loading those files from disk, yes?
I am assuming the answer is yes, you do.

But otherwise, yes it will work exactly how I want it to

Thank you again, for taking the time to explain.

[srod]

Absolutely yes.

The whole business of dealing with individual scripts revolves around what I have called an 'eScriptExecutionContext' object, an example of which you can see in the PB code I listed above. This is just a simple interface.

There are 2 ways of working with this object.

a) Load a script into it and compile it (as with the demo PB code above).

or

b) load a precompiled object file into it.

Either way, the result of A) or B) is that the eScriptExecutionContext object will have, sitting inside it, a compiled object file.

From here you can execute any part of that code or save it back out to disk and so on.

Incidentally, you don't have to keep your source files on disk; they can sit in memory. You can compile scripts from memory.