PureBasic Forum

New scripting engine - testers wanted!
Page 4 of 10

Author:  srod [ Thu Dec 28, 2017 11:53 pm ]
Post subject:  Re: New scripting engine - testers wanted!

**This post is in reply to a question which now seems to have been removed. :) The question revolved around how we create reusable and independent code modules so I may as well keep the reply here for interested parties. **


Yes you can do that.

Modules can separate code and variables etc. When you use the \Execute() method you are executing 'stand alone' code running in it's own inaccessible module. You would normally use this to call some function or other - but you don't have to.

If you call a function by it's direct name then minScript will search for any function matching the given name, starting first in the 'current module' (assuming the calling code lies within a module) and, if unsuccessful, it will then search all remaining modules. You can specify a particular module when you invoke a function using the 'moduleName:functionName' notation as with :

LET x = module1:foo(a, b, c)

etc. which will of course direct minScript to utilise the particular function implementation in the named module. In this way you can have loads of functions scattered about all sharing the same name and still be able to select the particular one of interest at any given time.

You can define module level global variables by placing the relevant DIM statements outside of a function block. These variables are then available to all functions and methods lying within that same module, but they will be inaccessible to code in a different module etc. Again this means you can reuse variable names and so on. (If you want global variables which are accessible to all modules then use 'script' variables... which I haven't documented yet! You can define these in script -e.g. DIM $x AS INTEGER, or through PB code. Details will be in the docs. Script variables always begin with a $ character.)

You cannot have classes in different modules sharing the same name, however. I did this for several reasons; not the least of which is the fact that I didn't want any confusion between modules when creating class objects. I want every module to be able to access the exact same classes etc.

Creating independent modules is thus quite straight forward. Consider using classes; one class per module etc.

There is also the global class library into which you can place classes (again, I would advise one class per module) and have them available not just to every other module in your code object, but to all independent code objects! This is the way to share code on an application level. Unfortunately, you will have to await the documentation to see how to make use of the global class library - though it is very straight forward. The qdMin test editor is not equipped to make use of it however.

I hope this helps.

Author:  HeX0R [ Fri Dec 29, 2017 12:10 am ]
Post subject:  Re: New scripting engine - testers wanted!

Sorry for deleting my own post, I thought i found it out by myself and didn't want to bother you.

Anyway, there is again some useful new information in your post, thanks!
To be true, it is a little tricky to get used to it with nothing but a (perfectly commented(!) anyway hard to get into detail, at least for me) source code.

Author:  srod [ Fri Dec 29, 2017 12:24 am ]
Post subject:  Re: New scripting engine - testers wanted!

Yes I accept that without documentation, minScript will be awkward to use. Not because the API is difficult in any way, but because anyone looking to integrate minScript in their own app will need not just a familiarity with the minScript API, but also to understand the finer details of the minScript scripting language itself - details such as those outlined in my previous post.

I had decided to concentrate initial testing on the scripts themselves which is why I created the qdMin test editor. When I am happy that I have a useable scripting system then I will concentrate on the documentation. Have to be in the new year though! :)

The place to look for the details of minScript's API is the main 'minScript.pbi' file. This lists the main interface methods comprising the minScript code object as well as a list of public function declarations.

Author:  srod [ Fri Dec 29, 2017 12:54 am ]
Post subject:  Re: New scripting engine - testers wanted!

HeX0R, I forgot to say that when declaring a function you can declare it as being 'private' which means that it will not be available outside of it's parent module even if you try and call it using the moduleName:functionName() construct.

This way you can make modules truly independent and not worry about name clashes etc.

An example of declaring a private function :

Author:  HeX0R [ Fri Dec 29, 2017 11:25 am ]
Post subject:  Re: New scripting engine - testers wanted!

I started to integrate minScript into my existing App to get rid of LUA finally.
Of course I will wait for the upcoming documentation, but anyway I want to make sure that I don't run into a dead-end.

One more question came up:
Runtime functions are global as far as I can see, no problem most of my runtime functions can be used by any of the plugins.
But the plugins offer their own scripts and also are allowed to offer own runtime functions.
This means there is a risk, that two users decided to use the same function name.
Is there a way to circumvent this somehow? Modular runtime procedures would be great, but I guess there is already something implemented I can't see?

Author:  srod [ Fri Dec 29, 2017 12:54 pm ]
Post subject:  Re: New scripting engine - testers wanted!

If you are willing to try and integrate minScript into an existing app (and without documentation! :) ) then I am more than happy to alter minScript in any way you need (within reason of course! :) )

However, let me make sure I understand what you are after.

Are you saying that your users will themselves create runtime functions in Purebasic and that these should be accessible on a per-module basis so that i) no other module will even 'see' another module's run-time functions and, ii) function names can be repeated through different modules?

If so... interesting. That most certainly is not possible at the moment as I didn't envisage a need to have run-time functions which were anything but global throughout an application. I'd have to give that some thought as to the best way of implementing that. I am planning on making some adjustments to run-time functions anyhow so I may be able to kill two birds with one stone there.

There is a massive potential problem, however, with allowing your users to create run-time functions in PB (if I have this right) and that revolves around garbage collection since when using Purebasic to effectively extend minScript, you introduce all sorts of possibilities to create situations which our garbage collector cannot cope with. Such situations can arise if a run-time function creates additional resources such as allocating memory, opening files and so on. minScript will know nothing about such things.

One example is our own 'AllocateMemory()' run-time function.

Any script making use of this particular function which then crashes out because of a run-time error or simply forgets to release the allocated memory, then creates a memory leak.

Take the following script for example :
FUNCTION DoStuff(bufferSize)
  DIM mem, a, b AS INTEGER
  IF bufferSize
    mem = ALLOCATEMEMORY(bufferSize)
    IF mem
      b = 10/a  ;This is a division by zero!

If the above script function was called then a run-time error will be generated at the line which causes a division by zero error. Execution will be halted before the FREEMEMORY() function has been invoked and thus we have a sizeable memory leak! :) minScript will of course release the variables 'bufferSize', 'mem', 'a' and 'b', but it knows nothing about the memory allocated in a chunk of Purebasic code servicing the AllocateMemory() run-time function. On top of this, some 'sloppy' coders may even forget to use FREEMEMORY() in an otherwise perfectly running script. :) Also, the design of the script may make it so that the appropriate FREEMEMORY() was called in a separate function and so on which of course makes automatic garbage collection of such dynamically allocated resources almost impossible. Almost, but not quite! :wink:

Now whilst there is a mechanism in place to clear up any such memory leaks, it really isn't a good option for you to expect your users to actually go down that 'non-trivial' path. You basically have to create what I have termed a garbage collector extension which is an advanced topic. I have created several for use in minScript (a simple one for AllocateMemory(), more complicated ones for arrays and hashmaps and a hugely complicated one for classes! :) ) Instead, you yourself (with my help in the first instance until the docs are complete) should perhaps create all such run-time functions which need to create dynamic resources (and thus potential memory leaks) and warn your users to only add run-time functions which do not themselves create additional dynamic resources unless they are prepared to create garbage collector extensions.

Mind you, as stated, the GC extension we created for AllocateMemory() was simple enough so perhaps I am being a bit pessimistic here.

Again, there was never any suggestion in my thinking that a user of a user of minScript would ever need to create a run-time function and so the thorny issue of garbage collection would only ever concern myself and immediate users of minScript such as yourself - not your users! :) Even then, garbage collection is only an issue if you create run-time functions which may misbehave in this way.

Honestly, garbage collection (especially with class objects) nearly brought minScript crashing down on my own head... and it may well do yet! :)

Author:  HeX0R [ Fri Dec 29, 2017 2:04 pm ]
Post subject:  Re: New scripting engine - testers wanted!

Let's try to explain it more detailed (I thought quite a while about the concept):

I have a main program, which loads plugin.dlls. Those can be created by end customers, or lets name them end customer developers (there will be a SDK included).
The main program also contains minScript, and ONLY the main program, the plugins will not have direct access to it.
Anyway, the plugin.dll's are allowed to be extended with script files, which could also create end customers of end customer developers.

With LUA I did it like this:
main program calls a procedure of the plugin.dll, which will tell the main program which "run time" procedures all of the script files (of this plugin) are allowed to use.
It will also tell the main program the function addresses (inside the dll) which handle the runtime procedures.

main program will add those run time procedures and as soon as a script will call it, the main program will directly root it to the dll function.
With LUA this worked, there I have been able to just use different instances and nothing will disturb each other.

Those script files will more or less be used to interact with dialoges, so the garbage collection might be no big deal here, but I understand your concerns.
I think I just need to point out what to be taken into account, anything else is not in my own responsibility.

Author:  srod [ Fri Dec 29, 2017 2:19 pm ]
Post subject:  Re: New scripting engine - testers wanted!

@HeX0R : I have given the module level run-time functions a little thought over a very late breakfast and have decided that it's a no go - it won't work! :) It will work, however, in a slightly modified form which I think is actually what you are after.

You need to understand how minScript works.

A host application creates one, or more, 'minScriptCode' objects. These objects are self-contained and you add as many script modules into one of these objects as you wish. You can then execute code within any one of these objects. If you have two minScriptCode objects kicking around then they are completely separate. You can have identically named modules in these two separate objects. No script in one module can access code in the other and so on.

Now, it really does not make sense to allow run-time functions to be added at the module level. It does make some sense, however, to allow run-time functions to be added at the minScriptCode object level. This way, scripts in each individual code object can access just it's own run-time functions as well as the global ones minScript already supports. Your application plug-ins would then simply be given their own instance of a minScriptCode object and voila, support for their own run-time functions. This would mean that each plugin was a separate entity not able to share script code other than through the 'global class library'.

Please let me know if this would work for you as it would be quite a bit of work for me to implement. Could be a good feature though.

As a point of interest, you can visualise a minScript application in the following way.

  • code object 1
    • Module 1
      • module variables
      • module constants
      • classes
      • functions/methods -> local variables etc.
    • Module 2
      • module variables
      • module constants
      • classes
      • functions/methods -> local variables etc.
    • Module 3
    • etc.
  • code object 2
    • Module 1
      • module variables
      • module constants
      • classes
      • functions/methods -> local variables etc.
    • etc
  • etc.

Author:  srod [ Fri Dec 29, 2017 2:25 pm ]
Post subject:  Re: New scripting engine - testers wanted!

Right, with your system, when you say a 'script calls a run-time function' are you talking about a script which resides within the plugin dll or can scripts from the main application then call these plug-in run-time functions?

Still not absolutely clear 'what is calling what' here?

Does the main app invoke scripts sitting in a plug-in which then can invoke their own run-time functions or do scripts running within the main app then invoke other scripts or run-time functions residing within the plugin?

Author:  HeX0R [ Fri Dec 29, 2017 7:01 pm ]
Post subject:  Re: New scripting engine - testers wanted!

HaHa, yeah, this gets quite complicated I know :mrgreen:

The option to create different instances of minScript would be perfect, but as far as I saw the source code I fear this would be horrible to add.
I wouldn't do that for just one dumbass if I would be you.

Anyhow, I think this discussion gets boring for other visitors, if you are still interested to understand my weird concept, I could try to explain it via PM.

Author:  srod [ Fri Dec 29, 2017 11:55 pm ]
Post subject:  Re: New scripting engine - testers wanted!

I am sure we can configure minScript to work for you without too much hassle.

pm is fine. I will be away for the new year so it will be a few days before I can reply.

Author:  HeX0R [ Sat Dec 30, 2017 8:37 pm ]
Post subject:  Re: New scripting engine - testers wanted!

Just keep it as is, I think I found a solution!

But more instances would have some charme also, just in case you have too much time :)

Author:  srod [ Mon Jan 01, 2018 11:20 am ]
Post subject:  Re: New scripting engine - testers wanted!

As I say, I want to update the run-time functions mechanism anyhow to include an ability for hosts to effectively add new keywords so I will look to extend the facility at the code-object level at the same time.

Author:  ChrisR [ Fri Jan 05, 2018 1:20 pm ]
Post subject:  Re: New scripting engine - testers wanted!

If time allow, count me in the hearty souls to help you test this promising swine :wink:

srod wrote:
As I say, I want to update the run-time functions mechanism anyhow to include an ability for hosts to effectively add new keywords so I will look to extend the facility at the code-object level at the same time.


Author:  srod [ Fri Jan 05, 2018 7:49 pm ]
Post subject:  Re: New scripting engine - testers wanted!

Hi ChrisR,

I will hopefully upload a new version over the weekend so feel free to get stuck in at that point. :) The more the merrier!

I am just finishing off bitwise operators and hexadecimal support and at that point I think we have all the features suitable for a proper first release. Still lots of testing to do mind before that. Oh yes and some documentation probably wouldn't go amiss!

Page 4 of 10 All times are UTC + 1 hour
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group