LUA?

[jack]

did you know about LuaJIT — a Just-In-Time Compiler for Lua ?
http://luajit.org/luajit.html

[idle]

I wouldn't think so if it only created a dll and it was statically linked into the program.
A number of years ago I made an experimental pb jit compiler with llvm in pb so its doable it was very limited but as an exercise it worked.

I'm just trying to visualise how that would operate. Would that require the PB compiler, and say, GCC or appropriate, installing on the executing machine — i.e. the machine with the runtime compiled PB executable?

And just as a wild idea, is the following sort of thing conceivable, or intended along these lines?

Code: Select all

mycode.s = "OpenConsole()" + #CRLF$
mycode.s + "PrintN(" + chr(34) + "Hello world" + chr(34)) + #CRLF$
mycode.s + "Delay(1000)" + #CRLF$

Jit(mycode.s, etc. etc.)

Therefore, I've created a PB executable and the executable itself is responsible for building the code dynamically. Then having constructed the code, it can call JIT.

This was the gist of using llvm to jit code however this was just a test of using llvm before I wrote the PB front end parser
the code is run with LLVMRunFunction(EE, FibF, 1, @Arg)

Code: Select all

;  Fibonacci function executed by JIT. 
;  Origonal c example from llvm fibonaci example 
;  
;  Author idle - a hack To see how llvm works 
;  Shows How to create a llvm module in memory
;  Build a Function    
;  Create a Function optimizer 
;  Dump the module To byte code 
;  Load the byte code file and produce native object code 
;  
;  The fib function calulates the number of factors    
;   
;  Procedure fib(argx.i)
;  If argx<=2
;     ProcedureReturn 1
;   Else  
;    ProcedureReturn fib(argx-1)+fib(argx-2)
;  EndIf
;  EndProcedure
;
; Once we have this, we compile the module via JIT, then execute the `fib'
; function And Return result To a driver, i.e. To a "host program".
;===----------------------------------------------------------------------===''

IncludeFile "llvm-2.9.pbi"

;dirty helper to read var arg strings 
Procedure.s ReadVarString(*chr)
 Protected strout.s,tmp.s 
 If *chr
 Repeat
    tmp = PeekS(*chr+ct)
    If tmp <> ""
      strout+tmp+#LF$ 
    Else 
      Break 
    EndIf  
    ct+Len(tmp)+1
  ForEver
 EndIf
ProcedureReturn strout
EndProcedure  

Procedure fibPB(argx.i)
  If argx<=2
    ProcedureReturn 1
  Else  
   ProcedureReturn fibPB(argx-1)+fibPB(argx-2)
  EndIf
EndProcedure

Procedure fibPB2(argx.i)
  Protected a,b,c
  If argx <= 2
    ProcedureReturn 1
  Else 
    a = 1
    b = 1
    While argx > 2
      c = a + b;
      b = a;
      a = c;
      argx-1;
    Wend
    ProcedureReturn c
 EndIf    
EndProcedure

Procedure CreateFibFunction(M.i,Context.i)     
    Protected FunctionReturnType
    Protected IntType 
    
    ;Procedure.i fib(x.i)
    ;function only uses and int type, create an int type.
    intType = LLVMInt32Type()
    
    ;Function returns an int And takes an int as the only parameter.
    FunctionReturnType = LLVMFunctionType(IntType,@IntType,1,0)
    
    ;Create the fib function definition And insert it into the module M. 
    FibF = LLVMAddFunction(M, "fib",FunctionReturnType)
    ;Set the function call convention to FastCall so it can utilize tail call 
    LLVMSetFunctionCallConv(FibF,#LLVMFastCallConv)
    
    
    ;Add a basic block To the function.
     BB = LLVMAppendBasicBlockInContext(Context, FibF, "EntryBlock")
        
    ;Make the Constants returns a pointer to the constant.
    ; fib(argx->>"1"<<)+fib(argx->>"2"<<)
    One = LLVMConstInt(IntType, 1, 0)
    Two = LLVMConstInt(IntType, 2, 0)
    
    ;Get a pointer to the ArgX.i and add to function...
    ArgX = LLVMGetFirstParam(FibF) ; Get the arg.
    LLVMSetValueName(ArgX, "ArgX") ; Give it a symbolic name.
   
    ;When using the C bindings, you have to use a BuilderRef To create any instructions.
    Builder = LLVMCreateBuilderInContext(Context)
    ;Set the builder to insert at the first BasicBlock.
    LLVMPositionBuilderAtEnd(Builder, BB)
    
    ;Create and build the conditional branch "if argX <= 2" 
    CondInst = LLVMBuildICmp(Builder, #LLVMIntSLE, ArgX, Two, "cond")
    
    ;Create two blocks for the Branches  If argx<=2 : RetBB : else : RecurseBB   
    RetBB = LLVMAppendBasicBlockInContext(Context, FibF, "return")
    RecurseBB = LLVMAppendBasicBlockInContext(Context, FibF, "recurse")
    
    LLVMBuildCondBr(Builder, CondInst, RetBB, RecurseBB)
    
   ;Create: ProcedureReturn 1
    LLVMPositionBuilderAtEnd(Builder, RetBB)
    LLVMBuildRet(Builder,One)
    
    ;Create fib(x-1)
    LLVMPositionBuilderAtEnd(Builder, RecurseBB)
    Subtr = LLVMBuildSub(Builder, ArgX, One, "arg")
    CallFibX1 = LLVMBuildCall(Builder, FibF, @Subtr, 1, "fibx1")
    ;LLVMSetTailCall(CallFibX1, 1)  ;let the optimzer deal with it
    
    ;Create fib(x-2)
    Subtr = LLVMBuildSub(Builder, ArgX, Two, "arg")
    CallFibX2 = LLVMBuildCall(Builder, FibF, @Subtr, 1, "fibx2")
    ;LLVMSetTailCall(CallFibX2, 1)  ;let the optimzer deal with it
    
    ;fib(x-1)+fib(x-2)
    Sum = LLVMBuildAdd(Builder, CallFibX1, CallFibX2, "addresult")
        
    ;Create the return instruction And add it To the basic block
    LLVMBuildRet(Builder, Sum)
    
    ;Don't forget to free the builder.
    LLVMDisposeBuilder(Builder)
    
    ;Return the function ref
    ProcedureReturn FibF
    
  EndProcedure

Procedure main(n)
   Protected errStr.s
   Protected EE.i
  
    LLVMLinkInJIT()
    Context = LLVMGetGlobalContext()

    ;Create a module for our function.
    M = LLVMModuleCreateWithNameInContext("test fib", Context)
    
    ;Create a pass manager optimizer for our funtions 
    PM =  LLVMCreateFunctionPassManagerForModule(M);
    ;add tail call elimination parse to it 
    LLVMAddTailCallEliminationPass(PM);
    LLVMInitializeFunctionPassManager(PM);
    
    ;Create the Fib function in the Module and Context
    FibF = CreateFibFunction(M, Context)
    ;Run function optimizer  
    If LLVMRunFunctionPassManager(PM,FibF);
        PrintN("Optimized Function")
    EndIf 
    LLVMFinalizeFunctionPassManager(PM)
    LLVMDisposePassManager(PM);
      
    ;Create JIT engine execution
   If  LLVMCreateExecutionEngineForModule(@EE, M, pStr)
     PrintN(ReadVarString(pStr))
     LLVMDisposeMessage(pStr)
     Input()
     End 
  EndIf
  
  ;read the target data 
  TD =LLVMGetExecutionEngineTargetData(EE);
  PrintN(ReadVarString(LLVMCopyStringRepOfTargetData(TD)))
   ;LLVMDisposeTargetData(TD); locks up for some reason 
  
  ;Verify module 
  PrintN("verifying module")
  If  LLVMVerifyModule(M, #LLVMPrintMessageAction,pStr)
     PrintN("Error constructing function!")
     PrintN(ReadVarString(pstr))
     LLVMDisposeMessage(pStr)
     Input()
     End
  EndIf

  PrintN("OK")
  PrintN("Test module succeded")
  PrintN("Run fibonacci(" + Str(n) + ") in JIT");

  ;Call the Fibonacci function With argument n:
  Arg = LLVMCreateGenericValueOfInt(LLVMInt32Type(), n, 1)
  
  st = ElapsedMilliseconds()
  GV = LLVMRunFunction(EE, FibF, 1, @Arg)
  et = ElapsedMilliseconds() 
  ;convert the result
  PrintN("Jit Result: ")
  PrintN(Str(LLVMGenericValueToInt(GV, 1)))
  PrintN("took " + Str(et-st))
  
  ;call same in PB 
   st = ElapsedMilliseconds()
   res = fibpb(n)
   et = ElapsedMilliseconds() 
  PrintN("PB Result: ")
  PrintN(Str(res))
  PrintN("took " + Str(et-st))
  
   ;call same in PB 
   st = ElapsedMilliseconds()
   res2 = fibpb2(n)
   et = ElapsedMilliseconds() 
  PrintN("PB Result 2: ")
  PrintN(Str(res2))
  PrintN("took " + Str(et-st))
    
  ;Dump the jit Bitcode to file 
  LLVMWriteBitcodeToFile(M,"fib.bc");
  PrintN("")
  PrintN("Dumped bytecode to 'fib.bc'") 
  
  ;clean up
  LLVMDisposeGenericValue(Arg)
  LLVMDisposeGenericValue(GV)
  LLVMDisposeExecutionEngine(EE)
 
  
EndProcedure 

OpenConsole()
PrintN("enter the number of factors < 35")
n = Val(Input())
main(n)
PrintN("")
PrintN("Press enter to quit") 
Input()
CloseConsole()

[PBJim]

This was the gist of using llvm to jit code however this was just a test of using llvm before I wrote the PB front end parser
the code is run with LLVMRunFunction(EE, FibF, 1, @Arg)

Just trying to see where it compiles — does your IncludeFile "llvm-2.9.pbi" invoke PB's compiler?

[idle]

No it compiles it with llvm, a lot has changed since llvm2.8 and it would be quite an effort to update and get going. I only got it working on Linux as windows llvm wouldn't compile at the time that was back in 2011 with pb 4.62

[benubi]

@Jack hey, thanks for the tips & codes. I may give it a try, even though I gave up a little while ago with those static libraries.

The greatest problem for me personally is the platform independence and availability for both (static and dynamic) library types using the same version, you have to install compilers and know how to use them on all systems, and my C expertise is very low concerning that.

Compatibility problems between 5.1 and 5.4- I prefer 5.4 a lot because of the now adopted bit operators. Compare that to the way even LuaJIT handles it: it uses an old 3rd party lib, but IDK if it does some tricky optimization on that. Last time I checked LuaJIT wasn't so widely available for all platforms which I found problematic back then (this seems to have changed a lot).

Also 5.4 has the package model (include 'xxxx' command mechanism), that doesn't work well with retro-compatibility, and the big "batteries" I entirely wrote for 5.4 won't work without package model. What man would like his package gone?

On raspberry I was unable to find a suitable .so file (apt get installed lua 5.1 for what ever reason), while on Windows the static files even renamed would always miss one or two unresolved things, using gcc, visual studio, and a third one. On the debian VM the test program crashed using the .so and it could be because of anything, as I only found & tried .so and linux port when the source code grew +30 files and 100,000's of lines long :/

It's a little problematic on how to implement it.

This is how I'd like to have it:

UseLuaScripting("engine3d.dll") <- have it totally backup-ed in the 3d engine (it's so big already nobody will notice ), so lua can use CEGUI and other things from inside, and if you don't do 3D, you just use a regular dll, and without parameter the static version.

The second problem is then how to put it in the documentation, because PB doesn'T "own" all those lua_xyz procedures, and they differ also in style etc. and keep the library up-to-date and bug free? should all lua_xyz functions be implemented, and all the standard libs etc. this may be a lot of work, beside the esthetics and the legal questions.

lua_createtable(*Stack) -> CreateLuaTable(#Lua)
lua_tostring(*stack,pos) -> LuaToString(#Lua, pos) ; ???
etc.

There are a lot of things to consider to make it nice and pretty. (IMO)