Page 2 of 4

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Tue Oct 10, 2017 8:15 pm
by nco2k
does this work?

Code: Select all

Global Test.s
ProcedureDLL.s Dummy()
  Protected Buffer.s = "bug"
  Test = Buffer
  ProcedureReturn Test
EndProcedure
c ya,
nco2k

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Tue Oct 10, 2017 8:41 pm
by falsam
nco2k wrote:does this work?
No. Thanks for your help :wink:
And I would prefer the userlib to work in the same way as this code.

Code: Select all

ProcedureDLL.s Dummy()
  Protected Buffer.s = "bug"
  ProcedureReturn Buffer
EndProcedure  

Debug "no " + Dummy()
Debug wrote:no bug

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Tue Oct 10, 2017 9:54 pm
by Lunasole
falsam wrote: @Lunasole : That's a very good idea. But it's already coded. You will not get your tracking ticket :wink:
Nice then ^^
I've finally checked this stuff in work and seen.
Btw principle you used to make it is really cool itself, like kind of "nice hack"

falsam wrote:I need your help.
I create a userlib with this code

Code: Select all

ProcedureDLL.s Dummy()...
And that's really interesting. Played with it a bit I can't say WTF it happens, here is just comparison of assembly generated for 2 variants:

Code: Select all

; here no is missing from result
Define C$ = "no" + Dummy()

Code: Select all

; here all is OK
Define C$ = "no"
C$ + Dummy()
-----------------------------

Code: Select all

 PUSH dword [_PB_StringBasePosition]
 MOV edx,_S1
 PUSH edx
 CALL _SYS_CopyString@4
 MOV edx,[_PB_StringBasePosition]
 PUSH edx
 PUSH edx
 CALL PB_Dummy
 POP eax
 PUSH dword v_C$
 CALL _SYS_AllocateString4@8

Code: Select all

 MOV edx,_S1
 LEA ecx,[v_C$]
 CALL SYS_FastAllocateStringFree
 MOV edx,dword [v_C$]
 PUSH dword [_PB_StringBasePosition]
 PUSH edx
 CALL _SYS_CopyString@4
 MOV edx,[_PB_StringBasePosition]
 PUSH edx
 PUSH edx
 CALL PB_Dummy
 POP eax
 PUSH dword v_C$
 CALL _SYS_AllocateString4@8
Oblious that "no" literal is present in output assembly in both cases. Noticeable difference is that 'wrong' variant doesn't use
_SYS_FastAllocateStringFree. In other parts (except quoted fragments) variants looking identical.

Maybe it is PB bug? Sometimes I'm still encountering a strange floating bug with string literals in large projects (some literals going missing from resulting exe). Maybe this one is of that kind.
Generally I don't know, maybe someone of true asm experts can tell more about fragments above ^^

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Tue Oct 10, 2017 10:08 pm
by Lunasole
As a workaround returning pointer instead of string is OK of course.

Also @falsam, have you tried using ProcedureCLL instead of ProcedureDLL? I was too lazy to modify a tool and try it ^^
UPDATE: already modified, also doesn't work

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Tue Oct 10, 2017 10:45 pm
by GPI
Code with userlib

Code: Select all

Define c$= "no"+Dummy()
Debug c$

Code: Select all

; Define c$= "no"+Dummy()
  PUSH   dword [_PB_StringBasePosition]
  MOV    edx,_S1
  PUSH   edx
  CALL  _SYS_CopyString@4
  MOV    edx,[_PB_StringBasePosition]
  PUSH   edx
  PUSH   edx
  CALL   PB_Dummy
  POP    eax
  PUSH   dword v_c$
  CALL  _SYS_AllocateString4@8
code without userlib

Code: Select all

Procedure.s dummy2()
    Protected Buffer.s = "bug"
  ProcedureReturn Buffer
EndProcedure
Define c$= "no"+Dummy2()
Debug c$

Code: Select all

;; Define c$= "no"+Dummy2()
  PUSH   dword [_PB_StringBasePosition]
  MOV    edx,_S2
  PUSH   edx
  CALL  _SYS_CopyString@4
  PUSH   dword [_PB_StringBasePosition]
  CALL  _Procedure0
  POP    eax
  ADD    dword [_PB_StringBasePosition],2
  PUSH   dword v_c$
  CALL  _SYS_AllocateString4@8
I don't think this is a bug in PB. PB use a diffrent string handling for userlibs.
It seems really dangerous to use any string-function...

edit: Copied wrong code, but the result are the same ;) There are diffrences

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Tue Oct 10, 2017 10:47 pm
by falsam
@Lunasole : Thank you for this detailed analysis.

If I test the library with this code, the result is good.

Code: Select all

Define Result$ = Dummy()
Debug "no " + Result$
I hope I should not touch the ASM file because I don't know how to use the assembler.

The code being simple, I was able to create the library with Tailbite. If I then test with this code, I have a good result!

Code: Select all

Debug "no " + Dummy()
Debug wrote:no bug

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Tue Oct 10, 2017 10:53 pm
by GPI
falsam wrote:The code being simple, I was able to create the library with Tailbite.
Does tailbite generate a asm-output?

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Tue Oct 10, 2017 11:41 pm
by falsam
GPI wrote:Does tailbite generate a asm-output?
Yes

ASM Code Extract with Tailbite

Code: Select all

_EndProcedure1:
  PUSH   dword [esp]
  CALL  _SYS_FreeString@4
  ADD    esp,4
  RET + 4
With MLF

Code: Select all

_EndProcedure1:
  PUSH   dword [esp]
  CALL  _SYS_FreeString@4
  ADD    esp,4
  RET
If I change this last line, the debug gives the right result. But I don't know what RET + 4 means.

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Wed Oct 11, 2017 12:16 am
by falsam
Another observation: With PB 5.61 x64 there is no bug.

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Wed Oct 11, 2017 12:56 am
by falsam
New version 1.36: Improved backup procedures for ASM and DESC files

This version does not correct the previously reported bug.

https://github.com/MLF4PB/MLF-Dev/archive/master.zip

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Wed Oct 11, 2017 8:48 am
by GPI
falsam wrote:If I change this last line, the debug gives the right result. But I don't know what RET + 4 means.
I google it: It means, that you return and remove 4 Bytes from the stack.

But I can't tell you, why this should fix this bug and does this fix everything or causes other situation really big troubles.

I think, for MLF the next big step is to learn assembler, or you create userlibs, that work in 99%, but for the rest you wonder, why it doesn't work. Or it work in the first 1000 calls and then the program crash.

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Wed Oct 11, 2017 1:42 pm
by falsam
Thanks for your feedback GPI. But i think there is a bug with pbcompiler. exe option /COMMENTED (Version x86)

http://www.purebasic.fr/english/viewtop ... =4&t=69381

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Wed Oct 11, 2017 3:00 pm
by GPI
falsam wrote: But i think there is a bug with pbcompiler. exe option /COMMENTED (Version x86)
Possible. Ok, i don't understand x86-Assembler, but I understand c64 and Atari-ST-Assembler and know how things work there. A change from "RET" to "RET +4" is not a simple or small error. It is a huge error. It will cause a corrupt stack and this is reallly bad and maybe one of the difficult to find errors. I personal y think, that when it is a bug, it has crash a long time before this.

The problem is: You don't understand assembler. You don't understand what you doing. It is more a try and error. When it doesn't work, try something else until it works. That in this case a very bad method of programming. Maybe it solve the problem in this one spezific way, but not in every position.

Have you read the readme in the SDK-folder? There is a discription how strings should be handled. And as my code-example aboves show: A Userlibrary must handle strings different from an "purebasic"-Procedure. PB does a complete different "call". So you use the Userlibrary in an undocumented way. Thats the problem. Thats also way the Program crash, when you try a MLF-Userlibrary with an array in the parameter.

There are other problems: Have you try to understand your ASM-Code for the Userlibrary? There are many "Public" in there. There are original designed to export this labels for other thing (PB-Libraries for example). Your code also export them.
Now you have to parts who export this label - the original PB-Main-File and your userlib. You have luck, that the linker seems to prefer the original-PB-Main-File and not your lib. Also it is possible, that the code in your userlib need the access to the label of the PB-Main-File, because it should be "extrn". It will fail and use the wrong one. Funny things can happen.

Also you handle Integers complete wrong.

Code: Select all

    
        Case "i", "l"  
          ProcedureType =  "Long | StdCall"
on a x86 system this is correct, but under x64 an integer is a quad! I use the Integer, when i want to return a pointer. Because a pointer is under x86 a long and under x64 a quad.

At the current state, i would say, that MLF is a very early alpha stage. With a high risk of bugs. Don't use it in productive systems. And maybe this thread should be moved to a different forum.
I personal y think, this bugs can be only solved, when LibraryMaker support not a new third Type (besides OBJ and LIB): PBOBJ. So the userlibs are called like a PB-Generated-Routine. But in this case, Fred should make the small step to add inbuild lib creation support ;)

I really like the idea, but the next step is: Learn Assembler to understand, what the problem is.

I think, it has a reason, why tailsbite died. Includes seems to be the much better way, with a higher chance of compatibility with future PB-Version.

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Wed Oct 11, 2017 4:55 pm
by falsam
GPI wrote:Have you read the readme in the SDK-folder?
Yes, of course! You seriously think I'd get into this project without reading the documentation ?

How I coded MLF ?
To code MLF I use only the tools in the PureBasic installation folder.
- pbcompiler. exe /COMMENTED to get the ASM file
- fasm. exe to get the obj file
- LibraryMaker. exe to obtain the user library.

These three tools are located in the PureBasic installation folder

Concerning the ASM file
The only transformations are to make the procedures public. I would never change the assembler code.

You're blaming me for my incompetence in assemblyman's language.
I insist: MLF does not generate the ASM code. I use the pbcompiler. exe tool to generate ASM !!.

About the DESC file
I read the ASM file for
- List dependencies
- List public procedures
- Determine the type of each procedure (Long, String, etc...)
- Determine the type of each parameter (Long, Stgring, etc...)

It's not complicated to code. I read the documentation (readme. txt) and follow the guidelines.
GPI wrote:Also you handle Integers complete wrong.
Code:

Case "i", "l"
ProcedureType = "Long | StdCall"

on a x86 system this is correct, but under x64 an integer is a quad!
If the user uses a variable that is not compatible with the compiler, MLF will not fix it.
GPI wrote:At the current state, i would say, that MLF is a very early alpha stage. With a high risk of bugs. Don't use it in productive systems.
Alert :!: :!: :!: !! MLF is very dangerous :!: :!: :!:
GPI wrote:And maybe this thread should be moved to a different forum.
No problem: Beginner Forum !!

You say "MLF is a very early alpha stage" Yes, I agree with that. This is the first sentence of the first message. But he didn't deserve this bashing ....

Ok! I'm looking for a bridge and I'm throwing myself into the void ....

Re: MLF : Make Lib Factory (For PureBasic)

Posted: Wed Oct 11, 2017 5:15 pm
by GPI
falsam wrote:Concerning the ASM file
The only transformations are to make the procedures public. I would never change the assembler code.
that is the problem. You must change the code or at least understand, what in the asm file. "Only put some public" will not lead to a function MLF. See the string problem (i don't think it is a PB bug, compare the code with and without userlibrary. The code of the dummy-procedure is always the same. The different is the "main-routine". It handles the call of the PB-Procedure different from the call of a Userlib-Procedure. And this why this bug happend. To solve this problem, you must understand, whats going on. Thats why it is important to lern Assembler, when your Project MLF should be a success. Otherwise you will never understand, why the "no" vanished or why the use of array will crash. Only then you could find solutions.
Also in the asm-code is rubbish and more things must be editied (for example all public in the original asm should be converted to extrn in my opinion).
The main problem is: The ASM-Code from pbcompiler is not designed to use in a userlibrary.
GPI wrote:Code:
Case "i", "l"
ProcedureType = "Long | StdCall"
btw. this is your code - you always translate integer to long in MLF. That is wrong. On x64 integer it is quad.
You say "MLF is a very early alpha stage" Yes, I agree with that. This is the first sentence of the first message. But he didn't deserve this bashing ....
Maybe it sounds harder then it should - my english and my social skills are bad ;)

When somebody post a code in Tricks'n'Tips, than this code should be useable in a productive way. Not for testing. And MLF is in the current state not useable in a productive system. Thats why i said it is dangerous. It is possible that the userlibs generated by MLF will crash under specific conditions or destroy strings. For example, when i don't know, that "define "no"+userlibproc()" could remove the no - i would search for hours to find this error.