MLF : Make Lib Factory (For PureBasic)

Share your advanced PureBasic knowledge/code with the community.
User avatar
falsam
Enthusiast
Enthusiast
Posts: 632
Joined: Wed Sep 21, 2011 9:11 am
Location: France
Contact:

Re: MLF : Make Lib Factory (For PureBasic)

Post by falsam »

GPI wrote:You must change the code or at least understand, what in the asm file.
I think it was you who hadn't read the readme. txt.

Making proceduresDLL public is within the reach of any beginner.

- Read the ASM file generated by Pbcompiler.
- Locate comments with a ProcedureDLL name
  Example : ; ProcedureDLL. s Dummy ()
- Insert after this public line PB_Dummy (very difficult to code)
- Replace _Procedure0: with PB_Dummy: (very difficult to code)
- Replaces all _ProcedureX with the real names of procedures
  Example CALL _Procedure0 => CALL PB_Dummy

Nothing complicated !! I don't need to understand ASM to do this.
GPI wrote:The main problem is: The ASM-Code from pbcompiler is not designed to use in a userlibrary.
That's your opinion.

➽ Windows 11 64-bit - PB 6.21 x64 - AMD Ryzen 7 - NVIDIA GeForce GTX 1650 Ti

Sorry for my bad english and the Dunning–Kruger effect 🤪
Bitblazer
Enthusiast
Enthusiast
Posts: 761
Joined: Mon Apr 10, 2017 6:17 pm
Location: Germany
Contact:

Re: MLF : Make Lib Factory (For PureBasic)

Post by Bitblazer »

Dunno if this is needed, but in any case: RET N and corrupting the stack by any kind of mistake or error, can lead to very hard to find problems due to the nature of the stack usage (local variables, return adresses, registers are all adressed/loaded through the stack) and because of its shared nature.

I have spend days on finding those kind of "mysterious" errors in software and it can be very tricky as the crashes often will show up much later and in correct code.
Last edited by Bitblazer on Wed Oct 11, 2017 6:14 pm, edited 1 time in total.
GPI
PureBasic Expert
PureBasic Expert
Posts: 1394
Joined: Fri Apr 25, 2003 6:41 pm

Re: MLF : Make Lib Factory (For PureBasic)

Post by GPI »

falsam wrote:
GPI wrote:You must change the code or at least understand, what in the asm file.
I think it was you who hadn't read the readme. txt.

Making proceduresDLL public is within the reach of any beginner.

- Read the ASM file generated by Pbcompiler.
- Locate comments with a ProcedureDLL name
  Example : ; ProcedureDLL. s Dummy ()
- Insert after this public line PB_Dummy (very difficult to code)
- Replace _Procedure0: with PB_Dummy: (very difficult to code)
- Replaces all _ProcedureX with the real names of procedures
  Example CALL _Procedure0 => CALL PB_Dummy

Nothing complicated !! I don't need to understand ASM to do this.
Do you know the Dunning-Kruger-Effect? Nobody is immun to this :)
https://en.wikipedia.org/wiki/Dunning%E ... ger_effect

First of all, you don't need to rename und replace - you can add a label. it is simpler and produces less errors.
Second: the ASM-File is designed to be the main-file of an exe. You want to use it as a Userlibrary. There are rubbish in the asm-code, you should remove. Did you never asked, why the programm "tailbite" is named that way?
Third: In the readme there is no manual, how to convert the asm-output for a library.
And finally: Tailbite is outdated as far as i know. when it is so "beginner" - why nobody else continued this work for this long time?
GPI wrote:The main problem is: The ASM-Code from pbcompiler is not designed to use in a userlibrary.
That's your opinion.
Ok, then explain, why it crahs, when you use a array in the parameter or why "no" vanish.
You can ask Fred about this. It think he know more than we are. And when it is so easy - why is there no Compile Userlib-option in the pbcompiler?
User avatar
falsam
Enthusiast
Enthusiast
Posts: 632
Joined: Wed Sep 21, 2011 9:11 am
Location: France
Contact:

Re: MLF : Make Lib Factory (For PureBasic)

Post by falsam »

GPI wrote:Do you know the Dunning-Kruger-Effect? Nobody is immun to this :)
https://en.wikipedia.org/wiki/Dunning%E ... ger_effect
Thank for the "the Dunning-Kruger-Effect"

Who are you to judge me? God? Continue to mastuberate your brain with your fakelibraries and your insipid OOP !!!

:arrow: Last October 8th you asked the question "how to create a userlibrary?" and you say "On windows it is easy. start fasm with the parameter "source. asm dest. obj", create a descfile and use librarymaker. exe."

http://www.purebasic.fr/english/viewtop ... 28#p513128

I confirm it is easy and I wait to see your code.
Last edited by falsam on Wed Oct 11, 2017 11:12 pm, edited 1 time in total.

➽ Windows 11 64-bit - PB 6.21 x64 - AMD Ryzen 7 - NVIDIA GeForce GTX 1650 Ti

Sorry for my bad english and the Dunning–Kruger effect 🤪
GPI
PureBasic Expert
PureBasic Expert
Posts: 1394
Joined: Fri Apr 25, 2003 6:41 pm

Re: MLF : Make Lib Factory (For PureBasic)

Post by GPI »

btw. look in the source from tailbite (you can found it in the forum).
RET+4 is generated by tailbite - not by the asm-Code from the pbcompiler. It is definitive not bug.
(form the inc_tailbite.pb)

Code: Select all

CompilerIf #PB_Compiler_OS = #PB_OS_Windows;extra safeguard
            If RETX4 ;- gnozal PB4.20 fix [Thanks Fred!]
              If PBVersionX64
                ;WriteString(File0, " + 8")
              Else
                WriteString(File0, " + 4")
              EndIf
            EndIf
          CompilerEndIf

A Tailbite does much more than only recompile, it rewrite the the prepared assembler-code.

I don't think, that fred will "fix" your "bug".
User avatar
falsam
Enthusiast
Enthusiast
Posts: 632
Joined: Wed Sep 21, 2011 9:11 am
Location: France
Contact:

Re: MLF : Make Lib Factory (For PureBasic)

Post by falsam »

GPI wrote:btw. look in the source from tailbite (you can found it in the forum).
RET+4 is generated by tailbite - not by the asm-Code from the pbcompiler. It is definitive not bug.
That's what I said with that comment. You're not teaching me anything.

http://www.purebasic.fr/english/viewtop ... 51#p513251

➽ Windows 11 64-bit - PB 6.21 x64 - AMD Ryzen 7 - NVIDIA GeForce GTX 1650 Ti

Sorry for my bad english and the Dunning–Kruger effect 🤪
G-Rom
User
User
Posts: 45
Joined: Sat Mar 19, 2011 3:49 pm
Contact:

Re: MLF : Make Lib Factory (For PureBasic)

Post by G-Rom »

GPI wrote:It is definitive not bug.
i not agree your point of view , a "procedure" inside dll or static lib , are in assembly the same , a simple "label" and a return statement , CALL and RET.
so, when you create the object file with fasm for use it with librarymaker , there is no PUSH dword before , but , when you call a procedure , in the final executable you have a push dword [_PB_StringBasePosition] , it is not possible to guest this, you have the same behavior when you pass an array in parameter , an another push dword.
for a procedure that return a string and have one array in parameter, you need to RET + 8 , one for the string , and last for the array.

When you call ProcedureDLL.s inside an executable program , Purebasic compiler write well RET 4 , but when you use a static lib, pbcompiler don't care about the return value and ignore the previous push , so , i think there is a problem with pbcompiler when he add the static lib to the final executable. the compiler dont check the push before the call and do not correct the ret value.
GPI wrote:I think, for MLF the next big step is to learn assembler, or you create userlibs, that work in 99%, but for the rest you wonder, why it doesn't work. Or it work in the first 1000 calls and then the program crash.
no , it's simple, working or not working. there is no middle behavior based on stars position...
The next step for Falsam is to detect string return , add 4 to the ret , and the same for array parameters, + 4 for each parameters, and probably the same for map / list
GPI
PureBasic Expert
PureBasic Expert
Posts: 1394
Joined: Fri Apr 25, 2003 6:41 pm

Re: MLF : Make Lib Factory (For PureBasic)

Post by GPI »

When you look in the Tailbite code, fred known this problem, because the +4-fix is mention by him. Why he doesn't fix it in the /commented - output?

When you look in the Readme.txt
When you want to return a string, you should use this function:
1) The function doesn't has string parameter

It's the easiest one, and you can use the SYS_GetOutputBuffer() funtion:.

char *Output = SYS_GetOutputBuffer(int Length, int PreviousPosition)

With this command you ask PureBasic to returns a string buffer of 'Length' (in characters, which means
than it's the same value in unicode or not) and store the pointer value in 'Output'. 'PreviousPosition' is a
value passed in the string function.

ex: Output = SYS_GetOutputBuffer(100, PreviousPosition);

Once you get this buffer, just write the string result in it and you're done.

Note: 'Length' should be the exact returned string length. It doesn't count the null terminating character
And only this function.
When you look in the ASM-Code, you find this
"CALL _SYS_FreeString@4"
That is not this function. It is a complete diffrent methode of returning a string. Thats why the trouble are here. Purebasic handles Userlibraries in a complete diffrent Method. You can test it.
Try it, userlib

Code: Select all

ProcedureDLL.s Dummy()
  Protected Buffer.s = "bug"
  ProcedureReturn Buffer
EndProcedure 
mainprogram:

Code: Select all

c$= "no " + Dummy()
debug c$
and
mainprogram2:

Code: Select all

ProcedureDLL.s Dummy2()
  Protected Buffer.s = "bug"
  ProcedureReturn Buffer
EndProcedure
c$= "no " + Dummy2()
debug c$
Now look at the assembler-Code of Mainprogram1 and mainprogram2
You will find diffrences, how Dummy() and Dummy2() are called. You don't need to understand Assembler to see, that there are differences! Why? Both procedures are identical (you can compare this part in Assembler too).

THAT is the Problem.

And sorry, do you understand why "RET+4" solve this problem? Did you understand *WHEN* it should added? Only on Strings? On Arrays? Or on every Routine? When?!!
Do you know what "RET+4" does? What is with 64Bit? Must you add here "RET+8" or you should leave it with "RET" and Why?

Sorry, only add a "RET+4" without knowing is really dangerous! Do you know what a stack is? What it is stored there? Why a corrupt stack is a really big problem? Did you know what "PUSH" does?

Did you understand, why when "RET+4" is added always by PBCompiler, it would create a DLL, which corrupt the stack and this dll will crash the calling programm? And in this case, we had here many Bug-Reports with DLLs!

The Big Problem is, that you don't understand what the ASM code does, you don't understand why this work and why this doesn't. Your answer is here "it work" - that is not enough.

The next step is: LEARNING ASSEMBLER! Than you can't answer my question without. Otherwise it is a Try-And-Error-Program without knowing when the error will happen. Otherwise you play "Russian roulette" with your stack and it will crash your Program. Not always of course.But it will crash on complete diffrent places in the code, it is difficult to find this kind of error.

What you here have is a big problem with Programming. The difference between an working code and understanding why it work. When you understand this, you can write a code, wich always work. Without you write a code which seems to work, but you can't guarantee it. It may work in 99%, but in the other 1% you will search for an eternity.
G-Rom
User
User
Posts: 45
Joined: Sat Mar 19, 2011 3:49 pm
Contact:

Re: MLF : Make Lib Factory (For PureBasic)

Post by G-Rom »

The next step is: LEARNING ASSEMBLER!
i've just write an asm compiler by the past for all pic microchip...
ok, so, you are an expert, tell me why we need to add ret 4 ? or ret 8 ?
GPI
PureBasic Expert
PureBasic Expert
Posts: 1394
Joined: Fri Apr 25, 2003 6:41 pm

Re: MLF : Make Lib Factory (For PureBasic)

Post by GPI »

G-Rom wrote:
The next step is: LEARNING ASSEMBLER!
i've just write an asm compiler by the past for all pic microchip...
ok, so, you are an expert, tell me why we need to add ret 4 ? or ret 8 ?
I never said, that I'm an expert. And that you need "ret 4" or "ret 8" is your solution, not mine. So you should expain this. Or since it is falsam project, he must know, if this will fix the Problem. Or is this Problem a specific problem and you should not always add this ret4. Or at least should find someone, who can answer this questions.

It seems, that fred mention this solution. Maybe you have luck and find here or in the french forum the dialog, where you find this answers. In this case, you have a "mentor". But in my opinion, when he want to proceed, it is better, when he understand the asm-code.
In my opinon a good programmer knows what he does. He read the manuals and understand, why this parameter are chosen and in which cases he should not use this function - even when it seems to work.
It is bad, when you write a code and it doesn't work and you don't know why. But it is even worse, when you write a code, it work and you don't know why. Then you don't know the limitation of your code, when it will work, when not and so on. And in this state is MLF. It seems to work - in the most parts. And falsam don't understand why it work (or not work). The next logical step is: Learning why it works and why not. And in this case it means: Learning Assembler. You can not write a program, which create ASM-userlibraries without knowing assembler!
In my opionion - even when the code works at 100% of all cases and the writer of the code doesn't understand why - this code should not be used. When you find somebody, who understands and says everything is ok, than you can use it. But MLF is far away from this state.
GPI
PureBasic Expert
PureBasic Expert
Posts: 1394
Joined: Fri Apr 25, 2003 6:41 pm

Re: MLF : Make Lib Factory (For PureBasic)

Post by GPI »

And maybe a missunderstanding: I don't want to falsam to abandon his MLF or that he is stupid and will never understand this. I said, that he need more knowledge to finish this project and where he will get it. It would be really nice to see the posibility to create Userlibraries with PB code. It is sad, that fred doesn't do anything in this direction.

I had also a suggesten two: It would be nice to call MLF with programmparameter like this

Code: Select all

mlf.exe <inputfile> <outputfile> /nores
And maybe it is a good idea to put the res-source-file in a seperate file. For example, when the lib-source-file is named "myuserlib.pb", it should scan for a "myuserlib.res.pb" and create only with this file the res file. You don't always want every structure/interface/constant in the res file and when you put a prototype in it, the pbcompiler will crash when he loads the file.

p.s.: I'm also not immune to the "Dunning-Kruger-Effect". I added a little modified method of creation of a userlib in my PreCompiler. I will remove it now, because I made the same mistakes like falsam. It seems to work and it was very easy.
And by the way. Download the Tailbite-Sourcecode. It isn't easy to understand, but maybe it is possible to avoid errors, which Tailbite has done and fixed in the past.
User avatar
dobro
Enthusiast
Enthusiast
Posts: 766
Joined: Sun Oct 31, 2004 10:54 am
Location: France
Contact:

Re: MLF : Make Lib Factory (For PureBasic)

Post by dobro »

GPI wrote: I said, that he need more knowledge to finish this project and where he will get it..
in the state, I like MLF.
there's no reason not to use it.
the remaining problems are still quite special in a "Standard" use.

the hardest part is done, now it's fun looking for bugs.
GPI, you give the impression of the jealous old woman, when I read you...

too bad, I liked your work on Japbe.

Let Falsam time to fix the problems.
Image
Windows 98/7/10 - PB 5.42
■ sites : http://michel.dobro.free.fr/
User avatar
nco2k
Addict
Addict
Posts: 1344
Joined: Mon Sep 15, 2003 5:55 am

Re: MLF : Make Lib Factory (For PureBasic)

Post by nco2k »

GPI's criticism is constructive and true. falsam should be happy to get such feedback. but instead, he is mocking the person trying to help. no offense, but that only shows his lack of confidence. without basic asm knowledge and especially without knowing how purebasic internally works, this project will suffer the same fate as tailbite. getting super defensive about it, wont make it any less true. why do you think maintaining tailbite was so difficult? new purebasic features and changes kept constantly breaking things, and tailbite had to adjust the asm code every time. the only person that could make a good userlib system, is the person that knows purebasics internals the best, and that is fred. every other userlib project is pretty much just a guessing game. i really hope that purebasic will be able to create userlibs at some point, since many people (including me) are wanting this feature very badly. then finally all this drama will have an end.

also @falsam, im not trying to discourage you from working on your project. having this, is better than having nothing, but in the long run, we need a native system. thats the only way forward.

c ya,
nco2k
If OSVersion() = #PB_OS_Windows_ME : End : EndIf
User avatar
Lunasole
Addict
Addict
Posts: 1091
Joined: Mon Oct 26, 2015 2:55 am
Location: UA
Contact:

Re: MLF : Make Lib Factory (For PureBasic)

Post by Lunasole »

Oh, that Tailbite's code is just a large piece of unreadable shit. Even my typical dirty shitty parsers often looks and are structured better.
Probably that's the answer why no one wants to maintain it ^^
I would rewrite it from scratch rather that diving into that badly commented chaotic mess of labels and procedures with lot of kb in size.

Anyway, here are some assembly parsing code parts, related to +4 fix:

1) Part where DllFunction value is defined (used later)

Code: Select all

*FileSeeker = FindNextString("; Procedure", IndexSeeker, FileStart)
*FileSeeker+11
DLLFunction = 0
;CallDebugger
If UCase(PeekS(*FileSeeker, 3))="DLL"
  *FileSeeker+3
  DLLFunction = 1
ElseIf UCase(PeekS(*FileSeeker, 4))="CDLL"
  *FileSeeker+4
  DLLFunction = 1
EndIf

2) Part where fix conditions determined

Code: Select all

While ProcSeeker<ProcEnd
  AlreadyThere = 0
  ThisLine$ = ""
  ThisLine$ = GetNextString(ProcSeeker, #SystemEOL)
  If function()\RetValue$="String"
    If Left(ThisLine$, 13)="_EndProcedure" And EndProcOffset=0
      EndProcOffset = ProcSeeker+Len(ThisLine$)
      EndProcOffset = FindNextString(#SystemEOL+"RET", EndProcOffset, ProcEnd)+Len(#SystemEOL)
      ;
      ;
      If function()\DLLFunction ; ONLY IF EXPORTED FUNCTION !!!
        CompilerIf #PB_Compiler_OS = #PB_OS_Windows;only used on windows!
          If IsPB420
            RETX4 = #True ;- gnozal PB4.20 fix
          EndIf
        CompilerEndIf
      EndIf
      ;
      ;
    EndIf
  EndIf

3) Finally, here fix is applied

Code: Select all

CompilerIf #PB_Compiler_OS = #PB_OS_Windows;extra safeguard
  If RETX4 ;- gnozal PB4.20 fix [Thanks Fred!]
    If PBVersionX64
      ;WriteString(File0, " + 8")
    Else
      WriteString(File0, " + 4")
    EndIf
  EndIf
CompilerEndIf

So it looks simple and hope really is so:
  • 1) +4 is added when library function is export (DLL/CDLL)
    2) It's added on Windows only
    3) It's added on x86 only, not needed on x64
    4) It's added only if function returns string (no other cases, like G-Rom described with arguments?)
Also @gnozal made that fix, maybe he can also help with that in MLF
"W̷i̷s̷h̷i̷n̷g o̷n a s̷t̷a̷r"
User avatar
nco2k
Addict
Addict
Posts: 1344
Joined: Mon Sep 15, 2003 5:55 am

Re: MLF : Make Lib Factory (For PureBasic)

Post by nco2k »

@Lunasole
i couldnt agree more. the problem is, tailbite was created without really knowing what the creater got himself into. the project began to grew quickly, without any real system or structure. instead of rewritting it from scratch, random people chipped in and patched things here and there. the endresult was just a huge, unmaintainable mess. in order for such a project to work, the code has to be well structured and easy to maintain. it has to be readable, so everyone who opens it, can understand which section does what and why.

c ya,
nco2k
If OSVersion() = #PB_OS_Windows_ME : End : EndIf
Post Reply