Need beta testers and/or ASM Gurus (no more subsystems)

TailBite specific forum

Moderators: gnozal, ABBKlaus, lexvictory

lexvictory
Addict
Addict
Posts: 1027
Joined: Sun May 15, 2005 5:15 am
Location: Australia
Contact:

Need beta testers and/or ASM Gurus (no more subsystems)

Post by lexvictory »

EDIT: This has now been merged into the Tailbite sources, so this topic simply serves as an indication as to where it started


What am I talking about? Having ASCII, Unicode, ThreadSafe and UnicodeThreadsafe versions of functions in the one Userlib file - no more UserLibUnicode etc subsystems needed.

I need more people to test this out, because I have a very basic knowledge of ASM - so I could be doing something wrong here.
What this code does is compile the Userlib 4 times (one time in each mode), it then modifies the Unicode/Threadsafe/UnicodeThreadsafe compiles to have the appropriate suffixes to ALL Public declarations (i.e. Procedures, variables) to separate completely the different compiles of the lib.
Then it merges them into the (unmodified) Ascii compile's folder, and then compiles it all into one userlib and puts it in the userlibraries folder.

So, it does NOT examine a function to see if it even uses strings - so the lib will be the same size as an Ascii userlib plus its subsystem counterparts.
I'm trying to figure out how to code this directly into TailBite, but I feel this should be tested properly first. (I also need time to figure out what does what in the TB sources!)

As usual with my test code, there are a bunch of constants at the top of the file. They more or less explain themselves.
The one that doesn't: #do64. Set it to 1 if you are compiling a 64 bit userlibrary, set it to 0 if you are doing an x86 one. Then edit the appropriate CompilerIf block. (Sorry non 64 bit people, I've left my 64 bit windows paths to PB x86 in)

WARNING: There is little error checking, and a LOT of debug info. Line 122 and 76 are the main culprits.
Single core users: If you want to use your computer while this runs (it takes a long while depending on the number of functions) you will want to add a few Delay calls, especially in the RenamePublicAndCopy procedure, as it maxes out one of my dual cores.

Your Tailbite NEEDS to be properly configured - including/especially the Asm source files folder, as this is the destination for the in work files (and where they will all be left once it is finished.

I have tested this with Droopy's Lib, and it seems to work well (but with 320 exported functions (there are more non exported ones) it takes quite a while - go-make-a-cup-of-coffee while)

If I've left anything out, don't shoot me please :P I'm a bit tired and headached..

Code: Select all

Procedure Error(errormsg.s) 
  MessageRequester("Error", errormsg, #MB_ICONERROR)
EndProcedure

#testfile = "H:\Documents\droopy svn\working copy\Droopy.pb"
#chm = "droopy"
#tb_libsrc = "C:\Documents and Settings\lex.XANDER\My Documents\TailBite Library Sources\";MUST BE EXACTLY THE SAME AS CONFIGURED IN TB
Global libname.s = RemoveString(GetFilePart(#testfile), ".pb")
#do64 = 0

CompilerIf #do64
#tailbitepath = "C:\Program Files\PureBasic4.30\TailBite\"
#fasm = "C:\Program Files\PureBasic4.30\Compilers\FAsm.exe"
#polib = "C:\Program Files\PureBasic4.30\Compilers\polib.exe"
#librarymaker = "C:\Program Files\PureBasic4.30\SDK\LibraryMaker.exe"
#purelibrariesfolder = "C:\Program Files\PureBasic4.30\PureLibraries\UserLibraries\"
CompilerElse
#tailbitepath = "C:\Program Files (x86)\PureBasic4.30\TailBite\"
#fasm = "C:\Program Files (x86)\PureBasic4.30\Compilers\FAsm.exe"
#polib = "C:\Program Files (x86)\PureBasic4.30\Compilers\polib.exe"
#librarymaker = "C:\Program Files (x86)\PureBasic4.30\SDK\LibraryMaker.exe"
#purelibrariesfolder = "C:\Program Files (x86)\PureBasic4.30\PureLibraries\UserLibraries\"
CompilerEndIf

RunProgram(#tailbitepath+"tailbite.exe", #DQUOTE$+#testfile+#DQUOTE$+" /DONT /KEEP /CHM:"+#chm, GetCurrentDirectory(),#PB_Program_Wait)
RunProgram(#tailbitepath+"tailbite.exe", #DQUOTE$+#testfile+#DQUOTE$+" /UCOD /DONT /KEEP /CHM:"+#chm+" /LIBN:"+libname+"_UNICODE", GetCurrentDirectory(),#PB_Program_Wait)
RunProgram(#tailbitepath+"tailbite.exe", #DQUOTE$+#testfile+#DQUOTE$+" /UCOD /THRD /DONT /KEEP /CHM:"+#chm+" /LIBN:"+libname+"_THREAD_UNICODE", GetCurrentDirectory(),#PB_Program_Wait)
RunProgram(#tailbitepath+"tailbite.exe", #DQUOTE$+#testfile+#DQUOTE$+" /THRD /DONT /KEEP /CHM:"+#chm+" /LIBN:"+libname+"_THREAD", GetCurrentDirectory(),#PB_Program_Wait)

Prototype RecurseProto(filename.s, subdir.s)

Procedure Recurse(dir.s, Func.RecurseProto,subdir.s, dirnum=1)
  If subdir : subdir+"\" :EndIf 
  If ExamineDirectory(dirnum, dir, "*.*");enumerate thru to find Public's
    While NextDirectoryEntry(dirnum)
      If DirectoryEntryType(dirnum) = #PB_DirectoryEntry_Directory
        If Left(DirectoryEntryName(dirnum), 1) <> ".";skip . and .. dirs.
          Recurse(dir+"\"+DirectoryEntryName(dirnum), Func, subdir+DirectoryEntryName(dirnum), dirnum+1)
        EndIf
      Else 
        Func(dir+"\"+DirectoryEntryName(dirnum), subdir)
      EndIf
    Wend
    FinishDirectory(dirnum)
  EndIf
EndProcedure

Global NewList PublicFuncs.s()
Procedure MakePublicList(filename.s, subdir.s)
  If LCase(GetExtensionPart(filename)) = "asm"
    Debug filename
    fileno = ReadFile(#PB_Any, filename)
    If fileno
      ReadStringFormat(fileno)
      While Eof(fileno) = 0
        line.s = ReadString(fileno)
        If FindString(line, "Public", 1)
          AddElement(PublicFuncs())
          PublicFuncs() = RemoveString(line, "Public ")
        EndIf
      Wend
      CloseFile(fileno)
    EndIf
  EndIf
EndProcedure

Global renamemode.s
Global NewList NewObjFiles.s()
Procedure RenamePublicAndCopy(filename.s, subdir.s)
  If LCase(Right(filename, 3)) <> "asm" : ProcedureReturn : EndIf ; so we only process asm files.
  destname.s = #tb_libsrc+libname+"\Functions\"+subdir+ReplaceString(GetFilePart(filename), libname+renamemode, libname)
  destname = Left(destname, Len(destname)-4)+renamemode+".asm";replace .asm with _MODE.asm eg. FunctionName.asm -> FunctionName_UNICODE.asm
  ;Debug destname
  AddElement(NewObjFiles())
  NewObjFiles() = RemoveString(destname, #tb_libsrc+libname+"\")
  Debug NewObjFiles()
  ;ProcedureReturn
  sourcefile = ReadFile(#PB_Any, filename)
  If sourcefile
    destfile = CreateFile(#PB_Any, destname)
    If destfile
      While Eof(sourcefile) = 0
        line.s = ReadString(sourcefile)
        If FindString(LCase(line), "init",1);it could be an init func
          line = ReplaceString(line, "PB_"+libname+"_Init", "PB_"+libname+"_Init"+renamemode, #PB_String_NoCase)
        EndIf
        If FindString(LCase(line), "end",1);it could be an end func
          line = ReplaceString(line, "PB_"+libname+"_End", "PB_"+libname+"_End"+renamemode, #PB_String_NoCase)
        EndIf
        found = 0
        For x = 0 To ListSize(PublicFuncs())-1
          SelectElement(PublicFuncs(),x)
          If MatchRegularExpression(x, line)
            found = 1
            Break
          EndIf
        Next x
        If found
          line = ReplaceString(line, ReplaceString(PublicFuncs(), libname, libname+renamemode), PublicFuncs()+renamemode)
          ;Debug line 
        EndIf
        WriteStringN(destfile, line)
      Wend
      CloseFile(sourcefile)
      CloseFile(destfile)
    Else
      Error("Could not open destfile"+#CRLF$+destname)
      End 
    EndIf
  Else
    Error("Could not open sourcefile"+#CRLF$+filename)
    End 
  EndIf
EndProcedure

Procedure CompileASM(filename.s, subdir.s)
  If LCase(Right(filename, 3)) <> "asm" : ProcedureReturn : EndIf ; so we only process asm files.
  fasm = RunProgram(#fasm, #DQUOTE$+filename+#DQUOTE$+" "+#DQUOTE$+ReplaceString(filename, ".asm", ".obj", #PB_String_NoCase)+#DQUOTE$, GetPathPart(filename), #PB_Program_Hide|#PB_Program_Open|#PB_Program_Read|#PB_Program_Error)
  If fasm
    While ProgramRunning(fasm)
      If AvailableProgramOutput(fasm)
        Debug ReadProgramString(fasm)
      Else
        errorstring.s = ReadProgramError(fasm)
        If errorstring : Debug errorstring : EndIf
      EndIf
    Wend
  Else
    Error("Could not run fasm")
  EndIf
EndProcedure

Recurse(#tb_libsrc+libname+"\functions", @MakePublicList(), "")
ForEach PublicFuncs()
  Debug PublicFuncs()
Next 

regex.s = "($|\]|,|:|\+| )"

renamemode = "_UNICODE"
ForEach PublicFuncs()
  If Not CreateRegularExpression(ListIndex(PublicFuncs()), ReplaceString(PublicFuncs(), libname, libname+renamemode)+regex)
    Error("could not create reg expression")
    End
  EndIf
Next
Recurse(#tb_libsrc+libname+renamemode+"\functions", @RenamePublicAndCopy(), "")

renamemode = "_THREAD"
ForEach PublicFuncs()
  If Not CreateRegularExpression(ListIndex(PublicFuncs()), ReplaceString(PublicFuncs(), libname, libname+renamemode)+regex)
    Error("could not create reg expression")
    End
  EndIf
Next
Recurse(#tb_libsrc+libname+renamemode+"\functions", @RenamePublicAndCopy(), "")

renamemode = "_THREAD_UNICODE"
ForEach PublicFuncs()
  If Not CreateRegularExpression(ListIndex(PublicFuncs()), ReplaceString(PublicFuncs(), libname, libname+renamemode)+regex)
    Error("could not create reg expression")
    End
  EndIf
Next
Recurse(#tb_libsrc+libname+renamemode+"\functions", @RenamePublicAndCopy(), "")

libnameObjFiles = OpenFile(#PB_Any, #tb_libsrc+libname+"\"+libname+"ObjFiles.txt")
If libnameObjFiles
  FileSeek(libnameObjFiles, Lof(libnameObjFiles));so we can append
  ForEach NewObjFiles()
    WriteStringN(libnameObjFiles, ReplaceString(NewObjFiles(), ".asm", ".obj", #PB_String_NoCase))
  Next 
  CloseFile(libnameObjFiles)
Else
  Error("Could not open "+#tb_libsrc+libname+"\"+libname+"ObjFiles.txt")
  End 
EndIf

NewList DescFileContent.s()
descFile = OpenFile(#PB_Any, #tb_libsrc+libname+"\"+libname+".Desc")
If descFile
  While Eof(descFile) = 0
    AddElement(DescFileContent())
    DescFileContent() = ReadString(descFile)
    If FindString(DescFileContent(), "|", 1); we need to add the extra flags to this line
      DescFileContent() = DescFileContent()+" | THREAD | UNICODE"
    EndIf
  Wend
  FileSeek(descFIle, 0)
  TruncateFile(descFile)
  ForEach DescFileContent()
    WriteStringN(descFile, DescFileContent())
  Next 
  CloseFile(descFile)
Else
  Error("Could not open "+#tb_libsrc+libname+"\"+libname+".Desc")
EndIf

Recurse(#tb_libsrc+libname+"\functions", @CompileASM(), "")

CompilerIf #do64
polib = RunProgram(#polib, "/MACHINE:X64 /OUT:"+#DQUOTE$+libname+".lib"+#DQUOTE$+" @"+#DQUOTE$+libname+"ObjFiles.txt"+#DQUOTE$, #tb_libsrc+libname, #PB_Program_Hide|#PB_Program_Open|#PB_Program_Read|#PB_Program_Error)
CompilerElse
polib = RunProgram(#polib, "/OUT:"+#DQUOTE$+libname+".lib"+#DQUOTE$+" @"+#DQUOTE$+libname+"ObjFiles.txt"+#DQUOTE$, #tb_libsrc+libname, #PB_Program_Hide|#PB_Program_Open|#PB_Program_Read|#PB_Program_Error)
CompilerEndIf
If polib
  While ProgramRunning(polib)
    If AvailableProgramOutput(polib)
      Debug ReadProgramString(polib)
    Else
      errorstring.s = ReadProgramError(polib)
      If errorstring : Debug errorstring : EndIf
    EndIf
  Wend
Else
  Error("Could not run polib")
EndIf

RunProgram(#librarymaker, #DQUOTE$+libname+".Desc"+#DQUOTE$+" /TO "+#DQUOTE$+#purelibrariesfolder+#DQUOTE$+" /NOUNICODEWARNING", #tb_libsrc+libname, #PB_Program_Hide|#PB_Program_Wait)
Last edited by lexvictory on Wed Mar 11, 2009 12:53 pm, edited 2 times in total.
Demonio Ardente

Currently managing Linux & OS X Tailbite
OS X TailBite now up to date with Windows!
ABBKlaus
Addict
Addict
Posts: 1143
Joined: Sat Apr 10, 2004 1:20 pm
Location: Germany

Post by ABBKlaus »

Works great here :D
Really goooooooddd work.

The only problems i have :

- Avira is detecting the trojan horse TR/Crypt.XPACK.Gen (false alarm)
- The PB-Lib is not created with LibraryMaker.exe [Fixed]

[EDIT] i found my bug, rewrote some constants
lexvictory
Addict
Addict
Posts: 1027
Joined: Sun May 15, 2005 5:15 am
Location: Australia
Contact:

Post by lexvictory »

ABBKlaus wrote:- The PB-Lib is not created with LibraryMaker.exe [Fixed]

[EDIT] i found my bug, rewrote some constants
Bug with TB, your lib or this program (or what you set the constants to at the beginning of this program?) ?
Demonio Ardente

Currently managing Linux & OS X Tailbite
OS X TailBite now up to date with Windows!
ABBKlaus
Addict
Addict
Posts: 1143
Joined: Sat Apr 10, 2004 1:20 pm
Location: Germany

Post by ABBKlaus »

lexvictory wrote:Bug with TB, your lib or this program (or what you set the constants to at the beginning of this program?) ?
no it was my fault, i modified the source to make things easier for me :wink:

I already compiled all my libs. No errors so far :wink:

Code: Select all

EnableExplicit

;#tb_libsrc : MUST BE EXACTLY THE SAME AS CONFIGURED IN TB 
#tb_libsrc = "c:\Users\Klaus\Documents\TailBite Library Sources"

;#testfile : MUST POINT TO YOUR LIBRARY
#testfile = "c:\Backup\ExcelWriter.pb"
;#testfile = "c:\Backup\GPIB_Lib.pb"
;#testfile = "c:\Backup\Printer_Lib.pb"
;#testfile = "c:\Backup\PrintBarcode.pb"
;#testfile = "c:\Backup\PureDYMO.pb"
;#testfile = "c:\Backup\PurePDF.pb"
;#testfile = "c:\Backup\PurePDFBarcode.pb"
;#testfile = "c:\Backup\RMChart.pbi"

;#do64 : '0' TO COMPILE IN X86 MODE / '1' TO COMPILE IN X64 MODE
#do64 = 0

Global libname.s, ext.s, chm.s, renamemode.s
Global NewList NewObjFiles.s() 
Global NewList PublicFuncs.s()
NewList DescFileContent.s() 
Define libnameObjFiles, descFile, polib, errorstring.s, regex.s

Prototype RecurseProto(filename.s, subdir.s) 

CompilerIf #do64=0
  #tailbitepath         = "C:\Program Files\TailBite" 
  #PBDir                = "C:\Program Files\PureBasic430X86"
CompilerElse 
  #tailbitepath         = "C:\Program Files\TailBite" 
  #PBDir                = "C:\Program Files\PureBasic430X64"
CompilerEndIf 

#fasm                 = #PBDir+"Compilers\FAsm.exe" 
#polib                = #PBDir+"Compilers\polib.exe" 
#librarymaker         = #PBDir+"SDK\LibraryMaker.exe" 
#purelibrariesfolder  = #PBDir+"PureLibraries\UserLibraries" 

libname = GetFilePart(#testfile)
ext = GetExtensionPart(libname)
If ext
  libname=Left(libname,Len(libname)-(Len(ext)+1))
EndIf
chm = libname

Procedure Error(errormsg.s) ; send an error messagebox to the user - also has an optional non default title 
  MessageRequester("Error", errormsg, #MB_ICONERROR) 
EndProcedure 

RunProgram(#tailbitepath+"tailbite.exe", #DQUOTE$+#testfile+#DQUOTE$+" /DONT /KEEP /CHM:"+chm, GetCurrentDirectory(),#PB_Program_Wait) 
RunProgram(#tailbitepath+"tailbite.exe", #DQUOTE$+#testfile+#DQUOTE$+" /UCOD /DONT /KEEP /CHM:"+chm+" /LIBN:"+libname+"_UNICODE", GetCurrentDirectory(),#PB_Program_Wait) 
RunProgram(#tailbitepath+"tailbite.exe", #DQUOTE$+#testfile+#DQUOTE$+" /UCOD /THRD /DONT /KEEP /CHM:"+chm+" /LIBN:"+libname+"_THREAD_UNICODE", GetCurrentDirectory(),#PB_Program_Wait) 
RunProgram(#tailbitepath+"tailbite.exe", #DQUOTE$+#testfile+#DQUOTE$+" /THRD /DONT /KEEP /CHM:"+chm+" /LIBN:"+libname+"_THREAD", GetCurrentDirectory(),#PB_Program_Wait) 

Procedure Recurse(dir.s, Func.RecurseProto,subdir.s, dirnum=1) 
  If subdir : subdir+"" :EndIf 
  If ExamineDirectory(dirnum, dir, "*.*");enumerate thru to find Public's 
    While NextDirectoryEntry(dirnum) 
      If DirectoryEntryType(dirnum) = #PB_DirectoryEntry_Directory 
        If Left(DirectoryEntryName(dirnum), 1) <> ".";skip . and .. dirs. 
          Recurse(dir+""+DirectoryEntryName(dirnum), Func, subdir+DirectoryEntryName(dirnum), dirnum+1) 
        EndIf 
      Else 
        Func(dir+""+DirectoryEntryName(dirnum), subdir) 
      EndIf 
    Wend 
    FinishDirectory(dirnum) 
  EndIf 
EndProcedure 

Procedure MakePublicList(filename.s, subdir.s) 
  Protected fileno, line.s
  
  If LCase(GetExtensionPart(filename)) = "asm" 
    Debug filename 
    fileno = ReadFile(#PB_Any, filename) 
    If fileno 
      ReadStringFormat(fileno) 
      While Eof(fileno) = 0 
        line = ReadString(fileno) 
        If FindString(line, "Public", 1) 
          AddElement(PublicFuncs()) 
          PublicFuncs() = RemoveString(line, "Public ") 
        EndIf 
      Wend 
      CloseFile(fileno) 
    EndIf 
  EndIf 
EndProcedure 

Procedure RenamePublicAndCopy(filename.s, subdir.s) 
  Protected destname.s, sourcefile, destfile, line.s, found, x
  
  If LCase(Right(filename, 3)) <> "asm" : ProcedureReturn : EndIf ; so we only process asm files. 
  destname = #tb_libsrc+libname+"\Functions"+subdir+ReplaceString(GetFilePart(filename), libname+renamemode, libname) 
  destname = Left(destname, Len(destname)-4)+renamemode+".asm";replace .asm with _MODE.asm eg. FunctionName.asm -> FunctionName_UNICODE.asm 
  ;Debug destname 
  AddElement(NewObjFiles()) 
  NewObjFiles() = RemoveString(destname, #tb_libsrc+libname+"") 
  Debug NewObjFiles() 
  ;ProcedureReturn 
  sourcefile = ReadFile(#PB_Any, filename) 
  If sourcefile 
    destfile = CreateFile(#PB_Any, destname) 
    If destfile 
      While Eof(sourcefile) = 0 
        line = ReadString(sourcefile) 
        If FindString(LCase(line), "init",1);it could be an init func 
          line = ReplaceString(line, "PB_"+libname+"_Init", "PB_"+libname+"_Init"+renamemode, #PB_String_NoCase) 
        EndIf 
        If FindString(LCase(line), "end",1);it could be an end func 
          line = ReplaceString(line, "PB_"+libname+"_End", "PB_"+libname+"_End"+renamemode, #PB_String_NoCase) 
        EndIf 
        found = 0 
        For x = 0 To ListSize(PublicFuncs())-1 
          SelectElement(PublicFuncs(),x) 
          If MatchRegularExpression(x, line) 
            found = 1 
            Break 
          EndIf 
        Next x 
        If found 
          line = ReplaceString(line, ReplaceString(PublicFuncs(), libname, libname+renamemode), PublicFuncs()+renamemode) 
          ;Debug line 
        EndIf 
        WriteStringN(destfile, line) 
      Wend 
      CloseFile(sourcefile) 
      CloseFile(destfile) 
    Else 
      Error("Could not open destfile"+#CRLF$+destname) 
      End 
    EndIf 
  Else 
    Error("Could not open sourcefile"+#CRLF$+filename) 
    End 
  EndIf 
EndProcedure 

Procedure CompileASM(filename.s, subdir.s) 
  Protected fasm, errorstring.s
  
  If LCase(Right(filename, 3)) <> "asm" : ProcedureReturn : EndIf ; so we only process asm files. 
  fasm = RunProgram(#fasm, #DQUOTE$+filename+#DQUOTE$+" "+#DQUOTE$+ReplaceString(filename, ".asm", ".obj", #PB_String_NoCase)+#DQUOTE$, GetPathPart(filename), #PB_Program_Hide|#PB_Program_Open|#PB_Program_Read|#PB_Program_Error) 
  If fasm 
    While ProgramRunning(fasm) 
      If AvailableProgramOutput(fasm) 
        Debug ReadProgramString(fasm) 
      Else 
        errorstring = ReadProgramError(fasm) 
        If errorstring : Debug errorstring : EndIf 
      EndIf 
    Wend 
  Else 
    Error("Could not run fasm") 
  EndIf 
EndProcedure 

Recurse(#tb_libsrc+libname+"\functions", @MakePublicList(), "") 
ForEach PublicFuncs() 
  Debug PublicFuncs() 
Next 

regex = "($|\]|,|:|\+| )" 

renamemode = "_UNICODE" 
ForEach PublicFuncs() 
  If Not CreateRegularExpression(ListIndex(PublicFuncs()), ReplaceString(PublicFuncs(), libname, libname+renamemode)+regex) 
    Error("could not create reg expression") 
    End 
  EndIf 
Next 
Recurse(#tb_libsrc+libname+renamemode+"\functions", @RenamePublicAndCopy(), "") 

renamemode = "_THREAD" 
ForEach PublicFuncs() 
  If Not CreateRegularExpression(ListIndex(PublicFuncs()), ReplaceString(PublicFuncs(), libname, libname+renamemode)+regex) 
    Error("could not create reg expression") 
    End 
  EndIf 
Next 
Recurse(#tb_libsrc+libname+renamemode+"\functions", @RenamePublicAndCopy(), "") 

renamemode = "_THREAD_UNICODE" 
ForEach PublicFuncs() 
  If Not CreateRegularExpression(ListIndex(PublicFuncs()), ReplaceString(PublicFuncs(), libname, libname+renamemode)+regex) 
    Error("could not create reg expression") 
    End 
  EndIf 
Next 
Recurse(#tb_libsrc+libname+renamemode+"\functions", @RenamePublicAndCopy(), "") 

libnameObjFiles = OpenFile(#PB_Any, #tb_libsrc+libname+""+libname+"ObjFiles.txt") 
If libnameObjFiles 
  FileSeek(libnameObjFiles, Lof(libnameObjFiles));so we can append 
  ForEach NewObjFiles() 
    WriteStringN(libnameObjFiles, ReplaceString(NewObjFiles(), ".asm", ".obj", #PB_String_NoCase)) 
  Next 
  CloseFile(libnameObjFiles) 
Else 
  Error("Could not open "+#tb_libsrc+libname+""+libname+"ObjFiles.txt") 
  End 
EndIf 

descFile = OpenFile(#PB_Any, #tb_libsrc+libname+""+libname+".Desc") 
If descFile 
  While Eof(descFile) = 0 
    AddElement(DescFileContent()) 
    DescFileContent() = ReadString(descFile) 
    If FindString(DescFileContent(), "|", 1); we need to add the extra flags to this line 
      DescFileContent() = DescFileContent()+" | THREAD | UNICODE" 
    EndIf 
  Wend 
  FileSeek(descFIle, 0) 
  TruncateFile(descFile) 
  ForEach DescFileContent() 
    WriteStringN(descFile, DescFileContent()) 
  Next 
  CloseFile(descFile) 
Else 
  Error("Could not open "+#tb_libsrc+libname+""+libname+".Desc") 
EndIf 

Recurse(#tb_libsrc+libname+"\functions", @CompileASM(), "") 

CompilerIf #do64
  polib = RunProgram(#polib, "/MACHINE:X64 /OUT:"+#DQUOTE$+libname+".lib"+#DQUOTE$+" @"+#DQUOTE$+libname+"ObjFiles.txt"+#DQUOTE$, #tb_libsrc+libname, #PB_Program_Hide|#PB_Program_Open|#PB_Program_Read|#PB_Program_Error) 
CompilerElse 
  polib = RunProgram(#polib, "/OUT:"+#DQUOTE$+libname+".lib"+#DQUOTE$+" @"+#DQUOTE$+libname+"ObjFiles.txt"+#DQUOTE$, #tb_libsrc+libname, #PB_Program_Hide|#PB_Program_Open|#PB_Program_Read|#PB_Program_Error) 
CompilerEndIf
If polib
  While ProgramRunning(polib) 
    If AvailableProgramOutput(polib) 
      Debug ReadProgramString(polib) 
    Else 
      errorstring = ReadProgramError(polib) 
      If errorstring : Debug errorstring : EndIf 
    EndIf 
  Wend 
Else 
  Error("Could not run polib") 
EndIf 

CompilerIf #do64
  RunProgram(#librarymaker, #DQUOTE$+libname+".Desc"+#DQUOTE$+" /TO "+#DQUOTE$+#purelibrariesfolder+#DQUOTE$+" /NOUNICODEWARNING", #tb_libsrc+libname, #PB_Program_Hide|#PB_Program_Wait)
CompilerElse
  RunProgram(#librarymaker, #DQUOTE$+libname+".Desc"+#DQUOTE$+" /TO "+#DQUOTE$+#purelibrariesfolder+#DQUOTE$+" /NOUNICODEWARNING /COMPRESSED", #tb_libsrc+libname, #PB_Program_Hide|#PB_Program_Wait)
CompilerEndIf
lexvictory
Addict
Addict
Posts: 1027
Joined: Sun May 15, 2005 5:15 am
Location: Australia
Contact:

Post by lexvictory »

Ah, I see :)
I just added constants as I needed them :D (it's not really meant for production use)
And I never usually code with enabled explicit (I use autocomplete a lot so it doesn't matter too much - until I get lots of local variables)

Should I be adding the /COMPRESSED flag to x86 Library maker? (saw it in your code)
And what does it compress exactly, do you know?

>I already compiled all my libs. No errors so far
I didn't really think there would be - if it compiles Droopy's lib, well, it should compile all (droopy's lib has SO many different coding styles...)
Although I haven't extensively tested Droopy's lib :lol:
Demonio Ardente

Currently managing Linux & OS X Tailbite
OS X TailBite now up to date with Windows!
El_Choni
TailBite Expert
TailBite Expert
Posts: 1007
Joined: Fri Apr 25, 2003 6:09 pm
Location: Spain

Post by El_Choni »

Great job, Lexvictory -this was a complex task indeed.

Regards,
El_Choni
lexvictory
Addict
Addict
Posts: 1027
Joined: Sun May 15, 2005 5:15 am
Location: Australia
Contact:

Post by lexvictory »

El_Choni wrote:Great job, Lexvictory -this was a complex task indeed
:shock:
Thank you!
I wasn't sure you were still keeping tabs on us :)

I just had a brainwave one day on how to do it, and after that it was just coding it all to process it (once I did it by hand the first time). It seemed fairly easy - I just had to experiment with Regular Expressions to match only the whole function name.
I was surprised it even worked - as I really don't know much ASM!

I just made a change on my copy of this code here so that it wouldn't do the renaming on variables (anything with _v_,_s_,_l_ in it) - is this wise?
It seems to work so far - though I had to restart the IDE to get the quick help to show up!
The less it has to loop through the quicker it is!

In my plan to code it into Tailbite, I'm still trying to find out where to influence a few things in the process. (heck, I still can't seem to find what line the call is to delete the subdir of TBTemp...! (I commented one out and it still happens!)

The reason I want to put it in TB is because it takes too long to post-process it...

Perhaps we need a group rewrite of part of it......
I shall do some more experimenting :twisted:
Demonio Ardente

Currently managing Linux & OS X Tailbite
OS X TailBite now up to date with Windows!
lexvictory
Addict
Addict
Posts: 1027
Joined: Sun May 15, 2005 5:15 am
Location: Australia
Contact:

Post by lexvictory »

I DID IT!!!

It's not as clean as the post processing method in the first post of this thread - but in my tests with droopy's lib it works!
eg some shared functions end up named Droopy_Droopy_Droopy_Droopy_FunctionName (when it compiles droopy's lib naturally)

download http://demonioardente.us.to/pb/Tailbite ... ltilib.zip

:D
Demonio Ardente

Currently managing Linux & OS X Tailbite
OS X TailBite now up to date with Windows!
ABBKlaus
Addict
Addict
Posts: 1143
Joined: Sat Apr 10, 2004 1:20 pm
Location: Germany

Post by ABBKlaus »

lexvictory wrote:eg some shared functions end up named Droopy_Droopy_Droopy_Droopy_FunctionName (when it compiles droopy's lib naturally)
It seems i found that bug, here´s a version to test : http://www.tailbite.com/downloads/TailB ... R1.880.zip

PS: i merged that SplitFunctions_MultiLib() function into the already existing SplitFunctions() and MultiLibFixStaticStrings() isn´t needed anymore.
lexvictory
Addict
Addict
Posts: 1027
Joined: Sun May 15, 2005 5:15 am
Location: Australia
Contact:

Post by lexvictory »

Now I get the same bug that I added a fix to the fix static strings for; some functions get an extrn declared in their own ASM file
But the extra prepending of the libname is confirmed fixed.

Code: Select all

format MS64 COFF

Public PB_NextDirectory_UNICODE <---


Extrn PB_NextDirectoryEntry_UNICODE
Extrn PB_DirectoryEntryName_UNICODE
Extrn SYS_AllocateString4
Extrn SYS_StringEqual
Extrn PB_NextDirectory_UNICODE <---
Extrn SYS_FreeString
Extrn PB_StringBasePosition
Extrn Droopy_UNICODE__S23
Extrn Droopy_UNICODE__S24


section '.text' code readable executable

PB_NextDirectory_UNICODE: <--- Fasm complains here
edit: fixed above error by replacing

Code: Select all

If Not (function()\DLLFunction=0 And function()\Name$+decoration=external())
With

Code: Select all

If Not (function()\DLLFunction=0 And function()\Name$+decoration=external() And "PB_"+function()\Name$+decoration=external())
on line 1346 of inc_tailbite.pb
However, I now get more or less the same error on a shared function Droopy_SearchDirectory (the ascii function - though I think all varieties are affected)

edit 2: This line makes it compile properly: (am I just making more bugs?)

Code: Select all

If Not (function()\Name$+decoration=external() Or "PB_"+function()\Name$+decoration=external())

The main reason I copied the Split functions procedure was so that I could continue to use the source in normal (perhaps now legacy) mode.
That was until I edited the MainProc
But on the whole, good job - I thought you might be able to merge it, as you understand the source much better than I do :lol:

One fix I didn't add is to make sure that Useunicodeoption and Usethreadoption are disabled.
I haven't had a detailed look at your code, but I think it wise to make a /LEGACYMODE parameter which will make TB compile just the Ascii, unicode, threadsafe or unicodethreadsafe lib - just in case the new compile method is not wanted - eg. a lib that does not use strings or threadsafe functions.
However I do plan to implement a method of intelligently making the decorated functions - like scanning for StringBase and/or _unicode/_thread and other appropriate PB internal variants of the functions.
Last edited by lexvictory on Mon Feb 02, 2009 8:07 am, edited 2 times in total.
Demonio Ardente

Currently managing Linux & OS X Tailbite
OS X TailBite now up to date with Windows!
lexvictory
Addict
Addict
Posts: 1027
Joined: Sun May 15, 2005 5:15 am
Location: Australia
Contact:

Post by lexvictory »

An idea I had a few days ago, is to put the Tailbite source into svn.
Obviously it's kind of a major management change, but it would also mean you might not need to do a "here's a version to test [link]" because people could just update their svn source copy between releases if need be.

The only real change you would have to make personally is that you would have to remember to do an svn commit after making changes.
Also, you'd need to decide on where the compiler options are to be saved by anyone who has write access to the svn repository - eg. I have mine set to use a common project.cfg for all files.

Basically it would involve doing what I did with droopy's lib - making a sourceforge project or similar - unless you can make an svn repository on your current setup. The website might have to be modded a bit - but that's easily done (I'd be happy to do it if required)

Just a thought/suggestion - you're naturally quite welcome to continue doing it the way you currently :)
Demonio Ardente

Currently managing Linux & OS X Tailbite
OS X TailBite now up to date with Windows!
ABBKlaus
Addict
Addict
Posts: 1143
Joined: Sat Apr 10, 2004 1:20 pm
Location: Germany

Post by ABBKlaus »

lexvictory wrote:Now I get the same bug that I added a fix to the fix static strings for
I already uploaded a fixed version yesterday, please test.
I also noticed that there is only one debug function used, this could be a problem if working with unicode strings.
lexvictory wrote:One fix I didn't add is to make sure that Useunicodeoption and Usethreadoption are disabled.
I haven't had a detailed look at your code, but I think it wise to make a /LEGACYMODE parameter which will make TB compile just the Ascii, unicode, threadsafe or unicodethreadsafe lib - just in case the new compile method is not wanted - eg. a lib that does not use strings or threadsafe functions.
I made the opposite, now there is a switch called /MULT for the new compiling method. The TBManager is updated to handle this.
lexvictory wrote:An idea I had a few days ago, is to put the Tailbite source into svn.
Obviously it's kind of a major management change, but it would also mean you might not need to do a "here's a version to test [link]" because people could just update their svn source copy between releases if need be.

The only real change you would have to make personally is that you would have to remember to do an svn commit after making changes.
Also, you'd need to decide on where the compiler options are to be saved by anyone who has write access to the svn repository - eg. I have mine set to use a common project.cfg for all files.

Basically it would involve doing what I did with droopy's lib - making a sourceforge project or similar - unless you can make an svn repository on your current setup. The website might have to be modded a bit - but that's easily done (I'd be happy to do it if required)
What is needed to setup this on the TailBite website (if possible) ?
If you need more info of my hoster, here is a link http://www.1blu.de/webhosting/homepagepakete/unlimited/
Mistrel
Addict
Addict
Posts: 3415
Joined: Sat Jun 30, 2007 8:04 pm

Post by Mistrel »

I'm very interested in testing this with all of the PureGDK's user libraries. Are there plans to integrate this functionality into Tailbite?
lexvictory
Addict
Addict
Posts: 1027
Joined: Sun May 15, 2005 5:15 am
Location: Australia
Contact:

Post by lexvictory »

Mistrel wrote:I'm very interested in testing this with all of the PureGDK's user libraries. Are there plans to integrate this functionality into Tailbite?
ABBKlaus wrote:now there is a switch called /MULT for the new compiling method. The TBManager is updated to handle this.
So yes, I'm thinking so. Check the download link in his previous post, ABB has been fixing my cruddy additions to TB :lol:

@ABBKlaus: Haven't tested yet, will do so later today.
About SVN, I'm not sure you could set up to repository on your webserver - I think it needs special extensions etc, I think all I was meaning to Mod the website was to put SVN instructions on it and to make it point to sourceforge download links (if you chose to use the file release system there)
I could also make it multilanguage friendly - see the droopy's lib website as an example.
Take a look at: https://sourceforge.net/svn/?group_id=211325 for sourceforge's SVN set up
and: https://sourceforge.net/project/showfil ... _id=211325 for an example of how I use the file releases.
File releases are stored permanently and are mirrored around the world (though I don't think you're obligated to use the file release system if you don't want to)
If you're not familiar with svn, I recommend http://svnbook.red-bean.com/en/1.4/index.html too.
Demonio Ardente

Currently managing Linux & OS X Tailbite
OS X TailBite now up to date with Windows!
lexvictory
Addict
Addict
Posts: 1027
Joined: Sun May 15, 2005 5:15 am
Location: Australia
Contact:

Post by lexvictory »

ok, have tested - excellent work, especially the changing of the status test :)
Droopy's Lib compiles fine

But in TBManager.pb, can #PB_CheckBox_Right be removed from line 416?
It creates an annoying little space between the checkbox and the text.

Just noticed: doesn't the /MULT make TB not check that it is already running?
line 43 of inc_parameter.pb

Also, in the TB manager code, it checks and disables unicode and threadsafe checkboxes, but I can't seem to find a similar safeguard in the Tailbite.exe code. Did I miss it?


Been doing some research, and Google Code is an SVN alternative to Sourceforge (possibly easier if you already have a google account).
But I don't have any experience with Google Code.
Demonio Ardente

Currently managing Linux & OS X Tailbite
OS X TailBite now up to date with Windows!
Post Reply