PureBasic Forums - English

Posted: **Sat Nov 26, 2022 4:50 pm**

Hello ;
Seems that I am frequently looking for working examples
in my own programs (1st) , thinking
"Hmm...how did I do that last time?" or "have I ever done that before?" .

My programming writing frequency has large gaps , some times a year or more .
So , I want to write a documentation program
'List-Keywords-In-Program-Source.pb' ,
so that I can put a 'List of Keywords used' in each Program Source .

But 1st , is there already a program that does this ?
If not then I will proceed .
I got a list of 1781 keywords from
https://www.purebasic.com/documentation ... index.html .

It looks like I will have to parse each line in *.pb file to see if line contains a keyword .
2nd question is ; What is the best/fastest structure to use , Arrays, Lists & Structures ?

Thanks for your Help...

Posted: **Sun Nov 27, 2022 12:21 pm**

Doesn't it make more sense to have a search program where you can enter a keyword, and it will show you in which code files it was found?

The PureBasic IDE already offers you the possibility to search for text in all files of a directory (including their subdirectories):

Click on "Edit" in the menu bar at the top of the PB IDE window.
Select "Find in files ...".

In the extension filter you can enter "*.pb,*.pbi" (without the quotation marks) so that only PB code files are searched.

Posted: **Sun Nov 27, 2022 6:26 pm**

Use my program

Posted: **Sun Nov 27, 2022 11:40 pm**

Thanks Sicro ;
Ya , I checked out "Find in Files" , pretty cool , unfortunately you can copy the list ,
but that's not the direction I need to take .
Sorry , guess I didnt explain things properly :
I have written only about 20 programs that I want to parse , mostly utility type .
A few are on : https://sourceforge.net/u/vmars-us/profile/ .
Someday I'll get caught up and put them all out there .
So with given the 20 pgms , I want to search each one separately to see what Keywords are used in the program ; then take those Keywords lines and put them into the Documentation section of each program (List-Keywords-In-Source-Program) ;
and then write the 'program-name & keywords' as line entries into a separate .txt file ; that file then becomes input to a second program possibly named 'Programs-Keywords-Matrix' .
I hope thats a better explanation .

Posted: **Sun Nov 27, 2022 11:54 pm**

Thanks Azjio ,
Yes , your text-replace program looks pretty cool . I like your Docs .
I have a Text-Manipulation-Program here:
https://vmars.us/freeware/Line-By-Line/ ... _HELP.html
Except mine doesnt have RegEx , I havent gotten into RegEx yet , maybe someday .
I have written only about 20 programs that I want to parse , mostly utility type .
A few are on : https://sourceforge.net/u/vmars-us/profile/ .
Someday I'll get caught up and put them all out there

Posted: **Mon Nov 28, 2022 12:02 am**

It looks like I will have to parse each line in each of my 20 *.pb programs to see if line contains one of the 1781 keywords .
So my Question is:
What is the best/fastest structure to use , Arrays, Lists , etc. ?

Thanks for your Help...

Posted: **Mon Nov 28, 2022 4:45 pm**

My initial thought was to put the keywords into a map and then tokenise the source lines and compare tokens against the map... Little John's mod of srods tokeniser should do the job. See viewtopic.php?p=332453#p332453

Kudos to Danilo who wrote the original program that I hacked below to test. It obtains the list directly from the compiler rather than having to build it yourself, see viewtopic.php?t=53701

(Just for the record, yes I know it would be faster without the debugger.)

Code: Select all

EnableExplicit

#Compiler = #PB_Compiler_Home+"compilers\pbcompiler.exe"

Procedure StartCompiler()
  ProcedureReturn RunProgram(#Compiler,"/STANDBY","",#PB_Program_Open|#PB_Program_Read|#PB_Program_Write|#PB_Program_Hide)
EndProcedure

Procedure StopCompiler(compiler)
  WriteProgramStringN(compiler, "END")
  WaitProgram(compiler,5000)
  CloseProgram(compiler)
EndProcedure

Procedure SendCompilerCommand(compiler,command$)
  If ProgramRunning(compiler)
    WriteProgramStringN(compiler, command$)
  EndIf
EndProcedure

Procedure.s GetCompilerOutput(compiler)
  If AvailableProgramOutput(compiler)
    ProcedureReturn ReadProgramString(compiler)
  EndIf
EndProcedure

Procedure FillList(compiler,List out.s(),space=0)
  Protected out$, tr
  Protected space$=Space(space)
  While out$<>"OUTPUT"+#TAB$+"COMPLETE" And Left(out$,5)<>"ERROR"
    out$=GetCompilerOutput(compiler)
    If out$ And out$<>"OUTPUT"+#TAB$+"COMPLETE" And Left(out$,5)<>"ERROR" And FindString("0123456789",Mid(out$,1,1))=0
      tr = FindString(out$, " (")
      out$ = Trim(Left(out$, tr))
      AddElement(out())
      out()=out$
    EndIf
  Wend
EndProcedure

Procedure GetProcedureList(compiler,List out.s())
  If ProgramRunning(compiler)
    SendCompilerCommand(compiler,"FUNCTIONLIST")
    FillList(compiler,out())
  EndIf
EndProcedure

Procedure WaitCompilerReady(compiler)
  Protected out$
  While out$<>"READY" And Left(out$,5)<>"ERROR"
    out$ = GetCompilerOutput(compiler)
    If out$
      Debug out$
    EndIf
  Wend
EndProcedure

Structure IX
  BlockStart.i
  BlockEnd.i
EndStructure

#Rounds = 10000

Define  pb, out$, starttime, endtime, i, j, x, e, s, cur, prev
NewList lstKeywords.S()
NewMap mapKeywords.S()

Debug "Obtaining keyword list..."
starttime = ElapsedMilliseconds()
pb = StartCompiler()
If pb
  WaitCompilerReady(pb)
  
  GetProcedureList(pb,lstKeywords())
  
  StopCompiler(pb)
EndIf
SortList(lstKeywords(), #PB_Sort_Ascending)
endtime = ElapsedMilliseconds()
Debug StrU(endtime - starttime) + " milliseconds."

Debug "Transfer to array and index..."
Dim arrIndex.IX(26)
i = ListSize(lstKeywords())
Dim arrKeywords.S(i)
starttime = ElapsedMilliseconds()
prev = 65
cur = 65
arrIndex(1)\BlockStart = 1
ForEach lstKeywords()
  cur = Asc(Left(lstKeywords(), 1))
  i = ListIndex(lstKeywords())
  If cur <> prev
    arrIndex(prev - 64)\BlockEnd = ListIndex(lstKeywords()) - 1
    arrIndex(cur - 64)\BlockStart = ListIndex(lstKeywords())
    prev = cur
  EndIf
  arrKeywords(ListIndex(lstKeywords())) = lstKeywords()
Next lstKeywords()
arrIndex(26)\BlockEnd = ListSize(lstKeywords())
endtime = ElapsedMilliseconds()
Debug StrU(endtime - starttime) + " milliseconds."

Debug "Unindexed List lookups..."
starttime = ElapsedMilliseconds()
For i = 1 To #Rounds
  ForEach lstKeywords()
    If lstKeywords() = "MemoryStatus"
      Break
    EndIf
  Next lstKeywords()
  ForEach lstKeywords()
    If lstKeywords() = "ACos"
      Break
    EndIf
  Next lstKeywords()
  ForEach lstKeywords()
    If lstKeywords() = "ZoomSprite"
      Break
    EndIf
  Next lstKeywords()
Next i
endtime = ElapsedMilliseconds()
Debug StrU(endtime - starttime) + " milliseconds."

Debug "Indexed List lookups..."
starttime = ElapsedMilliseconds()
For i = 1 To #Rounds
  
  j = 'M' - 64
  SelectElement(lstKeywords(), arrIndex(j)\BlockStart)
  Repeat
    If lstKeywords() = "MemoryStatus"
      Break
    EndIf
    NextElement(lstKeywords())
  Until ListIndex(lstKeywords()) = arrIndex(j)\BlockEnd
  
  j = 'A' - 64
  SelectElement(lstKeywords(), arrIndex(j)\BlockStart)
  Repeat
    If lstKeywords() = "ACos"
      Break
    EndIf
    NextElement(lstKeywords())
  Until ListIndex(lstKeywords()) = arrIndex(j)\BlockEnd
  
  j = 'Z' - 64
  SelectElement(lstKeywords(), arrIndex(j)\BlockStart)
  Repeat
    If lstKeywords() = "ZoomSprite"
      Break
    EndIf
    NextElement(lstKeywords())
  Until ListIndex(lstKeywords()) = arrIndex(j)\BlockEnd
  
Next i
endtime = ElapsedMilliseconds()
Debug StrU(endtime - starttime) + " milliseconds."

Debug "Unindexed array lookups..."
starttime = ElapsedMilliseconds()
For i = 1 To #Rounds
  e = ArraySize(arrKeywords())
  For j = 1 To e
    If arrKeywords(j) = "MemoryStatus"
      Break
    EndIf
  Next j
  For j = 1 To e
    If arrKeywords(j) = "Acos"
      Break
    EndIf
  Next j  
  For j = 1 To e
    If arrKeywords(j) = "ZoomSprite"
      Break
    EndIf
  Next j
Next i
endtime = ElapsedMilliseconds()
Debug StrU(endtime - starttime) + " milliseconds."

Debug "Indexed array lookups..."
For i = 1 To #Rounds
  
  j = 'M' - 64
  s = arrIndex(j)\BlockStart 
  e = arrIndex(j)\BlockEnd
  For j = s To e
    If arrKeywords(j) = "MemoryStatus"
      Break
    EndIf  
  Next J
  
  j = 'A' - 64
  s = arrIndex(j)\BlockStart 
  e = arrIndex(j)\BlockEnd
  For j = s To e
    If arrKeywords(j) = "Acos"
      Break
    EndIf
  Next j  
  
  j = 'Z' - 64
  s = arrIndex(j)\BlockStart 
  e = arrIndex(j)\BlockEnd
  For j = s To e
    If arrKeywords(j) = "ZoomSprite"
      Break
    EndIf
  Next j
  
Next i
endtime = ElapsedMilliseconds()
Debug StrU(endtime - starttime) + " milliseconds."

Debug "Transferring to map..."
starttime = ElapsedMilliseconds()
ForEach lstKeywords()
  AddMapElement(mapKeywords(), lstKeywords())
Next lstKeywords()
endtime = ElapsedMilliseconds()
Debug StrU(endtime - starttime) + " milliseconds."

Debug "Map lookups..."
starttime = ElapsedMilliseconds()
For i = 1 To #Rounds
  x = FindMapElement(mapKeywords(), "MemoryStatus")
  x = FindMapElement(mapKeywords(), "ACos")
  x = FindMapElement(mapKeywords(), "ZoomSprite")
Next i
endtime = ElapsedMilliseconds()
Debug StrU(endtime - starttime) + " milliseconds."

Posted: **Mon Nov 28, 2022 6:55 pm**

Thanks spikey ,
I ran all three , Danilo's is the only one that tries to output *.pb files .
But it gets an error here:
CompilerIf #PB_Compiler_Unicode
CompilerError "Please turn off compiler option 'Create unicode executable'"
CompilerEndIf
So I commented it out , and then it ran with no errors , but created no output *.pb files .
I could find no such option in compiler option:
"Please turn off compiler option 'Create unicode executable'"

No Structures.pb , Interfaces.pb , or Constants.pb .

How can I fix this ?
Thanks...

Posted: **Mon Nov 28, 2022 7:31 pm**

There are several versions of the code in the thread with modifications - which one exactly did you try?
The most recent one by gurj on Sun Nov 26, 2017 doesn't output to .pb files but files called "pb600.constants" etc.

Posted: **Mon Nov 28, 2022 10:51 pm**

Thanks spikey ,
This is the Danilo one I used ,
viewtopic.php?f=12&t=53701&hilit=GetPBInfo

I searched for all this , all and in parts :
gurj
Sun Nov 26, 2017 pb600.constants
But couldnt find it .
This is OK for now:
https://www.purebasic.com/documentation/list/index.html

It looks like I will have to parse each line in each of my 20 *.pb programs to see if line contains one of the 1781 keywords .
So my Question really is:
What is the best/fastest structure to use , Arrays, Lists , etc. ?

Thanks for your Help...

Posted: **Tue Nov 29, 2022 1:28 pm**

Not sure if you are able to solve this, but here's the version that works with Win 10 x64...

Code: Select all

;=========================================================
; GetPBInfo.pb by Danilo
; Version: PureBasic 5.10
; see: https://www.purebasic.fr/english/viewtopic.php?t=53701
;
; - add out Procedures by Denis
; - arranging out code by gurj on [pb_v5.61]
; - other various mods
;=========================================================
; works with 5.73
; works with 6.00 LTS
;
EnableExplicit

Global log.s
#Compiler = #PB_Compiler_Home + "compilers\pbcompiler.exe"

Procedure StartCompiler()
 ProcedureReturn RunProgram(#Compiler,"/STANDBY","",#PB_Program_Open|#PB_Program_Read|#PB_Program_Write|#PB_Program_Hide)
EndProcedure

Procedure StopCompiler(compiler)
 WriteProgramStringN(compiler, "END")
 WaitProgram(compiler,5000)
 CloseProgram(compiler)
EndProcedure

Procedure SendCompilerCommand(compiler,command$)
 If ProgramRunning(compiler)
  WriteProgramStringN(compiler, command$)
 EndIf
EndProcedure

Procedure.s GetCompilerOutput(compiler)
 If AvailableProgramOutput(compiler)
  ProcedureReturn ReadProgramString(compiler)
 EndIf
EndProcedure

Procedure FillList(compiler,List OUT.s(),space=0)
 Protected out$
 Protected space$=Space(space)
 While out$<>"OUTPUT"+#TAB$+"COMPLETE" And Left(out$,5)<>"ERROR"
  out$=GetCompilerOutput(compiler)
  If out$ And out$<>"OUTPUT"+#TAB$+"COMPLETE" And Left(out$,5)<>"ERROR" And FindString("0123456789",Mid(out$,1,1))=0
   AddElement(out())
   out()=space$+out$
  EndIf
 Wend
EndProcedure

Procedure FillConstantList(compiler,List OUT.s(),space=0)
 Protected out$
 Protected space$=Space(space)
 While out$<>"OUTPUT"+#TAB$+"COMPLETE" And Left(out$,5)<>"ERROR"
  out$=GetCompilerOutput(compiler)
  If out$<>"" And out$<>"OUTPUT"+#TAB$+"COMPLETE" And Left(out$,5)<>"ERROR" And FindString("0123456789",Mid(out$,2,1))=0
   If FindString("01",Mid(out$,1,1))
    out$ = "#"+Mid(out$,2,Len(out$)-1)
    out$ = ReplaceString(out$,#TAB$," = ")
    out$ = ReplaceString(out$,"# = ","#")
   ElseIf FindString("2",Mid(out$,1,1))
    Protected i, found_non_printable = #False
    Protected oldout$ = out$
    Protected sconst_value$  = StringField(oldout$,3,Chr(9))
    out$ = "#"+StringField(oldout$,2,#TAB$)
    For i = 1 To Len(sconst_value$)
     If Asc(Mid(sconst_value$,i)) < 32 Or Asc(Mid(sconst_value$,i)) > 126
      found_non_printable = #True
     EndIf
    Next i
    If out$ = "#TAB$"
     out$ + " = Chr(9)"
    ElseIf out$ = "#HT$"
     out$ + " = Chr(9)"
    ElseIf out$ = "#CRLF$"
     out$ + " = Chr(13) + Chr(10)"
    ElseIf out$ = "#LFCR$"
     out$ + " = Chr(10) + Chr(13)"
    ElseIf out$ = "#LF$"
     out$ + " = Chr(10)"
    ElseIf out$ = "#CR$"
     out$ + " = Chr(13)"
    ElseIf out$ = "#DOUBLEQUOTE$"
     out$ + " = Chr(34)"
    ElseIf out$ = "#DQUOTE$"
     out$ + " = Chr(34)"
    ElseIf found_non_printable = #False
     out$ + " = " + #DQUOTE$ + StringField(oldout$,3,#TAB$) + #DQUOTE$
    Else
     out$ + " ="
     Protected temp$ = StringField(oldout$,3,#TAB$)
     For i = 0 To Len(sconst_value$)-1
      out$ + " Chr("+Str(PeekB(@temp$+(i*SizeOf(Character)))) + ") +"
     Next
    EndIf
    out$ = RTrim(out$,"+")
    out$ = Trim(out$)
   Else
    log + out$+#LF$
   EndIf
   out$ = Trim(out$)
   If out$
    AddElement(out())
    out()=space$+out$
   EndIf
  EndIf
 Wend
EndProcedure

Procedure GetStructureList(compiler,List OUT.s())
 If ProgramRunning(compiler)
  SendCompilerCommand(compiler,"STRUCTURELIST")
  FillList(compiler,out())
 EndIf
EndProcedure

Procedure GetProcedureList(compiler,List OUT.s())
 If ProgramRunning(compiler)
  SendCompilerCommand(compiler,"FUNCTIONLIST")
  FillList(compiler,out())
 EndIf
EndProcedure

Procedure GetConstantsList(compiler,List OUT.s())
 If ProgramRunning(compiler)
  SendCompilerCommand(compiler,"CONSTANTLIST")
  FillConstantList(compiler,out())
 EndIf
EndProcedure


Procedure GetInterfaceList(compiler,List OUT.s())
 If ProgramRunning(compiler)
  SendCompilerCommand(compiler,"INTERFACELIST")
  FillList(compiler,out())
 EndIf
EndProcedure

Procedure GetStructureInfo(compiler,struct$,List OUT.s())
 If ProgramRunning(compiler)
  SendCompilerCommand(compiler,"STRUCTURE"+#TAB$+struct$)
  FillList(compiler,out(),4)
 EndIf
EndProcedure

Procedure GetInterfaceInfo(compiler,interf$,List OUT.s())
 If ProgramRunning(compiler)
  SendCompilerCommand(compiler,"INTERFACE"+#TAB$+interf$)
  FillList(compiler,out(),4)
 EndIf
EndProcedure


Procedure WaitCompilerReady(compiler)
 Protected out$
 While out$<>"READY" And Left(out$,5)<>"ERROR"
  out$ = GetCompilerOutput(compiler)
  If out$
   log + out$+#LF$
  EndIf
 Wend
EndProcedure

Define  pb, out$
NewList constants.s()
NewList structures.s()
NewList procedures.s()
NewList interfaces.s()

NewList structureInfo.s()
NewList interfaceInfo.s()

pb = StartCompiler()
If pb
 WaitCompilerReady(pb)
 
 GetStructureList(pb,structures())
 log + "found "+Str(ListSize(structures()))+" structures"+#LF$
 
 GetProcedureList(pb,procedures())
 log + "found "+Str(ListSize(procedures()))+" procedures"+#LF$
 
 GetConstantsList(pb,constants())
 log + "found "+Str(ListSize(constants()))+" constants"+#LF$
 SortList(constants(),#PB_Sort_Ascending|#PB_Sort_NoCase)
 
 GetInterfaceList(pb,interfaces())
 log + "found "+Str(ListSize(interfaces()))+" interfaces"+#LF$
 
 ClearList(structureInfo())
 ForEach structures()
  AddElement(structureInfo())
  structureInfo()="Structure "+structures()
  GetStructureInfo(pb,structures(),structureInfo())
  AddElement(structureInfo())
  structureInfo()="EndStructure"
  AddElement(structureInfo())
  structureInfo()=""
 Next
 
 ClearList(interfaceInfo())
 ForEach interfaces()
  AddElement(interfaceInfo())
  interfaceInfo()="Interface "+interfaces()
  GetInterfaceInfo(pb,interfaces(),interfaceInfo())
  AddElement(interfaceInfo())
  interfaceInfo()="EndInterface"
  AddElement(interfaceInfo())
  interfaceInfo()=""
 Next
 
 If CreateFile(0,"pb"+#PB_Compiler_Version+" Structures.txt")
  ForEach structureInfo()
   WriteStringN(0,structureInfo())
  Next
  CloseFile(0)
 EndIf
 
 If CreateFile(0,"pb"+#PB_Compiler_Version+" Interfaces.txt")
  ForEach interfaceInfo()
   WriteStringN(0,interfaceInfo())
  Next
  CloseFile(0)
 EndIf
 
 If CreateFile(0,"pb"+#PB_Compiler_Version+" Constants.txt")
  ForEach constants()
   WriteStringN(0,constants())
  Next
  CloseFile(0)
 EndIf
 
 If CreateFile(0,"pb"+#PB_Compiler_Version+" Procedures.txt")
  ForEach procedures()
   WriteStringN(0,procedures())
  Next
  CloseFile(0)
 EndIf
 
 If CreateFile(0,"pb"+#PB_Compiler_Version+".log")
  WriteString(0,log)
  CloseFile(0)
 EndIf
 
 StopCompiler(pb)
 MessageRequester( "GetPBInfo","Done, open the folder to view the results.")
EndIf

Posted: **Tue Nov 29, 2022 7:17 pm**

vmars316 wrote: Mon Nov 28, 2022 10:51 pm What is the best/fastest structure to use , Arrays, Lists , etc. ?

Use a simple list:

Code: Select all

NewList functionsList$()

; Place here the code that fills the list.

If Not ReadFile(0, "codeFile.pb")
  Debug "Error"
  End
EndIf

; Skip BOM
ReadStringFormat(0)

While Not Eof(0)
  codeLine$ = ReadString(0)
  ForEach functionsList$()
    If FindString(codeLine$, functionsList$(), 1, #PB_String_NoCase)
      Debug codeLine$
      Break
    EndIf
  Next
Wend

CloseFile(0)

If that's too slow for your purposes, we can switch to the more complicated variants.

Posted: **Wed Nov 30, 2022 5:29 pm**

Thanks blueb ;
Yes , that works great !
I Posted the Results , if anyone is interested here :
http://vmars.us/purebasic/
Keywords-Constants-WORKING-Danilo.pb 2022-11-30 09:27 6.8K
pb600.log 2022-11-30 09:23 142
pb600 Constants.txt 2022-11-30 09:23 382K
pb600 Interfaces.txt 2022-11-30 09:23 773K
pb600 Procedures.txt 2022-11-30 09:23 147K
pb600 Structures.txt 2022-11-30 09:23 104K

PureBasic Forums - English

Which structure to use , Arrays, Lists & Structures ?

Which structure to use , Arrays, Lists & Structures ?

Re: Which structure to use , Arrays, Lists & Structures ?

Re: Which structure to use , Arrays, Lists & Structures ?

Re: Which structure to use , Arrays, Lists & Structures ?

Re: Which structure to use , Arrays, Lists & Structures ?

Re: Which structure to use , Arrays, Lists & Structures ?

Re: Which structure to use , Arrays, Lists & Structures ?

Re: Which structure to use , Arrays, Lists & Structures ?

Re: Which structure to use , Arrays, Lists & Structures ?

Re: Which structure to use , Arrays, Lists & Structures ?

Re: Which structure to use , Arrays, Lists & Structures ?

Re: Which structure to use , Arrays, Lists & Structures ?

Re: Which structure to use , Arrays, Lists & Structures ?