Page 1 of 2
EnableExplicit generator?
Posted: Sun Jul 31, 2022 12:56 am
by BarryG
So I've never used EnableExplicit before, but I was thinking today: my source code has now grown far too large to suddenly add it, as I'd have to manually go through 1.6 MB of source code (55880 lines) to add definitions to every single variable that my app uses.
This is obviously way too hard and too much manual work for such a large source, so has anyone ever coded an EnableExplicit generator to do this automatically to a given source file?
Re: EnableExplicit generator?
Posted: Sun Jul 31, 2022 8:35 am
by STARGÅTE
What should this generator do?
What is the problem, to add
EnableExplicit at the beginning of the file?
When you mean a generator, which automatically define all undefined variables, then the sense of using EnableExplicit is obsolete.
EnableExplicit can be used to find spelling bugs like:
Code: Select all
Procedure MyProcedure(ParameterWithASpellingBug.i)
ProcedureReturn ParameterWitnASpellingBug * 20
EndProcedure
Debug MyProcedure(10)
With such a generator you end up with this, without showing you the spelling issue:
Code: Select all
EnableExplicit
Procedure MyProcedure(ParameterWithASpellingBug.i)
Protected ParameterWitnASpellingBug.i
ProcedureReturn ParameterWitnASpellingBug * 20
EndProcedure
Debug MyProcedure(10)
BarryG wrote: Sun Jul 31, 2022 12:56 am
my source code has now grown far too large to suddenly add it, as I'd have to manually go through 1.6 MB of source code (55880 lines) to add definitions to every single variable that my app uses.
If you haven't used explicite definitions until now, why do you care about EnableExplicit now?
Re: EnableExplicit generator?
Posted: Sun Jul 31, 2022 10:13 am
by BarryG
Yes, I meant a generator, which automatically define all undefined variables. And no, it won't make EnableExplicit obsolete because all my variables are valid at the the time the generator would be used, so there's no situation of typos becoming valid generated names.
The idea is to make all variables explicit at the time the generator is done, so that future variable names can be defensively coded thereafter. The generator isn't something that you'd run every day on the same source, but only once, to clean it up.
Re: EnableExplicit generator?
Posted: Sun Jul 31, 2022 11:16 am
by STARGÅTE
BarryG wrote: Sun Jul 31, 2022 10:13 am
And no, it won't make EnableExplicit obsolete because all my variables are valid at the the time the generator would be used, so there's no situation of typos becoming valid generated names.
Why you are sure that all your variable names are "valid" if you havn't used EnableExplicit at the moment?
Wrong definitions or spelling issue do not always and up in syntax errors or calculation issues.
Whatever, my answers/question are slightly off-topic.
So, I havn't such a generator. There are some generators for creating procedure declarations at one shot, but I don't know if they also create Protected or Define keywords.
Re: EnableExplicit generator?
Posted: Sun Jul 31, 2022 12:41 pm
by BarryG
STARGÅTE wrote: Sun Jul 31, 2022 11:16 amWhy you are sure that all your variable names are "valid" if you havn't used EnableExplicit at the moment?
Re: EnableExplicit generator?
Posted: Sun Jul 31, 2022 2:00 pm
by Oso
BarryG wrote: Sun Jul 31, 2022 10:13 am
Yes, I meant a generator, which automatically define all undefined variables. And no, it won't make EnableExplicit obsolete because all my variables are valid at the the time the generator would be used, so there's no situation of typos becoming valid generated names.
I often write code to modify my source code, in cases where I need to make the same change to multiple programmes, but that's not in PB, it's in PICK DataBasic (which I've been using practically my entire life
), but I fully understand your motivation in wanting to do this. It can save an enormous amount of time. There's no easy way, in my view, because your automatic routine is going to need to identify variable names within all the various lines of code and make a list of them. It's then going to need to make a determination regarding the variable type and then generate some form of output for you, either in the form of a list of all the variable names that it has found, which presumably aren't already defined, or alternatively add them automatically into the top of the code itself.
If it's any help though, I noticed (as a very new user of PureBasic) that if you add EnableExplicit to a routine in which it wasn't already set, that the compiler will throw an error anyway, until you define each variable. So it's not as bad as having a routine that compiles successfully but fails under certain conditions when it is executed.
I would say that if your code is mature and is well written, despite not having EnableExplicit, there isn't a strong argument for changing what already works well. I guess it's more of a personal desire to improve it, but it doesn't seem worthwhile. I suppose one way of doing it, is that whenever you work on an existing routine, improve it at that point and work through the system in that way. Just my thoughts anyway.
Re: EnableExplicit generator?
Posted: Sun Jul 31, 2022 2:13 pm
by #NULL
In case you don't know: There is DisableExplicit as well. So you can convert only part of your code at times. Or you start at the bottom of your files, moving EnableExplicit up in chunks to convert new parts.
Re: EnableExplicit generator?
Posted: Sun Jul 31, 2022 7:27 pm
by Oso
I have written a source code analyser which may help @BarryG 's requirement, the purpose being to look through the code and pick out all the variable names used. It builds these into a sorted list, then outputs them at the end. It would be easy to modify this to write "Define" statements for each variable. This is the first programme I've written with PureBasic and consequently my knowledge of the language is very limited. It's likely that it doesn't cover all relevant statements that assign values to variables. Nevertheless, it looks for the following:
Code: Select all
variable = 'x'
For variable = 'x' to 'y'
It properly ignores:
Code: Select all
Define variable = 'x'
#constant = 'x'
If variable = 'x'
The code is fully commented, which should make it easier for others to add missing PureBasic statements that might be responsible for assigning values to variables, that I'm unfamiliar with.
Code: Select all
OpenConsole()
EnableExplicit
#ENABLE_EXPLICIT = "EnableExplicit"
Define filename.s ; File path of source code file to process
Define text.s ; Input text line read from file
Define recno ; Input record counter to show serial no. of source line
Define length ; Input line length, before removing multiple spaces
Define newstring.s ; Input line cleaned up, by remocing multiple unwanted spaces
Define varname.s ; Extracted variable name from the source line
Define stpos ; Input line character count
Define lastchr.s ; Previous character processed from input line, to detected multiple spaces
Define chr.s ; Input character read one-by-one
Define expfound ; Flag to indicate "EnableExplicit" was found in the source item
Define pos ; Character position when searching for specific symbols in input line
Define duped ; Flag to indicate that variable is already in the output list
Define outcount ; Output variable counter
NewList varlist.s() ; List for building each variable name found
NewList vardupe.s() ; List for de-duping the varlist.s list
Print ("Please enter the file path of the source file you want to process (or press Enter for default) : ")
filename.s = Input()
If filename.s = ""
filename.s="C:\test\source_util.pb"
EndIf
If ReadFile(1,filename.s)
While Not(Eof(1))
text.s = ReadString(1) ; Read the input line
; \\\
; \\\ Remove trailing semi-colons from input lines, retaining only the code which precedes it
; \\\
pos=FindString(text.s, ";") ; Locate semi-colon (marker for comments, which can trail on end of code)
If pos; ; Found a semi-colon
text.s=Left(text.s,pos-1) ; Retain all text up to the semi-colon, discard everything from semi-colon onwards
EndIf
; \\\
; \\\ Determine line length and reset working variables
; \\\
length=Len(text.s)
newstring.s = ""
lastchr.s=" "
; \\\
; \\\ Process the input line and look for repeating spaces, removing the unwanted spaces to maintain consistency. Write the cleaned-up line
; \\\ to the new variable newstring.s
; \\\
For stpos=1 To length ; Go through each character in the input line
chr.s=Mid(text.s,stpos,1) ; Extract each character in turn
If (chr.s <> " " Or lastchr.s <> " ") ; If character is not a space, or the previous character was not a space, include it
newstring.s=newstring.s+chr.s ; Include the character in the 'newstring.s' output
EndIf
lastchr.s=chr.s ; Save this character for the next iteration in the loop
Next stpos
recno=recno+1 ; Increment the record counter, if only to provide the user with a running count of lines
Debug Str(recno)+" : "+newstring.s ; Show the user the newly amended line (i.e. the line with unwanted spacing removed)
If CountString(newstring.s, #ENABLE_EXPLICIT) ; If we encounter EnableExplicit, set to flag to indicate as such
expfound=#True ; and the user can therefore choose to ignore those source files which already contain it
EndIf
If Left(newstring.s,3) <> "If " And Left(newstring.s,1) <> "#" ; Ignore 'If' statements and constants, since we're not interested in those vars.
; but other commands can be added to this list, as necessary
If Left(newstring.s,4) = "For " ; Special case for 'For' statement, since the variable follows the statement
newstring.s=Mid(newstring.s,5) ; In this case, remove 'For' since we don't want this to form part of var. name
EndIf
If Left(newstring.s,7) <> "Define " ; Special case for 'Define' with equate of an initialised value, since these
; presumably fulfil our requirement of being defined, hence we
; don't need to know about these
pos=FindString(newstring.s, "=") ; Look for '=' assignment in the line, so we can determine the variable name preceding it
If pos>1 ; If we find an '=' symbol, process the line
If Mid(newstring.s,pos-1,1)=" " ; If the previous character before the '=' symbol is a space, adjust our pointer to ignore it
pos=pos-1 ; because we don't want the space in the variable name
EndIf
varname.s=Left(newstring.s,pos-1) ; Extract the variable name preceding the '=' symbol
Debug " "
Debug varname.s+" IS A VARIABLE" ; Show the user the extracted variable name, so this can be checked in the source if req'd.
Debug " "
AddElement(varlist.s()) ; Create a new list entry in varlist.s()
varlist.s() = varname.s ; Store the extracted variable name in list varlist.s()
EndIf
EndIf
EndIf
Wend
Else
Debug "Cannot open the file specified"
End
EndIf
Debug " "
If expfound
Debug "This source code file already contains "+ #ENABLE_EXPLICIT
Else
Debug "This source code file DOES NOT contain "+ #ENABLE_EXPLICIT
EndIf
; \\\
; \\\ Now we have our list of variable names found in varlist.s(), go through each of them and de-dupe them against a new list vardupe.s(), so we
; \\\ end up with the list of unique variable names
; \\\
ResetList(varlist.s()) ; Reset the varlist.s() point to the beginning of the list
While NextElement(varlist.s()) ; Go through each list element
duped=#False
ResetList(vardupe.s()) ; Reset the de-duped output list vardupe.s() so we can check every entry in there
While NextElement(vardupe.s()) ; Go through the output de-duped list
If vardupe.s() = varlist.s() ; Check if the input list element varlist.s() already exists in the de-duped list
duped=#True ; It's already there, so we don't need to store it again
EndIf
Wend
If Not(duped) ; Is it a duplication?
AddElement(vardupe.s()) ; Not a duplication, so we create a new element in vardupe.s()
vardupe.s() = varlist.s() ; Store the variable in the de-duped list vardupe.s()
EndIf
Wend
; \\\
; \\\ Sort the de-duped output list in alphabetical sequence
; \\\
SortList(vardupe(),#PB_Sort_Ascending)
Debug "These are the variables found : "
; \\\
; \\\ Show the complete list to the user
; \\\
ResetList(vardupe.s())
While NextElement(vardupe.s())
outcount=outcount+1
Debug Str(outcount)+" : "+vardupe.s()
Wend
Re: EnableExplicit generator?
Posted: Sun Jul 31, 2022 7:40 pm
by Oso
Incidentally, a good 'first test' of this is to run the routine on its own source code file. Without wishing to include the full result, the tail-end of the output will be as follows...
Code: Select all
135 :
136 :
137 : ResetList(vardupe.s())
138 : While NextElement(vardupe.s())
139 : outcount=outcount+1
outcount IS A VARIABLE
140 : Debug Str(outcount)+" : "+vardupe.s()
141 : Wend
This source code file already contains EnableExplicit
These are the variables found :
1 : chr.s
2 : duped
3 : expfound
4 : filename.s
5 : lastchr.s
6 : length
7 : newstring.s
8 : outcount
9 : pos
10 : recno
11 : stpos
12 : text.s
13 : vardupe.s()
14 : varlist.s()
15 : varname.s
Re: EnableExplicit generator?
Posted: Sun Jul 31, 2022 9:57 pm
by BarryG
Thanks Oso! That's the type of practical help I was looking for. I'll try it soon.
Re: EnableExplicit generator?
Posted: Sun Jul 31, 2022 11:22 pm
by Oso
BarryG wrote: Sun Jul 31, 2022 9:57 pm
Thanks Oso! That's the type of practical help I was looking for. I'll try it soon.
We might need to refine it further, because it's designed for the code as I would write it. PB is still a bit of an unknown science to me!
Re: EnableExplicit generator?
Posted: Mon Aug 01, 2022 3:47 am
by TassyJim
I wrote a variable report for another much simpler Basic as part of an IDE. It was also written in something other than PureBasic.
It would not transfer easily to testing PureBasic but one of my early "gotchas" was handling quoted text in the code.
Code: Select all
pos=FindString(text.s, ";") ; Locate semi-colon (marker for comments, which can trail on end of code)
If pos; ; Found a semi-colon
text.s=Left(text.s,pos-1) ; Retain all text up to the semi-colon, discard everything from semi-colon onwards
EndIf
The above section of Oso's code will stop at any ; that is in quotes. Any strings in quotes should be stepped over when scanning the line.
My flow was
Scan the file and make a list of all procedures.
Scan again and find each 'word' and test to see if it a procedure form the earlier list or a language keyword.
If it's not in either list, assume that it is a variable.
Check to see if it has been found already.
You will have to handle within procedures as individual entities.
You will also have to include any Includes in the scan for procedures
I couldn't live without enableexplicit.
My left hand doesn't do what I tell it to do any longer, so lots of spelling errors.
Jim
Re: EnableExplicit generator?
Posted: Mon Aug 01, 2022 12:50 pm
by Oso
TassyJim wrote: Mon Aug 01, 2022 3:47 am
Code: Select all
pos=FindString(text.s, ";") ; Locate semi-colon (marker for comments, which can trail on end of code)
If pos; ; Found a semi-colon
text.s=Left(text.s,pos-1) ; Retain all text up to the semi-colon, discard everything from semi-colon onwards
EndIf
The above section of Oso's code will stop at any ; that is in quotes. Any strings in quotes should be stepped over when scanning the line.
These things are easy to do, but I think there's limited value in attempting to take this to the n'th degree, since trying to do so makes it just as sophisticated as the compiler's own parser. In fact many compilers provide the option to do this anyway - it's usually known as the 'symbol table'
I have added those you mention anyway and the complete code is below...
1. Contents of double-quote pairs are removed, so reserved words or special characters within quotes are ignored
2. Procedure names are included within the table of variable names, thus when the variable names are sorted, they appear in order of parent procedure and the individual procedure can be identified
Code: Select all
OpenConsole()
EnableExplicit
#ENABLE_EXPLICIT = "EnableExplicit"
#CHR34 = Chr(34)
Define filename.s ; File path of source code file to process
Define text.s ; Input text line read from file
Define recno ; Input record counter to show serial no. of source line
Define length ; Input line length, before removing multiple spaces
Define newstring.s ; Input line cleaned up, by remocing multiple unwanted spaces
Define varname.s ; Extracted variable name from the source line
Define stpos ; Input line character count
Define lastchr.s ; Previous character processed from input line, to detected multiple spaces
Define chr.s ; Input character read one-by-one
Define expfound ; Flag to indicate "EnableExplicit" was found in the source item
Define pos ; Character position when searching for specific symbols in input line
Define duped ; Flag to indicate that variable is already in the output list
Define outcount ; Output variable counter
Define lastproc.s ; Last procedure name
Define quotemark ; Quote mark toggle flag
NewList varlist.s() ; List for building each variable name found
NewList vardupe.s() ; List for de-duping the varlist.s list
Print ("Please enter the file path of the source file you want to process (or press Enter for default) : ")
filename.s = Input()
If filename.s = ""
filename.s="C:\test\source_util.pb"
EndIf
If ReadFile(1,filename.s)
While Not(Eof(1))
text.s = ReadString(1) ; Read the input line
; \\\
; \\\ Determine line length and reset working variables
; \\\
length=Len(text.s)
newstring.s = ""
lastchr.s=" "
; \\\
; \\\ Process the input line and look for repeating spaces, removing the unwanted spaces to maintain consistency. Write the cleaned-up line
; \\\ to the new variable newstring.s
; \\\
For stpos=1 To length ; Go through each character in the input line
chr.s=Mid(text.s,stpos,1) ; Extract each character in turn
If (chr.s <> " " Or lastchr.s <> " ") ; If character is not a space, or the previous character was not a space, include it
newstring.s=newstring.s+chr.s ; Include the character in the 'newstring.s' output
EndIf
lastchr.s=chr.s ; Save this character for the next iteration in the loop
Next stpos
; \\\
; \\\ Remove text between double quotes to eliminate problems in the event that reserved words or characters processed by this routine happen to
; \\\ be within double quotes
; \\\
quotemark=#False; ; Reset quote mark 'active' flag
length=Len(newstring.s) ; Length of processed line
For stpos=1 To length ; Go through each character in the resulting processed line
chr.s=Mid(newstring.s,stpos,1) ; Extract each character in turn
If chr.s = #CHR34 ; Look for double quote mark
If quotemark=#True ; We're already inside a quote pair, so turn off the quote flag, since we've reached
quotemark=#False ; the end of the pair
Else
quotemark=#True ; First of the quote pair so turn on the flag
EndIf
EndIf
If quotemark=#True And chr.s <> #CHR34 ; If we're inside a quote pair and this is not the actual quote, then change to a space
newstring.s=Left(newstring.s,stpos-1) + " " + Mid(newstring.s,stpos+1)
EndIf
Next stpos
; \\\
; \\\ Remove trailing semi-colons from input lines, retaining only the code which precedes it
; \\\
pos=FindString(newstring.s, ";") ; Locate semi-colon (marker for comments, which can trail on end of code)
If pos; ; Found a semi-colon
newstring.s=Left(newstring.s,pos-1) ; Retain all text up to the semi-colon, discard everything from semi-colon onwards
EndIf
recno=recno+1 ; Increment the record counter, if only to provide the user with a running count of lines
Debug Str(recno)+" : "+newstring.s ; Show the user the newly amended line (i.e. the line with unwanted spacing removed)
If CountString(newstring.s, #ENABLE_EXPLICIT) ; If we encounter EnableExplicit, set the flag to indicate as such
expfound=#True ; and the user can therefore choose to ignore those source files in which EnableExplicit
EndIf ; already appears
If Left(newstring.s,10)="Procedure " ; If we meet a Procedure section, record the name so we can associate variables with it
lastproc.s=Mid(newstring.s,11); ; Extract the procedure name
pos=FindString(lastproc.s, "(") ; Find the opening parenthesis so we know where the name ends
If pos
lastproc.s=Left(lastproc.s,pos-1) ; Extract the procedure name, up to the character before opening parenthesis
EndIf
EndIf
If Left(newstring.s,12)="EndProcedure" ; For EndProcedure, drop the last procedure name, so that any further variables are
lastproc.s="" ; associated without a procedure name (the main programme), until we subsequently find
EndIf ; another procedure
If Left(newstring.s,3) <> "If " And Left(newstring.s,1) <> "#" ; Ignore 'If' statements and constants, since we're not interested in those vars.
; but other commands can be added to this list, as necessary
If Left(newstring.s,4) = "For " ; Special case for 'For' statement, since the variable follows the statement
newstring.s=Mid(newstring.s,5) ; In this case, remove 'For' since we don't want this to form part of var. name
EndIf
If Left(newstring.s,6) <> "Define" ; Special case for 'Define' with equate of an initialised value, since these
; presumably fulfil our requirement of being defined, hence we
; don't need to know about these
pos=FindString(newstring.s, "=") ; Look for '=' assignment in the line, so we can determine the variable name preceding it
If pos>1 ; If we find an '=' symbol, process the line
If Mid(newstring.s,pos-1,1)=" " ; If the previous character before the '=' symbol is a space, adjust our pointer to ignore it
pos=pos-1 ; because we don't want the space in the variable name
EndIf
varname.s=Left(newstring.s,pos-1) ; Extract the variable name preceding the '=' symbol
Debug " "
If lastproc.s <> "" ; Show the user the extracted variable name, so this can be checked in the source if
Debug lastproc.s + "/" + varname.s+" IS A VARIABLE" ; req'd. also add the procedure name, so we can see the name of the procedure with which
Else ; the variable is associated
Debug varname.s+" IS A VARIABLE"
EndIf
Debug " ";
AddElement(varlist.s()) ; Create a new list entry in varlist.s()
If lastproc.s <> ""
varname.s=lastproc.s + " / " + varname.s ; Prefix with procedure name
EndIf
varlist.s() = varname.s ; Store the extracted variable name in list varlist.s()
EndIf
EndIf
EndIf
Wend
Else
Debug "Cannot open the file specified"
End
EndIf
Debug " "
If expfound
Debug "This source code file already contains "+ #ENABLE_EXPLICIT
Else
Debug "This source code file DOES NOT contain "+ #ENABLE_EXPLICIT
EndIf
; \\\
; \\\ Now we have our list of variable names found in varlist.s(), go through each of them and de-dupe them against a new list vardupe.s(), so we
; \\\ end up with the list of unique variable names
; \\\
ResetList(varlist.s()) ; Reset the varlist.s() point to the beginning of the list
While NextElement(varlist.s()) ; Go through each list element
duped=#False
ResetList(vardupe.s()) ; Reset the de-duped output list vardupe.s() so we can check every entry in there
While NextElement(vardupe.s()) ; Go through the output de-duped list
If vardupe.s() = varlist.s() ; Check if the input list element varlist.s() already exists in the de-duped list
duped=#True ; It's already there, so we don't need to store it again
EndIf
Wend
If Not(duped) ; Is it a duplication?
AddElement(vardupe.s()) ; Not a duplication, so we create a new element in vardupe.s()
vardupe.s() = varlist.s() ; Store the variable in the de-duped list vardupe.s()
EndIf
Wend
; \\\
; \\\ Sort the de-duped output list in alphabetical sequence
; \\\
SortList(vardupe(),#PB_Sort_Ascending)
Debug "These are the variables found : "
; \\\
; \\\ Show the complete list to the user
; \\\
ResetList(vardupe.s()) ; Start at the beginning of the de-duplicated list
While NextElement(vardupe.s())
outcount=outcount+1
Debug Str(outcount)+" : "+vardupe.s() ; Output the numeric counter and the variable name (note that variable names can be
Wend ; prefixed with the procedure name (if found) followed by a '/'
Re: EnableExplicit generator?
Posted: Mon Aug 01, 2022 1:00 pm
by Oso
Just to give an example of what to expect. Someone else's code I'm looking at gives this output and we can see how it has identified a set of variables associated with the procedure 'OpenImage', whereas the other two (x and y) are in the main body.
Code: Select all
41 : Procedure OpenImage()
42 : Define.s filePath$
43 : filePath$=OpenFileRequester(" ",
OpenImage/filePath$ IS A VARIABLE
59 : imageWidth = ImageWidth(#Image_Copy)
OpenImage/imageWidth IS A VARIABLE
60 : imageHeight = ImageHeight(#Image_Copy)
OpenImage/imageHeight IS A VARIABLE
This source code file already contains EnableExplicit
These are the variables found :
1 : OpenImage / filePath$
2 : OpenImage / imageHeight
3 : OpenImage / imageWidth
4 : x
5 : y
Re: EnableExplicit generator?
Posted: Mon Aug 01, 2022 1:58 pm
by blueb
BarryG wrote: Sun Jul 31, 2022 12:56 am
So I've never used EnableExplicit before, but I was thinking today: my source code has now grown far too large to suddenly add it, as I'd have to manually go through 1.6 MB of source code (55880 lines) to add definitions to every single variable that my app uses.
This is obviously way too hard and too much manual work for such a large source, so has anyone ever coded an EnableExplicit generator to do this automatically to a given source file?
BarryG
SnifftheGlove wrote something a few years ago that might assist you...
https://www.purebasic.fr/english/viewto ... 41#p517741