Trimmer : remove unused procedures

[maker]

I've submitted a utility (with source) to Andre to see if he thinks it could be on PureArea.net in the developer tools section.

The program, called Trimmer, creates a single file version of your source + includes with unused procedures removed. I'm a recently registered user of PB, and I was unable to find a standalone version of this sort of thing, so...I did what we all do, I wrote it myself. I've used it on over fifty programs to test it (part of a large example code conversion I'm doing), including its own source, and it seems to work fine. I use a batch file to call it first, then call pbcompiler with its output; I use a 3rd party editor, but I assume the PB IDE 'trigger' facility could do something similiar.

Anyway, I saw a number of other people's comments about wanting this kind of program, so here it is, or rather there it will be, hopefully soon.

EDIT 1: I jumped the gun on submitting it; the first version only did one pass, so it missed some things. Version 1.1 (now also submitted to Andre at PureArea) continues to go over the file until it makes a pass with no unreferenced procs found.

EDIT 2: Andre advised me it may be a while before he updates the dev tools area, so I guess I'll be posting the prog elsewhere; I don't know where yet. And yes, I know about the showcase, but it does not allow me to upload the file directly, it seems to only allow linking offsite, so I still have to find a place to put the program. I'll update this when I do.

EDIT 3:Never did come back and put the link up here, so here it is:http://programmingforfun.net/trimmer/trimmer.htm

[viiartz]

Sound useful...I would be interested in the utility.

[Michael Vogel]

Some options woul be fine, like...

[_] removing unused procedures
[_] removing unused variable definitions

[_] just do a report without removing anything

[maker]

Some options woul be fine, like...

[_] removing unused procedures
[_] removing unused variable definitions

[_] just do a report without removing anything

I agree; I'll try to stick those in the next version.

I believe I have the hosting situation sorted out, so I should be able to post a link to the prog today or early tomorrow.

[maker]

Trimmer now has a home! You can download it at

http://www.programmingforfun.net/trimmer/trimmer.htm

[Seymour Clufley]

Hi Maker,

I've gone to your site but it seems to be offline???

SC.

[maker]

Hi Maker,

I've gone to your site but it seems to be offline???

SC.

Not sure what's happening with that; it seems to be working from here. Maybe try again? If it still doesn't work, could you post the URL you're using?

Anybody else having trouble with the site?

[Seymour Clufley]

I meant to thank you for PM'ing me about your Trimmer program. That was very kind of you.

I'm just using the URL you gave. I've shortened it to just the domain name as well. Neither works. I get a Google page saying the site is offline. If I go to Google's cache of the page and try to download the Trimmer exe, it doesn't work.

Hope you get it sorted, as I'd like to try Trimmer!

All the best,
SC.

[Michael Vogel]

The home page works for me, just downloaded Trimmer now

[maker]

I meant to thank you ...

You're welcome!

I'm just using the URL you gave. I've shortened it to just the domain name as well. Neither works.

Hm. I'm stumped. The only thing I can think of is maybe an unusual lag in the domain name getting spread around all the name servers; the site hasn't existed for very long. If that's the case, it should start working for you soon.

If not, I'll be glad to email it.

Michael, I'm glad to hear it worked for you.

If anybody else has problems, by all means let me know; I'd like to get this fixed.

[Seymour Clufley]

IE says it's a 403 error.

[maker]

IE says it's a 403 error.

Do you get the 403 error trying to access the site at all, or just when
downloading the file? If the latter, your browser/proxy may have the referrer field blocked; I have some .htaccess mod_rewrite rules set up that would cause a 403 on attempts to download the file without the referrer pointing to one of the site pages.

[maker]

that would cause a 403 on attempts to download the file without the referrer pointing to one of the site pages.

Wait, no, I just checked and it turns out I already changed those rules; the current behavior is to redirect to the trimmer html page if the file is requested with a blank referrer field. So it still shouldn't trigger a 403.

I also just realized I was getting too focused on the web site issue and not thinking about the fact that I could just post the source here. I'll do that later today.

[maker]

The source for Trimmer 1.1 is at the end of this post. It depends on several chunks of code from srod.

The file named srod_merge.pbi came from this post:
http://www.purebasic.fr/english/viewtop ... ource+file

Note: I changed the line:

Code: Select all

XIncludeFile "stackClass.pbi"

to:

Code: Select all

XIncludeFile "srod_stackClass.pbi"

The files named "srod_stackClass.pbi" and "srod_stackClass_Residents.pbi" came from here:
http://www.purecoder.net/stack.zip

They do not have the srod_ prefix in the archive; I added that and made the necessary include
file changes to be sure srod got proper credit (after all, his code is doing most of the work,
mine is just a hack sitting on top of it).

In the file "srod_stackClass.pbi", I changed:

Code: Select all

XIncludeFile "stackClass_Residents.pbi"

to

Code: Select all

XIncludeFile "srod_stackClass_Residents.pbi"

Code: Select all

; Trimmer 1.1 - made it multipass
; by maker
; trimmer1.1.pb
DataSection
    usageText:
    Data.s "=============="
    Data.s "= Trimmer 1.1 "
    Data.s "=============="
    Data.s "Takes your source code set and creates two files from it:         "
    Data.s "      1) a merged file containing all code in your main file +    "
    Data.s "         all included file text                                   "
    Data.s "      2) a merged-and-trimmed file, same as #1 but with unused    "
    Data.s "         procedures replaced by comments.                         "
    Data.s "By compiling the 2nd file, you should get much smaller executables"
    Data.s "(if you had a lot of library code in the include files).          "
    Data.s " "
    Data.s "    Usage: "
    Data.s "    trimmer mainFileName.pb"
    Data.s " "
    Data.s "    Credits: "
    Data.s "      Trimmer uses stack and file merge code by Stephen Rodriguez (srod)."
    Data.s "      The rest can be blamed on Craig Gilbert (maker)."
EndDataSection
Macro USAGETEXTLINES: 17 : EndMacro

Macro extensionExtension : "pp"  : EndMacro
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; support routines
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
Procedure showUsageAndExit()
    Define outtext.s
    Restore usageText
    For i = 1 To USAGETEXTLINES
        Read.s outtext
        PrintN(outtext);
    Next i
    End 1
EndProcedure

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; main program
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
If Not OpenConsole()
    End 1
EndIf
;
; get target file from command line
;
If Not CountProgramParameters()
    showUsageAndExit()
EndIf
Define mainFile.s
;
; this is pretty crude but it gets around a problem With 3rd party editors and
; batch files causing nested double quoted file names. Note that it also
; means this prog can only have one command line arg.
For i = 1 To CountProgramParameters()
    mainFile = mainFile + ProgramParameter()
Next i
Define temp.s
For i = 1 To Len(mainFile)
    If Mid(mainFile, i, 1) <> Chr(34)
        temp = temp + Mid(mainFile,i,1)
    EndIf
Next i
mainFile = temp

;
; get cur dir
;
Define homeDir.s = GetCurrentDirectory()

;
; build one large file from main file + included files
;
XIncludeFile "srod_merge.pbi"
Define mergedFilename.s = mainFile+"MERGED"
Define retval = MergePBSources(mainFile, mergedFilename)
If retval <> #MPBS_OKAY
    Select retval
        Case #MPBS_INVALIDSOURCEFILE
            PrintN("file merger says: MPBS_INVALIDSOURCEFILE:")
            PrintN("                  " + mainFile)
        Case #MPBS_INSUFFICIENTMEMORY
            PrintN("file merger says: MPBS_INSUFFICIENTMEMORY")
        Case #MPBS_FILEOPENERROR
            PrintN("file merger says: MPBS_FILEOPENERROR, one of these:")
            PrintN("                  " + mainFile)
            PrintN("                  " + mergedFilename)
        Case #MPBS_INVALIDPATH
            PrintN("file merger says: MPBS_INVALIDPATH")
        Case #MPBS_INVALIDINCLUDEFILE
            PrintN("file merger says: MPBS_INVALIDINCLUDEFILE")
        Case #MPBS_INCLUDEFILENOTFOUND ;In this case, gMPBS_Include$ will contain the path\filename
                                       ;of the file which cannot be located.
            PrintN("file merger says: MPBS_INCLUDEFILENOTFOUND:")
            PrintN("                  " + gMPBS_Include$)
        Case #MPBS_NESTEDINCLUDESOVERFLOW  ;Probably a recursive use of a certain include file.
            PrintN("file merger says: MPBS_NESTEDINCLUDESOVERFLOW")
    EndSelect
    End 1
EndIf
;
; get merged file
;
Define f = ReadFile(#PB_Any, mergedFilename)
If Not f
    PrintN("Unable to open merged file: " + mergedFilename)
    End 1
EndIf

Define textLine.s
NewList lineList.s()
While Not Eof(f)
    textLine = ReadString(f)
    AddElement(lineList())
    lineList() = textLine
Wend

CloseFile(f)

If ListSize(lineList()) < 1
    PrintN("No merged file available!")
    End 1
EndIf
;
; purge unused procedures
;
Structure procstruct
    name.s
    used.l
    *start.procstruct    ; buildProcList() must set this
EndStructure
Procedure buildProcList(List lines.s(), List procList.procstruct())
; expects procList to be empty.
    Define l.s
    ResetList(lines())
    For i = 1 To ListSize(lines())
        If Not NextElement(lines())
            PrintN("Error in buildProcList()! Linked list lines() error!")
            End 1
        EndIf
        l = lines()
        found = FindString(LCase(l), "procedure", 1)
        If found
            Define ch.s = ""
            For x = 1 To found-1   ; make sure nothing before 'Procedure' except spaces or tabs
                ch = Mid(l,x,1)
                If ch <> " " And ch <> Chr(9)
                    Break
                EndIf
            Next x
            If ch <> " " And ch <> Chr(9) And ch <> ""
                Continue
            EndIf

            If ch = " " Or ch = Chr(9) Or ch = "" ; is procedure start line
                ;
                ; get name
                ;
                found + 9
                If found > Len(l):  PrintN("Line ended too early! Merged code Line " + Str(i)) : End 1 : EndIf
                ch = Mid(l,found,1)
                If ch <> " " And ch <> Chr(9) And ch <> "."  ; Not a procedure
                    Continue
                EndIf
                If ch = "."
                    While ch <> " " And ch <> Chr(9) ; roll past procedure return type
                        found + 1
                        If found > Len(l):  PrintN("Line ended too early! Merged code Line " + Str(i)) : End 1 : EndIf
                        ch = Mid(l,found,1)
                    Wend
                EndIf
                ch = Mid(l,found,1)
                While ch = " " Or ch = Chr(9)  ; roll past spaces and tabs
                    found + 1
                    ch = Mid(l,found,1)
                Wend
                AddElement(proclist())
                While ch <> "(" And ch <> " " And ch <> Chr(9)
                    procList()\name + ch
                    found + 1
                    ch = Mid(l,found,1)
                Wend
                ;
                ; get line location for quick return later
                ;
                procList()\start = @lines()
            EndIf
        EndIf
    Next i
EndProcedure

Procedure isIdentifierChar(ch.c)
    If (ch >= '0' And ch <= '9') Or (ch >= 'A' And ch <= 'Z') Or (ch >= 'a' And ch <= 'z') Or ch = '_' Or ch = '$'
        ProcedureReturn 1
    Else
        ProcedureReturn 0
    EndIf
EndProcedure

Procedure declarationLine(l.s)
    Define found = FindString(LCase(l), "declare",1)
    If Not found : ProcedureReturn 0 : EndIf

    If found > 1
        Define ch.l = Asc(Mid(l,found-1,1))
        If ch <> ' ' And ch <> 9
            ProcedureReturn 0
        EndIf
    EndIf
    found + 7
    If found > Len(l): ProcedureReturn 0 : EndIf
    ch = Asc(Mid(l,found,1))
    If ch <> ' ' And ch <> 9
        ProcedureReturn 0
    EndIf

    ProcedureReturn 1
EndProcedure

Macro NORMAL: 1: EndMacro
Macro INQUOTES: 0: EndMacro
Procedure markProcs(List lines.s(), List procList.procstruct())
; look for procedure references on the line l
    Define ch.s, identifierBuffer.s, mode.l = NORMAL, l.s = lines()
    For i = 1 To Len(l)
        ch = Mid(l, i, 1)
        If Asc(ch) = '"'
            If mode = NORMAL
                mode = INQUOTES
            Else
                mode = NORMAL
            EndIf
        Else
            If mode = NORMAL
                If Asc(ch) = ';' ; no need to look further on this line
                    Break
                ElseIf isIdentifierChar(Asc(ch))
                    identifierBuffer + ch
                ElseIf ch = "("  ;; ding ding ding! we may have a procedure reference

                    If Len(identifierBuffer) > 0
                        ResetList(procList())           ;; loop through all proc names
                        For x = 1 To ListSize(procList())
                            If Not NextElement(procList())
                                PrintN("Error in markProcs()! Linked list procList() error!")
                                End 1
                            EndIf
                            If LCase(procList()\name) = LCase(identifierBuffer)
                                If procList()\start <> @lines()  ; not the procedure definition line
                                    If Not declarationLine(l)
                                        procList()\used + 1
                                        Break
                                    EndIf
                                EndIf
                            EndIf
                        Next x
                        identifierBuffer = ""
                    EndIf
                Else
                    identifierBuffer = ""
                EndIf
            EndIf
        EndIf
    Next i
    If Len(identifierBuffer) > 0
            ResetList(procList())
            For x = 1 To ListSize(procList())
                If Not NextElement(procList())
                    PrintN("Error in markProcs()! Linked list procList() error!")
                    End 1
                EndIf
                If procList()\name = identifierBuffer
                    procList()\used + 1
                    Break
                EndIf
            Next x
    EndIf
EndProcedure

Procedure blankToEndProcedure(List lines.s(), List procList.procstruct())
    For i = ListIndex(lines()) To ListSize(lines())
        Define found.l = FindString(LCase(lines()), "endprocedure",1)
        lines() = "; unused Procedure " + procList()\name + "() was here"
        If found
            Break
        EndIf
        If Not NextElement(lines())
            PrintN("Error in blankToEndProcedure()! Linked list lines() error!")
            End 1
        EndIf
    Next i
EndProcedure

Procedure purgeUnusedProcedures(List lines.s())
    NewList procList.procstruct()
    buildProcList(lines(), procList())
    ;
    ; check each line for procedure references
    ;
    ResetList(lines())
    For i = 1 To ListSize(lines())
        If Not NextElement(lines())
            PrintN("Error in purgeUnusedProcedures()! Linked list lines() error!")
            End 1
        EndIf
        markProcs(lines(), procList())
    Next i
    ;
    ; replace unused procedure lines with blank lines
    ;
    Define foundUnused
    ResetList(procList())
    For i = 1 To ListSize(procList())
        If Not NextElement(procList())
            PrintN("Error in purgeUnusedProcedures()! Linked list procList() error!")
            End 1
        EndIf
        If Not procList()\used
            ChangeCurrentElement(lines(), procList()\start)
            blankToEndProcedure(lines(), procList())
            foundUnused + 1
        EndIf
    Next i
    ProcedureReturn foundUnused
EndProcedure

; keep purging until nothing to purge
While purgeUnusedProcedures(lineList())
Wend

;
; save to file with name formed by mainFile + "TRIMMED"
;
Define outfile.s
ResetList(lineList())
For i = 1 To ListSize(lineList())
    If Not NextElement(lineList())
        ; uh-oh, should never get here
        PrintN("My linked list got foobared!")
        End 1
    EndIf
    If Len(lineList()) > 0
        outfile + lineList() + Chr(13) + Chr(10)
    EndIf
Next i
ClearList(lineList())

Define outfilename.s = mainFile + "TRIMMED"
f = CreateFile(#PB_Any, outfilename)
If Not f
    PrintN("Unable to create trimmed file: " + outfilename)
    End 1
EndIf

WriteString(f, outfile)

CloseFile(f)

End 0

[maker]

v 1.2 is available - fixed a couple of detection errors that allowed some unreferenced procedures to slip through the cracks (mainly recursive routines).

http://www.programmingforfun.net/trimmer/trimmer.htm