Little Purebasic Preprocessor - LPP

Share your advanced PureBasic knowledge/code with the community.
Little John
Addict
Addict
Posts: 4527
Joined: Thu Jun 07, 2007 3:25 pm
Location: Berlin, Germany

Little Purebasic Preprocessor - LPP

Post by Little John »

Works also with PB 5.20

Hi all,

according to another discussion, I've revived an old tool of mine that allows for PureBasic statements on multiple lines. It was mainly developed as tool for the PureBasic IDE, but can also be used as standalone program.

//edit:
I've rewritten the whole program, so that it now can handle include files. I was able to do so because srod gave me some fine code, and his kind permission to abuse it. ;-) Many, many thanks!!

Bug reports, suggestions for improvement and other comments are welcome. :-)

Version 0.10
  • First public release of LPP.
Version 0.50
  • Pre-processed include files are not merged into a single file anymore.
    So in case of a syntax error, the IDE now shows the correct line.
  • Fixed some bugs.
Version 0.60
  • Additional argument when used as IDE tool, so that LPP now can handle code which has not yet been saved to a file.
  • Reading the environment variables PB_TOOLS_IDE and PUREBASIC_HOME makes LPP now more flexible and versatile.
    This e.g. makes it easier to use LPP on systems where both the x86 version and the x64 version of PB is installed.
  • LPP now takes into account, that on Windows not only the back slash but also the forward slash can be used as path separator.
  • Better adaptation to Mac OS.
  • Changed some details.
Version 0.61
  • Added support for #PB_Compiler_FilePath in Include statements.
    This constant is very useful for nested include files.
  • Improved error handling.
  • Improved comments.
  • Changed several details.
Overview about the following code
- LPP including documentation
- LppClean (companion program for deleting temporary files)
- Demo 1
- Demo 2 (consists of two files)

Regards, Little John


LPP including documentation

Code: Select all

; -- Little Purebasic Preprocessor (LPP)
; -- Version 0.61, 2011-01-01
; Public Domain, written by Juergen Luethje <http://luethje.eu/>.
; Developed with Purebasic 4.51.
; Cross-platform, Unicode compliant.
; Tested on Windows XP x86, and Ubuntu 10.10 x86.
; Standard disclaimer: USE AT YOUR OWN RISK!
;
; Features
; --------
; Optional line continuation in the source code by adding a particular
; mark at the end of the regarding lines. This is possible in the main
; file and in include files as well.
;
; Installation and call
; ---------------------
; Compile the program to an executable file.
;
; 1) This program was mainly developed as tool for the Purebasic IDE.
;    In the IDE, choose from the menu "Configure Tools ...".
;    In the window that appears, add the program 2 times with
;    different names by clicking the [ New ] button.
;
;    Use as trigger
;    - for one name      : Before Compile/Run
;    - for the other name: Before Create Executable
;
;    and for both names choose the following settings
;
;    Arguments: "%FILE" "%COMPILEFILE" "%TEMPFILE"    (with the quotes!)
;    [v] Wait until tool quits
;    [v] Hide Tool from the Main menu
;
; 2) You can use it as standalone program, for instance in order to
;    pre-process code that is going to be tested with PureUnit. In this
;    case, it might be useful to set the environment variable
;    PUREBASIC_HOME, e.g. on Ubuntu:
;        export PUREBASIC_HOME=/opt/purebasic/
;    (for details please see your system specific instructions).
;
;    If this environment variable is not set on your system, the LPP
;    executable must be in a subdirectory of the PureBasic installation
;    directory, in order to be able to handle #PB_Compiler_Home in the
;    processed code correctly.
;
;    Command line:  Lpp <Main PB source code file> <Main output file>
;
; Usage
; -----
; For using line continuation in your code, type
;     $LPP      (lower/upper case doesn't matter)
; as first executable statement in your main source code file.
;
; This tool allows to continue any line by adding a trailing _ like in
; the first example (a comment may follow). Open strings cannot be
; continued. In case that mark is part of an identifier (e.g. procedure
; or variable name), it is *not* interpreted as line continuation mark.
;
; If you want, you can define a custom string (e.g. &) as continuation
; mark for each file separately. In order to do so, write:
;     $LPP_MULTILINE = &
; on the line after $LPP in your main source code file, or as first
; executable statement in an include file. Note that the custom string
; is not enclosed in quotes.
;
; In case of an error in an include file, the IDE will display a
; temporary file with the flawed line, generated by LPP. If e.g. the
; error is on line 12 in the file "foo.pbi", in combination with LPP
; the IDE will highlight line 12 in the file "~Lpp_foo.pbi". Bear in
; mind to correct the code in the original source file (which is in the
; same directory), not in the temporary file!
;
; For working with your normal code, it's not required to switch this
; tool off in the IDE. If it does not read $LPP at the beginning of the
; source code, then the pre-processor will see that there's nothing to
; do for it and will immediately terminate.
;
; Notes
; -----
; All XincludeFile, IncludeFile, and IncludePath statements in the
; source code which is to be processed by this program
; - must not be part of multi-statemented (colon separated) lines
; - must only involve constant literal strings, #PB_Compiler_Home,
;   and #PB_Compiler_FilePath
; - must not depend on macros
;
; Include files may have different encodings.
;
; Return code of the program
; --------------------------
; 0 = source code doesn't require preprocessing
; 1 = source code successfully processed
; 2 = error (also a message is shown in this case)
;
; Credits
; -------
; This program makes use of code by Stephen Rodriguez for merging a
; Purebasic source file and all included files into a single file.
; Thank you very much!
; Thanks to all members of the German and the English Purebasic
; forums, from whom I learned tips, and who made suggestions for
; improvement of this program.


EnableExplicit

;-- CONSTANTS AND STRUCTURES

#PROGRAM_TITLE$ = "LPP 0.61"
#ABORT_MESSAGE$ = #LF$ + "Program aborted."
#TMPFILE_PREFIX$ = "~Lpp_"
#SIZEOFSTACK = 1024                        ; Max number of include files

#SLASH_CHARS$ = "/\"
#WHITESPACE$ = " " + #TAB$
#KEYWORD_SEPARATOR$ = "=" + #WHITESPACE$
#INCLUDE_SEPARATOR$ = "+" + #WHITESPACE$ + #DQUOTE$
#SYMBOLS$ = ",=-*/\&|!~()<>%:'" + #INCLUDE_SEPARATOR$
#LPP_KEYWORD$ = "$LPP"                     ; Upper case here!
#MULTILINE_KEYWORD$ = "$LPP_MULTILINE"     ;    -- " --


; Return values of PreProcessSources()
Enumeration 0 Step -1
   #LPP_NOTHINGTODO
   #LPP_OKAY
   #LPP_MAINOPENERROR
   #LPP_FILECREATEERROR
   #LPP_INVALIDCOMPILERHOME
   #LPP_INVALIDPATH
   #LPP_INVALIDINCLUDEFILE
   #LPP_NESTEDINCLUDESOVERFLOW
   #LPP_INCLUDEOPENERROR
   #LPP_INVALIDSTRINGFORMAT
   #LPP_INVALIDMULTILINESYNTAX
EndEnumeration

Structure File        ; used for the stack
   inFile$
   inFile.i
   outFile.i
   format.i
   mark$
EndStructure


;-- PLATFORM SPECIFIC CODE

CompilerSelect #PB_Compiler_OS
   CompilerCase #PB_OS_Windows
      #PB_IDE$ = "purebasic.exe"        ; file name relative to #PB_Compiler_Home
      Macro PathIsRelative (_file_)
         (Mid(_file_, 2, 1) <> ":")
      EndMacro

   CompilerCase #PB_OS_Linux
      #PB_IDE$ = "compilers/purebasic"
      Macro PathIsRelative (_file_)
         (Left(_file_, 1) <> "/")
      EndMacro

   CompilerCase #PB_OS_MacOS
      #PB_IDE$ = "PureBasic.app"
      Macro PathIsRelative (_file_)
         (Left(_file_, 1) <> "/")
      EndMacro
CompilerEndSelect


;-- GLOBAL VARIABLES

Global g_Include$      ; used internally
Global g_ErrMessage$   ; used in case of an error


;-- PROCEDURES

Procedure Help()
   Protected msg$

   msg$ = "Little Purebasic Preprocessor" + #LF$
   msg$ + "Juergen Luethje 2011" + #LF$
   msg$ + #LF$
   msg$ + "Parameters when using as IDE tool : " + #DQUOTE$ + "%FILE" + #DQUOTE$ + " "
   msg$ + #DQUOTE$ + "%COMPILEFILE" + #DQUOTE$ + " " + #DQUOTE$ + "%TEMPFILE" + #DQUOTE$ + #LF$
   msg$ + #LF$
   msg$ + "Usage as standalone program : Lpp <Main PB source code file> <Main output file>"
   MessageRequester(#PROGRAM_TITLE$, msg$)
EndProcedure


Procedure.s LTrimChars (source$, charList$=#WHITESPACE$)
   ; in : string
   ; out: string without any leading characters, that are contained in 'charList$'
   Protected left, length=Len(source$)

   left = 1
   While (left <= length) And FindString(charList$, Mid(source$,left,1), 1)
      left + 1
   Wend
   ProcedureReturn Mid(source$, left)
EndProcedure


Procedure.s RTrimChars (source$, charList$=#WHITESPACE$)
   ; in : string
   ; out: string without any trailing characters, that are contained in 'charList$'
   Protected right

   right = Len(source$)
   While (right >= 1) And FindString(charList$, Mid(source$,right,1), 1)
      right - 1
   Wend
   ProcedureReturn Left(source$, right)
EndProcedure


Procedure.s GetPBFolder()
   ; returns the absolute path of the PB installation directory
   ; (with trailing slash)

 CompilerIf #PB_Compiler_Debugger
   ; LPP runs as source code in the IDE with debugger on.
   ProcedureReturn #PB_Compiler_Home

 CompilerElse                                                  ; EXE file
   Protected folder$

   ; First look whether LPP runs as IDE tool.
   folder$ = GetEnvironmentVariable("PB_TOOLS_IDE")
   If folder$
      ProcedureReturn Left(folder$, Len(folder$)-Len(#PB_IDE$))
   EndIf

   ; Look for a regarding environment variable on the system.
   folder$ = GetEnvironmentVariable("PUREBASIC_HOME")
   If folder$
      If FindString(#SLASH_CHARS$, Right(folder$,1),1) = 0
         folder$ + "/"
      EndIf
      ProcedureReturn folder$
   EndIf

   ; LPP runs as standalone program, and the environment variable PUREBASIC_HOME is not set.
   ; In this case, the executable file of LPP must be in a SUBDIRECTORY of the PB installation directory.
   folder$ = GetPathPart(ProgramFilename())
   Repeat
      folder$ = GetPathPart(Left(folder$, Len(folder$)-1))  ; parent folder
      If folder$ = ""
         Break
      EndIf
   Until FileSize(folder$ + #PB_IDE$) > 0
   ProcedureReturn folder$
 CompilerEndIf
EndProcedure


Procedure.s ExtractCode (line$)
   ; in : source code line
   ; out: source code line without comment, and
   ;      without trailing whitespace (with a
   ;      trailing " " in case of an error, though)
   Protected char, i, length=Len(line$)

   i = 1
   While i <= length
      char = Asc(Mid(line$, i, 1))
      If char = '"'      ; skip quoted string
         i = FindString(line$, #DQUOTE$, i+1)
         If i = 0
            Break
         EndIf
      ElseIf char = 39   ; skip 'quoted number'
         i = FindString(line$, "'", i+1)
         If i = 0
            Break
         EndIf
      ElseIf char = ';'  ; end of code in given line
         line$ = Left(line$, i-1)
         Break
      EndIf
      i + 1
   Wend

   line$ = RTrimChars(line$)
   If i = 0              ; error: " or ' are unbalanced on the line
      line$ + " "
   EndIf
   ProcedureReturn line$
EndProcedure


Procedure.s GetLine (inFileID, format, cMark$, lenMark, *lineCount.Integer)
   ; in : inFileID    : input file number
   ;      format      : #PB_Ascii or #PB_UTF8
   ;      cMark$      : line continuation mark
   ;      lenMark     : length of line continuation mark
   ; out: *lineCount\i: number of blank lines to pad
   ;      return value: current whole (code) line
   Protected lenCode
   Protected line$, code$, ret$

   *lineCount\i = 0
   While Eof(inFileID) = #False
      line$ = ReadString(inFileID, format)
      code$ = ExtractCode(line$)
      lenCode = Len(code$)
      If (UCase(Right(code$,lenMark)) = cMark$) And FindString(#SYMBOLS$, Mid(code$,lenCode-lenMark,1), 1)
         If *lineCount\i = 0
            ret$ = Left(code$, lenCode-lenMark)
         Else
            ret$ + LTrimChars(Left(code$, lenCode-lenMark))
         EndIf
         *lineCount\i + 1
      ElseIf *lineCount\i > 0
         ProcedureReturn ret$ + LTrimChars(code$)
      Else
         ProcedureReturn line$
      EndIf
   Wend

   ProcedureReturn ""                      ; end of file
EndProcedure


Procedure.s NextStatement (inFileID, format, List buffer$(), *filePntr.Quad)
   ; in : inFileID    : input file number
   ;      format      : #PB_Ascii or #PB_UTF8
   ; out: buffer$()   : skipped lines
   ;      *filePntr\q : pointer of file 'inFileID' before doing ReadString()
   ;      return value: next statement in input file
   Protected line$, code$=""

   ClearList(buffer$())
   While Eof(inFileID) = #False
      *filePntr\q = Loc(inFileID)
      line$ = ReadString(inFileID, format)
      code$ = LTrimChars(ExtractCode(line$))
      If code$
         Break
      EndIf
      AddElement(buffer$())
      buffer$() = line$
   Wend

   ProcedureReturn code$
EndProcedure


Procedure.s ContinuationMark (inFileID, outFileID, format)
   ; in : inFileID : input file number
   ;      outFileID: output file number
   ;      format   : #PB_Ascii or #PB_UTF8
   ; out: custom line continuation mark,
   ;      ""  on error,
   ;      " " if #MULTILINE_KEYWORD$ statement was not found
   Protected filePntr.q, multilineKeyLength=Len(#MULTILINE_KEYWORD$)
   Protected line$, param$, ret$

   NewList buffer$()
   line$ = NextStatement(inFileID, format, buffer$(), @filePntr)
   If (UCase(Left(line$, multilineKeyLength)) = #MULTILINE_KEYWORD$) And FindString(#KEYWORD_SEPARATOR$, Mid(line$+" ", multilineKeyLength+1, 1), 1)
      param$ = LTrim(Mid(line$, multilineKeyLength+1))
      If Left(param$,1) = "="
         ret$ = UCase(LTrim(Mid(param$, 2)))         ; custom string to denote line continuation
      Else
         ret$ = ""                                   ; Syntax error
      EndIf
      AddElement(buffer$())
   Else                                              ; #MULTILINE_KEYWORD$ not found
      ret$ = " "
      FileSeek(inFileID, filePntr)                   ; rewind file pointer
   EndIf

   ForEach buffer$()
      WriteStringN(outFileID, buffer$(), format)     ; write skipped lines
   Next

   ProcedureReturn ret$
EndProcedure


Procedure.i ProcessIncludeString (inFile$, line$)
   ; in : inFile$: name of current source file
   ;      line$  : include path / filename
   ;               (can contain concatenations or
   ;                #PB_Compiler_Home, #PB_Compiler_FilePath)
   ; out: g_Include$  : string literal corresponding to line$;
   ;                    this is not necessarily a perfectly validated
   ;                    path/filename, but that will be picked up later.
   ;      return value: 0 on success, or an error code
   Protected char$, pbFolder$
   Protected length, left, right, errCode, i
   NewList token$()

   errCode = 0
   g_Include$ = ""
   
   line$ = ExtractCode(line$)
   length = Len(line$)
   If length = 0
      ProcedureReturn -2
   EndIf

   ; First we tokenise the line.
   left=1 : right=1
   Repeat
      char$ = Mid(line$, right, 1)
      If FindString(#INCLUDE_SEPARATOR$, char$, 1)
         If left < right
            AddElement(token$())
            token$() = Mid(line$, left, right-left)
            left = right
         ElseIf char$ = #DQUOTE$     ; open quote, left=right
            right = FindString(line$, char$, left+1)
            If right = 0             ; no end quote
               ClearList(token$())
               errCode = -2
               Break
            ElseIf right-left > 1
               AddElement(token$())
               token$() = Mid(line$, left, right-left+1)
            EndIf
            right+1
            left = right
         ElseIf char$ <> " "         ; left=right
            AddElement(token$())
            token$() = Mid(line$,left,1)
            left+1 : right+1
         Else
            left+1 : right+1
         EndIf
      ElseIf right = length
         right+1
         AddElement(token$())
         token$() = Mid(line$, left, right-left)
      Else
         right+1
      EndIf
   Until right > length

   ; Now process the tokens.
   i = 1
   ForEach token$()   
      If i % 2 = 0
         If token$() <> "+"
            errCode = -2
            Break
         EndIf
      ElseIf UCase(token$()) = "#PB_COMPILER_HOME"
         pbFolder$ = GetPBFOlder()
         If pbFolder$ = ""
            errCode = -1
            Break
         EndIf
         g_Include$ + pbFolder$
      ElseIf UCase(token$()) = "#PB_COMPILER_FILEPATH"
         g_Include$ + GetPathPart(inFile$)
      ElseIf Left(token$(),1) = #DQUOTE$
         g_Include$ + Mid(token$(), 2, Len(token$())-2)
      Else
         errCode = -2
         Break
      EndIf
      i + 1
   Next

   ProcedureReturn errCode
EndProcedure


Procedure.i PreProcessSources (inFile$, outFile$)
   ; -- Main procedure
   ; in : inFile$ : name of main source code file to process
   ;      outFile$: name of target file
   ; out: one of the #LPP_* constants listed at the top
   Protected inFileID, outFileID, format, stackPtr, logFileID, filePntr.q
   Protected k, inc, padLines, lenMark, processInclude, result
   Protected path$, line$, cMark$, customMark$, xInclude$, incStatement$
   Protected LogFile$ = GetTemporaryDirectory() + "~Lpp.tmp"

   ; Create file stack and line buffer
   Dim stack.File(#SIZEOFSTACK)
   stackPtr = 0                               ; number of elements on the stack
   NewList buffer$()

   ; Attempt to open the main input file
   path$ = GetPathPart(inFile$)
   inFileID = ReadFile(#PB_Any, inFile$)
   If inFileID = 0
      ProcedureReturn #LPP_MAINOPENERROR
   EndIf

   ; Identify the string encoding used in the file
   format = ReadStringFormat(inFileID)
   Select format
      Case #PB_Ascii, #PB_UTF8
      Default
         CloseFile(inFileID)
         g_ErrMessage$ = inFile$
         ProcedureReturn #LPP_INVALIDSTRINGFORMAT
   EndSelect

   ; Check the first statement of the main input file.
   If UCase(NextStatement(inFileID, format, buffer$(), @filePntr)) <> #LPP_KEYWORD$
      CloseFile(inFileID)
      ProcedureReturn #LPP_NOTHINGTODO   ; no preprocessing necessary
   EndIf

   ; Attempt to create the main output file
   outFileID = CreateFile(#PB_Any, outFile$)
   If outFileID = 0
      CloseFile(inFileID)
      g_ErrMessage$ = outFile$
      ProcedureReturn #LPP_FILECREATEERROR
   EndIf

   ; Write BOM and skipped lines to main output file.
   WriteStringFormat(outFileID, format)
   ForEach buffer$()
      WriteStringN(outFileID, buffer$(), format)
   Next
   WriteStringN(outFileID, "")

   cMark$ = "_"                              ; default line continuation mark

   ; Look for custom line continuation mark
   customMark$ = ContinuationMark(inFileID, outFileID, format)
   If customMark$ = ""
      CloseFile(inFileID)
      CloseFile(outFileID)
      g_ErrMessage$ = inFile$
      ProcedureReturn #LPP_INVALIDMULTILINESYNTAX  ; Error
   ElseIf customMark$ <> " "
      cMark$ = customMark$
   EndIf

   logFileID = CreateFile(#PB_Any, logFile$)
   If logFileID = 0
      CloseFile(inFileID)
      CloseFile(outFileID)
      g_ErrMessage$ = logFile$
      ProcedureReturn #LPP_FILECREATEERROR
   EndIf

   ; Main loop
   result = #LPP_OKAY
   Repeat
      lenMark = Len(cMark$)

      While Eof(inFileID) = #False
         line$ = GetLine(inFileID, format, cMark$, lenMark, @padLines)

         ; Check the three possible 'include' options.
         If (FindString(UCase(line$), "INCLUDEPATH",1) = 1) And FindString(#SYMBOLS$, Mid(line$,12,1),1)
            line$ = LTrimChars(RTrimChars(Mid(line$, 12)))
            If line$ = #DQUOTE$ + #DQUOTE$
               path$ = ""
            Else
               processInclude = ProcessIncludeString(inFile$, line$)
               If processInclude < 0                 ; Error
                  CloseFile(inFileID)
                  CloseFile(outFileID)
                  If processInclude = -1
                     result = #LPP_INVALIDCOMPILERHOME
                  Else
                     g_ErrMessage$ = line$
                     result = #LPP_INVALIDPATH
                  EndIf
                  Break 2
               EndIf
               path$ = g_Include$
               If path$ And (FindString(#SLASH_CHARS$, Right(path$,1),1) = 0)
                  path$ + "/"
               EndIf
            EndIf
            For k = 1 To padLines+1                        ; Write appropriate number of blank
               WriteStringN(outFileID, "")                 ; lines, in order to preserve line
            Next                                           ; numbers of subsequent statements

         Else
            inc = FindString(UCase(line$), "INCLUDEFILE",1) + 11
            If (inc = 12 Or (inc=13 And UCase(Left(line$,1))="X")) And FindString(#SYMBOLS$, Mid(line$,inc,1),1)
               incStatement$ = Left(line$, inc-1) + " "
               line$ = LTrimChars(RTrimChars(Mid(line$, inc)))
               processInclude = ProcessIncludeString(inFile$, line$)
               If processInclude < 0                        ; Error
                  CloseFile(inFileID)
                  CloseFile(outFileID)
                  If processInclude = -1
                     result = #LPP_INVALIDCOMPILERHOME
                  Else
                     g_ErrMessage$ = line$
                     result = #LPP_INVALIDINCLUDEFILE
                  EndIf
                  Break 2
               EndIf
               If PathIsRelative(g_Include$)
                  g_Include$ = path$ + g_Include$
               EndIf
               ; Write Include statement with new file name into including file.
               outFile$ = GetPathPart(g_Include$) + #TMPFILE_PREFIX$ + GetFilePart(g_Include$)
               WriteStringN(outFileID, incStatement$ + #DQUOTE$ + outFile$ + #DQUOTE$, format)
               For k = 1 To padLines                       ; Write appropriate number of blank
                  WriteStringN(outFileID, "")              ; lines, in order to preserve line
               Next                                        ; numbers of subsequent statements

               ; Check if it's OK to include the file.
               If inc = 12 Or FindString(XInclude$, "<"+g_Include$+">",1) = 0
                  ; Save information about the current file on the stack.
                  If stackPtr = #SIZEOFSTACK
                     CloseFile(inFileID)
                     CloseFile(outFileID)
                     result = #LPP_NESTEDINCLUDESOVERFLOW  ; Error
                     Break 2
                  EndIf
                  stackPtr + 1
                  stack(stackPtr)\inFile$ = inFile$
                  stack(stackPtr)\inFile  = inFileID
                  stack(stackPtr)\outFile = outFileID
                  stack(stackPtr)\format  = format
                  stack(stackPtr)\mark$   = cMark$

                  ; Attempt to open the include file.
                  inFile$ = g_Include$
                  inFileID = ReadFile(#PB_Any, inFile$)
                  If inFileID = 0
                     g_ErrMessage$ = inFile$
                     result = #LPP_INCLUDEOPENERROR        ; Error
                     Break 2
                  EndIf
                  ; Identify the string encoding used in the file.
                  format = ReadStringFormat(inFileID)
                  Select format
                     Case #PB_Ascii, #PB_UTF8
                     Default
                        CloseFile(inFileID)
                        g_ErrMessage$ = inFile$
                        result = #LPP_INVALIDSTRINGFORMAT  ; Error
                        Break 2
                  EndSelect
                  ; Attempt to create temporary output file.
                  outFileID = CreateFile(#PB_Any, outFile$)
                  If outFileID = 0
                     CloseFile(inFileID)
                     g_ErrMessage$ = outFile$
                     result = #LPP_FILECREATEERROR         ; Error
                     Break 2
                  EndIf
                  ; Write BOM to output file.
                  WriteStringFormat(outFileID, format)
                  ; Look for custom line continuation mark
                  customMark$ = ContinuationMark(inFileID, outFileID, format)
                  If customMark$ = ""
                     CloseFile(inFileID)
                     CloseFile(outFileID)
                     g_ErrMessage$ = inFile$
                     result = #LPP_INVALIDMULTILINESYNTAX  ; Error
                     Break 2
                  ElseIf customMark$ <> " "
                     cMark$ = customMark$
                     lenMark = Len(cMark$)
                  EndIf

                  ; Add this file to the list of already included files.
                  xinclude$ + "<"+inFile$+">"
                  ; Save absolute name of the temporary output file.
                  If PathIsRelative(outFile$)
                     outFile$ = GetCurrentDirectory() + outFile$
                  EndIf
                  WriteStringN(logFileID, outFile$, #PB_UTF8)
               EndIf

            Else                              ; no 'Include' command
               WriteStringN(outFileID, line$, format)

               For k = 1 To padLines                    ; Write appropriate number of blank
                  WriteStringN(outFileID, "")           ; lines, in order to preserve line
               Next                                     ; numbers of subsequent statements.
            EndIf
         EndIf
      Wend
      CloseFile(inFileID)
      CloseFile(outFileID)

      If stackPtr = 0
         Break
      EndIf

      inFile$   = stack(stackPtr)\inFile$  ; restore inFile name
      inFileID  = stack(stackPtr)\inFile   ; restore inFile ID
      outFileID = stack(stackPtr)\outFile  ; restore outFile ID
      format    = stack(stackPtr)\format   ; restore encoding
      cMark$    = stack(stackPtr)\mark$    ; restore line continuation mark
      stackPtr - 1
   ForEver

   CloseFile(logFileID)

   ; Clear the stack if there was an error.
   For k = 1 To stackPtr
      CloseFile(stack(k)\inFile)
      CloseFile(stack(k)\outFile)
   Next

   ProcedureReturn result
EndProcedure


;===========================
;-- EXECUTION ENTRY POINT
;===========================

Define ExitCode
Define Infile$, Outfile$, Msg$

If CountProgramParameters() < 2 Or CountProgramParameters() > 3
   Help()
   End 2
EndIf

Infile$ = ProgramParameter(0)        ; "%FILE"
If Infile$ = ""
   If CountProgramParameters() = 3   ; Tool in the IDE, main source code not saved
      Infile$ = ProgramParameter(2)  ; "%TEMPFILE"
   Else
      Help()
      End 2
   EndIf
EndIf
Outfile$ = ProgramParameter(1)       ; "%COMPILEFILE"

ExitCode = PreProcessSources(Infile$, Outfile$)

Select ExitCode
   ; normal termination
   Case #LPP_NOTHINGTODO
      Debug "Nothing got to do."
      End 0
   Case #LPP_OKAY
      Debug "File '" + Infile$ + "' successfully processed."
      End 1

   ; errors
   Case #LPP_MAINOPENERROR
      Msg$ = "Couldn't open main source file '" + Infile$ + "'." + #ABORT_MESSAGE$
   Case #LPP_FILECREATEERROR
      Msg$ = "Couldn't create file '" + g_ErrMessage$ + "'." + #ABORT_MESSAGE$
   Case #LPP_INVALIDCOMPILERHOME
      Msg$ = "#PB_Compiler_Home not found."
      Msg$ + #ABORT_MESSAGE$
      Msg$ + #LF$ + #LF$
      Msg$ + "Put the executable file of this program into a sub-" + #LF$
      Msg$ + "directory of the PureBasic installation directory."
   Case #LPP_INVALIDPATH
      Msg$ = "Invalid parameter for IncludePath: " + g_ErrMessage$ + "." + #ABORT_MESSAGE$
   Case #LPP_INVALIDINCLUDEFILE
      Msg$ = "Invalid parameter for XIncludeFile or IncludeFile: " + g_ErrMessage$ + "." + #ABORT_MESSAGE$
   Case #LPP_NESTEDINCLUDESOVERFLOW
      Msg$ = "Too many nested includes" + #LF$
      Msg$ + "(probably a recursive use of a certain include file)." + #ABORT_MESSAGE$
   Case #LPP_INCLUDEOPENERROR
      Msg$ = "Couldn't open include file '" + g_ErrMessage$ + "'." + #ABORT_MESSAGE$
   Case #LPP_INVALIDSTRINGFORMAT
      Msg$ = "Unsupported encoding in source file '" + g_ErrMessage$ + "'." + #ABORT_MESSAGE$
   Case #LPP_INVALIDMULTILINESYNTAX
      Msg$ = "Invalid syntax for " + #MULTILINE_KEYWORD$ + " in file '" + g_ErrMessage$ + "'." + #ABORT_MESSAGE$
EndSelect

MessageRequester(#PROGRAM_TITLE$, Msg$)
End 2
LppClean

Code: Select all

; -- LppClean: Companion program for LPP
; -- Version 0.60, 2009-02-28
; Public Domain, written by Juergen Luethje <http://luethje.eu/>.
; Developed with Purebasic 4.30.
; Cross-platform, Unicode compliant.
; Tested on Windows XP, and Ubuntu 8.10.
; Standard disclaimer: USE AT YOUR OWN RISK!
;
; Purpose
; -------
; Delete temporary files that are generated by LPP.
;
; Installation
; ------------
; Compile the program to an EXE file.
;
; In the IDE, choose from the menu "Configure Tools ...".
; In the window that appears, add the program 2 times with
; different names by clicking the [ New ] button.
;
; Use as trigger
; - for one name      : After Compile/Run
; - for the other name: After Create Executable
;
; and for both names choose the setting
;
; [v] Hide Tool from the Main menu


EnableExplicit

Define LogFile$, Line$
Define LogFileID, Ignore

LogFile$ = GetTemporaryDirectory() + "~Lpp.tmp"
LogFileID = ReadFile(#PB_Any, LogFile$)

If LogFileID
   Ignore = #False
   While Eof(LogFileID) = #False
      Line$ = ReadString(logFileID, #PB_UTF8)
      If DeleteFile(Line$) = 0 And Ignore = #False
         If MessageRequester("LppClean", "Couldn't delete file '" + Line$ + "'." + #LF$ + "Ignore such problems?", #PB_MessageRequester_YesNo) = #PB_MessageRequester_Yes
            Ignore = #True
         EndIf
      EndIf
   Wend
   CloseFile(LogFileID)
   DeleteFile(LogFile$)
EndIf
Demo 1

Code: Select all

; Demo program 1 for LPP.
; The possibility to write 1 statement on multiple lines allows to view
; long statements without horizontal scrolling. As shown in the example,
; this is also useful e.g. for nicely adding a comment to each parameter
; of a procedure.

$Lpp                                     ; comment

Procedure Demo (ParameterA.l, _          ; cool parameter
                ParameterB.w, _          ; important parameter
                ParameterC.s, _          ; special parameter
                ParameterD.c)            ; another parameter

   Debug ParameterA + ParameterB         ; Line numbers of subsequent
   Debug "ASCII of the semicolon is:"    ; statements remain unchanged.
   Debug ';'
   a$ = "Hello " _
      + "world!"
   Debug a$
EndProcedure

Demo(7,3,"",0)
Demo 2

Code: Select all

; File "animals.pbi"

$Lpp_Multiline = To be continued ...

Animal1$ To be continued ...
   = "dog"

Animal2$ To be continued ...
   = "cat"

Code: Select all

; Demo program 2 for LPP.
; The possibility to write 1 statement on multiple lines allows to view
; long statements without horizontal scrolling.

$Lpp

XIncludeFile _                            ; Include file
   #PB_Compiler_FilePath + "animals.pbi"  ; with pets.


Procedure Demo (ParameterA.l, _           ; cool parameter
                ParameterB.w, _           ; important parameter
                ParameterC.s, _           ; special parameter
                ParameterD.c)             ; another parameter

   Debug ParameterA + ParameterB          ; Line numbers of subsequent
   Debug "ASCII of the semicolon is:"     ; statements remain unchanged.
   Debug ';'
   a$ = "Hello " _
      + "world!"
   Debug a$
EndProcedure

Demo(7,3,"",0)

Debug Animal1$
Debug Animal2$
Last edited by Little John on Mon Aug 19, 2013 7:53 pm, edited 14 times in total.
srod
PureBasic Expert
PureBasic Expert
Posts: 10589
Joined: Wed Oct 29, 2003 4:35 pm
Location: Beyond the pale...

Post by srod »

Hi Little John,

I have some code which takes a single PB source file and creates a separate source file containing the original source + all include files embedded in the correct order etc. It can deal with #PB_Compiler_Home etc. You are welcome to the code if you would like it? Perhaps it can be combined with your tool.
I may look like a mule, but I'm not a complete ass.
PB
PureBasic Expert
PureBasic Expert
Posts: 7581
Joined: Fri Apr 25, 2003 5:24 pm

Re: JoinLines

Post by PB »

@Little John: Do the extra lines still have syntax coloring? (I can't test it now).
I compile using 5.31 (x86) on Win 7 Ultimate (64-bit).
"PureBasic won't be object oriented, period" - Fred.
Little John
Addict
Addict
Posts: 4527
Joined: Thu Jun 07, 2007 3:25 pm
Location: Berlin, Germany

Post by Little John »

Hi srod,

oh pleeease ... yes, yes. :D

Hi PB,

as far as I can see, syntax coloring is not disturbed by using code lines with a trailing _.

Regards, Little John
User avatar
Blue
Addict
Addict
Posts: 884
Joined: Fri Oct 06, 2006 4:41 am
Location: Canada

Post by Blue »

Very, very nice tool, Little John.
I really like it.
I love being able to display and comment functions just the way you present them in your demos. It greatly adds to the clarity of the code.
Thanks.
-------------------------------------------------------------------

3 questions/comments:
(1) Any idea about how much it would impact a large source file ? I couldn't test it because all my large projects are broken down in many smaller included subFiles.

(2) multiline preprocessing is activated by the instruction (?) "Multiline on". Since there is no symmetrical "Multiline off", wouldn't "Multiline" by itself be clear enough ?

(3) You advise that the instruction appear at the beginning of the source file. However, since it can be used repetitively in various circumstances, I don't see any need for it to appear at the beginning of the file. It only has to appear on a line by itself, no?
-------------------------------------------------------------------

Incidentally, you made me stumble onto something i didn't even know was possible!
In demo #1, the procedure exits with

Code: Select all

 Debug ParameterA + ParameterB + ';' 
which, to my amazement, outputs 69 in the Debug window !!!
It took me a while to figure out where the result 69 was coming from !
I always took it for granted that ';' would mean the character ; .
I didn't realize a symbol between single quotes could have its ascii value evaluated in this manner. :shock:
When did that come about ?
-------------------------------------------------------------------

BTW: typo on line 10 of JoinLines.pb:
"the concernig line" should read "the concerned line""
Last edited by Blue on Mon Jan 05, 2009 9:51 am, edited 3 times in total.
"That's not a bug..." said the programmer. "it's a feature! "
"Oh! I see..." replied the blind man.
User avatar
Blue
Addict
Addict
Posts: 884
Joined: Fri Oct 06, 2006 4:41 am
Location: Canada

Re: JoinLines

Post by Blue »

PB wrote:@Little John: Do the extra lines still have syntax coloring? (I can't test it now).
Tested and verified, PB: syntax coloring is maintained .
Last edited by Blue on Mon Jan 05, 2009 9:55 am, edited 1 time in total.
"That's not a bug..." said the programmer. "it's a feature! "
"Oh! I see..." replied the blind man.
srod
PureBasic Expert
PureBasic Expert
Posts: 10589
Joined: Wed Oct 29, 2003 4:35 pm
Location: Beyond the pale...

Post by srod »

Blue wrote:I didn't realize a symbol between single quotes could have its ascii value evaluated in this manner. :shock:
When did that come about ?
That's been a feature of PB since a time before women took to saying to me : "Not if you were the richest man alive!". Well the final laugh is on me because I am far from the richest man on Earth...... wait, hang on... :wink: Seriously, that's been in for a long long time. You can even use something like 'abcdefgh' to get a quad in which the highest byte has the value of Asc("a"), the next byte Asc("b") and so on. Be careful with Unicode however.

@Little John : pm me an e-mail address and I'll get the code to you. :)
I may look like a mule, but I'm not a complete ass.
User avatar
Blue
Addict
Addict
Posts: 884
Joined: Fri Oct 06, 2006 4:41 am
Location: Canada

Post by Blue »

srod wrote:... Well the final laugh is on me because I am far from the richest man on Earth...... wait, hang on... :wink:
8) Yeah! I finally get to understand why Srod is represented by a car crash.

Oupse :oops: Sorry Little John. i didn't mean to hijack your topic in this way...
"That's not a bug..." said the programmer. "it's a feature! "
"Oh! I see..." replied the blind man.
Little John
Addict
Addict
Posts: 4527
Joined: Thu Jun 07, 2007 3:25 pm
Location: Berlin, Germany

Post by Little John »

2Blue:

thanks for your kind words and your suggestions.

I didn't test the code with large source files myself. :-)
When I had been aware that for an IDE tool, the argument "%COMPILEFILE" does only allow access to the main file in the editor without the include files, I had cancelled this project. Now, in combination with srod's code, maybe I can decvelop a serious tool, and then we'll see how long it takes to process our projects. It shouldn't take too long, I suppose.

Yes, "Multiline" by itself would be clear enough. I cannot remember why I chose "Multiline on". I agree, someone who reads that might expect that there is a corresponding "Mulitiline off" command (which is not the case). I think I'll change that in the next version.

On principle, of course there is no need for such an instruction to appear at the beginning of the source file. But I deliberately wrote the program in a way, so that there is a need for this. It's for the sake of speed, for files that do not contain any "Multiline" instruction. So when my program realises that the first command in a source file is not "Multiline", then it immediately cancels reading that file. Otherwise, the program would have to read the whole file and check each line. That could waste time for huge files, that do not contain any "Multiline" instruction or any line continuation mark at all.

The demo programs are pretty simple, because of lazyness. :-)
I mainly used ';' there in order to verify, that the pre-processor does not confuse that with the beginning of a comment.

And no worries, I don't feel like the topic was hijacked. :-)

2srod:
Done. Thanks again!

Regards, Little John
Last edited by Little John on Tue Jan 06, 2009 9:01 pm, edited 1 time in total.
User avatar
blueznl
PureBasic Expert
PureBasic Expert
Posts: 6161
Joined: Sat May 17, 2003 11:31 am
Contact:

Post by blueznl »

I just might steal that 'multiline' command idea of yours... :-)
( PB6.00 LTS Win11 x64 Asrock AB350 Pro4 Ryzen 5 3600 32GB GTX1060 6GB)
( The path to enlightenment and the PureBasic Survival Guide right here... )
Little John
Addict
Addict
Posts: 4527
Joined: Thu Jun 07, 2007 3:25 pm
Location: Berlin, Germany

Post by Little John »

You're welcome. :-)

Regards, Little John
PB
PureBasic Expert
PureBasic Expert
Posts: 7581
Joined: Fri Apr 25, 2003 5:24 pm

Re: JoinLines

Post by PB »

> Tested and verified, PB: syntax coloring is maintained

Actually, I meant with strings. See how this fails to maintain color:

Image
I compile using 5.31 (x86) on Win 7 Ultimate (64-bit).
"PureBasic won't be object oriented, period" - Fred.
User avatar
Rook Zimbabwe
Addict
Addict
Posts: 4326
Joined: Tue Jan 02, 2007 8:16 pm
Location: Cypress TX
Contact:

Post by Rook Zimbabwe »

LittleJohn this looks great!

@PB: does it work? It looks like the coloring for the " marks works OK?
Binarily speaking... it takes 10 to Tango!!!

Image
http://www.bluemesapc.com/
User avatar
Kaeru Gaman
Addict
Addict
Posts: 4826
Joined: Sun Mar 19, 2006 1:57 pm
Location: Germany

Post by Kaeru Gaman »

sure, because it's the mark...

if he'd continued the line

Code: Select all

there" + "!"
the plus wd'f been green and the exclamation orange.
oh... and have a nice day.
PB
PureBasic Expert
PureBasic Expert
Posts: 7581
Joined: Fri Apr 25, 2003 5:24 pm

Post by PB »

In my example, the word "there" is supposed to be green. But it's not.
And you should be able to code it that way without workarounds, like
Visual Basic does. ;) That's my dream for a line continuation method.
I compile using 5.31 (x86) on Win 7 Ultimate (64-bit).
"PureBasic won't be object oriented, period" - Fred.
Post Reply