Hello!
I found a "frightening" bug on Linux. It has the strongest effect on the ARM-64 version (PB6.00 still because I need to change OS), but it also happens on Debian with PB6.11 (64 bit), when wildcard strings are being parsed recursively.
I use strings that I parse and contain wildcards in the form ${variable-or-function a b c} directly inspired from what I saw in Linux, PHP etc what have you. But when I try to parse those strings, which would remove all those $ in the output, the string gets "pre-parsed" and escaped in some way I can't figure out (mostly on arm-64 linux, sometimes the other linux, but NEVER on windows x86/x64).
Now here is where it gets creepy and ghosty: this happens when the console gets opened, and some parser procedures use the output (print) and get more corrupted/ghostly-escaped strings that those parts where the string manipulation doesn't "touch" Print calls. As if something would "forsee" that ${variables} could/would be used, or maybe there's a compiler switch and "ghosty things" happen there. Maybe only "because" of those escape/wildcard sequences.
Here you can compare the results, there are no visible problems on the Debian version but they will happen eventually.
https://ibb.co/1MCV5m2
I can send via network strings parsed on the raspberry to the debian machine, and the debian machine will parse & display them correctly, because on the raspberry side those strings are only assembled but not parsed+printed... it bypasses the Console-Procedures code parts which would print the result or change colors etc.
In the output strings no $ character should be seen since I use it only for formatting/wildcarding in the strings for now, to print it just use two $$ like in other standards.
String manipulation with ${escape sequences}
Re: [PB6.xx] String manipulation bug with ${escape sequences}
I don't understand your report, please post some code. To me it seems like a terminal issue when using printn() ?
Re: [PB6.xx] String manipulation bug with ${escape sequences}
Well I only have a bit of spaghetti code (in part)
It happens when $ dollar signs appear in a string. It seems cut out the first appearances of $ with a few chars after that $, and then escape the following $ in the string like that: $ -> $$. But that is only my impression.
The Parsing happens "recursively" with ParseWildcards() / ParseWildcards_Back() and the argument exploder functions ExplodeArgString() and ExplodArgstring2().
Both those function pairs use the lookup functions to fill the wildcards/parameter arrays,
and the lookup functions use those functions too.
The "line input" procedures I wrote are not working 100% on linux, backspace and cursor keys aren't, and on Windows there might be glitches especially in the password input field. That's not the focus but you can simply hit return 3 times and you are through the test program.
On windows (all backends/x86/x64): no errors - the title/head is well drawn with - + | characters around the username
Linux x64: colors work, inputs work (tho yet incomplete), but the head/title is corrupted
Linux arm64: does not work at all, all strings are "corrupted"
I thought of "DangerousSmiley$" to be the problem but it isn't; it shows in all debug windows and even the Linux terminal; only windows (10) terminal prints an "undefined" symbol.
It happens when $ dollar signs appear in a string. It seems cut out the first appearances of $ with a few chars after that $, and then escape the following $ in the string like that: $ -> $$. But that is only my impression.
The Parsing happens "recursively" with ParseWildcards() / ParseWildcards_Back() and the argument exploder functions ExplodeArgString() and ExplodArgstring2().
Both those function pairs use the lookup functions to fill the wildcards/parameter arrays,
and the lookup functions use those functions too.
The "line input" procedures I wrote are not working 100% on linux, backspace and cursor keys aren't, and on Windows there might be glitches especially in the password input field. That's not the focus but you can simply hit return 3 times and you are through the test program.
On windows (all backends/x86/x64): no errors - the title/head is well drawn with - + | characters around the username
Linux x64: colors work, inputs work (tho yet incomplete), but the head/title is corrupted
Linux arm64: does not work at all, all strings are "corrupted"
I thought of "DangerousSmiley$" to be the problem but it isn't; it shows in all debug windows and even the Linux terminal; only windows (10) terminal prints an "undefined" symbol.
Code: Select all
; =====================================================
; = UnescapeString2.pb
; =====================================================
; Author: benubi
; Filename: UnescapeString2.pb
; Status: Secret
;
; Declare$ _UnescapeString(String$)
; Declare$ _UnescapeString2(String$, LookUp.UE_LookupProcedure, *udata)
; Declare.i _UnescapeString_Back(String$, *udataWriter, *udataLookup, Writer.UE_WriterProcedure, Lookup.UE_LookupProcedure)
;
; Declare ExplodeArgString(String$, Array Result.s(1))
; + a few little helpers
;
;
; \a: alarm Chr(7)
; \b: backspace Chr(8)
; \f: formfeed Chr(12)
; \n: newline Chr(10)
; \r: carriage return Chr(13)
; \t: horizontal tab Chr(9)
; \v: vertical tab Chr(11)
; \": double quote Chr(34)
; \\: backslash Chr(92)
;
;
;++ extended part ++
;
; \e: escape Chr(27)
; \xnn: ascii hex Chr(0-255)
; \unnnn: unicode hex Chr(0-65535)
; \nnn: octal digits[0-7] Chr(0-255)
; \0: octal null-byte Chr(0)
;
;
; Lookup: ${variable-name}
; ${var "name" 'value' some-thing}
; ${x}
; $[with brackets]
; $(with parenthesis)
; ${pretty}
; $short
;
; Lookup2: with AND without parenthesis:
; $varname_until_unfitting_char.
; The lookup strings remain escaped at lookup function call time
;
; to do/want:
;
;
; "maybe" (low probability of implementation)
; c++ 23 conventions
;
; \0{nnnnn,...}
; \x{nnnnn,...}
; \u{nnnnnn...}
;
;
EnableExplicit
; Like EplodeArgString() except that the results are parsed with a lookup procedure
; Prototype for lookup procedures
Prototype$ UE_LookupProcedure(*udata, key$, parenthesis, isWriter)
; Prototype for writer (output) procedures
Prototype UE_WriterProcedure(*udata, *buff, bytes) ; returns 0 to continue, other to abort
; Prototype for reader (input) procedures
Prototype UE_ReaderProcedure(*udata, *buff, bytes) ; returns 0 to continue, other to abort
Structure LookupStruct
*VT ; = @LookupStruct\LookUp
*LookupCookie
*WriterCookie
Lookup.UE_LookupProcedure
Writer.UE_WriterProcedure
EndStructure
; Extended UnescapeString procedure
Procedure$ _UnescapeString(String$)
Protected resultLength
Protected *C.Character = @String$
Protected octalvalue, i
If String$=#Null$
ProcedureReturn ""
EndIf
; \a: alarm Chr(7)
; \b: backspace Chr(8)
; \f: formfeed Chr(12)
; \n: newline Chr(10)
; \r: carriage return Chr(13)
; \t: horizontal tab Chr(9)
; \v: vertical tab Chr(11)
; \": double quote Chr(34)
; \\: backslash Chr(92)
;++
; \e: escape Chr(27)
; \0: null byte Chr(0)
; \nnn: octal 1-3 digits [0-7]
; \x##: ascii hex Chr(0-255)
; \u####: unicode hex Chr(0-65535)
;
While *C\c
If *C\c = '\'
*C + SizeOf(Character)
Select *C\c
Case '0' To '9'
i = 0
While i < 3 And *C\c >= '0' And *C\c <= '9'
i + 1
*C + SizeOf(character)
Wend
resultLength + 1
Case 'a', 'b', 'f', 'n', 'r', 't', 'v', 32, 39, 92, 'e'
resultLength + 1
Case 'x', 'X' ; ascii character (hex)
*C + SizeOf(character):If *C\c = 0 : Break : EndIf
*C + SizeOf(character):If *C\c = 0 : Break : EndIf
resultLength + 1
Case 'u', 'U' ; unicode character
*C + SizeOf(character):If *C\c = 0 : Break : EndIf
*C + SizeOf(character):If *C\c = 0 : Break : EndIf
*C + SizeOf(character):If *C\c = 0 : Break : EndIf
*C + SizeOf(character):If *C\c = 0 : Break : EndIf
resultLength + 1
Case 0
Break
Default
resultLength + 1
EndSelect
Else
resultLength + 1
EndIf
*C + SizeOf(Character)
Wend
Protected result$ = Space(resultLength)
Protected *Z.Character = @result$
Protected num.c
*C = @String$
While *C\c
If *C\c = '\'
*C + SizeOf(Character)
Select *C\c
Case '0' To '9' ; octal
octalvalue = 0
While *C\c >= '0' And *C\c <= '9' And i < 3
octalvalue * 8
octalvalue + *c\c - '0'
*C + SizeOf(character)
i + 1
Wend
*Z\c = octalvalue
Case 'a'
*Z\c = 7
Case 'b'
*Z\c = 8
Case 'f'
*Z\c = 12
Case 'n'
*Z\c = 10
Case 'r'
*Z\c = 13
Case 't'
*Z\c = 9
Case 'v'
*Z\c = 11
Case 32, 39, 92
*Z\c = *c\c
Case 'e'
*Z\c = 27
Case 'x', 'X' ; ascii character (hex)
*C + SizeOf(character):If *C\c = 0 : Break : EndIf
Select *c\c
Case '0' To '9'
num = *c\c - '0'
Case 'a' To 'f'
num = *c\c - 'a' + 10
Case 'A' To 'F'
num = *c\c - 'A' + 10
EndSelect
num << 4
*C + SizeOf(character):If *C\c = 0 : Break : EndIf
Select *c\c
Case '0' To '9'
num = num | (*c\c - '0')
Case 'a' To 'f'
num = num | (*c\c - 'a' + 10)
Case 'A' To 'F'
num = num | (*c\c - 'A' + 10)
EndSelect
*Z\c = num
Case 'u', 'U' ; unicode character
*C + SizeOf(character):If *C\c = 0 : Break : EndIf
Select *c\c
Case '0' To '9'
num = *c\c - '0'
Case 'a' To 'f'
num = *c\c - 'a' + 10
Case 'A' To 'F'
num = *c\c - 'A' + 10
EndSelect
num << 4
*C + SizeOf(character):If *C\c = 0 : Break : EndIf
Select *c\c
Case '0' To '9'
num = num | (*c\c - '0')
Case 'a' To 'f'
num = num | (*c\c - 'a' + 10)
Case 'A' To 'F'
num = num | (*c\c - 'A' + 10)
EndSelect
num << 4
*C + SizeOf(character):If *C\c = 0 : Break : EndIf
Select *c\c
Case '0' To '9'
num = num | (*c\c - '0')
Case 'a' To 'f'
num = num | (*c\c - 'a' + 10)
Case 'A' To 'F'
num = num | (*c\c - 'A' + 10)
EndSelect
num << 4
*C + SizeOf(character):If *C\c = 0 : Break : EndIf
Select *c\c
Case '0' To '9'
num = num | (*c\c - '0')
Case 'a' To 'f'
num = num | (*c\c - 'a' + 10)
Case 'A' To 'F'
num = num | (*c\c - 'A' + 10)
EndSelect
*Z\c = num
Case 0
Break
Default
*Z\c = *c\c
EndSelect
Else
*Z\c = *C\c
EndIf
*C + SizeOf(Character)
*Z + SizeOf(Character)
Wend
ProcedureReturn result$
EndProcedure
; UnescapeString() with user-defined lookup function (searching for ${abcdefgh} variables)
Procedure$ ParseWildcards(String$, LookUp.UE_LookupProcedure, *udata, iswriter = #False)
Protected resultLength
Protected *C.Character = @String$
Protected t$, *s, enca, lims, lime
Protected tlen, i
Protected NewList token.s()
While *C\c
If *C\c = '$' ; dollar sign (will be omitted)
*C + SizeOf(Character)
Select *C\c
Case 'a' To 'z', 'A' To 'Z', '_'
tlen = 0
*S = *C
While *C\c
Select *C\c
Case 'a' To 'z', 'A' To 'Z', '_', '0' To '9'
tlen + 1
Default
Break
EndSelect
*C + SizeOf(character)
Wend
AddElement(Token())
t$ = LookUp(*udata, PeekS(*S, tlen), 0, isWriter)
Token() = t$
resultLength + Len(t$)
Continue
Case '{', '[', '('
lims = *c\c
If lims = '('
lime = lims + 1
Else
lime = lims + 2
EndIf
*C + SizeOf(character)
*S = *C
enca = 0
tlen = 0
While *C\c
If (*C\c = 34 Or *C\c = 39)
If enca = *C\c
enca = 0
ElseIf enca = 0
enca = *C\c
EndIf
ElseIf *C\c = '\'
resultLength + 1
*C + SizeOf(Character)
If Not *C\c
Break
EndIf
Continue
ElseIf *C\c = lime
If enca = 0
Break
EndIf
EndIf
tlen + 1
*C + SizeOf(Character)
Wend
If *C\c = lime
AddElement(Token())
t$ = LookUp(*udata, PeekS(*S, tlen), lims, isWriter)
Token() = t$
resultLength + Len(t$)
*C + SizeOf(Character)
Continue
Else
Break
EndIf
Default
EndSelect
EndIf
resultLength + 1 ; next character will be copied (no matter what it is) -> best choice $
*C + SizeOf(Character)
Wend
Protected result$ = Space(resultLength)
Protected *Z.Character = @result$
Protected num.c
*C = @String$
While *C\c
If *C\c = '$'
*C + SizeOf(Character)
Select *C\c
Case 'a' To 'z', 'A' To 'Z', '_'
While *C\c
Select *C\c
Case 'a' To 'z', 'A' To 'Z', '_', '0' To '9'
Default
Break
EndSelect
*C + SizeOf(character)
Wend
FirstElement(Token())
*Z + PokeS(*Z, token())
DeleteElement(token())
Continue
Case '{', '(', '['
lims = *C\c
If lims = '('
lime = lims + 1
Else
lime = lims + 2
EndIf
While *C\c
If (*C\c = 34 Or *C\c = 39)
If enca = *C\c
enca = 0
ElseIf enca = 0
enca = *C\c
EndIf
ElseIf *C\c = '\'
*Z\c = *C\c
*C + SizeOf(Character)
If Not *C\c
Break
EndIf
*Z\c = *C\c
*C + SizeOf(Character)
*S = *C
Continue
ElseIf *C\c = lime
If enca = 0
Break
EndIf
EndIf
*C + SizeOf(Character)
Wend
If *C\c = lime
FirstElement(Token())
*Z + PokeS(*Z, Token())
DeleteElement(token())
*c + SizeOf(Character)
Continue
EndIf
Break
Default
*Z\c = *C\c
EndSelect
Else
*Z\c = *C\c
EndIf
*C + SizeOf(Character)
*Z + SizeOf(Character)
Wend
ProcedureReturn result$
EndProcedure
; UnescapeString() with user-defined lookup and output/writer function
Procedure.i ParseWildcards_Back(String$, *udataWriter, *udataLookup, Writer.UE_WriterProcedure, Lookup.UE_LookupProcedure)
Structure UE_Token
Token.s
toklen.i
EndStructure
Protected *C.Character = @String$
Protected t$, *s, num.c
Protected result , enca, lims, lime
Protected toklen, ttlen, x$, c, *Z
Protected NewList token.UE_Token()
*S = *C
Macro WR(_X_)
AddElement(Token())
Token()\token = _X_
Token()\toklen = Len(_X_)
ttlen = ttlen + token()\toklen
toklen = 0
*S = *C + SizeOf(Character)
EndMacro
Macro WR1(_X_)
AddElement(Token())
Token()\token = _X_
Token()\toklen = 1
ttlen + 1
toklen = 0
*S = *C + SizeOf(Character)
EndMacro
Macro WRFLUSH()
c = 0
x$ = Space(ttlen + 2)
*z = @x$
ForEach token()
c + PokeS(*z + c, token()\token)
Next
ClearList(token())
ttlen = 0
toklen = 0
*S = *C + SizeOf(Character)
If c
result = Writer(*udataWriter, *z, c)
If result : ProcedureReturn result : EndIf
EndIf
EndMacro
While *C\c
If toklen = 0
*S = *C
EndIf
If *C\c = '$' ; dollar sign (will be omitted)
If toklen
WR(PeekS(*S, toklen))
EndIf
*C + SizeOf(Character)
Select *C\c
Case 'a' To 'z', 'A' To 'Z', '_'
WRFLUSH()
*S = *C
toklen = 0
While *C\c
Select *C\c
Case 'a' To 'z', 'A' To 'Z', '_', '0' To '9'
toklen + 1
Default
Break
EndSelect
*C + SizeOf(Character)
Wend
If toklen
t$ = Lookup(*udataLookup, PeekS(*S, toklen), 0, 1)
toklen = 0
WR(t$)
EndIf
*S = *C
toklen = 0
Continue
Case '{', '[', '(' ; look up procedure
WRFLUSH()
lims = *C\c
If lims = '(' : lime = lims + 1 : Else : lime = lims + 2 : EndIf
*C + SizeOf(character)
*S = *C ;+ SizeOf(Character)
enca = 0
While *C\c
If enca And enca = *C\c
enca = 0
ElseIf *c\c = 34 Or *C\c = 39 And enca = 0
enca = *C\c
ElseIf *C\c = '\'
*C + SizeOf(Character)
If Not *C\c
Break
EndIf
ElseIf enca = 0 And *C\c = lime
Break
EndIf
toklen + 1
*C + SizeOf(Character)
Wend
If *C\c = lime
t$ = LookUp(*udataLookup, PeekS(*S, toklen), lims, 1)
If t$ <> #Empty$ And T$ <> #Empty$
toklen = 0
WR(t$)
EndIf
toklen = 0
*C + SizeOf(Character)
*S = *C
Continue
Else
Break
EndIf
toklen = 0
*S = *C + SizeOf(Character)
Default
WR1(Chr(*C\c))
EndSelect
Else
toklen + 1
EndIf
*C + SizeOf(Character)
Wend
If toklen
; Debug #PB_Compiler_Procedure +" exit tail peek call"
WR(PeekS(*S, toklen))
EndIf
WRFLUSH()
; Debug #PB_Compiler_Procedure +" exit bottom"
ProcedureReturn 0
EndProcedure
; Explodes a string like if it was a command/batch line taking care of " ' parameter encapsulation
Procedure ExplodeArgString(String$, Array Result.s(1)) ;
Protected *C.Character
Protected count, c
Protected *argstart, slen, argend, *n.Character
*C = @String$
While *C\c
If (*C\c = 32 Or *C\c = #TAB) And *argstart
*argstart = 0
slen = 0
count + 1
EndIf
While *C\c = 32 Or *C\c = #TAB : *C + SizeOf(Character) : Wend
Select *C\c
Case 0, '#' ; commentary ; End of string
Break
Case '\'
If *argstart = 0
*argstart = *c
*C + SizeOf(Character)
If *c\c = 0
Break
EndIf
slen = 2
*C + SizeOf(Character)
Continue
Else
slen + 1
EndIf
Case 34, 39 ; Encapsulation
count + 1
argend = *C\c
*argstart = *C + SizeOf(Character)
slen = 0
*C + SizeOf(Character)
While *C\c
If *C\c = '\'
slen + 1
*C + SizeOf(Character)
If *C\c = 0
Break
EndIf
ElseIf *C\c = argend
;slen+1
Break
EndIf
slen + 1
*C + SizeOf(Character)
Wend
slen = 0
*argstart = 0
Default
If *argstart = 0
*argstart = *C
slen = 1
count = count + 1
Else
slen = slen + 1
EndIf
EndSelect
If *C\c
*C = *C + SizeOf(Character)
Else
Break
EndIf
Wend
If slen
count = count + 1
EndIf
ReDim Result.s(count)
*argstart = 0
*C = @String$
c = 0
While *C\c
If (*C\c = 32 Or *c\c = #TAB) And *argstart
If slen
result(c) = PeekS(*argstart, slen)
Else
result(c) = ""
EndIf
c = c + 1
slen = 0
*argstart = 0
EndIf
While *C\c = 32 Or *C\c = #TAB : *C + SizeOf(Character) : Wend
Select *C\c
Case 0, '#' ; commentary ; End of string
Break
Case '\'
If *argstart = 0
*argstart = *c
*C + SizeOf(Character)
If *c\c = 0
Break
EndIf
slen = 2
*C + SizeOf(Character)
Continue
Else
slen + 1
EndIf
Case 34, 39 ; Encapsulation
argend = *C\c
*argstart = *C + SizeOf(Character)
slen = 0
*C + SizeOf(Character)
While *c\c
If *c\c = '\'
*C = *c + SizeOf(Character)
slen = slen + 1
If *C\c = 0
Break
EndIf
ElseIf *C\c = argend
;slen+1
Break
EndIf
slen = slen + 1
*C + SizeOf(Character)
Wend
If slen
result(c) = PeekS(*argstart, slen)
Else
result(c) = ""
EndIf
*argstart = 0
slen = 0
c = c + 1
Default
If *argstart = 0
*argstart = *C
slen = 1
Else
slen = slen + 1
EndIf
EndSelect
If *C\c
*C + SizeOf(Character)
Else
Break
EndIf
Wend
If slen
result(c) = PeekS(*argstart, slen)
c = c + 1
EndIf
ReDim Result.s(c)
ProcedureReturn c
EndProcedure
; Like EplodeArgString() except that the results are parsed with a lookup procedure
Procedure ExplodeArgString2(String$, Array Result.s(1), LookUp.UE_LookupProcedure, *udata, isWriter) ;
Protected *C.Character
Protected count, c
Protected *argstart.character, slen, argend, *n.Character
;Protected old_show=show_it
;show_it=#True
*C = @String$
While *C\c
If (*C\c = 32 Or *C\c = #TAB) And *argstart
*argstart = 0
slen = 0
count + 1
EndIf
While *C\c = 32 Or *C\c = #TAB : *C + SizeOf(Character) : Wend
Select *C\c
Case 0, '#' ; commentary ; End of string
Break
Case '\'
If *argstart = 0
*argstart = *c
*C + SizeOf(Character)
If *c\c = 0
Break
EndIf
slen = 2
*C + SizeOf(Character)
Continue
Else
slen + 1
EndIf
Case 34, 39 ; Encapsulation
count + 1
argend = *C\c
*argstart = *C ;+ SizeOf(Character)
slen = 1
*C + SizeOf(Character)
While *C\c
If *C\c = '\'
slen + 1
*C + SizeOf(Character)
If *C\c = 0
Break
EndIf
ElseIf *C\c = argend
slen + 1
Break
EndIf
slen + 1
*C + SizeOf(Character)
Wend
slen = 0
*argstart = 0
Default
If *argstart = 0
*argstart = *C
slen = 1
count = count + 1
Else
slen = slen + 1
EndIf
EndSelect
If *C\c
*C = *C + SizeOf(Character)
Else
Break
EndIf
Wend
If slen
count = count + 1
EndIf
ReDim Result.s(count)
*argstart = 0
*C = @String$
c = 0
While *C\c
If (*C\c = 32 Or *c\c = #TAB) And *argstart
If slen
result(c) = PeekS(*argstart, slen)
Else
result(c) = ""
EndIf
c = c + 1
slen = 0
*argstart = 0
EndIf
While *C\c = 32 Or *C\c = #TAB : *C + SizeOf(Character) : Wend
Select *C\c
Case 0, '#' ; commentary ; End of string
Break
Case '\'
If *argstart = 0
*argstart = *c
*C + SizeOf(Character)
If *c\c = 0
Break
EndIf
slen = 2
*C + SizeOf(Character)
Continue
Else
slen + 1
EndIf
Case 34, 39 ; Encapsulation
argend = *C\c
*argstart = *C + SizeOf(Character)
slen = 0
*C + SizeOf(Character)
While *c\c
If *c\c = '\'
*C = *c + SizeOf(Character)
slen = slen + 1
If *C\c = 0
Break
EndIf
ElseIf *C\c = argend
;slen+1
Break
EndIf
slen = slen + 1
*C + SizeOf(Character)
Wend
If slen
If *argstart\c = 39
result(c) = ParseWildcards(PeekS(*argstart + SizeOf(Character), slen - (2*SizeOf(Character))), LookUp, *udata, isWriter)
ElseIf *argstart\c = 34
result(c) = ParseWildcards(PeekS(*argstart + SizeOf(Character), slen - (2*SizeOf(Character))), LookUp, *udata, isWriter)
Else
result(c) = ParseWildcards(PeekS(*argstart, slen), LookUp, *udata, isWriter)
EndIf
Else
result(c) = ""
EndIf
*argstart = 0
slen = 0
c = c + 1
Default
If *argstart = 0
*argstart = *C
slen = 1
Else
slen = slen + 1
EndIf
EndSelect
If *C\c
*C + SizeOf(Character)
Else
Break
EndIf
Wend
If slen
If *argstart\c = 39
result(c) = ParseWildcards(PeekS(*argstart + SizeOf(Character), slen - (2*SizeOf(Character))), LookUp, *udata, isWriter)
ElseIf *argstart\c = 34
result(c) = ParseWildcards(PeekS(*argstart + SizeOf(Character), slen - (2*SizeOf(Character))), LookUp, *udata, isWriter)
Else
result(c) = ParseWildcards(PeekS(*argstart, slen), LookUp, *udata, isWriter)
EndIf
c = c + 1
EndIf
ReDim Result.s(c)
ProcedureReturn c
EndProcedure
; returns the named flag's value (flag1:val1 flag2:val2) values are in decimal
Procedure.i StringFlag(String$, FlagList$)
Protected *S
Protected klen
Protected flag
Protected value
Protected *C.character = @flaglist$
While *C\c
While *C\c = #TAB Or *C\c = 32
*C + SizeOf(character)
Wend
*S = *C
klen = 0
While *C\c And *C\c <> 32 And *c\c <> #TAB
If *C\c = ':'
If PeekS(*S, klen) = String$
;Debug "found! "+String$
; continue
*C + SizeOf(Character)
While *C\c And *C\c <> 32 And *C\c <> #TAB
value * 10
value + (*C\c - '0')
*C + SizeOf(Character)
Wend
ProcedureReturn value
Else
; Debug "next... ("+PeekS(*S,klen)+")"
While *C\c And *c\c <> 32 And *C\c <> #TAB
*C + SizeOf(character)
Wend
*S = *C
If Not *C\c
;Debug "reached eol"
ProcedureReturn
EndIf
Break
EndIf
EndIf
klen + 1
*C + SizeOf(Character)
Wend
*C + SizeOf(Character)
Wend
;Debug "not found :("
ProcedureReturn flag
EndProcedure
; returns #true is string$ is in the memberlist
Procedure.i IsMember(String$, MemberList$)
Protected *Z1.Character
Protected *Z2.character = @MemberList$
While *Z2\c
While *Z2\c = #TAB Or *Z2\c = 32
*Z2 + SizeOf(Character)
Wend
*Z1 = @String$
While *Z1\c = *Z2\c And *Z1\c And *Z2\c
*Z1 + SizeOf(Character)
*Z2 + SizeOf(Character)
Wend
If *Z1\c = 0 And (*Z2\c = 0 Or *Z2\c = 32 Or *Z2\c = #TAB)
ProcedureReturn #True
EndIf
Wend
ProcedureReturn #False
EndProcedure
CompilerIf #PB_Compiler_IsMainFile
; Examples
; --------------------------------------------------------------------------------------------------------
; How to make your own lookup function to return unescaped+filled strings
; Use helpfer functions like ExplodeArgString and StringFlag
;
; use StringFlag(String$,) with flaglists like these:
#INCLUDE_MSBOX = #True
#USE_CONSOLE = #True
Global msgboxflaglist$ ; pairs are name:value, pairs are separated by space(s) or tab(s)
msgboxflaglist$ = "cancel:" + Str(#PB_MessageRequester_Cancel)
msgboxflaglist$ + " error:" + Str(#PB_MessageRequester_Error)
msgboxflaglist$ + " info:" + Str(#PB_MessageRequester_Info)
msgboxflaglist$ + " no:" + Str(#PB_MessageRequester_No)
msgboxflaglist$ + " ok:" + Str(#PB_MessageRequester_Ok)
msgboxflaglist$ + " warning:" + Str(#PB_MessageRequester_Warning)
msgboxflaglist$ + " yes:" + Str(#PB_MessageRequester_Yes)
msgboxflaglist$ + " yes-no-cancel:" + Str(#PB_MessageRequester_YesNoCancel)
msgboxflaglist$ + " yes-no:" + Str(#PB_MessageRequester_YesNo)
; Example 1:
; look up procedures receive "CONTENT" key$ when found inside parsed/unescaped string "${CONTENT}"
NewMap _L.s()
Declare$ prime_lookup(*udata, key$, parenthesis, isWriter)
Procedure$ mylookup(*udata, key$, parenthesis, isWriter)
Shared _L()
Protected result$ = ""
Protected Dim p$(8)
; Protected pc = ExplodeArgString2(key$, p$(), @mylookup(), *udata, isWriter)
Protected pc = ExplodeArgString2(key$, p$(), @prime_lookup(), *udata, Bool(isWriter) * 2)
;Debug key$
If pc < 6
ReDim p$(6)
EndIf
If parenthesis = 0
Select LCase(p$(0))
Case "username"
result$ = UserName()
Case "computername"
result$ = ComputerName()
Default
result$ = GetRuntimeString(p$(0))
EndSelect
ProcedureReturn result$
ElseIf parenthesis = '['
; lang
result$ = _L(p$(0))
ProcedureReturn result$
EndIf
Select p$(0)
Case "n" ; CRLF 'Telnet' 'network-line' standard ${n} may be easier to type than \r\n for some !?
result$ = #CRLF$
Case "env"
If pc = 2
result$ = GetEnvironmentVariable(p$(1))
EndIf
Case "rts"
result$ = GetRuntimeString(p$(1))
Case "rti"
result$ = FormatNumber(GetRuntimeInteger(p$(1)), 0, "", p$(2))
Case "rtd"
result$ = FormatNumber(GetRuntimeDouble(p$(1)), Val(p$(2)), Left(p$(3) + ",", 1), Mid(p$(3), 2, 1))
Case "setrts"
SetRuntimeString(p$(1), p$(2))
Case "setrti"
SetRuntimeInteger(p$(1), Val(p$(2)))
Case "setrtd"
SetRuntimeDouble(p$(1), ValD(p$(2)))
Case "lset"
result$ = LSet(p$(1), Val(p$(2)), p$(3))
Case "rset"
result$ = RSet(p$(1), Val(p$(2)), p$(3))
Case "cset"
P$(1) = ParseWildcards(p$(1), @prime_lookup(), *udata, 0)
result$ = LSet(Space((Val(p$(2)) / 2) - (Len(p$(1)) / 2)), Val(p$(2)), P$(3))
Case "mid"
result$ = Mid(p$(1), Val(p$(2)), Val(p$(3)))
Case "rf"
Protected fh = ReadFile( - 1, p$(1))
Protected format, remove_eol, replace_eol
If fh
format = ReadStringFormat(fh)
result$ = ReadString(fh, format | #PB_File_IgnoreEOL)
CloseFile(fh)
Select p$(2)
Case ""
Case "remove-eol"
Repeat
Select Right(result$, 1)
Case #CR$, #LF$
result$ = Left(result$, Len(result$) - 1)
Default
Break
EndSelect
ForEver
Case "replace-eol"
result$ = ReplaceString(result$, #CRLF$, " ")
result$ = ReplaceString(result$, #LFCR$, " ")
result$ = ReplaceString(result$, #LF$, " ")
result$ = ReplaceString(result$, #CR$, " ")
result$ = ReplaceString(result$, #TAB$, LSet("", 8))
result$ = Trim(result$)
EndSelect
EndIf
CompilerIf #INCLUDE_MSBOX
Case "msgbox"
;Debug ">>"+key$
If pc => 4
Select MessageRequester(ParseWildcards(p$(1), @mylookup(), *udata), ParseWildcards(p$(2), @mylookup(), *udata), StringFlag(p$(3), msgboxflaglist$) | StringFlag(p$(4), msgboxflaglist$))
Case #PB_MessageRequester_Cancel
result$ = "cancel"
Case #PB_MessageRequester_No
result$ = "no"
Case #PB_MessageRequester_Yes
result$ = "yes"
EndSelect
ElseIf pc >= 3
MessageRequester(ParseWildcards(p$(1), @mylookup(), *udata), ParseWildcards(p$(2), @mylookup(), *udata))
ElseIf pc => 2
MessageRequester("Nested msgbox", ParseWildcards(p$(1), @mylookup(), *udata))
EndIf
CompilerEndIf
EndSelect
ProcedureReturn result$
EndProcedure
Procedure$ MyUnescapeString(String$)
EnableGraphicalConsole(0)
ProcedureReturn _UnescapeString(ParseWildcards(String$, @prime_lookup(), 0))
EndProcedure
; msgbox in mylookup() needs to be uncommented For it To work
;MessageRequester("Too much??", MyUnescapeString("Hello world\r\nSomething wrong?\r\n${msgbox 'Heh heh heh :)' 'A msgbox hiding in a msgbox!!' yes-no-cancel warning}!!!"), #PB_MessageRequester_YesNoCancel)
; --------------------------------------------------------------------------------------------------------
; How to make your own output writer
; How to make it work with the lookup function
; How to simply cascade lookup functions with select statements
Define CurrentConsoleINV
Define CurrentConsoleBG = 0, CurrentConsoleFG = 7
Declare _Print(String$)
Procedure$ Lang(id$)
Shared _L()
If FindMapElement(_L(), id$)
ProcedureReturn _L()
EndIf
ProcedureReturn ""
EndProcedure
Procedure$ PrimeParse(String$, isWriter = 0)
Protected result$ = ParseWildcards(String$, @prime_lookup(), 0, isWriter)
ProcedureReturn result$
EndProcedure
Procedure$ prime_lookup(*udata, key$, parenthesis, isWriter)
Shared CurrentConsoleFG
Shared CurrentConsoleBG
Shared CurrentConsoleINV
Protected Dim p$(0)
Protected pc = ExplodeArgString2(key$, p$(), @prime_lookup(), *udata, Bool(isWriter) * 2)
Protected result$ = Space(100)
Protected Raw
Protected inkey$
result$ = #Empty$
ReDim p$(8)
; Graphical console functions
Select LCase(P$(0))
Case "fg"
If isWriter = 1
CompilerIf #USE_CONSOLE
EnableGraphicalConsole(1)
CurrentConsoleFG = Val(p$(1))
If CurrentConsoleINV = 0
ConsoleColor(CurrentConsoleFG, CurrentConsoleBG)
Else
ConsoleColor(CurrentConsoleBG, CurrentConsoleFG)
EndIf
EnableGraphicalConsole(0)
CompilerEndIf
ElseIf isWriter = 2
result$ = "${" + key$ + "}"
EndIf
Case "bg"
If isWriter = 1
CompilerIf #USE_CONSOLE
EnableGraphicalConsole(1)
CurrentConsoleBG = Val(p$(1))
If CurrentConsoleINV = 0
ConsoleColor(CurrentConsoleFG, CurrentConsoleBG)
Else
ConsoleColor(CurrentConsoleBG, CurrentConsoleFG)
EndIf
EnableGraphicalConsole(0)
CompilerEndIf
ElseIf isWriter = 2
result$ = "${" + key$ + "}"
EndIf
Case "cls"
If isWriter = 1
;ClearConsole()
CurrentConsoleBG = 0
CurrentConsoleFG = 7
CompilerIf #USE_CONSOLE
EnableGraphicalConsole(1)
If CurrentConsoleINV = 0
ConsoleColor(CurrentConsoleFG, CurrentConsoleBG)
Else
ConsoleColor(CurrentConsoleBG, CurrentConsoleFG)
EndIf
EnableGraphicalConsole(0)
CompilerEndIf
ElseIf isWriter = 2
result$ = "${" + key$ + "}"
EndIf
Case "col"
If isWriter = 1
CurrentConsoleFG = Val(p$(1))
CurrentConsoleBG = Val(p$(2))
CompilerIf #USE_CONSOLE
EnableGraphicalConsole(1)
If CurrentConsoleINV = 0
ConsoleColor(CurrentConsoleFG, CurrentConsoleBG)
Else
ConsoleColor(CurrentConsoleBG, CurrentConsoleFG)
EndIf
EnableGraphicalConsole(0)
CompilerEndIf
ElseIf isWriter = 2
result$ = "${" + key$ + "}"
EndIf
Case "n"
result$ = #CRLF$
Case "h"
If isWriter = 1
CompilerIf #USE_CONSOLE
EnableGraphicalConsole(1)
CompilerIf #PB_Compiler_OS = #PB_OS_Linux
ConsoleLocate(1, 1)
CompilerElse
ConsoleLocate(0, 0)
CompilerEndIf
EnableGraphicalConsole(0)
CompilerEndIf
ElseIf isWriter = 2
result$ = "${" + key$ + "}"
EndIf
Case "inv"
If isWriter = 1
CurrentConsoleINV ! 1
CompilerIf #USE_CONSOLE
EnableGraphicalConsole(1)
If CurrentConsoleINV = 0
ConsoleColor(CurrentConsoleFG, CurrentConsoleBG)
Else
ConsoleColor(CurrentConsoleBG, CurrentConsoleFG)
EndIf
EnableGraphicalConsole(0)
CompilerEndIf
EndIf
Case "cur"
If isWriter = 1
If pc = 1
CompilerIf #USE_CONSOLE
ConsoleCursor(Val(p$(1)))
CompilerEndIf
EndIf
EndIf
Case "loc"
If isWriter = 1
If pc = 2
CompilerIf #USE_CONSOLE
EnableGraphicalConsole(1)
CompilerIf #PB_Compiler_OS = #PB_OS_Linux
ConsoleLocate(Val(p$(0)), Val(p$(1)))
CompilerElse
ConsoleLocate(Val(p$(0)), Val(p$(1)))
CompilerEndIf
EnableGraphicalConsole(0)
CompilerEndIf
EndIf
ElseIf isWriter = 2
result$ = "${" + key$ + "}"
EndIf
Case "rep"
result$ = LSet("", Val(p$(1)), p$(2))
Case "reset", "r"
If isWriter = 1
CompilerIf #USE_CONSOLE
EnableGraphicalConsole(1)
CurrentConsoleBG = 0
CurrentConsoleFG = 7
CurrentConsoleINV = 0
ConsoleColor(7, 0)
EnableGraphicalConsole(0)
CompilerEndIf
ElseIf isWriter = 2
result$ = "${" + key$ + "}"
EndIf
Case "hd", "head"
Protected tt$ = ParseWildcards(p$(1), @prime_lookup(), 0, 0)
tt$ = Left(tt$, 76)
p$(1) = _UnescapeString(P$(1))
p$(2) = _UnescapeString(P$(2))
p$(3) = _UnescapeString(P$(3))
p$(4) = _UnescapeString(P$(4))
Protected ind$ = Space(Val(p$(4)))
Protected tlen = Len(tt$)
Protected horiz$ = Left(p$(2), 1)
If horiz$ = "" : horiz$ = "*" : EndIf
Protected vert$ = Mid(p$(2), 2, 1)
If vert$ = "" : vert$ = horiz$ : EndIf
Protected tl$ = Left(p$(3), 1)
If tl$ = "" : If vert$ = horiz$ : tl$ = horiz$:Else:tl$ = "+":EndIf :EndIf
Protected tr$ = Mid(p$(3), 2, 1)
If tr$ = "":tr$ = tl$:EndIf
Protected bl$ = Mid(p$(3), 3, 1)
If bl$ = "":bl$ = tl$:EndIf
Protected br$ = Mid(p$(3), 4, 1)
If br$ = "":br$ = tl$:EndIf
If isWriter = 0
result$ =ind$ + tl$ + LSet("", tlen + 2, horiz$) + tr$ + #CRLF$ + ind$ + vert$ + " " + tt$ + " " + vert$ + #CRLF$ + ind$ + bl$ + LSet("", tlen + 2, horiz$) + br$ + #CRLF$
ElseIf isWriter = 2
result$ = ind$ + tl$ + LSet("", tlen + 2, horiz$) + tr$ + #CRLF$ + ind$ + vert$ + " " + _UnescapeString(ParseWildcards(p$(1), @prime_lookup(), 0, 2)) + " " + vert$ + #CRLF$ +ind$ + bl$ + LSet("", tlen + 2, horiz$) + br$ + #CRLF$
Else
CompilerIf #USE_CONSOLE
_Print(ind$ + tl$ + LSet("", tlen + 2, horiz$) + tr$ + #CRLF$)
_Print(ind$ + vert$ + " " + p$(1) + " " + vert$ + #CRLF$)
_Print(ind$ + bl$ + LSet("", tlen + 2, horiz$) + br$ + #CRLF$)
CompilerEndIf
result$ = ""
EndIf
Case "inln"
CompilerIf #USE_CONSOLE
If isWriter = 1
Protected input_len = Val(p$(1))
Protected fill_char$ = Left(p$(3) + " ", 1)
Protected value$ = p$(2)
Protected inloc
Protected escape_sequence$
EnableGraphicalConsole(0)
Print(LSet(value$, input_len, fill_char$))
Print(LSet("", input_len - Len(value$), Chr(8)))
inloc = Len(value$)
Repeat
EnableGraphicalConsole(1)
inkey$ = Inkey()
raw = RawKey()
Select raw
Case 0
Delay(0)
Case 8, 127
If Len(value$) > 0 And inloc > 0
EnableGraphicalConsole(0)
value$ = Left(value$, inloc - 1) + Mid(value$, inloc + 1)
Print(Chr(8) + fill_char$ + Chr(8))
Print(Mid(value$, inloc) + fill_char$)
Print(LSet("", 2 + Len(value$) - inloc, Chr(8)))
inloc - 1
;
EndIf
Case 13, 10
result$ = value$
Break
Case 27
If Len(inkey$) < 2
While Len(value$)
value$ = Left(value$, Len(value$) - 1)
Print(Chr(8) + fill_char$ + Chr(8))
Wend
inloc = 0
EndIf
Case 37
; Debug "left"
If inloc > 0
inloc - 1
Print(Chr(8))
EndIf
Case 38
; Debug "up"
Case 39
; Debug "right"
If inloc < Len(value$)
inloc + 1
Print(Mid(value$, inloc, 1))
EndIf
Case 40
; Debug "down"
Case 46
; delete key
If Len(value$) > 0
If inloc < Len(value$)
value$ = Left(value$, inloc) + Mid(value$, inloc + 2)
Print(Mid(value$, inloc + 1) + fill_char$)
Print(LSet("", 1 + Len(Mid(value$, inloc + 1)), Chr(8)))
EndIf
EndIf
Default
Debug "default... raw: " + Raw
EndSelect
If raw >= 32 And Len(inkey$) = 1
If Len(value$) < input_len
value$ = Left(value$, inloc) + inkey$ + Mid(value$, inloc + 1)
inloc + 1
Print(Mid(value$, inloc))
Print(LSet("", Len(value$) - inloc, Chr(8)))
EndIf
ElseIf raw = 27
EndIf
ForEver
; Debug "value$="
; Debug value$
If IsRuntime(p$(4))
SetRuntimeString(p$(4), value$)
result$ = ""
Else
result$ = value$
EndIf
EndIf
CompilerEndIf
Case "inpw"
CompilerIf #USE_CONSOLE
If isWriter = 1
input_len = Val(p$(1))
fill_char$ = Left(p$(3) + " ", 1)
value$ = p$(2)
Protected pw_char$ = p$(4)
If pw_char$ = ""
pw_char$ = "*"
ElseIf pw_char$ = "no-char"
pw_char$ = ""
EndIf
inloc = Len(value$)
EnableGraphicalConsole(0)
If value$ <> "" And pw_char$ <> ""
Print(LSet(LSet("", Len(value$), pw_char$), input_len, fill_char$))
Print(LSet("", input_len - Len(value$), Chr(8)))
Else
Print(LSet("", input_len, fill_char$))
Print(LSet("", input_len, Chr(8)))
EndIf
Repeat
inkey$ = Inkey()
raw = RawKey()
Select raw
Case 0
Delay(0)
Case 10, 13
result$ = value$
Break
Case 27
If pw_char$ <> ""
While Len(value$)
value$ = Left(value$, Len(value$) - 1)
Print(Chr(8) + fill_char$ + Chr(8))
Wend
EndIf
inloc = 0
Case 37
If inloc > 0
inloc - 1
If pw_char$ <> ""
Print(Chr(8))
EndIf
EndIf
Case 38
Case 39
If inloc < Len(value$)
inloc + 1
Print(pw_char$)
EndIf
Case 40
EndSelect
If raw >= 32 And inkey$ <> ""
If Len(value$) < input_len
value$ = Left(value$, inloc) + inkey$ + Mid(value$, inloc + 1)
inloc + 1
If pw_char$ <> ""
Print(LSet("", 1 + Len(value$) - inloc, pw_char$))
Print(LSet("", Len(value$) - inloc, Chr(8)))
EndIf
EndIf
ElseIf raw = 8
If Len(value$) > 0 And inloc > 0
value$ = Left(value$, Len(value$) - 1)
If pw_char$ <> ""
Print(Chr(8) + fill_char$ + Chr(8))
EndIf
inloc - 1
If pw_char$ <> ""
Print(LSet("", 1 + Len(value$) - inloc, pw_char$))
Print(LSet("", Len(value$) - inloc, Chr(8)))
EndIf
EndIf
EndIf
ForEver
If IsRuntime(p$(5))
SetRuntimeString(p$(5), value$)
result$ = ""
Else
result$ = value$
EndIf
EndIf
CompilerEndIf
Case "waitkey"
Protected kl = Len(p$(1))
CompilerIf #USE_CONSOLE
If isWriter = 1
If kl = 0
Repeat
result$ = Inkey()
Raw = RawKey()
If Not Raw
Delay(0)
EndIf
Until result$ <> #Empty$ Or raw
result$ = result$ + ":" + Str(raw)
Else
Repeat
result$ = Inkey()
Raw = RawKey()
If result$ <> #Empty$
If Not FindString(result$, p$(1))
result$ = ""
EndIf
EndIf
If Not Raw
Delay(0)
EndIf
Until result$ <> ""
result$ = result$ + ":" + Str(raw)
EndIf
EndIf
If IsRuntime(p$(2))
SetRuntimeString(p$(2), result$)
result$ = ""
EndIf
CompilerEndIf
Default
ProcedureReturn mylookup(*udata, key$, parenthesis, isWriter) ; Cascading lookup-procedures
EndSelect
ProcedureReturn result$
EndProcedure
Procedure prime_writer(*udata, *buff, bytes)
;MessageRequester("Writer",string$)
CompilerIf #USE_CONSOLE
If *buff ; string$ <> #Null$
If bytes ; string$ <> #Empty$
EnableGraphicalConsole(0)
Print(PeekS(*buff, bytes))
EnableGraphicalConsole(1)
EndIf
EndIf
CompilerEndIf
ProcedureReturn 0
EndProcedure
Procedure$ PrimeUnescape(String$, isWriter = 0)
If isWriter
CompilerIf #USE_CONSOLE
EnableGraphicalConsole(0)
CompilerEndIf
EndIf
ProcedureReturn _UnescapeString(PrimeParse(String$, IsWriter))
EndProcedure
Procedure _Print(String$)
CompilerIf #USE_CONSOLE
EnableGraphicalConsole(0)
ParseWildcards_Back(_UnescapeString(String$), 0, 0, @prime_writer(), @prime_lookup())
EnableGraphicalConsole(0)
CompilerEndIf
EndProcedure
Global myUsername.s
Global myPassword.s
Global Var1.d
Global Var2.d
Runtime myUsername, myPassword
Runtime Var1, Var2
Procedure Main()
Protected DangerousSmiley$ = "\uD83D\uDE0A" ; <--- is this the reason?
;Protected DangerousSmiley$ = "" ; <--- is this the reason?
Protected str$
Protected uname$
;
; this caused a bug on linux with the title/header frame $(hd ...)
; CompilerIf #PB_Compiler_OS = #PB_OS_Windows
; uname$= "${env Username}$${fg 15}@$${fg 6}${env Computername}"
; CompilerElse
; uname$= "${env USER}@${rf '/etc/hostname' replace-eol}"
; CompilerEndIf
;
; directly setting virtual variables (returns result from UserName() and ComputerName())
uname$ = "$username$${fg 15}@$${fg 5}$computername$${r}"
myUsername = ParseWildcards(ParseWildcards(uname$, @prime_lookup(), 0, 0), @prime_lookup(), 0, 0)
;myUsername = ParseWildcards(uname$,@prime_lookup(),0,0)
str$ = "$(hd 'Hello $${fg 3}" + uname$
str$ = str$ + "$${fg 7}!' '-|' '+' 24)\r\n This message is \xA9 ${fg 10}free${fg 7} \U00A9.${n}" + DangerousSmiley$ + ""
str$ = str$ + "\r\n${lset 'Physical mem free' 20 '.'}: ${inv}${rset '${rtd var1 0 .,}' 24 ' '}${inv}"
str$ = str$ + "${n}${lset 'Total virtual' 20 '.'}: ${inv}${rset '${rtd var2 0 ,.}' 24 ' '}${inv}\r\n"
;str$ = str$ + "But now beware of $$ -> $$$$ $ $ $strange things can ${happen} if you don't consider ;)"
str$ = str$ + " \r\n"
str$ = str$ + "${lset 'Username ' 30 '.' } : ${inv}${inln 40 '$myUsername' '_' 'myUsername'}${inv}\r\n"
str$ = str$ + "${lset 'Password ' 30 '.' '*'} : ${inv}${inpw 40 '' '_' '*' 'myPassword'}${inv}\r\n"
;Define DangerousSmiley$="<no-smiley>" ; <--- is this the reason?
; Let's fill runtime variables that are accessible via the lookup procedure
Define log$ = "./testcoloroutput.txt"
CreateFile(0, log$, #PB_UTF8)
If Not ReadStringFormat(0)
WriteStringFormat(0, #PB_UTF8)
EndIf
WriteStringN(0, "String before parsing:")
WriteStringN(0, str$)
var1 = MemoryStatus(#PB_System_FreePhysical)
var2 = MemoryStatus(#PB_System_TotalVirtual)
CompilerIf #USE_CONSOLE
OpenConsole()
EnableGraphicalConsole(1)
Debug MyUnescapeString(str$)
Debug _UnescapeString(PrimeParse(str$))
_Print(str$)
PrintN("")
PrintN("")
PrintN("")
PrintN("")
PrintN("Entered Username=" + myUsername)
PrintN("Entered Password=" + myPassword)
EnableGraphicalConsole(0)
PrintN("The same without writer, using user-defined MyUnescape() procedure returning a string (therefor no color formatting inside)")
PrintN("")
Print(PrimeUnescape(str$))
; Print(MyUnescapeString(str$))
_Print("\r\nEnter ${inv}RETURN${inv} to exit\r\n")
EnableGraphicalConsole(0)
Input()
CloseConsole()
CompilerElse
WriteStringN(0, "Parsed:")
WriteStringN(0, PrimeParse(str$, 0))
CompilerEndIf
CloseFile(0)
CompilerIf #PB_Compiler_OS = #PB_OS_Windows
;RunProgram(log$)
CompilerElse
;RunProgram("gedit", log$, "")
CompilerEndIf
EndProcedure
main()
CompilerEndIf
Last edited by benubi on Tue Jun 18, 2024 12:14 pm, edited 2 times in total.
-
DarkDragon
- Addict
- Posts: 2345
- Joined: Mon Jun 02, 2003 9:16 am
- Location: Germany
- Contact:
Re: [PB6.xx] String manipulation bug with ${escape sequences}
So there are multiple ideas which come to my head, but therefore we need some information:
- What terminal emulators do you use?
- What shell do you use?
- Where do you get the format string from?
- What are the locale settings?
bye,
Daniel
Daniel
Re: [PB6.xx] String manipulation bug with ${escape sequences}
To answer that I have to learn a few things about Linux first. I needed a bit of exercise anyway 
I only use the pre-installed programs so far.
Shell+Terminal emulators:
- bash+"LXTerminal 0.4.0 Terminal emulator for LXDE project" (Raspberry 4 arm-64)
- bash+"GNOME Terminal 3.46.8 for GNOME 43" (Debian x64)
Format string: I am not sure what you exactly mean. For now this is a "standalone" include that isn't finished. I hard code the ${xyz} escape sequences in a string or write them simply in a text file.
Locales:
- en_US.UTF-8 (raspberry pi arm64)
- de_DE.UTF-8 (Debian x64 VM)
There is still a part that runs on the RPi machine; for example the 2 messageboxes are displayed, and the second half of the demo text is displayed. But both linux machines they have trouble rendering the "header" formater, which puts a string into a box like that
To be able to put the title in the box, even with colors, I have to filter out all the $ escape sequences; this happens by Parsing recursively the string parameter containing the title - up to 4 times.
Either the recursion causes the strings to be corrupted (kernel switch), or they come in corrupted (compiler switch); then the frame around the title seems to contain letters that appear at other places in the "parent string" from which the command & arguments were extracted "like CLI parameters".(I never heard or don't know anything about those kind switches, or if they exist - it's just a speculation)
In the Main() procedure the String "str$" contains the escape sequences/formatters that make up most of the demo code (except the 2 messagebox calls part).
I wonder if it's on the c compiler/backend side or maybe a secret kernel switch that happens, because it happens more "around" Print() and graphical console commands.
The ParseWildcards() that returns a string has better integrity than the ParseWildcards_Back() that uses a writer function instead of returning a string; the writer uses the console commands to render the text in colors etc.
I will have to make an other demo/test code that has 0 console commands.
I only use the pre-installed programs so far.
Shell+Terminal emulators:
- bash+"LXTerminal 0.4.0 Terminal emulator for LXDE project" (Raspberry 4 arm-64)
- bash+"GNOME Terminal 3.46.8 for GNOME 43" (Debian x64)
Format string: I am not sure what you exactly mean. For now this is a "standalone" include that isn't finished. I hard code the ${xyz} escape sequences in a string or write them simply in a text file.
Locales:
- en_US.UTF-8 (raspberry pi arm64)
- de_DE.UTF-8 (Debian x64 VM)
There is still a part that runs on the RPi machine; for example the 2 messageboxes are displayed, and the second half of the demo text is displayed. But both linux machines they have trouble rendering the "header" formater, which puts a string into a box like that
Code: Select all
**********
* demo 1 *
**********
+-------+
| Demo2 |
+-------+
etc. are only correctly rendered on windows
Either the recursion causes the strings to be corrupted (kernel switch), or they come in corrupted (compiler switch); then the frame around the title seems to contain letters that appear at other places in the "parent string" from which the command & arguments were extracted "like CLI parameters".(I never heard or don't know anything about those kind switches, or if they exist - it's just a speculation)
In the Main() procedure the String "str$" contains the escape sequences/formatters that make up most of the demo code (except the 2 messagebox calls part).
I wonder if it's on the c compiler/backend side or maybe a secret kernel switch that happens, because it happens more "around" Print() and graphical console commands.
The ParseWildcards() that returns a string has better integrity than the ParseWildcards_Back() that uses a writer function instead of returning a string; the writer uses the console commands to render the text in colors etc.
I will have to make an other demo/test code that has 0 console commands.
Re: [PB6.xx] String manipulation bug with ${escape sequences}
I don't think it's a PB bug actually.
Re: [PB6.xx] String manipulation bug with ${escape sequences}
I updated the code in Post #3
The bug seems not to be related to Console/Terminal, this was a false hint. Sorry. I would not exclude it completely, that there may be some magic "bash injection" happening but I don't know even what I am imagining
Now the header formatting works a little better concerning the frame drawing, "because" the username is now constructed differently.
I modified the code and in main() and (look-up procedures) you can see that before I read the /etc/hostname file on Linux, now the lookup function uses directly UserName() and ComputerName() when the variables $username and $computername are looked up.
But this happened when I changed it: the username would pass through, but the computername sometimes not!
When I changed
to
results improved but now on Linux x64 the computer name is empty after the @ in the user name; it seems to be tabulator and not spaces since I can't mouse select parts of it, or maybe it's a console color trick but it looks very void (there are no tabs in the escape sequences/string).
IMO it has to do with recursion and returning strings from "far away" - when a prototype.s returns a string from an other prototype.s etc. in a chain "recursively"; it remembers me the problem with strings and DLL's. I had similar bugs I could not clearly define when using interfaces. Suddenly strings won't be returned or are broken. But these problems always arise when the code already has reached a greater length and complexity.
The bug seems not to be related to Console/Terminal, this was a false hint. Sorry. I would not exclude it completely, that there may be some magic "bash injection" happening but I don't know even what I am imagining
Now the header formatting works a little better concerning the frame drawing, "because" the username is now constructed differently.
I modified the code and in main() and (look-up procedures) you can see that before I read the /etc/hostname file on Linux, now the lookup function uses directly UserName() and ComputerName() when the variables $username and $computername are looked up.
But this happened when I changed it: the username would pass through, but the computername sometimes not!
When I changed
Code: Select all
ProcedureReturn ComputerName()
Code: Select all
result$=ComputerName()
; (... select/endselect)
ProcedureReturn result$
IMO it has to do with recursion and returning strings from "far away" - when a prototype.s returns a string from an other prototype.s etc. in a chain "recursively"; it remembers me the problem with strings and DLL's. I had similar bugs I could not clearly define when using interfaces. Suddenly strings won't be returned or are broken. But these problems always arise when the code already has reached a greater length and complexity.
Re: [PB6.xx] String manipulation bug with ${escape sequences}
Well, without a reproducing snippet, it's impossible to fix as it could be a side effect from another error.
Re: [PB6.xx] String manipulation bug with ${escape sequences}
I updated the code in post #3 again; little change the "header/title" function now takes a 4th parameter for indentation.
The code above produces faulty output on Raspberry (right) and the expected output on the other systems. My Debian VM terminal display yellow color (6) in black which is why the computer name looked void but I changed to color 5 in the formatter string, as the Terminal refuses to display yellow (thanks to HTML-conversion feature in the menu I found out the computer name is not void but "only invisible").
https://ibb.co/T129zpx
The faulty part is processed with ParseWildcard_Back(), written chunk/token -wise using print().
The "good" part is processed with ParseWildcard(), which returns the parsed content a string (no print/console operations involved).
I'll try this next:
- try to make/find the bug in a much shorter example
- draw text on a bitmap instead of printing in console
- change the OS for the raspberry (needs to be done to use new 6.11+ versions).
I think it's a raspberry installation related bug or side effect, either on the kernel or in the static libraries the compiler may use. I fear that I could have stolen much time for a bug I "invented" but it makes no sense that it works "flawlessly" on all systems except the raspberry. The only difference is the byte ordering on RPi is big endian but this shouldn't be of any importance for string manipulation.
The code above produces faulty output on Raspberry (right) and the expected output on the other systems. My Debian VM terminal display yellow color (6) in black which is why the computer name looked void but I changed to color 5 in the formatter string, as the Terminal refuses to display yellow (thanks to HTML-conversion feature in the menu I found out the computer name is not void but "only invisible").
https://ibb.co/T129zpx
The faulty part is processed with ParseWildcard_Back(), written chunk/token -wise using print().
The "good" part is processed with ParseWildcard(), which returns the parsed content a string (no print/console operations involved).
I'll try this next:
- try to make/find the bug in a much shorter example
- draw text on a bitmap instead of printing in console
- change the OS for the raspberry (needs to be done to use new 6.11+ versions).
I think it's a raspberry installation related bug or side effect, either on the kernel or in the static libraries the compiler may use. I fear that I could have stolen much time for a bug I "invented" but it makes no sense that it works "flawlessly" on all systems except the raspberry. The only difference is the byte ordering on RPi is big endian but this shouldn't be of any importance for string manipulation.
Re: String manipulation with ${escape sequences}
UPDATE
Finally it works. I made an OS upgrade on the RPi, all by myself using the text editor and the console
Now I can start PB6.11 and later versions, on the later version the string problem is resolved.
But when I use the old PB6.00 version, new compiled programs still have that bug, and the old compiled programs also display the bug after upgrade to the new OS.
From my POV this weird behavior happens during compilation or linking, and while IDK what the reason is I could imagine a third party library locally installed to be the culprit (or configuration), since the problem could not be reproduced on other machines but it still persists after my OS upgrade (on the Raspberry Pi 400).
It remains mysterious but for me I consider the problem "fixed"
Finally it works. I made an OS upgrade on the RPi, all by myself using the text editor and the console
Now I can start PB6.11 and later versions, on the later version the string problem is resolved.
But when I use the old PB6.00 version, new compiled programs still have that bug, and the old compiled programs also display the bug after upgrade to the new OS.
From my POV this weird behavior happens during compilation or linking, and while IDK what the reason is I could imagine a third party library locally installed to be the culprit (or configuration), since the problem could not be reproduced on other machines but it still persists after my OS upgrade (on the Raspberry Pi 400).
It remains mysterious but for me I consider the problem "fixed"