PokeS() not working with #PB_Ascii flag

[charvista]

I understand that the flag #PB_Ascii in the PokeS() command forces the program using Ascii characters in Memory, even when the program is compiled in Unicode.

My System : Vista Home Premium x86, and PB 4.50 x86. (Same problem on W7 x86)

1) Try this code below in normal compilation. You should then get two messages that the colors are the same. This works fine.

2) Try this code in Unicode compilation (Compiler Options: Create unicode executable). You will receive two different results. Is there a bug here or is there someting I am not aware of?

Code: Select all

*A=AllocateMemory(3+1)
Color1.s=Chr($0F)+Chr($7E)+Chr($65)
PokeS(*A,Color1,3,#PB_Ascii)
Color2.s=PeekS(*A,3,#PB_Ascii)
If Color1=Color2
    MessageRequester("Verify","Color1 and Color2 are identical.  Excellent.")
Else
    MessageRequester("Verify","Color1 and Color2 are different!  What happened???")
EndIf

*M=AllocateMemory(3+1)
Color1.s=Chr($0F)+Chr($7E)+Chr($9C)
PokeS(*M,Color1,3,#PB_Ascii)
Color2.s=PeekS(*M,3,#PB_Ascii)
If Color1=Color2
    MessageRequester("Verify","Color1 and Color2 are identical.  Excellent.")
Else
    MessageRequester("Verify","Color1 and Color2 are different!  What happened???")
EndIf

[charvista]

On further investigation, it seems that only *some* characters are not passing the PokeS/PeekS, as well PokeA, PokeB, PokeC etc when using their ascii value.
These are the characters $80, $82838485868788898A8B8C, $8E, $9192939495969798999A9B9C, $9E9F
meaning Chr(128,130,131,132,133,134,135,136,137,138,139,140,142,145,146,147,148,149,150,151,152,153,154,155,156,158,159).
All other characters seem to be OK.

[Little John]

My System : Vista Home Premium x86, and PB 4.50 x86. (Same problem on W7 x86)

Same problem here with PB 4.51 RC 2 x86 on Windows XP x86, compiled to Unicode.

//edit:

On further investigation, it seems that only *some* characters are not passing the PokeS/PeekS, as well PokeA, PokeB, PokeC etc when using their ascii value.

I didn't encounter a problem with PokeA() and PeekA(). What is the code that showed problems with these functions to you?

The following code -- when compiled to Unicode -- confirms the problem regarding PokeS() and PeekS():

Code: Select all

Macro Hb (_x_)
   RSet(Hex(_x_),2,"0")
EndMacro


Define *m.Ascii, i.a, p.a

*m = AllocateMemory(10)

Debug "-- PokeS() errors:"
For i = $00 To $FF
;    PokeA(*m, i)     ; works well
   PokeS(*m, Chr(i), 1, #PB_Ascii)
   If i <> *m\a
      Debug "i = $" + Hb(i) + ", *m\a = $" + Hb(*m\a)
   EndIf
Next

Debug ""

Debug "-- PeekS() errors:"
For i = $00 To $FF
   *m\a = i
;    p = PeekA(*m)    ; works well
   p = Asc(PeekS(*m, 1, #PB_Ascii))
   If *m\a <> p
      Debug "*m\a = $" + Hb(*m\a) + ", p = $" + Hb(p)
   EndIf
Next

Output:

Code: Select all

-- PokeS() errors:
i = $80, *m\a = $3F
i = $82, *m\a = $3F
i = $83, *m\a = $3F
i = $84, *m\a = $3F
i = $85, *m\a = $3F
i = $86, *m\a = $3F
i = $87, *m\a = $3F
i = $88, *m\a = $3F
i = $89, *m\a = $3F
i = $8A, *m\a = $3F
i = $8B, *m\a = $3F
i = $8C, *m\a = $3F
i = $8E, *m\a = $3F
i = $91, *m\a = $3F
i = $92, *m\a = $3F
i = $93, *m\a = $3F
i = $94, *m\a = $3F
i = $95, *m\a = $3F
i = $96, *m\a = $3F
i = $97, *m\a = $3F
i = $98, *m\a = $3F
i = $99, *m\a = $3F
i = $9A, *m\a = $3F
i = $9B, *m\a = $3F
i = $9C, *m\a = $3F
i = $9E, *m\a = $3F
i = $9F, *m\a = $3F

-- PeekS() errors:
*m\a = $80, p = $AC
*m\a = $82, p = $1A
*m\a = $83, p = $92
*m\a = $84, p = $1E
*m\a = $85, p = $26
*m\a = $86, p = $20
*m\a = $87, p = $21
*m\a = $88, p = $C6
*m\a = $89, p = $30
*m\a = $8A, p = $60
*m\a = $8B, p = $39
*m\a = $8C, p = $52
*m\a = $8E, p = $7D
*m\a = $91, p = $18
*m\a = $92, p = $19
*m\a = $93, p = $1C
*m\a = $94, p = $1D
*m\a = $95, p = $22
*m\a = $96, p = $13
*m\a = $97, p = $14
*m\a = $98, p = $DC
*m\a = $99, p = $22
*m\a = $9A, p = $61
*m\a = $9B, p = $3A
*m\a = $9C, p = $53
*m\a = $9E, p = $7E
*m\a = $9F, p = $78

Regards, Little John

[PMV]

ASCII has only 7-Bit, our "ASCII" has nothing to do with it
The "High ASCII" or "Extended ASCII" is known as code page 437,
but PB uses a wrongly called ASCII-Table: Windows-1252 which is based on
ISO-8859-1. So many of this 256 Characters can not be used with unicode in
Chr() like you did. If you are using some "special" characters from
this, you need the right encoding for this character. The code is given in
the wikipedia article if needed.

Code: Select all

Procedure.s PeekHexBytes (*source.Ascii, numBytes)
   Protected *eofSource, ret$=""

   *eofSource = *source + numBytes
   While *source < *eofSource
      ret$ + RSet(Hex(*source\a), 2, "0")
      *source + 1
   Wend

   ProcedureReturn ret$
EndProcedure


Define *M, Color1.s, Color2.s

*M = AllocateMemory(10)
CompilerIf #PB_Compiler_Unicode
  Color1 = Chr($000F) + Chr($007E) + Chr($0153)
CompilerElse
  Color1 = Chr($0F) + Chr($7E) + Chr($9C)  
CompilerEndIf
Debug "source string: " + Color1
Debug "Color1: " + PeekHexBytes(@Color1, 2*SizeOf(Character) + SizeOf(CHARACTER))
PokeS(*M, Color1, 3, #PB_Ascii)
Debug "Memory: " + PeekHexBytes(*M, 3)
Color2 = PeekS(*M, 3, #PB_Ascii)
Debug "Color2: " + PeekHexBytes(@Color2, 2*SizeOf(Character) + SizeOf(CHARACTER))
Debug "result string: " + Color2

MFG PMV

[charvista]

@Little John
With PokeA() in Unicode, the problem is the same. With PokeS() we use the complete string, while with PokeA() we poke the ascii value one by one. See the code below for clarification.

Code: Select all

*A=AllocateMemory(3+1)
Color1.s=Chr($0F)+Chr($7E)+Chr($9C)
;PokeS(*A,Color1,3,#PB_Ascii)
PokeA(*A,$0F)
PokeA(*A+1,$7E)
PokeA(*A+2,$65)
Color2.s=PeekS(*A,3,#PB_Ascii)
If Color1=Color2
    MessageRequester("Verify","Color1 and Color2 are identical.  Excellent.")
Else
    MessageRequester("Verify","Color1 and Color2 are different!  What happened???")
EndIf

However, I think PMV is right. Phew, what an extreme complication!
I need to keep the Unicode compilation because my programs sometimes display Japanese characters.
But when I open a file for binary reading AS IS, or want to simply read asciis from 0 to 255, it becomes very difficult to treat this as an ASCII file if there are "some" exceptions......
What a mess!
Okay, I am now trying to write a procedure to make it compatible with ISO-8859-1..........

@PMV
Thank you very much for clarifying the mystery....

[Little John]

@PMV:
Thank you for clarifying this!

@Little John
With PokeA() in Unicode, the problem is the same. With PokeS() we use the complete string, while with PokeA() we poke the ascii value one by one.

I know the difference between PokeS() and PokeA(), that's not the question.

See the code below for clarification.

The problem with PokeA() in Unicode is not the same as with PokeS(..., #PB_Ascii), see the code in my previous post for clarification.

Code: Select all

*A=AllocateMemory(3+1)
Color1.s=Chr($0F)+Chr($7E)+Chr($9C)
;PokeS(*A,Color1,3,#PB_Ascii)
PokeA(*A,$0F)
PokeA(*A+1,$7E)
PokeA(*A+2,$65)
Color2.s=PeekS(*A,3,#PB_Ascii)
If Color1=Color2
    MessageRequester("Verify","Color1 and Color2 are identical.  Excellent.")
Else
    MessageRequester("Verify","Color1 and Color2 are different!  What happened???")
EndIf

In this code, it's obviously PeekS(..., #PB_Ascii) that causes the problem, not PokeA().

Regards, Little John

[PMV]

However, I think PMV is right. Phew, what an extreme complication!
I need to keep the Unicode compilation because my programs sometimes display Japanese characters.
But when I open a file for binary reading AS IS, or want to simply read asciis from 0 to 255, it becomes very difficult to treat this as an ASCII file if there are "some" exceptions......
What a mess!
Okay, I am now trying to write a procedure to make it compatible with ISO-8859-1..........

The first 256 characters of the Unicode-Standard are the
same as ISO-8859-1. Like you two have seen with your for-loop
for all the 256 characters, only the characters of $80 till $9F are
changed in Windows-1252. So you only need to map this 32
characters, if you are using Windows-1252 encoding some
where in your code base. This is no problem for Windows-1252
text-files, because with optional Flag, you can say PB to read
this characters and converts it for you. No more action is
needed.

And i repeat it again, it is not PeekS who makes the mistake. It
is the programmer who expects another encoding as used. In
ISO-8859-1, $80-$9F are non-printable characters. If you print
those characters, the result is a questionmark (?), which encoding
is $3F. The second for-loop is a good example of how to get the
right encodings. PokeA can only handle 8-bit-characters, so
it doesn't know anything about this encodingproblem.

MFG PMV

[charvista]

OK, PokeA seems to be the right thing in this case.
I wrote quickly a visual Memory dumper to see the contents and indeed, it is perfect. I am glad that there is at least one simple solution!
In the good (?) old DOS time, the characters $00-$19 were the non-printable characters, and now with Windows-1252 encoding, we must take care of $80-$9F. This is completely new to me, and thanks to your Wikipedia link I learned a lot.
I would like to thank Little John and PMV for their nice support!

Code: Select all

Macro Hb (_x_)
    RSet(Hex(_x_),2,"0")
EndMacro

Procedure.d zMax(Value1.d,Value2.d)
    If Value1>Value2
        ProcedureReturn Value1
    Else
        ProcedureReturn Value2
    EndIf
EndProcedure 

Procedure zMemDump(*Buffer)
    Win=OpenWindow(#PB_Any,100,100,800,600,"Memory Dump")
    F0=LoadFont(#PB_Any,"Fixedsys",12)
    Dim T(15)
    For i=0 To 15
        T(i)=TextGadget(#PB_Any,20,40+i*20,590,15,"")
        SetGadgetFont(T(i),FontID(F0))
    Next
    Length=MemorySize(*Buffer)
    For i=0 To zMax(Length-(i*16),15*16) Step 16
        For j=0 To 15
            p=PeekA(*Buffer+i+j)
            H.s+Hb(p)+" "
        Next
        SetGadgetText(T(i/16),H)
        H.s=""
    Next

    Repeat
        Event = WaitWindowEvent()
        Select Event
            Case #PB_Event_CloseWindow
                ExitEventLoop = #True
;             Case #PB_Event_Gadget
;                 Select EventGadget()
;                     Case BtnOk
;                         ExitEventLoop = #True
;                 EndSelect
        EndSelect
    Until ExitEventLoop
    CloseWindow(Win)
EndProcedure



   *b=AllocateMemory(16*16+1)
   For i=$00 To $ff
       PokeA(*b+i,i)
   Next
   zMemDump(*b)