Curious FILO behavior

[SMaag]

here an updated version to get all Characters of a String wiht pChar

Code: Select all

EnableExplicit

Structure pChar
  a.a[0]    ; Byte access, 'ASCii'
  c.c[0]    ; Char access
  u.u[0]    ; unsigned WORD access 'unicode'
EndStructure

Define s$ = "12" + #CRLF$
Define *Char.pChar

*Char = @s$

; Debug *Char\a[0]
; Debug *Char\a[1]  ; on unicode systems this will be 0

Define I
For I = 0 To Len(s$)-1
  Debug *Char\c[I]  ; with .c char access it's always the character, 1Byte at ASCii System 2Byte at Unicode 
Next

49
50
13 ; here the CR
10 ; and here the LF

[Randy Walker]

Ok, but is it possible to do that without using structure? That structure stuff just warps my brain.

[jacdelad]

Sorry if this sounds rude, but structures and EnableExplicit viewtopic.php?p=632644#p632644 are fundamental in PureBasic. Plus this structure isn't even hard to understand. You joined the forum 20 years ago, how can you code for so long avoiding both at will?

[SMaag]

it is possible without Structure too. With PeekC you will get same result.
But as long as you do not explain what exactly you want do, we have to
speculate what's your problem and how to solve it.

here the example with Pointer and PeekC, what's doing same as the Code with Structure

Code: Select all

EnableExplicit

Structure pChar
  a.a[0]    ; Byte access, 'ASCii'
  c.c[0]    ; Char access
  u.u[0]    ; unsigned WORD access 'unicode'
EndStructure

Define s$ = "12" + #CRLF$
Define *Char.pChar

*Char = @s$

; Debug *Char\a[0]
; Debug *Char\a[1]  ; on unicode systems this will be 0

Define I
For I = 0 To Len(s$)-1
  Debug *Char\c[I]  ; with .c char access it's always the character, 1Byte at ASCii System 2Byte at Unicode 
  Debug PeekC(*Char + I * SizeOf(Character))  ; same as Debug *Char\c[I] 
Next

[idle]

it is possible without Structure too. With PeekC you will get same result.
But as long as you do not explain what exactly you want do, we have to
speculate what's your problem and how to solve it.

here the example with Pointer and PeekC, what's doing same as the Code with Structure

Code: Select all

EnableExplicit

Structure pChar
  a.a[0]    ; Byte access, 'ASCii'
  c.c[0]    ; Char access
  u.u[0]    ; unsigned WORD access 'unicode'
EndStructure

Define s$ = "12" + #CRLF$
Define *Char.pChar

*Char = @s$

; Debug *Char\a[0]
; Debug *Char\a[1]  ; on unicode systems this will be 0

Define I
For I = 0 To Len(s$)-1
  Debug *Char\c[I]  ; with .c char access it's always the character, 1Byte at ASCii System 2Byte at Unicode 
  Debug PeekC(*Char + I * SizeOf(Character))  ; same as Debug *Char\c[I] 
Next

you should use a structure union

Code: Select all

Structure pChar
  StructureUnion
  a.a[0]    ; Byte access, 'ASCii'
  c.c[0]    ; Char access
  u.u[0]    ; unsigned WORD access 'unicode'
  EndStructureUnion
EndStructure

[Randy Walker]

Sorry if this sounds rude, but structures and EnableExplicit viewtopic.php?p=632644#p632644 are fundamental in PureBasic. Plus this structure isn't even hard to understand. You joined the forum 20 years ago, how can you code for so long avoiding both at will?

Huh Huh Huh Not rude. A little Naive perhaps. Most things you understand are easy. I don't get structures, and hence never use them unless I have a working Sample and no other alternative. Even then, I still need to understand what to plug in where to get the results I'm looking for and that structured sample makes no sense to me at all. Call me stupid. Duhhh. Not going make any difference. I also noticed people who understand SQL can't seem to write code to do anything without putting it into an SQL database. Seems the same aplies to structures. To each his own I guess.

[Randy Walker]

You use a word variable.
This uses 2 bytes and ... they are stored with the endianess of the CPU
So your bytes needs top be swapped.

That's the reason why I used ShowMemoryViewer, that you can see what happens.

You need to know about endianess if you use such things.

Ok, so I went and read up a little bit on those messed up endians, and I Guess it kind of explains what happened to my CRLF getting reversed. Can't say I understand completely but I see there is a reason for it and something I just have to live with. Thanks for that tip.

[Randy Walker]

You joined the forum 20 years ago, how can you code for so long avoiding both at will?

I dabble a LOT more than I code. Here is my life's story in a nutshell if you really want to understand how it is I code out of necessity, not out of pure brilliance.
viewtopic.php?t=85980
If I don't need it then why do I need it? Makes no sense to me. PureBasic offers a LOT of stuff that I have no use for and no interest in exploring. Why should I? I don't need it.

[SMaag]

once again speculating what you want to do!

Do you want to find the position of the CRLF?
Do you want to remove it from the String?


What is the String Data format in the File?
- 1Byte per Character ASCII or 2 Byte per Character Unicode?
- if it is 2Byte unicode: where is the Data from: Intel Byte format (little Endian) or Motorola Byte format (big endian)

here an example how to remove the CRLF from a String

Code: Select all

EnableExplicit 

Define s$         ; the String with a CRLF
Define res$       ; the result without CRLF

s$="12345" + #CRLF$
Debug Len(s$)

Define pos = FindString(s$, #CRLF$)

If pos = 0
  Debug "No CRLF found!"
  res$ = s$  
Else
  res$ = Left(s$, pos-1) ; remove the CRLF from the String
EndIf

Debug Len(res$) 

[Randy Walker]

I'm sorry SMaag. Not trying to remove the CRLF. I think what I was doing is reading through a help file until I reached a specific string ending with the CRLF, with that line marking the start of a new record. It's all so very unclear because I was such a novice at the time I didn't realize how important it was to comment everything. This particular area of the code is followed with more bit shifting that makes no sense to me. Here is a snippet with the other bit shifting magic:

Code: Select all

      Repeat
        c.i = c.i + 1
        a.i = (a.i >> 8) + (h.i << 24)
        ;Debug a.i
        h.i = ReadByte(0)
        redBytes = redBytes + 1    ; count only text and tabs
        Xit1.W = Xit1.W << 8 + h.i
        xitl =  Val(Right(Hex(Xit1),2))
        ; If h.i = 13
        ; Break
        ; EndIf
      Until Xit1.W & $FF = $0D    ; CR at end of line, bail out

I'm quite sure there was a more logical way to do whatever the hell I was I was trying to do. Whatever that is I should use it to replace that code. All I remember at this point is I created some goofy algorithm to prioritize records in a user search. Trying to replicate that during a total rewrite would be a huge undertaking since I don't have a clue how it all worked. Clear in my mind at the time I created it. I expect. Best I can come up with now is a big DuHhhh. I was hoping if I could get this CRLF thing sorted out then I could begin to unravel the rest of that algorithm. This probably all made a lot more sense back in the 16 bit era.
The most frustrating thing about this is it all worked fine in ver 5.40 until recently when I added a new record, and removing the record does not allow the code to work again. Never worked in ver 6.xx.

[normeus]

Randy,
I did not read the whole thread but here is an example ( taken from your example) that might help you with your current code, it uses ShowMemoryViewer() and CallDebuger, you can add those two to your code while debugging to see what's happening to your bytes:

Code: Select all

; using ShowMemoryViewer()   will help you.



s$ = "12"
Debug  StringByteLength(s$) ; this command will tell you the number of bytes = 4 , number of chars = 2
Debug "^^^ number of bytes fo s$ equals 4 , number of characters = 2" 
Xit1.W = 33
Debug Xit1.W
Debug "^^^^ xit1.W has 2 charactes and it will always be 2 bytes for numbers -32768 to +32767 "
ShowMemoryViewer(@Xit1,2); it will show "21 00"
Debug "PRESS PLAY ON THE MENU AT THE TOP SO THE PROGRAM CONTIUES EXECUTION" 
Debug"- CONTENTS OF Xit1.W IN MEMORY VIEWER 21 00"
CallDebugger 

TAG_CR$ = Chr(13) + Chr(10)
redByte =13
Xit1.W = (Xit1.W << 8) + redByte ;move right byte To the left & add $0D
Debug Xit1.W
ShowMemoryViewer(@Xit1,2)
Debug "PRESS PLAY ON THE MENU AT THE TOP SO THE PROGRAM CONTIUES EXECUTION "
Debug "- CONTENTS OF Xit1.W IN MEMORY VIEWER  WENT FROM 21 00    TO  0D 21"
CallDebugger
redByte =10
Xit1.W = (Xit1.W << 8) + redByte ;move right byte To the left & add $0A
ShowMemoryViewer(@Xit1,2)
Debug "PRESS PLAY ON THE MENU AT THE TOP SO THE PROGRAM CONTIUES EXECUTION "
Debug "- CONTENTS OF Xit1.W IN MEMORY VIEWER  WENT FROM 0D 21    TO  0A 0D"
CallDebugger
PokeW(@s$,Xit1.W)
ShowMemoryViewer(@S$,4)
Debug "PRESS PLAY ON THE MENU AT THE TOP SO THE PROGRAM CONTIUES EXECUTION "
Debug "- CONTENTS OF MEMORY VIEWER  NOW WITH s$ =  0A 0D 32 00"
CallDebugger

Debug s$
Debug "^^^ NUMBER 2 IS AT THE END BECAUSE THOSE BYTES WERE NEVER CHANGED"
Debug "     NUMBER 3 WAS REPLACED BY THE CHARACTER REPRESENTATION OF 0A 0D"

this code was writen with PB5.40LTSx86 on a windows 10 pc
Norm

[Randy Walker]

"- CONTENTS OF Xit1.W IN MEMORY VIEWER WENT FROM 21 00 TO 0D 21"
"- CONTENTS OF Xit1.W IN MEMORY VIEWER WENT FROM 0D 21 TO 0A 0D"

That's what's messiing me up. I guess that comes back around to the endian thing, and I don't like it.
I think of a string as existing left to right, like the way we type, so left("Word",1) would equal "w", NOT "d", uless I push that word 3 bytes to the left using << 24.
Just going to take some time for me to wrap my head around that endian bolony.

[normeus]

Randy,
If you create a fake example file of what the input file should be, it would be much easier to help you. It sounds like it is a tab delimited file with a carriage return at the end of the line. The last snipet of code has no meaning whatsoever without and input file, and by the way I completely forgot what the orginal problem was.

here is a site where you can create a random text file:
https://onlinefiletools.com/generate-random-text-file

[Randy Walker]

Randy,
If you create a fake example file of what the input file should be, it would be much easier to help you. It sounds like it is a tab delimited file with a carriage return at the end of the line. The last snippet of code has no meaning whatsoever without and input file, and by the way I completely forgot what the original problem was.

here is a site where you can create a random text file:
https://onlinefiletools.com/generate-random-text-file

Yes, I've considered creating fake data files and that might help with troubleshooting some of the code but I would have to strip out code for the FTP file sharing parts too. That's a complex mess. The data files are not any standard format except for the customer database file which includes encrypted login credentials for stores. So, that random text file thing you offered would be of little to no use. I would have to strip out the decryption stuff too. I would also have to write code to create the fake files with random data. Getting the source code and fake files to you is a last challenge in one big zip file. It's a lot of tedious work. Oh how I hate to think it would become necessary.

[SMaag]

Now I understand more or less.
Yout try to understand your very old undocumented code in GFA Basic und try to convert it to Purebasic!

What you are basically doing is:

Read a File Byte by Byte, and combine the last 2 Bytes read to a Word Var using Bitshift. Then you check this 2 Bytes for
'CRLF'. But you do not find 'CRLF' you find 'LFCR'. That's a thing what drives you crazy. It looks like changed Endianess but it isn't!
You use ASCII Strings and they do not have Endianess because it's only 1 Byte per Character.

So you changend the Endianess with the Bitshift Operation!

Remember: Intel = Little Endian (low Byte first)

If you Read 'CR' and shift it left 8Bit => 'CR' is now in the HightByte of Xit1
Now you add 'LF' what it's in the low Byte of Xit1.

As result you get LoByte = 'LF' : HiByte = 'CR'
Remember LittleEndian => LoByte first
if you use a Peek command to write it back to Memory you get => 'LFCR'
That's absolute correct!

For seperating the Datasets delimited by 'CRLF' I would use the PureBasic command ReadLine()
what's doing this automatically. My normal way is to read all such Datasets into a Stringlist() and then
step trough all Datasets with ForEach()