Read Binary File to String Problem

[datachild]

Hey im new on the board and purebasic

I hope u can help me
I want to open a BINARY file and read the data into blocks as STRING (size 1022) ... *later it should be used in a TCP Filetransfer)
The problem is PeekS seem to ignore special characters it only returns "MZ" with a textfile *.TXT all is working well :/
Please give me a hint or a solution how to Read a binary file into blocks as a STRING

This is my current code:

Code: Select all

If ReadFile(0,"D:\TEST.exe")
    While Eof(0) = 0 
      *sendID = AllocateMemory(1022)
      ReadData(0,*sendID, 1022)
      stack.s = "S#" + PeekS(*sendID,1022)
      Debug stack.s
      FreeMemory(*sendID);CLEAN UP MEMORY
    Wend
  CloseFile(0)
  Debug "EOF"
  Else
  ;ERROR
  EndIf

In VB6 u can read a Binary file in a String with this Code:

Code: Select all

Dim mystring as String

Open "FILEPATH" For Binary Access Read As #1
   mystring = Input(LOF(1), 1)
Close #1

msgbox mystring ,, ""

Thx datachild
sorry about my bad english

[DarkPlayer]

Hello,

PureBasic does not store the length of a string internal. For PureBasic a string ends when the Character 0 is detected. So if you could read all the data into a string, PureBasic would think that the string is over when there is the character 0 somewhere.

The only possibility to read binary data into a string is, to read the data into a memory block and encode it with the Base64Encoder(). The 0 character gets replaced by other characters, so that you dont have any problems with special characters. The disadvantage is that the Base64 Encoded String is > 33 % bigger than the original data.

The safest and best way is to use memory blocks with a specified size instead of strings.

DarkPlayer

[kenmo]

DarkPlayer is right - PB uses null-terminated strings, so any zero byte would falsely mark the end of it. The best plan is probably to read and write raw blocks of binary data, something like the following:

Code: Select all

Procedure.i ReadBlock(File.i, Size.i)
  Protected BytesRead.i, *Block.i = #Null
  
  If (IsFile(File))
    *Block = AllocateMemory(Size)
    If (*Block)
      BytesRead = ReadData(File, *Block, Size)
      If (BytesRead = 0)
        FreeMemory(*Block)
        *Block = #Null
      EndIf
    EndIf
  EndIf
  
  ProcedureReturn *Block
EndProcedure

Procedure.i FreeBlock(*Block.i)
  If (*Block)
    FreeMemory(*Block)
  EndIf
EndProcedure

[datachild]

thx DarkPlayer and kenmo

How could i encode the data with base64 before using Peeks() ?

Code: Select all

*Block = AllocateMemory(Size)
BytesRead = ReadData(File, *Block, Size);< -BASE64 ?

I dont really know how to implement the ur solution kenmo
Does it work like this :

Code: Select all

Procedure.i ReadBlock(File.i, Size.i)
  Protected BytesRead.i, *Block.i = #Null
  
  If (IsFile(File))
    *Block = AllocateMemory(Size)
    If (*Block)
      BytesRead = ReadData(File, *Block, Size)
      If (BytesRead = 0)
        FreeMemory(*Block)
        *Block = #Null
      EndIf
    EndIf
  EndIf
  
  ProcedureReturn *Block
EndProcedure

Procedure.i FreeBlock(*Block.i)
  If (*Block)
    FreeMemory(*Block)
  EndIf

;********************************

If ReadFile(0,"D:\TEST.exe")
    While Eof(0) = 0 
      stack.s = "S#" + PeekS(ReadBlock(0, 1022),1022); <------- Like that ?
      Debug stack.s
    Wend
  CloseFile(0)
  Debug "EOF"
  Else
  ;ERROR
  EndIf
EndProcedure

thx 4 ur time
datachild

[kenmo]

That's basically correct, but:
1. You would want to store the pointer returned from ReadBlock() in a variable, so that you can reference it again and release it when done.
2. PeekS() would still stop when it meets a 0 byte, which could lead to missing text.

But the "block" itself holds all the original binary data. Depending on if you're writing it to a file, or sending it through a network connection like you mentioned, you might use WriteData, SendNetworkData, etc.

If you convert the binary data to a base-64 string (Wikipedia link if you don't what it's for), then you can treat it just like a normal string without worrying about tricky binary characters. However you must convert it back to raw data before displaying, writing, or sending it.

To get you started:

Code: Select all

Procedure.s ReadBlock64(File.i, Size.i)
  Protected BytesRead.i, *Block.i, *Encode.i
  Protected Result.s = ""
 
  If (IsFile(File))
    *Block = AllocateMemory(Size)
    If (*Block)
      BytesRead = ReadData(File, *Block, Size)
      If (BytesRead > 0)
        *Encode = AllocateMemory(Int(BytesRead * 1.4))
        If (*Encode)
          Base64Encoder(*Block, BytesRead, *Encode, Int(BytesRead * 1.4))
          Result = PeekS(*Encode)
          FreeMemory(*Encode)
        EndIf
      Else
      EndIf
      FreeMemory(*Block)
    EndIf
  EndIf
 
  ProcedureReturn Result
EndProcedure

; Write 100 random bytes between 0 and 32
If CreateFile(0, "c:\temp.dat")
  For i = 1 To 100
    WriteByte(0, Random(32))
  Next i
  CloseFile(0)
EndIf

; Read the data back as a safe, alphanumeric string
If ReadFile(0, "c:\temp.dat")
  Encoded.s = ReadBlock64(0, 1000)
  Debug Encoded
  CloseFile(0)
EndIf

[datachild]

thx kenmo i got it working by implementing a simple base64 encoding like DarkPlayer and u said
however this should be away from professional code but iam still learning... :>

Heres is my solution for those who interrested !
... maybe somone can improve it

Code: Select all

If ReadFile(0,sfile.s);sfile.s = "C:\binaryfile.samle"
    While Eof(0) = 0 
      *sendID = AllocateMemory(1024)
      *baseID = AllocateMemory(2048)
      ReadData(0,*sendID, 1024)
      Base64Encoder(*sendID, 1024, *baseID, 2048);thats the trick :)
      stack.s = PeekS(*baseID,2048)
      Debug stack.s
      FreeMemory(*sendID);Buffer Cleanup maybe this could be done in anotherway so i dont need to Allocate every "LOOP" again dunno...
      FreeMemory(*baseID)
    Wend
  CloseFile(0)
 Else
 EndIf

thx 4 ur help
datachild

[kenmo]

;Buffer Cleanup maybe this could be done in anotherway so i dont need to Allocate every "LOOP" again dunno...

Yes it sure can - just move the AllocateMemory()s before the loop and the FreeMemory()s after it.

And if you want to be completely safe, you should check that the two memory blocks were created (pointers aren't zero) such as

Code: Select all

;allocatememory
if (*sendID and *baseID)
;loop here
;free memory
endif

EDIT - just realized a possible error. If the final block doesn't fill the entire 1024 bytes, it will have the previous block's data leftover (using the method in this post). This will give you the wrong base64 string..... so your way is fine. But if speed becomes an issue, consider my tips and you will probably have to write a slightly more complicated reading algorith.

[datachild]

EDIT - just realized a possible error. If the final block doesn't fill the entire 1024 bytes, it will have the previous block's data leftover (using the method in this post). This will give you the wrong base64 string..... so your way is fine. But if speed becomes an issue, consider my tips and you will probably have to write a slightly more complicated reading algorith.

...i thought if u free the memory and reallocate it there should be no data left or iam wrong ?

but at least its not working as well as i thought first :/
it would be nice to get some further advice how to read & send binary data also without the use of base64...

thx DC

[TerryHough]

Unless you have a need to view or modify the block of the file you are reading, then you are on the wrong track.

If you just want to transmit via FTP the "binary" file, an .exe file as in your example, then you should just use the FTP Library SendFTPFile command. See this post for an example code
http://www.purebasic.fr/english/viewtop ... endftpfile

That should get you started, and if you get confused let me know. I think I have a better example that I can send you.

[datachild]

I dont need ftp filetransfer...
im looking for how to transfer files over the network...
using NetworkSendData ... without the need of encoding it in base64...

[cas]

Client part would be something like this (warning:this code is not tested):

Code: Select all

EnableExplicit
Define FileName.s="D:\TEST.exe"
Define Server.s="127.0.0.1"
Define Port=8956

If InitNetwork() = 0
  Debug "there is no TCP/IP stack available on the system"
  End
EndIf

Define connectionID=OpenNetworkConnection(Server.s,Port)

If connectionID
  Select FileSize(FileName.s)
    Case -2
      Debug "file is a directory"
    Case -1
      Debug "file not found"
    Case 0
      Debug "file is empty"
    Default
      Define FilePBHandle=ReadFile(#PB_Any,FileName.s)
      If FilePBHandle
        Define TotalFileSize=Lof(FilePBHandle)
        Define *DataToSend=AllocateMemory(TotalFileSize) ;allocate memory for file data
        ReadData(FilePBHandle,*DataToSend,TotalFileSize) ;read file to allocated memory
        CloseFile(FilePBHandle) ;close file
        Define DataSent_total=0
        Define DataSent_now=0
        Repeat
          DataSent_now=SendNetworkData(connectionID,*DataToSend+DataSent_total,TotalFileSize-DataSent_total)
          If DataSent_now=<0
            Delay(50) ; TODO: WSAGetLastError_() !!!
          Else
            DataSent_total+DataSent_now
          EndIf
        Until DataSent_total=TotalFileSize
        CloseNetworkConnection(connectionID)
        FreeMemory(*DataToSend) ;free allocated memory
      Else
        Debug "failed to read file"
      EndIf
  EndSelect
Else
  Debug "failed to connect"
EndIf

[datachild]

thx cas but this wont work for me because ur sample wont support big files...

there was a example by kenmo...

Code: Select all

Procedure.i ReadBlock(File.i, Size.i)
  Protected BytesRead.i, *Block.i = #Null
  
  If (IsFile(File))
    *Block = AllocateMemory(Size)
    If (*Block)
      BytesRead = ReadData(File, *Block, Size)
      If (BytesRead = 0)
        FreeMemory(*Block)
        *Block = #Null
      EndIf
    EndIf
  EndIf
  
  ProcedureReturn *Block
EndProcedure

Procedure.i FreeBlock(*Block.i)
  If (*Block)
    FreeMemory(*Block)
  EndIf
EndProcedure

the problem is that binary files containing 0 bytes...
so it will come to troulble using PeekS() function...

when i understand it right the sample of kenmo is splitting the data by 0bytes into a block...
the problem might be the size of the blocks... which vary...

so i coded my solution with the help of kenmo and darkplayer with their idea to encode the data with base64
so i have the option to get set the block size of the data i read in...

Code: Select all

If ReadFile(0,sfile.s);sfile.s = "C:\binaryfile.samle"
    While Eof(0) = 0 
      *sendID = AllocateMemory(1024)
      *baseID = AllocateMemory(2048)
      ReadData(0,*sendID, 1024)
      Base64Encoder(*sendID, 1024, *baseID, 2048);thats the trick :)
      stack.s = PeekS(*baseID,2048)
      Debug stack.s
      FreeMemory(*sendID);Buffer Cleanup maybe this could be done in anotherway so i dont need to Allocate every "LOOP" again dunno...
      FreeMemory(*baseID)
    Wend
  CloseFile(0)
Else
EndIf

but iam not happy with my solution...

so my question is there some other way to send files of any size over the net ?
maybe without sending a string using PeekS() & base64...

i hope u can help me and sorry about my english

[helpy]

so my question is there some other way to send files of any size over the net ?
maybe without sending a string using PeekS() & base64...

i hope u can help me and sorry about my english

Darkplayer already said:

The safest and best way is to use memory blocks with a specified size instead of strings.

Have a look at:

• SendNetworkData
• ReceiveNetworkData
• AllocateMemory
• FreeMemory
• PureBasic Reference Manual

cu,
guido

[cas]

thx cas but this wont work for me because ur sample wont support big files...

It's easy to add another loop for ReadData and one variable for storing offset.

so my question is there some other way to send files of any size over the net ?
maybe without sending a string using PeekS() & base64...

i hope u can help me and sorry about my english

Actually, there is Search function integrated in forums, take a look at http://www.purebasic.fr/english/viewtop ... 12&t=23202 which has more advanced code with calculating speed of transfer and gui examples...
Or take helpy's advice.