Buffered file transfer with unlimited filesize

[Inf0Byt3]

Code updated For 5.20+


Hi, i just managed to make a buffered file transfer test and it seems to work great. The file was sent successfully here. There is not too much error-checking inside but this can be optimized and improved by anyone. Don't forget to enter a valid file path in the client's code and after executing the server and the client to press a key in the client screen to begin the transfer. If you manage to make it better please share it because this is a pretty good improvement to the network functions. We can add a callback too .

The server.pb file

Code: Select all

  ;by Inf0Byt3
  OpenConsole()
  
  #Command = "!ASLKLK#@*)SAUJ!"
  #DataBuf = "!SI#)()U!J:LSAJ!"
  
  Global FileHandle

  *Buffer = AllocateMemory(1024)

  If InitNetwork()
   CreateNetworkServer(0, 6443, #PB_Network_TCP)
   PrintN("Server created...")
   Repeat
    SEvent = NetworkServerEvent()
     If SEvent
      ClientID = EventClient()
      Select SEvent
       
       Case 1

       Case 2
        ReceiveNetworkData(ClientID, *Buffer, 1024)
        Command.s = PeekS(*Buffer,128)
        
       ;00000000000000000000000000000
        Select Left(Command,16)
         
         Case #Command
          DataRecv.s = Mid(Command,17,Len(Command)-16)
          Filesize.l = Val(StringField(DataRecv,1,"|"))
          Filename.s = StringField(DataRecv,2,"|")
          FileHandle = CreateFile(#PB_Any,"Received.txt")
         
         Case #DataBuf
          LengthRec = Val(RemoveString(Mid(Command,17,8),"X"))
          WriteData(FileHandle,*Buffer+24,LengthRec)
        
        EndSelect
        FreeMemory(*Buffer)
       ;00000000000000000000000000000

       Case 4
        Quit = 1
      
      EndSelect
    EndIf
   Until Quit = 1 
   CloseNetworkServer(0)
  Else
   MessageRequester("Error", "Can't create the server (port in use ?).", 0)
  EndIf

The client.pb file (execute after the server and press a key in the console)

Code: Select all

OpenConsole()

#Command = "!ASLKLK#@*)SAUJ!"
#DataBuf = "!SI#)()U!J:LSAJ!"

If InitNetwork() = 0
  PrintN("Can't initialize the network !")
  Input()
  End
EndIf

;!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
File.s = "yourfilepathhere"
;!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Global Sent,Length

ConnectionID = OpenNetworkConnection("127.0.0.1", 6443, #PB_Network_TCP)

If ConnectionID

  ;Repeat
   
   SendNetworkString(ConnectionID, #Command + Str(FileSize(File))+"|"+GetFilePart(File))
   Input()
   If ReadFile(0,File)
    Length = Lof(0)
    
    While Sent < Length
     *DataSend = AllocateMemory(1024)
     *Temp = AllocateMemory(1000)
     DataR = ReadData(0,*Temp,1000)
     BufferLength.s = LSet(Str(DataR),8,"X")
     Signature.s = #DataBuf+BufferLength+"!"
     PokeS(*DataSend,Signature)
     CopyMemory(*Temp,*DataSend+24,DataR)
     SendNetworkData(ConnectionID,*DataSend,DataR+24)
     FreeMemory(*Temp)
     FreeMemory(*DataSend)
     Sent + DataR
     PrintN(Str(Sent)+" -> "+Str(Length)+" -> "+Str(x))
     x + 1
    Wend
    
    CloseFile(0)
    CloseNetworkConnection(ConnectionID)
   EndIf
   Input()

Else
  MessageRequester("Client", "Can't find the server!!! Is it launched ?", 0)
EndIf

[NoahPhense]

nice code..

- np

[Anden]

Yipieeh. At last someone (thanks Inf0Byt3) started the revolution

Things to do:

Supply "filepart number" and check it (must be +1 for each part. That ensures that the file is always put together in the right sequence). Could also be used as a progress indicator!

Unique #Command and #DataBuf values (never underestimate the odds)

Optimize E.g. Move Allocate and FreeMem in client out of the While loop

[Inf0Byt3]

Thanks! I'm on it... For now i'm implementing a hash check on the parts and the filepart number.

[Bonne_den_kule]

I made a similar program with compression, start-stop function and progress bar.

[Inf0Byt3]

Wow that's very nice... Any chance to share it (or the source) ?. I am doing something similar but not with compression for now. Although it can be easily implemented.

[Konne]

Ok I think there's a problem with your Receive Part.
The PB-Help tells u that the Result of ReceiveNetworkData() is te Size of Data realy read. So if the program didn't receive 1024 Byte yet u have wrong values in the file. I can tell u It's much easyer to use the Api (And it's Linux etc compatible too!).

[Inf0Byt3]

Oh, I know that... This is just a skeleton... It can be improved, but that parts must be coded inside your program to make them work as you wish. It would be nice to make it send a confirmation after the bytes were received and if it is wrong (e.g. not all bytes received or a hash fail) to resend that piece.

[Bonne_den_kule]

Wow that's very nice... Any chance to share it (or the source) ?. I am doing something similar but not with compression for now. Although it can be easily implemented.

I can send you the source tomorrow.
But is still a bit unstable.
PM me your email or Skype user if any.

[Bonne_den_kule]

Oh, I know that... This is just a skeleton... It can be improved, but that parts must be coded inside your program to make them work as you wish. It would be nice to make it send a confirmation after the bytes were received and if it is wrong (e.g. not all bytes received or a hash fail) to resend that piece.

I think that the TCP/IP protocol ensures that the data is not corrupted.

[Shannara]

Cool, would be sweet to have updated and/or bug fixed code

[Inf0Byt3]

I'm on it .

[Tranquil]

I think that the TCP/IP protocol ensures that the data is not corrupted.

TCP ensures this, you are correct. But you have also to check for TCP errors (eg buffer overflows). If not, you will loose datas. Probably not on localhost/localhost connections but for sure on internet ones.

[Inf0Byt3]

It's a bit harder than it seems... Now I handle the network events in a thread. The problem is that it's using too much CPU (98-99%) and if I put a delay() in it it does not receive the whole file . Any advices?

[Edit]

I got it... Delay when the events are 0 . It's working ok now (0-1%).

[Pantcho!!]

Very Nice!
Looking forward to see it done with hash error checking