Correctly(!) networking (send & receive strings)

Share your advanced PureBasic knowledge/code with the community.
BackupUser
PureBasic Guru
PureBasic Guru
Posts: 16777133
Joined: Tue Apr 22, 2003 7:42 pm

Correctly(!) networking (send & receive strings)

Post by BackupUser »

Restored from previous forum. Originally posted by tranquil.

Hi ppl,

as I announced it, I would show how to send and receive datas via network correctly couse I saw many codes in this forum who does not. I tried to use some variablenames to make it clear. This routine can be optimized a lot but possible its not as clear as it is yet. Have fun.

Code: Select all

; How to send and receive Strings via network correctly

; The Send-Routine is quite simple
; You should use a threads to send datas, espacially on the server to avoid blockings.
; Here is nothing special but I will try to explain every line.


    Procedure SendString(id,txt$)     ; connected Socket, string$
    
    ; calculate bufferlength
    
    length.l=Len(txt$)+1        ; String + EOL
    length.l+4                  ; Packet length at beginning
    length.l+4                  ; CRC32

    *mem.l=GlobalAlloc_(#GMEM_FIXED | #GMEM_ZEROINIT, length.l) ; Allocate needed memory
    
    PokeL(*mem.l,length.l)      ; Store the Header (Size of Packet)
    PokeS(*mem.l+4,txt$)        ; Store String (Datas)

    crc.l=CRC32Fingerprint(*mem.l+4,length.l-    ; Create CRC32 Checksum - Not really needed
    PokeL(*mem.l+length.l-4,crc.l)                ; Store it at the end of the packet
    
    Repeat                                       ; Loop for Datasend
      send=send_(id,*mem.l,length,0)              ; Send our Datas
      sended+send                                 ; Count our sended Bytes
      If send=#SOCKET_ERROR                       ; Check for socket error
          wsaerror=wsagetlasterror_()
          If wsaerror=#WSANOTINITIALISED:err$="Wsa not initialized":EndIf
          If wsaerror=#WSAENETDOWN :err$="The Windows Sockets implementation has detected that the network subsystem has failed.":EndIf
          If wsaerror=#WSAENOTCONN :err$="Socket not connected.":EndIf
          If wsaerror=#WSAEINTR :err$="The (blocking) call was canceled using WSACancelBlocking":EndIf
          If wsaerror=#WSAEINPROGRESS :err$="A blocking Windows Sockets operation is in progress.":EndIf
          If wsaerror=#WSAENOTSOCK :err$="The descriptor is not a socket.":EndIf
          If wsaerror=#WSAEOPNOTSUPP :err$="MSG_OOB was specified, but the socket is not of type SOCK_STREAM.":EndIf
          If wsaerror=#WSAESHUTDOWN :err$="The socket has been shut down; it is not possible to recv on a socket after shutdown has been invoked with how set to 0 or 2.":EndIf
          If wsaerror=#WSAEMSGSIZE :err$="The datagram was too large to fit into the specified buffer and was truncated.":EndIf
          If wsaerror=#WSAEINVAL :err$="The socket has not been bound with bind.":EndIf
          If wsaerror=#WSAECONNABORTED :err$="The virtual circuit was aborted due to timeout or other failure.":EndIf
          If wsaerror=#WSAECONNRESET :err$="The virtual circuit was reset by the remote side.":EndIf
          If wsaerror=#WSAEWOULDBLOCK :err$="The socket is marked as nonblocking":EndIf        
          
          ; This is VERY importend to check for.
          ; If the TCP/IP Buffer is full and your datas can not send as fast as your app it sends, you get
          ; this error messages. You have to send your datas again and again till its complete.
          ; There is another methode too, but I think this is the easiest way.
      
          If wsaerror=#WSAEWOULDBLOCK:send=0:sended=0:Delay(25):EndIf 
      
      EndIf ; Errorcheck
    
    Until sended>=length Or send=#SOCKET_ERROR   ; Loop as long as all Datas has been send
    GlobalFree_(*mem)                                 ; Free our Memory again
    EndProcedure

 
; The following receive-code receives a string that is send with my send procedure above.
; I make it a little bit shorter as the normal one in our app couse that is MUCH bigger.
; Short explanation to the variables:
; totalreceived   ->   amount of datas that are allready received on the buffer
; receiveexpected ->   The expected Size of the sended String
;
; The Datapackage is send in the following shema:
;
; Packetsite.l -> 4 Bytes Packetsize
; string$      -> unknown length, chr(0) terminated
; CRC32        -> optional a CRC32 at the end (not realy needed)
;
; HOW TO USE:
; The receive routine should only be called if an networkevent on this socket is received.
; The code shown here is only to handle one user, that means you can only use it on a clients side.
; If you want to use it on server side you will need a buffer for every user connected.
; The vars Totalreceived.l and receiveexpected.l should be in a list for every user too.
;
; After connecting to a server you have to catch incoming events you have to wait for.
; Use WSAAsyncSelect_() to do this. Its easy to use. Add the flags FD_Read to it. Thats all you need.
; Now you can use WaitWindowEvent() for waiting for network events too. 
;
; Dont forget to initialize the network and allocate the *receivebuffer to eg 10000 Bytes first.
; Should be enough but depends on you stringsize.

; here we go...


     ; --->>> RECEIVEROUTINE 0             ; We allready know our Packetsize, so we can receive it
      received=recv_(serversocket.l,*receivebuffer.l+totalreceived.l,receiveexpected-totalreceived,0) ; Receive and attach the datas to our buffer
      
      If received#SOCKET_ERROR                      ;WSA Error detected? No? Then....
        If received>0:totalreceived+received:EndIf   ;add the received bytes to our Bytecounter in totalreceived
        If totalreceived=PeekL(*receivebuffer.l) ; Is our packet complete?
         
          command$=PeekS(*receivebuffer+4)       ; Get our String from Memorybuffer +4 (couse 0 is packetsize)

          totalreceived=0                        ; Datapacket complete, reset value and wait for new one
          receiveexpected=-1                     ; Wait for new packetheader but do not execute the next part!
        
        EndIf
      
      Else
      
        ; YOUR ERRORHANDLING HERE
      
      EndIf
     EndIf
      
     If totalreceived#SOCKET_ERROR
        If received=4:receiveexpected=PeekL(*receivebuffer):EndIf        ; Completed yet?
        If received>0:totalreceived+received:EndIf                       ; Count our Received bytes (we need 4!)
      EndIf
     
     
      ; Your Errorhandling here
      
     EndIf
         
     If receiveexpected=-1:receiveexpected=0:EndIf    ; This is only to avoid receiving datas that has to wait in the queue
                                                       ; Until the next event is processed

Mike

Tranquilizer/ Secretly!
http://www.secretly.de
Registred PureBasic User
BackupUser
PureBasic Guru
PureBasic Guru
Posts: 16777133
Joined: Tue Apr 22, 2003 7:42 pm

Post by BackupUser »

Restored from previous forum. Originally posted by MrVainSCL.

Cool... This is the first really correct working network example :wink: Very nice work Mike! Now we have solved the damn network problem and can still continue with our project! I´m back with FULL MOTIVATION, yeah :wink:))



PIII450, 256MB Ram, 6GB HD, RivaTNT, DirectX8.1, SB AWE64, Win2000 + all Updates...

greetz
MrVainSCL! aka Thorsten
BackupUser
PureBasic Guru
PureBasic Guru
Posts: 16777133
Joined: Tue Apr 22, 2003 7:42 pm

Post by BackupUser »

Restored from previous forum. Originally posted by fred.

Pff, I've done a webserver, a FTP server many other network tools and never encounter any problems... So I don't understand your sentence :)
BackupUser
PureBasic Guru
PureBasic Guru
Posts: 16777133
Joined: Tue Apr 22, 2003 7:42 pm

Post by BackupUser »

Restored from previous forum. Originally posted by tranquil.
Originally posted by fred

Pff, I've done a webserver, a FTP server many other network tools and never encounter any problems... So I don't understand your sentence :)
Sorry Fred, I took a closer look on your FTP the last 30 minutes and tried it also in my LAN and it definitly do not work correctly. More fails encountered while due to more connections to the server. The client received trash data-strings (commandos).

I tried with WinCommander FTP and FlashFXP. Both latest versions.

Cheers
Mike

Tranquilizer/ Secretly!
http://www.secretly.de
Registred PureBasic User
BackupUser
PureBasic Guru
PureBasic Guru
Posts: 16777133
Joined: Tue Apr 22, 2003 7:42 pm

Post by BackupUser »

Restored from previous forum. Originally posted by fred.

The FTP server you have is not finished and do nothing :). Better try the web server. I use it 24/24 on my gateway and it's still running :)

Fred - AlphaSND
BackupUser
PureBasic Guru
PureBasic Guru
Posts: 16777133
Joined: Tue Apr 22, 2003 7:42 pm

Post by BackupUser »

Restored from previous forum. Originally posted by MrVainSCL.

Hello Fred
I think the problem that Tranquil tried to explain here with his good documentated and 100% working source example is how to handle and solve a network problem that nearly all network coders on this forum dont check nor notice in there appz!

The problem is just simple... Send a lot of data packets from SOURCE to DESTINATION... If the DESTINATION cant handle all the incoming data packets as fast as the SOURCE tried to send, the dataway is blocked and so you will get the systemfailure message: #WSAEWOULDBLOCK !!! If so, you have only to wait a small moment (for example 25 millisecs) and try to send the last packet again and again, until the DESTINATION can handle it... Thats all!!!

Have a look to many network-coding-related-sites and on most of it you will find the same problem and how to solve it... Same way as Tranquil used... (very easy and 100% working one)...

I saw some PB network examples but nobody who handles the stuff correctly! I cant believe that so many (prof coders too) are using delay(1) or higher in there mainloops for handling the network events... Have you ever thinked about how to code a stable and fast industry-network-based-program? Will you use there a delay(1) every loop too??? Please beware...

For all
In my opinion tranquils source is very easy to understand and if you take a closer look to his handle/solve blocking problem... you will see that its correct and logical... If you dont understand it, just ask tranquil... he will explain it again if you dont understand his example in any parts...

Happy coding and dont make the old mistake in your network appz guys!



PIII450, 256MB Ram, 6GB HD, RivaTNT, DirectX8.1, SB AWE64, Win2000 + all Updates...

greetz
MrVainSCL! aka Thorsten
BackupUser
PureBasic Guru
PureBasic Guru
Posts: 16777133
Joined: Tue Apr 22, 2003 7:42 pm

Post by BackupUser »

Restored from previous forum. Originally posted by MrVainSCL.

Hi @ all
Just take a look to following link, to learn more about all kind of network and how to code it:

http://www.codeproject.com/internet/

PIII450, 256MB Ram, 6GB HD, RivaTNT, DirectX8.1, SB AWE64, Win2000 + all Updates...

greetz
MrVainSCL! aka Thorsten
BackupUser
PureBasic Guru
PureBasic Guru
Posts: 16777133
Joined: Tue Apr 22, 2003 7:42 pm

Post by BackupUser »

Restored from previous forum. Originally posted by fred.
Originally posted by MrVainSCL
I saw some PB network examples but nobody who handles the stuff correctly! I cant believe that so many (prof coders too) are using delay(1) or higher in there mainloops for handling the network events... Have you ever thinked about how to code a stable and fast industry-network-based-program? Will you use there a delay(1) every loop too??? Please beware...
Hum, if you're telling this about Atomic Web Server, take a better look to the code. Delay(1) is only used when no events are in the queue, so if the source send continously data to the destination, delay(1) will be never executed.

Fred - AlphaSND
soerenkj
User
User
Posts: 95
Joined: Mon Jun 14, 2004 10:19 pm

Post by soerenkj »

Based on the principles in tranquil's code, I've created two (practically identical) functions that can be used instead of SendNetworkData and ReceiveNetworkData - the difference is that these functions give the network some extra time to catch up on sending and receiving data.
I hope I understood the principles right. Please correct me if there are some problems with the code.

Code: Select all

#Timeout = 100

Procedure.b SendData(clientID, *p, length)
  Repeat 
    sent = SendNetworkData(clientID, *p + sent_total, length - sent_total)
    If sent > 0 : sent_total + sent : EndIf     ;sent<0 means that some error code has been returned
    If sent_total <> length : Delay(1) : EndIf  ;give the network some time to empty the send buffer
    timeout + 1
  Until sent_total = length Or timeout = #Timeout ;if an error ocurred (sent<0) the loop runs until timeout
  
  If timeout = #Timeout : ProcedureReturn 0
  Else : ProcedureReturn 1 : EndIf
EndProcedure

Procedure.b ReceiveData(clientID, *p, length)
  Repeat 
    received = ReceiveNetworkData(clientID, *p + received_total, length - received_total)
    If received > 0 : received_total + received : EndIf
    If received_total <> length : Delay(1) : EndIf
    timeout + 1
  Until received_total = length Or timeout = #Timeout
  
  If timeout = #Timeout : ProcedureReturn 0
  Else : ProcedureReturn 1 : EndIf
EndProcedure
thefool
Always Here
Always Here
Posts: 5875
Joined: Sat Aug 30, 2003 5:58 pm
Location: Denmark

Post by thefool »

But isn't fred right about that atomic webserver only executes delay(1) when no events?
soerenkj
User
User
Posts: 95
Joined: Mon Jun 14, 2004 10:19 pm

Post by soerenkj »

I haven't looked at Fred's server. I was just concerned with the problem of receiving data in spite of network delays, and for me the performance isn't that important - ie. I make the delay(1) even if there are events to be processed..

By the way, another difference with the functions is that the 'length' parameter has got to be the exact length of what is to be sent/ received, for the functions to return 'success' (1)...
Post Reply