Wrappers to do full sendNetworkData and ReceiveNetworkData

[HeX0R]

You can't compare UDP with TCP/IP.
For UDP the maximum packet you can send is of size MTU, because there is no functionality to recover packets on the receivers side.
You just throw out small packets and hope they arrive, you also don't know at which order they arrive.
It is usually used for protocols, where that doesn't really matter, e.g. voice transmissions or players positions in multiplayer games.
UDP is faster, because it has less overhead, but with above disadvantages then.

For TCP/IP MTU holds the size the bigger data you are sending will be devided into (internally at the lower TCP levels and usually not of much interest for the app).

If Quin is using TCP/IP I hardly believe it has anything to do with the MTU setting.
Idles procedure would just go on (assumption after a quick look, I'm not using it) calling SendNetworkData() and add new packets, I see no reason for any such limitation.

[Quin]

For TCP/IP MTU holds the size the bigger data you are sending will be devided into (internally at the lower TCP levels and usually not of much interest for the app).

If Quin is using TCP/IP I hardly believe it has anything to do with the MTU setting.
Idles procedure would just go on (assumption after a quick look, I'm not using it) calling SendNetworkData() and add new packets, I see no reason for any such limitation.

This is all TCP/IP, sending UTF-8 encoded JSON. I'm skeptical if it's the MTU myself, because my MTU packet size is 1500 bytes, and I can send around double that.

[mk-soft]

You can also send over 64kB with TCP/IP. This is then split into individual packets and passed to your Receive in the correct order. However, you must also transfer the entire length so that you know when the data is complete.

[miso]

As I thougth, TCP handles this on its layer. (as for games, I have experiences with only UDP.)

There are other usecases, you can keep and mark important packets, and resend if not acknowledged.

Also for reliable filesending there are usable protocols. (send how many parts there are in the file, client asks for the first x part, send those to it.
Parts are numbered in the header, can be written to a file to its position. Client remove from a list those parts that arrived, and asks for the next X that is in it's list until finished, aka list is empty.) Works pretty well.

[idle]

The functions work fine on windows, they are both used in atomic web servers reverse proxy which is used to host multiple servers but I haven't got around to testing them on linux.

If the send or receive functions returns 0 its an error and if you want to retrieve the error you need to pass in the address of an integer which will report either
#PB_Network_Error_Fatal = -1 ; something went wrong which isn't recoverable
#PB_Network_Error_Timeout = -2 ; the function timed out trying
#PB_Network_Error_Dropped = -3 ; the client closed
#PB_Network_Error_Memory = -4 ; it failed to reallocate the buffer

if your doing this in threads you need to ensure that your not trying to send or receive to the same connection simultaneously, your data needs to be serialized into the network functions in that case.

[Quin]

The functions work fine on windows, they are both used in atomic web servers reverse proxy which is used to host multiple servers but I haven't got around to testing them on linux.

If the send or receive functions returns 0 its an error and if you want to retrieve the error you need to pass in the address of an integer which will report either
#PB_Network_Error_Fatal = -1 ; something went wrong which isn't recoverable
#PB_Network_Error_Timeout = -2 ; the function timed out trying
#PB_Network_Error_Dropped = -3 ; the client closed
#PB_Network_Error_Memory = -4 ; it failed to reallocate the buffer

if your doing this in threads you need to ensure that your not trying to send or receive to the same connection simultaneously, your data needs to be serialized into the network functions in that case.

Thanks for all that info, Idle!
I am indeed using a thread, at least on the client side. Would you mind expanding a bit on how to "serialize my data into the network functions"?
Thanks!

[idle]

It might be easier if you posted your code so we can see what your doing

[Quin]

Okay.
This is a large application with tons of components, so I obviously cannot post all of it. However, here's my macro for sending packets to the server:

Code: Select all

Macro SendPacket(_Buffer_, _Structure_)
	Protected JSONID.i, *Res
	JSONID = CreateJSON(#PB_Any, #PB_JSON_NoCase)
	If JSONID
		InsertJSONStructure(JSONValue(JSONID), _Buffer_, _Structure_)
		*Res = UTF8(ComposeJSON(JSONID))
		FreeJSON(JSONID)
	EndIf
	SendNetworkDataEX(App\ClientID, *Res, MemorySize(*Res))
	FreeMemory(*Res)
EndMacro

and I then have functions to send my packets like so:

Code: Select all

Procedure SendChat(Message$)
	Protected Chat.ChatPacket
	With Chat
		\Type = #PacketType_Chat
		\Sender$ = Prefs\Name$
		\Message$ = Message$
		\Recipient$ = GetGadgetText(#Gadget_Main_SendToCombo)
	EndWith
	SendPacket(@Chat, ChatPacket)
EndProcedure

These functions can be called from in my net loop, from the main UI thread, etc.

[idle]

I wouldn't recommend using macros until you're sure the functions they perform are bullet proof since you cant debug them properly.

you need something like this

Code: Select all

Structure Client 
  IP.s
  ClientID.i
  ThreadID.i
  Semephore.i ;might want it  
  Mutex.i
EndStructure 

Structure App 
  ClientID.i
  Map clients.Client() 
EndStructure   

Global app.app 

;network connect 
key.s = Str(clientID)
If Not FindMapElement(app\Clients(),key) 
  AddMapElement(app\clients(),key)
  app\clients()\mutex = CreateMutex()
  app\clients()\ClientID = clientid ;network client ID  
  app\clients()\tid = CreateThread(@ProcessClients())   
EndIf   

Procedure SendPacket(_Buffer_, _Structure_)
	Protected JSONID.i, *Res
	JSONID = CreateJSON(#PB_Any, #PB_JSON_NoCase)
	If JSONID
		InsertJSONStructure(JSONValue(JSONID), _Buffer_, _Structure_)
		*Res = UTF8(ComposeJSON(JSONID))
		FreeJSON(JSONID)
	EndIf
	
	result = SendNetworkDataEX(App\Clients()\clientID,*Res,MemorySize(*Res),5000,mutex,@error)
	FreeMemory(*Res)
	
EndProcedure 

Procedure SendChat(Message$)
  Protected Chat.ChatPacket,result,error,mutex,key.s
   
  With Chat
    \Type = #PacketType_Chat
    \Sender$ = Prefs\Name$
    \Message$ = Message$
    \Recipient$ = GetGadgetText(#Gadget_Main_SendToCombo)
  EndWith
  
  key = Str(app\Clientid)
  If FindMapElement(app\clients(),key)
    
    mutex = app\Clients()\mutex 
    
    SendPacket(@Chat, ChatPacket)
    If result = 0 ;there was an unrecoverable error 
      Select error 
        Case #PB_Network_Error_Fatal
          Debug "unrecoverable close connection"  
        Case #PB_Network_Error_timeout 
          Debug "timed out close connection" 
        Case #PB_Network_Error_Dropped 
          Debug "client closeed connection"
      EndSelect   
      FreeMutex(app\clients()\Mutex) 
      CloseNetworkConnection(app\clients()\ClientID)  
      DeleteMapElement(app\clients(),key) 
      
    EndIf   
  EndIf   
  
EndProcedure

[Quin]

Hi Idle,
Sorry, that code I sent was for my client, not server.
The reason I used a macro is because it's not possible to pass the type of a structure as a parameter to a procedure, never mind that some built-in PB functions can do it...
For the server, I handle all network events on the main loop. I assume this is unadvisable, and I should use something like what you posted. What about for the client, though?
If it matters, here's the macro for the server (slightly different to account for broadcast packets):

Code: Select all

Macro SendPacket(_Buffer_, _Structure_)
	Protected JSONID.i, *Res
	JSONID = CreateJSON(#PB_Any, #PB_JSON_NoCase)
	If JSONID
		InsertJSONStructure(JSONValue(JSONID), _Buffer_, _Structure_)
		*Res = UTF8(ComposeJSON(JSONID))
		FreeJSON(JSONID)
	EndIf
	If ID = -1
		ForEach Srv\Users()
			SendNetworkDataEX(Srv\Users()\ID, *Res, MemorySize(*Res))
		Next
	Else
		SendNetworkDataEX(ID, *Res, MemorySize(*Res))
	EndIf
	FreeMemory(*Res)
EndMacro

[mk-soft]

I once customised the constants for macOS from the error.h, but haven't tested them yet.

Code: Select all

  CompilerSelect #PB_Compiler_OS
    CompilerCase #PB_OS_Windows 
      #WSAEINTR = 10004
      #WSAEMFILE = 10024
      #WSAEWOULDBLOCK = 10035
      #WSAEINPROGRESS = 10036
      #WSAEALREADY = 10037
    CompilerCase #PB_OS_Linux
      #WSAEINTR = 4         ; EINTR 
      #WSAEMFILE = 17       ; ENOFILE 
      #WSAEWOULDBLOCK = 11  ; Eagain  
      #WSAEINPROGRESS = 115 ; EINPROGRESS
      #WSAEALREADY = 114    ; EALREADY 
    CompilerCase #PB_OS_MacOS
      #WSAEINTR = 4         ; EINTR 
      #WSAEMFILE = 24       ; EMFILE 
      #WSAEWOULDBLOCK = 35  ; EWOULDBLOCK = EAGAIN  
      #WSAEINPROGRESS = 36  ; EINPROGRESS
      #WSAEALREADY = 37     ; EALREADY 
  CompilerEndSelect

[idle]

thanks for that, updated 1st post