Another threaded server.

[RichAlgeni]

Because sometimes we can't see the forest for the trees! Allow me to explain: I had requested from Fred a command that would 'increment' if you will the NetworkServerEvent(), so that in a loop, the next query of NetworkServerEvent() would return the next client connected. I asked for this because I wanted to create a thread that would receive the data in the socket input buffer, without the system indicating #PB_NetworkEvent_Data for an existing client connection. In a sense, a duplicate #PB_NetworkEvent_Data notification was occurring. Note that this is not what was really happening, and it's NOT a bug. It's just that it took longer for the thread to spawn and read the data from the client, then it took for my loop to come back around and interrogate NetworkServerEvent() again. I found in my testing that only in receiving data from the client connection is what increments the NetworkServerEvent().

I swear I do my best thinking in the bathroom, as it occurred to me there that I should instead create the thread to handle the new connection based upon the server event #PB_NetworkEvent_Connect instead of #PB_NetworkEvent_Data. I'm not sure how much of a difference it makes in the overall performance of a program, but I wanted to offload the work of receiving and sending network data entirely to the threads.

Here's what I ended up with, if it can help anyone out there, great! Please note that this code uses a Windows socket API to return the amount of data in the receive buffer.

Code: Select all

EnableExplicit

Procedure ProcessRequest(clientNumber)

    Protected *memoryLocation
    Protected length.i
    Protected result.i
    Protected amountRead.i
    Protected attemptCount.i
    Protected socketHandle.i
    Protected memLength.i     = 10000
    Protected maxReadTrys.i   = 20
    Protected socketTimeout.i = 100
    Protected errNumber.i     = 0

    PrintN("Client " + Str(clientNumber) + " connected")

    socketHandle = ConnectionID(clientNumber)

    *memoryLocation = AllocateMemory(10000)

    Repeat
        attemptCount = 0
        amountRead   = 0

; loop until we have received data, or our timeout has been exceeded

        Repeat
            result = ioctlsocket_(socketHandle, #FIONREAD, @length)
            If result < 0; socket error
                amountRead = result
                errNumber  = WSAGetLastError_()
                Break
            ElseIf result > 0; socket error
                amountRead = result * -1
                errNumber  = WSAGetLastError_()
                Break
            ElseIf length > 0; we have data in the receive buffer
                If  length > memLength
                    length = memLength
                EndIf

; loop here until we've read in all the data in the buffer

                While length  > 0
                    result    = ReceiveNetworkData(clientNumber, *memoryLocation + amountRead, length)
                    If result > 0
                        amountRead = amountRead + result
                        length     = length     - result
                    Else
                        amountRead = result * -1
                        errNumber  = WSAGetLastError_()
                        length     = 0
                    EndIf
                Wend
                Break
            Else
                Delay(socketTimeout)
                attemptCount = attemptCount + 1
                If attemptCount > maxReadTrys; if still nothing received, just get out
                    Break
                EndIf
            EndIf
        ForEver

; show what we ended up with

        Select amountRead
        Case 0
            PrintN("Client " + Str(clientNumber) + " received nothing from socket")
        Case 1 To 65535
            PrintN("Client " + Str(clientNumber) + " received " + Str(amountRead) + " bytes: " + PeekS(*memoryLocation, result))
        Default
            PrintN("Client " + Str(clientNumber) + " received an error: " + Str(errNumber) + ", thread terminating")
        EndSelect

        If amountRead < 0
            Break
        EndIf
    ForEver

    FreeMemory(*memoryLocation)

EndProcedure

Define NSEvent.i
Define thisClient.i
Define keyPressed.s
Define serverNumber.i
Define portNumber.i = 8901

; initialize the network and create the server needed

InitNetwork()

serverNumber = CreateNetworkServer(#PB_Any, portNumber)

OpenConsole()

PrintN("Started Network Server on port " + Str(portNumber))
PrintN("Press Escape to exit")
PrintN("")

Repeat
    NSEvent = NetworkServerEvent() ; if we receive data, it will be indicated here

    Select NSEvent
    Case #PB_NetworkEvent_Connect   ; a socket has connected
        thisClient = EventClient()  ; get the event client identifier
        CreateThread(@ProcessRequest(), thisClient); threaded procedure to process incoming data
    Case #PB_NetworkEvent_Data      ; raw data has been received
    Case #PB_NetworkEvent_File      ; uploading a file
    Case #PB_NetworkEvent_Disconnect; a socket disconnected
    Default; no server event occurred
    EndSelect

    Delay(20); sleep so we don't hammer the processor

    keyPressed = Inkey()

Until keyPressed = #ESC$

PrintN("Escape pressed, press <enter> to terminate this process")

CloseNetworkServer(serverNumber)

Input()

End

[Warmonger]

I don't see why you couldn't do something like this to maintain cross platform capability. Because it works exactly the same for me using native PB functions, and it is much simpler.

Code: Select all

Procedure ProcessRequest(ClientID)
  
  Protected *MemoryLocation
  Protected AmountRead.i
  Protected AttemptCount.i
  Protected MemLength.i = 65535
  Protected MaxReadTrys.i = 20
  Protected SocketTimeout.i = 100
  
  *MemoryLocation = AllocateMemory(MemLength)
  If *MemoryLocation = 0
    PrintN("Could Not Allocate Memory For Client "+Str(ClientID))
  EndIf
  
  Repeat
    AmountRead = 0
    AttemptCount = 0
    Repeat
      AmountRead = ReceiveNetworkData(ClientID, *MemoryLocation, MemLength)
      If AmountRead > 0
        Break
      Else
        Delay(SocketTimeout)
        AttemptCount = AttemptCount + 1
        If AttemptCount > MaxReadTrys
          Break
        EndIf
      EndIf
    ForEver
    
    Select AmountRead
        
      Case 1 To 65535
        
        Select PeekA(*MemoryLocation + 17)
            
          Case $03
            PrintN("LOGIN!")
            
          Default
            PrintN("Received Unknown Packet From "+IPString(GetClientIP(ClientID)))
            
        EndSelect
        
      Default
        Break
        
    EndSelect
    
  ForEver
  
  FreeMemory(*MemoryLocation)

EndProcedure

[RichAlgeni]

I'm using PB 4.61, on Windows 7 64 bit SP1

When I run the above server process without ioctlsocket_(), and nothing is received on the socket, I receive a -1 as the result, which according to the documentation is a socket error. I know the socket is still connected, as my test client program can still send data, it just goes to the bit bucket. I see that your code treats negative number the same as a 0, which in theory wouldn't trap a socket error. The socket procedure would just time out, which is one way to handle it.

In my earlier code, if ReceiveNetworkData() returned with a negative number, the procedure would exit. That's why I have requested this:

http://purebasic.fr/english/viewtopic.php?f=3&t=51194

Take a look and see if you think my request would be useful. If you feel it could be, please respond to the thread so that we can convince Fred and Freak.

Rich

[Warmonger]

I'm using PB 4.61, on Windows 7 64 bit SP1

When I run the above server process without ioctlsocket_(), and nothing is received on the socket, I receive a -1 as the result, which according to the documentation is a socket error. I know the socket is still connected, as my test client program can still send data, it just goes to the bit bucket. I see that your code treats negative number the same as a 0, which in theory wouldn't trap a socket error. The socket procedure would just time out, which is one way to handle it.

In my earlier code, if ReceiveNetworkData() returned with a negative number, the procedure would exit. That's why I have requested this:

http://purebasic.fr/english/viewtopic.php?f=3&t=51194

Take a look and see if you think my request would be useful. If you feel it could be, please respond to the thread so that we can convince Fred and Freak.

Rich

I see, after connecting ReceiveNetworkData() constantly returns -1 when there is no received data. So you couldn't possibly know when a client is idle, or disconnected. Maybe Fred should change it to return 0 if there is no received data, and -1 when there is a socket error. It would also be handy if there was some sort of NetworkError() function to capture the network error code.

[RichAlgeni]

I see, after connecting ReceiveNetworkData() constantly returns -1 when there is no received data. So you couldn't possibly know when a client is idle, or disconnected. Maybe Fred should change it to return 0 if there is no received data, and -1 when there is a socket error. It would also be handy if there was some sort of NetworkError() function to capture the network error code.

Exactly right! That's why I use (since I only write in Windows now) result = ioctlsocket_(socketHandle, #FIONREAD, @length), if the result <> 0, there is a socket error, or the socket closed, and I can use errNumber = WSAGetLastError_() to get the Winsock error. If result = 0, I can look at the length variable, to see if it's greater than 0, if it is, there is data to be read.

Returning the '-1' instead of '0' really through me off for awhile, I'm wondering if that should be listed as a bug? What do you think?

[Warmonger]

I see, after connecting ReceiveNetworkData() constantly returns -1 when there is no received data. So you couldn't possibly know when a client is idle, or disconnected. Maybe Fred should change it to return 0 if there is no received data, and -1 when there is a socket error. It would also be handy if there was some sort of NetworkError() function to capture the network error code.

Exactly right! That's why I use (since I only write in Windows now) result = ioctlsocket_(socketHandle, #FIONREAD, @length), if the result <> 0, there is a socket error, or the socket closed, and I can use errNumber = WSAGetLastError_() to get the Winsock error. If result = 0, I can look at the length variable, to see if it's greater than 0, if it is, there is data to be read.

Returning the '-1' instead of '0' really through me off for awhile, I'm wondering if that should be listed as a bug? What do you think?

I don't think it should be listed under bugs, but more over something that should defiantly be changed. Tho after toying with it a bit, one way to check if a client is disconnected is using #PB_NetworkEvent_Disconnect to kill the thread the client uses. Of course you would have to store a list of thread ID's and client ID's in order for it to know which thread to kill. Tho ReceiveNetworkData() still shouldn't return error just because it hasn't received any data. It would be ten times easier if.

Code: Select all

    Select ReceiveNetworkData()
        
      Case -1
        ;Network Error
        
      Case 0
        ;Received Nothing
        
      Case 1 To 65535
        ;Received Data
        
    EndSelect

Edit: I guess a temporary work around would be to tally the -1's being returned by the ReceiveNetworkData() function sorta like the timeout works. So if you get 20 -1's in a row then mark the client as disconnected. Tho you will still never know if its throwing -1 because of a error.

[Opcode]

Some code cleanup and updated for PB 5.30. Haven't tested since I am on a Linux platform.

Code: Select all

; --------------------------------------------------
;   Threaded Server
; --------------------------------------------------

EnableExplicit

; --------------------------------------------------
;   Client Thread
; --------------------------------------------------
Procedure ProcessRequest(ClientID)
  
  Protected *MemoryID
  Protected Length.l
  Protected Result.l
  Protected AmountRead.l
  Protected AttemptCount.l
  Protected SocketHandle.l
  Protected MemLength.l     = 10000
  Protected MaxReadTrys.l   = 20
  Protected SocketTimeout.l = 100
  Protected ErrorNumber.l   = 0
  
  PrintN("[Client " + Str(ClientID) + "] Connected")
  
  SocketHandle = ConnectionID(ClientID)
  
  *MemoryID = AllocateMemory(MemLength)
  
  Repeat
    
    AttemptCount = 0
    AmountRead   = 0
    
    ; --------------------------------------------------
    ;   Loop Until Received Data Or Timeout
    ; --------------------------------------------------
    Repeat
      Result = ioctlsocket_(SocketHandle, #FIONREAD, @Length)
      If Result <> 0
        AmountRead  = -1
        ErrorNumber = WSAGetLastError_()
        Break
      ElseIf Length > 0
        If Length > MemLength
          Length = MemLength
        EndIf
        
        ; --------------------------------------------------
        ;   Loop Until We Have All Data
        ; --------------------------------------------------
        While Length > 0
          Result = ReceiveNetworkData(ClientID, *MemoryID + AmountRead, Length)
          If Result > 0
            AmountRead = AmountRead + Result
            Length     = Length     - Result
          Else
            AmountRead  = -1
            ErrorNumber = WSAGetLastError_()
            Break
          EndIf
        Wend
        Break
      Else
        Delay(SocketTimeout)
        AttemptCount = AttemptCount + 1
        If AttemptCount > MaxReadTrys
          Break
        EndIf
      EndIf
    ForEver
    
    ; --------------------------------------------------
    ;   Handle Received Data
    ; --------------------------------------------------
    Select AmountRead
      Case 0
        ;PrintN("[Client " + Str(ClientID) + " Received Nothing From Socket")
      Case 1 To 65535
        PrintN("[Client " + Str(ClientID) + "] Received " + Str(AmountRead) + " Bytes")
        ;ShowMemoryViewer(*MemoryID, Result)
      Default
        PrintN("[Client " + Str(ClientID) + "] Socket Error Code " + Str(ErrorNumber) + " (Thread Terminating)")
    EndSelect
    
    If AmountRead < 0
      Break
    EndIf
  ForEver
  
  FreeMemory(*MemoryID)
  
EndProcedure

; --------------------------------------------------
;   Main Variables
; --------------------------------------------------
Define ServerEvent.l
Define ServerIP.s = "127.0.0.1"
Define ServerPort.l = 4001

; --------------------------------------------------
;   Initialize Network And Create Server
; --------------------------------------------------
InitNetwork()

If CreateNetworkServer(0, ServerPort, #PB_Network_TCP, ServerIP)
  
  OpenConsole()
  
  PrintN("Server Listening On "+ServerIP+":"+Str(ServerPort))
  
  Repeat
    
    ServerEvent = NetworkServerEvent(0)
    
    Select ServerEvent
      Case #PB_NetworkEvent_Connect
        CreateThread(@ProcessRequest(), EventClient())
      Case #PB_NetworkEvent_Data
      Case #PB_NetworkEvent_Disconnect
      Default
    EndSelect
    
    Delay(10) ; Idle
    
    If Inkey() = #ESC$
      CloseNetworkServer(0)
      End
    EndIf
    
  ForEver
  
EndIf

[RichAlgeni]

Are you saying my code needed cleanup? Did you just call me fat?

[tatanas]

Sorry to dig up this topic, but I have a question concerning the "receiving data" thread.
How many clients this architecture could handle ?
Can I use this code for 500 to 1000 clients ?

Thanks.

[mk-soft]

You can manage clients over a Map.

client = EventClient()
keyClient.s = Str(client)

Show example module NetworkTCP

[tatanas]

Thanks for your very nice module.

If I understand your server example correctly, you create only one thread to handle the network events.
So one thread could handle that many clients ?

[mk-soft]

One thread for the server that manages and receives data from all clients.
Then you can create a thread for each client to continue working, which will process the data and send the data from the client thread and then stop.

[tatanas]

Ok thank you for the advices.