Module NetworkTCP - Send and Receive Data over 64kB

[mk-soft]

Your code ...

[08:55:39] [COMPILER] Line 12: Syntax error.

[RichAlgeni]

Your code ...
[08:55:39] [COMPILER] Line 12: Syntax error.

You're right! I apologize. My brain has been in a fog for the last year or so. This should be better:

Code: Select all

; ------------------------------------------------------------
; Program name: network_procs.pbi
; Written by:   Rich Algeni, Jr.
; Date written: 02/17/2011
;
; Purpose: to include network procedures into PB programs
; ------------------------------------------------------------

; constants needed for udp and tcp lengths

#maxUDPLength =  2048
#maxTCPLength = 65535

Declare.i CloseNetworkSocket(*socketNumber)
Declare.i SetTCPBufferSize(*socketNumber, bufferSize.i = #maxTCPLength)
Declare.i SetServerSocketTimeout(*socketNumber, bufferSize.i = #maxTCPLength)
Declare.i OpenClientSocket(*serverName, serverPortNumber.i, *errorReturn, bufferSize.i = #maxTCPLength, networkMode.i = #PB_Network_TCP, openTimeout.i = 5000)
Declare.i NetworkWrite(*socketNumber, *errorReturn, *memoryLoc, lenSendText.i, *netSource, debugThis.i)
Declare.i NetworkRead(*socketNumber, *errorReturn, *memoryLoc, memLength.i, *netSource, debugThis.i, socketTimeout.i, maxReadTrys.i)
Declare.i NetworkReadFixed(*socketNumber, *errorReturn, *memoryLoc, memLength.i, *netSource, debugThis.i, socketTimeout.i, maxReadTrys.i)
Declare.i CleanNetworkInputBuffer(*socketNumber, *errorReturn)
Declare.i NetworkInputBufferAmount(*socketNumber, *errorReturn)
Declare.i NetworkUDPRead(*socketNumber, *errorReturn, *memoryLoc, memLength.i, *netSource, debugThis.i, socketTimeout.i, maxReadTrys.i)

; constant for keepalive function values

#SIO_KEEPALIVE_VALS = 2550136836

; Argument Structure For SIO_KEEPALIVE_VALS

Structure TCP_KEEPALIVE
    onOff.l
    keepAliveTime.l
    keepAliveInterval.l
EndStructure

; **********************************************************************************
; initialization of networking variables
; **********************************************************************************

Procedure.i InitializeNetwork()

; add this program name and version to the list of includes the main program is running

    AddElement(programList())
    programList() = "network_procs.pbi"

    AddElement(progVersions())
    progVersions() = "7.0.01b"

EndProcedure

; **********************************************************************************
; this procedure attempts to close the network connection via the socket handle
; **********************************************************************************

Procedure.i CloseNetworkSocket(*socketNumber)

    Protected result.l       = 0; winsock utlizes a 32 bit integer only!
    Protected socketNumber.i = PeekI(*socketNumber)
    Protected socketHandle.i

    If socketNumber > 0
        socketHandle = ConnectionID(socketNumber)

        result = closesocket_(socketHandle)
        If  result <> 0
            result = WSAGetLastError_()
        EndIf
    EndIf

    PokeI(*socketNumber, 0)

; this should return 0 if all went ok, per Microsoft

    ProcedureReturn result

EndProcedure

; **********************************************************************************
; this procedure set the TCP receive buffer size, and keepalive
; **********************************************************************************

Procedure.i SetTCPBufferSize(*socketNumber, bufferSize.i = #maxTCPLength)

    Protected result.l;    winsock utlizes a 32 bit integer only!
    Protected keepAlive.i
    Protected socketNumber.i = PeekI(*socketNumber)
    Protected socketHandle.i

; if we already have an error encountered, or we don't have a socket, just return

    If socketNumber < 1 Or bufferSize < 1
        ProcedureReturn -999
    EndIf

    socketHandle = ConnectionID(socketNumber)

; version 2.3.01b, set the buffer size, the keep alive and no delay options

    If socketHandle > 0
        result = setsockopt_(socketHandle, #SOL_SOCKET, #SO_RCVBUF, @bufferSize, SizeOf(bufferSize))
    Else
        result = -1
    EndIf

; set the keepalive option and the no delay ack option, if result = 0 per Microsoft, all should be ok!

    keepAlive = #True
    If result = 0
        result = setsockopt_(socketHandle, #SOL_SOCKET, #SO_KEEPALIVE, @keepAlive, SizeOf(keepAlive))
        If result = 0
            result = setsockopt_(socketHandle, #IPPROTO_TCP, #TCP_NODELAY, @keepAlive, SizeOf(keepAlive))
            If result <> 0
                result = WSAGetLastError_()
            EndIf
        Else
            result = WSAGetLastError_()
            CloseNetworkSocket(*socketNumber)
        EndIf
    Else
        result = WSAGetLastError_()
    EndIf

; this should return 0 if all went ok, per Microsoft

    ProcedureReturn result

EndProcedure

; **********************************************************************************
; this procedure sets a server socket's keepalive and timeout values
; **********************************************************************************

; !!!!!!!!!!! this is for a server socket !!!!!!!!!!!!!

Procedure.i SetServerSocketTimeout(*serverNumber, bufferSize.i = #maxTCPLength)

    Protected result.l;              winsock utlizes a 32 bit integer only!
    Protected serverNumber.i = PeekI(*serverNumber)
    Protected socketHandle.i
    Protected bytesReturned.i
    Protected alive.TCP_KEEPALIVE

; if we already have an error encountered, or we don't have a socket, just return

    If serverNumber < 1 Or bufferSize < 1
        ProcedureReturn -999
    EndIf

    socketHandle = ConnectionID(serverNumber)

; see for more info on keepalive:             https://msdn.microsoft.com/en-us/library/windows/desktop/dd877220%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396

    alive\onOff                 = 1;              If the onOff entry is set to a nonzero value, TCP keep-alive is enabled and the other members in the structure are used.
    alive\keepAliveTime         = 1000 * 60;      The keepalivetime entry specifies the timeout, in milliseconds, with no activity until the first keep-alive packet is sent.
    alive\keepAliveInterval     = 1000 * 10;      The keepaliveinterval entry specifies the interval, in milliseconds, between when successive keep-alive packets are sent if no acknowledgement is received.

; turn on keep alive for the server socket

    result = SetTCPBufferSize(*serverNumber, bufferSize)
    If  result = 0
        result = WSAIoctl_(socketHandle, #SIO_KEEPALIVE_VALS, @alive, SizeOf(alive), #Null, 0, @bytesReturned, #Null, #Null)
    EndIf

; if this procedure completes successfully, the WSAIoctl function returns zero

    ProcedureReturn result

EndProcedure

; **********************************************************************************
; process to open a client socket connection, and set the receive buffer size
; **********************************************************************************

Procedure.i OpenClientSocket(*serverName, serverPortNumber.i, *errorReturn, bufferSize.i = #maxTCPLength, networkMode.i = #PB_Network_TCP, openTimeout.i = 5000)

    Protected result.l; winsock utlizes a 32 bit integer only!
    Protected serverName.s
    Protected socketNumber.i
    Protected sErrorNumber.i

; attempt to open the socket here

    serverName   = PeekS(*serverName)
    socketNumber = OpenNetworkConnection(serverName, serverPortNumber, networkMode, openTimeout)

; if the socket opened successfully, set the receive buffer size

    If socketNumber > 0
        If networkMode = #PB_Network_TCP
            result = SetTCPBufferSize(@socketNumber, bufferSize)
            If result <> 0
                socketNumber = -2
                sErrorNumber = WSAGetLastError_()
                PokeI(*errorReturn, sErrorNumber)
                CloseNetworkSocket(@socketNumber)
            EndIf
        EndIf
    Else
        socketNumber = 0
        sErrorNumber = WSAGetLastError_()
        PokeI(*errorReturn, sErrorNumber)
    EndIf

; this procedure returns the socket number if the socket is open, the socket number is also Poked into *socketNumber

    ProcedureReturn socketNumber

EndProcedure

; **********************************************************************************
; this procedure returns the amount of data in the network input buffer, but nothing is returned
; **********************************************************************************

Procedure.i NetworkInputBufferAmount(*socketNumber, *errorReturn)

    Protected socketNumber.i = PeekI(*socketNumber)
    Protected length.i       = 0
    Protected result.l       = 0; winsock utlizes a 32 bit integer only!
    Protected returnAmount.i = 0
    Protected sErrorNumber.i = PeekI(*errorReturn)
    Protected socketHandle.i
    Protected logText.s

; see if we have previously encountered an error

    If sErrorNumber <> 0
        ProcedureReturn -896
    EndIf

; if we already have an error encountered, just return

    If socketNumber < 1
        logText = "Invalid Socket number " + Str(socketNumber)
        WriteToLog(@logText)
        sErrorNumber = -899
        PokeI(*errorReturn, sErrorNumber)
        ProcedureReturn sErrorNumber
    EndIf

    socketHandle = ConnectionID(socketNumber)
    result       = ioctlsocket_(socketHandle, #FIONREAD, @length)

    If result = 0
        returnAmount = length
    Else
        sErrorNumber = WSAGetLastError_()
        PokeI(*errorReturn, sErrorNumber)
        CloseNetworkSocket(*socketNumber)
        logText      = "Socket error " + Str(sErrorNumber)
        WriteToLog(@logText)

        returnAmount = -898
    EndIf

    ProcedureReturn returnAmount

EndProcedure

; **********************************************************************************
; this procedure writes data to a socket, can be used with TCP or UDP!
; **********************************************************************************

Procedure.i NetworkWrite(*socketNumber, *errorReturn, *memoryLoc, lenSendText.i, *netSource, debugThis.i)

    Protected result.l       = 0; winsock utlizes a 32 bit integer only!
    Protected totalSent.i    = 0
    Protected socketNumber.i = PeekI(*socketNumber)
    Protected sErrorNumber.i = PeekI(*errorReturn)
    Protected logText.s

; see if we have previously encountered an error

    If sErrorNumber <> 0
        ProcedureReturn -896
    EndIf

; make sure the socket is valid!

    result = NetworkInputBufferAmount(*socketNumber, *errorReturn)

; make sure our parameters are valid!

    If result < 0
        logText = "Socket " + Str(socketNumber) + " is not a valid socket"
        WriteToLog(@logText)
        ProcedureReturn -999
    EndIf
    If *memoryLoc < 1
        logText = "Pointer *memoryLoc is invalid"
        WriteToLog(@logText)
        ProcedureReturn -998
    EndIf
    If lenSendText < 1
        logText = "lenSendText is " + Str(lenSendText) + ", must be reater than 0"
        WriteToLog(@logText)
        ProcedureReturn -997
    EndIf
    If *netSource < 1
        logText = "Pointer *netSource is invalid"
        WriteToLog(@logText)
        ProcedureReturn -996
    EndIf

; try to send the data, loop until all done, or error encountered

    While lenSendText > 0
        result = SendNetworkData(socketNumber, *memoryLoc + totalSent, lenSendText)

; if we've received an error, record it, close the socket, and set the condition to exit the loop

        If result < 0
            totalSent    = result
            sErrorNumber = WSAGetLastError_()
            PokeI(*errorReturn, sErrorNumber)
            CloseNetworkSocket(*socketNumber)
            lenSendText = 0
        Else
            totalSent   = totalSent   + result
            lenSendText = lenSendText - result
        EndIf

; send the data we have transmitted to the debugger listing, or if needed the error

        If debugThis
            DebugString(*memoryLoc + totalSent, result, *netSource, #debugOut, sErrorNumber, #Null)
        EndIf
    Wend

    ProcedureReturn totalSent

EndProcedure

; **********************************************************************************
; this procedure reads binary data from a network connection in one read attempt
; **********************************************************************************

Procedure.i NetworkRead(*socketNumber, *errorReturn, *memoryLoc, memLength.i, *netSource, debugThis.i, socketTimeout.i, maxReadTrys.i)

    Protected socketNumber.i = PeekI(*socketNumber)
    Protected attemptCount.i = 0
    Protected amountRead.i   = 0
    Protected length.l       = 0
    Protected sErrorNumber.i = PeekI(*errorReturn)
    Protected logText.s

; see if we have previously encountered an error

    If sErrorNumber <> 0
        ProcedureReturn -895
    EndIf

; make sure our parameters are valid!

    If *memoryLoc < 1
        logText = "Pointer *memoryLoc is invalid"
        WriteToLog(@logText)
        ProcedureReturn -995
    EndIf
    If memLength < 1
        logText = "memLength is " + Str(memLength) + ", must be reater than 0"
        WriteToLog(@logText)
        ProcedureReturn -994
    EndIf
    If *netSource < 1
        logText = "Pointer *netSource is invalid"
        WriteToLog(@logText)
        ProcedureReturn -993
    EndIf

; loop until we have received data, or our timeout has been exceeded

    While amountRead = 0
        length = NetworkInputBufferAmount(*socketNumber, *errorReturn)

; if we encountered a socket error, exit the loop

        If length < 0
            amountRead   = -1
            sErrorNumber = PeekI(*errorReturn)

; if nothing has been received, increment the attempt counter and sleep, else get out of the loop

        ElseIf length = 0
            attemptCount = attemptCount + 1

; if maxReadTrys < 0 (-1), then reads will attempt forever, only returning if data was received, or shutdown = true

            If maxReadTrys > 0 And attemptCount > maxReadTrys
                Break
            Else
                Delay(socketTimeout)
            EndIf

; else get the data in the network receive buffer, but only up to the amount we have buffered for

        Else
            If length > memLength
                amountRead = ReceiveNetworkData(socketNumber, *memoryLoc, memLength)
            Else
                amountRead = ReceiveNetworkData(socketNumber, *memoryLoc, length)
            EndIf
            If  amountRead < 0
                sErrorNumber = WSAGetLastError_()
                PokeI(*errorReturn, sErrorNumber)
                CloseNetworkSocket(*socketNumber)
            EndIf
        EndIf

; send the data we have read to the debugger listing, if needed

        If debugThis
            DebugString(*memoryLoc, amountRead, *netSource, #debugIn, sErrorNumber, #Null)
        EndIf
    Wend

; now return the amount we have received, if anything

    ProcedureReturn amountRead

EndProcedure

; **********************************************************************************
; this procedure reads fixed amount of binary data from a network connection
; **********************************************************************************

Procedure.i NetworkReadFixed(*socketNumber, *errorReturn, *memoryLoc, memLength.i, *netSource, debugThis.i, socketTimeout.i, maxReadTrys.i)

    Protected socketNumber.i = PeekI(*socketNumber)
    Protected attemptCount.i = 0
    Protected amountRead.i   = 0
    Protected length.i       = 0
    Protected result.i       = 0
    Protected sErrorNumber.i = PeekI(*errorReturn)
    Protected logText.s

; see if we have previously encountered an error

    If sErrorNumber <> 0
        ProcedureReturn -895
    EndIf

; make sure our parameters are valid!

    If *memoryLoc < 1
        logText = "Pointer *memoryLoc is invalid"
        WriteToLog(@logText)
        ProcedureReturn -992
    EndIf
    If memLength < 1
        logText = "memLength is " + Str(memLength) + ", must be reater than 0"
        WriteToLog(@logText)
        ProcedureReturn -991
    EndIf
    If *netSource < 1
        logText = "Pointer *netSource is invalid"
        WriteToLog(@logText)
        ProcedureReturn -990
    EndIf

; this procedure COULD read multiple packets to get the amount of data needed

; loop until we have received the predetermined amount of data, or our timeout has been exceeded

    Repeat
        length = NetworkInputBufferAmount(*socketNumber, *errorReturn)

; if we encountered a socket error, exit the loop

        If length < 0
            amountRead   = -1
            sErrorNumber = PeekI(*errorReturn)
            Break

; if nothing has been received, increment the attempt counter and sleep, else get out of the loop

        ElseIf length = 0
            attemptCount = attemptCount + 1
            If attemptCount > maxReadTrys
                Break
            Else
                Delay(socketTimeout)
            EndIf

; else get the data in the network receive buffer, but only up to the amount we have buffered for

        Else
            result = ReceiveNetworkData(socketNumber, *memoryLoc + amountRead, memLength)
            If  result     > 0
                amountRead = amountRead + result
                memLength  = memLength  - result
                If memLength = 0
                    Break
                EndIf
            Else
                sErrorNumber = WSAGetLastError_()
                PokeI(*errorReturn, sErrorNumber)
                CloseNetworkSocket(*socketNumber)
                amountRead = -1
                Break
            EndIf
        EndIf
    ForEver

; send the data we have read to the debugger listing, if needed

    If debugThis
        DebugString(*memoryLoc, amountRead, *netSource, #debugIn, sErrorNumber, #Null)
    EndIf

; now return the amount we have received, if anything

    ProcedureReturn amountRead

EndProcedure

; **********************************************************************************
; this procedure cleans out current network input buffer, nothing is returned
; **********************************************************************************

Procedure.i CleanNetworkInputBuffer(*socketNumber, *errorReturn)

    Protected socketNumber.i = PeekI(*socketNumber)
    Protected length.i       = 0
    Protected returnAmount.i = 0
    Protected sErrorNumber.i = PeekI(*errorReturn)
    Protected *memoryLoc

; see if we have previously encountered an error

    If sErrorNumber <> 0
        ProcedureReturn -894
    EndIf

    length = NetworkInputBufferAmount(*socketNumber, *errorReturn)

    If length > 0
        *memoryLoc   = AllocateMemory(length + 2)
        length       = ReceiveNetworkData(socketNumber, *memoryLoc, length)
        FreeMemory(*memoryLoc)
        returnAmount = length
    EndIf

    ProcedureReturn returnAmount

EndProcedure

; **********************************************************************************
; this procedure reads binary data from a UDP network connection in one read attempt
; **********************************************************************************

Procedure.i NetworkUDPRead(*socketNumber, *errorReturn, *memoryLoc, memLength.i, *netSource, debugThis.i, socketTimeout.i, maxReadTrys.i)

    Protected socketNumber.i = PeekI(*socketNumber)
    Protected attemptCount.i = 0
    Protected amountRead.i   = 0
    Protected length.l       = 0
    Protected sErrorNumber.i = PeekI(*errorReturn)
    Protected logText.s

; see if we have previously encountered an error

    If sErrorNumber <> 0
        ProcedureReturn -895
    EndIf

; make sure our parameters are valid!

    If *memoryLoc < 1
        logText = "Pointer *memoryLoc is invalid"
        WriteToLog(@logText)
        ProcedureReturn -995
    EndIf
    If memLength < 1
        logText = "memLength is " + Str(memLength) + ", must be reater than 0"
        WriteToLog(@logText)
        ProcedureReturn -994
    EndIf
    If *netSource < 1
        logText = "Pointer *netSource is invalid"
        WriteToLog(@logText)
        ProcedureReturn -993
    EndIf

; loop until we have received data, or our timeout has been exceeded

    While amountRead = 0

; get the data in the network receive buffer, but only up to the amount we have buffered for

        amountRead = ReceiveNetworkData(socketNumber, *memoryLoc, memLength)

        If  amountRead < 0
            sErrorNumber = WSAGetLastError_()
            PokeI(*errorReturn, sErrorNumber)
            CloseNetworkSocket(*socketNumber)

; if maxReadTrys < 0 (-1), then reads will attempt forever, only returning if data was received, or shutdown = true

        ElseIf amountRead = 0
            attemptCount = attemptCount + 1
            If maxReadTrys > 0 And attemptCount > maxReadTrys
                Break
            Else
                Delay(socketTimeout)
            EndIf
        EndIf

; send the data we have read to the debugger listing, if needed

        If debugThis
            DebugString(*memoryLoc, amountRead, *netSource, #debugIn, sErrorNumber, #Null)
        EndIf
    Wend

; now return the amount we have received, if anything

    ProcedureReturn amountRead

EndProcedure

[tatanas]

Hi mk-soft,

How does the server handle reception errors ? Is it capable of automatically requesting the client to resend the data, or does it need to be managed manually ? I am looking for a robust TCP client/server library capable of handling packet loss/errors.

Thanks.

[miso]

How does the server handle reception errors ? Is it capable of automatically requesting the client to resend the data, or does it need to be managed manually ?

TCP/IP protocoll handles these internally.

[tatanas]

Hi mk-soft,

Did you try your Module with the new embeded TLS ?
I'm strugling with my program with strange receiving problems (I didn't test with your code).