Windows sockets close returning memory errors

[RichAlgeni]

I've tested this pretty thoroughly, and have found the following: when using threads serving web pages, windows will rarely, but consistently throw a memory error when attempting to close a socket. From my research, this happens once every few thousand connections, and it only happens when running threads. It happens using the PB 'CloseNetworkConnection' function, or the Windows API 'closesocket_' function. This is the strange part, it doesn't seem to occur in a peer to peer connection, I can only get it to fail in a web server context. I've ran multiple tests on each, spanning 100's of thousands of connections. I've held back reporting my findings until I was completely sure of my work.

I have found a way to get close to work, and not error out, I'll share that below. First, there are indeed bug in Windows Sockets API. This has to do with the 'graceful' closing of a socket:

http://msdn.microsoft.com/en-us/library ... 85%29.aspx

See the bottom section. The Procedure I wrote does not return an error, but leaves the socket open for a short time. Doing a 'netstat -a', you will see it with a status of 'time_wait'. This condition will clear itself based upon setting in the Windows registry. Usually three minutes in newer versions of Windows, but you may need to change a setting in Windows 2003 Server. Just ask me if you need to know where these are.

Thanks to many who helped me stumble on to this serviceable, if not perfect solution. Here are the procedures for opening and closing a client socket. Server sockets will close the same way, you just need to add the function to get the Socket Handle.

Note that I am using 'integer' variables for consistency, as most of the code I am writing is x64.

Code: Select all

; **********************************************************************************
; this procedure attempts to open a client socket
; **********************************************************************************

Procedure.i OpenClientSocket(hostName.s, hostPort.i, *connectNumber, *socketHandle)

    Protected connectNumber.i
    Protected socketHandle.i

    connectNumber = OpenNetworkConnection(hostName, hostPort)
    If connectNumber
        socketHandle = ConnectionID(connectNumber)
        If socketHandle
            PokeI(*connectNumber, connectNumber)
            PokeI(*socketHandle,  socketHandle)
            ProcedureReturn #True; open socket was successful
        Else
            ProcedureReturn #False
        Endif
    Else
        ProcedureReturn #False
    Endif

EndProcedure


; **********************************************************************************
; close a windows socket using the API
; **********************************************************************************

Procedure.i CloseWindowsSocket(connectNumber.i, socketHandle.i)

    Protected Result.i
    Protected socket.LINGER

; set the socket options to close gracefully, in the background, structure LINGER alreadY defined in PB

    socket\l_onoff  = 1; socket should remain open...
    socket\l_linger = 1; wait # seconds, or until registry setting expires (3 minutes)

    Result = setsockopt_(socketHandle, #SOL_SOCKET, #SO_LINGER, @socket.LINGER, SizeOf(LINGER))

    If Result = 0; setting socket options was successful
        Result = closesocket_(socketHandle);
        ProcedureReturn Result; 0 means socket was successfully closed
    Else
        ProcedureReturn Result; socket error, already closed?
    EndIf

EndProcedure

Define hostName.s = "localhost"
Define hostPort.i = "8080"
Define connectNumber.i
Define socketHandle.i
Define socketOpen.i
socketOpen = OpenClientSocket(hostName, hostPort, @connectNumber, @socketHandle)

If socketOpen
    CloseWindowsSocket(connectNumber, socketHandle)
EndIf

The linger options do NOT work correctly, if you set 'linger_onoff' to 0, the socket will be crushed, and data lost. The linger_linger variable is supposed to determine how long a socket lingers, in seconds, before it closes. Windows ignores this, and reverts to the registry setting.

Supposedly there is a work around using the 'DisconnectEx(sock, NULL, 0, 0);' function as detailed at the bottom of the page, but right now it's over my head as to how to get it to work.

I just want to emphasize, I only encountered the memory error problem using sockets in threads, not as a single task, and when I wrote peer connection processes in PB, even using threads, I also did not encounter the problem.

On a better note, I do have a thread-safe web server processing running, manipulating strings, now with no memory errors. If anyone would like to see that code, I'd be happy to post it.

Rich

[cxAlex]

Thank you. I implemented this via compiler switch optionally in my ENM - Network System

Greets, Alex

[SFSxOI]

I've tested this pretty thoroughly, and have found the following: when using threads serving web pages, windows will rarely, but consistently throw a memory error when attempting to close a socket. From my research, this happens once every few thousand connections, and it only happens when running threads. It happens using the PB 'CloseNetworkConnection' function, or the Windows API 'closesocket_' function. This is the strange part, it doesn't seem to occur in a peer to peer connection, I can only get it to fail in a web server context. I've ran multiple tests on each, spanning 100's of thousands of connections. I've held back reporting my findings until I was completely sure of my work.

I have found a way to get close to work, and not error out, I'll share that below. First, there are indeed bug in Windows Sockets API. This has to do with the 'graceful' closing of a socket:

http://msdn.microsoft.com/en-us/library ... 85%29.aspx

See the bottom section. The Procedure I wrote does not return an error, but leaves the socket open for a short time. Doing a 'netstat -a', you will see it with a status of 'time_wait'. This condition will clear itself based upon setting in the Windows registry. Usually three minutes in newer versions of Windows, but you may need to change a setting in Windows 2003 Server. Just ask me if you need to know where these are.

Thanks to many who helped me stumble on to this serviceable, if not perfect solution. Here are the procedures for opening and closing a client socket. Server sockets will close the same way, you just need to add the function to get the Socket Handle.

Note that I am using 'integer' variables for consistency, as most of the code I am writing is x64.

Code: Select all

; **********************************************************************************
; this procedure attempts to open a client socket
; **********************************************************************************

Procedure.i OpenClientSocket(hostName.s, hostPort.i, *connectNumber, *socketHandle)

    Protected connectNumber.i
    Protected socketHandle.i

    connectNumber = OpenNetworkConnection(hostName, hostPort)
    If connectNumber
        socketHandle = ConnectionID(connectNumber)
        If socketHandle
            PokeI(*connectNumber, connectNumber)
            PokeI(*socketHandle,  socketHandle)
            ProcedureReturn #True; open socket was successful
        Else
            ProcedureReturn #False
        Endif
    Else
        ProcedureReturn #False
    Endif

EndProcedure


; **********************************************************************************
; close a windows socket using the API
; **********************************************************************************

Procedure.i CloseWindowsSocket(connectNumber.i, socketHandle.i)

    Protected Result.i
    Protected socket.LINGER

; set the socket options to close gracefully, in the background, structure LINGER alreadY defined in PB

    socket\l_onoff  = 1; socket should remain open...
    socket\l_linger = 1; wait # seconds, or until registry setting expires (3 minutes)

    Result = setsockopt_(socketHandle, #SOL_SOCKET, #SO_LINGER, @socket.LINGER, SizeOf(LINGER))

    If Result = 0; setting socket options was successful
        Result = closesocket_(socketHandle);
        ProcedureReturn Result; 0 means socket was successfully closed
    Else
        ProcedureReturn Result; socket error, already closed?
    EndIf

EndProcedure

Define hostName.s = "localhost"
Define hostPort.i = "8080"
Define connectNumber.i
Define socketHandle.i
Define socketOpen.i
socketOpen = OpenClientSocket(hostName, hostPort, @connectNumber, @socketHandle)

If socketOpen
    CloseWindowsSocket(connectNumber, socketHandle)
EndIf

The linger options do NOT work correctly, if you set 'linger_onoff' to 0, the socket will be crushed, and data lost. The linger_linger variable is supposed to determine how long a socket lingers, in seconds, before it closes. Windows ignores this, and reverts to the registry setting.

Supposedly there is a work around using the 'DisconnectEx(sock, NULL, 0, 0);' function as detailed at the bottom of the page, but right now it's over my head as to how to get it to work.

I just want to emphasize, I only encountered the memory error problem using sockets in threads, not as a single task, and when I wrote peer connection processes in PB, even using threads, I also did not encounter the problem.

On a better note, I do have a thread-safe web server processing running, manipulating strings, now with no memory errors. If anyone would like to see that code, I'd be happy to post it.

Rich

Try doing WSAStartup with your winsock version before the rest of your code executes and see if the errors happen:

Code: Select all

; for winsock 2.2 and up
WSAStartup_($0202,@wsaData.WSADATA)
socketOpen = OpenClientSocket(hostName, hostPort, @connectNumber, @socketHandle)