Question regarding UDP network connections

[Thumper]

Infratec provided a cross-platform (Linux, MacOS and Windows) socket-level example UDP program that proved very useful in resolving this problem. I suggest that those finding this post be certain to review it at http://www.purebasic.fr/english/viewtop ... 18#p451018

----- Begin original post -----

Greetings!

Well, I've been back in the PureBasic world for a while now and things are progressing nicely. So far everything I have been testing has been working in Windows and on the Mac.

One little difficulty that I have encountered is with regards to UDP communications. Here's the scoop:

The background is I have to send from the PC an "I'm ready" broadcast packet from port 50500 to port 50501 and almost immediately I will receive an "We're Here" UDP response from all listening devices back on port 50500. These devices also periodically send out the same packet every couple of minutes.

I have created the network server to receive the packets and that seems to work perfectly. I am able to get the periodic device messages and process them.

So far, so good, but here is the catch: When I send the "I'm ready" message, I'm not getting the responses that I just received.

What it seems to me is that when send a message on a particular UDP port, I cannot receive any messages on that port for a brief period of time. (I am guessing this quiet period to be able two seconds but I could be very wrong on that.) Now after that period of time had passed, I have no issue in receiving the page that are sent out periodically.

Thanks to Fred adding the ability to specify the originating port, the program appears to be sending packet correctly according to WireShark. Also according to WireShark, the devices are sending their responses. It's just that I'm not getting a packet notification.

Referring to the attached screen shot from WireShark, here is a chronology as to what is happening:

• Record 19 - I sent the "I'm ready" packet from .120 to .255
• Record 20 - 1/100th of a second later, WireShark sees the "We're here" packet from .147 but my program does not
• Record 112 - 30 seconds later, .147 says "We're here" again and this time my program does receive it
• Record 115 - My program interprets the packet, decides to establish communications and sends this packet direct from .120 to .147 with instructions to establish the TCP connection
• Record 118 - TCP communications commence between the PC at .120 and the device at .147; the program is working as it should
• Record 122 - The device at .147 sends its first data packet to the PC at .120; the program is also working as it should

Screen shot link: http://www.bullthumper.com/images/wireshark-udp.png

One thing I have tried was to close the server established by CreateNetworkServer(), send the packet and then re-open the server, just in case there was some limitation, but interesting the characteristic seems to be same, or at least similar.

I'm not sure where to go in diagnosing this and am wondering if anyone has any ideas or suggestions. Also, if I need to provide more information, just let me know and I will try to oblige.

Thanks in advance for your help,

Dan

[deesko]

Hello.

Since you don't specify this, I assume you're not using threads in the server/client processing.

Probably that's what causes you not having "time" to catch those events.

[Thumper]

Since you don't specify this, I assume you're not using threads in the server/client processing.

Probably that's what causes you not having "time" to catch those events.

You are correct. I am not using threads however it is interesting you mentioning that. I did try to send the UDP from within a short-lived thread as an experiment but the result was the same.

My theory has been to avoid threads and let the event engine manage the GUI and communications "traffic."

If I recall recall, in my thread-based sender, I created a static variable to hold the handle and if I didn't close the connection, I wouldn't get anything from that point on. That's how I discovered that I have open and close the sending connection. It's like I cannot have both at the same time.

Here is the procedure that opens the UDP connection:

Code: Select all

     Procedure lan_udp_open ()

          if udp_receiver_handle     ; << This is defined as a global
               udp_receiver_handle = 0
          endif

          udp_receiver_handle = CreateNetworkServer (#PB_Any, 50500, #PB_Network_UDP)

     EndProcedure

Here is the essence my sender routine that can be called directly or via a thread:

Code: Select all

     procedure lan_udp_send_st_online ( *value )
          define i, k, result

          Protected *p = AllocateMemory(72)

          static ConnectionHandle.i

          define IPAddress.s = "192.168.13.255"   ; (hard coded for my testing purposes)
          define myIPAddress.s = "192.168.13.120"   ; (hard coded for my testing purposes)

          if value = -1
               if connectionHandle
                    CloseNetworkConnection ( connectionHandle )
                    connectionHandle = 0
                    ProcedureReturn 
               endif
          endif

          ; skipping the boring part of putting the message into *p

          if connectionhandle = 0
               ConnectionHandle = OpenNetworkConnection(IPAddress, 50501, #PB_Network_UDP | #PB_Network_IPv4, 5000, myIPAddress, 50500)
          endif

          if ConnectionHandle
               Result = SendNetworkData (ConnectionHandle, *p, 24)

               CloseNetworkConnection (ConnectionHandle)  ; <-- see note
               connectionHandle = 0 
          endif

     endprocedure

Note: I have tried this many ways. Originally, I had it leave the connection open but when I did that, I did not receive any events on the udp_receiver_handle server.

It boils down to close and have a quiet period or leave it open and get nothing.

Also, in case you are interested, this is the essence of my event loop:

Code: Select all

     While ProgramRunning    ; < this is a global

          ThisEvent = WindowEvent ()
          if ThisEvent
               Select ThisEvent
                    ; do window stuff here
               EndSelect
               Continue
          EndIf

          NetEvent = NetworkServerEvent()
          If NetEvent
               Select NetEvent
                    Case #PB_NetworkEvent_Connect
                         if EventServer () = tcp_receiver_handle
                              lan_tcp_connect ()
                              Continue
                         endif


                    Case #PB_NetworkEvent_Data

                         if EventServer() = udp_receiver_handle ; receive the udp packet
                              lan_udp_receive ()
                              Continue
                         endif

                         if EventServer() = tcp_receiver_handle ;
                              lan_tcp_receive ()
                              Continue
                         endif


                    Case #PB_NetworkEvent_Disconnect
                         Continue

                    Default
                         Continue

               EndSelect
               Continue
          EndIf

          Delay (20)
     Wend

[infratec]

Hi,

if you running the server part on port 50500, you can not open the same port continiously for sending.

But why are you sending from port 50500 ???
Normally only the target port is from interest.
Remove the 50500 and you can hold the connection 'open'.

In your code above your target is a broadcast address.
I don't know if this is possible with PB.

Bernd

[infratec]

Since you didn't provide working code,
I had to do it by my self (not really my work)

Part 1:

Code: Select all

InitNetwork()

Buffer$ = Space(1024)

ServerID = CreateNetworkServer(#PB_Any, 50501, #PB_Network_UDP, "127.0.0.1")
If ServerID
  
  Timeout = 1
  Repeat
    
    ServerEvent = NetworkServerEvent(ServerID)
    If ServerEvent <> #PB_NetworkEvent_None
      Select ServerEvent
        Case #PB_NetworkEvent_Data
          Client = EventClient()
          
          Length = ReceiveNetworkData(Client, @Buffer$, 1024)
          Debug Length
          If Length
            SendNetworkData(Client, @Buffer$, Length)
          EndIf
      EndSelect
    Else
      Delay(10)
    EndIf
    
    Timeout - 1
    If Timeout = 0
      Timeout = 300
      Conn = OpenNetworkConnection("127.0.0.1", 50500, #PB_Network_UDP)
      If Conn
        SendNetworkData(Conn, @"Ping", 5)
        CloseNetworkConnection(Conn)
      EndIf
    EndIf
    
  ForEver
  
EndIf

Part 2:

Code: Select all

Global udp_receiver_handle.i


Procedure lan_tcp_connect()
EndProcedure


Procedure lan_tcp_receive()
EndProcedure


Procedure lan_udp_receive()
  
  Buffer$ = Space(1024)
  
  Length = ReceiveNetworkData(EventClient(), @Buffer$, 1024)
  If Length
    Debug Buffer$
  EndIf
  
EndProcedure


Procedure lan_udp_open ()
  
  If udp_receiver_handle     ; << This is defined as a global
    udp_receiver_handle = 0
  EndIf
  
  udp_receiver_handle = CreateNetworkServer(#PB_Any, 50500, #PB_Network_UDP)
  
EndProcedure



Procedure lan_udp_send_st_online(*value)
  
  Define i, k, result
  
  Debug "UDP Send"
  
  Protected *p = AllocateMemory(72)

  Static ConnectionHandle.i

  Define IPAddress.s = "127.0.0.1"   ; (hard coded for my testing purposes)
  Define myIPAddress.s = "127.0.0.1"   ; (hard coded for my testing purposes)
  
  If *value = -1
    If connectionHandle
      CloseNetworkConnection(connectionHandle)
      connectionHandle = 0
      ProcedureReturn
    EndIf
  EndIf

  ; skipping the boring part of putting the message into *p
  PokeS(*p, "Hello world!")
  
  If connectionhandle = 0
    ConnectionHandle = OpenNetworkConnection(IPAddress, 50501, #PB_Network_UDP | #PB_Network_IPv4, 0, myIPAddress, 50500)
  EndIf
  
  If ConnectionHandle
    Result = SendNetworkData(ConnectionHandle, *p, 24)
    
    CloseNetworkConnection(ConnectionHandle)  ; <-- see note
    connectionHandle = 0
  EndIf

EndProcedure




InitNetwork()

lan_udp_open()

OpenWindow(0, 0, 0, 300, 200, "Test 1", #PB_Window_SystemMenu)
ButtonGadget(0, 10, 10, 50, 20, "Send")


ProgramRunning = #True

While ProgramRunning    ; < this is a global
  
  DelayFlag = #True
  
  ThisEvent = WindowEvent()
  If ThisEvent
    Select ThisEvent
      Case #PB_Event_Gadget
        Select EventGadget()
          Case 0
            lan_udp_send_st_online(#Null)
        EndSelect
      Case #PB_Event_CloseWindow
        ProgramRunning = #False
    EndSelect
    DelayFlag = #False
  EndIf
  
  Repeat
    NetServerEvent = NetworkServerEvent()
    If NetServerEvent <> #PB_NetworkEvent_None
      Select NetServerEvent
        Case #PB_NetworkEvent_Connect
          If EventServer() = tcp_receiver_handle
            lan_tcp_connect()
          EndIf
        Case #PB_NetworkEvent_Data
          Select EventServer()
            Case udp_receiver_handle ; receive the udp packet
              lan_udp_receive()
            Case tcp_receiver_handle ;
              lan_tcp_receive()
          EndSelect
      EndSelect
      DelayFlag = #False
    EndIf
  Until NetServerEvent = #PB_NetworkEvent_None
  
  If DelayFlag
    Delay(10)
  EndIf
  
Wend

[Thumper]

Hi,

if you running the server part on port 50500, you can not open the same port continiously for sending.

But why are you sending from port 50500 ???
Normally only the target port is from interest.
Remove the 50500 and you can hold the connection 'open'.

In your code above your target is a broadcast address.
I don't know if this is possible with PB.

Bernd

Greetings Bernd!

This is how the hardware is setup. The device responds on whatever port the UDP packet was sent from to the address it was sent from. I do not have any choice in the matter.

In fact, Fred was great in helping me with an early problem in that UDP packets were sent from (seemingly) random ports to which the device responded on an unknown port. The update provided the ability to specify the source port and now that is working great as WireShark has confirmed.

And yes, the broadcast seems to be working just fine. As you can see from the WireShark screen shot, the device responds almost instantly to the packet being sent out. It's just that I'm not able to see that response.

I'm going to look at what you did but I don't see that you did anything different or if it will even work with the equipment in question.

For example, using 127.0.0.1 with the source address, no packet is received by the device nor is WireShark indicating anything. What is the logic for sending or receiving the packet to/from the localhost?

I did try grouping NetworkServerEvent()s in a Repeat loop as you did but that didn't change anything. Is that necessary? I'm guessing the benefit is that it would process the LAN events in a group but then again, it could process LAN and GUI events out of sequence. Thoughts?

I do appreciate your effort in making a "running" program. The reason I did not make a "running" source is that without this equipment, nobody would be able to test or validate anything. I omitted stuff like my debug logging, multiple connection handling and the couple pages of source just to put the message into *p. I was just trying to present the essence of what I was doing. The full source with the UI is so far a little over 27,000 lines that nobody would want to pile through.

I have considered writing a simulator that responds on a port to be run on a second computer on a LAN. I think that would be productive but it would take me a bit to write something like that. Remember, I do not consider myself fluent yet with PureBasic.

In the OpenNetworkConnection(), you have the TimeOut set to 0. Does PureBasic take that as to use the default? And for what it is worth, I have tried a variety of values ranging from the 1 to 5000 to no effect.

After reviewing what your example and comparing with mine, aside from differences in coding style, I don't see that I am doing anything different.

I'll toss in this little piece of information. If I set my timer frequent enough, say 1 second increments to send the broadcast packet, I will never see a response from the device, even the ones it sends on its on volition. If more I increase the value, and not make it divisor of the automatic sends from the device, eventually it will hit outside the quiet time window.

Thank you again for your response, Bernd!

[infratec]

Hi,

for UDP you have no timeout at OpenNetworkConnection(), since this procedure is only a dummy.

I used 127.0.0.1 because I tested it on one PC

My main loop is a bit different to yours.
Simply give it a try with your real device hardware. It can only fail.

Does my 'simulator' something different as your device?
It sends peridic stuff and answers directly if it receives something.

Bernd

[Thumper]

I used 127.0.0.1 because I tested it on one PC

My main loop is a bit different to yours.
Simply give it a try with your real device hardware. It can only fail.

Does my 'simulator' something different as your device?
It sends peridic stuff and answers directly if it receives something.

Bernd

What kind of setup are you using? I'm still getting the same results here with your program.

(I'm using a combination of XP and two 8.1 machines and haven't tried to introduce the Mac in to the equation.)

I've even added to your version the strings and calculations to make the devices talk back correctly and I'm seeing the exact same traffic via WireShark as I did with my version.

(Pulling hair)

[infratec]

In my office I use Win 7 64bit with PB 5.30 x86.
My 2 programs work like expected.

Here at home I use Win XP 32bit with PB 5.30 and when I send the packet via the button
it is not received.

I'll have to look deeper now.

[Thumper]

In my office I use Win 7 64bit with PB 5.30 x86.
My 2 programs work like expected.

Here at home I use Win XP 32bit with PB 5.30 and when I send the packet via the button
it is not received.

I'll have to look deeper now.

Hmmm, I think that I am using 5.21. And they are the 64-bit versions. I wonder if these are making a difference?

I'm on my BlackBerry right now and will check that later this evening. Why don't you hold off on deeper investigation until I can get back to my computer and download 5.30.

[infratec]

Hi,

Code: Select all

InitNetwork()

udp_receiver_handle = CreateNetworkServer(#PB_Any, 50500, #PB_Network_UDP)

Debug OpenNetworkConnection("127.0.0.1", 50501, #PB_Network_UDP, 0, "127.0.0.1", 50500)

fails on Win XP.
Works on Win 7.

[Thumper]

Quick update.

I simply recompiled my program using 5.30 and now the UDP packets are going out correctly on my Windows 8.1 machine.

For some reason, the exact same EXE is failing on the UDP creation on an XP netbook, similar to what you said.

Continuing to investigate...

[infratec]

Hi,

an other approach:

Code: Select all

EnableExplicit


Procedure lan_tcp_connect()
EndProcedure


Procedure lan_tcp_receive()
EndProcedure


Procedure lan_udp_receive(ConID.i)
  
  Protected.i Length
  Protected Buffer$
  
  
  Buffer$ = Space(1024)
 
  Length = ReceiveNetworkData(ConID, @Buffer$, 1024)
  If Length
    Debug Buffer$
  EndIf
 
EndProcedure


Procedure.i lan_udp_open ()
  
  ProcedureReturn OpenNetworkConnection("127.0.0.1", 50501, #PB_Network_UDP, 0, "127.0.0.1", 50500)
 
EndProcedure



Procedure lan_udp_send_st_online(ConnectionHandle.i)
 
  Protected.i Result
  Protected *p
  
  
  Debug "UDP Send"
 
  If ConnectionHandle
    *p = AllocateMemory(72)
    If *p
      PokeS(*p, "Hello world!")
      Result = SendNetworkData(ConnectionHandle, *p, 24)
      FreeMemory(*p)
    EndIf
  EndIf

EndProcedure



Define.i UDPCon, ProgramRunning, DelayFlag, WinEvent, UDPEvent, TCPEvent, TCPServer


InitNetwork()

UDPCon = lan_udp_open()

OpenWindow(0, 0, 0, 300, 200, "Test 1", #PB_Window_SystemMenu)
ButtonGadget(0, 10, 10, 50, 20, "Send")


ProgramRunning = #True

While ProgramRunning    ; < this is a global
 
  DelayFlag = #True
 
  WinEvent = WindowEvent()
  If WinEvent
    Select WinEvent
      Case #PB_Event_Gadget
        Select EventGadget()
          Case 0
            lan_udp_send_st_online(UDPCon)
        EndSelect
      Case #PB_Event_CloseWindow
        ProgramRunning = #False
    EndSelect
    DelayFlag = #False
  EndIf
 
  UDPEvent = NetworkClientEvent(UDPCon)
  If UDPEvent = #PB_NetworkEvent_Data
    lan_udp_receive(UDPCon)
    DelayFlag = #False
  EndIf
  
  If TCPServer <> 0
    TCPEvent = NetworkServerEvent(TCPServer)
    If TCPEvent <> #PB_NetworkEvent_None
      
      DelayFlag = #False
    EndIf
  EndIf
  
  If DelayFlag
    Delay(10)
  EndIf
 
Wend

Since the PC is not the 'server', I use only the client stuff.
So you don't need to close the 'connection' and it should also work on XP.

Bernd

[Thumper]

Well, I reworked my program to follow your latest example but in this case, I cannot see any packets from the devices.

I then took your program and replaced the IP addresses with my own computer and one device, replaced the strings and calculations and still couldn't get messages from the device.

I appreciate this attempt but I'm going to go back a couple of steps where Windows 8 works great and XP doesn't.

Oh, and for what it is worth, I'm having the same problem on the Mac. In fact, if the UDP receiver is open, I cannot send anything and am completely dead in the water. The Mac side of PureBasic is where I am going.

Not sure what to do. Suggestions?

[Thumper]

Something else to throw into the mix.

If I follow these three steps:

1. receiverHandle = CreateNetworkServer
2. senderHandle = OpenNetworkConnection
3. SendNetworkData on senderHandle

Windows 8.1 seems to work okay with broadcast or specific device IP addresses

Windows XP seems to vary but I'm not concerned as the primary focus of this program is for the Mac platform and not Windows.

The Mac version
- works with a specific IP address (i.e. 192.168.13.133)
- but fails at step 2 if the a broadcast address (i.e. 192.168.13.255)


If it works on the Mac without the broadcast, I can kinda-sorta deal with for the time being since the devices send out a beacon every couple of minutes. That means it will take some time before my program will "see" the device but at least it something I can proceed with until a solution can be found.