SendNetworkFile still buggy or can i use it safely?

[netmaestro]

Sorry, but posting a bug report is like yelling in a black hole in this case ...

I disagree. The team has fixed hundreds of bugs since PB v4.0 beta 1 was released, and they continue to fix them as they arise. Often, when bugs are reported and you never see a "fixed." reply, it's because the problem was a result of weak coding and not a flaw in PureBasic.

Multithreading/Multiconnections are not really possible (strange things will happen if you try)

Try the following example, which sets up four server ports on four threads and sends the position of a different colored dot on each port. The server-window dots move with the mouse, and all four dots on the client window read their position from the network. If you notice something strange happening, be sure to let me know:

Code: Select all

; Program 1 - Run this first 
; 
; Then run program 2 
; 
; Then click in this program's window and move the mouse around 

Global FirstPort = 6832 

InitSprite():InitMouse() 

Structure mousemessage 
  x.l 
  y.l 
  send.l 
EndStructure 

Global Dim connection(4) 
Global Dim *mousemsg.mousemessage(4) 

Procedure SendMouse1(thread) ; sends the red dot 
  Repeat 
   If *mousemsg(thread)\send 
     *mousemsg(thread)\x = MouseX() 
     *mousemsg(thread)\y = MouseY() 
     If connection(thread) 
       SendNetworkData(connection(1), *mousemsg(thread), 512) 
       *mousemsg(thread)\send = #False 
     EndIf 
   Else 
     Delay(1) 
   EndIf 
  ForEver 
EndProcedure 

Procedure SendMouse2(thread) ; sends the green dot 
  Repeat 
   If *mousemsg(thread)\send 
     *mousemsg(thread)\x = MouseX()+32 
     *mousemsg(thread)\y = MouseY() 
     If connection(thread) 
       SendNetworkData(connection(thread), *mousemsg(thread), 512) 
       *mousemsg(thread)\send = #False 
     EndIf 
   Else 
     Delay(1) 
   EndIf 
  ForEver 
EndProcedure 

Procedure SendMouse3(thread) ; sends the yellow dot 
  Repeat 
   If *mousemsg(thread)\send 
     *mousemsg(thread)\x = MouseX() 
     *mousemsg(thread)\y = MouseY()+32 
     If connection(thread) 
       SendNetworkData(connection(thread), *mousemsg(thread), 512) 
       *mousemsg(thread)\send = #False 
     EndIf 
   Else 
     Delay(1) 
   EndIf 
  ForEver 
EndProcedure 

Procedure SendMouse4(thread) ; sends the blue dot 
  Repeat 
   If *mousemsg(thread)\send 
     *mousemsg(thread)\x = MouseX()+32 
     *mousemsg(thread)\y = MouseY()+32 
     If connection(thread) 
       SendNetworkData(connection(thread), *mousemsg(thread), 512) 
       *mousemsg(thread)\send = #False 
     EndIf 
   Else 
     Delay(1) 
   EndIf 
  ForEver 
EndProcedure 

screencolor = GetSysColor_(#COLOR_BTNFACE) 
If InitNetwork() = 0 
  MessageRequester("Error", "Can't initialize the network !", 0) 
  End 
EndIf 

If CreateNetworkServer(1, FirstPort) And CreateNetworkServer(2, FirstPort+1) And CreateNetworkServer(3, FirstPort+2) And CreateNetworkServer(4, FirstPort+3) 
  OpenWindow(0,0,200,500,500,"Right-clicking this window ends both programs...") 
  StickyWindow(0,1) 
  OpenWindowedScreen(WindowID(0),0,0,500,500,0,0,0) 
  Dim clr(4):clr(1)=#Red:clr(2)=#Green:clr(3)=#Yellow:clr(4)=#Blue 
  For i = 1 To 4 
    CreateSprite(i,32,32) 
    StartDrawing(SpriteOutput(i)) 
      Box(0,0,32,32,screencolor) 
      Circle(15,15,16,clr(i)) 
    StopDrawing() 
  Next 
  
  ClearScreen(screencolor) 
  StartDrawing(ScreenOutput()) 
    DrawText(180,245,"Run the client now...",#Black,screencolor) 
  StopDrawing() 
  FlipBuffers() 

  cc=0  
  While cc < 4 
    SEvent = NetworkServerEvent() 
    If SEvent = 1 
      cc+1 
      connection(cc) = EventClient() 
    EndIf 
    If WaitWindowEvent(1) = #WM_RBUTTONDOWN 
      End 
    EndIf 
  Wend 
  For i=1 To 4 : *mousemsg(i) = AllocateMemory(512) : Next
  CreateThread(@SendMouse1(), 1) 
  CreateThread(@SendMouse2(), 2) 
  CreateThread(@SendMouse3(), 3) 
  CreateThread(@SendMouse4(), 4) 
   Repeat 
    While WindowEvent():Wend 
    ClearScreen(screencolor) 
    ExamineMouse() 
    If MouseDeltaX() Or MouseDeltaY() 
      For i = 1 To 4 
        *mousemsg(i)\send = #True 
        *mousemsg(i)\x = MouseX() 
        *mousemsg(i)\y = MouseY() 
      Next 
    EndIf 
    DisplaySprite(1,MouseX(),MouseY()) 
    DisplaySprite(2,MouseX()+32,MouseY()) 
    DisplaySprite(3,MouseX(),MouseY()+32) 
    DisplaySprite(4,MouseX()+32,MouseY()+32) 
    FlipBuffers() 
    If MouseButton(#PB_MouseButton_Right) 
      quit=1 
    EndIf 
    Delay(1)    
  Until Quit 
  killmsg.l = 999 
  SendNetworkData(connection(1), @killmsg, 512) 
Else 
  MessageRequester("Error", "Can't create the servers! (ports in use ?)") 
EndIf 
End

Client:

Code: Select all

If FindWindow_(0,"Right-clicking this window ends both programs...") = 0 
  MessageRequester("OOPS!","Run the server program first please",0) 
  End 
EndIf 

Global FirstPort = 6832 

Global Dim Connection(4) 

If InitNetwork()=0 Or InitSprite()=0 
  MessageRequester("Error", "Can't run the program", 0) 
  End 
EndIf 

Global Dim sprite.Point(4) 
Global *buffer = AllocateMemory(512) 
Global *mousemsg.point = *buffer 

sprite(1)\x = 0  : sprite(1)\y = 0 
sprite(2)\x = 32 : sprite(2)\y = 0 
sprite(3)\x = 0  : sprite(3)\y = 32 
sprite(4)\x = 32 : sprite(4)\y = 32 

For i = 1 To 4 
  While Not connection(i) 
    Connection(i) = OpenNetworkConnection("127.0.0.1", FirstPort-1 + i) 
    Delay(1) 
  Wend 
Next 

OpenWindow(0,550,200,500,500,"Click in the first window, then move the mouse around...") 
OpenWindowedScreen(WindowID(0),0,0,500,500,0,0,0) 
Dim clr(4):clr(1)=#Red:clr(2)=#Green:clr(3)=#Yellow:clr(4)=#Blue 
For i = 1 To 4 
  CreateSprite(i,32,32) 
  StartDrawing(SpriteOutput(i)) 
    Box(0,0,32,32,GetSysColor_(#COLOR_BTNFACE)) 
    Circle(15,15,16,clr(i)) 
  StopDrawing() 
Next 

Repeat 
  While WindowEvent():Wend 
  delaying=#True ; only delay if no network events to deal with 
  For i = 1 To 4 
    If NetworkClientEvent(Connection(i)) = #PB_NetworkEvent_Data 
      result=ReceiveNetworkData(Connection(i), *buffer, 512) 
      If PeekL(*buffer)=999 : End : EndIf 
      sprite(i)\x = *mousemsg\x 
      sprite(i)\y = *mousemsg\y 
      delaying=#False 
    EndIf 
  Next 
  ClearScreen(GetSysColor_(#COLOR_BTNFACE)) 
  For i=1 To 4 : DisplaySprite(i,sprite(i)\x,sprite(i)\y) : Next 
  FlipBuffers() 
  If delaying 
    Delay(1) 
  EndIf 
Until quit 
End  

[real]

I disagree totally with that. The team has fixed hundreds of bugs since PB v4.0 beta 1 was released, and they continue to fix them as they arise. Often, when bugs are reported and you never see a "fixed." reply, it's because the problem was a result of weak coding and not a flaw in PureBasic.

Some functions of PB are still trial and error.

I would recommend to write your own sockets or maybe use an Userlib to be sure.

[Poshu]

Can we find a very fast socket Userlib around?

[blueznl]

it's pretty easy to use the sockets, do a search for winsock on the forum

[Anden]

Sorry, but posting a bug report is like yelling in a black hole in this case ...

I disagree totally with that. The team has fixed hundreds of bugs since PB v4.0 beta 1 was released, and they continue to fix them as they arise.

Very good and appreciated, but i'm talking about a bug that hasn't been fixed so far ...

Often, when bugs are reported and you never see a "fixed." reply, it's because the problem was a result of weak coding and not a flaw in PureBasic.

Replace "Often" with "Sometimes" and using a SINGLE command is very unlikely to be considered as "weak coding".

Multithreading/Multiconnections are not really possible (strange things will happen if you try)

Unmitigated poppycock. Try the following example, which sets up four server ports on four threads and sends the position of a different colored dot on each port. The server-window dots move with the mouse, and all four dots on the client window read their position from the network. If you notice something strange happening, be sure to let me know:

Very interesting example (ignoring your personal attacks completely, fanboy), but where's the usage of Send/ReceiveNetworkfile? Are you sure you're posting in the right topic?

[Shannara]

Can we find a very fast socket Userlib around?

For a userlib, I do not know, but an excellent replacement could be found (and costs ...) @ http://www.sockettools.com

[Anden]

Yeah, sockettools are great (but the devs know that too, thus they can be a little bit pricey )

[Fred]

Anden, please fill a bug report with some small snippets in the appriopriate section, it would be more constructive.

[Anden]

There are numerous bug reports, feature requests and other topics in this forum regarding to Send/ReceiveNetworkFile problems. Another bug report is as easy as follows:

Code: Select all

ReceiveNetworkFile(x, y)

I'm sorry, but the problem is WITHIN the command. So nobody can be more specific (and it seems that you are the only one who knows what's happening inside but don't won't to share your insight).

Facts are:
1)
If there are network problems (failure in transmissions, retransmissions necessary, aso ...) ReceiveNetworkFile() never returns from it's task and the PB app "hangs" with 100% CPU load (a ping command terminates with a timeout in the same situation btw)

2)
If one closes the connection immediately after SendNetworkFile() returns, it could happen, that a network "broadcast" is triggered (=loads of small packets at the receiving client queue = 100% cpu load again).
This is only the case, if ReceiveNetworkFile() on the client side has NOT fully received the file at that time.
Which can happen sometimes (seems to be more frequent on slower client machines), because it seems that Send/ReceiveNetworkFile use some kind of caching algo (i have to guess here again, i don't know your code and you don't tell us. Thus that's all the help i can give you right now).

3)
The missing progress information is obvious and would help a lot in nailing down or avoiding the problems described above.

4)
There seems to be a limit for file size. That limit seems to be connected to the RAM available in the sending machine. I've not investigated that case very much. Instead i've limited the max file size for senders to 32 MB (which gave me some harsh critics from my users ), that seems to be reliable for machines with 512 MB main memory installed (and Win 2k/XP).

1+2 are not that easy to reproduce.
1 is very likely to happen from time to time in larger networks with heavy load and some routers and switches involved.
2 can be avoided, if only the receiving client closes the connection (not the sender). The sender could only close safely if he knows for sure, that the transmission is 100% complete (needs progress info or fix it by not returning from SendNetworkFile() BEFORE that).

Hope that helps, but it's only a summary of some former posts in this forum ...

BTW:
To nail down 1+2 we used the following:
Large network
Network specialist with hardware sniffer
Manageable switches (because of traps and logs)
At least 2 users willing to help and test
PB with debugger ON as client AND server
Veeeeery much endurance

Estimated cost: app. 20 specialist hours = 2500 Euros (excluding hardware and equipment). You're welcome ...

[Fred]

Could you try with this one ? www.purebasic.com/beta/Network

[Anden]

Of course. Test for 1, 2, 3 or 4? (i'd really like to minimize complexity)

1 needs special network setup to force errors (= complex)
2 needs "the right code" (close immediately after send) and a slow client to max chances of occurrence
3 needs documentation
4 is easy to test

[Trond]

If it only happens on a special network under extremely strict conditions and under the supervision of networking experts waged at 125 euro per hour, will it happen for your client?

[Jan Vooijs]

Facts are:
1)
If there are network problems (failure in transmissions, retransmissions necessary, aso ...) ReceiveNetworkFile() never returns from it's task and the PB app "hangs" with 100% CPU load (a ping command terminates with a timeout in the same situation btw)

2)
If one closes the connection immediately after SendNetworkFile() returns, it could happen, that a network "broadcast" is triggered (=loads of small packets at the receiving client queue = 100% cpu load again).
This is only the case, if ReceiveNetworkFile() on the client side has NOT fully received the file at that time.
Which can happen sometimes (seems to be more frequent on slower client machines), because it seems that Send/ReceiveNetworkFile use some kind of caching algo (i have to guess here again, i don't know your code and you don't tell us. Thus that's all the help i can give you right now).

3)
The missing progress information is obvious and would help a lot in nailing down or avoiding the problems described above.

For the numbers 1 and 2 this is NOT a 100% PB problem i had (have) the same problems with an old version of PowerBasic (7.04 to be precise, never upgraded since i use PB exclusively now). And also a version of RealBasic has the same problem (but you can 'interrupt' the call in a way by a maximum time to wait). I think it is more a windows problem than a PB problem. Or the implementation of a win xp dll (which one i never found out) which has the problem. But giving the call a max time to wait helps enormously like some implementations use (RapidQ for instance, and yes PowerBasic [in the open statement]).

Number 3, a progress report is not possible i believe PowerBasic has the same problem (never returns on 'no completion' of a recieve) and yes it goes to 100% then (hit the reset--) also.

Jan V.

[Anden]

If it only happens on a special network under extremely strict conditions and under the supervision of networking experts waged at 125 euro per hour, will it happen for your client?

It will, because the "special conditions" (it's just a hefty network load) are only necessary to increase/force errors. It just happens less often if you don't force it.

It's part of the bug fixing process to raise certain error conditions, you know ...

[netmaestro]

I don't believe SendNetworkFile() was ever intended for large file transfers. I don't need anything that'll transfer files larger than a couple of mb over a network at the moment, but if I did I wouldn't use SendNetworkFile(). I'd take an MD5 fingerprint of the file, read it in in chunks of 100k, send them one at a time and when they are gone I'd send the fingerprint. The whole thing would go in its own thread and with this approach, health-monitoring as well as a progress bar are a snap. And your main loop can just keep on truckin'. Process is reversed on the receiving end and when the whole file is in its md5 fingerprint gets checked against the one it received and if they match you can display a messagebox with confidence: File received successfully. If you do it this way you can send gig-sized files across the network, cancel anytime, watch the progress, etc. Might take a couple hours to program it but the result would be worth the effort, imho.

Another advantage to this approach is that you can manipulate the transfer speed. If time isn't a concern you can put sizable delays in the loops and avoid bogging the network traffic down. Take a little care in the design and you can even do pause-stop-resumes.