Network Packet Delivery

[Effigy]

I don't have any code to show for this yet however, before I get an example together I would like to ask a question.

I'm running into some problems with netoroking in PB.

1. When I'm running a server sometimes I receive a Long negative integer when I'm receiveing RAW data. I can force this to happen if there is no delay in the repeat.

2. My program sends a lot of small packets over the network quite quick. I'm finding many of these packets are simply going missing. My client says he has properly sent it, however the server is simply not recognizing there were packets available. When I put a messagerequester in to slow it down I do receive every packet.

With PB, can I only send one packet at a time untill delivery and processing are complete? I'm somewhat new to network programming and am quite unclear on how much PB does for me and how much I need to do.

Thanks

[Dreglor]

I'm not sure about question 1 but question 2 tcp/ip protocol some times decides to bunch packet data together or split them up so plan on recieving both and parse data accordingly a good way of doing this is to tag both beginning and end of the packets and search for those. If there is data left over parse that and if a packet is unfinished wait for another packet.

[Nik]

IMHO the easiest way for tcp/ip is to use the blocking api functions recv_ and send_ directly and opnly use PB comands for establisishing the connection. Then you can use the ClientID() or ConnectionID() for the API, but the biggest advantage is that when using the MSG_WAITALL flag you can simply wait until it received all the data you wanted and then return with all teh data in the buffer, the advangate is that waiting like that consumes very little cpu time and makes the code very fast. The disadvantage on the other hand is that you will probably need to program multithreaded.
Btw: Though this uses API it's absolutely cross platrform since Windows stole the network API from Unix and Linux and OS X are Unix-like.

[Effigy]

Thanks for the replies... I'm now looping through the code which is working (Kind of).

If I'm sending the packets locally to my computer it works great...

However if I send them over the Internet something weird happens.

I'm checking the results with ReceiveNetworkDatag. This command is supposed to return a value of the bytes read. My SendNetworkData is saying that the whole data has been sent but on the receiving end I only get Bytes read of 1420. I read the buffer again and I get a -1. The first 1420 does look correct. But then I get 0's for the rest of the data. It's like it has disappeard. Now my buffer is 250000 so I thought that would be enough to handle it. I tried reducing the buffer to 1200. And the weird part is I'm still getting a return of 1420? How can I get a return bigger than the buffer.

And to make it even weider. If I step through it, it works correctly. It's only if I let it run at full speed do I see this issue.

Now again this only does this over the Internet not on my local network.

Any thoughts...My next step will be to code using the API commands as suggested.

Thanks

[blueznl]

yeah, looks like your buffers are out of space, did you increase send buffer as well as receive buffer?

[Effigy]

SOme Code for you.. Bassically I'm just pulling a screenshot zipping it and sending it over the network. You will need the purezip librabry installed.

This code works with 127.0.0.1 but does not work over the internet. I get a corrupted screenshot. Some of it is sometimes visible.

Source for Sender

Code: Select all

Global sheight,swidth, connectionid, headersize

swidth=1024
sheight=768



Procedure MakeDesktopScreenshot(ImageNr,x,y,Width,Height)
   hImage = CreateImage(ImageNr,Width,Height,16)
   hDC    = StartDrawing(ImageOutput(ImageNR))
   DeskDC = GetDC_(GetDesktopWindow_())
      BitBlt_(hDC,0,0,Width,Height,DeskDC,x,y,#SRCCOPY)
   StopDrawing()
   ReleaseDC_(GetDesktopWindow_(),DeskDC)
   ProcedureReturn hImage
EndProcedure

Procedure.l CopyImageToMemory(ImageID.l, Memory.l)
    hDC = CreateDC_("DISPLAY", #Null,#Null, #Null)
    GetObject_(ImageID, SizeOf(BITMAP), bm.BITMAP)
   
    bmi.BITMAPINFO
    bmi\bmiHeader\biSize        = SizeOf(BITMAPINFOHEADER)
    bmi\bmiHeader\biWidth       =  bm\bmWidth
    bmi\bmiHeader\biHeight      =  -bm\bmHeight
    bmi\bmiHeader\biPlanes      =  1
    bmi\bmiHeader\biBitCount    = 16
    bmi\bmiHeader\biCompression = #BI_RGB

    ;Debug GetDIBits_(hDC, ImageID, 0, bm\bmHeight, Memory, bmi, #DIB_RGB_COLORS)
    If GetDIBits_(hDC, ImageID, 0, bm\bmHeight, Memory, bmi, #DIB_RGB_COLORS)
      DeleteDC_(hDC)
      ProcedureReturn #True
    Else
      DeleteDC_(hDC)
      ProcedureReturn #False
    EndIf
  EndProcedure 



hDeskBmp = MakeDesktopScreenshot(2,0,0,swidth,sheight)
CurSS=AllocateMemory(swidth*sheight*2)
ReduceSS=AllocateMemory(swidth*sheight*2)
If CopyImageToMemory(hDeskBmp, curss)
    CompSS=AllocateMemory((swidth*sheight*2)+4)
    DestinationLength = swidth*sheight*2
    result=PureZIP_PackMemory(curSS, DestinationLength, CompSS+4, @DestinationLength)
    PokeL(compss,destinationlength)
EndIf

If InitNetwork() = 0
  MessageRequester("Error", "Can't initialize the network !", 0)
  End
EndIf

Port = 5502

ConnectionID = OpenNetworkConnection("127.0.0.1", Port)
If ConnectionID
  
  Result = SendNetworkData(ConnectionID, CompSS, DestinationLength+4)
  
Else
  MessageRequester("PureBasic - Client", "Can't find the server (Is it launched ?).", 0)
EndIf


FreeMemory(CompSS)

Delay(20000)

Source for Viewer

Code: Select all

;Review Captured Screens
Global imageid, mainimage, headersize
Global sheight,swidth,snum


Procedure.l CopyMemoryToImage(Memory.l, ImageID.l)
    hDC = CreateDC_("DISPLAY", #Null,#Null, #Null)
    GetObject_(ImageID, SizeOf(BITMAP), bm.BITMAP)
   
    bmi.BITMAPINFO
    bmi\bmiHeader\biSize        = SizeOf(BITMAPINFOHEADER)
    bmi\bmiHeader\biWidth       =  bm\bmWidth
    bmi\bmiHeader\biHeight      = -bm\bmHeight
    bmi\bmiHeader\biPlanes      =  1
    bmi\bmiHeader\biBitCount    = 16
    bmi\bmiHeader\biCompression = #BI_RGB
   
    If SetDIBits_(hDC, ImageID, 0, bm\bmHeight, Memory, bmi, #DIB_RGB_COLORS)
      DeleteDC_(hDC)
      ProcedureReturn #True
    Else
      DeleteDC_(hDC)
      ProcedureReturn #False
    EndIf
  EndProcedure 



hWindow = OpenWindow(0, 0, 0, 921, 692,"Load picture example", #PB_Window_SystemMenu|#PB_Window_MinimizeGadget )
If hWindow
   sheight=768
   swidth=1024
   MainImage=AllocateMemory(swidth*sheight*2)  
  If InitNetwork() = 0
    MessageRequester("Error", "Can't initialize the network !", 0)
    End
  EndIf

  Port = 5502

#curimage=3
imageid = CreateImage(#curimage,swidth ,sheight,16)
If CreateGadgetList(hWindow)
      If ImageID
        smheight=600
        smwidth=800
        Result = ScrollAreaGadget(5, 0, 0, smWidth, smHeight, sWidth,sHeight, 1 ,#PB_ScrollArea_BorderLess ) 
        ImageGadget(0, 0, 0, swidth, sheight, ImageID)
        SetGadgetState(0,ImageID)
        ResizeWindow(0,0,0,smwidth, smheight)
        HideWindow(0, 0)
        AddKeyboardShortcut(0, #PB_Shortcut_Space, 0)
        SetForegroundWindow_(hWindow)
        ;dodepth=1
      EndIf
EndIf

If CreateNetworkServer(0, Port)

  first=0
  Repeat
      
    SEvent = NetworkServerEvent()
  
    If SEvent
    
      ClientID = EventClient()
  
      Select SEvent
      
        Case 1
          ;MessageRequester("PureBasic - Server", "A new client has connected !", 0)
  
        Case 2
          
          
           Buffer = AllocateMemory(250000)
           bytesread=ReceiveNetworkData(ClientID, Buffer, 250000)
           newlength=swidth*sheight*2
           totallength=PeekL(buffer)
           Result2 =PureZIP_UnpackMemory(buffer+4, totallength, MainImage, @newlength) 
           CopyMemoryToImage(MainImage, imageid) 
           
          ;MessageRequester("Info", "String: "+PeekS(Buffer), 0)
           SetGadgetState(0,imageid)
           FreeMemory(buffer)
        Case 4
          MessageRequester("PureBasic - Server", "Client "+Str(ClientID)+" has closed the connexion...", 0)
          Quit = 1
    
      EndSelect
    EndIf
    
    EventID = WindowEvent()
    Delay(20)
    ;calldebugger
  Until Quit = 1 Or  EventID=#PB_Event_CloseWindow

   
   EndIf
EndIf

This is a simplified version of what my real code is... I would appreciate any help in figuring this one out.

Thanks

[CadeX]

Ask it from me, i'm a real geek when it comes to networking clients and servers. When sending ALOT of packets at once, make sure you use delay(#) every now and again, trust me.. it works.

As for question one, i have no idea what your saying.

[Effigy]

Yes everyone should just ignore question 1... That was not clear and it is just a symptom of the problems I am having.

As for delay... I have used it in the loops... I just forgot to add it to this example. Plus the Max size of the package I'm sending is much smaller than the receive buffer so I really shouldn't need to get the data more that once at least for this example.

The question stands why is the package I'm receving not the full length of the data I'm sending. The code works on my localhost, I only see the problem over a "real" internet connection.

I hope someone can test this code and verify that it's not just me.

[Effigy]

Has anyone been able to test this code over the Internet. I would love to get some testers to see if the problem isn't a local issue. Again you need to test it over the Internet to see the problem. It works fine over a local network.

Thanks for testing...

[Effigy]

Apparently I was just very confused on how the buffers for the Netowrk delivery work... I seem to have found a solution...

Thanks for the suggestions...

[Nik]

Well your code assumes that the data comes in one big chunk wich apparently is wwrong when sending in realistic cirumstances, tcp/ip doesn't guarantee that the packets to arrive in one big chung but it gurantees that everything arrives in the right order. So when dealing with receiving you have to receive as many times as it takes to receive all the data you wanted, you can either receive in a loop using ReceiveNetwork data and checking for new data every time, which will take unnesassary CPU time or you can use the API Command len=recv_(Connection(ClientID), Buffer, length, flag); with the #MSG_WAITALL flag but even then you will need to check the length, which also means you will hae to send the size of the picture before thet data itself in order to know when you are done receiving this would for example work like this:
1. Send a long containig the size of the picture
(Beware that when orting this to PowerPC Macs you will need to check for endianess)

First Sending picture and size(two commands will arrive in one buffer so no need for alocating extra memory for the long and poking it into it)

Code: Select all

len.l=#SIZE_OF_PICTURE_IN_BYES
SendNetworkData(ClientID, @len,sizof(len)); Sending the length
SendNetworkData(ClientID,*PicturePointer,Picturesize)

Then on the other side we will do the following

Code: Select all

PictureSize.l
recv_(Connection(ClientID),@PictureSize,sizeof(PictureSize),#MSG_WAITALL)
; PictureSize will hold the Size of the Picture in bytes

len=recv_( Connection(ClientID), Buffer,PictureLength, #MSG_WAITALL);
;if len is 0 client disconnected before the picture was received 
;if len  < 0 ERROR
if len=PictureSize
;Picture complete display 
endif

[Nik]

One more comment on how the network stuff works:
You have to imagine the network connection established over tcp/ip as sort of a queue everything you put in comes out on the other end in the correct order but you never know how much at a time.
So if sending foir example 1000 byte you will probably receive 10 packets of each being 100 bytes long at the other side, so you will need to wait until you got 1000 bytes, thus you need to know that you are waiting for exactly 1000 bytes, and you need to use the #MSG_WAITALL flag so that recv doesn't return until you got 1000 bytes. The other way would be to use recv without thatz flag putting everything you get into a buffer taking care that you don't overwrite anything and then wait until the buffer is filled with everything you wanted.