Bug (or something weird) in ReceiveNetworkData

Seldon · Post by **Seldon** » Tue Sep 09, 2003 6:49 am

Hi,

I don't know if it's a bug: I've this call on my code:

nbytes=ReceiveNetworkData(DatCon,membuf,16384)

DatCon and membuf are perfectly initialized, the matter is that even if the file I'm getting from the net is longer than 16384 bytes, *sometimes* the function returns a value less than 16384 but I think that shouldn't happen. As, according to manual, when nbytes is equal to the passed length, it means I have more bytes to get from the net. It isn't a problem of my network as I use a similar function in BlitzBasic and it DOES return ALWAYS 16384 bytes. That is a bit annoying as I can't know if a file is finished and I had to resolve it in other way. Thanks for help.

Karbon · Post by **Karbon** » Tue Sep 09, 2003 8:05 am

It's not guaranteed that ReceiveNetworkData is going to give you as much data as you ask for at one time.. You should always call ReceiveNetworkData and SendNetworkData checking their return values and comparing against how much you expect..

I believe that SendNetworkData and ReceiveNetworkData are both direct pass through functions to the Windows sockets recv() and send() functions so I doubt there is a bug there...

Try replacing your ReceiveNetworkData(DatCon,membuf,16384) with recv_(DatCon,*membuf,16384,0) - that's using the windows socket function directly - see if the results vary...

As far as Blitz working all the time.. I went looking for my copy of BB and I can't find it right now to do up a test. If it does *always* return the number of bytes you ask for then I'd say you're either really really lucky or it's using some additional internal buffering that does multiple calls to the windows socket recv() until the amount of data requested is available..

Example, checking the return value like I mentioned..

Code: Select all


;
; You'll have to get your expected_bytes number some how before you 
; use this!
;
While total_received_bytes < expected_bytes 

  received_bytes = ReceiveNetworkData(DatCon,*membuf,16384)

  total_received_bytes + received_bytes

Wend

Good luck!!

Tranquil · Post by **Tranquil** » Tue Sep 09, 2003 2:18 pm

I think BB uses other networkroutines , maybe own of Marks, that handles the complete networkthingy. In PB you have to do it yourselfe!

You only have to check how many bytes are send and how many bytes are received.

Expect the following:
Your application sends datas to the internet (you are connected with a 56kbit modem). Exactly at the same time the user sends mail, again, over the same connection.
Atm there are much more datas to send then the modem can handle. For that case there are some buffers. Buffers on Network-Cards, a Socketbuffer in Windows, and a TCP/IP Buffer again in windows. If these buffers are full your datas can not be sent at this moment, you you have to wait a little bit until there is a little bit more space on the buffers.

Same on the Recipient-side.
Suggest you are a LAN users and MANY users sends datas to your application. You only have a very slow CPU (only to make it clear) that can not handle all these incoming datas, so again your buffers will be run full and other "Senders" in the network are not able to send more datas to your application.

Hope it helps to understand network circumstances.

Cheers
Mike

AngelSoul · Post by **AngelSoul** » Tue Sep 09, 2003 2:49 pm

What if the file size is 49152 bytes (3 x 16384). How would you know the end of the file?

In some case the remote close the connection when file transfer is complete. If you do it that way, the sender could send a line with CrLF with some instructions such as the filename and size then what comes next to CrLF is the actual file data, but of course if it is decided to use a single port that would be the way to do it.

Tranquil · Post by **Tranquil** » Tue Sep 09, 2003 4:45 pm

I created an own protocoll to handle that. It looks like That:

size.w -> Packsetsize in Bytes
command.l -> 4 Chars CommandString
Datas... -> Your Datas or parameters to command string here
CRC32 -> Optionala crc32 at the end of the packet to see if its corectly received

Anyway, the CRC is not needed couse TCP/IP does it for you. So you could synchronize your incoming Datas with your expected packets.
I posted a source some times ago, search for "correctly networking" in the tipps and tricks section. you should find it there.

Mike

Karbon · Post by **Karbon** » Tue Sep 09, 2003 8:24 pm

I meant to post this last night.. You can see an implementation of what Mike is talking about here : http://www.purebasic.org/comments.php?id=P20_0_1_0

It's not perfect by any means but should get you started. Feel free to ask questions!

Seldon · Post by **Seldon** » Tue Sep 09, 2003 9:52 pm

Thanks alls for reply. Karbon, I had already done as you've suggested, but I wanted to be sure if that was the only way as the help seems to suggest something different:

[...]
The 'Result' returns the number of byte effectively read. If 'Result' is equal to DataBufferLength, then more data is available to be read.
[...]

I thought I could use such feature to download a file without knowing its size in advance. Actually it suggests that if a file is bigger than DataBufferLength, DataBufferLength will be equal to Result.. so I can stay in a loop till DataBufferLength will be different. Of course it doesn't work that way.. but I do think this line: "If 'Result' is equal to DataBufferLength, then more data is available to be read", should be removed or stated that it doesn't happen always.

Karbon · Post by **Karbon** » Tue Sep 09, 2003 11:07 pm

That is confusing for sure... Fred or someone else more in the know about the internals of PureBasic are going to have to chime in here... That explanation of the value of Result doesn't make much sense to me.. That function returns the number of bytes successfully read, period. *shrug*

To get all the data available just keep calling ReceiveNetowkrData until it returns 0 I suppose...

Remember there are the SendNetworkFile and ReceiveNetworkFile procedures too..

Post by **freak** » Tue Sep 09, 2003 11:23 pm

The 'Result' returns the number of bytes actually read.
Well, if there are more bytes to read, than the size of the buffer you
specify, then the function can't read more than that.

Then the result is equal to 'DataBufferLength', because more can't be
read. Simple logic, isn't it?

Now, if that happens, you have to assume, that there is more to read.
If that was all, you'll get a 0 as result next time.

If the result is less than 'DataBufferLength', then there was more space
in the buffer than data to be read, so all of it was read.
So in this case, you can be sure, there is no more to read.

I guess the "read until result=0" also works fine.

Timo

Karbon · Post by **Karbon** » Wed Sep 10, 2003 12:56 am

doh. You're right freak - it does make perfect sense.. I must have forgot my thinking hat in Arizona when I moved!

fsw · Post by **fsw** » Wed Sep 10, 2003 2:19 am

Karbon wrote:doh. You're right freak - it does make perfect sense.. I must have forgot my thinking hat in Arizona when I moved!

[off topic]
You know, that you could have bought Kentucky Bourbon in Arizona - you didn't had to move...

[/off topic]

Karbon · Post by **Karbon** » Wed Sep 10, 2003 2:30 am

Hahaha

I moved around for several years - there always comes a time to go home! And the moonshine here is better than anywhere else

Seldon · Post by **Seldon** » Wed Sep 10, 2003 6:47 am

@Freak: the matter is that it DOESN'T work that way.. otherwise this thread would have no sense.

For example, I have a file of 129379 bytes, I do this: result=ReceiveNetWorkData(Con,Buf,16384) , well.. Result can be 7000 or 16384 or 2000 or anything else (<=16384 of course). So it happens that even if "Result is NOT equal to DataBufferLength" , I still have MORE data to read. So I think the help file line, should be updated (IMO).

Post by **freak** » Wed Sep 10, 2003 11:05 am

I guess the reason why this happens, is because at the time of the
call, there really wasn't more to read. What point would there be, to
have more data avaiable, and not receive it (even though the buffer
could handle it) ?

You can't expect that file to be sent and be avaiable at once.

So if that happens, and you expect more data than you could get,
then you just have to wait for another 'data received' Network event,
and then go on receiving the data.

Timo

Tranquil · Post by **Tranquil** » Thu Sep 11, 2003 7:23 pm

Seldon wrote:@Freak: the matter is that it DOESN'T work that way.. otherwise this thread would have no sense. For example, I have a file of 129379 bytes, I do this: result=ReceiveNetWorkData(Con,Buf,16384) , well.. Result can be 7000 or 16384 or 2000 or anything else (<=16384 of course). So it happens that even if "Result is NOT equal to DataBufferLength" , I still have MORE data to read. So I think the help file line, should be updated (IMO).

Okay, I will try to explain again. Its very simple if you know how it works:
If you >>>successfully<<< send a datapacket of, lets say 20.000 Bytes (sendnetworkdata returns exactly these value then everything is all right), it could happen the following thingie:

You have a receivebuffer of 1.000.000 Bytes (too much, but just to say a number). With your first call of recv_() you only received 10.000 of 20.000 Bytes. Thats a normal TCP/IP behavior. Just call recv_() again and again till your packet is completed. Of course, you need to know how large your expected package must be. But there are sooo many examples here in the forum made by karbon and me thats show how to do it.

By the way: A working network-routing (just working - nothing more) does not close security holes! Espiecially if you use an own protocol.

Mike