Networking newbie alert!

[srod]

Hi,

I just have a quick question regarding threads and servers and a multi-threaded server app. which I am working on. Just looking for some advice from those more experienced than I with network stuff.

I have a server (TCP) running with a single 'network event loop' running within the main process. When a new client connects, the server creates a new thread to service that connection etc. The main process will then send 'messages' to these individual threads as and when appropriate as data is received etc.

My question regards whether my following 'plan' is sound or not?

My intention is to proceed as follows : when a client makes a request to the server; the server sends the client's thread a message to tell it to read some data from the network connection and to process the request etc. This of course means that multiple threads could be engaged within the business of reading data from different clients simultaneously, and I am unsure if this is a good idea or not?

The alternative of course is for the main process to retrieve all the data first from the network connection and to then send that to the client thread? This way, only the main process undertakes the task of reading data streams from the network connection etc.

My preference at this stage is for the first option of having the individual client threads read from the various data streams. I am just unsure how the network lib will hold up to this kind of multi-threaded abuse though?

Any and all advice gratefully received.

Thanks.

[Trond]

I don't know anything about networking, but I do know about threads, and I think that having one thread per client is excessive. Let's say you have 1000 clients. That would mean 1000 threads, which is "just too much"™. I would create a fixed number of threads, usually something like Int(processorCount * 1.4)+1.

Then you have a mutex-protected event queue. Every time NetworkServerEvent() returns an event, put the event and EventClient() into the queue.

All the threads then look like this:

Code: Select all

while quit = 0
  GetEvent(*Event.Event) ; Event contains event and client. This function gets the event from the queue and blocks if there are no events
  select *Event\Event
    case #PB_NetworkEvent_Data
      ReceiveNetworkData(*Event\client, ...)
    ; blah blah blah
  endselect
wend

[milan1612]

You're some kind of mind reader! That's exactly the same question which I asked
in the German forums yesterday

I'm very interrested in the concept of having a server system where every client
has his dedicated thread and every single thread is running absolutely independant
of the main server loop and other threads.

Though I'm not a networking expert, I tend to advice the second approach, namely
to let the server loop receive all data and then pass it on to the client threads.
This way the complexity of the thread functions is greatly reduced and besides, the whole
data receiving process takes place centrally - in the server's main loop.

@Trond
This problem you mention was discussed extensively in a question on stackoverflow.com.
They came to the conclusion that as long as you don't expect more than 100 clients to
communicate concurrently, you are not likely to run into problems with your threads.

[srod]

I certainly intend limiting the number of threads - though I was intending doing this by limiting the number of connections.

Your queuing aside for the moment... you seem to be suggesting reading the network data within the individual threads which is what I am hoping to do. If you think the network lib is up to that then I shall indeed be taking that path?

Now, as for limiting the number of threads in the way you suggest.... interesting. I guess really you are hinting at using a single queuing thread to deal with all client requests... interesting. Because of the nature of this server though, I would favour individual threads, not for each client, but for each of the different kinds of request - with some care that could work quite well.

Better get the old thinking cap on...

Thanks Trond.

[srod]

You're some kind of mind reader! That's exactly the same question which I asked
in the German forums yesterday

If I was a mind reader then I would have answered the question before you'd asked it!

@Trond
This problem you mention was discussed extensively in a question on stackoverflow.com.
They came to the conclusion that as long as you don't expect more than 100 clients to
communicate concurrently, you are not likely to run into problems with your threads.

100 threads/clients would be more than enough. Having one thread per client would fit very well into the code as it stands right now.

[milan1612]

Now that I read Tronds post twice ( ) I remember that the alternative answer at
that thread on SO was exactly what Trond supposes.

It is certainly a good approach if your calculations per client aren't too expensive
(e.g. the responses of the server take very long to create). However if they take
some time, this approach will become a serious bottleneck...


EDIT: This website here talks about this problem: http://www.cim.mcgill.ca/~franco/OpSys- ... ode68.html
If you hit "Next" on top of it, it takes you right to "Worker Threads", so this seems to be a well accepted method.

[Trond]

Now that I read Tronds post twice ( ) I remember that the alternative answer at
that thread on SO was exactly what Trond supposes.

It is certainly a good approach if your calculations per client aren't too expensive
(e.g. the responses of the server take very long to create). However if they take
some time, this approach will become a serious bottleneck...

It wouldn't become a bottleneck, I think. Because even though there are fewer threads, they will run faster. If you have 100 threads they won't do any more work, in fact they will do less. In networking, generating the reply can't take a lot of time anyways, because the connection would timeout.

With a small number of worker threads every connection accepted by a worker thread will get served rather quickly, though some may timeout before they get served.

With a large number of threads, all connections may timeout because the threads all steal each other's time, thus if you are unlucky, none may finish before the timeout.

Note that if you want to make the server do some heavy work and then call the client back, you'd not do the heavy work in the networking worker threads, you'd make a separate set of lower-priority (lower than the networking threads) worker threads, with a separate task queue, to do the work.

[milan1612]

I'd always prefer a slow response from a server over no response at all

[Trond]

I'd always prefer a slow response from a server over no response at all

You have a greater chance of getting a response with a smaller number of worker threads, since they are less likely to timeout.

Remember: timeout = no response. We want to have as few timeouts as possible. If we process later request in parallel with the first requests, the first requests will run slower and have a greater possibility of timeout.

Note: I don't really know this, but it seems obvious once you think about it.

[Rook Zimbabwe]

I have been playing with the idea of using MAP to multithread...

considering what I don't know about MAP it is going slow!

[idle]

I don't think you would need to do threads at all, the data arrives serially anyhow
so you just need to maintain a table of clients with their own data buffers.

you can easily thread the client and server so it's independent of your app
though I really don't think it's necessary to spawn a thread for each client

I pm'd you a networking lib I'm currently working on it should be easy enough to abstract for multiple clients, though it's currently provides multiple connections with the aim to reduce effects of latency.

[srod]

Well, some requests will involve quite a lot of work from the server whilst others will complete very quickly. Multiple threads will avoid the lengthier operations holding up the shorter ones.

Thanks for the code, it looks very good, but it is unlikely that I will be able to use it as my server needs to be fully crossplatform and, well, I learn best by coding my own you know.

[idle]

Well, some requests will involve quite a lot of work from the server whilst others will complete very quickly. Multiple threads will avoid the lengthier operations holding up the shorter ones.

Thanks for the code, it looks very good, but it is unlikely that I will be able to use it as my server needs to be fully cross platform and, well, I learn best by coding my own you know.

No worries, It might save you a few hair follicles.

I thought about trying a thread per client but decided that it's easier not to and simply limit clients to a max chunk size smaller chunks more clients serviced.

A thread per client is probably doable though from what I've googled a single thread is faster than a multi threaded server, but then you won't really know until you've tried.

The code is more or less cross platform, it's just the error code that's not

[netmaestro]

I made this demo a few years ago, dunno if you'll find anything useful in it, probably not as it's quite simple:

http://www.purebasic.fr/english/viewtop ... 75&start=5

[srod]

Thanks, but not quite the same as you are running 4 different servers rather than one server creating multiple threads etc.

I am going to go with the multiple threaded model in this first instance. If it doesn't work well up to a 'reasonable' number of connections then I will start limiting the threads and queuing requests. At the moment there is no need to queue because each client must await an acknowledgement anyhow before sending out any subsequent requests and so each connection having a corresponding 'worker' thread seems ideal right now.

Only one way to find out...