Sharing data in multithreaded server -- methods

Quote:

Original post by JonW how much of a hit does entering and leaving critical sections over and over realistically incur on performance?

Implementation of a lock will have some minimal constant cost.

In non-contested case, the penalty is effectively constant cost of lock acquisition. This cost depends on which primitive is used.

But the important cost is the cost of contention. If a thread spends 1ms with exclusive access to a resource, and there are 8 threads running on 8 cores, then the theoretical limit will be 125 accesses to resource per second. Even if more are made, the contention will stall other threads. This is also worthless example, since contention often causes cascading effects, but it merely demonstrates the problem.

This is why replication of resources is a good idea, regardless of various details. It allows different parts to progress without contention.

Quote:

What would be the best approach to handle the scenario above?

Impossible to say, since you don't mention the actual architecture. Is the server using IOCP?

Quote:

I feel hesitant to lock every read of the list so that the rare writing operation does not affect the reading operations.

Gaining more performance is poor excuse if algorithm is prone to crashing or resulting in corrupted data.

While lock-free structure might help, they are not the definitive answer. What is this list, why does it need to be shared, how accurately must its state be replicated across threads, ....

Thank you Antheus,

Mainly my synchronization needs involve looking up a particular structure in a linked list. There are many instances where I will find myself needing to perform lookups like these. For a particular example, connected clients are stored in a linked list. Each thread locks the list and appends a new entry each time an "accept" operation completes. Clients connect and disconnect fairly infrequently when compared with the number of times the list must be traversed by reading threads, so there will be a lot of locking going on but little modification of the list. So it seems almost a waste to always have to enter and leave a critical section to traverse the list, especially when reading threads have to lock each other out needlessly.

Note that this is just a particular example of a more general case: maintaining a linked list between threads. And yes, I am using I/O completion ports.

Quote:

Original post by JonW Note that this is just a particular example of a more general case:

It's also a case which is almost guaranteed to not happen in practice. Accept is performed in single thread, which also manages client list. Requests are then handled in IOCP's thread pool, but they know which client they belong to. Completion handlers do not need to know about client list.

Here's the reason why. Since accept and request handlers are performed in different threads, there are no guarantees which will run first. Now consider the following scenario:

When a client completes a request, it wants to let all peers know about it. But - while the client is running a request, either:
- no new peers connected
- 500 new peers connected

Since threads are not synchronized to each other but by nature run in parallel, both of those states are equally probable. So just because your completion thread completed a few ms faster, you don't notify 500 clients because the accept thread simply completed a few ms too late.

Even if you make the list thread-safe, there is no transaction. So while data will be correct, it will not be logically consistent.

----

A more common scenario is this:
On accept, client is stored into "connected" list by accept thread. Completion handlers then negotiate connection, identify the client, perform authentication, and then install this client's ID into some global state. Each of these clients only ever has one outstanding IOCP request. This ensures that completion callbacks do not need any synchronization (two threads will not be handling same socket).

Only at that point is the client put into shared model that will determine when and how it's updated. And this operation needs to be thread-safe. The performance impact of locking isn't relevant here, since it cannot be avoided.

From here on, shared state is updated in yet another thread (only one), and on each loop, it determines what each client has sent, and marks what each client needs to send.