Would anyone like a nice IO Completion Ports example?

I think this is a great idea. Good examples are always helpful, especially when it is clearly stated why they do things in a certain way (plus, perhaps, a note on why it is wrong to do the same thing in a different way).
Plus, I've been wanting to learn IOCP for a while now :D. You wouldn't want to leave somebody salivating for a great example and not getting it, would you?

I'm also interested in it, I want to learn IOCP too, and its true that is difficult to find good information and code examples in the net.

Personally, I have no more use for such an example (apart from being curious, you never know whether you can still learn something from someone else's code), but it is certainly something that would be very valuable. It might save people who are still struggling on that particular thing several weeks of pain.

It might be even more helpful if it included file i/o as well, and a Linux epoll port. Completion ports are much easier (almost trivial, once you have waded through all documentation) to get right with only sockets, and the same is true for epoll. Getting both sockets and file read/writes into one "thing" without blocking, however, is rather non-trivial.

Quote:

Original post by samoth It might be even more helpful if it included file i/o as well, and a Linux epoll port.

Or, instead of dealing with API pecularities, just focus on actual tasks.

Quote:

Original post by Antheus Quote: Original post by samoth It might be even more helpful if it included file i/o as well, and a Linux epoll port. Or, instead of dealing with API pecularities, just focus on actual tasks.

This has been an area of contention for me for quite some time. In most applications abstracting the OS with a library is a good thing. It lets developers concentrate more on what they are doing and less on what the underlying OS is doing. This changes when you enter the arena of system and/or server software. Modern operating systems provide lots of support for high performance applications and in order to take full advantage of that support one must intimately tie the application to the OS. You need to be just as concerned with the underlying OS and what it is doing as well as what you are doing.

Adding general purpose abstraction layers so close to the hardware does make your life easier, but you lose a key advantage. The vast majority of the time that advantage does not matter (super performant code on a quad core 2.4ghz machine gains very little for user applications.) and you can use C#.NET, Java, Boost, whatever your flavor is, to its fullest potential and be just fine. For pedagogical reasons, its best that people learn what is really happening on the metal. It keeps people from writing 10,000 element linked lists that have to be iterated though every 100ms because it is easy to write "LinkedList LList; LList.Find(Whatever);" without understanding the consequences of big O and cache poisoning.

Anyway, it seems as if some people are interested so I will do a write up and post once interview season is over.

PS. The MSDN example for AcceptEx uses IOCP, and it is completely broken. If anyone is looking for a good IOCP QA exercise, go to http://msdn.microsoft.com/en-us/library/ms737524(VS.85).aspx and figure out why it is broken. If you post your answer here I will let you know if you are right or not. It's right on the surface, no digging required, so most people should be able to find it just by looking at the code sample long enough, and I doubt throwing the code into a debugger will be any help.

-Karl Strings

Quote:

Original post by Karl Strings This has been an area of contention for me for quite some time. In most applications abstracting the OS with a library is a good thing. It lets developers concentrate more on what they are doing and less on what the underlying OS is doing. This changes when you enter the arena of system and/or server software. Modern operating systems provide lots of support for high performance applications and in order to take full advantage of that support one must intimately tie the application to the OS. You need to be just as concerned with the underlying OS and what it is doing as well as what you are doing.

Why would that be necessary when abstraction layers such as Boost.Asio which Antheus suggested simply provide wrappers around these high performance OS facilities? One may argue that the abstraction layer might be slightly slower than calling the native API directly, but knowing the Boost community I would say the performance loss is negligible. Besides, that sort of reasoning can be taken just as far as you like: "The C++ compiler generates slow code. I'm better off writing the assembly code by hand."

Quote:

Original post by Karl Strings Adding general purpose abstraction layers so close to the hardware does make your life easier, but you lose a key advantage. The vast majority of the time that advantage does not matter (super performant code on a quad core 2.4ghz machine gains very little for user applications.) and you can use C#.NET, Java, Boost, whatever your flavor is, to its fullest potential and be just fine. For pedagogical reasons, its best that people learn what is really happening on the metal. It keeps people from writing 10,000 element linked lists that have to be iterated though every 100ms because it is easy to write "LinkedList LList; LList.Find(Whatever);" without understanding the consequences of big O and cache poisoning.

In a sense, I agree with you on this. It is important to understand your tools. However, whether or not it's worth your time really depends on what your goal is. If you are trying to gain a deeper understanding of the technology you are working with (in this case, Operating System specific scalable IO), playing around with that is obviously the way to go. However, if you're making a game, you will probably be better off using a reliable abstraction layer written people who know what they are doing (such as the Boost developers). That way, you just might even finish your project. ;)

A few days ago I ran across a thread on an electronics forum where a guy presented his idea for making an ADC (Analog-to-digital converter) from discrete components. The replies were generally discouraging, and the consensus appeared to be that "10 years ago, it might've been fun; today, you're probably better off making something with an ADC." Once again, making the ADC from discrete components would enable him to learn something. Building something else using an ADC IC would allow him to accomplish something.

Quote:

Original post by Windryder Why would that be necessary when abstraction layers such as Boost.Asio which Antheus suggested simply provide wrappers around these high performance OS facilities?

My reply was regarding "linux and file handling". Not the rest of thread or OP.

Quote:

Original post by Antheus Quote: Original post by Windryder Why would that be necessary when abstraction layers such as Boost.Asio which Antheus suggested simply provide wrappers around these high performance OS facilities? My reply was regarding "linux and file handling". Not the rest of thread or OP.

From your post I did get the feeling that you encourage people to apply this principle in a more general context ("focus on the actual tasks"). Hence I thought it appropriate to refer to your suggestion in my post.

Quote:

Original post by Windryder From your post I did get the feeling that you encourage people to apply this principle in a more general context ("focus on the actual tasks").

If someone wants to study IOCP using plain C, that's fine.

But when it comes to cross-platform portability and related abstractions, the problem is solved satisfactory by asio or ACE. And reinventing the wheel in that particular case is somewhat counter-productive.

Portability, especially in C and C++ is a tedious nightmare of typedefs and macros, and lots of obscure debugging around undocumented or other edge cases, and studying something like that for purpose of learning is, IMHO, a waste of time. It's just one of those things that requires years-long tenure of hands-on practice and simply cannot be taught from some book or source code tutorial.

Or, at very least, it's probably best to start with some existing abstraction, then drill down as to why certain choices were made the way they were.

I am an engineer, not a clergyman. I try to stay away from such religious topics as language and libraries, but apparently I have slipped into the trap again.

I think this thread has deviated to far from its original intention.

-Karl Strings

Try to search something on CodeProject.com, there are a lot of tutorials about IOCP in C++.

Quote:

Original post by Windryder Quote: Original post by Karl Strings This has been an area of contention for me for quite some time. In most applications abstracting the OS with a library is a good thing. It lets developers concentrate more on what they are doing and less on what the underlying OS is doing. This changes when you enter the arena of system and/or server software. Modern operating systems provide lots of support for high performance applications and in order to take full advantage of that support one must intimately tie the application to the OS. You need to be just as concerned with the underlying OS and what it is doing as well as what you are doing. Why would that be necessary when abstraction layers such as Boost.Asio which Antheus suggested simply provide wrappers around these high performance OS facilities? One may argue that the abstraction layer might be slightly slower than calling the native API directly, but knowing the Boost community I would say the performance loss is negligible.

My advice when it comes to Boost is to "Chew the meat and spit the bones." Some of the libraries are excellent but not all of them. The performance of the Boost Serialization library is significantly worse than that of the C++ Middleware Writer across a number of tests. I'm not very familiar with the Asio library, but want to point out that you should be careful regarding what you use. If you're not, your company/work will go the way of the dodo bird.

Another weakness of traditional C++ serialization libraries is they don't automate the production of serialization functions. This is a bonus that one gets with C++ Middleware Writer.

Brian Wood
www.webEbenezer.net

It's quite nice to read that someone else has been working with similar project that I've been on for a long time. However instead of just focusing on sockets, I'm writing a platform for handling streams efficiently. Just like you, I originally I worked with C, but after complexity of code increased I decided to swap to C++ to keep the code relatively trivial, clean and easy to maintain.

Even though completion ports do their job when trying to avoid obsolete memory bandwidth usage and cache trashing. From performance point of view (and to keep things trivial), it would make sense to be able to map kernel read and write buffers directly to user space instead of mapping user space buffers to kernel space (completion ports).

Quote:

Original post by skorhone ... From performance point of view (and to keep things trivial), it would make sense to be able to map kernel read and write buffers directly to user space instead of mapping user space buffers to kernel space (completion ports).

Explain? I would also like to know the reson you think that mapping memory from kernel space to user space is trivial...

Just to clarify, mapping memory from one ring to another is not usually trivial. Yes, it may not be hard to "do", but it is very easy to open your code up to some wicked race conditions that are very hard to debug.

If your goal is to map memory, have fun, just be aware that you can have some weird race conditions that may not be apparent just by looking at the code.

I think the general idea is that if the device driver gets to specify the parameters for a buffer (alignment requirement, contiguity requirement, physical address requirement, etc) then it may be more efficient (due to hardware limitations) than some arbitrary buffer allocated by the application.

For example, if some DMA operation needs 64-byte aligned buffers, and a user hands in a 16-byte aligned buffer (or even a 4-byte aligned buffer from malloc()) then chances are that the driver will need to re-buffer, which triples the cost of the operation(!). Original write into internal buffer; read out of internal buffer; write into user buffer.

And before you say that all hardware should support DMA to arbitrary scattered memory pages with arbitrary alignment, let me point at the 99% of hardware out there that doesn't :-)

I didn't mean that the implementation of this kind of interface was trivial - it obviously isn't. What I meant was that such interface could be easier to utilize efficiently (as you already noted, most completion port implementations are more or less wrong)

Interface could be something simple as:
flags |= WSA_MEM_MAP;
buffers[0].buf = NULL; /* wsarecv updates this */
buffers[0].len = 4096; /* max read length */
WSARecv(sock, buffers, 1, &bytesRead, &flags, overlapped, NULL);
.

.
WSAFree(buffers, 1);

Instead of:
3* WSARecv(sock, buffers, 1, &bytesRead, &flags, overlapped, NULL);
.

Quote:

Original post by skorhone I didn't mean that the implementation of this kind of interface was trivial - it obviously isn't. What I meant was that such interface could be easier to utilize efficiently (as you already noted, most completion port implementations are more or less wrong) Interface could be something simple as: flags |= WSA_MEM_MAP; buffers[0].buf = NULL; /* wsarecv updates this */ buffers[0].len = 4096; /* max read length */ WSARecv(sock, buffers, 1, &bytesRead, &flags, overlapped, NULL); . . WSAFree(buffers, 1); Instead of: 3* WSARecv(sock, buffers, 1, &bytesRead, &flags, overlapped, NULL); .

Let me make sure I understand this... You want the allocation to happen in the kernel and WSARecv will return a kernel allocated buffer to you on completion? I am assuming that you are doing this to ensure proper memory alignment and no double allocations?

I think you will find better results and less bugs by using look aside lists. You can allocate your memory however you need to in user land at startup and free at program close. One round of allocations and frees, everything is correctly aligned however you want, and most important - no kernel code is needed.

True, but the point would be to make the application developement trivial. Completion ports have been there over a decade, still there's only handful of proper implementations and documentation is above average - there must be a reason for that :)

In my opinion interface that is hard to use, has a design flaw. Enough ranting, just trying to point out that currently completion ports are propably not the ultimate solution. With a proper interface, perhaps.

Quote:

In my opinion interface that is hard to use, has a design flaw.

If an operation is, at the core, complex, then any proper interface must, by extension, be complex. If you try to put a simple interface on a complex operation, you will be doing something wrong, or something inefficient.

That being said, the wrapper idiom for I/O completion ports that's found in the .NET framework is pretty elegant in my opinion. Most well-written .NET programs use I/O completion ports, and may not even know it. Anytime you call BeginRead() or similar, you're really using I/O completion ports with the Windows thread worker pool.