std::stringstream not thread safe?

Keep in mind that C++ is not thread-aware and so the standard library makes no attempt to ever be thread safe about anything. You will generally have to handle locking yourself.

To add to what Promit said, although the standard library does not attempt to be threadsafe it may appear to be. Adding things to buffers for example, unless heavily used by multiple threads (and even in this case) may appear to be threadsafe because the adding is so simple.

Quote:

Original post by Nemesis2k2 After repeated crashes at random times in my heavily threaded program, always in the destructor for a specific std::wstringstream object that only exists for literally 3 lines of code, I did some research, and I'm hearing whispers here and there that stringstream objects may not be entirely thread safe.

If your stringstream is created, used, and destroyed as an automatic variable by a single thread, it should be entirely threadsafe, unless your std::allocator is not threadsafe.

If your std::allocator is not threadsafe, find another compiler vendor. Yours can't be trusted.

If you are sharing a stringstream between threads, you need to treat it like any other datum shared between threads. You most likely need to serialize access to it. As mentioned, the C++ runtime does not automatically provide such functionality, nor should it.

Quote:

Keep in mind that C++ is not thread-aware and so the standard library makes no attempt to ever be thread safe about anything. You will generally have to handle locking yourself.

That's a fair point, but there has to be a limit to that line of thinking. I haven't read every line of code for every standard library, nor do I plan to. I mean, I use std::string and std::vector everywhere. I've never looked at the code for them. I've got no plans to wrap global locks around every access to a vector or string object just in case it's not thread safe. If I can't have any confidence in the C++ standard library in a multithreaded program, that's a BIG problem for me. I use the STL to make programming easier, and to reduce errors. Most of my apps nowadays are multithreaded, and I've got no intention of re-writing the STL.

The simple fact is, C++ is moving into a more "thread aware" state. C++0x should introduce native threading support. With that, I would expect the standard library itself to become thread safe, or at least, for the rules to be clearly spelt out. I just want to work around this minefield until it's cleared. My concern right now is the stringstream class. My exposure to the STL in general is limited, and I can control access to any affected areas, I just need to know what exactly needs to be controlled. I've been coding in C++ for 10 years now, and this is the first I've heard of a stringstream potentially not being thread safe. Where does this end? Does it affect the iostream library itself? Does anyone have a list of known safe or known danger areas?

Consult the documentation for your standard library implementation. For example, for MSVC, there's a document titled "Thread Safety in the Standard C++ Library" in MSDN that details what threading guarantees the MSVC standard library implementation provides.

Quote:

If your stringstream is created, used, and destroyed as an automatic variable by a single thread, it should be entirely threadsafe, unless your std::allocator is not threadsafe.

Are you absolutely sure about that? In the absence of anything to the contrary, I would assume the same thing. I did assume the same thing. Right now though, all I can say is I'm getting random crashes in the destructor for stringstream objects. It's either a buffer overflow in some random part of my code (and I've virtually ruled out that possibility already), or it's not in my code at all.

Quote:

Original post by SiCrane Consult the documentation for your standard library implementation. For example, for MSVC, there's a document titled "Thread Safety in the Standard C++ Library" in MSDN that details what threading guarantees the MSVC standard library implementation provides.

Thanks, I'll look that up. I am using VC++2005 BTW.

Quote:

Original post by Nemesis2k2 The simple fact is, C++ is moving into a more "thread aware" state. C++0x should introduce native threading support. With that, I would expect the standard library itself to become thread safe, or at least, for the rules to be clearly spelt out.

The rules are clearly spelled out by implementations. At this point, pretty much all new versions of them have the same rule: Containers allow unlimited concurrent reading, but writing requires exclusive access to that container. Older versions have some thread-unsafe-on-read CoW stuff... offhand, I think that's limited to std::basic_string and its derivatives.

Oh, and for the record, I am talking about thread safety in the context of two completely separate, independent objects being created and used at the same time in separate threads, NOT two separate threads accessing the same object at the same time. This should be thread safe, as long as the objects don't rely on a common global or static variable. Sorry, I just realised I never qualified this.

Quote:

The rules are clearly spelled out by implementations. At this point, pretty much all of them have the same rule: Containers allow unlimited concurrent reading, but writing requires exclusive access to that container.

Sorry, I think you mistook what I meant by thread safety. I've clarified what I meant in the above post.

The only manner in which different objects of this sort ever affect each other is memory allocation. Unless you're doing special pooled allocator stuff (and you'll know if you are) you're fine. (Assuming a thread-safe allocator. You're using a threadsafe runtime, right?)

If it's a completely thread local object that isn't being used from anywhere else, your crash is probably not threading related. So let me ask -- what exactly is the crash? What is it doing at the point of failure? Is it an assertion or an exception?

Quote:

The only manner in which different objects of this sort ever affect each other is memory allocation. Unless you're doing special pooled allocator stuff (and you'll know if you are) you're fine. (Assuming a thread-safe allocator. You're using a threadsafe runtime, right?)

No, I'm not doing my own allocators, and yes, I am using the multi-threaded runtime.

Quote:

If it's a completely thread local object that isn't being used from anywhere else, your crash is probably not threading related. So let me ask -- what exactly is the crash? What is it doing at the point of failure? Is it an assertion or an exception?

When I did some debugging on the last crash, it looked like it might be a double-free problem with a critical section object which is owned by the stringstream, IE, the object was deleted twice. The crash occurs within ntdll.dll, in a function to destroy a critical section. Specifically, it's an access violation reading from address 0, while trying to parse a structure within the object. I can't be more precise right now, as I unfortunately didn't save any more info on the crash, and since it's a random kind of problem, I'll just have to wait for it to happen again.

Actually, I've just had a look at the CRITICAL_SECTION object to refresh my memory, and the crash occurs in the DeleteCriticalSection function, in code which handles the DebugInfo struct within the object, while parsing the "ProcessLocksList" member. The members of the ProcessLocksList were zero, and the code assumed they pointed to valid memory addresses. Unfortunately, there's no way I can determine the real cause of the crash without finding out where and how these members were set to 0. The other members of the DebugInfo struct did appear to be valid at the time of the crash.

On the surface, it looks like the kind of thing you'd blame on a buffer overflow. I think the nature of this crash means it's one I'm going to have to debug myself. The extremely short-lived and localized nature of this object however, the fact I haven't got crashes anywhere else, the fact no other corruption appeared in the affected structure, and, quite arrogently, the fact that IMO I've basically eliminated the potential for buffer overflows in my coding style over the last few years led me to do some research first. Before I spend the next week chasing a phantom buffer overflow, I wanted to check if a stringstream was even thread safe to begin with. That discussion in the boost mailing list implies that this isn't necessarily the case. Specifically, there's this comment:

Quote:

"Perhaps there is more to std::stringstream in this regard, as I remember you talked about some global variables in it, but it appeared the issue stopped at replacing it with the customized stringstream (in addition to replacing std::string)."

and this comment:

Quote:

"> It looks like the "thread-dangerosity" you're > speaking about is like the same we would have by used two different > stringstream objects in two different threads... "

This is what I'm doing. I'm using two (or more) different stringstream objects in two (or more) different threads concurrently. If this is the cause of my bug, I can deal with it, but I was alarmed to hear that this might not be safe. I wanted to check here to see if anyone else had some more info about this.

Quote:

Original post by Nemesis2k2 When I did some debugging on the last crash, it looked like it might be a double-free problem with a critical section object which is owned by the stringstream, IE, the object was deleted twice. The crash occurs within ntdll.dll, in a function to destroy a critical section. Specifically, it's an access violation reading from address 0, while trying to parse a structure within the object.

That usually means you trashed the object at some point.

Quote:

I wanted to check if a stringstream was even thread safe to begin with.

No, but then your definition of thread-safe is not the one most people use, hence not being entirely relevant to the problem. It's almost always fine to use 2 'non-thread-safe' objects in independent threads, and I'm pretty sure std::stringstream is guaranteed to be safe in that context. I doubt there's any static or global state in stringstreams.

Quote:

Original post by Kylotan It's almost always fine to use 2 'non-thread-safe' objects in independent threads, and I'm pretty sure std::stringstream is guaranteed to be safe in that context. I doubt there's any static or global state in stringstreams.

There is no guarantee about thread safety in the C++ standard. Period.

Normally the std::stringstream uses std::allocator, which through inference must use global objects. That's why I pointed out that if std::allocator is not threadsafe, all bets are off when it comes to creating stringstreams in automatic storage.

The std::stringstream also uses the global std::locale object. I would expect a sane implementation of the std::facets would be threadsafe, but again there is no guarantee. Think about a naive implementation of the std::codecvt facet in a multibyte locale.

So, right there are two global objects (or at least objects in static storage) that must be explicitly threadsafe before a stringstream in automatic storage could be threadsafe.

Now, from the description, the problem is occurring in some debug object that has a critical section that gets destroyed and zeroed. My guess is that the standard library vendor has introduced an additional global object behind the scenes to use for a debugging implementation. My guess is that that non-standard debug object is entirely non-threadsafe and the root of the problem. The implementation is still conforming, but certainly unusable in today's environment.

There may be some way to disable the debug objects through a compiler switch or globally defined macro name. It might be worth checking with the library or compiler vendor.

Ok, if your problem was really the creation and destruction of independent std::stringstream objects, then this program should die in the same way:

#include <windows.h>#include <sstream>DWORD __stdcall function(void *) {  for (;;) {    std::stringstream sstr;    sstr << "foo";  }}int main(int, char **) {  CreateThread(0, 0, &function, 0, 0, 0);  CreateThread(0, 0, &function, 0, 0, 0);  function(0);}

Three threads all creating and destroying stringstream objects constantly. I've built and run this in MSVC and have had this running for about ten minutes now and nothing's crashed.

Wasn't there a bug in the Visual Studio 2005 pre SP1 version of std::basic_stringstream?

Are you using VS2005? Have you installed SP1?

This is looking more like a buffer overflow or rouge write. I got another crash yesterday while exiting the program which was clearly caused by heap corruption. I also got another crash in the stringstream destructor where the entire DebugInfo struct had been overwritten. Now I've just got the fun task of identifying which section of code is responsible, with a dozen actively running threads and over 60,000 lines of code to check. I guess I'm just (un)lucky that this one temporary object seems to be the only thing that's being regularly affected.

Quote:

Ok, if your problem was really the creation and destruction of independent std::stringstream objects, then this program should die in the same way:

Yeah, I've tried similar tests and I haven't been able to reproduce this crash. Mind you, tests like this don't rule out an interaction of other standard library components which share common globals in the back-end, but I'm becoming more confident that the standard library isn't the cause of this problem.

Quote:

Wasn't there a bug in the Visual Studio 2005 pre SP1 version of std::basic_stringstream? Are you using VS2005? Have you installed SP1?

Yeah, it was a memory leak, and it caused me big problems when I migrated to VS2005. I installed SP1 the day it came out.

Could you post some code, or could you condense the problem to a few lines?