dll heaps

Your heap will grow until it exhausts the entire virtual memory available to the system (unless you have a quota set, which generally you will not). Oh, and the DLL does not have its own heap; there is one memory space per process.

If your DLL is using a gig and a half of memory, though, it generally means you're doing something wrong. What is taking up so much room?

Quote:

Original post by Sneftel Oh, and the DLL does not have its own heap; there is one memory space per process.

I though each process has one address space, but within that space, each DLL has its own private heap? I think thats why you can't malloc some memory in one DLL and try to free it in another DLL (within the same process of course)


I'm trying to do some operations on very large data sets. There isn't much I can do to decrease the data set size. So I'm hoping that I manage memory better to handle the biggest data sets possible.

Quote:

Original post by indigox3 I though each process has one address space, but within that space, each DLL has its own private heap? I think thats why you can't malloc some memory in one DLL and try to free it in another DLL (within the same process of course)

That's sort of the old way. The new way is to dynamically link the memory allocation functions themselves, such that the DLL uses the host process' own allocation functions to allocate memory. Regardless, though, both would be in the same memory space, using the same system calls to get more pages allocated, and either one could potentially allocate "all the memory".

Quote:

Original post by Sneftel The new way is to dynamically link the memory allocation functions themselves, such that the DLL uses the host process' own allocation functions to allocate memory.

Do you have any docs or references that explain how this works? It would be good to know the details of this.

Although, as long as the heap that my DLL is using (private or the application's) can grow until running out of memory there isn't much I can do...

Quote:

Original post by indigox3 Do you have any docs or references that explain how this works?

Your compiler documentation should talk about it. In Visual C++, this is enabled by selecting one of the "DLL" options for the "runtime library" field in Project Options.

Whether or not a dll has it's own heap depends on the details of your compiler and how you built your app.

One common case is if the C runtime library (e.g. the library where the code for malloc lives) is built into your dll or lives in it's own dll that is shared by everybody. In the case where it's built in then the runtime will commonly (by not necessarily) allocate a big chunk of memory from the OS that only it knows about and dole it out in bits as you request. In the shared case the same thing happens but this time everybody is using the same chunk of memory instead of each having it's own.

This is the reason for the cautions about allocating in one dll and freeing in another. You don't know, in general, whether a given dll is using a private or a shared heap. If you try to mess with the control structures of a heap you don't know about then you'll likely corrupt them somehow.

Even though there's only one address space you can have multiple heaps. As mentioned a heap is just a chunk of memory and some control structures. e.g. see HeapCreate in Win32.

If you're using 1.5GB it's unlikely that fiddling with private or shared heaps will do much good. You're probably really out of memory. Or at least you are out of address space in which to allocate more heap areas. Remember that your address space is shared not only by the heap(s) but by code, stacks, memory mapped files, various areas used for OS bookkeeping, etc, etc. In normal circumstances on 32-bit OS's you only have 2GB of address space. And oftentimes allocating big new structures (such as heaps) requires continuous ranges of space and you are likely to be fragmented.

Quote:

Original post by Anon Mike you are likely to be fragmented.

This was my next thought. Are there any tools for determining the amount of fragmentation? Right now, I'm just looking at the task manager to get an idea of whats going.


Thanks for the help guys,