Ok I get it, thanks!
However I'm still a bit confused about some points:
My first version doesn't shrink in memory (os) when freeing stuff, but doesn't grow neither if I reallocate stuff, because of the free space in the heap. That means that the memory allocated in the heap doesn't get returned to the system, but is free for my program to reallocate. This far I understand.
2 questions come to my mind:
1) why does the memory actually gets returned to the system and the heap shrinks if I use the 2nd program I sent? why does it get returned to the system when I use these 2 programs? (files attached)
2) is there a way for a program to yield to the system the free memory it has in the heap? (or for the system to reclaim it?)
The actual problem I am trying to solve is that I am writing a program that needs to process huge amounts of data, but then I can free nearly everything and only keep the results in memory. I don't care if it swaps during processing, or uses a huge amount of memory, but I'd like to free it afterwards (in the sense: return it to the system) as it is going to be ran as a server and stay resident on a machine that maybe could make better use of that "wasted" memory in the heap...
Thanks again for your help!
Nicolas.
Eljay Love-Jensen wrote:
Hi Nicolas,
I'm watching the memory consumption with 'top' (that's the reason for the call to sleep)
Top does not monitor "free space versus consumed space" in your program's heap.
A different compiler (SAS/C++) gave me an API that I could interact with the heap in my program, asking it such questions as "How many OS extants do you have?", "How big is the extant?", "How much heap free space is in the extant?" And a few knobs to adjust where small allocations versus large allocations go.
I don't know if the Standard C Library provides an API for interacting with the heap manager -- I presume it does not.
There are alternative heap managers that have different performance characteristics, some of which give the programmer more control over heap behavior -- including when an extant is returned to the OS. That'd be for OS's such as Mac OS Classic, OS/2, Amiga OS, Win16 applications. Other OS's, such as Unix-based ones, provide each application a virtual memory space up to their ulimit parameters. (I haven't looked into the underpinnings of Win32 applications under NT -- I don't know if heap management is done via extants, or done the Unix ulimit way.)
What's weird is that if I launch the original program a second time while the first already has supposedly freed its memory, it eats everything and bring my box to its knees, killing most of the processes that use memory (so I guess the memory is not freed at all).
I presume you are seeing the behavior of swap space being utilized.
Yes, that's it, but my thunderbird got killed as well (I guess the kernel should only kill the offending process...) However this is offtopic here :) ...
You should use ulimit, to prevent a process from consuming too much space.
Also, top tells me that the whole memory stays resident (which means it doesn't even return to the program's free memory pool, am I right?)
No. Top does not monitor the free memory pool. The resident memory is not a reflection of free or consumed space in the heap. It's whether the memory is resident (in actual RAM) or swapped (paged out, via the magic of the PMMU).
HTH, --Eljay
#include <iostream>
#include <vector>
using namespace std;
const unsigned int N = 300000;
class Test {
float* _data;
public:
Test() { _data = new float[256]; }
Test(const Test& t) { _data = new float[256]; }
~Test() { delete[] _data; }
};
int main() {
vector<Test> v(N);
cout << "1st step" << endl;
sleep(5);
v.clear();
cout << "2nd step" << endl;
sleep(15);
return 0;
}
#include <iostream>
#include <vector>
using namespace std;
const unsigned int N = 150000;
class Test {
float* _data;
public:
Test() { _data = new float[256]; }
Test(const Test& t) { _data = new float[256]; }
~Test() { delete[] _data; }
};
int main() {
vector<Test> v;
for (uint i=0; i<N; i++) {
v.push_back(Test());
}
cout << "1st step" << endl;
sleep(5);
v.clear();
cout << "2nd step" << endl;
sleep(15);
return 0;
}
