www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Wrestling the Garbage Collector.. And loosing..

reply "Simen Haugen" <simen norstat.no> writes:
Hi.

I'm having a problem with my application using extremly much memory, and the 
only way I'm able to free it is by deleting objects explicitly. I guess I'm 
not understanding the GC right, but I thought that it should free all 
objects without references to it when it's collecting, but my testing below 
shows I'm doing something wrong.

// Test 1
auto str = new char[512];
delete str;
str = new char[512]; // OK. No new allocation

// Test 2
auto str = new char[512];
str = new char[512]; // New memory allocated. GC isn't run. Guess it's 
expected behavior.

// Test 3
auto str = new char[512];
str = null;
str = new char[512]; // New memory allocated. GC isn't run. Guess it's 
expected behavior.

// Test 4
auto str = new char[512];
str = null;
GC.collect;
str = new char[512]; // New memory allocated. Why?

// Test 5
auto str = new char[512];
str.length = 0;
GC.collect;
str = new char[512]; // New memory allocated. Why?

I would have thought Test 4 and 5 shouldn't allocate more memory either as I 
remove the reference (or zero length) and run the GC, but it does.
I would also think Test 2 and 3 should make the GC reclaim memory when it is 
run.

The same applies for classes too, but there I need to explicitly delete all 
members in the destructor also..

And what does GC.minimize actually do? Even if I free an object and run 
minimize, the application still uses the same amout of memory (or at least 
it looks like it)

This is tested with dmd 1.028, tango ca. 0.99.5 (used trunk from back then) 
and win xp.

At this point my program is basically useless because of this. You can run 
it and restart it before the next run, but this limits its use very much. I 
can fix it by explicity deleting objects when I don't need them, but I 
really hope I don't have to.
I'm already using scope everywhere I can, and my tests with those shows it 
helps a lot, but the application will still require a good deal of memory 
and I need to limit this as much as I can.

I really hope someone could enlighten me on the correct usage of the GC 
(I've read the garbage collector documentation and memory article on 
digitalmars and the wikipedia article without getting much smarter)
Apr 30 2008
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Simen Haugen" wrote
 Hi.

 I'm having a problem with my application using extremly much memory, and 
 the only way I'm able to free it is by deleting objects explicitly. I 
 guess I'm not understanding the GC right, but I thought that it should 
 free all objects without references to it when it's collecting, but my 
 testing below shows I'm doing something wrong.

 // Test 1
 auto str = new char[512];
 delete str;
 str = new char[512]; // OK. No new allocation

 // Test 2
 auto str = new char[512];
 str = new char[512]; // New memory allocated. GC isn't run. Guess it's 
 expected behavior.

 // Test 3
 auto str = new char[512];
 str = null;
 str = new char[512]; // New memory allocated. GC isn't run. Guess it's 
 expected behavior.

 // Test 4
 auto str = new char[512];
 str = null;
 GC.collect;
 str = new char[512]; // New memory allocated. Why?

 // Test 5
 auto str = new char[512];
 str.length = 0;
 GC.collect;
 str = new char[512]; // New memory allocated. Why?

 I would have thought Test 4 and 5 shouldn't allocate more memory either as 
 I remove the reference (or zero length) and run the GC, but it does.
 I would also think Test 2 and 3 should make the GC reclaim memory when it 
 is run.

The GC is based on pointers. So if a thread stack has pointers to any GC data, it will not collect that data. In test 5, the str array has length 0, but the array ptr is still a pointer pointing to the beginning of that allocated data. I would not expect Test 5 to collect the original data. In Test 4, this one may be due to the pointer still being in the registers. You might want to try executing some other functions or statements before calling GC.collect, but even then, it might not clear out the particular register that held the pointer. There are other causes to the GC not reclaiming memory. The GC only allocates memory based on bin sizes, in 16, 32, ..., 4096+ sizes. So if you allocate just over one of those sizes, then you are still allocating the whole bin size. Do this alot, and you will see a huge waste of memory. This is a tradeoff of minimal memory size vs. efficiency to find an empty memory location. Another cause of lost memory is appending to an array. The GC never releases pages back to the OS (this might have been fixed in Tango). If you append a byte to an array over and over again, for instance, what happens? // array.length == 16 array ~= 'a'; This needs to move array into another bin, but what happens to the original bin? It's marked as free, but the memory is still held by the GC for the next <=16 byte allocation. What ends up happening is as you append more and more data, the number of pages allocated grows a lot. Even though this memory is marked as 'free', it's still being used by your application. -Steve
Apr 30 2008