www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Garbage Collection, Allocators/Deallocators and

reply Ivo Kasiuk <i.kasiuk gmx.de> writes:
Hi,

to improve my understanding of the GC and when/how
allocators/deallocators and constructors/destructors get called, I wrote
a little test program. And now I understand even less than before...
Here is the program:

------------------------------------
import core.memory;
import std.stdio;

struct S1 {
  ubyte[1_000_000] buf;
  new(size_t size) {
    void* ptr =3D GC.malloc(size);
    writefln("S1.new(%d) =3D %x", size, ptr);
    return ptr;
  }
  delete(void* ptr) {
    writefln("S1.delete %x", ptr);
    if (ptr) GC.free(ptr);
  }
  this(int x) { writefln("S1(%d)", x); }
  ~this() { writeln("~S1()"); }
}

struct S2 {
  ubyte[1_000_000] buf;
  this(int x) { writefln("S2(%d)", x); }
  ~this() { writeln("~S2()"); }
}

class C1 {
  ubyte[1_000_000] buf;
  new(size_t size) {
    void* ptr =3D GC.malloc(size);
    writefln("C1.new(%d) =3D %x", size, ptr);
   return ptr;
  }
  delete(void* ptr) {
    writefln("C1.delete %x", ptr);
    if (ptr) GC.free(ptr);
  }
  this() { writeln("C1()"); }
  ~this() { writeln("~C1()"); }
}

class C2 {
  ubyte[1_000_000] buf;
  this() { writeln("C2()"); }
  ~this() { writeln("~C2()"); }
}

void f() {
  for (int i=3D0; i<10_000; i++) {
    S1* s1 =3D new S1(i);
    S2* s2 =3D new S2(i);
    C1 c1 =3D new C1;
    C2 c2 =3D new C2;
  }
}

void main() {
  f();
  GC.collect();
}
-----------------------------------------

Running this with DMD 2.049, I observed the following:

- S1(int), S2(int), C1(), C2(), S1.new and S2.new get invoked in every
cycle of the loop, as you would expect.

- ~C2() also gets invoked frequently.

- ~S1(), ~S2(), ~C1(), S1.delete and C1.delete never ever get called.

- The program does not run out of memory and actually has a relatively
modest memory footprint, so it does not seem to leak memory. (When using
std.c.stdlib.malloc instead of GC.malloc it runs out of memory almost
immediately).

It is good to see that the GC apparently really frees the unreferenced
memory again. However, I don't understand why the deallocators and the
destructors (apart from ~C2) do not get called. If the memory for the
objects gets freed, as is apparently the case, then why are there no
destructor and deallocator calls for these objects?

Thanks
Ivo Kasiuk
Sep 18 2010
parent reply Sean Kelly <sean invisibleduck.org> writes:
Ivo Kasiuk Wrote:

 Hi,
 
 to improve my understanding of the GC and when/how
 allocators/deallocators and constructors/destructors get called, I wrote
 a little test program. And now I understand even less than before...
...
 Running this with DMD 2.049, I observed the following:
 
 - S1(int), S2(int), C1(), C2(), S1.new and S2.new get invoked in every
 cycle of the loop, as you would expect.
 
 - ~C2() also gets invoked frequently.
 
 - ~S1(), ~S2(), ~C1(), S1.delete and C1.delete never ever get called.
 
 - The program does not run out of memory and actually has a relatively
 modest memory footprint, so it does not seem to leak memory. (When using
 std.c.stdlib.malloc instead of GC.malloc it runs out of memory almost
 immediately).
 
 It is good to see that the GC apparently really frees the unreferenced
 memory again. However, I don't understand why the deallocators and the
 destructors (apart from ~C2) do not get called. If the memory for the
 objects gets freed, as is apparently the case, then why are there no
 destructor and deallocator calls for these objects?
The deallocator is only called if you delete the object, not when it's finalized by the GC. The GC will only finalize something that is in its memory space, so if this happens there's no need to call the deallocator.
Sep 18 2010
parent reply Ivo Kasiuk <i.kasiuk gmx.de> writes:
Am Samstag, den 18.09.2010, 10:08 -0400 schrieb Sean Kelly:
 Ivo Kasiuk Wrote:
=20
 Hi,
=20
 to improve my understanding of the GC and when/how
 allocators/deallocators and constructors/destructors get called, I wrot=
e
 a little test program. And now I understand even less than before...
...
 Running this with DMD 2.049, I observed the following:
=20
 - S1(int), S2(int), C1(), C2(), S1.new and S2.new get invoked in every
 cycle of the loop, as you would expect.
=20
 - ~C2() also gets invoked frequently.
=20
 - ~S1(), ~S2(), ~C1(), S1.delete and C1.delete never ever get called.
=20
 - The program does not run out of memory and actually has a relatively
 modest memory footprint, so it does not seem to leak memory. (When usin=
g
 std.c.stdlib.malloc instead of GC.malloc it runs out of memory almost
 immediately).
=20
 It is good to see that the GC apparently really frees the unreferenced
 memory again. However, I don't understand why the deallocators and the
 destructors (apart from ~C2) do not get called. If the memory for the
 objects gets freed, as is apparently the case, then why are there no
 destructor and deallocator calls for these objects?
=20 The deallocator is only called if you delete the object, not when it's fi=
nalized by the GC. The GC will only finalize something that is in its memo= ry space, so if this happens there's no need to call the deallocator. Ok, that makes sense. So the deallocators really should not get called in this case. But why are the destructors not invoked when the GC finalizes the objects? Exploring the example a bit further: If C's malloc is used instead of GC.malloc then the deallocators also are not called and the program runs out of memory. How are the objects supposed to get finalized in this case - do I have to use the delete keyword explicitly? An interesting case is when using C's malloc for C1 and using the scope attribute for c1: class C1 { ubyte[1_000_000] buf; new(size_t size) { void* ptr =3D std.c.stdlib.malloc(size); if (ptr is null) throw new OutOfMemoryError(__FILE__, __LINE__); writefln("C1.new(%d) =3D %x", size, ptr); return ptr; } delete(void* ptr) { writefln("C1.delete %x", ptr); if (ptr) std.c.stdlib.free(ptr); } this() { writeln("C1()"); } ~this() { writeln("~C1()"); } } ... scope C1 c1 =3D new C1; In this case, ~C1 gets invoked but not C1.delete. Nevertheless, the memory appears to get freed (normal memory consumption, no OutOfMemory). How does this happen?
Sep 18 2010
next sibling parent Ivo Kasiuk <i.kasiuk gmx.de> writes:
 An interesting case is when using C's malloc for C1 and using the scope
 attribute for c1:
=20
 class C1 {
   ubyte[1_000_000] buf;
   new(size_t size) {
     void* ptr =3D std.c.stdlib.malloc(size);
     if (ptr is null)
       throw new OutOfMemoryError(__FILE__, __LINE__);
     writefln("C1.new(%d) =3D %x", size, ptr);
    return ptr;
   }
   delete(void* ptr) {
     writefln("C1.delete %x", ptr);
     if (ptr) std.c.stdlib.free(ptr);
   }
   this() { writeln("C1()"); }
   ~this() { writeln("~C1()"); }
 }
 ...
   scope C1 c1 =3D new C1;
=20
 In this case, ~C1 gets invoked but not C1.delete. Nevertheless, the
 memory appears to get freed (normal memory consumption, no OutOfMemory).
 How does this happen?
Ok, I figured that out myself: c1 is allocated on the stack in this case, so no neither the allocator nor the deallocator need to get called.
Sep 18 2010
prev sibling parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Ivo Kasiuk <i.kasiuk gmx.de> wrote:

 Ok, that makes sense. So the deallocators really should not get called
 in this case. But why are the destructors not invoked when the GC
 finalizes the objects?
For S1 and S2, this is a known bug - destructors of structs on the heap don't get called. As for C1, I have no idea what's happening. Definitely a bug, though.
 Exploring the example a bit further:
 If C's malloc is used instead of GC.malloc then the deallocators also
 are not called and the program runs out of memory. How are the objects
 supposed to get finalized in this case - do I have to use the delete
 keyword explicitly?
If you use C's malloc, you will also have to use C's free. -- Simen
Sep 18 2010
next sibling parent reply Ivo Kasiuk <i.kasiuk gmx.de> writes:
 Exploring the example a bit further:
 If C's malloc is used instead of GC.malloc then the deallocators also
 are not called and the program runs out of memory. How are the objects
 supposed to get finalized in this case - do I have to use the delete
 keyword explicitly?
=20 If you use C's malloc, you will also have to use C's free.
Yes, obviously. But the deallocator which contains the call to C's free has to be invoked by someone, otherwise it's useless. So that is my question: how should finalization and deallocation get triggered in this scenario (where non-GC memory is used)?
Sep 18 2010
parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Ivo Kasiuk <i.kasiuk gmx.de> wrote:

 Exploring the example a bit further:
 If C's malloc is used instead of GC.malloc then the deallocators also
 are not called and the program runs out of memory. How are the objects
 supposed to get finalized in this case - do I have to use the delete
 keyword explicitly?
If you use C's malloc, you will also have to use C's free.
Yes, obviously. But the deallocator which contains the call to C's free has to be invoked by someone, otherwise it's useless. So that is my question: how should finalization and deallocation get triggered in this scenario (where non-GC memory is used)?
delete will, as long as it is kept in D, call the deallocator method. Unless the allocated memory is not registered with the garbage collector, it will be ignored by the garbage collector, and you will have to manually delete allocated objects. The point of using C's malloc and free is exactly that - the GC will ignore your objects, and you are free to (that is, have to) manage their lifetimes yourself. -- Simen
Sep 18 2010
parent reply Ivo Kasiuk <i.kasiuk gmx.de> writes:
 Exploring the example a bit further:
 If C's malloc is used instead of GC.malloc then the deallocators als=
o
 are not called and the program runs out of memory. How are the objec=
ts
 supposed to get finalized in this case - do I have to use the delete
 keyword explicitly?
If you use C's malloc, you will also have to use C's free.
Yes, obviously. But the deallocator which contains the call to C's free has to be invoked by someone, otherwise it's useless. So that is my question: how should finalization and deallocation get triggered in thi=
s
 scenario (where non-GC memory is used)?
=20 delete will, as long as it is kept in D, call the deallocator method. Unless the allocated memory is not registered with the garbage collector, it will be ignored by the garbage collector, and you will have to manuall=
y
 delete allocated objects. The point of using C's malloc and free is
 exactly that - the GC will ignore your objects, and you are free to (that
 is, have to) manage their lifetimes yourself.
=20
Thanks for the explanation. Summing up what I have learned now: - When GC-allocated memory is used the deallocator is not called when the GC automatically finalizes the object, but only when the delete keyword is used explicitly. - When non-GC memory is used the deallocator is also only called when the delete keyword is used, and deallocation/finalization is never triggered automatically. In other words: the deallocator only ever gets called when the delete keyword is used. What does this mean if the delete keyword is not kept in D: - Will deallocators also be obsolete then? - Is there any way the regular finalization sequence (i.e. including automatic invoking of the destructors) can be run for an object that uses non-GC memory?
Sep 19 2010
parent Sean Kelly <sean invisibleduck.org> writes:
Ivo Kasiuk Wrote:
 
 - Is there any way the regular finalization sequence (i.e. including
 automatic invoking of the destructors) can be run for an object that
 uses non-GC memory?
It should happen when delete is called. Here's the code executed for delete: http://dsource.org/projects/druntime/browser/trunk/src/rt/lifetime.d#L167 _d_delclass
Sep 19 2010
prev sibling next sibling parent Ivo Kasiuk <i.kasiuk gmx.de> writes:
 Ok, that makes sense. So the deallocators really should not get called
 in this case. But why are the destructors not invoked when the GC
 finalizes the objects?
=20 For S1 and S2, this is a known bug - destructors of structs on the heap don't get called.
 As for C1, I have no idea what's happening. Definitely a bug, though.
I am not sure that it is a bug that ~C1 (and also ~S1) is not invoked. After all, we called GC.malloc manually. So the GC knows about the memory region but is possibly not aware that it contains the object. If this is true it would be logical that the memory is freed but the destructor is not called.
Sep 19 2010
prev sibling parent Ivo Kasiuk <i.kasiuk gmx.de> writes:
Simen kjaeraas wrote:
 Ivo Kasiuk <i.kasiuk gmx.de> wrote:
=20
 Ok, that makes sense. So the deallocators really should not get called
 in this case. But why are the destructors not invoked when the GC
 finalizes the objects?
=20 For S1 and S2, this is a known bug - destructors of structs on the heap don't get called. =20 As for C1, I have no idea what's happening. Definitely a bug, though.
Actually it is not a bug but a problem with the implementation: class C1 { ubyte[1_000_000] buf; new(size_t size) { void* ptr =3D GC.malloc(size); writefln("C1.new(%d) =3D %x", size, ptr); return ptr; } this() { writeln("C1()"); } ~this() { writeln("~C1()"); } } GC.malloc is defined as follows: static void* malloc(size_t sz, uint ba =3D 0); So by calling it without a second argument, I implicitly specified bitmask 0. The problem with this is that one of these bits is GC.BlkAttr.FINALIZE. As this is not set, the GC does not finalise the data contained in the memory block. The following correction is necessary: ... void* ptr =3D GC.malloc(size, GC.BlkAttr.FINALIZE); ... Then ~C1() gets called as expected. I think that in theory this should also work for structs. But currently the same code with a struct instead of a class results in a segfault. I guess I will have to try again when the struct finalisation bug has been fixed. Ivo
Oct 15 2010