www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - current GC behavior

reply luka8088 <luka8088 owave.net> writes:
Hello everyone,

I was writing some unit tests and I also wanted to test that in certain 
cases object references are properly removed everywhere so that GC can 
collect them in order to make sure there is no memory leak. While trying 
to achieve this I learned that objects are not always collected at the 
time I would expect them to, so I documented current behavior and some tips.


/**
  * Compiled with:
  *   DMD32 D Compiler v2.060
  *   Copyright (c) 1999-2012 by Digital Mars written by Walter Bright
  *   Documentation: http://www.dlang.org/index.html
  */

module program;

import std.stdio;
import std.string;
import core.memory;

void main () {

   writeln("// ---------- beginning of main");

   mixin(writeAndExec(4, q{

     GC.collect();
     new a("a1");
     // GC will collect a1
     GC.collect();

     auto a2 = new a("a2");
     // GC will collect a2 because a2 will not be used in the future
     GC.collect();

     auto a3 = new a("a3");
     // GC will not collect a3 because a3 will be used in the future
     // tip: make some interaction with the object if you don't want
     // it to get collected before the end of scope GC.collect();
     GC.collect();
     a3.i = 1;

     {
       auto a4 = new a("a4");
       // GC will collect a4 because a4 will not be used in the future
       GC.collect();
     }

     {
       auto a5 = new a("a5");
       a5.i = 1;
       // maybe GC could collect a5 because a5 will be used
       // in the future, but it has been used and for some
       // (probably implementation) reasons GC does not collect it
       GC.collect();
     }

     ({
       auto a6 = new a("a6");
       a6.i = 1;
       // same as previous GC.collect (for a5)
       GC.collect();
     })();

     // maybe GC could collect a5 because a5 has gone out of scope
     // but for some (probably implementation) reasons GC does not
     // collect it, however, it will collect a6 that was declared
     // and used in a function call
     // tip: use function call (like a6 case) for a scope if you
     // want GC.collect(); to collect objects declared inside it
     GC.collect();

   }));

   writeln("// ---------- end of main");

}

class a {

   string id;
   int i;

   this (string id) {
     this.id = id;
     writeln("// ! ", this.id, " is now constructed");
   }

   ~this () {
     writeln("// ! ", this.id, " is now destructed");
   }

}

string writeAndExec (uint ident, string code) {
   string result = "";
   foreach (line; code.splitLines())
     result ~= "writeln(`" ~ line[line.length > ident ? ident : 0 .. $] 
~ "`);\n" ~ line ~ "\n";
   return result;
}
Nov 06 2012
next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 11/06/2012 03:27 AM, luka8088 wrote:


 I was writing some unit tests and I also wanted to test that in certain
 cases object references are properly removed everywhere so that GC can
 collect them in order to make sure there is no memory leak. While trying
 to achieve this I learned that objects are not always collected at the
 time I would expect them to, so I documented current behavior and some
 tips.


Thanks for the analysis and the tips but they all depend on observations 
made by a particular test program on a particular version of a 
particular compiler. It is conceivable that even the same GC algorithm 
can behave differently in another program.

In fact, some objects may never be collected even if there are no more 
references to them.

In comparison, destroy() and scoped() do call the destructors at precise 
times.

Ali
Nov 06 2012
parent reply luka8088 <luka8088 owave.net> writes:
On 6.11.2012 18:00, Ali Çehreli wrote:

 On 11/06/2012 03:27 AM, luka8088 wrote:

  > I was writing some unit tests and I also wanted to test that in certain
  > cases object references are properly removed everywhere so that GC can
  > collect them in order to make sure there is no memory leak. While trying
  > to achieve this I learned that objects are not always collected at the
  > time I would expect them to, so I documented current behavior and some
  > tips.

 Thanks for the analysis and the tips but they all depend on observations
 made by a particular test program on a particular version of a
 particular compiler. It is conceivable that even the same GC algorithm
 can behave differently in another program.


Yes, but it seems that we can in general say that the following code 
will never fail... or am I wrong ?

import core.memory;

class a {
   static int totalRefCount = 0;
   this () { totalRefCount++; }
   ~this () { totalRefCount--; }
}

void main () {
   assert(a.totalRefCount == 0);
   ({
     auto a1 = new a();
     assert(a.totalRefCount == 1);
   })();
   GC.collect();
   assert(a.totalRefCount == 0);
}


 In fact, some objects may never be collected even if there are no more
 references to them.


Can you give me an example ?


 In comparison, destroy() and scoped() do call the destructors at precise
 times.


Yes, but the main reason for doing this is testing for memory leaks so 
this cases are not much of a concern =)


 Ali


Luka
Nov 06 2012
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 11/06/2012 12:00 PM, luka8088 wrote:


 Yes, but it seems that we can in general say that the following code
 will never fail... or am I wrong ?

 import core.memory;

 class a {
 static int totalRefCount = 0;
 this () { totalRefCount++; }
 ~this () { totalRefCount--; }
 }

 void main () {
 assert(a.totalRefCount == 0);
 ({
 auto a1 = new a();
 assert(a.totalRefCount == 1);
 })();
 GC.collect();


The current definition of GC.collect() explicitly says that its 
definition may change:

   http://dlang.org/phobos/core_memory.html#collect

"the meaning of this may change based on the garbage collector 
implementation".


 assert(a.totalRefCount == 0);
 }


 In fact, some objects may never be collected even if there are no more
 references to them.


 Can you give me an example ?


I can't come up with an example but the spec is clear on that issue:

   http://dlang.org/class.html#destructors

"The garbage collector is not guaranteed to run the destructor for all 
unreferenced objects. Furthermore, the order in which the garbage 
collector calls destructors for unreference objects is not specified. 
This means that when the garbage collector calls a destructor for an 
object of a class that has members that are references to garbage 
collected objects, those references may no longer be valid. This means 
that destructors cannot reference sub objects."

Ali
Nov 06 2012
next sibling parent luka8088 <luka8088 owave.net> writes:
On 6.11.2012 21:59, Ali Çehreli wrote:

 On 11/06/2012 12:00 PM, luka8088 wrote:

  > Yes, but it seems that we can in general say that the following code
  > will never fail... or am I wrong ?
  >
  > import core.memory;
  >
  > class a {
  > static int totalRefCount = 0;
  > this () { totalRefCount++; }
  > ~this () { totalRefCount--; }
  > }
  >
  > void main () {
  > assert(a.totalRefCount == 0);
  > ({
  > auto a1 = new a();
  > assert(a.totalRefCount == 1);
  > })();
  > GC.collect();

 The current definition of GC.collect() explicitly says that its
 definition may change:

 http://dlang.org/phobos/core_memory.html#collect

 "the meaning of this may change based on the garbage collector
 implementation".

  > assert(a.totalRefCount == 0);
  > }
  >
  >>
  >> In fact, some objects may never be collected even if there are no more
  >> references to them.
  >
  > Can you give me an example ?

 I can't come up with an example but the spec is clear on that issue:

 http://dlang.org/class.html#destructors

 "The garbage collector is not guaranteed to run the destructor for all
 unreferenced objects. Furthermore, the order in which the garbage
 collector calls destructors for unreference objects is not specified.
 This means that when the garbage collector calls a destructor for an
 object of a class that has members that are references to garbage
 collected objects, those references may no longer be valid. This means
 that destructors cannot reference sub objects."

 Ali


I see, so is there some alternative for memory leak tests ?

Luka
Nov 06 2012
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Nov 6, 2012, at 12:59 PM, Ali =C7ehreli <acehreli yahoo.com> wrote:

=20
 I can't come up with an example but the spec is clear on that issue:
=20
  http://dlang.org/class.html#destructors
=20
 "The garbage collector is not guaranteed to run the destructor for all =


unreferenced objects. Furthermore, the order in which the garbage =
collector calls destructors for unreference objects is not specified. =
This means that when the garbage collector calls a destructor for an =
object of a class that has members that are references to garbage =
collected objects, those references may no longer be valid. This means =
that destructors cannot reference sub objects."

This clause is mostly intended to say that the GC may not detect that =
every unreferenced object is truly unreferenced.  It's also possible, =
however, that the GC could delay finalizing collected objects until some =
later time.=
Nov 06 2012
prev sibling parent reply "thedeemon" <dlang thedeemon.com> writes:
On Tuesday, 6 November 2012 at 11:27:25 UTC, luka8088 wrote:

 Hello everyone,

 I was writing some unit tests and I also wanted to test that in 
 certain cases object references are properly removed everywhere 
 so that GC can collect them in order to make sure there is no 
 memory leak. While trying to achieve this I learned that 
 objects are not always collected at the time I would expect 
 them to, so I documented current behavior and some tips.


Those are some sample cases which all fit into the general rule: 
on collection the stack is scanned and everything that looks like 
a pointer to live heap is used to mark heap objects as live, then 
the heap objects are scanned and marked transitively. Unmarked 
objects get collected. In each case described in the original 
post the decision whether some value will be collected or not 
depends on whether the local variable is still on the stack and 
not overwritten by some newer stack variable. As a consequence:
If it's going to be used later, it's still live on stack.
If it's not going to be used later, it all depends on whether 
some other variable defined later is placed by compiler in the 
same stack slot, and whether that new variable was initialized 
already.
If it was used in a function called and returned, it's not in the 
active stack, so it should be collected. And so on.
Nov 06 2012
parent luka8088 <luka8088 owave.net> writes:
On 6.11.2012 18:02, thedeemon wrote:

 On Tuesday, 6 November 2012 at 11:27:25 UTC, luka8088 wrote:

 Hello everyone,

 I was writing some unit tests and I also wanted to test that in
 certain cases object references are properly removed everywhere so
 that GC can collect them in order to make sure there is no memory
 leak. While trying to achieve this I learned that objects are not
 always collected at the time I would expect them to, so I documented
 current behavior and some tips.


 Those are some sample cases which all fit into the general rule: on
 collection the stack is scanned and everything that looks like a pointer
 to live heap is used to mark heap objects as live, then the heap objects
 are scanned and marked transitively. Unmarked objects get collected. In
 each case described in the original post the decision whether some value
 will be collected or not depends on whether the local variable is still
 on the stack and not overwritten by some newer stack variable. As a
 consequence:
 If it's going to be used later, it's still live on stack.
 If it's not going to be used later, it all depends on whether some other
 variable defined later is placed by compiler in the same stack slot, and
 whether that new variable was initialized already.
 If it was used in a function called and returned, it's not in the active
 stack, so it should be collected. And so on.


Thx for the more detailed implementation explanation =)
Nov 06 2012