www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Weak References

reply PJP <pete.poulos gmail.com> writes:
Is there any plan to add support for weak references to the D garbage collector?

Weak references are references which don't prevent the object from being
garbage collected.  If it has been garbage collected then the weak reference
returns null.

This is useful for things like resource caches, maintaining references to all
instances of a specific class (ex: to implement an operation such as
Window.minimizeAllWindows()), or to register a callback/listener with an object
such that the registration won't prevent that object from being garbage
collected.
Aug 06 2008
next sibling parent reply Jason House <jason.james.house gmail.com> writes:
A weak reference library exists but doesn't always work...

PJP Wrote:


 Is there any plan to add support for weak references to the D garbage
collector?
 
 Weak references are references which don't prevent the object from being
garbage collected.  If it has been garbage collected then the weak reference
returns null.
 
 This is useful for things like resource caches, maintaining references to all
instances of a specific class (ex: to implement an operation such as
Window.minimizeAllWindows()), or to register a callback/listener with an object
such that the registration won't prevent that object from being garbage
collected.
Aug 06 2008
parent reply "Bill Baxter" <wbaxter gmail.com> writes:
On Thu, Aug 7, 2008 at 8:12 AM, Jason House <jason.james.house gmail.com> wrote:

 A weak reference library exists but doesn't always work...


By doesn't always work you mean that it seems to you that sometimes it
prevents the target object from being collected, right?

But I think you said your only test case for this was something that
is not reproducible, correct?

Very frustrating to not be able to say for sure.  It would be a huge
help if the GC exposed some API for querying where a particular
pointer is being referenced.  Without that it's really hard to say
where the reference actually is coming from.

One possible gotcha that just crossed my mind:  I think the current GC
allocates chunks in an all or nothing manner -- hasPointers is just a
single bit.  So could it be that the fact that all Objects have a
vtable pointer mean that everything in an Object is always treated as
having pointers?

--bb


 PJP Wrote:


 Is there any plan to add support for weak references to the D garbage
collector?

 Weak references are references which don't prevent the object from being
garbage collected.  If it has been garbage collected then the weak reference
returns null.

 This is useful for things like resource caches, maintaining references to all
instances of a specific class (ex: to implement an operation such as
Window.minimizeAllWindows()), or to register a callback/listener with an object
such that the registration won't prevent that object from being garbage
collected.
Aug 06 2008
next sibling parent reply Jason House <jason.james.house gmail.com> writes:
Bill Baxter Wrote:


 On Thu, Aug 7, 2008 at 8:12 AM, Jason House <jason.james.house gmail.com>
wrote:

 A weak reference library exists but doesn't always work...


 
 By doesn't always work you mean that it seems to you that sometimes it
 prevents the target object from being collected, right?


It causes crashes under the right circumstances. I posted test cases to
reproduce the behavior when I was experimenting with it...





 
 But I think you said your only test case for this was something that
 is not reproducible, correct?
 
 Very frustrating to not be able to say for sure.  It would be a huge
 help if the GC exposed some API for querying where a particular
 pointer is being referenced.  Without that it's really hard to say
 where the reference actually is coming from.
 
 One possible gotcha that just crossed my mind:  I think the current GC
 allocates chunks in an all or nothing manner -- hasPointers is just a
 single bit.  So could it be that the fact that all Objects have a
 vtable pointer mean that everything in an Object is always treated as
 having pointers?
 
 --bb
 

 PJP Wrote:


 Is there any plan to add support for weak references to the D garbage
collector?

 Weak references are references which don't prevent the object from being
garbage collected.  If it has been garbage collected then the weak reference
returns null.

 This is useful for things like resource caches, maintaining references to all
instances of a specific class (ex: to implement an operation such as
Window.minimizeAllWindows()), or to register a callback/listener with an object
such that the registration won't prevent that object from being garbage
collected.
Aug 06 2008
parent PJP <pete.poulos gmail.com> writes:
Jason House Wrote:


 Bill Baxter Wrote:
 

 On Thu, Aug 7, 2008 at 8:12 AM, Jason House <jason.james.house gmail.com>
wrote:

 A weak reference library exists but doesn't always work...


 
 By doesn't always work you mean that it seems to you that sometimes it
 prevents the target object from being collected, right?


 
 It causes crashes under the right circumstances. I posted test cases to
reproduce the behavior when I was experimenting with it...
 
 
 
 

 
 But I think you said your only test case for this was something that
 is not reproducible, correct?
 
 Very frustrating to not be able to say for sure.  It would be a huge
 help if the GC exposed some API for querying where a particular
 pointer is being referenced.  Without that it's really hard to say
 where the reference actually is coming from.
 
 One possible gotcha that just crossed my mind:  I think the current GC
 allocates chunks in an all or nothing manner -- hasPointers is just a
 single bit.  So could it be that the fact that all Objects have a
 vtable pointer mean that everything in an Object is always treated as
 having pointers?
 
 --bb
 

 PJP Wrote:


 Is there any plan to add support for weak references to the D garbage
collector?

 Weak references are references which don't prevent the object from being
garbage collected.  If it has been garbage collected then the weak reference
returns null.

 This is useful for things like resource caches, maintaining references to all
instances of a specific class (ex: to implement an operation such as
Window.minimizeAllWindows()), or to register a callback/listener with an object
such that the registration won't prevent that object from being garbage
collected.
Aug 07 2008
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Bill Baxter" wrote

 On Thu, Aug 7, 2008 at 8:12 AM, Jason House wrote:

 A weak reference library exists but doesn't always work...


 By doesn't always work you mean that it seems to you that sometimes it
 prevents the target object from being collected, right?

 But I think you said your only test case for this was something that
 is not reproducible, correct?

 Very frustrating to not be able to say for sure.  It would be a huge
 help if the GC exposed some API for querying where a particular
 pointer is being referenced.  Without that it's really hard to say
 where the reference actually is coming from.

 One possible gotcha that just crossed my mind:  I think the current GC
 allocates chunks in an all or nothing manner -- hasPointers is just a
 single bit.  So could it be that the fact that all Objects have a
 vtable pointer mean that everything in an Object is always treated as
 having pointers?


At least on Tango, this is not the case:

import tango.core.Memory;

class X
{
    int x;
}

class Y
{
    int *y;
}

void main()
{
    auto n = new X;
    assert(GC.getAttr(cast(void*)n) & GC.BlkAttr.NO_SCAN);
    auto m = new Y;
    assert(!(GC.getAttr(cast(void*)m) & GC.BlkAttr.NO_SCAN));
}

-Steve
Aug 07 2008
parent reply "Bill Baxter" <wbaxter gmail.com> writes:
On Thu, Aug 7, 2008 at 11:38 PM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:

 "Bill Baxter" wrote

 On Thu, Aug 7, 2008 at 8:12 AM, Jason House wrote:

 One possible gotcha that just crossed my mind:  I think the current GC
 allocates chunks in an all or nothing manner -- hasPointers is just a
 single bit.  So could it be that the fact that all Objects have a
 vtable pointer mean that everything in an Object is always treated as
 having pointers?


 At least on Tango, this is not the case:

 import tango.core.Memory;

 class X
 {
    int x;
 }

 class Y
 {
    int *y;
 }

 void main()
 {
    auto n = new X;
    assert(GC.getAttr(cast(void*)n) & GC.BlkAttr.NO_SCAN);
    auto m = new Y;
    assert(!(GC.getAttr(cast(void*)m) & GC.BlkAttr.NO_SCAN));
 }


Hmm, yeh I also went looking for the previous discussion on this topic
and found there that someone had already checked that the GC flag on
the WeakRef class was getting set properly.

The only other potential problem I can see is the manipulation of the
member variable that goes on in the destructor of the WeakRef.  It
makes the assumption that if the referred object has been collected
then the notification function will have been called, and thus the
pointer-as-size_t variable will be null.  But that code is very
similar to what std.signals does in its destructor.  So if that's the
source of the bug then it's probably a bug that's in std.signal too.

--bb
Aug 07 2008
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Bill Baxter" wrote

 On Thu, Aug 7, 2008 at 11:38 PM, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:

 "Bill Baxter" wrote

 On Thu, Aug 7, 2008 at 8:12 AM, Jason House wrote:

 One possible gotcha that just crossed my mind:  I think the current GC
 allocates chunks in an all or nothing manner -- hasPointers is just a
 single bit.  So could it be that the fact that all Objects have a
 vtable pointer mean that everything in an Object is always treated as
 having pointers?


 At least on Tango, this is not the case:

 import tango.core.Memory;

 class X
 {
    int x;
 }

 class Y
 {
    int *y;
 }

 void main()
 {
    auto n = new X;
    assert(GC.getAttr(cast(void*)n) & GC.BlkAttr.NO_SCAN);
    auto m = new Y;
    assert(!(GC.getAttr(cast(void*)m) & GC.BlkAttr.NO_SCAN));
 }


 Hmm, yeh I also went looking for the previous discussion on this topic
 and found there that someone had already checked that the GC flag on
 the WeakRef class was getting set properly.

 The only other potential problem I can see is the manipulation of the
 member variable that goes on in the destructor of the WeakRef.  It
 makes the assumption that if the referred object has been collected
 then the notification function will have been called, and thus the
 pointer-as-size_t variable will be null.  But that code is very
 similar to what std.signals does in its destructor.  So if that's the
 source of the bug then it's probably a bug that's in std.signal too.


I don't think that's possible.  In order for this to cause a problem, the GC 
would have to collect the memory from the referred object before calling 
your hook, or else that it doesn't call your hook at all.  Neither of these 
seem likely.

Looking through your code with thread-oriented eyes, and assuming that any 
thread can cause a collect cycle at any time, it seems that it is impossible 
for a race to occur.

-Steve
Aug 07 2008
prev sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from Bill Baxter (wbaxter gmail.com)'s article

 On Thu, Aug 7, 2008 at 11:38 PM, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:

 "Bill Baxter" wrote

 On Thu, Aug 7, 2008 at 8:12 AM, Jason House wrote:

 One possible gotcha that just crossed my mind:  I think the current GC
 allocates chunks in an all or nothing manner -- hasPointers is just a
 single bit.  So could it be that the fact that all Objects have a
 vtable pointer mean that everything in an Object is always treated as
 having pointers?


 At least on Tango, this is not the case:

 import tango.core.Memory;

 class X
 {
    int x;
 }

 class Y
 {
    int *y;
 }

 void main()
 {
    auto n = new X;
    assert(GC.getAttr(cast(void*)n) & GC.BlkAttr.NO_SCAN);
    auto m = new Y;
    assert(!(GC.getAttr(cast(void*)m) & GC.BlkAttr.NO_SCAN));
 }


 Hmm, yeh I also went looking for the previous discussion on this topic
 and found there that someone had already checked that the GC flag on
 the WeakRef class was getting set properly.
 The only other potential problem I can see is the manipulation of the
 member variable that goes on in the destructor of the WeakRef.  It
 makes the assumption that if the referred object has been collected
 then the notification function will have been called, and thus the
 pointer-as-size_t variable will be null.  But that code is very
 similar to what std.signals does in its destructor.  So if that's the
 source of the bug then it's probably a bug that's in std.signal too.


The last time I tested the WeakRef code the unittest was simply broken
in that it tried something like this:

    WeakRef!(int) r = new int;
    GC.collect();
    assert( r.get() is null );

Problem being that a register still may have a reference to the int value
that the test expected to be collected.  The easiest way around this is to
perform a second "dummy" allocation in hopes that the relevant registers
will be overwritten.  Once I made this change in the WeakRef code the
unittest passed for Tango.


Sean
Aug 07 2008
next sibling parent "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Sean Kelly" <sean invisibleduck.org> wrote in message 
news:g7fv4t$2bce$1 digitalmars.com...


 The last time I tested the WeakRef code the unittest was simply broken
 in that it tried something like this:

    WeakRef!(int) r = new int;
    GC.collect();
    assert( r.get() is null );

 Problem being that a register still may have a reference to the int value
 that the test expected to be collected.  The easiest way around this is to
 perform a second "dummy" allocation in hopes that the relevant registers
 will be overwritten.  Once I made this change in the WeakRef code the
 unittest passed for Tango.


For you.  For me, it still fails.
Aug 07 2008
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Sean Kelly" wrote

 == Quote from Bill Baxter (wbaxter gmail.com)'s article

 On Thu, Aug 7, 2008 at 11:38 PM, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:

 "Bill Baxter" wrote

 On Thu, Aug 7, 2008 at 8:12 AM, Jason House wrote:

 One possible gotcha that just crossed my mind:  I think the current GC
 allocates chunks in an all or nothing manner -- hasPointers is just a
 single bit.  So could it be that the fact that all Objects have a
 vtable pointer mean that everything in an Object is always treated as
 having pointers?


 At least on Tango, this is not the case:

 import tango.core.Memory;

 class X
 {
    int x;
 }

 class Y
 {
    int *y;
 }

 void main()
 {
    auto n = new X;
    assert(GC.getAttr(cast(void*)n) & GC.BlkAttr.NO_SCAN);
    auto m = new Y;
    assert(!(GC.getAttr(cast(void*)m) & GC.BlkAttr.NO_SCAN));
 }


 Hmm, yeh I also went looking for the previous discussion on this topic
 and found there that someone had already checked that the GC flag on
 the WeakRef class was getting set properly.
 The only other potential problem I can see is the manipulation of the
 member variable that goes on in the destructor of the WeakRef.  It
 makes the assumption that if the referred object has been collected
 then the notification function will have been called, and thus the
 pointer-as-size_t variable will be null.  But that code is very
 similar to what std.signals does in its destructor.  So if that's the
 source of the bug then it's probably a bug that's in std.signal too.


 The last time I tested the WeakRef code the unittest was simply broken
 in that it tried something like this:

    WeakRef!(int) r = new int;


This is probably just a typo, but the WeakRef class can only point to an 
Object derivative, because it adds an extra monitor code in a hidden field, 
which wouldn't be on an int.


    GC.collect();
    assert( r.get() is null );

 Problem being that a register still may have a reference to the int value
 that the test expected to be collected.  The easiest way around this is to
 perform a second "dummy" allocation in hopes that the relevant registers
 will be overwritten.  Once I made this change in the WeakRef code the
 unittest passed for Tango.


I don't see this as an issue, except that the unittest needs updating.  The 
object still functions as designed.  Jarrett has some other issue (which he 
has yet to share :) )

I'd like to see this get solved, as I can see a benefit to having a WeakRef 
class in Tango.

-Steve
Aug 07 2008
parent "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in message 
news:g7g9if$10vt$1 digitalmars.com...


 I don't see this as an issue, except that the unittest needs updating. 
 The object still functions as designed.  Jarrett has some other issue 
 (which he has yet to share :) )


http://dsource.org/projects/tango/ticket/1046
Aug 08 2008
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
Bill,

As an aside, I notice you have a dup() function in WeakRef.  When would you 
ever want to dup the WeakRef?  If anything, dup() should just return 'this'.

-Steve
Aug 07 2008
parent "Bill Baxter" <wbaxter gmail.com> writes:
On Fri, Aug 8, 2008 at 11:09 AM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:

 Bill,

 As an aside, I notice you have a dup() function in WeakRef.  When would you
 ever want to dup the WeakRef?  If anything, dup() should just return 'this'.


Hmm, yeh.  You're right.  It doesn't serve any useful purpose there.

--bb
Aug 07 2008
prev sibling parent reply Frank Benoit <keinfarbton googlemail.com> writes:
PJP schrieb:

 Is there any plan to add support for weak references to the D garbage
 collector?
 
 Weak references are references which don't prevent the object from
 being garbage collected.  If it has been garbage collected then the
 weak reference returns null.
 
 This is useful for things like resource caches, maintaining
 references to all instances of a specific class (ex: to implement an
 operation such as Window.minimizeAllWindows()), or to register a
 callback/listener with an object such that the registration won't
 prevent that object from being garbage collected.


And please, put a WeakHashMap on the TODO list too :)
http://java.sun.com/j2se/1.4.2/docs/api/java/util/WeakHashMap.html
Aug 06 2008
parent Jason House <jason.james.house gmail.com> writes:
Frank Benoit Wrote:
 

 And please, put a WeakHashMap on the TODO list too :)
 http://java.sun.com/j2se/1.4.2/docs/api/java/util/WeakHashMap.html


I wrote one of those, but it does not work. When investigating, I found issues
with the basic weak reference.
Aug 06 2008