www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Weak references.

reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
I've been looking at some of my old nonagon code, and as I'm sure you've all 
experienced looking at your old code, I'm just _cringing_ at some of the 
things I did in it ;)

One thing that I've noticed would be nice to have, however, would be weak 
references to some objects.  For example, I have "brush" objects which 
encapsulate a bunch of visual settings (textures, material, blending modes 
etc.).  These are referenced by objects that can be rendered.  Some of these 
brushes need to be updated often (i.e. every frame to update the environment 
mapping matrices).  So I keep a list (table) of all instances of this brush 
class so I can easily loop through them and update them all.

However when their owners disappear, I end up with uncollectable brushes, 
since they are still being referenced by that table.

So what would be nice to have happen would be to have that table be a table 
of _weak references_ to brush objects.  Then, when no more real references 
existed, they could be collected, and I wouldn't be keeping them around for 
no reason anymore.

I've looked into the scrapple weakref implementation, but it doesn't seem to 
do what I want.  Seemingly it only works if the object that it references is 
explicitly deleted.  I want the weakref to become null if the object that it 
references is collected in any way.

Is this possible at all?  Am I just not using Bill's weakref properly? 
Apr 12 2008
next sibling parent Jason House <jason.james.house gmail.com> writes:
Take another look... Weak references are not viewed as references. This means
the objects will get collected when only weak references exist. It sounds like
that is what you want...

Jarrett Billingsley Wrote:

 I've been looking at some of my old nonagon code, and as I'm sure you've all 
 experienced looking at your old code, I'm just _cringing_ at some of the 
 things I did in it ;)
 
 One thing that I've noticed would be nice to have, however, would be weak 
 references to some objects.  For example, I have "brush" objects which 
 encapsulate a bunch of visual settings (textures, material, blending modes 
 etc.).  These are referenced by objects that can be rendered.  Some of these 
 brushes need to be updated often (i.e. every frame to update the environment 
 mapping matrices).  So I keep a list (table) of all instances of this brush 
 class so I can easily loop through them and update them all.
 
 However when their owners disappear, I end up with uncollectable brushes, 
 since they are still being referenced by that table.
 
 So what would be nice to have happen would be to have that table be a table 
 of _weak references_ to brush objects.  Then, when no more real references 
 existed, they could be collected, and I wouldn't be keeping them around for 
 no reason anymore.
 
 I've looked into the scrapple weakref implementation, but it doesn't seem to 
 do what I want.  Seemingly it only works if the object that it references is 
 explicitly deleted.  I want the weakref to become null if the object that it 
 references is collected in any way.
 
 Is this possible at all?  Am I just not using Bill's weakref properly? 
 
 

Apr 12 2008
prev sibling next sibling parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Jarrett Billingsley wrote:
 I've been looking at some of my old nonagon code, and as I'm sure you've all 
 experienced looking at your old code, I'm just _cringing_ at some of the 
 things I did in it ;)
 
 One thing that I've noticed would be nice to have, however, would be weak 
 references to some objects.  For example, I have "brush" objects which 
 encapsulate a bunch of visual settings (textures, material, blending modes 
 etc.).  These are referenced by objects that can be rendered.  Some of these 
 brushes need to be updated often (i.e. every frame to update the environment 
 mapping matrices).  So I keep a list (table) of all instances of this brush 
 class so I can easily loop through them and update them all.
 
 However when their owners disappear, I end up with uncollectable brushes, 
 since they are still being referenced by that table.
 
 So what would be nice to have happen would be to have that table be a table 
 of _weak references_ to brush objects.  Then, when no more real references 
 existed, they could be collected, and I wouldn't be keeping them around for 
 no reason anymore.
 
 I've looked into the scrapple weakref implementation, but it doesn't seem to 
 do what I want.  Seemingly it only works if the object that it references is 
 explicitly deleted.  I want the weakref to become null if the object that it 
 references is collected in any way.
 
 Is this possible at all?  Am I just not using Bill's weakref properly? 
 
 

It does exactly what you're asking for ... as long as your "Brushes" are class instances, which it sounds like they are. Instead of filling the table with type Brush, fill it with type WeakRef!(Brush). Then when you go to use them from the table you should check for null first. If the ptr value is null, it means the object got collected at some point, so you can remove it from the table. --bb
Apr 12 2008
prev sibling next sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Jarrett Billingsley" <kb3ctd2 yahoo.com> wrote in message 
news:ftr9as$12s2$1 digitalmars.com...
 I've been looking at some of my old nonagon code, and as I'm sure you've 
 all experienced looking at your old code, I'm just _cringing_ at some of 
 the things I did in it ;)

 One thing that I've noticed would be nice to have, however, would be weak 
 references to some objects.  For example, I have "brush" objects which 
 encapsulate a bunch of visual settings (textures, material, blending modes 
 etc.).  These are referenced by objects that can be rendered.  Some of 
 these brushes need to be updated often (i.e. every frame to update the 
 environment mapping matrices).  So I keep a list (table) of all instances 
 of this brush class so I can easily loop through them and update them all.

 However when their owners disappear, I end up with uncollectable brushes, 
 since they are still being referenced by that table.

 So what would be nice to have happen would be to have that table be a 
 table of _weak references_ to brush objects.  Then, when no more real 
 references existed, they could be collected, and I wouldn't be keeping 
 them around for no reason anymore.

 I've looked into the scrapple weakref implementation, but it doesn't seem 
 to do what I want.  Seemingly it only works if the object that it 
 references is explicitly deleted.  I want the weakref to become null if 
 the object that it references is collected in any way.

 Is this possible at all?  Am I just not using Bill's weakref properly?

To both: yes, that's what I would have expected, but I can't seem to get the object to be collected even if the only reference to it is a weak reference. Tried forcing a full collect, allocating bunches of memory, collecting again, allocating at the beginning of a long-running program etc. it never seems to get collected, even if I never assign the reference into an actual pointer type. I'm using Tango if it makes any difference.
Apr 12 2008
next sibling parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Jarrett Billingsley wrote:
 To both: yes, that's what I would have expected, but I can't seem to get the 
 object to be collected even if the only reference to it is a weak reference. 
 Tried forcing a full collect, allocating bunches of memory, collecting 
 again, allocating at the beginning of a long-running program etc.  it never 
 seems to get collected, even if I never assign the reference into an actual 
 pointer type.
 
 I'm using Tango if it makes any difference. 

Have you tried looking at the generated assembler code? Perhaps the compiler is leaving a reference in some register that's never overwritten afterwards or something...
Apr 13 2008
parent "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Frits van Bommel" <fvbommel REMwOVExCAPSs.nl> wrote in message 
news:ftsq01$f05$1 digitalmars.com...
 Jarrett Billingsley wrote:
 To both: yes, that's what I would have expected, but I can't seem to get 
 the object to be collected even if the only reference to it is a weak 
 reference. Tried forcing a full collect, allocating bunches of memory, 
 collecting again, allocating at the beginning of a long-running program 
 etc.  it never seems to get collected, even if I never assign the 
 reference into an actual pointer type.

 I'm using Tango if it makes any difference.

Have you tried looking at the generated assembler code? Perhaps the compiler is leaving a reference in some register that's never overwritten afterwards or something...

That's what I was thinking, which is why I tried running lots of code after the allocation, to clear out the registers. I've checked the memory block allocated for the WeakRef instance and it's marked as "finalize" and "no scan", which are both right. I absolutely cannot get it to collect the instance that it references. I really wonder if this is a bug in the Tango GC.
Apr 13 2008
prev sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from Jarrett Billingsley (kb3ctd2 yahoo.com)'s article
 "Jarrett Billingsley" <kb3ctd2 yahoo.com> wrote in message
 news:ftr9as$12s2$1 digitalmars.com...
 I've been looking at some of my old nonagon code, and as I'm sure you've
 all experienced looking at your old code, I'm just _cringing_ at some of
 the things I did in it ;)

 One thing that I've noticed would be nice to have, however, would be weak
 references to some objects.  For example, I have "brush" objects which
 encapsulate a bunch of visual settings (textures, material, blending modes
 etc.).  These are referenced by objects that can be rendered.  Some of
 these brushes need to be updated often (i.e. every frame to update the
 environment mapping matrices).  So I keep a list (table) of all instances
 of this brush class so I can easily loop through them and update them all.

 However when their owners disappear, I end up with uncollectable brushes,
 since they are still being referenced by that table.

 So what would be nice to have happen would be to have that table be a
 table of _weak references_ to brush objects.  Then, when no more real
 references existed, they could be collected, and I wouldn't be keeping
 them around for no reason anymore.

 I've looked into the scrapple weakref implementation, but it doesn't seem
 to do what I want.  Seemingly it only works if the object that it
 references is explicitly deleted.  I want the weakref to become null if
 the object that it references is collected in any way.

 Is this possible at all?  Am I just not using Bill's weakref properly?

object to be collected even if the only reference to it is a weak reference. Tried forcing a full collect, allocating bunches of memory, collecting again, allocating at the beginning of a long-running program etc. it never seems to get collected, even if I never assign the reference into an actual pointer type. I'm using Tango if it makes any difference.

Are these tables AAs? And how do the weak references store their reference? The type information provided to AAs is a bit weak, and so some non-pointer data is currently still scanned by the GC. I'm going to improve this a bit in the next Tango release, but it still won't be perfect (because the compiler doesn't provide enough info for it to be). Sean
Apr 13 2008
parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Sean Kelly" <sean invisibleduck.org> wrote in message 
news:fttd02$1mp1$1 digitalmars.com...
 Are these tables AAs?  And how do the weak references store their 
 reference?
 The type information provided to AAs is a bit weak, and so some 
 non-pointer
 data is currently still scanned by the GC.  I'm going to improve this a 
 bit in the
 next Tango release, but it still won't be perfect (because the compiler 
 doesn't
 provide enough info for it to be).

In my tests I've just been doing auto weakref = new WeakRef!(A)(new A); and then checking that after doing god-knows-what to try to get it to collect that instance of A, so it's not even that I'm doing anything that crazy.
Apr 13 2008
next sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Jarrett Billingsley" <kb3ctd2 yahoo.com> wrote in message 
news:fttdej$1nfs$1 digitalmars.com...
 "Sean Kelly" <sean invisibleduck.org> wrote in message 
 news:fttd02$1mp1$1 digitalmars.com...
 Are these tables AAs?  And how do the weak references store their 
 reference?
 The type information provided to AAs is a bit weak, and so some 
 non-pointer
 data is currently still scanned by the GC.  I'm going to improve this a 
 bit in the
 next Tango release, but it still won't be perfect (because the compiler 
 doesn't
 provide enough info for it to be).

In my tests I've just been doing auto weakref = new WeakRef!(A)(new A); and then checking that after doing god-knows-what to try to get it to collect that instance of A, so it's not even that I'm doing anything that crazy.

Testing my code in Linux causes a crash in glibc's free() after the program exits. (And the assertion still fails.) Something really weird is happening.
Apr 13 2008
parent reply Jason House <jason.james.house gmail.com> writes:
Jarrett Billingsley wrote:

 "Jarrett Billingsley" <kb3ctd2 yahoo.com> wrote in message
 news:fttdej$1nfs$1 digitalmars.com...
 "Sean Kelly" <sean invisibleduck.org> wrote in message
 news:fttd02$1mp1$1 digitalmars.com...
 Are these tables AAs?  And how do the weak references store their
 reference?
 The type information provided to AAs is a bit weak, and so some
 non-pointer
 data is currently still scanned by the GC.  I'm going to improve this a
 bit in the
 next Tango release, but it still won't be perfect (because the compiler
 doesn't
 provide enough info for it to be).

In my tests I've just been doing auto weakref = new WeakRef!(A)(new A); and then checking that after doing god-knows-what to try to get it to collect that instance of A, so it's not even that I'm doing anything that crazy.

Testing my code in Linux causes a crash in glibc's free() after the program exits. (And the assertion still fails.) Something really weird is happening.

Ok, my latest testing (and testing by others for configurations I don't have) are collected in the list below. I don't see any pattern. tango+linux+dmd = works tango+linux+gdc = crash in glibc tango+windows+dmd = does not collect tango+windows+gdc = ??? phobos+linux+dmd = works phobos+linux+gdc = does not collect phobos+windows+dmd = ??? phobos+windows+gdc = ??? These tests are based off the code at http://tango.pastebin.com/m10aa57b6 Note that phobos users must comment out the version=Tango line. Any additional tests would be appreciated.
Apr 13 2008
parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Jason House wrote:
 Jarrett Billingsley wrote:
 
 "Jarrett Billingsley" <kb3ctd2 yahoo.com> wrote in message
 news:fttdej$1nfs$1 digitalmars.com...
 "Sean Kelly" <sean invisibleduck.org> wrote in message
 news:fttd02$1mp1$1 digitalmars.com...
 Are these tables AAs?  And how do the weak references store their
 reference?
 The type information provided to AAs is a bit weak, and so some
 non-pointer
 data is currently still scanned by the GC.  I'm going to improve this a
 bit in the
 next Tango release, but it still won't be perfect (because the compiler
 doesn't
 provide enough info for it to be).

auto weakref = new WeakRef!(A)(new A); and then checking that after doing god-knows-what to try to get it to collect that instance of A, so it's not even that I'm doing anything that crazy.

program exits. (And the assertion still fails.) Something really weird is happening.

Ok, my latest testing (and testing by others for configurations I don't have) are collected in the list below. I don't see any pattern. tango+linux+dmd = works tango+linux+gdc = crash in glibc tango+windows+dmd = does not collect tango+windows+gdc = ??? phobos+linux+dmd = works phobos+linux+gdc = does not collect phobos+windows+dmd = ??? phobos+windows+gdc = ??? These tests are based off the code at http://tango.pastebin.com/m10aa57b6 Note that phobos users must comment out the version=Tango line. Any additional tests would be appreciated.

I'd be worried about stack references in the second example. That inner function makeWa may be getting inlined or something with some compiler combos. The first unittest there works ok right? Maybe explicitly overwriting a with null after using it would help. Anyway, I'm pretty sure Sean said after he added the functions to Tango that he thought they should work but they weren't tested very well. So I wouldn't be surprised if there were issues. And really, Phobos's gc functions for weak ref support probably haven't been tested all that well either. --bb
Apr 13 2008
parent Jason House <jason.james.house gmail.com> writes:
Bill Baxter wrote:

 Jason House wrote:
 Jarrett Billingsley wrote:
 
 "Jarrett Billingsley" <kb3ctd2 yahoo.com> wrote in message
 news:fttdej$1nfs$1 digitalmars.com...
 "Sean Kelly" <sean invisibleduck.org> wrote in message
 news:fttd02$1mp1$1 digitalmars.com...
 Are these tables AAs?  And how do the weak references store their
 reference?
 The type information provided to AAs is a bit weak, and so some
 non-pointer
 data is currently still scanned by the GC.  I'm going to improve this
 a bit in the
 next Tango release, but it still won't be perfect (because the
 compiler doesn't
 provide enough info for it to be).

auto weakref = new WeakRef!(A)(new A); and then checking that after doing god-knows-what to try to get it to collect that instance of A, so it's not even that I'm doing anything that crazy.

program exits. (And the assertion still fails.) Something really weird is happening.

Ok, my latest testing (and testing by others for configurations I don't have) are collected in the list below. I don't see any pattern. tango+linux+dmd = works tango+linux+gdc = crash in glibc tango+windows+dmd = does not collect tango+windows+gdc = ??? phobos+linux+dmd = works phobos+linux+gdc = does not collect phobos+windows+dmd = ??? phobos+windows+gdc = ??? These tests are based off the code at http://tango.pastebin.com/m10aa57b6 Note that phobos users must comment out the version=Tango line. Any additional tests would be appreciated.

I'd be worried about stack references in the second example. That inner function makeWa may be getting inlined or something with some compiler combos. The first unittest there works ok right? Maybe explicitly overwriting a with null after using it would help. Anyway, I'm pretty sure Sean said after he added the functions to Tango that he thought they should work but they weren't tested very well. So I wouldn't be surprised if there were issues. And really, Phobos's gc functions for weak ref support probably haven't been tested all that well either. --bb

I'm mostly trying to extract as much info as I can from this example to better help me file a bug report. The unit test can't differentiate between "never cleans up" and "didn't clean up from register values hanging around"... Nevertheless, it's a good start for further investigation. Certainly the crash inside glibc is a bug in either gdc, tango, or both.
Apr 13 2008
prev sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from Jarrett Billingsley (kb3ctd2 yahoo.com)'s article
 "Sean Kelly" <sean invisibleduck.org> wrote in message
 news:fttd02$1mp1$1 digitalmars.com...
 Are these tables AAs?  And how do the weak references store their
 reference?
 The type information provided to AAs is a bit weak, and so some
 non-pointer
 data is currently still scanned by the GC.  I'm going to improve this a
 bit in the
 next Tango release, but it still won't be perfect (because the compiler
 doesn't
 provide enough info for it to be).

auto weakref = new WeakRef!(A)(new A); and then checking that after doing god-knows-what to try to get it to collect that instance of A, so it's not even that I'm doing anything that crazy.

Just for kicks, try querying the GC to determine whether the GC thinks weakref contains pointers. Sean
Apr 13 2008
next sibling parent reply Jason House <jason.james.house gmail.com> writes:
Sean Kelly wrote:

 == Quote from Jarrett Billingsley (kb3ctd2 yahoo.com)'s article
 "Sean Kelly" <sean invisibleduck.org> wrote in message
 news:fttd02$1mp1$1 digitalmars.com...
 Are these tables AAs?  And how do the weak references store their
 reference?
 The type information provided to AAs is a bit weak, and so some
 non-pointer
 data is currently still scanned by the GC.  I'm going to improve this a
 bit in the
 next Tango release, but it still won't be perfect (because the compiler
 doesn't
 provide enough info for it to be).

auto weakref = new WeakRef!(A)(new A); and then checking that after doing god-knows-what to try to get it to collect that instance of A, so it's not even that I'm doing anything that crazy.

Just for kicks, try querying the GC to determine whether the GC thinks weakref contains pointers. Sean

How would one do that?
Apr 13 2008
parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from Jason House (jason.james.house gmail.com)'s article
 Sean Kelly wrote:
 == Quote from Jarrett Billingsley (kb3ctd2 yahoo.com)'s article
 "Sean Kelly" <sean invisibleduck.org> wrote in message
 news:fttd02$1mp1$1 digitalmars.com...
 Are these tables AAs?  And how do the weak references store their
 reference?
 The type information provided to AAs is a bit weak, and so some
 non-pointer
 data is currently still scanned by the GC.  I'm going to improve this a
 bit in the
 next Tango release, but it still won't be perfect (because the compiler
 doesn't
 provide enough info for it to be).

auto weakref = new WeakRef!(A)(new A); and then checking that after doing god-knows-what to try to get it to collect that instance of A, so it's not even that I'm doing anything that crazy.

Just for kicks, try querying the GC to determine whether the GC thinks weakref contains pointers. Sean


(from memory) if( GC.getAttr( weakref ) & GC.BlkAttr.NO_SCAN ) printf( "The GC will not scan weakref\n" ); Sean
Apr 13 2008
parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Sean Kelly wrote:
 == Quote from Jason House (jason.james.house gmail.com)'s article
 Sean Kelly wrote:
 == Quote from Jarrett Billingsley (kb3ctd2 yahoo.com)'s article
 "Sean Kelly" <sean invisibleduck.org> wrote in message
 news:fttd02$1mp1$1 digitalmars.com...
 Are these tables AAs?  And how do the weak references store their
 reference?
 The type information provided to AAs is a bit weak, and so some
 non-pointer
 data is currently still scanned by the GC.  I'm going to improve this a
 bit in the
 next Tango release, but it still won't be perfect (because the compiler
 doesn't
 provide enough info for it to be).

auto weakref = new WeakRef!(A)(new A); and then checking that after doing god-knows-what to try to get it to collect that instance of A, so it's not even that I'm doing anything that crazy.

weakref contains pointers. Sean


(from memory) if( GC.getAttr( weakref ) & GC.BlkAttr.NO_SCAN ) printf( "The GC will not scan weakref\n" );

Is that how it's supposed to work? I mean WeakRef is basically class WeakRef { size_t hidden_pointer; } Does that mean Tango shouldn't be trying to scan instances of it? I just ask because I was thinking the GC *would* scan the WeakRef's memory just not find anything that looks like a pointer there. Also (for Jason, in particular) it occurs to me that this code: assert(wa.ptr !is null); collect(); assert(wa.ptr is null); Has to store wa.ptr somewhere (register or stack) when you ask to check if it is null or not. So that act of checking itself will likely make a reference that prevents the following collect() from cleaning it up. --bb
Apr 13 2008
parent reply Jason House <jason.james.house gmail.com> writes:
Bill Baxter Wrote:
     if( GC.getAttr( weakref ) & GC.BlkAttr.NO_SCAN )
         printf( "The GC will not scan weakref\n" );

Is that how it's supposed to work? I mean WeakRef is basically class WeakRef { size_t hidden_pointer; } Does that mean Tango shouldn't be trying to scan instances of it? I just ask because I was thinking the GC *would* scan the WeakRef's memory just not find anything that looks like a pointer there.

That's the intent. It keeps a reference without the GC detecting it. The only extra wrinkle to the implementation is that it needs to null out its internal pointer when the object gets collected.
 Also (for Jason, in particular) it occurs to me that this code:
 
    assert(wa.ptr !is null);
    collect();
    assert(wa.ptr is null);
 
 Has to store wa.ptr somewhere (register or stack) when you ask to check 
 if it is null or not.  So that act of checking itself will likely make a 
 reference that prevents the following collect() from cleaning it up.

That's true that that might cause the last assert to fail, oops. I don't know what the correct way to clear everything out is (including from the preceding function call). Any more tips on making a more correct unit test are appreciated. Jarret was having issues with non-collection (ever) when using weak ref's and I had written this test to try and exercise it (his problem is windows + tango + dmd). An assert failure doesn't mean it won't ever collect but will narrow the search down and allow closer examination... and the crash on linux+tango+gdc is definitely not correct behavior.
Apr 14 2008
parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from Jason House (jason.james.house gmail.com)'s article
 Bill Baxter Wrote:
     if( GC.getAttr( weakref ) & GC.BlkAttr.NO_SCAN )
         printf( "The GC will not scan weakref\n" );

Is that how it's supposed to work? I mean WeakRef is basically class WeakRef { size_t hidden_pointer; } Does that mean Tango shouldn't be trying to scan instances of it? I just ask because I was thinking the GC *would* scan the WeakRef's memory just not find anything that looks like a pointer there.


 Also (for Jason, in particular) it occurs to me that this code:

    assert(wa.ptr !is null);
    collect();
    assert(wa.ptr is null);

 Has to store wa.ptr somewhere (register or stack) when you ask to check
 if it is null or not.  So that act of checking itself will likely make a
 reference that prevents the following collect() from cleaning it up.


The easiest thing I can think of would be to do an access of a dummy weakptr after each access of the real weakptr: auto dummy = new WeakPtr!(void); assert(wa.ptr !is null); assert(dummy.ptr is null); collect(); assert(wa.ptr is null); Since the dummy op is identical to the real op, it should hopefully hit the same registers. Best way to be sure would be to test with -O turned off.
 Jarret was having issues with non-collection (ever) when using weak ref's and
I had written this test to try and exercise it (his problem

examination... and the crash on linux+tango+gdc is definitely not correct behavior. Thanks for any investigation you do for this. If you can find a bug in Tango, I'd be happy to fix it, but I'm too busy to do the research myself at the moment. Sean
Apr 14 2008
parent reply Jason House <jason.james.house gmail.com> writes:
Sean Kelly Wrote:
 Thanks for any investigation you do for this.  If you can find a bug in Tango,
I'd be
 happy to fix it, but I'm too busy to do the research myself at the moment.

I have a vested interest in it... I want my code to work ;) Just to be clear, what is enough to proclaim stuff as a bug in Tango? For the failed assert: prior to collect, show a lack of lingering references through disassembly? For a runtime crash: Isn't any crash in core libraries used by Tango a Tango bug? For whatever I can prove to be a Tango bug, I'll submit a ticket.
Apr 14 2008
next sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
Jason House wrote:

 Sean Kelly Wrote:
 Thanks for any investigation you do for this.  If you can find a bug in
 Tango, I'd be happy to fix it, but I'm too busy to do the research myself
 at the moment.

I have a vested interest in it... I want my code to work ;) Just to be clear, what is enough to proclaim stuff as a bug in Tango? For the failed assert: prior to collect, show a lack of lingering references through disassembly? For a runtime crash: Isn't any crash in core libraries used by Tango a Tango bug? For whatever I can prove to be a Tango bug, I'll submit a ticket.

You can create a Tango ticket for anything directly related to Tango that doesn't work. In this particular case, a compiler bug is at least as likely (due to what is working with what where). We have many tickets for problems that have been verified to be toolchain problems, as it is easier for us to track how it affect us that way. -- Lars Ivar Igesund blog at http://larsivi.net DSource, #d.tango & #D: larsivi Dancing the Tango
Apr 14 2008
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
== Quote from Jason House (jason.james.house gmail.com)'s article
 Sean Kelly Wrote:
 Thanks for any investigation you do for this.  If you can find a bug in Tango,
I'd be
 happy to fix it, but I'm too busy to do the research myself at the moment.

Just to be clear, what is enough to proclaim stuff as a bug in Tango?

Ideally, any bug report will contain a simple repro and/or mention of where the actual problem is in Tango. But we see ambiguous bug reports or things that, after investigation, turn out to be compiler problems and the like. But the easier you can make things on our end, the faster the problem is likely to be fixed :-)
 For the failed assert: prior to collect, show a lack of lingering references
through disassembly?

What I'd do to diagnose this is rebuild the Tango runtime with debug reporting turned on for the GC, possibly adding something to print the address of all blocks being collected (if that doesn't exist already). We know that weakptr data isn't being collected on Win32... the question is why.
 For a runtime crash: Isn't any crash in core libraries used by Tango a Tango
bug?

Not necessarily. We've seen a bunch of crashes that GDB reported were in the runtime that actually turned out to be compiler bugs in GDC, or in some cases errors in the user program that ended up surfacing as runtime code exploding. But in theory you're right.
 For whatever I can prove to be a Tango bug, I'll submit a ticket.

Thank you :-) Sean
Apr 14 2008
prev sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Sean Kelly" <sean invisibleduck.org> wrote in message 
news:ftubv9$kph$1 digitalmars.com...
 == Quote from Jarrett Billingsley (kb3ctd2 yahoo.com)'s article
 "Sean Kelly" <sean invisibleduck.org> wrote in message
 news:fttd02$1mp1$1 digitalmars.com...
 Are these tables AAs?  And how do the weak references store their
 reference?
 The type information provided to AAs is a bit weak, and so some
 non-pointer
 data is currently still scanned by the GC.  I'm going to improve this a
 bit in the
 next Tango release, but it still won't be perfect (because the compiler
 doesn't
 provide enough info for it to be).

auto weakref = new WeakRef!(A)(new A); and then checking that after doing god-knows-what to try to get it to collect that instance of A, so it's not even that I'm doing anything that crazy.

Just for kicks, try querying the GC to determine whether the GC thinks weakref contains pointers.

Already tried it. The GC sees the weakref instance's memory as NO_SCAN and FINALIZE, both correct.
Apr 13 2008
parent Sean Kelly <sean invisibleduck.org> writes:
== Quote from Jarrett Billingsley (kb3ctd2 yahoo.com)'s article
 "Sean Kelly" <sean invisibleduck.org> wrote in message
 news:ftubv9$kph$1 digitalmars.com...
 == Quote from Jarrett Billingsley (kb3ctd2 yahoo.com)'s article
 "Sean Kelly" <sean invisibleduck.org> wrote in message
 news:fttd02$1mp1$1 digitalmars.com...
 Are these tables AAs?  And how do the weak references store their
 reference?
 The type information provided to AAs is a bit weak, and so some
 non-pointer
 data is currently still scanned by the GC.  I'm going to improve this a
 bit in the
 next Tango release, but it still won't be perfect (because the compiler
 doesn't
 provide enough info for it to be).

auto weakref = new WeakRef!(A)(new A); and then checking that after doing god-knows-what to try to get it to collect that instance of A, so it's not even that I'm doing anything that crazy.

Just for kicks, try querying the GC to determine whether the GC thinks weakref contains pointers.

FINALIZE, both correct.

Huh. Afraid I'm at a loss as to what's going on then. Even stranger that it works on linux but not Win32, since the code is basically the same for both. I suppose I could add some profiling code to try and figure out where the lingering reference is, assuming there is one. But not sure when I'll have the time to do this. Sean
Apr 14 2008
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2008-04-12 17:24:44 -0400, "Jarrett Billingsley" <kb3ctd2 yahoo.com> said:

 Is this possible at all?  Am I just not using Bill's weakref properly?

I've been using an alternate strategy in the D/Objective-C bridge for the bridged object lookup table. Basically, I'm casting object references to size_t (so they're no longer pointers) and mangling the pointers (so the garbage collector can't see them as pointers, I think this is no longer be necessary). When the bridged object gets deleted on the D side, it removes its mangled pointer from the table. So if your referenced object knows about its weak reference in the table, you could implement it as such and don't have to worry about null pointers polluting your table. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Apr 13 2008
parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Michel Fortin wrote:
 On 2008-04-12 17:24:44 -0400, "Jarrett Billingsley" <kb3ctd2 yahoo.com> 
 said:
 
 Is this possible at all?  Am I just not using Bill's weakref properly?

I've been using an alternate strategy in the D/Objective-C bridge for the bridged object lookup table. Basically, I'm casting object references to size_t (so they're no longer pointers)

Yes, that's what the weak ref class does.
 and mangling the 
 pointers (so the garbage collector can't see them as pointers, I think 
 this is no longer be necessary). 

I agree that that's not necessary.
 When the bridged object gets deleted on 
 the D side, it removes its mangled pointer from the table.
 
 So if your referenced object knows about its weak reference in the 
 table, you could implement it as such and don't have to worry about null 
 pointers polluting your table.

Good idea. I was thinking to suggest something like that too. A WeakRef class is not really an optimal way to keep track of a collection of weak refs. In addition to null pointers polluting the table, there's also just the basic overhead of needing to create an extra object for each entry. Ideally there would be WeakArray, WeakSet, WeakHeap... etc. I think I've seen such things in some language. Python maybe? --bb
Apr 13 2008
next sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Bill Baxter" <dnewsgroup billbaxter.com> wrote in message 
news:fttn2m$2emt$1 digitalmars.com...
 Michel Fortin wrote:
 On 2008-04-12 17:24:44 -0400, "Jarrett Billingsley" <kb3ctd2 yahoo.com> 
 said:

 Is this possible at all?  Am I just not using Bill's weakref properly?

I've been using an alternate strategy in the D/Objective-C bridge for the bridged object lookup table. Basically, I'm casting object references to size_t (so they're no longer pointers)

Yes, that's what the weak ref class does.
 and mangling the pointers (so the garbage collector can't see them as 
 pointers, I think this is no longer be necessary).

I agree that that's not necessary.
 When the bridged object gets deleted on the D side, it removes its 
 mangled pointer from the table.

 So if your referenced object knows about its weak reference in the table, 
 you could implement it as such and don't have to worry about null 
 pointers polluting your table.

Good idea. I was thinking to suggest something like that too. A WeakRef class is not really an optimal way to keep track of a collection of weak refs. In addition to null pointers polluting the table, there's also just the basic overhead of needing to create an extra object for each entry. Ideally there would be WeakArray, WeakSet, WeakHeap... etc. I think I've seen such things in some language. Python maybe?

Lua's tables can have weak keys, weak values, or both.
Apr 13 2008
parent reply Robert Fraser <fraserofthenight gmail.com> writes:
Jarrett Billingsley wrote:
 Lua's tables can have weak keys, weak values, or both. 

And MiniD's...? ;-P
Apr 13 2008
parent "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Robert Fraser" <fraserofthenight gmail.com> wrote in message 
news:fttt4r$2oic$2 digitalmars.com...
 Jarrett Billingsley wrote:
 Lua's tables can have weak keys, weak values, or both.

And MiniD's...? ;-P

If I had a weak reference lib for D it could X(
Apr 13 2008
prev sibling parent reply Bjoern <nanali nospam-wanadoo.fr> writes:
Bill Baxter schrieb:
 of weak refs.  In addition to null pointers polluting the table, there's 
 also just the basic overhead of needing to create an extra object for 
 each entry.  Ideally there would be WeakArray, WeakSet, WeakHeap... etc. 
  I think I've seen such things in some language.  Python maybe?
 
 --bb

weakHashMap, weakTreeMap ??? implemented in Java.... ? OpenJDK/ GNU Classpath Well, Frank is looking for a D weakHashMap implementation. Due to the fact that I am actually doing plombering, plastering, electric, establish tons of cement concrete ... I am not able to offer a solution. (Beeing allready happy that I am able to write a msg) Nevertheless I did like to suggest to use Bill B's weakRef! as inner adaptor class of WeakHashMap. (pretty much the same as in GNU classpath) as static array of WeakRef objects. Further the dCollection : see dSource / collection classes are <IMO> fitting better than Tango Collection. Bjoern
Apr 23 2008
parent Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Bjoern wrote:
 Bill Baxter schrieb:
 of weak refs.  In addition to null pointers polluting the table, 
 there's also just the basic overhead of needing to create an extra 
 object for each entry.  Ideally there would be WeakArray, WeakSet, 
 WeakHeap... etc.  I think I've seen such things in some language.  
 Python maybe?

 --bb

weakHashMap, weakTreeMap ??? implemented in Java.... ? OpenJDK/ GNU Classpath Well, Frank is looking for a D weakHashMap implementation. Due to the fact that I am actually doing plombering, plastering, electric, establish tons of cement concrete ... I am not able to offer a solution. (Beeing allready happy that I am able to write a msg) Nevertheless I did like to suggest to use Bill B's weakRef! as inner adaptor class of WeakHashMap. (pretty much the same as in GNU classpath) as static array of WeakRef objects. Further the dCollection : see dSource / collection classes are <IMO> fitting better than Tango Collection.

Using the wrappers might have too much overhead. But perhaps MinTL with an allocator using malloc would do the trick? -- Tomasz Stachowiak http://h3.team0xf.com/ h3/h3r3tic on #D freenode
Apr 23 2008