www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - Delegate-object invariant bug on program exit.

reply pragma <pragma_member pathlink.com> writes:
DMD v100.00
OPTLINK v7.50B1
Win2k SP4

The following creates a general protection fault.  I've traced it down to
phobos\internal\invariant.d (_d_invariant).

import std.stdio;

class Foo{
public void notify(){
writefln("notified");
}
}
class Bar{
public void delegate() event;
~this(){
event();
}	
}
void main(){
Bar b = new Bar();
Foo f = new Foo();
b.event = &(f.notify);
}

It looks the instance of Foo is being collected before the instance of Bar, thus
invalidating the delegate.

- Pragma

[[ EricAnderton at (don't spam me anymore) yahoo.com ]]
Aug 24 2004
parent reply Ben Hinkle <bhinkle4 juno.com> writes:
pragma wrote:

 DMD v100.00
 OPTLINK v7.50B1
 Win2k SP4
 
 The following creates a general protection fault.  I've traced it down to
 phobos\internal\invariant.d (_d_invariant).
 
 import std.stdio;
 
 class Foo{
 public void notify(){
 writefln("notified");
 }
 }
 class Bar{
 public void delegate() event;
 ~this(){
 event();
 }
 }
 void main(){
 Bar b = new Bar();
 Foo f = new Foo();
 b.event = &(f.notify);
 }
 
 It looks the instance of Foo is being collected before the instance of
 Bar, thus invalidating the delegate.
 
 - Pragma
 
 [[ EricAnderton at (don't spam me anymore) yahoo.com ]]

I think it will be quite hard getting this kind of thing to work. A destructor should never try to reference another object - since as you found out it might be a bogus reference. I'd like to see a sentance or two in the D destructor doc to avoid referencing other objects - or anything that belongs to the GC. See my post on the main newsgroup digitalmars.D/9023 -Ben
Aug 24 2004
parent reply "antiAlias" <fu bar.com> writes:
"Ben Hinkle" <bhinkle4 juno.com> wrote in message
news:cggsr7$1i9o$1 digitaldaemon.com...
 pragma wrote:

 DMD v100.00
 OPTLINK v7.50B1
 Win2k SP4

 The following creates a general protection fault.  I've traced it down


 phobos\internal\invariant.d (_d_invariant).

 import std.stdio;

 class Foo{
 public void notify(){
 writefln("notified");
 }
 }
 class Bar{
 public void delegate() event;
 ~this(){
 event();
 }
 }
 void main(){
 Bar b = new Bar();
 Foo f = new Foo();
 b.event = &(f.notify);
 }

 It looks the instance of Foo is being collected before the instance of
 Bar, thus invalidating the delegate.

 - Pragma

 [[ EricAnderton at (don't spam me anymore) yahoo.com ]]

I think it will be quite hard getting this kind of thing to work. A destructor should never try to reference another object - since as you found out it might be a bogus reference. I'd like to see a sentance or two in the D destructor doc to avoid referencing other objects - or anything that belongs to the GC. See my post on the main newsgroup digitalmars.D/9023 -Ben

Yes, that would be helpful. But it's not /always/ practical to avoid touching other objects during destruction, though perhaps 'desirable'. And it's not an obvious error either, since one could hardly be blamed for thinking the delegate holds a reference to its enclosing class instance (it's a delegate; not a function). If so, then the instance should still be live as long as the delegate is referenced ... yes? Apparently that's not how it works. Sure looks like a bug. Do you know what the delegate holds a reference to, Ben?
Aug 24 2004
next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
antiAlias wrote:

 "Ben Hinkle" <bhinkle4 juno.com> wrote in message
 news:cggsr7$1i9o$1 digitaldaemon.com...
 pragma wrote:

 DMD v100.00
 OPTLINK v7.50B1
 Win2k SP4

 The following creates a general protection fault.  I've traced it down


 phobos\internal\invariant.d (_d_invariant).

 import std.stdio;

 class Foo{
 public void notify(){
 writefln("notified");
 }
 }
 class Bar{
 public void delegate() event;
 ~this(){
 event();
 }
 }
 void main(){
 Bar b = new Bar();
 Foo f = new Foo();
 b.event = &(f.notify);
 }

 It looks the instance of Foo is being collected before the instance of
 Bar, thus invalidating the delegate.

 - Pragma

 [[ EricAnderton at (don't spam me anymore) yahoo.com ]]

I think it will be quite hard getting this kind of thing to work. A destructor should never try to reference another object - since as you found out it might be a bogus reference. I'd like to see a sentance or two in the D destructor doc to avoid referencing other objects - or anything that belongs to the GC. See my post on the main newsgroup digitalmars.D/9023 -Ben

Yes, that would be helpful. But it's not /always/ practical to avoid touching other objects during destruction, though perhaps 'desirable'.

well... just as long as none of those objects need collecting at program exit. Otherwise it's a game of chance which one gets GC'ed first.
 And it's not an obvious error either, since one could hardly be blamed for
 thinking the delegate holds a reference to its enclosing class instance
 (it's a delegate; not a function). If so, then the instance should still
 be live as long as the delegate is referenced ... yes? Apparently that's
 not how it works. Sure looks like a bug.

True - it should be considered a bug. Java (I think) allows other objects to be referenced in finalizers. I don't know the details, though, since finalizers are frowned upon I've never really gotten into it.
 Do you know what the delegate holds a reference to, Ben?

in the case above, f.
Aug 24 2004
parent reply pragma <pragma_member pathlink.com> writes:
 
 Yes, that would be helpful. But it's not /always/ practical to avoid
 touching other objects during destruction, though perhaps 'desirable'.

well... just as long as none of those objects need collecting at program exit. Otherwise it's a game of chance which one gets GC'ed first.

It gets even more frustrating than that. Similar to what you ran into with streams, I have a .dll Library class that has delegate array (event) that functions as a multicast callback to other interested parties when it is unloaded. This is a much needed runtime behavior, and is especially useful to fire this event when the library is unloaded during finalization.
 And it's not an obvious error either, since one could hardly be blamed for
 thinking the delegate holds a reference to its enclosing class instance
 (it's a delegate; not a function). If so, then the instance should still
 be live as long as the delegate is referenced ... yes? Apparently that's
 not how it works. Sure looks like a bug.


Yep, that's basically where I was when I filed the report. The delegate has an outstanding reference, yet it's object was destroyed.
True - it should be considered a bug. Java (I think) allows other objects to
be referenced in finalizers. I don't know the details, though, since
finalizers are frowned upon I've never really gotten into it.

It creates a lot of problems, especially when you're relying on the GC to perform collections during the execution of the program. Its nice to know when something is completely destroyed, and releases its resources: # class Foo{ # int value; # ~this(){ Logger.getLogger.put("Foo Destroyed (%d).",value); } # } Something like this becomes a problem with the current revision of D: it works great during runtime and then (very likely) fails miserably once you exit the program. The problem also gets worse when expecting to be able to free a resource on termination, but can't since some random destructor causes a GPF. Honestly, my expectation was that during the final collection (before program exit) all the finalizers would be run *first* before any memory is freed. # // pseudocode for GC termination # // (multithreading issues not withstanding) # alias void* function(void*) Finalizer; # void*[] gc_objects; # Finalizer[void*] gc_finalizers; # # void shutdown(){ # while(gc_finalizers.length > 0){ # foreach(void* obj,Finalizer fin; gc_finalizers){ # fin(obj); # } # } # foreach(void* obj; gc_objects) free(obj); # } I get the impression that it's doing something like this instead: # void shutdown(){ # foreach(void* obj,Finalizer fin; gc_finalizers){ # fin(obj); # free(obj); # } # } - Pragma [[ EricAnderton at (pounding away on code) yahoo.com ]]
Aug 25 2004
parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
 Honestly, my expectation was that during the final collection (before

 exit) all the finalizers would be run *first* before any memory is freed.

That makes sense - though it would cause two loops over the garbage instead of one. Also no-one has brought up the cases when garbage can be brought back to life. That's one reason why Java's finalizers are frowned upon - they can bring back dead references and generally cause the GC to jump through hoops to make sure it isn't cleaning up live data. What if one of the destructors assigns a reference to some garbage to a static variable and thus brings it back to life? The GC needs to check all the program's references after the destructors are run to make sure no garbage was brought to life. That would be a nasty performance hit. It occured to me that this problem of having destructors reference other objects isn't limited to program exit. It can happen any time there is a collection. The reason it shows up at exit often is that D *always* collects at exit and our programs that have the problem don't run long enough to force collections during normal run-time (or the object being referenced isn't garbage during runtime). Or we can keep the current architecture and just document the fact that destructors can't reference any other GC managed resources (or anything that has managed GC resources). Nasty but it lets the GC do whatever it wants.
Aug 25 2004
parent reply pragma <pragma_member pathlink.com> writes:
In article <cgia4c$273m$1 digitaldaemon.com>, Ben Hinkle says...
 Honestly, my expectation was that during the final collection (before

 exit) all the finalizers would be run *first* before any memory is freed.

That makes sense - though it would cause two loops over the garbage instead of one. Also no-one has brought up the cases when garbage can be brought back to life. That's one reason why Java's finalizers are frowned upon - they can bring back dead references and generally cause the GC to jump through hoops to make sure it isn't cleaning up live data.

I see what you mean. Granted, there's nothing keeping a destructor from creating *more* data on the heap in a growing spiral of allocations. But I really don't see that as any more a problem as infinite recursion or mutually-referenced functions (like in the invariant 'bug' menioned earlier in this NG). But I strongly feel that reducing destructors to playing with only scalar types (as we've discovered that object references are all "weak" in the present D implementation during the final collect) reduces their capability to near uselessness. I also just noticed that my pseudocode on the previous post was also a little flawed, and hence misleading. If one were to loop over a /copy/ of the current set of finalizers, after clearing out the old list, the outer loop stands a good chance of actually terminating.
What if one of
the destructors assigns a reference to some garbage to a static variable and
thus brings it back to life? The GC needs to check all the program's
references after the destructors are run to make sure no garbage was brought
to life. That would be a nasty performance hit.

Well, "ressurecting" an object properly would require the GC to register that object's finalizer, in which case it looks like a normal object. That /would/ thwart the present behavior as well as my informal proposal in my previous post. Personally, I'd rather have all my objects a /chance/ of cleaning up propertly on program exit rather than have unpredictable behavior inside of destructors. I know what I'm asking for is deterministic destruction of objects, in all cases, which has been something of a holy war on the D NG. I think its possible to finialize in the proposed method and abort to a full free of the heap if the loop runs for too long, or if there's no change in the number of finalizers for a number of iterations. At least that way, you have given the program a fighting chance of being well behaved on exit without bending over backwards in your code. Another way to go would be to allow the GC to track a set of privileged references that are guaranteed to be finalized before a dealloc on exit. These would be explicitly set in the client code and would be understood to delay program termination by ever so much. At the very least, it'd be nice to have a way to check if a reference is still valid, and hasn't been collected. That way one could treat object references as 'weak' if you're using code inside of a destructor. -Pragma [[ EricAnderton at (let my objects go) yahoo.com ]]
Aug 25 2004
next sibling parent reply "antiAlias" <fu bar.com> writes:
If this behavior is a trade-off between correct program-execution and
"performance" during program termination, then there's something very
seriously wrong with the foundations of this language.

As Pragma points out, if you can't reliably release resources during a
destructor then their value is diminished to the level of worthless.
Actually better not to have destructors at all. Now where would that leave
us?

What's the point of tracking references and dependencies if they're then
simply ignored during Object destruction? Utter lunacy.


"pragma" <pragma_member pathlink.com> wrote in message
news:cgif07$29ev$1 digitaldaemon.com...
 In article <cgia4c$273m$1 digitaldaemon.com>, Ben Hinkle says...
 Honestly, my expectation was that during the final collection (before

 exit) all the finalizers would be run *first* before any memory is



That makes sense - though it would cause two loops over the garbage


of one. Also no-one has brought up the cases when garbage can be brought
back to life. That's one reason why Java's finalizers are frowned upon -
they can bring back dead references and generally cause the GC to jump
through hoops to make sure it isn't cleaning up live data.

I see what you mean. Granted, there's nothing keeping a destructor from creating *more* data on the heap in a growing spiral of allocations. But

 really don't see that as any more a problem as infinite recursion or
 mutually-referenced functions (like in the invariant 'bug' menioned

 this NG).  But I strongly feel that reducing destructors to playing with

 scalar types (as we've discovered that object references are all "weak" in

 present D implementation during the final collect) reduces their

 near uselessness.

 I also just noticed that my pseudocode on the previous post was also a

 flawed, and hence misleading.  If one were to loop over a /copy/ of the

 set of finalizers, after clearing out the old list, the outer loop stands

 chance of actually terminating.

What if one of
the destructors assigns a reference to some garbage to a static variable


thus brings it back to life? The GC needs to check all the program's
references after the destructors are run to make sure no garbage was


to life. That would be a nasty performance hit.

Well, "ressurecting" an object properly would require the GC to register

 object's finalizer, in which case it looks like a normal object.  That

 thwart the present behavior as well as my informal proposal in my previous

 Personally, I'd rather have all my objects a /chance/ of cleaning up

 on program exit rather than have unpredictable behavior inside of

 I know what I'm asking for is deterministic destruction of objects, in all
 cases, which has been something of a holy war on the D NG.

 I think its possible to finialize in the proposed method and abort to a

 free of the heap if the loop runs for too long, or if there's no change in

 number of finalizers for a number of iterations.  At least that way, you

 given the program a fighting chance of being well behaved on exit without
 bending over backwards in your code.

 Another way to go would be to allow the GC to track a set of privileged
 references that are guaranteed to be finalized before a dealloc on exit.

 would be explicitly set in the client code and would be understood to

 program termination by ever so much.

 At the very least, it'd be nice to have a way to check if a reference is

 valid, and hasn't been collected.  That way one could treat object

 'weak' if you're using code inside of a destructor.

 -Pragma
 [[ EricAnderton at (let my objects go) yahoo.com ]]

Aug 25 2004
parent reply pragma <pragma_member pathlink.com> writes:
In article <cgiiff$2bio$1 digitaldaemon.com>, antiAlias says...
If this behavior is a trade-off between correct program-execution and
"performance" during program termination, then there's something very
seriously wrong with the foundations of this language.

As Pragma points out, if you can't reliably release resources during a
destructor then their value is diminished to the level of worthless.
Actually better not to have destructors at all. Now where would that leave
us?

I'll stay off my soapbox on this one. However, I'll be happy to provide any support to the cause should there be a weighing-in of pros and cons. All I want is to write good software in D, and through that have D succeed.
What's the point of tracking references and dependencies if they're then
simply ignored during Object destruction? Utter lunacy.

Well let's not jump to conclusions... I would like to contend that this seems like a mistake in implementation and is still likely a "bug" until proven otherwise. :) (Now, if it's been said before that the current behavior is quite deliberate, then I'll stand corrected.) -Pragma [[ EricAnderton at (making D better) yahoo.com ]]
Aug 25 2004
parent reply "antiAlias" <fu bar.com> writes:
 Well let's not jump to conclusions... I would like to contend that this

 like a mistake in implementation and is still likely a "bug" until proven
 otherwise. :)

Fair point Eric. Having come-a-cropper over so many fundamental "no ... surely not!" type issues, I'm afraid my resilience against more of same has been frayed to tatters. Part of the whole "library development issues: what to do?" thread.
 All I want is to write good software in D, and through that have D

Hear hear. There's no other reason why I've spent four months 80-hours a week on Mango. Unfortunately, it sometimes feels as though the language, and /particularly/ the maturation 'process' behind it, is actually working against that goal (note that the "what to do?" thread all but disappeared down the black-hole). More's the pity.
Aug 25 2004
parent "Matthew" <admin.hat stlsoft.dot.org> writes:
"antiAlias" <fu bar.com> wrote in message
news:cgipm8$2f1j$1 digitaldaemon.com...
 Well let's not jump to conclusions... I would like to contend that this

 like a mistake in implementation and is still likely a "bug" until proven
 otherwise. :)

Fair point Eric. Having come-a-cropper over so many fundamental "no ... surely not!" type issues, I'm afraid my resilience against more of same has been frayed to tatters. Part of the whole "library development issues: what to do?" thread.

LOL! Shared, pain, old buddy. But fear not, you have several weeks while Phoenix founders before you must face reality. Or, optimistically, it might actually succeed, in which case we'll all be happy. :)
 All I want is to write good software in D, and through that have D

Hear hear. There's no other reason why I've spent four months 80-hours a week on Mango. Unfortunately, it sometimes feels as though the language, and /particularly/ the maturation 'process' behind it, is actually working against that goal (note that the "what to do?" thread all but disappeared down the black-hole). More's the pity.

Aug 26 2004
prev sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
"pragma" <pragma_member pathlink.com> wrote in message
news:cgif07$29ev$1 digitaldaemon.com...
 In article <cgia4c$273m$1 digitaldaemon.com>, Ben Hinkle says...
 Honestly, my expectation was that during the final collection (before

 exit) all the finalizers would be run *first* before any memory is



That makes sense - though it would cause two loops over the garbage


of one. Also no-one has brought up the cases when garbage can be brought
back to life. That's one reason why Java's finalizers are frowned upon -
they can bring back dead references and generally cause the GC to jump
through hoops to make sure it isn't cleaning up live data.

I see what you mean. Granted, there's nothing keeping a destructor from creating *more* data on the heap in a growing spiral of allocations. But

 really don't see that as any more a problem as infinite recursion or
 mutually-referenced functions (like in the invariant 'bug' menioned

 this NG).  But I strongly feel that reducing destructors to playing with

 scalar types (as we've discovered that object references are all "weak" in

 present D implementation during the final collect) reduces their

 near uselessness.

Destructors should only be used to release external resources - like file handles, malloc'ed memory etc. To quote from the D documentation on the section about destructors: <quote> When the garbage collector calls a destructor for an object of a class that has members that are references to garbage collected objects, those references are no longer valid. This means that destructors cannot reference sub objects. This rule does not apply to auto objects or objects deleted with the DeleteExpression. The garbage collector is not guaranteed to run the destructor for all unreferenced objects. Furthermore, the order in which the garbage collector calls destructors for unreference objects is not specified. <\quote> It would be nice if we lived in a world where the GC could be fast and run destructors "nicely" but given the choice I would go for GC performance. Destructors should be a very weak construct. You never know when or if they will run so most of the time relying on them is a bug in disguise. Google around for web pages about Finalizers and Java and you'll get an idea how everyone in the Java world says not to use them. Remember all this destructor running happens on every GC pass, not just program exit.
 I also just noticed that my pseudocode on the previous post was also a

 flawed, and hence misleading.  If one were to loop over a /copy/ of the

 set of finalizers, after clearing out the old list, the outer loop stands

 chance of actually terminating.

What if one of
the destructors assigns a reference to some garbage to a static variable


thus brings it back to life? The GC needs to check all the program's
references after the destructors are run to make sure no garbage was


to life. That would be a nasty performance hit.

Well, "ressurecting" an object properly would require the GC to register

 object's finalizer, in which case it looks like a normal object.  That

 thwart the present behavior as well as my informal proposal in my previous

 Personally, I'd rather have all my objects a /chance/ of cleaning up

 on program exit rather than have unpredictable behavior inside of

 I know what I'm asking for is deterministic destruction of objects, in all
 cases, which has been something of a holy war on the D NG.

 I think its possible to finialize in the proposed method and abort to a

 free of the heap if the loop runs for too long, or if there's no change in

 number of finalizers for a number of iterations.  At least that way, you

 given the program a fighting chance of being well behaved on exit without
 bending over backwards in your code.

 Another way to go would be to allow the GC to track a set of privileged
 references that are guaranteed to be finalized before a dealloc on exit.

 would be explicitly set in the client code and would be understood to

 program termination by ever so much.

 At the very least, it'd be nice to have a way to check if a reference is

 valid, and hasn't been collected.  That way one could treat object

 'weak' if you're using code inside of a destructor.

 -Pragma
 [[ EricAnderton at (let my objects go) yahoo.com ]]

Aug 25 2004
next sibling parent reply pragma <pragma_member pathlink.com> writes:
In article <cgisdb$2g83$1 digitaldaemon.com>, Ben Hinkle says...
Destructors should only be used to release external resources - like file
handles, malloc'ed memory etc. To quote from the D documentation on the
section about destructors:
<quote>
When the garbage collector calls a destructor for an object of a class that
has members that are references to garbage collected objects, those
references are no longer valid. This means that destructors cannot reference
sub objects. This rule does not apply to auto objects or objects deleted
with the DeleteExpression.
The garbage collector is not guaranteed to run the destructor for all
unreferenced objects. Furthermore, the order in which the garbage collector
calls destructors for unreference objects is not specified.
<\quote>

Yikes. All apologies to Walter, who has very kindly saved me the embarassment of being told this from himself. I guess that says it. Its not a bug, just not at all what I expected. All the same, I'd like to request that this be seriously reconsidered for V2.0. So I suppose the proper "D" way to proceed is to use an auto object on the "main" thread, that manages critical resources for me?
It would be nice if we lived in a world where the GC could be fast and run
destructors "nicely" but given the choice I would go for GC performance.
Destructors should be a very weak construct. You never know when or if they
will run so most of the time relying on them is a bug in disguise. Google
around for web pages about Finalizers and Java and you'll get an idea how
everyone in the Java world says not to use them. Remember all this
destructor running happens on every GC pass, not just program exit.

I see what you mean. Having developers code destructors in this fashion can have the consequence of increased performance. I still contend that D supporting "strong" constructors changes little but D's flexibility. Thanks everybody (Ben, Kris, Sean) for their help here on this issue. Again, all apologies to Walter for my having wasted his time with something that was in the documentation. - Pragma [[ EricAnderton at (auto + deamon = fix?) yahoo.com ]]
Aug 25 2004
next sibling parent Ben Hinkle <bhinkle4 juno.com> writes:
 Thanks everybody (Ben, Kris, Sean) for their help here on this issue. 
 Again, all apologies to Walter for my having wasted his time with
 something that was in the documentation.

Actually I hadn't read that part of the doc until today either. My eariler post when I was messing with std.stream was trying to do too much in destructors so you're definitely not alone in writing ambitious destructors. Maybe if that part of the doc was in bold and blinking or something... It's very easy to just think destructors are just like C++ destructors only you don't have to worry about freeing memory. oh boy is that dangerous.
Aug 25 2004
prev sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cgj8qb$2ler$1 digitaldaemon.com>, pragma says...

The garbage collector is not guaranteed to run the destructor for all
unreferenced objects.


Please see THIS POST from Walter, and the discussion before and after it: digitalmars.D/3488 In particular, Walter said: "I must have misspoke, because when the gc collects an object, it *does* run the destructor." (Walter's emphasis on *does*). I have security apps planned which rely on Walter's statement being true. Where is this "not guaranteed" quote in the manual? I couldn't find it. Basically - help! Who's right? Walter, or the manual? Walter, please could you clarify (again), and fix the manual if it's wrong. Jill
Aug 26 2004
next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
Arcane Jill wrote:

 In article <cgj8qb$2ler$1 digitaldaemon.com>, pragma says...
 
The garbage collector is not guaranteed to run the destructor for all
unreferenced objects.


Please see THIS POST from Walter, and the discussion before and after it: digitalmars.D/3488 In particular, Walter said: "I must have misspoke, because when the gc collects an object, it *does* run the destructor." (Walter's emphasis on *does*). I have security apps planned which rely on Walter's statement being true. Where is this "not guaranteed" quote in the manual? I couldn't find it. Basically - help! Who's right? Walter, or the manual? Walter, please could you clarify (again), and fix the manual if it's wrong. Jill

The manual and Walter's post are consistent. The GC runs the destructor when it collects an object but it isn't guaranteed to collect all unreferenced objects (due to ambiguous references etc).
Aug 26 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cgkkeq$925$1 digitaldaemon.com>, Ben Hinkle says...

The manual and Walter's post are consistent. The GC runs the destructor when
it collects an object but it isn't guaranteed to collect all unreferenced
objects (due to ambiguous references etc). 

Okay, but presumably it *is* guaranteed to collect everything not yet collected (and to call all remaining destructors) at program exit. Right?
Aug 26 2004
parent reply Ben Hinkle <bhinkle4 juno.com> writes:
Arcane Jill wrote:

 In article <cgkkeq$925$1 digitaldaemon.com>, Ben Hinkle says...
 
The manual and Walter's post are consistent. The GC runs the destructor
when it collects an object but it isn't guaranteed to collect all
unreferenced objects (due to ambiguous references etc).

Okay, but presumably it *is* guaranteed to collect everything not yet collected (and to call all remaining destructors) at program exit. Right?

nope - at program exit it scans static data for roots so any object reference stored in a static variable will never get collected. I don't know if removing static data from the final scan would cause problems.
Aug 26 2004
next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cgknav$ab6$1 digitaldaemon.com>, Ben Hinkle says...

 Okay, but presumably it *is* guaranteed to collect everything not yet
 collected (and to call all remaining destructors) at program exit. Right?


nope - at program exit it scans static data for roots so any object
reference stored in a static variable will never get collected. I don't
know if removing static data from the final scan would cause problems.

So if I have a static variable: # int n; and its value happens by coincidence to be the run-time address of an instance of some class or other, then the destructor of that instance will /never/ get called? Not even at program exit? This is worrying. I need a workaround, and I'm going to have to think hard to come up with it. (Fortunately it's not urgent). Jill
Aug 26 2004
next sibling parent reply pragma <pragma_member pathlink.com> writes:
In article <cgkpfo$b62$1 digitaldaemon.com>, Arcane Jill says...
This is worrying. I need a workaround, and I'm going to have to think hard to
come up with it. (Fortunately it's not urgent).

As distressing as this inconsistency with destructors is, I managed to compose a decent workaround last night in a fit of inspiration. http://svn.dsource.org/svn/projects/dsp/trunk/misc/systemFinalizer.d I'm basically eating my own dog food here: I suggested some time ago that we be given a way to flag objects for explicit deletion on exit if nothing else. :) There are other bits in that directory as well all are welcome to critique. These are common pieces that have fallen out of DSP that are useful outside the scope of the project. The idea is that the SystemFinalizer is a singleton that is instantiated and controlled via a separate AutoSystemFinalizer in main() (or your daemon thread if you prefer). This way you have a single point of access from which to mark objects for deletion once the AutoSystemFinalizer goes out of scope. Simply call SystemFinalizer.register(obj) anywhere in your app to flag an object, and off you go. A second register() method exists that allows for a custom finalizer function, just in case the first form isn't flexible enough. -Pragma [[ EricAnderton at (yummy!) yahoo.com ]]
Aug 26 2004
next sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
nifty. you could also use module constructors and destructors instead of
auto class ctors and dtors. Module ctors and dtors will always run. It
probably will be about the same - I guess the difference is that user code
doesn't have to worry about doing anything in main().

"pragma" <pragma_member pathlink.com> wrote in message
news:cgku7g$dhv$1 digitaldaemon.com...
 In article <cgkpfo$b62$1 digitaldaemon.com>, Arcane Jill says...
This is worrying. I need a workaround, and I'm going to have to think


come up with it. (Fortunately it's not urgent).

As distressing as this inconsistency with destructors is, I managed to

 decent workaround last night in a fit of inspiration.

 http://svn.dsource.org/svn/projects/dsp/trunk/misc/systemFinalizer.d

 I'm basically eating my own dog food here: I suggested some time ago that

 given a way to flag objects for explicit deletion on exit if nothing else.

 There are other bits in that directory as well all are welcome to

 These are common pieces that have fallen out of DSP that are useful

 scope of the project.

 The idea is that the SystemFinalizer is a singleton that is instantiated

 controlled via a separate AutoSystemFinalizer in main() (or your daemon

 if you prefer).  This way you have a single point of access from which to

 objects for deletion once the AutoSystemFinalizer goes out of scope.

 Simply call SystemFinalizer.register(obj) anywhere in your app to flag an
 object, and off you go.  A second register() method exists that allows for

 custom finalizer function, just in case the first form isn't flexible

 -Pragma
 [[ EricAnderton at (yummy!) yahoo.com ]]

Aug 26 2004
parent pragma <pragma_member pathlink.com> writes:
In article <cgkvmr$ec5$1 digitaldaemon.com>, Ben Hinkle says...
nifty. you could also use module constructors and destructors instead of
auto class ctors and dtors. Module ctors and dtors will always run. It
probably will be about the same - I guess the difference is that user code
doesn't have to worry about doing anything in main().

Well, the main advantage here is not /if/ the destructor will run, but rather /how/. In this case, use of a destructor registry (like SystemFinalizer) guarantees that a registered destructor (via a call to 'delete') will run while all other referenced objects are still alive. This is in sharp contrast to how D may-or-may-not run all outsitanding finalizers and delete their corresponding objects one-by-one per finalizer. Another way to look at it is like an event callback mechanism. SystemFinalizer is analagous to a "System Shutdown Event" that warns registered listeners to clean up. In retrospect, the latter would probably be more useful. -Pragma [[ EricAnderton at (D needs events) yahoo.com ]]
Aug 26 2004
prev sibling parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"pragma" <pragma_member pathlink.com> wrote in message
news:cgku7g$dhv$1 digitaldaemon.com...
 In article <cgkpfo$b62$1 digitaldaemon.com>, Arcane Jill says...
This is worrying. I need a workaround, and I'm going to have to think hard to
come up with it. (Fortunately it's not urgent).

As distressing as this inconsistency with destructors is, I managed to compose a decent workaround last night in a fit of inspiration. http://svn.dsource.org/svn/projects/dsp/trunk/misc/systemFinalizer.d

Why not just have the instance in a static initialiser? (Need to force linking via explicit mention of class??)
 I'm basically eating my own dog food here: I suggested some time ago that we be
 given a way to flag objects for explicit deletion on exit if nothing else. :)

 There are other bits in that directory as well all are welcome to critique.
 These are common pieces that have fallen out of DSP that are useful outside the
 scope of the project.

 The idea is that the SystemFinalizer is a singleton that is instantiated and
 controlled via a separate AutoSystemFinalizer in main() (or your daemon thread
 if you prefer).  This way you have a single point of access from which to mark
 objects for deletion once the AutoSystemFinalizer goes out of scope.

 Simply call SystemFinalizer.register(obj) anywhere in your app to flag an
 object, and off you go.  A second register() method exists that allows for a
 custom finalizer function, just in case the first form isn't flexible enough.

 -Pragma
 [[ EricAnderton at (yummy!) yahoo.com ]]

Aug 26 2004
parent pragma <pragma_member pathlink.com> writes:
In article <cglhec$nn3$1 digitaldaemon.com>, Matthew says...

"pragma" <pragma_member pathlink.com> wrote in message
news:cgku7g$dhv$1 digitaldaemon.com...
 http://svn.dsource.org/svn/projects/dsp/trunk/misc/systemFinalizer.d

Why not just have the instance in a static initialiser? (Need to force linking via explicit mention of class??)

# // so I should do this? # static this(){ # _instance = new SystemFinalizer(); # } You're right. That would make a better singleton wouldn't it? -Pragma [[ EricAnderton at (_instance) yahoo dot com ]]
Aug 26 2004
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <cgkpfo$b62$1 digitaldaemon.com>, Arcane Jill says...
In article <cgknav$ab6$1 digitaldaemon.com>, Ben Hinkle says...

 Okay, but presumably it *is* guaranteed to collect everything not yet
 collected (and to call all remaining destructors) at program exit. Right?


nope - at program exit it scans static data for roots so any object
reference stored in a static variable will never get collected. I don't
know if removing static data from the final scan would cause problems.

So if I have a static variable: # int n; and its value happens by coincidence to be the run-time address of an instance of some class or other, then the destructor of that instance will /never/ get called? Not even at program exit? This is worrying. I need a workaround, and I'm going to have to think hard to come up with it. (Fortunately it's not urgent).

Does the final GC pass occur after module destructors are run? If so, a slightly lame workaround would be to have such destructors default initialize all statics in the module. ie. static ~this() { n = n.init; } Basically saying that every module is effectively responsible for picking up its toys when it's done playing with them. Sean
Aug 26 2004
prev sibling parent reply Nick <Nick_member pathlink.com> writes:
In article <cgknav$ab6$1 digitaldaemon.com>, Ben Hinkle says...
Arcane Jill wrote:
 [...]
 Okay, but presumably it *is* guaranteed to collect everything not yet
 collected (and to call all remaining destructors) at program exit. Right?

nope - at program exit it scans static data for roots so any object reference stored in a static variable will never get collected. I don't know if removing static data from the final scan would cause problems.

Are you sure about this? Why does it scan static data but not dynamic? Nick
Aug 29 2004
parent Ben Hinkle <bhinkle4 juno.com> writes:
Nick wrote:

 In article <cgknav$ab6$1 digitaldaemon.com>, Ben Hinkle says...
Arcane Jill wrote:
 [...]
 Okay, but presumably it *is* guaranteed to collect everything not yet
 collected (and to call all remaining destructors) at program exit.
 Right?

nope - at program exit it scans static data for roots so any object reference stored in a static variable will never get collected. I don't know if removing static data from the final scan would cause problems.

Are you sure about this? Why does it scan static data but not dynamic? Nick

The static data is scanned for roots into the dynamic data. Once a root is found it continues to mark anything reachable from that point. Normally it looks for root in the static data and on all stacks, but since the program is exiting the stacks are skipped. Users can add arbitrary chunks of memory to list of things to scan for roots, too. Once it is done marking all reachable dynamic data it garbage collects everything else. Does that help? -Ben
Aug 29 2004
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <cgkcsq$5fi$1 digitaldaemon.com>, Arcane Jill says...
In article <cgj8qb$2ler$1 digitaldaemon.com>, pragma says...

The garbage collector is not guaranteed to run the destructor for all
unreferenced objects.


Please see THIS POST from Walter, and the discussion before and after it: digitalmars.D/3488 In particular, Walter said: "I must have misspoke, because when the gc collects an object, it *does* run the destructor." (Walter's emphasis on *does*). I have security apps planned which rely on Walter's statement being true. Where is this "not guaranteed" quote in the manual? I couldn't find it. Basically - help! Who's right? Walter, or the manual? Walter, please could you clarify (again), and fix the manual if it's wrong.

IIRC this came up in the original discussion and that was Walter's reply. The spec should likely be corrected. Sean
Aug 26 2004
prev sibling parent "Matthew" <admin.hat stlsoft.dot.org> writes:
"Ben Hinkle" <bhinkle mathworks.com> wrote in message
news:cgisdb$2g83$1 digitaldaemon.com...
 "pragma" <pragma_member pathlink.com> wrote in message
 news:cgif07$29ev$1 digitaldaemon.com...
 In article <cgia4c$273m$1 digitaldaemon.com>, Ben Hinkle says...
 Honestly, my expectation was that during the final collection (before

 exit) all the finalizers would be run *first* before any memory is



That makes sense - though it would cause two loops over the garbage


of one. Also no-one has brought up the cases when garbage can be brought
back to life. That's one reason why Java's finalizers are frowned upon -
they can bring back dead references and generally cause the GC to jump
through hoops to make sure it isn't cleaning up live data.

I see what you mean. Granted, there's nothing keeping a destructor from creating *more* data on the heap in a growing spiral of allocations. But

 really don't see that as any more a problem as infinite recursion or
 mutually-referenced functions (like in the invariant 'bug' menioned

 this NG).  But I strongly feel that reducing destructors to playing with

 scalar types (as we've discovered that object references are all "weak" in

 present D implementation during the final collect) reduces their

 near uselessness.

Destructors should only be used to release external resources - like file handles, malloc'ed memory etc. To quote from the D documentation on the section about destructors: <quote> When the garbage collector calls a destructor for an object of a class that has members that are references to garbage collected objects, those references are no longer valid. This means that destructors cannot reference sub objects. This rule does not apply to auto objects or objects deleted with the DeleteExpression. The garbage collector is not guaranteed to run the destructor for all unreferenced objects. Furthermore, the order in which the garbage collector calls destructors for unreference objects is not specified. <\quote>

That does preclude the use of any kind of resource-management object being used in composition, does it not? In which case, D's OO is seriously hamstrung. Now if D would allow embedding of auto members (of auto or non-auto classes), then we'd have an answer. (Sounds suspiciously like C++, of course, but then this is something C++ got very right.)
 It would be nice if we lived in a world where the GC could be fast and run
 destructors "nicely" but given the choice I would go for GC performance.
 Destructors should be a very weak construct. You never know when or if they
 will run so most of the time relying on them is a bug in disguise. Google
 around for web pages about Finalizers and Java and you'll get an idea how
 everyone in the Java world says not to use them. Remember all this
 destructor running happens on every GC pass, not just program exit.

I agree with all that you say, but just want to mention that I think your emphasis is a little off. They're a deprecated technique for correctness rather than efficiency, surely?
Aug 26 2004
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <cgguse$1jab$1 digitaldaemon.com>, antiAlias says...
Yes, that would be helpful. But it's not /always/ practical to avoid
touching other objects during destruction, though perhaps 'desirable'.

I bet this works: void main(){ auto Bar b = new Bar(); auto Foo f = new Foo(); b.event = &(f.notify); }
And it's not an obvious error either, since one could hardly be blamed for
thinking the delegate holds a reference to its enclosing class instance
(it's a delegate; not a function). If so, then the instance should still be
live as long as the delegate is referenced ... yes? Apparently that's not
how it works. Sure looks like a bug.

The problem is that once execution has left main(), all bets are off. So what's happening here is that the references to Foo and Bar are being lost when main() exits and then the gc is run. I imagine more common problems with this might possibly be addressed by having the gc make multiple passes on application exit, but I have a feeling that this would be shaky at best. I have no problem with objects referencing other objects on destruction, but the user should be aware that this is unreliable in this particular situation. Sean
Aug 25 2004