www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Destructor nonsense on dlang.org

reply =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
Hi,

http://dlang.org/class.html#Destructor

"The garbage collector is not guaranteed to run the destructor for all 
unreferenced objects."

What the *hell*? So resources are allowed to arbitrarily leak and the 
programmer has to actually expect this to happen?

I really, really hope that this is a documentation error or early design 
decision that has since been rectified but with lack of documentation 
updates.

-- 
Alex Rřnne Petersen
alex lycus.org
http://lycus.org
May 24 2012
next sibling parent reply "Thor" <Thor hotmail.com> writes:
On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen 
wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor 
 for all unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak 
 and the programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or 
 early design decision that has since been rectified but with 
 lack of documentation updates.

use "clear", or "scope (exit)" or "structs" or scoped!... etc. There could always be a false reference... so you cannot depend on automatically releasing resources in a class destructor.
May 24 2012
parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 24-05-2012 14:33, Thor wrote:
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

use "clear", or "scope (exit)" or "structs" or scoped!... etc.

I know.
 There could always be a false reference... so you cannot depend on
 automatically releasing resources in a class destructor.

False pointers have nothing to do with it. The GC should free and finalize all objects on shutdown, meaning the finalizer runs *sooner or later*. If this is the case (which I do believe it is), then the docs are very wrong. -- Alex Rønne Petersen alex lycus.org http://lycus.org
May 24 2012
parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 24-05-2012 14:48, Thor wrote:
 On Thursday, 24 May 2012 at 12:38:45 UTC, Alex Rønne Petersen wrote:
 On 24-05-2012 14:33, Thor wrote:
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

use "clear", or "scope (exit)" or "structs" or scoped!... etc.

I know.
 There could always be a false reference... so you cannot depend on
 automatically releasing resources in a class destructor.

False pointers have nothing to do with it. The GC should free and finalize all objects on shutdown, meaning the finalizer runs *sooner or later*. If this is the case (which I do believe it is), then the docs are very wrong.

__gshared uint my_false_ptr; even if we are shutting down, the static references doesn't disappear... or did I miss something?

The GC should (and probably does) assume at shutdown that all objects are unreferenced, and therefore reclaim and finalize them. -- Alex Rønne Petersen alex lycus.org http://lycus.org
May 24 2012
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

They may refer to one another. Andrei
May 24 2012
next sibling parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 24-05-2012 16:54, Andrei Alexandrescu wrote:
 On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

They may refer to one another. Andrei

Doesn't matter: Nothing is guaranteed about order of finalization (and this is reasonable). Thus, the finalizers can be run in any arbitrary order. The important point here is that they are run *at all*. -- Alex Rønne Petersen alex lycus.org http://lycus.org
May 24 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/24/12 9:57 AM, Alex Rønne Petersen wrote:
 On 24-05-2012 16:54, Andrei Alexandrescu wrote:
 On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

They may refer to one another. Andrei

Doesn't matter: Nothing is guaranteed about order of finalization (and this is reasonable). Thus, the finalizers can be run in any arbitrary order. The important point here is that they are run *at all*.

It does matter because a destructor may use an object that has just been destroyed. Andrei
May 24 2012
next sibling parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 24-05-2012 18:06, Andrei Alexandrescu wrote:
 On 5/24/12 9:57 AM, Alex Rønne Petersen wrote:
 On 24-05-2012 16:54, Andrei Alexandrescu wrote:
 On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

They may refer to one another. Andrei

Doesn't matter: Nothing is guaranteed about order of finalization (and this is reasonable). Thus, the finalizers can be run in any arbitrary order. The important point here is that they are run *at all*.

It does matter because a destructor may use an object that has just been destroyed. Andrei

No, the docs specifically state that this is invalid (and it currently throws InvalidMemoryOperationError in most cases). Whether it *should* be allowed is arguable, but it isn't currently, both in docs and impl. -- Alex Rønne Petersen alex lycus.org http://lycus.org
May 24 2012
parent reply ponce <spam spam.org> writes:
Le 24/05/2012 18:10, Alex Rønne Petersen a écrit :
 On 24-05-2012 18:06, Andrei Alexandrescu wrote:
 It does matter because a destructor may use an object that has just been
 destroyed.

 Andrei

No, the docs specifically state that this is invalid (and it currently throws InvalidMemoryOperationError in most cases). Whether it *should* be allowed is arguable, but it isn't currently, both in docs and impl.

I really had a hard time to believe it when #D told me so, but there is no guaranteed order of destruction and as you cannot relies on members still being alive in a class destructor. All of it can happen when making absolutely no cycles in the object graph. What I do now is having a close function for each class which hold a non-memory resource. This is writtent in TDPL, but I wish I was told earlier :)
May 24 2012
next sibling parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 24-05-2012 19:47, Tove wrote:
 On Thursday, 24 May 2012 at 17:06:19 UTC, ponce wrote:
 I really had a hard time to believe it when #D told me so, but there
 is no guaranteed order of destruction and as you cannot relies on
 members still being alive in a class destructor.
 All of it can happen when making absolutely no cycles in the object
 graph.

 What I do now is having a close function for each class which hold a
 non-memory resource.

 This is writtent in TDPL, but I wish I was told earlier :)

http://dlang.org/class.html#destructors "This rule does not apply to auto objects or objects deleted with the DeleteExpression, as the destructor is not being run by the garbage collector, meaning all references are valid." i.e. non gc resources are fine... and it's also fine if you call clear()... it's only a problem if you rely on automatic collection and reference a object... so there's no need for close, as clear() will do the trick.

I would strongly advise against that, because a missed clear() means your finalizer may be run by the runtime's finalization machinery, and thus invalidate any invariants you were relying on in the finalizer. -- Alex Rønne Petersen alex lycus.org http://lycus.org
May 24 2012
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-05-24 21:46, foobar wrote:

 Looks to me like an issue with separation of concerns. I think that
 dtors need to only provide deterministic management of resources and not
 affect GC algorithms:
 1. classes should *not* have dtors at all.
 2. struct values should *not* be gc managed [*].

 Composition of classes and structs should be handled as follows:
 1. If a class contains a pointer to a struct it doesn't scan it in a GC
 cycle. The runtime can provide a hook so that structs could register a
 callback for RC purposes.
 2. When a class contains a strcut value, they share the same lifetime
 thus the GC will call the struct's dtor when the object is collected.

How is that any different than having destructors for classes. -- /Jacob Carlborg
May 25 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-05-25 11:02, foobar wrote:

 It makes the call order deterministic like in C++.

 e.g.
 class Foo {}

 struct A {
 resource r;
 ~this() { release(r); }
 }

 struct B {
 A* a;
 Foo foo;
 ~this() { delete a; } // [1]
 }

I though we were talking about classes holding structs: class B { A* a; Foo foo; ~this() { delete a; } } In this case you don't know when/if the destructor of B is called. It doesn't help to wrap it in a struct, you could just have put it directly in A. Is that correct?
 Lets look at point [1]:
 The "foo" instance is "managed" by the GC since the only resource it
 holds is memory. The "a" member wraps some "non-managed" resource (e.g.
 file descriptor) and in this model is still valid, thus allows me to
 deterministically dispose of it as in c++.

Ok, but if B is a class?
 This can be simply checked at compile-time - you can only reference non
 class instance members in the destructor, so adding a "delete foo;"
 statement at point [1] simply won't compile.

If you have a pointer to a struct you don't know how it was created. It's possible it's been created with "new", which means the garbage collector needs to delete it. -- /Jacob Carlborg
May 25 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-05-25 14:05, foobar wrote:

 If you have a pointer to a struct you don't know how it was created.
 It's possible it's been created with "new", which means the garbage
 collector needs to delete it.

let's say we add two classes: class FooA { A a; } class FooPA { A* pa; } For the first case, both the class and the struct share the same lifetime thus when an instance of FooA is GC-ed, the GC would call A's d-tor and allow it to do what-ever (self) cleaning it requires. This means the d-tor will always be called.

Is that the cases even if the destructor of FooA isn't called?
 For the second case, The GC will only scan "pa" to find inner class
 instances but will *not* handle the struct value itself.
 In order to clean what "pa" points to, you need to explicitly call the
 destructor yourself.

Are you saying that the GC won't collect a struct allocated with "new"? http://dlang.org/expression.html#NewExpression "NewExpressions are used to allocate memory on the garbage collected heap...". I though that everything allocated via the GC was also collected by the GC.
 One way to do this would be to register a callback
 with the GC to get notified when an instance of FooPA is collected and
 inside the callback function maintain a reference-counter.
 This also means that if you allocate a struct value on the heap via
 "new" you are responsible to call delete _yourself_ and the gc will not
 call it for you.
 I think that loosing this small convenience is worth it - we gay more
 orthogonal semantics that are easier to reason about.

-- /Jacob Carlborg
May 26 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-05-26 14:28, foobar wrote:

 Huh? In my model FooA has no destructor.

Hm, right. But if a destructor of a class isn't guaranteed to be called, how can it guarantee that the struct's destructor will be called?
 I indeed propose that structs allocated with "new" will be put in region
 of the heap *not* managed by the GC. It's a tiny price to pay to get
 more orthogonal semantics which are easier to reason about.
 Please note that this only affects code that directly uses pointers
 which is not common in D and is geared towards more advanced use cases
 where the programmer will likely want to manage the memory explicitly
 anyway.

I see. -- /Jacob Carlborg
May 26 2012
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/25/12 12:07 AM, Mehrdad wrote:
 Now, there are two ways a FileStream can get destroyed:

 1. Through a manual call to FileStream.Dispose(). In this case, all
 embedded objects (e.g. SafeFileHandle) are *guaranteed* to be valid, so
 we simply flush the file and call SafeFileHandle.Dispose() to dispose of
 the managed resources, and then dispose of all the unmanaged resources
 (which are primitive fields, guaranteed to be accessible). Furthermore,
 the object suppresses its own finalizer.

 2. Through a garbage-collected call to ~FileStream(). In this case, the
 managed resources such as SafeFileHandle will be (or is already)
 destroyed SEPARATELY, and so we do _NOT_ access them. We ONLY dispose of
 the unmanaged resources, if any, and let the managed resources take care
 of themselves.

What happens in C# if an object A that has a field referring to object B, and the object B has in turn a field referring to object A? That is: class C { C another; ~this() { writeln(another.another); } } void main() { auto a = new C; auto b = new C; a.another = b; b.another = a; } What happens then? Will the GC nullify references to destroyed objects, or will it put them in a zombie state? Thanks, Andrei
May 25 2012
next sibling parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 25-05-2012 16:35, Andrei Alexandrescu wrote:
 On 5/25/12 12:07 AM, Mehrdad wrote:
 Now, there are two ways a FileStream can get destroyed:

 1. Through a manual call to FileStream.Dispose(). In this case, all
 embedded objects (e.g. SafeFileHandle) are *guaranteed* to be valid, so
 we simply flush the file and call SafeFileHandle.Dispose() to dispose of
 the managed resources, and then dispose of all the unmanaged resources
 (which are primitive fields, guaranteed to be accessible). Furthermore,
 the object suppresses its own finalizer.

 2. Through a garbage-collected call to ~FileStream(). In this case, the
 managed resources such as SafeFileHandle will be (or is already)
 destroyed SEPARATELY, and so we do _NOT_ access them. We ONLY dispose of
 the unmanaged resources, if any, and let the managed resources take care
 of themselves.

What happens in C# if an object A that has a field referring to object B, and the object B has in turn a field referring to object A? That is: class C { C another; ~this() { writeln(another.another); } } void main() { auto a = new C; auto b = new C; a.another = b; b.another = a; } What happens then? Will the GC nullify references to destroyed objects, or will it put them in a zombie state? Thanks, Andrei

This is called resurrection: http://msdn.microsoft.com/en-us/magazine/bb985010.aspx (scroll down to Resurrection) -- Alex Rønne Petersen alex lycus.org http://lycus.org
May 25 2012
next sibling parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 25-05-2012 17:53, Mehrdad wrote:
 On Friday, 25 May 2012 at 14:38:29 UTC, Alex Rønne Petersen wrote:
 This is called resurrection:
 http://msdn.microsoft.com/en-us/magazine/bb985010.aspx (scroll down to
 Resurrection)

Ah, yes, you're completely right; I missed this fact. Apparently under these conditions, you _can_ resurrect objects, but it's bad practice (and unnecessary) in most situations. Andrei: The reason this is allowed is that finalization is _separate_ from garbage collection in .NET. So an object can be finalized and yet still not GC'd. Or its finalizer might be suppressed, allowing it to get GC'd directly. This allows for many possibilities, although you don't usually need them.

This is, in fact, how most GCs other than D's work. :) -- Alex Rønne Petersen alex lycus.org http://lycus.org
May 25 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, May 25, 2012 17:53:45 Mehrdad wrote:
  Andrei: The reason this is allowed is that finalization is
 _separate_ from garbage collection in .NET. So an object can be
 finalized and yet still not GC'd. Or its finalizer might be
 suppressed, allowing it to get GC'd directly. This allows for
 many possibilities, although you don't usually need them.

Finalization _can_ be separate from the GC in D thanks to clear, but it does normally occur as part of a collection cycle. - Jonathan M Davis
May 25 2012
prev sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Friday, 25 May 2012 at 19:30:35 UTC, Jonathan M Davis wrote:
 On Friday, May 25, 2012 17:53:45 Mehrdad wrote:
  Andrei: The reason this is allowed is that finalization is
 _separate_ from garbage collection in .NET. So an object can be
 finalized and yet still not GC'd. Or its finalizer might be
 suppressed, allowing it to get GC'd directly. This allows for
 many possibilities, although you don't usually need them.

Finalization _can_ be separate from the GC in D thanks to clear, but it does normally occur as part of a collection cycle. - Jonathan M Davis

Uhm... sure... I wasn't really talking about D, so I'm not sure what you mean. But, comparing to D: I'm _not_ talking about the fact that you can call the finalizer manually. That has _nothing_ to do with the "separation" I was referring to (even though it's nevertheless necessary for separating the GC from the finalization queue). I'm talking about the fact that, if an object has a finalizer, its finalization stage is SEPARATE from (and obviously, before) the GC stage. In other words, there can be TWO passes over all objects that are going to be GC'd: 1. Unreachable finalizable objects are finalized, and their finalizers are suppressed. 2. Unreachable objects with no finalizers are GC'd. Therefore, resurrection is possible because after an object goes through stage 1, it may no longer be eligible for stage 2 (it may have strong references). Note that this means merely _having_ a finalizer causes an object to take 2 passes to be GC'd, instead of 1 -- even if the finalizer is empty. Also notice that NOTHING is left in an undefined state, and yet everything is guaranteed to be reclaimed at some point. And circular references cause no problems whatsoever, because if they're unreachable from the GC root, no one cares if they have references to each other. Andrei: Does that make sense?
May 25 2012
prev sibling parent deadalnix <deadalnix gmail.com> writes:
Le 25/05/2012 16:35, Andrei Alexandrescu a Ă©crit :
 On 5/25/12 12:07 AM, Mehrdad wrote:
 Now, there are two ways a FileStream can get destroyed:

 1. Through a manual call to FileStream.Dispose(). In this case, all
 embedded objects (e.g. SafeFileHandle) are *guaranteed* to be valid, so
 we simply flush the file and call SafeFileHandle.Dispose() to dispose of
 the managed resources, and then dispose of all the unmanaged resources
 (which are primitive fields, guaranteed to be accessible). Furthermore,
 the object suppresses its own finalizer.

 2. Through a garbage-collected call to ~FileStream(). In this case, the
 managed resources such as SafeFileHandle will be (or is already)
 destroyed SEPARATELY, and so we do _NOT_ access them. We ONLY dispose of
 the unmanaged resources, if any, and let the managed resources take care
 of themselves.

What happens in C# if an object A that has a field referring to object B, and the object B has in turn a field referring to object A? That is: class C { C another; ~this() { writeln(another.another); } } void main() { auto a = new C; auto b = new C; a.another = b; b.another = a; } What happens then? Will the GC nullify references to destroyed objects, or will it put them in a zombie state? Thanks, Andrei

Here is what I suggest : 1/ what is in ~this stay in ~this. You cannot escape a reference to this or anything reached throw this of ~this. In other terms, the hidden this parameter get scope storage class. This avoid resurrection (something we really want, it have caused much trouble in java). 2/ The GC will call all finalizers on objects (here a and b) when they are garbage. Finalizers are called in undefined order. 3/ The GC will recycle the memory after all finalizers ran. In the example above, both writeln will executed and then, both object's memory will be recycled.
May 26 2012
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 24/05/2012 16:54, Andrei Alexandrescu a Ă©crit :
 On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

They may refer to one another. Andrei

So what ? Each GC passes must, mark object that have to be removed, call finalizer on them all, THEN recycle memory. So « zombie » object can still refer to one another in finalization. The real problem is resurrection, which should be 100% forbiden and this must be enforced by the language (ie, the scopeness of this parameter is something important).
May 24 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/24/12 10:27 AM, deadalnix wrote:
 Le 24/05/2012 16:54, Andrei Alexandrescu a Ă©crit :
 On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

They may refer to one another. Andrei

So what ? Each GC passes must, mark object that have to be removed, call finalizer on them all, THEN recycle memory. So « zombie » object can still refer to one another in finalization.

This is possible but not trivial as the state of zombie objects must be properly defined. Often such objects will fail their invariant (a reasonable state of a zombie object is with the correct vtable, no mutex, and all fields in the pre-constructor state).
 The real problem is resurrection, which should be 100% forbiden and this
 must be enforced by the language (ie, the scopeness of this parameter is
 something important).

As one aspect, calls to new should fail during destruction. Andrei
May 24 2012
parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 24-05-2012 18:10, Andrei Alexandrescu wrote:
 On 5/24/12 10:27 AM, deadalnix wrote:
 Le 24/05/2012 16:54, Andrei Alexandrescu a Ă©crit :
 On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

They may refer to one another. Andrei

So what ? Each GC passes must, mark object that have to be removed, call finalizer on them all, THEN recycle memory. So « zombie » object can still refer to one another in finalization.

This is possible but not trivial as the state of zombie objects must be properly defined. Often such objects will fail their invariant (a reasonable state of a zombie object is with the correct vtable, no mutex, and all fields in the pre-constructor state).
 The real problem is resurrection, which should be 100% forbiden and this
 must be enforced by the language (ie, the scopeness of this parameter is
 something important).

As one aspect, calls to new should fail during destruction. Andrei

Finalization happens once the world has been resumed, meaning GC allocation (and even explicit deallocation) should be perfectly safe. This is absolutely essential: Finalization models where finalizers run in a paused world are doomed to fail miserably. -- Alex Rønne Petersen alex lycus.org http://lycus.org
May 24 2012
prev sibling parent Artur Skawina <art.08.09 gmail.com> writes:
On 05/26/12 13:35, Jacob Carlborg wrote:
 On 2012-05-25 14:05, foobar wrote:
 
 If you have a pointer to a struct you don't know how it was created.
 It's possible it's been created with "new", which means the garbage
 collector needs to delete it.

let's say we add two classes: class FooA { A a; } class FooPA { A* pa; } For the first case, both the class and the struct share the same lifetime thus when an instance of FooA is GC-ed, the GC would call A's d-tor and allow it to do what-ever (self) cleaning it requires. This means the d-tor will always be called.

Is that the cases even if the destructor of FooA isn't called?
 For the second case, The GC will only scan "pa" to find inner class
 instances but will *not* handle the struct value itself.
 In order to clean what "pa" points to, you need to explicitly call the
 destructor yourself.

Are you saying that the GC won't collect a struct allocated with "new"? http://dlang.org/expression.html#NewExpression "NewExpressions are used to allocate memory on the garbage collected heap...". I though that everything allocated via the GC was also collected by the GC.

Everything allocated is collected, though not necessarily destructed. In case of structs, the GC currently doesn't know about the dtors, so those are never called. import std.stdio; struct S { ~this() { writeln("never called"); } } void main() { S* sp; while (1) sp = new S(); } Right now you have to use a class if you need dtors for heap allocated objects. artur
May 26 2012
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-05-24 12:21:01 +0000, Alex Rřnne Petersen <alex lycus.org> said:

 Hi,
 
 http://dlang.org/class.html#Destructor
 
 "The garbage collector is not guaranteed to run the destructor for all 
 unreferenced objects."
 
 What the *hell*? So resources are allowed to arbitrarily leak and the 
 programmer has to actually expect this to happen?
 
 I really, really hope that this is a documentation error or early 
 design decision that has since been rectified but with lack of 
 documentation updates.

I think it means that objects not collected when the program terminates will never be, and thus the destructor will not be called. There's also the issue of false pointers that can prevent some objects from being collected. More generally, you can't really count on the destructor being called because the GC is free to decide when to recycle memory. An implementation that never collects and always allocate new memory is a perfectly valid GC implementation, even though it might not be very practical in most cases. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 24 2012
next sibling parent reply =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
On 24-05-2012 14:43, Michel Fortin wrote:
 On 2012-05-24 12:21:01 +0000, Alex Rřnne Petersen <alex lycus.org> said:

 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

I think it means that objects not collected when the program terminates will never be, and thus the destructor will not be called.

That's silly. Microsoft .NET and Mono have been running finalizers on runtime shutdown for a long time without problems.
 There's also the issue of false pointers that can prevent some objects
 from being collected.

Right, but I think if we guarantee finalization at shutdown, that shouldn't matter.
 More generally, you can't really count on the destructor being called
 because the GC is free to decide when to recycle memory. An
 implementation that never collects and always allocate new memory is a
 perfectly valid GC implementation, even though it might not be very
 practical in most cases.

Even such an implementation should free all memory at shutdown, and, at that point, run finalizers. -- Alex Rřnne Petersen alex lycus.org http://lycus.org
May 24 2012
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-05-24 14:35:45 +0000, Alex Rřnne Petersen <alex lycus.org> said:

 On 24-05-2012 14:43, Michel Fortin wrote:
 I think it means that objects not collected when the program terminates
 will never be, and thus the destructor will not be called.

That's silly. Microsoft .NET and Mono have been running finalizers on runtime shutdown for a long time without problems.
 There's also the issue of false pointers that can prevent some objects
 from being collected.

Right, but I think if we guarantee finalization at shutdown, that shouldn't matter.

.NET is a virtual machine which has total control of all the code that runs. D has to work with the C runtime and other non-D code that might use D code more or less directly. The interesting question is *how* do they do it without causing thread-safety issues? Perhaps they wait until all threads have been terminated, or perhaps they're unsafe too. Would doing the same be appropriate for D? Does this mean we can't use anything using static and global variables in destructors because they might have been finalized? If so, how does the compiler detects this? If there's a way and it is not overly costly in performance, it might be a good idea. But we're not in a virtual machine, there are more constrains we must abide to and we might have to make different compromises. Enlarging the scope of all this, there is already some impending problems with how destructors are handled in D that can easily create low-level races. And this applies to struct destructors too, when the struct is put on the heap or is a member of an object. We're in the need of a global solution for all this. <http://d.puremagic.com/issues/show_bug.cgi?id=4621> <http://d.puremagic.com/issues/show_bug.cgi?id=4624>
 More generally, you can't really count on the destructor being called
 because the GC is free to decide when to recycle memory. An
 implementation that never collects and always allocate new memory is a
 perfectly valid GC implementation, even though it might not be very
 practical in most cases.

Even such an implementation should free all memory at shutdown, and, at that point, run finalizers.

Well, not according to the current spec. It could make sense to do so, although I'm not sure. Freeing "external" resources can mean many things: if we're talking about externally allocated memory, open files, sockets, mutexes, etc., there's no need to finalize that at the end of the program: the OS will do the cleanup for us. If we're talking about advisory locks on files, then there's definitely a need to clear the lock, although I'm not sure it makes sense to make the release dependent on a non-deterministic GC. So I'm curious, what resource are we trying to free here? -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 24 2012
parent =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
On 24-05-2012 17:18, Michel Fortin wrote:
 On 2012-05-24 14:35:45 +0000, Alex Rřnne Petersen <alex lycus.org> said:

 On 24-05-2012 14:43, Michel Fortin wrote:
 I think it means that objects not collected when the program terminates
 will never be, and thus the destructor will not be called.

That's silly. Microsoft .NET and Mono have been running finalizers on runtime shutdown for a long time without problems.
 There's also the issue of false pointers that can prevent some objects
 from being collected.

Right, but I think if we guarantee finalization at shutdown, that shouldn't matter.

.NET is a virtual machine which has total control of all the code that runs. D has to work with the C runtime and other non-D code that might use D code more or less directly.

So does .NET. It just does it with some trampoline magic. Non-D code just has to call thread_attachThis()/thread_detachThis() and friends.
 The interesting question is *how* do they do it without causing
 thread-safety issues? Perhaps they wait until all threads have been
 terminated, or perhaps they're unsafe too. Would doing the same be
 appropriate for D? Does this mean we can't use anything using static and
 global variables in destructors because they might have been finalized?
 If so, how does the compiler detects this?

The CLR waits for all threads to have shut down (including daemon threads). That's not equal to all static/__gshared data being nulled out, mind you. All data is cleared out once all threads, including the finalizer thread, have been terminated one way or another. And yes, it is thread safe.
 If there's a way and it is not overly costly in performance, it might be
 a good idea. But we're not in a virtual machine, there are more
 constrains we must abide to and we might have to make different
 compromises.

Don't need a virtual machine. We already have the infrastructure to do it in druntime, we just need to make the lifetime and GC code respect the C#/CLR finalization semantics if we want to go with those.
 Enlarging the scope of all this, there is already some impending
 problems with how destructors are handled in D that can easily create
 low-level races. And this applies to struct destructors too, when the
 struct is put on the heap or is a member of an object. We're in the need
 of a global solution for all this.

 <http://d.puremagic.com/issues/show_bug.cgi?id=4621>
 <http://d.puremagic.com/issues/show_bug.cgi?id=4624>


 More generally, you can't really count on the destructor being called
 because the GC is free to decide when to recycle memory. An
 implementation that never collects and always allocate new memory is a
 perfectly valid GC implementation, even though it might not be very
 practical in most cases.

Even such an implementation should free all memory at shutdown, and, at that point, run finalizers.

Well, not according to the current spec.

No, but a sane implementation made for a sane spec should. ;)
 It could make sense to do so, although I'm not sure. Freeing "external"
 resources can mean many things: if we're talking about externally
 allocated memory, open files, sockets, mutexes, etc., there's no need to
 finalize that at the end of the program: the OS will do the cleanup for
 us. If we're talking about advisory locks on files, then there's
 definitely a need to clear the lock, although I'm not sure it makes
 sense to make the release dependent on a non-deterministic GC.

We just need a dispose pattern whereby explicit dispose() instructs the GC to not finalize.
 So I'm curious, what resource are we trying to free here?

None. I just came across it in the docs and found it completely insane. -- Alex Rřnne Petersen alex lycus.org http://lycus.org
May 24 2012
prev sibling parent "Tove" <tove fransson.se> writes:
On Thursday, 24 May 2012 at 15:43:57 UTC, Alex Rønne Petersen 
wrote:
 We just need a dispose pattern whereby explicit dispose() 
 instructs the GC to not finalize.

 So I'm curious, what resource are we trying to free here?

None. I just came across it in the docs and found it completely insane.

Hmm... well, as long as it's optional behavior... as in my case I actually want to go in the opposite direction... short-lived tool which claims x resources and is run once for every file... So in this case, resources should be "free:ed" _unless_ it's at program termination... as then it just slows down the shutdown procedure, the OS reclaim it faster anyway.
May 24 2012
prev sibling next sibling parent "Thor" <Thor hotmail.com> writes:
On Thursday, 24 May 2012 at 12:38:45 UTC, Alex Rønne Petersen 
wrote:
 On 24-05-2012 14:33, Thor wrote:
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen 
 wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the 
 destructor for all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak 
 and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or 
 early
 design decision that has since been rectified but with lack of
 documentation updates.

use "clear", or "scope (exit)" or "structs" or scoped!... etc.

I know.
 There could always be a false reference... so you cannot 
 depend on
 automatically releasing resources in a class destructor.

False pointers have nothing to do with it. The GC should free and finalize all objects on shutdown, meaning the finalizer runs *sooner or later*. If this is the case (which I do believe it is), then the docs are very wrong.

__gshared uint my_false_ptr; even if we are shutting down, the static references doesn't disappear... or did I miss something?
May 24 2012
prev sibling next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen 
wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor 
 for all unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak 
 and the programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or 
 early design decision that has since been rectified but with 
 lack of documentation updates.

I'm pretty sure it's the same in Java. Finalizers (a.k.a. class destructors) are practically useless.
May 24 2012
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 24/05/2012 14:54, Peter Alexander a Ă©crit :
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

I'm pretty sure it's the same in Java. Finalizers (a.k.a. class destructors) are practically useless.

Java finalizer is a pretty bad design decision. Let's not reproduce error made in Java in D's destructors.
May 24 2012
next sibling parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 24-05-2012 15:49, Steven Schveighoffer wrote:
 On Thu, 24 May 2012 09:47:23 -0400, deadalnix <deadalnix gmail.com> wrote:

 Le 24/05/2012 14:54, Peter Alexander a Ă©crit :
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

I'm pretty sure it's the same in Java. Finalizers (a.k.a. class destructors) are practically useless.

Java finalizer is a pretty bad design decision. Let's not reproduce error made in Java in D's destructors.

You actually need a finalizer if you want to have resources that aren't GC allocated. -Steve

But that doesn't mean we should have Java finalization. There are many different forms of finalization, and I do agree that Java is the worst of all of them. -- Alex Rønne Petersen alex lycus.org http://lycus.org
May 24 2012
next sibling parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 24-05-2012 16:53, Steven Schveighoffer wrote:
 On Thu, 24 May 2012 10:30:02 -0400, Alex Rønne Petersen <alex lycus.org>
 wrote:

 On 24-05-2012 15:49, Steven Schveighoffer wrote:
 On Thu, 24 May 2012 09:47:23 -0400, deadalnix <deadalnix gmail.com>
 wrote:

 Le 24/05/2012 14:54, Peter Alexander a Ă©crit :
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for
 all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

I'm pretty sure it's the same in Java. Finalizers (a.k.a. class destructors) are practically useless.

Java finalizer is a pretty bad design decision. Let's not reproduce error made in Java in D's destructors.

You actually need a finalizer if you want to have resources that aren't GC allocated. -Steve

But that doesn't mean we should have Java finalization. There are many different forms of finalization, and I do agree that Java is the worst of all of them.

I only found one definition for finalizer on wikipedia, and it fits D's definition. What I think we need is a dispose pattern for objects, like Tango has. -Steve

Just look at /usr/include/gc/gc.h (from libgc, the Boehm-Demers-Weiser GC). It has 3 (if not 4) different kinds of finalization. To be specific, the finalization I believe we need is the *_no_order behavior. This is the behavior C# and the CLR follow. -- Alex Rønne Petersen alex lycus.org http://lycus.org
May 24 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-05-24 16:53, Steven Schveighoffer wrote:

 What I think we need is a dispose pattern for objects, like Tango has.

Object.dispose in Tango is called on scope exit if the object is variable is declared "scope". "scope" is deprecated in D2. -- /Jacob Carlborg
May 24 2012
prev sibling parent deadalnix <deadalnix gmail.com> writes:
Le 24/05/2012 15:49, Steven Schveighoffer a Ă©crit :
 You actually need a finalizer if you want to have resources that aren't
 GC allocated.

 -Steve

Indeed. But java's way of doing it is very poor.
May 24 2012
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-05-24 13:38:01 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 However, I'd tend to believe Java implementations will attempt to invoke
 all finalizers of objects left on the heap at program shutdown.

In Java you can call System.runFinalizersOnExit(true), but the default is false and this method has been deprecated because unsafe in multithreaded environments. <http://docs.oracle.com/javase/6/docs/api/java/lang/System.html#runFinalization()> -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 24 2012
parent =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
On 24-05-2012 16:00, Michel Fortin wrote:
 On 2012-05-24 13:38:01 +0000, "Steven Schveighoffer"
 <schveiguy yahoo.com> said:

 However, I'd tend to believe Java implementations will attempt to invoke
 all finalizers of objects left on the heap at program shutdown.

In Java you can call System.runFinalizersOnExit(true), but the default is false and this method has been deprecated because unsafe in multithreaded environments. <http://docs.oracle.com/javase/6/docs/api/java/lang/System.html#runFinalization()>

It's only deprecated because Java's way of handling threading and finalization is apparently completely broken. See C# and the CLR for a system that actually works. -- Alex Rřnne Petersen alex lycus.org http://lycus.org
May 24 2012
prev sibling next sibling parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 24-05-2012 15:38, Steven Schveighoffer wrote:
 On Thu, 24 May 2012 08:54:45 -0400, Peter Alexander
 <peter.alexander.au gmail.com> wrote:

 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for
 all unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

I'm pretty sure it's the same in Java. Finalizers (a.k.a. class destructors) are practically useless.

From Java spec: "The Java programming language does not specify how soon a finalizer will be invoked, except to say that it will happen before the storage for the object is reused" So yeah, there is no guarantee when a finalizer will be invoked. However, I'd tend to believe Java implementations will attempt to invoke all finalizers of objects left on the heap at program shutdown. I think D is the same too, as long as termination is normal (i.e. not from throwing an Error).

This shouldn't just be an implementation detail IMO. It should be a documented feature.
 -Steve

-- Alex Rønne Petersen alex lycus.org http://lycus.org
May 24 2012
prev sibling parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 24-05-2012 16:03, Russel Winder wrote:
 On Thu, 2012-05-24 at 09:38 -0400, Steven Schveighoffer wrote:
 [...]
 However, I'd tend to believe Java implementations will attempt to invoke
 all finalizers of objects left on the heap at program shutdown.

As far as I am aware Java implementations do no finalization on exit unless System.runFinalizersOnExit(true) has been called. This method is deprecated since it can cause incorrect finalization. But like all things Java that have been deprecated, they never actually go away. There is System.runFinalization() which executes finalize on all objects in the pending finalization queue.

We should really expose a waitForFinalizers() function in core.memory. -- Alex Rønne Petersen alex lycus.org http://lycus.org
May 24 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 24 May 2012 08:54:45 -0400, Peter Alexander  =

<peter.alexander.au gmail.com> wrote:

 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex R=C3=B8nne Petersen wro=

 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for al=


 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the=


 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early  =


 design decision that has since been rectified but with lack of  =


 documentation updates.

I'm pretty sure it's the same in Java. Finalizers (a.k.a. class destructors) are practically useless.

From Java spec: "The Java programming language does not specify how soon a finalizer wil= l = be invoked, except to say that it will happen before the storage for the= = object is reused" So yeah, there is no guarantee when a finalizer will be invoked. However, I'd tend to believe Java implementations will attempt to invoke= = all finalizers of objects left on the heap at program shutdown. I think D is the same too, as long as termination is normal (i.e. not fr= om = throwing an Error). -Steve
May 24 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 24 May 2012 09:47:23 -0400, deadalnix <deadalnix gmail.com> wrot=
e:

 Le 24/05/2012 14:54, Peter Alexander a =C3=A9crit :
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex R=C3=B8nne Petersen wr=


 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for a=



 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and th=



 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

I'm pretty sure it's the same in Java. Finalizers (a.k.a. class destructors) are practically useless.

Java finalizer is a pretty bad design decision. Let's not reproduce =

 error made in Java in D's destructors.

You actually need a finalizer if you want to have resources that aren't = GC = allocated. -Steve
May 24 2012
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, 2012-05-24 at 09:38 -0400, Steven Schveighoffer wrote:
[...]
 However, I'd tend to believe Java implementations will attempt to invoke =

 all finalizers of objects left on the heap at program shutdown.

As far as I am aware Java implementations do no finalization on exit unless System.runFinalizersOnExit(true) has been called. This method is deprecated since it can cause incorrect finalization. But like all things Java that have been deprecated, they never actually go away. There is System.runFinalization() which executes finalize on all objects in the pending finalization queue. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
May 24 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 24 May 2012 10:30:02 -0400, Alex R=C3=B8nne Petersen <alex lycus=
.org>  =

wrote:

 On 24-05-2012 15:49, Steven Schveighoffer wrote:
 On Thu, 24 May 2012 09:47:23 -0400, deadalnix <deadalnix gmail.com>  =


 wrote:

 Le 24/05/2012 14:54, Peter Alexander a =C3=A9crit :
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex R=C3=B8nne Petersen =




 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for=





 all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and =





 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

I'm pretty sure it's the same in Java. Finalizers (a.k.a. class destructors) are practically useless.

Java finalizer is a pretty bad design decision. Let's not reproduce error made in Java in D's destructors.

You actually need a finalizer if you want to have resources that aren=


 GC allocated.

 -Steve

But that doesn't mean we should have Java finalization. There are many=

 different forms of finalization, and I do agree that Java is the worst=

 of all of them.

I only found one definition for finalizer on wikipedia, and it fits D's = = definition. What I think we need is a dispose pattern for objects, like Tango has. -Steve
May 24 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 24 May 2012 11:04:06 -0400, Jacob Carlborg <doob me.com> wrote:

 On 2012-05-24 16:53, Steven Schveighoffer wrote:

 What I think we need is a dispose pattern for objects, like Tango has.

Object.dispose in Tango is called on scope exit if the object is variable is declared "scope". "scope" is deprecated in D2.

We can easily hook that in object.clear, which any scoped library implementation should be calling. -Steve
May 24 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Thursday, 24 May 2012 at 15:04:06 UTC, Jacob Carlborg wrote:
 On 2012-05-24 16:53, Steven Schveighoffer wrote:
 What I think we need is a dispose pattern for objects, like 
 Tango has.

Object.dispose in Tango is called on scope exit if the object is variable is declared "scope". "scope" is deprecated in D2.

As for replicating that functionality, Scoped!T could always check for a magic dispose() method (maybe with another name, a marker parameter, …) and if it exists, call it on destruction. But of course, the »universality« of the Tango runtimes solution is lost with this. David
May 24 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Thursday, 24 May 2012 at 16:06:23 UTC, Andrei Alexandrescu 
wrote:
 On 5/24/12 9:57 AM, Alex Rønne Petersen wrote:
 Doesn't matter: Nothing is guaranteed about order of 
 finalization (and
 this is reasonable). Thus, the finalizers can be run in any 
 arbitrary
 order. The important point here is that they are run *at all*.

just been destroyed.

You can't do that in today's D either, going by the spec as well by the actual implementation. David
May 24 2012
prev sibling next sibling parent "Tove" <tove fransson.se> writes:
On Thursday, 24 May 2012 at 17:06:19 UTC, ponce wrote:
 I really had a hard time to believe it when #D told me so, but 
 there is no guaranteed order of destruction and as you cannot 
 relies on members still being alive in a class destructor.
 All of it can happen when making absolutely no cycles in the 
 object graph.

 What I do now is having a close function for each class which 
 hold a non-memory resource.

 This is writtent in TDPL, but I wish I was told earlier :)

http://dlang.org/class.html#destructors "This rule does not apply to auto objects or objects deleted with the DeleteExpression, as the destructor is not being run by the garbage collector, meaning all references are valid." i.e. non gc resources are fine... and it's also fine if you call clear()... it's only a problem if you rely on automatic collection and reference a object... so there's no need for close, as clear() will do the trick.
May 24 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Thursday, 24 May 2012 at 17:55:02 UTC, Alex Rønne Petersen 
wrote:
 I would strongly advise against that, because a missed clear() 
 means your finalizer may be run by the runtime's finalization 
 machinery, and thus invalidate any invariants you were relying 
 on in the finalizer.

Yes – the »correc†
May 24 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 24 May 2012 13:47:31 -0400, Tove <tove fransson.se> wrote:

 On Thursday, 24 May 2012 at 17:06:19 UTC, ponce wrote:
 I really had a hard time to believe it when #D told me so, but there is  
 no guaranteed order of destruction and as you cannot relies on members  
 still being alive in a class destructor.
 All of it can happen when making absolutely no cycles in the object  
 graph.

 What I do now is having a close function for each class which hold a  
 non-memory resource.

 This is writtent in TDPL, but I wish I was told earlier :)

http://dlang.org/class.html#destructors "This rule does not apply to auto objects or objects deleted with the DeleteExpression, as the destructor is not being run by the garbage collector, meaning all references are valid." i.e. non gc resources are fine... and it's also fine if you call clear()... it's only a problem if you rely on automatic collection and reference a object... so there's no need for close, as clear() will do the trick.

There's a big problem with this though. Your destructor *has no idea* whether it's being called from within a collection cycle, or from clear. You must assume the most restrictive environment, i.e. that the dtor is being called from the GC. This is even true with struct dtors! -Steve
May 24 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Thursday, 24 May 2012 at 17:55:02 UTC, Alex Rønne Petersen 
wrote:
 I would strongly advise against that, because a missed clear() 
 means your finalizer may be run by the runtime's finalization 
 machinery, and thus invalidate any invariants you were relying 
 on in the finalizer.

Yes – the »correct« way to handle situations where you need deterministic finalization is to use structs on the stack, possibly in conjunction with reference counting. Of course, there are some situations (e.g. when there are cycles) where this doesn't work, but at least it covers most of the »external non-memory resource« cases. Composability can still be a problem, though, because holding a reference in a (GC-managed) class object might leave you with exactly the same problem you tried to avoid in the first place. David
May 24 2012
prev sibling next sibling parent "Tove" <tove fransson.se> writes:
On Thursday, 24 May 2012 at 17:57:11 UTC, Steven Schveighoffer 
wrote:
 On Thu, 24 May 2012 13:47:31 -0400, Tove <tove fransson.se> 
 wrote:

 There's a big problem with this though.  Your destructor *has 
 no idea* whether it's being called from within a collection 
 cycle, or from clear.  You must assume the most restrictive 
 environment, i.e. that the dtor is being called from the GC.

 This is even true with struct dtors!

 -Steve

If there is a clear location where a manual close() function can be called... then there are many safe solutions to automatically and safely call clear instead. std.typecons.Unique If you are a library creator, you could even use a factory to enforce wrapping in Unique... But I don't see any point of adding a non standard destructor function name, there are numerous ways to facilitate RAII.
May 24 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Thursday, 24 May 2012 at 17:57:11 UTC, Steven Schveighoffer 
wrote:
 On Thu, 24 May 2012 13:47:31 -0400, Tove <tove fransson.se> 
 wrote:

 On Thursday, 24 May 2012 at 17:06:19 UTC, ponce wrote:
 I really had a hard time to believe it when #D told me so, 
 but there is no guaranteed order of destruction and as you 
 cannot relies on members still being alive in a class 
 destructor.
 All of it can happen when making absolutely no cycles in the 
 object graph.

 What I do now is having a close function for each class which 
 hold a non-memory resource.

 This is writtent in TDPL, but I wish I was told earlier :)

http://dlang.org/class.html#destructors "This rule does not apply to auto objects or objects deleted with the DeleteExpression, as the destructor is not being run by the garbage collector, meaning all references are valid." i.e. non gc resources are fine... and it's also fine if you call clear()... it's only a problem if you rely on automatic collection and reference a object... so there's no need for close, as clear() will do the trick.

There's a big problem with this though. Your destructor *has no idea* whether it's being called from within a collection cycle, or from clear. You must assume the most restrictive environment, i.e. that the dtor is being called from the GC. This is even true with struct dtors! -Steve

Looks to me like an issue with separation of concerns. I think that dtors need to only provide deterministic management of resources and not affect GC algorithms: 1. classes should *not* have dtors at all. 2. struct values should *not* be gc managed [*]. Composition of classes and structs should be handled as follows: 1. If a class contains a pointer to a struct it doesn't scan it in a GC cycle. The runtime can provide a hook so that structs could register a callback for RC purposes. 2. When a class contains a strcut value, they share the same lifetime thus the GC will call the struct's dtor when the object is collected. 3. If a struct contains a reference to an object (class instance) than *that* object instance is scanned by the GC. [*] point 3 above means that struct pointers are scanned in a pass-thru way only for the purpose of scanning contained object references. With the above semantics, the dtor does know that all its members (except for class references) are valid, including any pointers to other structs and has similar semantics to c++ dtors. This scheme can be enforced by the compiler and is safe since objects won't inherently hold any resources by themselves (they don't have dtors). This also allows to implement more advanced finalization schemes (e.g. dependencies between resources).
May 24 2012
prev sibling next sibling parent "Tove" <tove fransson.se> writes:
On Thursday, 24 May 2012 at 19:46:07 UTC, foobar wrote:
 Looks to me like an issue with separation of concerns. I think 
 that dtors need to only provide deterministic management of 
 resources and not affect GC algorithms:
 1. classes should *not* have dtors at all.
 2. struct values should *not* be gc managed [*].

Why not simply set "BlkAttr.NO_SCAN" on ourselves if we need certain resources in the destructor? Assuming we one day get user defined attributes, it can be make quite simple...
May 24 2012
prev sibling next sibling parent "Tove" <tove fransson.se> writes:
On Thursday, 24 May 2012 at 20:53:33 UTC, Tove wrote:
 On Thursday, 24 May 2012 at 19:46:07 UTC, foobar wrote:
 Looks to me like an issue with separation of concerns. I think 
 that dtors need to only provide deterministic management of 
 resources and not affect GC algorithms:
 1. classes should *not* have dtors at all.
 2. struct values should *not* be gc managed [*].

Why not simply set "BlkAttr.NO_SCAN" on ourselves if we need certain resources in the destructor? Assuming we one day get user defined attributes, it can be make quite simple...

Tested my idea... unfortunately it's broken... GC.collect() while the program is running, is OK... so I was hoping to add: GC.disable() just before main() ends, but apparently this request is ignored. i.e. back to square 1, undefined collecting order once the program exits. import std.stdio; import core.memory; class Important { this() { us ~= this; } ~this() { writeln("2"); } private: static Important[] us; } class CollectMe { Important resource; this() { resource = new Important(); } ~this() { writeln("1"); clear(resource); } } void main() { GC.setAttr(cast(void*)new CollectMe(), GC.BlkAttr.NO_SCAN); GC.collect(); GC.disable(); writeln("3"); }
May 24 2012
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Thursday, 24 May 2012 at 13:49:38 UTC, Steven Schveighoffer 
wrote:
 You actually need a finalizer if you want to have resources 
 that aren't GC allocated.

Does that ring a bell? ;)
May 24 2012
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Thursday, 24 May 2012 at 16:06:23 UTC, Andrei Alexandrescu 
wrote:
 It does matter because a destructor may use an object that has 
 just been destroyed.

 Andrei

Andrei: .NET has this exact problem, and handles it pretty well. There are two types of "Dispose()": manual and automatic. Whenever you're wrapping some unmanaged resource (e.g. file handle), you wrap it inside some managed object (e.g. SafeFileHandle). Then you embed _that_ resource in the actual object you want (e.g. FileStream). Now, there are two ways a FileStream can get destroyed: 1. Through a manual call to FileStream.Dispose(). In this case, all embedded objects (e.g. SafeFileHandle) are *guaranteed* to be valid, so we simply flush the file and call SafeFileHandle.Dispose() to dispose of the managed resources, and then dispose of all the unmanaged resources (which are primitive fields, guaranteed to be accessible). Furthermore, the object suppresses its own finalizer. 2. Through a garbage-collected call to ~FileStream(). In this case, the managed resources such as SafeFileHandle will be (or is already) destroyed SEPARATELY, and so we do _NOT_ access them. We ONLY dispose of the unmanaged resources, if any, and let the managed resources take care of themselves. It's a pretty well-defined sequence, and it works well in practice. (Of course, you don't actually _need_ this double-indirection here: You could instead just wrap the unmanaged resource manually, and do everything in FileStream. The reason for the double-indirection is something slightly unrelated. I was just explaining how to take care of the managed resource disposal problem that you mentioned.) You could point out that, in this case, the FileStream doesn't flush its buffers before the file handle is destroyed, if the GC collects the object. That problem is solvable in two ways, although .NET simply chose to not worry about it, as far as I know: 1. Simply wrap the handle inside FileStream. Since it will be unmanaged, you can access it during disposal. 2. If that isn't possible, keep a _strong_, *unmanaged* reference to your _managed_ SafeFileHandle object. (This is accomplished through acquiring a cookie from the GC.) Because of this, SafeFileHandle will NOT be destroyed before FileStream. You can then use this fact to access SafeFileHandle inside FileStream's finalizer, through the unmanaged (but safe) cookie. tl;dr: It's a completely solved problem in .NET; there really shouldn't be any issues with it in D either.
May 24 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Friday, 25 May 2012 at 07:13:11 UTC, Jacob Carlborg wrote:
 On 2012-05-24 21:46, foobar wrote:

 Looks to me like an issue with separation of concerns. I think 
 that
 dtors need to only provide deterministic management of 
 resources and not
 affect GC algorithms:
 1. classes should *not* have dtors at all.
 2. struct values should *not* be gc managed [*].

 Composition of classes and structs should be handled as 
 follows:
 1. If a class contains a pointer to a struct it doesn't scan 
 it in a GC
 cycle. The runtime can provide a hook so that structs could 
 register a
 callback for RC purposes.
 2. When a class contains a strcut value, they share the same 
 lifetime
 thus the GC will call the struct's dtor when the object is 
 collected.

How is that any different than having destructors for classes.

It makes the call order deterministic like in C++. e.g. class Foo {} struct A { resource r; ~this() { release(r); } } struct B { A* a; Foo foo; ~this() { delete a; } // [1] } Lets look at point [1]: The "foo" instance is "managed" by the GC since the only resource it holds is memory. The "a" member wraps some "non-managed" resource (e.g. file descriptor) and in this model is still valid, thus allows me to deterministically dispose of it as in c++. This can be simply checked at compile-time - you can only reference non class instance members in the destructor, so adding a "delete foo;" statement at point [1] simply won't compile.
May 25 2012
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Thu, 24 May 2012 18:57:11 +0100, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:
 On Thu, 24 May 2012 13:47:31 -0400, Tove <tove fransson.se> wrote:

 On Thursday, 24 May 2012 at 17:06:19 UTC, ponce wrote:
 I really had a hard time to believe it when #D told me so, but there  
 is no guaranteed order of destruction and as you cannot relies on  
 members still being alive in a class destructor.
 All of it can happen when making absolutely no cycles in the object  
 graph.

 What I do now is having a close function for each class which hold a  
 non-memory resource.

 This is writtent in TDPL, but I wish I was told earlier :)

http://dlang.org/class.html#destructors "This rule does not apply to auto objects or objects deleted with the DeleteExpression, as the destructor is not being run by the garbage collector, meaning all references are valid." i.e. non gc resources are fine... and it's also fine if you call clear()... it's only a problem if you rely on automatic collection and reference a object... so there's no need for close, as clear() will do the trick.

There's a big problem with this though. Your destructor *has no idea* whether it's being called from within a collection cycle, or from clear. You must assume the most restrictive environment, i.e. that the dtor is being called from the GC. This is even true with struct dtors!

The C# dispose model suggests/gives examples of handling this sort of problem using a bool and 2 dispose methods, see: http://msdn.microsoft.com/en-us/library/fs2xkftw.aspx Could we do something similar in D, i.e. provide a template class which could wrap any reference and implement this.. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
May 25 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Friday, 25 May 2012 at 11:23:40 UTC, Jacob Carlborg wrote:
 On 2012-05-25 11:02, foobar wrote:

 It makes the call order deterministic like in C++.

 e.g.
 class Foo {}

 struct A {
 resource r;
 ~this() { release(r); }
 }

 struct B {
 A* a;
 Foo foo;
 ~this() { delete a; } // [1]
 }

I though we were talking about classes holding structs: class B { A* a; Foo foo; ~this() { delete a; } } In this case you don't know when/if the destructor of B is called. It doesn't help to wrap it in a struct, you could just have put it directly in A. Is that correct?

No. see below.
 Lets look at point [1]:
 The "foo" instance is "managed" by the GC since the only 
 resource it
 holds is memory. The "a" member wraps some "non-managed" 
 resource (e.g.
 file descriptor) and in this model is still valid, thus allows 
 me to
 deterministically dispose of it as in c++.

Ok, but if B is a class?
 This can be simply checked at compile-time - you can only 
 reference non
 class instance members in the destructor, so adding a "delete 
 foo;"
 statement at point [1] simply won't compile.

If you have a pointer to a struct you don't know how it was created. It's possible it's been created with "new", which means the garbage collector needs to delete it.

let's say we add two classes: class FooA { A a; } class FooPA { A* pa; } For the first case, both the class and the struct share the same lifetime thus when an instance of FooA is GC-ed, the GC would call A's d-tor and allow it to do what-ever (self) cleaning it requires. This means the d-tor will always be called. For the second case, The GC will only scan "pa" to find inner class instances but will *not* handle the struct value itself. In order to clean what "pa" points to, you need to explicitly call the destructor yourself. One way to do this would be to register a callback with the GC to get notified when an instance of FooPA is collected and inside the callback function maintain a reference-counter. This also means that if you allocate a struct value on the heap via "new" you are responsible to call delete _yourself_ and the gc will not call it for you. I think that loosing this small convenience is worth it - we gay more orthogonal semantics that are easier to reason about.
May 25 2012
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Friday, 25 May 2012 at 14:35:58 UTC, Andrei Alexandrescu wrote:
 What happens in C# if an object A that has a field referring to 
 object B, and the object B has in turn a field referring to 
 object A? That is:
 What happens then? Will the GC nullify references to destroyed 
 objects, or will it put them in a zombie state?

Depends. If it's a _manual_ disposal, everything is fine -- neither is GC'd yet. If it's an _automatic_ disposal (a.k.a. garbage collection), then the cross references must not be used. I believe their contents are either undefined or null, but in either case, you don't worry about disposing them because the objects will take care of themselves.
May 25 2012
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Friday, 25 May 2012 at 14:38:29 UTC, Alex Rønne Petersen 
wrote:
 This is called resurrection: 
 http://msdn.microsoft.com/en-us/magazine/bb985010.aspx (scroll 
 down to Resurrection)

Ah, yes, you're completely right; I missed this fact. Apparently under these conditions, you _can_ resurrect objects, but it's bad practice (and unnecessary) in most situations. Andrei: The reason this is allowed is that finalization is _separate_ from garbage collection in .NET. So an object can be finalized and yet still not GC'd. Or its finalizer might be suppressed, allowing it to get GC'd directly. This allows for many possibilities, although you don't usually need them.
May 25 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Saturday, 26 May 2012 at 11:35:29 UTC, Jacob Carlborg wrote:
 On 2012-05-25 14:05, foobar wrote:

 If you have a pointer to a struct you don't know how it was 
 created.
 It's possible it's been created with "new", which means the 
 garbage
 collector needs to delete it.

let's say we add two classes: class FooA { A a; } class FooPA { A* pa; } For the first case, both the class and the struct share the same lifetime thus when an instance of FooA is GC-ed, the GC would call A's d-tor and allow it to do what-ever (self) cleaning it requires. This means the d-tor will always be called.

Is that the cases even if the destructor of FooA isn't called?

Huh? In my model FooA has no destructor.
 For the second case, The GC will only scan "pa" to find inner 
 class
 instances but will *not* handle the struct value itself.
 In order to clean what "pa" points to, you need to explicitly 
 call the
 destructor yourself.

Are you saying that the GC won't collect a struct allocated with "new"? http://dlang.org/expression.html#NewExpression "NewExpressions are used to allocate memory on the garbage collected heap...". I though that everything allocated via the GC was also collected by the GC.

I indeed propose that structs allocated with "new" will be put in region of the heap *not* managed by the GC. It's a tiny price to pay to get more orthogonal semantics which are easier to reason about. Please note that this only affects code that directly uses pointers which is not common in D and is geared towards more advanced use cases where the programmer will likely want to manage the memory explicitly anyway.
 One way to do this would be to register a callback
 with the GC to get notified when an instance of FooPA is 
 collected and
 inside the callback function maintain a reference-counter.
 This also means that if you allocate a struct value on the 
 heap via
 "new" you are responsible to call delete _yourself_ and the gc 
 will not
 call it for you.
 I think that loosing this small convenience is worth it - we 
 gay more
 orthogonal semantics that are easier to reason about.


May 26 2012
prev sibling parent "Michael" <pr m1xa.com> writes:
On Friday, 25 May 2012 at 14:35:58 UTC, Andrei Alexandrescu wrote:
 What happens in C# if an object A that has a field referring to 
 object B, and the object B has in turn a field referring to 
 object A? That is:

 class C { C another; ~this() { writeln(another.another); } }

 void main() {
     auto a = new C;
     auto b = new C;
     a.another = b;
     b.another = a;
 }

 What happens then? Will the GC nullify references to destroyed 
 objects, or will it put them in a zombie state?


 Thanks,

 Andrei

In this case the a and b objects will be collected by GC and memory freed. It's a one of most popular questions about .NET GC. Maybe something was changed in .NET >= 4.5. I have another question: there available a good example of idiomatic destructor usage in the D? Something without calling a rt_ hooks on destroy?
Nov 12 2013