=?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:

Hi,

http://dlang.org/class.html#Destructor

"The garbage collector is not guaranteed to run the destructor for all 
unreferenced objects."

What the *hell*? So resources are allowed to arbitrarily leak and the 
programmer has to actually expect this to happen?

I really, really hope that this is a documentation error or early design 
decision that has since been rectified but with lack of documentation 
updates.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

"Thor" <Thor hotmail.com> writes:

On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen 
wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor 
 for all unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak 
 and the programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or 
 early design decision that has since been rectified but with 
 lack of documentation updates.

use "clear", or "scope (exit)" or "structs" or scoped!... etc.

There could always be a false reference... so you cannot depend 
on automatically releasing resources in a class destructor.

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 24-05-2012 14:33, Thor wrote:
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

 use "clear", or "scope (exit)" or "structs" or scoped!... etc.

I know.

 There could always be a false reference... so you cannot depend on
 automatically releasing resources in a class destructor.

False pointers have nothing to do with it. The GC should free and 
finalize all objects on shutdown, meaning the finalizer runs *sooner or 
later*. If this is the case (which I do believe it is), then the docs 
are very wrong.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

"Thor" <Thor hotmail.com> writes:

On Thursday, 24 May 2012 at 12:38:45 UTC, Alex Rønne Petersen 
wrote:
 On 24-05-2012 14:33, Thor wrote:
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen 
 wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the 
 destructor for all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak 
 and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or 
 early
 design decision that has since been rectified but with lack of
 documentation updates.

 use "clear", or "scope (exit)" or "structs" or scoped!... etc.

 I know.

 There could always be a false reference... so you cannot 
 depend on
 automatically releasing resources in a class destructor.

 False pointers have nothing to do with it. The GC should free 
 and finalize all objects on shutdown, meaning the finalizer 
 runs *sooner or later*. If this is the case (which I do believe 
 it is), then the docs are very wrong.

__gshared uint my_false_ptr;

even if we are shutting down, the static references doesn't 
disappear... or did I miss something?

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 24-05-2012 14:48, Thor wrote:
 On Thursday, 24 May 2012 at 12:38:45 UTC, Alex Rønne Petersen wrote:
 On 24-05-2012 14:33, Thor wrote:
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

 use "clear", or "scope (exit)" or "structs" or scoped!... etc.

 I know.

 There could always be a false reference... so you cannot depend on
 automatically releasing resources in a class destructor.

 False pointers have nothing to do with it. The GC should free and
 finalize all objects on shutdown, meaning the finalizer runs *sooner
 or later*. If this is the case (which I do believe it is), then the
 docs are very wrong.

 __gshared uint my_false_ptr;

 even if we are shutting down, the static references doesn't disappear...
 or did I miss something?

The GC should (and probably does) assume at shutdown that all objects 
are unreferenced, and therefore reclaim and finalize them.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

They may refer to one another.

Andrei

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 24-05-2012 16:54, Andrei Alexandrescu wrote:
 On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

 They may refer to one another.

 Andrei

Doesn't matter: Nothing is guaranteed about order of finalization (and 
this is reasonable). Thus, the finalizers can be run in any arbitrary 
order. The important point here is that they are run *at all*.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 5/24/12 9:57 AM, Alex Rønne Petersen wrote:
 On 24-05-2012 16:54, Andrei Alexandrescu wrote:
 On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

 They may refer to one another.

 Andrei

 Doesn't matter: Nothing is guaranteed about order of finalization (and
 this is reasonable). Thus, the finalizers can be run in any arbitrary
 order. The important point here is that they are run *at all*.

It does matter because a destructor may use an object that has just been 
destroyed.

Andrei

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 24-05-2012 18:06, Andrei Alexandrescu wrote:
 On 5/24/12 9:57 AM, Alex Rønne Petersen wrote:
 On 24-05-2012 16:54, Andrei Alexandrescu wrote:
 On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

 They may refer to one another.

 Andrei

 Doesn't matter: Nothing is guaranteed about order of finalization (and
 this is reasonable). Thus, the finalizers can be run in any arbitrary
 order. The important point here is that they are run *at all*.

 It does matter because a destructor may use an object that has just been
 destroyed.

 Andrei

No, the docs specifically state that this is invalid (and it currently 
throws InvalidMemoryOperationError in most cases).

Whether it *should* be allowed is arguable, but it isn't currently, both 
in docs and impl.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

ponce <spam spam.org> writes:

Le 24/05/2012 18:10, Alex Rønne Petersen a écrit :
 On 24-05-2012 18:06, Andrei Alexandrescu wrote:
 It does matter because a destructor may use an object that has just been
 destroyed.

 Andrei

 No, the docs specifically state that this is invalid (and it currently
 throws InvalidMemoryOperationError in most cases).

 Whether it *should* be allowed is arguable, but it isn't currently, both
 in docs and impl.

I really had a hard time to believe it when #D told me so, but there is 
no guaranteed order of destruction and as you cannot relies on members 
still being alive in a class destructor.
All of it can happen when making absolutely no cycles in the object graph.

What I do now is having a close function for each class which hold a 
non-memory resource.

This is writtent in TDPL, but I wish I was told earlier :)

"Tove" <tove fransson.se> writes:

On Thursday, 24 May 2012 at 17:06:19 UTC, ponce wrote:
 I really had a hard time to believe it when #D told me so, but 
 there is no guaranteed order of destruction and as you cannot 
 relies on members still being alive in a class destructor.
 All of it can happen when making absolutely no cycles in the 
 object graph.

 What I do now is having a close function for each class which 
 hold a non-memory resource.

 This is writtent in TDPL, but I wish I was told earlier :)

http://dlang.org/class.html#destructors

"This rule does not apply to auto objects or objects deleted with 
the DeleteExpression, as the destructor is not being run by the 
garbage collector, meaning all references are valid."

i.e. non gc resources are fine... and it's also fine if you call 
clear()... it's only a problem if you rely on automatic 
collection and reference a object... so there's no need for 
close, as clear() will do the trick.

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 24-05-2012 19:47, Tove wrote:
 On Thursday, 24 May 2012 at 17:06:19 UTC, ponce wrote:
 I really had a hard time to believe it when #D told me so, but there
 is no guaranteed order of destruction and as you cannot relies on
 members still being alive in a class destructor.
 All of it can happen when making absolutely no cycles in the object
 graph.

 What I do now is having a close function for each class which hold a
 non-memory resource.

 This is writtent in TDPL, but I wish I was told earlier :)

 http://dlang.org/class.html#destructors

 "This rule does not apply to auto objects or objects deleted with the
 DeleteExpression, as the destructor is not being run by the garbage
 collector, meaning all references are valid."

 i.e. non gc resources are fine... and it's also fine if you call
 clear()... it's only a problem if you rely on automatic collection and
 reference a object... so there's no need for close, as clear() will do
 the trick.

I would strongly advise against that, because a missed clear() means 
your finalizer may be run by the runtime's finalization machinery, and 
thus invalidate any invariants you were relying on in the finalizer.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

"David Nadlinger" <see klickverbot.at> writes:

On Thursday, 24 May 2012 at 17:55:02 UTC, Alex Rønne Petersen 
wrote:
 I would strongly advise against that, because a missed clear() 
 means your finalizer may be run by the runtime's finalization 
 machinery, and thus invalidate any invariants you were relying 
 on in the finalizer.

Yes – the »correc†

"David Nadlinger" <see klickverbot.at> writes:

On Thursday, 24 May 2012 at 17:55:02 UTC, Alex Rønne Petersen 
wrote:
 I would strongly advise against that, because a missed clear() 
 means your finalizer may be run by the runtime's finalization 
 machinery, and thus invalidate any invariants you were relying 
 on in the finalizer.

Yes – the »correct« way to handle situations where you need 
deterministic finalization is to use structs on the stack, 
possibly in conjunction with reference counting. Of course, there 
are some situations (e.g. when there are cycles) where this 
doesn't work, but at least it covers most of the »external 
non-memory resource« cases.

Composability can still be a problem, though, because holding a 
reference in a (GC-managed) class object might leave you with 
exactly the same problem you tried to avoid in the first place.

David

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Thu, 24 May 2012 13:47:31 -0400, Tove <tove fransson.se> wrote:

 On Thursday, 24 May 2012 at 17:06:19 UTC, ponce wrote:
 I really had a hard time to believe it when #D told me so, but there is  
 no guaranteed order of destruction and as you cannot relies on members  
 still being alive in a class destructor.
 All of it can happen when making absolutely no cycles in the object  
 graph.

 What I do now is having a close function for each class which hold a  
 non-memory resource.

 This is writtent in TDPL, but I wish I was told earlier :)

 http://dlang.org/class.html#destructors

 "This rule does not apply to auto objects or objects deleted with the  
 DeleteExpression, as the destructor is not being run by the garbage  
 collector, meaning all references are valid."

 i.e. non gc resources are fine... and it's also fine if you call  
 clear()... it's only a problem if you rely on automatic collection and  
 reference a object... so there's no need for close, as clear() will do  
 the trick.

There's a big problem with this though.  Your destructor *has no idea*  
whether it's being called from within a collection cycle, or from clear.   
You must assume the most restrictive environment, i.e. that the dtor is  
being called from the GC.

This is even true with struct dtors!

-Steve

"Tove" <tove fransson.se> writes:

On Thursday, 24 May 2012 at 17:57:11 UTC, Steven Schveighoffer 
wrote:
 On Thu, 24 May 2012 13:47:31 -0400, Tove <tove fransson.se> 
 wrote:

 There's a big problem with this though.  Your destructor *has 
 no idea* whether it's being called from within a collection 
 cycle, or from clear.  You must assume the most restrictive 
 environment, i.e. that the dtor is being called from the GC.

 This is even true with struct dtors!

 -Steve

If there is a clear location where a manual close() function can 
be called... then there are many safe solutions to automatically 
and safely call clear instead.

std.typecons.Unique

If you are a library creator, you could even use a factory to 
enforce wrapping in Unique...  But I don't see any point of 
adding a non standard destructor function name, there are 
numerous ways to facilitate RAII.

"foobar" <foo bar.com> writes:

On Thursday, 24 May 2012 at 17:57:11 UTC, Steven Schveighoffer 
wrote:
 On Thu, 24 May 2012 13:47:31 -0400, Tove <tove fransson.se> 
 wrote:

 On Thursday, 24 May 2012 at 17:06:19 UTC, ponce wrote:
 I really had a hard time to believe it when #D told me so, 
 but there is no guaranteed order of destruction and as you 
 cannot relies on members still being alive in a class 
 destructor.
 All of it can happen when making absolutely no cycles in the 
 object graph.

 What I do now is having a close function for each class which 
 hold a non-memory resource.

 This is writtent in TDPL, but I wish I was told earlier :)

 http://dlang.org/class.html#destructors

 "This rule does not apply to auto objects or objects deleted 
 with the DeleteExpression, as the destructor is not being run 
 by the garbage collector, meaning all references are valid."

 i.e. non gc resources are fine... and it's also fine if you 
 call clear()... it's only a problem if you rely on automatic 
 collection and reference a object... so there's no need for 
 close, as clear() will do the trick.

 There's a big problem with this though.  Your destructor *has 
 no idea* whether it's being called from within a collection 
 cycle, or from clear.  You must assume the most restrictive 
 environment, i.e. that the dtor is being called from the GC.

 This is even true with struct dtors!

 -Steve

Looks to me like an issue with separation of concerns. I think 
that dtors need to only provide deterministic management of 
resources and not affect GC algorithms:
1. classes should *not* have dtors at all.
2. struct values should *not* be gc managed [*].

Composition of classes and structs should be handled as follows:
1. If a class contains a pointer to a struct it doesn't scan it 
in a GC cycle. The runtime can provide a hook so that structs 
could register a callback for RC purposes.
2. When a class contains a strcut value, they share the same 
lifetime thus the GC will call the struct's dtor when the object 
is collected.
3. If a struct contains a reference to an object (class instance) 
than *that* object instance is scanned by the GC.

[*] point 3 above means that struct pointers are scanned in a 
pass-thru way only for the purpose of scanning contained object 
references.

With the above semantics, the dtor does know that all its members 
(except for class references) are valid, including any pointers 
to other structs and has similar semantics to c++ dtors. This 
scheme can be enforced by the compiler and is safe since objects 
won't inherently hold any resources by themselves (they don't 
have dtors). This also allows to implement more advanced 
finalization schemes (e.g. dependencies between resources).

"Tove" <tove fransson.se> writes:

On Thursday, 24 May 2012 at 19:46:07 UTC, foobar wrote:
 Looks to me like an issue with separation of concerns. I think 
 that dtors need to only provide deterministic management of 
 resources and not affect GC algorithms:
 1. classes should *not* have dtors at all.
 2. struct values should *not* be gc managed [*].

Why not simply set "BlkAttr.NO_SCAN" on ourselves if we need 
certain resources in the destructor? Assuming we one day get user 
defined attributes, it can be make quite simple...

"Tove" <tove fransson.se> writes:

On Thursday, 24 May 2012 at 20:53:33 UTC, Tove wrote:
 On Thursday, 24 May 2012 at 19:46:07 UTC, foobar wrote:
 Looks to me like an issue with separation of concerns. I think 
 that dtors need to only provide deterministic management of 
 resources and not affect GC algorithms:
 1. classes should *not* have dtors at all.
 2. struct values should *not* be gc managed [*].

 Why not simply set "BlkAttr.NO_SCAN" on ourselves if we need 
 certain resources in the destructor? Assuming we one day get 
 user defined attributes, it can be make quite simple...

Tested my idea... unfortunately it's broken...

GC.collect() while the program is running, is OK... so I was 
hoping to add:
GC.disable() just before main() ends, but apparently this request 
is ignored.

i.e. back to square 1, undefined collecting order once the 
program exits.

import std.stdio;
import core.memory;

class Important
{
   this()
   {
     us ~= this;
   }
   ~this()
   {
     writeln("2");
   }

private:
   static Important[] us;
}

class CollectMe
{
   Important resource;

   this()
   {
     resource = new Important();
   }
   ~this()
   {
     writeln("1");
     clear(resource);
   }
}

void main()
{
   GC.setAttr(cast(void*)new CollectMe(), GC.BlkAttr.NO_SCAN);
   GC.collect();
   GC.disable();

   writeln("3");
}

Jacob Carlborg <doob me.com> writes:

On 2012-05-24 21:46, foobar wrote:

 Looks to me like an issue with separation of concerns. I think that
 dtors need to only provide deterministic management of resources and not
 affect GC algorithms:
 1. classes should *not* have dtors at all.
 2. struct values should *not* be gc managed [*].

 Composition of classes and structs should be handled as follows:
 1. If a class contains a pointer to a struct it doesn't scan it in a GC
 cycle. The runtime can provide a hook so that structs could register a
 callback for RC purposes.
 2. When a class contains a strcut value, they share the same lifetime
 thus the GC will call the struct's dtor when the object is collected.

How is that any different than having destructors for classes.

-- 
/Jacob Carlborg

"foobar" <foo bar.com> writes:

On Friday, 25 May 2012 at 07:13:11 UTC, Jacob Carlborg wrote:
 On 2012-05-24 21:46, foobar wrote:

 Looks to me like an issue with separation of concerns. I think 
 that
 dtors need to only provide deterministic management of 
 resources and not
 affect GC algorithms:
 1. classes should *not* have dtors at all.
 2. struct values should *not* be gc managed [*].

 Composition of classes and structs should be handled as 
 follows:
 1. If a class contains a pointer to a struct it doesn't scan 
 it in a GC
 cycle. The runtime can provide a hook so that structs could 
 register a
 callback for RC purposes.
 2. When a class contains a strcut value, they share the same 
 lifetime
 thus the GC will call the struct's dtor when the object is 
 collected.

 How is that any different than having destructors for classes.

It makes the call order deterministic like in C++.

e.g.
class Foo {}

struct A {
   resource r;
   ~this() { release(r); }
}

struct B {
   A* a;
   Foo foo;
   ~this() { delete a; } // [1]
}

Lets look at point [1]:
The "foo" instance is "managed" by the GC since the only resource 
it holds is memory. The "a" member wraps some "non-managed" 
resource (e.g. file descriptor) and in this model is still valid, 
thus allows me to deterministically dispose of it as in c++.

This can be simply checked at compile-time - you can only 
reference non class instance members in the destructor, so adding 
a "delete foo;" statement at point [1] simply won't compile.

Jacob Carlborg <doob me.com> writes:

On 2012-05-25 11:02, foobar wrote:

 It makes the call order deterministic like in C++.

 e.g.
 class Foo {}

 struct A {
 resource r;
 ~this() { release(r); }
 }

 struct B {
 A* a;
 Foo foo;
 ~this() { delete a; } // [1]
 }

I though we were talking about classes holding structs:

class B {
     A* a;
     Foo foo;
     ~this() { delete a; }
}

In this case you don't know when/if the destructor of B is called. It 
doesn't help to wrap it in a struct, you could just have put it directly 
in A. Is that correct?

 Lets look at point [1]:
 The "foo" instance is "managed" by the GC since the only resource it
 holds is memory. The "a" member wraps some "non-managed" resource (e.g.
 file descriptor) and in this model is still valid, thus allows me to
 deterministically dispose of it as in c++.

Ok, but if B is a class?

 This can be simply checked at compile-time - you can only reference non
 class instance members in the destructor, so adding a "delete foo;"
 statement at point [1] simply won't compile.

If you have a pointer to a struct you don't know how it was created. 
It's possible it's been created with "new", which means the garbage 
collector needs to delete it.

-- 
/Jacob Carlborg

"foobar" <foo bar.com> writes:

On Friday, 25 May 2012 at 11:23:40 UTC, Jacob Carlborg wrote:
 On 2012-05-25 11:02, foobar wrote:

 It makes the call order deterministic like in C++.

 e.g.
 class Foo {}

 struct A {
 resource r;
 ~this() { release(r); }
 }

 struct B {
 A* a;
 Foo foo;
 ~this() { delete a; } // [1]
 }

 I though we were talking about classes holding structs:

 class B {
     A* a;
     Foo foo;
     ~this() { delete a; }
 }

 In this case you don't know when/if the destructor of B is 
 called. It doesn't help to wrap it in a struct, you could just 
 have put it directly in A. Is that correct?

No. see below.

 Lets look at point [1]:
 The "foo" instance is "managed" by the GC since the only 
 resource it
 holds is memory. The "a" member wraps some "non-managed" 
 resource (e.g.
 file descriptor) and in this model is still valid, thus allows 
 me to
 deterministically dispose of it as in c++.

 Ok, but if B is a class?

 This can be simply checked at compile-time - you can only 
 reference non
 class instance members in the destructor, so adding a "delete 
 foo;"
 statement at point [1] simply won't compile.

 If you have a pointer to a struct you don't know how it was 
 created. It's possible it's been created with "new", which 
 means the garbage collector needs to delete it.

let's say we add two classes:
class FooA {
   A a;
}
class FooPA {
   A* pa;
}

For the first case, both the class and the struct share the same 
lifetime thus when an instance of FooA is GC-ed, the GC would 
call A's d-tor and allow it to do what-ever (self) cleaning it 
requires. This means the d-tor will always be called.

For the second case, The GC will only scan "pa" to find inner 
class instances but will *not* handle the struct value itself.
In order to clean what "pa" points to, you need to explicitly 
call the destructor yourself. One way to do this would be to 
register a callback with the GC to get notified when an instance 
of FooPA is collected and inside the callback function maintain a 
reference-counter.
This also means that if you allocate a struct value on the heap 
via "new" you are responsible to call delete _yourself_ and the 
gc will not call it for you.
I think that loosing this small convenience is worth it - we gay 
more orthogonal semantics that are easier to reason about.

Jacob Carlborg <doob me.com> writes:

On 2012-05-25 14:05, foobar wrote:

 If you have a pointer to a struct you don't know how it was created.
 It's possible it's been created with "new", which means the garbage
 collector needs to delete it.

 let's say we add two classes:
 class FooA {
 A a;
 }
 class FooPA {
 A* pa;
 }

 For the first case, both the class and the struct share the same
 lifetime thus when an instance of FooA is GC-ed, the GC would call A's
 d-tor and allow it to do what-ever (self) cleaning it requires. This
 means the d-tor will always be called.

Is that the cases even if the destructor of FooA isn't called?

 For the second case, The GC will only scan "pa" to find inner class
 instances but will *not* handle the struct value itself.
 In order to clean what "pa" points to, you need to explicitly call the
 destructor yourself.

Are you saying that the GC won't collect a struct allocated with "new"?

http://dlang.org/expression.html#NewExpression

"NewExpressions are used to allocate memory on the garbage collected 
heap...". I though that everything allocated via the GC was also 
collected by the GC.

 One way to do this would be to register a callback
 with the GC to get notified when an instance of FooPA is collected and
 inside the callback function maintain a reference-counter.
 This also means that if you allocate a struct value on the heap via
 "new" you are responsible to call delete _yourself_ and the gc will not
 call it for you.
 I think that loosing this small convenience is worth it - we gay more
 orthogonal semantics that are easier to reason about.


-- 
/Jacob Carlborg

"foobar" <foo bar.com> writes:

On Saturday, 26 May 2012 at 11:35:29 UTC, Jacob Carlborg wrote:
 On 2012-05-25 14:05, foobar wrote:

 If you have a pointer to a struct you don't know how it was 
 created.
 It's possible it's been created with "new", which means the 
 garbage
 collector needs to delete it.

 let's say we add two classes:
 class FooA {
 A a;
 }
 class FooPA {
 A* pa;
 }

 For the first case, both the class and the struct share the 
 same
 lifetime thus when an instance of FooA is GC-ed, the GC would 
 call A's
 d-tor and allow it to do what-ever (self) cleaning it 
 requires. This
 means the d-tor will always be called.

 Is that the cases even if the destructor of FooA isn't called?

Huh? In my model FooA has no destructor.

 For the second case, The GC will only scan "pa" to find inner 
 class
 instances but will *not* handle the struct value itself.
 In order to clean what "pa" points to, you need to explicitly 
 call the
 destructor yourself.

 Are you saying that the GC won't collect a struct allocated 
 with "new"?

 http://dlang.org/expression.html#NewExpression

 "NewExpressions are used to allocate memory on the garbage 
 collected heap...". I though that everything allocated via the 
 GC was also collected by the GC.

I indeed propose that structs allocated with "new" will be put in 
region of the heap *not* managed by the GC. It's a tiny price to 
pay to get more orthogonal semantics which are easier to reason 
about.
Please note that this only affects code that directly uses 
pointers which is not common in D and is geared towards more 
advanced use cases where the programmer will likely want to 
manage the memory explicitly anyway.

 One way to do this would be to register a callback
 with the GC to get notified when an instance of FooPA is 
 collected and
 inside the callback function maintain a reference-counter.
 This also means that if you allocate a struct value on the 
 heap via
 "new" you are responsible to call delete _yourself_ and the gc 
 will not
 call it for you.
 I think that loosing this small convenience is worth it - we 
 gay more
 orthogonal semantics that are easier to reason about.

Jacob Carlborg <doob me.com> writes:

On 2012-05-26 14:28, foobar wrote:

 Huh? In my model FooA has no destructor.

Hm, right. But if a destructor of a class isn't guaranteed to be called, 
how can it guarantee that the struct's destructor will be called?

 I indeed propose that structs allocated with "new" will be put in region
 of the heap *not* managed by the GC. It's a tiny price to pay to get
 more orthogonal semantics which are easier to reason about.
 Please note that this only affects code that directly uses pointers
 which is not common in D and is geared towards more advanced use cases
 where the programmer will likely want to manage the memory explicitly
 anyway.

I see.

-- 
/Jacob Carlborg

Artur Skawina <art.08.09 gmail.com> writes:

On 05/26/12 13:35, Jacob Carlborg wrote:
 On 2012-05-25 14:05, foobar wrote:
 
 If you have a pointer to a struct you don't know how it was created.
 It's possible it's been created with "new", which means the garbage
 collector needs to delete it.

 let's say we add two classes:
 class FooA {
 A a;
 }
 class FooPA {
 A* pa;
 }

 For the first case, both the class and the struct share the same
 lifetime thus when an instance of FooA is GC-ed, the GC would call A's
 d-tor and allow it to do what-ever (self) cleaning it requires. This
 means the d-tor will always be called.

 
 Is that the cases even if the destructor of FooA isn't called?
 
 For the second case, The GC will only scan "pa" to find inner class
 instances but will *not* handle the struct value itself.
 In order to clean what "pa" points to, you need to explicitly call the
 destructor yourself.

 
 Are you saying that the GC won't collect a struct allocated with "new"?
 
 http://dlang.org/expression.html#NewExpression
 
 "NewExpressions are used to allocate memory on the garbage collected heap...".
I though that everything allocated via the GC was also collected by the GC.

Everything allocated is collected, though not necessarily destructed. In case
of structs, the GC currently doesn't know about the dtors, so those are never
called.

   import std.stdio;
   struct S { ~this() { writeln("never called"); } }
   void main() { S* sp; while (1) sp = new S(); }

Right now you have to use a class if you need dtors for heap allocated
objects.

artur

"Regan Heath" <regan netmail.co.nz> writes:

On Thu, 24 May 2012 18:57:11 +0100, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:
 On Thu, 24 May 2012 13:47:31 -0400, Tove <tove fransson.se> wrote:

 On Thursday, 24 May 2012 at 17:06:19 UTC, ponce wrote:
 I really had a hard time to believe it when #D told me so, but there  
 is no guaranteed order of destruction and as you cannot relies on  
 members still being alive in a class destructor.
 All of it can happen when making absolutely no cycles in the object  
 graph.

 What I do now is having a close function for each class which hold a  
 non-memory resource.

 This is writtent in TDPL, but I wish I was told earlier :)

 http://dlang.org/class.html#destructors

 "This rule does not apply to auto objects or objects deleted with the  
 DeleteExpression, as the destructor is not being run by the garbage  
 collector, meaning all references are valid."

 i.e. non gc resources are fine... and it's also fine if you call  
 clear()... it's only a problem if you rely on automatic collection and  
 reference a object... so there's no need for close, as clear() will do  
 the trick.

 There's a big problem with this though.  Your destructor *has no idea*  
 whether it's being called from within a collection cycle, or from  
 clear.  You must assume the most restrictive environment, i.e. that the  
 dtor is being called from the GC.

 This is even true with struct dtors!


problem using a bool and 2 dispose methods, see:
http://msdn.microsoft.com/en-us/library/fs2xkftw.aspx

Could we do something similar in D, i.e. provide a template class which  
could wrap any reference and implement this..

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

"David Nadlinger" <see klickverbot.at> writes:

On Thursday, 24 May 2012 at 16:06:23 UTC, Andrei Alexandrescu 
wrote:
 On 5/24/12 9:57 AM, Alex Rønne Petersen wrote:
 Doesn't matter: Nothing is guaranteed about order of 
 finalization (and
 this is reasonable). Thus, the finalizers can be run in any 
 arbitrary
 order. The important point here is that they are run *at all*.

 It does matter because a destructor may use an object that has 
 just been destroyed.

You can't do that in today's D either, going by the spec as well 
by the actual implementation.

David

"Mehrdad" <wfunction hotmail.com> writes:

On Thursday, 24 May 2012 at 16:06:23 UTC, Andrei Alexandrescu 
wrote:
 It does matter because a destructor may use an object that has 
 just been destroyed.

 Andrei


Andrei: .NET has this exact problem, and handles it pretty well.

There are two types of "Dispose()": manual and automatic.

Whenever you're wrapping some unmanaged resource (e.g. file 
handle), you wrap it inside some managed object (e.g. 
SafeFileHandle).

Then you embed _that_ resource in the actual object you want 
(e.g. FileStream).

Now, there are two ways a FileStream can get destroyed:

1. Through a manual call to FileStream.Dispose(). In this case, 
all embedded objects (e.g. SafeFileHandle) are *guaranteed* to be 
valid, so we simply flush the file and call 
SafeFileHandle.Dispose() to dispose of the managed resources, and 
then dispose of all the unmanaged resources (which are primitive 
fields, guaranteed to be accessible). Furthermore, the object 
suppresses its own finalizer.

2. Through a garbage-collected call to ~FileStream(). In this 
case, the managed resources such as SafeFileHandle will be (or is 
already) destroyed SEPARATELY, and so we do _NOT_ access them. We 
ONLY dispose of the unmanaged resources, if any, and let the 
managed resources take care of themselves.


It's a pretty well-defined sequence, and it works well in 
practice.

(Of course, you don't actually _need_ this double-indirection 
here: You could instead just wrap the unmanaged resource 
manually, and do everything in FileStream. The reason for the 
double-indirection is something slightly unrelated. I was just 
explaining how to take care of the managed resource disposal 
problem that you mentioned.)


You could point out that, in this case, the FileStream doesn't 
flush its buffers  before the file handle is destroyed, if the GC 
collects the object.

That problem is solvable in two ways, although .NET simply chose 
to not worry about it, as far as I know:

1. Simply wrap the handle inside FileStream. Since it will be 
unmanaged, you can access it during disposal.

2. If that isn't possible, keep a _strong_, *unmanaged* reference 
to your _managed_ SafeFileHandle object. (This is accomplished 
through acquiring a cookie from the GC.) Because of this, 
SafeFileHandle will NOT be destroyed before FileStream. You can 
then use this fact to access SafeFileHandle inside FileStream's 
finalizer, through the unmanaged (but safe) cookie.



tl;dr: It's a completely solved problem in .NET; there really 
shouldn't be any issues with it in D either.

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 5/25/12 12:07 AM, Mehrdad wrote:
 Now, there are two ways a FileStream can get destroyed:

 1. Through a manual call to FileStream.Dispose(). In this case, all
 embedded objects (e.g. SafeFileHandle) are *guaranteed* to be valid, so
 we simply flush the file and call SafeFileHandle.Dispose() to dispose of
 the managed resources, and then dispose of all the unmanaged resources
 (which are primitive fields, guaranteed to be accessible). Furthermore,
 the object suppresses its own finalizer.

 2. Through a garbage-collected call to ~FileStream(). In this case, the
 managed resources such as SafeFileHandle will be (or is already)
 destroyed SEPARATELY, and so we do _NOT_ access them. We ONLY dispose of
 the unmanaged resources, if any, and let the managed resources take care
 of themselves.


B, and the object B has in turn a field referring to object A? That is:

class C { C another; ~this() { writeln(another.another); } }

void main() {
     auto a = new C;
     auto b = new C;
     a.another = b;
     b.another = a;
}

What happens then? Will the GC nullify references to destroyed objects, 
or will it put them in a zombie state?


Thanks,

Andrei

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 25-05-2012 16:35, Andrei Alexandrescu wrote:
 On 5/25/12 12:07 AM, Mehrdad wrote:
 Now, there are two ways a FileStream can get destroyed:

 1. Through a manual call to FileStream.Dispose(). In this case, all
 embedded objects (e.g. SafeFileHandle) are *guaranteed* to be valid, so
 we simply flush the file and call SafeFileHandle.Dispose() to dispose of
 the managed resources, and then dispose of all the unmanaged resources
 (which are primitive fields, guaranteed to be accessible). Furthermore,
 the object suppresses its own finalizer.

 2. Through a garbage-collected call to ~FileStream(). In this case, the
 managed resources such as SafeFileHandle will be (or is already)
 destroyed SEPARATELY, and so we do _NOT_ access them. We ONLY dispose of
 the unmanaged resources, if any, and let the managed resources take care
 of themselves.


 B, and the object B has in turn a field referring to object A? That is:

 class C { C another; ~this() { writeln(another.another); } }

 void main() {
 auto a = new C;
 auto b = new C;
 a.another = b;
 b.another = a;
 }

 What happens then? Will the GC nullify references to destroyed objects,
 or will it put them in a zombie state?


 Thanks,

 Andrei

This is called resurrection: 
http://msdn.microsoft.com/en-us/magazine/bb985010.aspx (scroll down to 
Resurrection)

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

"Mehrdad" <wfunction hotmail.com> writes:

On Friday, 25 May 2012 at 14:38:29 UTC, Alex Rønne Petersen 
wrote:
 This is called resurrection: 
 http://msdn.microsoft.com/en-us/magazine/bb985010.aspx (scroll 
 down to Resurrection)



Ah, yes, you're completely right; I missed this fact. Apparently 
under these conditions, you _can_ resurrect objects, but it's bad 
practice (and unnecessary) in most situations.


 Andrei: The reason this is allowed is that finalization is 
_separate_ from garbage collection in .NET. So an object can be 
finalized and yet still not GC'd. Or its finalizer might be 
suppressed, allowing it to get GC'd directly. This allows for 
many possibilities, although you don't usually need them.

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 25-05-2012 17:53, Mehrdad wrote:
 On Friday, 25 May 2012 at 14:38:29 UTC, Alex Rønne Petersen wrote:
 This is called resurrection:
 http://msdn.microsoft.com/en-us/magazine/bb985010.aspx (scroll down to
 Resurrection)



 Ah, yes, you're completely right; I missed this fact. Apparently under
 these conditions, you _can_ resurrect objects, but it's bad practice
 (and unnecessary) in most situations.


  Andrei: The reason this is allowed is that finalization is _separate_
 from garbage collection in .NET. So an object can be finalized and yet
 still not GC'd. Or its finalizer might be suppressed, allowing it to get
 GC'd directly. This allows for many possibilities, although you don't
 usually need them.

This is, in fact, how most GCs other than D's work. :)

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

Jonathan M Davis <jmdavisProg gmx.com> writes:

On Friday, May 25, 2012 17:53:45 Mehrdad wrote:
  Andrei: The reason this is allowed is that finalization is
 _separate_ from garbage collection in .NET. So an object can be
 finalized and yet still not GC'd. Or its finalizer might be
 suppressed, allowing it to get GC'd directly. This allows for
 many possibilities, although you don't usually need them.

Finalization _can_ be separate from the GC in D thanks to clear, but it does 
normally occur as part of a collection cycle.

- Jonathan M Davis

"Mehrdad" <wfunction hotmail.com> writes:

On Friday, 25 May 2012 at 19:30:35 UTC, Jonathan M Davis wrote:
 On Friday, May 25, 2012 17:53:45 Mehrdad wrote:
  Andrei: The reason this is allowed is that finalization is
 _separate_ from garbage collection in .NET. So an object can be
 finalized and yet still not GC'd. Or its finalizer might be
 suppressed, allowing it to get GC'd directly. This allows for
 many possibilities, although you don't usually need them.

 Finalization _can_ be separate from the GC in D thanks to 
 clear, but it does
 normally occur as part of a collection cycle.

 - Jonathan M Davis

Uhm... sure...

I wasn't really talking about D, so I'm not sure what you mean.

But, comparing to D:

I'm _not_ talking about the fact that you can call the finalizer
manually.
That has _nothing_ to do with the "separation" I was referring to
(even though it's nevertheless necessary for separating the GC
from the finalization queue).

I'm talking about the fact that, if an object has a finalizer,
its finalization stage is SEPARATE from (and obviously, before)
the GC stage. In other words, there can be TWO passes over all
objects that are going to be GC'd:

1. Unreachable finalizable objects are finalized, and their
finalizers are suppressed.
2. Unreachable objects with no finalizers are GC'd.


Therefore, resurrection is possible because after an object goes
through stage 1, it may no longer be eligible for stage 2 (it may
have strong references).

Note that this means merely _having_ a finalizer causes an object
to take 2 passes to be GC'd, instead of 1 -- even if the
finalizer is empty.

Also notice that NOTHING is left in an undefined state, and yet
everything is guaranteed to be reclaimed at some point. And
circular references cause no problems whatsoever, because if
they're unreachable from the GC root, no one cares if they have
references to each other.


 Andrei: Does that make sense?

"Mehrdad" <wfunction hotmail.com> writes:

On Friday, 25 May 2012 at 14:35:58 UTC, Andrei Alexandrescu wrote:

 object B, and the object B has in turn a field referring to 
 object A? That is:
 What happens then? Will the GC nullify references to destroyed 
 objects, or will it put them in a zombie state?

Depends.

If it's a _manual_ disposal, everything is fine -- neither is
GC'd yet.

If it's an _automatic_ disposal (a.k.a. garbage collection), then
the cross references must not be used. I believe their contents
are either undefined or null, but in either case, you don't worry
about disposing them because the objects will take care of
themselves.

deadalnix <deadalnix gmail.com> writes:

Le 25/05/2012 16:35, Andrei Alexandrescu a écrit :
 On 5/25/12 12:07 AM, Mehrdad wrote:
 Now, there are two ways a FileStream can get destroyed:

 1. Through a manual call to FileStream.Dispose(). In this case, all
 embedded objects (e.g. SafeFileHandle) are *guaranteed* to be valid, so
 we simply flush the file and call SafeFileHandle.Dispose() to dispose of
 the managed resources, and then dispose of all the unmanaged resources
 (which are primitive fields, guaranteed to be accessible). Furthermore,
 the object suppresses its own finalizer.

 2. Through a garbage-collected call to ~FileStream(). In this case, the
 managed resources such as SafeFileHandle will be (or is already)
 destroyed SEPARATELY, and so we do _NOT_ access them. We ONLY dispose of
 the unmanaged resources, if any, and let the managed resources take care
 of themselves.


 B, and the object B has in turn a field referring to object A? That is:

 class C { C another; ~this() { writeln(another.another); } }

 void main() {
 auto a = new C;
 auto b = new C;
 a.another = b;
 b.another = a;
 }

 What happens then? Will the GC nullify references to destroyed objects,
 or will it put them in a zombie state?


 Thanks,

 Andrei

Here is what I suggest :
1/ what is in ~this stay in ~this. You cannot escape a reference to this 
or anything reached throw this of ~this. In other terms, the hidden this 
parameter get scope storage class. This avoid resurrection (something we 
really want, it have caused much trouble in java).
2/ The GC will call all finalizers on objects (here a and b) when they 
are garbage. Finalizers are called in undefined order.
3/ The GC will recycle the memory after all finalizers ran.

In the example above, both writeln will executed and then, both object's 
memory will be recycled.

"Michael" <pr m1xa.com> writes:

On Friday, 25 May 2012 at 14:35:58 UTC, Andrei Alexandrescu wrote:

 object B, and the object B has in turn a field referring to 
 object A? That is:

 class C { C another; ~this() { writeln(another.another); } }

 void main() {
     auto a = new C;
     auto b = new C;
     a.another = b;
     b.another = a;
 }

 What happens then? Will the GC nullify references to destroyed 
 objects, or will it put them in a zombie state?


 Thanks,

 Andrei

In this case the a and b objects will be collected by GC and 
memory freed.
It's a one of most popular questions about .NET GC. Maybe 
something was changed in .NET >= 4.5.

I have another question: there available a good example of 
idiomatic destructor usage in the D?
Something without calling a rt_ hooks on destroy?

deadalnix <deadalnix gmail.com> writes:

Le 24/05/2012 16:54, Andrei Alexandrescu a écrit :
 On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

 They may refer to one another.

 Andrei

So what ?

Each GC passes must, mark object that have to be removed, call finalizer 
on them all, THEN recycle memory.

So « zombie » object can still refer to one another in finalization.

The real problem is resurrection, which should be 100% forbiden and this 
must be enforced by the language (ie, the scopeness of this parameter is 
something important).

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 5/24/12 10:27 AM, deadalnix wrote:
 Le 24/05/2012 16:54, Andrei Alexandrescu a écrit :
 On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

 They may refer to one another.

 Andrei

 So what ?

 Each GC passes must, mark object that have to be removed, call finalizer
 on them all, THEN recycle memory.

 So « zombie » object can still refer to one another in finalization.

This is possible but not trivial as the state of zombie objects must be 
properly defined. Often such objects will fail their invariant (a 
reasonable state of a zombie object is with the correct vtable, no 
mutex, and all fields in the pre-constructor state).

 The real problem is resurrection, which should be 100% forbiden and this
 must be enforced by the language (ie, the scopeness of this parameter is
 something important).

As one aspect, calls to new should fail during destruction.


Andrei

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 24-05-2012 18:10, Andrei Alexandrescu wrote:
 On 5/24/12 10:27 AM, deadalnix wrote:
 Le 24/05/2012 16:54, Andrei Alexandrescu a écrit :
 On 5/24/12 9:28 AM, Alex Rønne Petersen wrote:
 The GC should (and probably does) assume at shutdown that all objects
 are unreferenced, and therefore reclaim and finalize them.

 They may refer to one another.

 Andrei

 So what ?

 Each GC passes must, mark object that have to be removed, call finalizer
 on them all, THEN recycle memory.

 So « zombie » object can still refer to one another in finalization.

 This is possible but not trivial as the state of zombie objects must be
 properly defined. Often such objects will fail their invariant (a
 reasonable state of a zombie object is with the correct vtable, no
 mutex, and all fields in the pre-constructor state).

 The real problem is resurrection, which should be 100% forbiden and this
 must be enforced by the language (ie, the scopeness of this parameter is
 something important).

 As one aspect, calls to new should fail during destruction.


 Andrei

Finalization happens once the world has been resumed, meaning GC 
allocation (and even explicit deallocation) should be perfectly safe. 
This is absolutely essential: Finalization models where finalizers run 
in a paused world are doomed to fail miserably.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

Michel Fortin <michel.fortin michelf.com> writes:

On 2012-05-24 12:21:01 +0000, Alex Rønne Petersen <alex lycus.org> said:

 Hi,
 
 http://dlang.org/class.html#Destructor
 
 "The garbage collector is not guaranteed to run the destructor for all 
 unreferenced objects."
 
 What the *hell*? So resources are allowed to arbitrarily leak and the 
 programmer has to actually expect this to happen?
 
 I really, really hope that this is a documentation error or early 
 design decision that has since been rectified but with lack of 
 documentation updates.

I think it means that objects not collected when the program terminates 
will never be, and thus the destructor will not be called.

There's also the issue of false pointers that can prevent some objects 
from being collected.

More generally, you can't really count on the destructor being called 
because the GC is free to decide when to recycle memory. An 
implementation that never collects and always allocate new memory is a 
perfectly valid GC implementation, even though it might not be very 
practical in most cases.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/

=?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:

On 24-05-2012 14:43, Michel Fortin wrote:
 On 2012-05-24 12:21:01 +0000, Alex Rønne Petersen <alex lycus.org> said:

 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

 I think it means that objects not collected when the program terminates
 will never be, and thus the destructor will not be called.

That's silly. Microsoft .NET and Mono have been running finalizers on 
runtime shutdown for a long time without problems.

 There's also the issue of false pointers that can prevent some objects
 from being collected.

Right, but I think if we guarantee finalization at shutdown, that 
shouldn't matter.

 More generally, you can't really count on the destructor being called
 because the GC is free to decide when to recycle memory. An
 implementation that never collects and always allocate new memory is a
 perfectly valid GC implementation, even though it might not be very
 practical in most cases.

Even such an implementation should free all memory at shutdown, and, at 
that point, run finalizers.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

Michel Fortin <michel.fortin michelf.com> writes:

On 2012-05-24 14:35:45 +0000, Alex Rønne Petersen <alex lycus.org> said:

 On 24-05-2012 14:43, Michel Fortin wrote:
 I think it means that objects not collected when the program terminates
 will never be, and thus the destructor will not be called.

 
 That's silly. Microsoft .NET and Mono have been running finalizers on 
 runtime shutdown for a long time without problems.
 
 There's also the issue of false pointers that can prevent some objects
 from being collected.

 
 Right, but I think if we guarantee finalization at shutdown, that 
 shouldn't matter.

.NET is a virtual machine which has total control of all the code that 
runs. D has to work with the C runtime and other non-D code that might 
use D code more or less directly.

The interesting question is *how* do they do it without causing 
thread-safety issues? Perhaps they wait until all threads have been 
terminated, or perhaps they're unsafe too. Would doing the same be 
appropriate for D? Does this mean we can't use anything using static 
and global variables in destructors because they might have been 
finalized? If so, how does the compiler detects this?

If there's a way and it is not overly costly in performance, it might 
be a good idea. But we're not in a virtual machine, there are more 
constrains we must abide to and we might have to make different 
compromises.

Enlarging the scope of all this, there is already some impending 
problems with how destructors are handled in D that can easily create 
low-level races. And this applies to struct destructors too, when the 
struct is put on the heap or is a member of an object. We're in the 
need of a global solution for all this.

<http://d.puremagic.com/issues/show_bug.cgi?id=4621>
<http://d.puremagic.com/issues/show_bug.cgi?id=4624>


 More generally, you can't really count on the destructor being called
 because the GC is free to decide when to recycle memory. An
 implementation that never collects and always allocate new memory is a
 perfectly valid GC implementation, even though it might not be very
 practical in most cases.

 
 Even such an implementation should free all memory at shutdown, and, at 
 that point, run finalizers.

Well, not according to the current spec.

It could make sense to do so, although I'm not sure. Freeing "external" 
resources can mean many things: if we're talking about externally 
allocated memory, open files, sockets, mutexes, etc., there's no need 
to finalize that at the end of the program: the OS will do the cleanup 
for us. If we're talking about advisory locks on files, then there's 
definitely a need to clear the lock, although I'm not sure it makes 
sense to make the release dependent on a non-deterministic GC.

So I'm curious, what resource are we trying to free here?


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/

=?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:

On 24-05-2012 17:18, Michel Fortin wrote:
 On 2012-05-24 14:35:45 +0000, Alex Rønne Petersen <alex lycus.org> said:

 On 24-05-2012 14:43, Michel Fortin wrote:
 I think it means that objects not collected when the program terminates
 will never be, and thus the destructor will not be called.

 That's silly. Microsoft .NET and Mono have been running finalizers on
 runtime shutdown for a long time without problems.

 There's also the issue of false pointers that can prevent some objects
 from being collected.

 Right, but I think if we guarantee finalization at shutdown, that
 shouldn't matter.

 .NET is a virtual machine which has total control of all the code that
 runs. D has to work with the C runtime and other non-D code that might
 use D code more or less directly.

So does .NET. It just does it with some trampoline magic. Non-D code 
just has to call thread_attachThis()/thread_detachThis() and friends.

 The interesting question is *how* do they do it without causing
 thread-safety issues? Perhaps they wait until all threads have been
 terminated, or perhaps they're unsafe too. Would doing the same be
 appropriate for D? Does this mean we can't use anything using static and
 global variables in destructors because they might have been finalized?
 If so, how does the compiler detects this?

The CLR waits for all threads to have shut down (including daemon 
threads). That's not equal to all static/__gshared data being nulled 
out, mind you. All data is cleared out once all threads, including the 
finalizer thread, have been terminated one way or another.

And yes, it is thread safe.

 If there's a way and it is not overly costly in performance, it might be
 a good idea. But we're not in a virtual machine, there are more
 constrains we must abide to and we might have to make different
 compromises.

Don't need a virtual machine. We already have the infrastructure to do 
it in druntime, we just need to make the lifetime and GC code respect 


 Enlarging the scope of all this, there is already some impending
 problems with how destructors are handled in D that can easily create
 low-level races. And this applies to struct destructors too, when the
 struct is put on the heap or is a member of an object. We're in the need
 of a global solution for all this.

 <http://d.puremagic.com/issues/show_bug.cgi?id=4621>
 <http://d.puremagic.com/issues/show_bug.cgi?id=4624>


 More generally, you can't really count on the destructor being called
 because the GC is free to decide when to recycle memory. An
 implementation that never collects and always allocate new memory is a
 perfectly valid GC implementation, even though it might not be very
 practical in most cases.

 Even such an implementation should free all memory at shutdown, and,
 at that point, run finalizers.

 Well, not according to the current spec.

No, but a sane implementation made for a sane spec should. ;)

 It could make sense to do so, although I'm not sure. Freeing "external"
 resources can mean many things: if we're talking about externally
 allocated memory, open files, sockets, mutexes, etc., there's no need to
 finalize that at the end of the program: the OS will do the cleanup for
 us. If we're talking about advisory locks on files, then there's
 definitely a need to clear the lock, although I'm not sure it makes
 sense to make the release dependent on a non-deterministic GC.

We just need a dispose pattern whereby explicit dispose() instructs the 
GC to not finalize.

 So I'm curious, what resource are we trying to free here?

None. I just came across it in the docs and found it completely insane.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

"Tove" <tove fransson.se> writes:

On Thursday, 24 May 2012 at 15:43:57 UTC, Alex Rønne Petersen 
wrote:
 We just need a dispose pattern whereby explicit dispose() 
 instructs the GC to not finalize.

 So I'm curious, what resource are we trying to free here?

 None. I just came across it in the docs and found it completely 
 insane.

Hmm... well, as long as it's optional behavior... as in my case I 
actually want to go in the opposite direction... short-lived tool 
which claims x resources and is run once for every file...

So in this case, resources should be "free:ed" _unless_ it's at 
program termination... as then it just slows down the shutdown 
procedure, the OS reclaim it faster anyway.

"Peter Alexander" <peter.alexander.au gmail.com> writes:

On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen 
wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor 
 for all unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak 
 and the programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or 
 early design decision that has since been rectified but with 
 lack of documentation updates.

I'm pretty sure it's the same in Java.

Finalizers (a.k.a. class destructors) are practically useless.

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Thu, 24 May 2012 08:54:45 -0400, Peter Alexander  =

<peter.alexander.au gmail.com> wrote:

 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex R=C3=B8nne Petersen wro=

te:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for al=


l  =

 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the=


  =

 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early  =


 design decision that has since been rectified but with lack of  =


 documentation updates.

 I'm pretty sure it's the same in Java.

 Finalizers (a.k.a. class destructors) are practically useless.

 From Java spec:

"The Java programming language does not specify how soon a finalizer wil=
l  =

be invoked, except to say that it will happen before the storage for the=
  =

object is reused"

So yeah, there is no guarantee when a finalizer will be invoked.

However, I'd tend to believe Java implementations will attempt to invoke=
  =

all finalizers of objects left on the heap at program shutdown.

I think D is the same too, as long as termination is normal (i.e. not fr=
om  =

throwing an Error).

-Steve

Michel Fortin <michel.fortin michelf.com> writes:

On 2012-05-24 13:38:01 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 However, I'd tend to believe Java implementations will attempt to invoke
 all finalizers of objects left on the heap at program shutdown.

In Java you can call System.runFinalizersOnExit(true), but the default 
is false and this method has been deprecated because unsafe in 
multithreaded environments.

<http://docs.oracle.com/javase/6/docs/api/java/lang/System.html#runFinalization()>


-- 


Michel Fortin
michel.fortin michelf.com
http://michelf.com/

=?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:

On 24-05-2012 16:00, Michel Fortin wrote:
 On 2012-05-24 13:38:01 +0000, "Steven Schveighoffer"
 <schveiguy yahoo.com> said:

 However, I'd tend to believe Java implementations will attempt to invoke
 all finalizers of objects left on the heap at program shutdown.

 In Java you can call System.runFinalizersOnExit(true), but the default
 is false and this method has been deprecated because unsafe in
 multithreaded environments.

 <http://docs.oracle.com/javase/6/docs/api/java/lang/System.html#runFinalization()>

It's only deprecated because Java's way of handling threading and 

system that actually works.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

Russel Winder <russel winder.org.uk> writes:

On Thu, 2012-05-24 at 09:38 -0400, Steven Schveighoffer wrote:
[...]
 However, I'd tend to believe Java implementations will attempt to invoke =

=20
 all finalizers of objects left on the heap at program shutdown.

As far as I am aware Java implementations do no finalization on exit
unless System.runFinalizersOnExit(true) has been called. This method is
deprecated since it can cause incorrect finalization. But like all
things Java that have been deprecated, they never actually go away.

There is System.runFinalization() which executes finalize on all objects
in the pending finalization queue.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 24-05-2012 16:03, Russel Winder wrote:
 On Thu, 2012-05-24 at 09:38 -0400, Steven Schveighoffer wrote:
 [...]
 However, I'd tend to believe Java implementations will attempt to invoke
 all finalizers of objects left on the heap at program shutdown.

 As far as I am aware Java implementations do no finalization on exit
 unless System.runFinalizersOnExit(true) has been called. This method is
 deprecated since it can cause incorrect finalization. But like all
 things Java that have been deprecated, they never actually go away.

 There is System.runFinalization() which executes finalize on all objects
 in the pending finalization queue.

We should really expose a waitForFinalizers() function in core.memory.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 24-05-2012 15:38, Steven Schveighoffer wrote:
 On Thu, 24 May 2012 08:54:45 -0400, Peter Alexander
 <peter.alexander.au gmail.com> wrote:

 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for
 all unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

 I'm pretty sure it's the same in Java.

 Finalizers (a.k.a. class destructors) are practically useless.

  From Java spec:

 "The Java programming language does not specify how soon a finalizer
 will be invoked, except to say that it will happen before the storage
 for the object is reused"

 So yeah, there is no guarantee when a finalizer will be invoked.

 However, I'd tend to believe Java implementations will attempt to invoke
 all finalizers of objects left on the heap at program shutdown.

 I think D is the same too, as long as termination is normal (i.e. not
 from throwing an Error).


This shouldn't just be an implementation detail IMO. It should be a 
documented feature.

 -Steve

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

deadalnix <deadalnix gmail.com> writes:

Le 24/05/2012 14:54, Peter Alexander a écrit :
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

 I'm pretty sure it's the same in Java.

 Finalizers (a.k.a. class destructors) are practically useless.

Java finalizer is a pretty bad design decision. Let's not reproduce 
error made in Java in D's destructors.

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Thu, 24 May 2012 09:47:23 -0400, deadalnix <deadalnix gmail.com> wrot=
e:

 Le 24/05/2012 14:54, Peter Alexander a =C3=A9crit :
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex R=C3=B8nne Petersen wr=


ote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for a=



ll
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and th=



e
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

 I'm pretty sure it's the same in Java.

 Finalizers (a.k.a. class destructors) are practically useless.

 Java finalizer is a pretty bad design decision. Let's not reproduce  =

 error made in Java in D's destructors.

You actually need a finalizer if you want to have resources that aren't =
GC  =

allocated.

-Steve

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 24-05-2012 15:49, Steven Schveighoffer wrote:
 On Thu, 24 May 2012 09:47:23 -0400, deadalnix <deadalnix gmail.com> wrote:

 Le 24/05/2012 14:54, Peter Alexander a écrit :
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

 I'm pretty sure it's the same in Java.

 Finalizers (a.k.a. class destructors) are practically useless.

 Java finalizer is a pretty bad design decision. Let's not reproduce
 error made in Java in D's destructors.

 You actually need a finalizer if you want to have resources that aren't
 GC allocated.

 -Steve

But that doesn't mean we should have Java finalization. There are many 
different forms of finalization, and I do agree that Java is the worst 
of all of them.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Thu, 24 May 2012 10:30:02 -0400, Alex R=C3=B8nne Petersen <alex lycus=
.org>  =

wrote:

 On 24-05-2012 15:49, Steven Schveighoffer wrote:
 On Thu, 24 May 2012 09:47:23 -0400, deadalnix <deadalnix gmail.com>  =


 wrote:

 Le 24/05/2012 14:54, Peter Alexander a =C3=A9crit :
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex R=C3=B8nne Petersen =




wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for=





  =

 all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and =





the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

 I'm pretty sure it's the same in Java.

 Finalizers (a.k.a. class destructors) are practically useless.

 Java finalizer is a pretty bad design decision. Let's not reproduce
 error made in Java in D's destructors.

 You actually need a finalizer if you want to have resources that aren=


't
 GC allocated.

 -Steve

 But that doesn't mean we should have Java finalization. There are many=

  =

 different forms of finalization, and I do agree that Java is the worst=

  =

 of all of them.

I only found one definition for finalizer on wikipedia, and it fits D's =
 =

definition.

What I think we need is a dispose pattern for objects, like Tango has.

-Steve

=?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:

On 24-05-2012 16:53, Steven Schveighoffer wrote:
 On Thu, 24 May 2012 10:30:02 -0400, Alex Rønne Petersen <alex lycus.org>
 wrote:

 On 24-05-2012 15:49, Steven Schveighoffer wrote:
 On Thu, 24 May 2012 09:47:23 -0400, deadalnix <deadalnix gmail.com>
 wrote:

 Le 24/05/2012 14:54, Peter Alexander a écrit :
 On Thursday, 24 May 2012 at 12:21:02 UTC, Alex Rønne Petersen wrote:
 Hi,

 http://dlang.org/class.html#Destructor

 "The garbage collector is not guaranteed to run the destructor for
 all
 unreferenced objects."

 What the *hell*? So resources are allowed to arbitrarily leak and the
 programmer has to actually expect this to happen?

 I really, really hope that this is a documentation error or early
 design decision that has since been rectified but with lack of
 documentation updates.

 I'm pretty sure it's the same in Java.

 Finalizers (a.k.a. class destructors) are practically useless.

 Java finalizer is a pretty bad design decision. Let's not reproduce
 error made in Java in D's destructors.

 You actually need a finalizer if you want to have resources that aren't
 GC allocated.

 -Steve

 But that doesn't mean we should have Java finalization. There are many
 different forms of finalization, and I do agree that Java is the worst
 of all of them.

 I only found one definition for finalizer on wikipedia, and it fits D's
 definition.

 What I think we need is a dispose pattern for objects, like Tango has.

 -Steve

Just look at /usr/include/gc/gc.h (from libgc, the Boehm-Demers-Weiser 
GC). It has 3 (if not 4) different kinds of finalization. To be 
specific, the finalization I believe we need is the *_no_order behavior. 


-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org

Jacob Carlborg <doob me.com> writes:

On 2012-05-24 16:53, Steven Schveighoffer wrote:

 What I think we need is a dispose pattern for objects, like Tango has.

Object.dispose in Tango is called on scope exit if the object is 
variable is declared "scope". "scope" is deprecated in D2.

-- 
/Jacob Carlborg

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Thu, 24 May 2012 11:04:06 -0400, Jacob Carlborg <doob me.com> wrote:

 On 2012-05-24 16:53, Steven Schveighoffer wrote:

 What I think we need is a dispose pattern for objects, like Tango has.

 Object.dispose in Tango is called on scope exit if the object is  
 variable is declared "scope". "scope" is deprecated in D2.

We can easily hook that in object.clear, which any scoped library  
implementation should be calling.

-Steve

"David Nadlinger" <see klickverbot.at> writes:

On Thursday, 24 May 2012 at 15:04:06 UTC, Jacob Carlborg wrote:
 On 2012-05-24 16:53, Steven Schveighoffer wrote:
 What I think we need is a dispose pattern for objects, like 
 Tango has.

 Object.dispose in Tango is called on scope exit if the object 
 is variable is declared "scope". "scope" is deprecated in D2.

As for replicating that functionality, Scoped!T could always 
check for a magic dispose() method (maybe with another name, a 
marker parameter, …) and if it exists, call it on destruction.

But of course, the »universality« of the Tango runtimes 
solution is lost with this.

David

deadalnix <deadalnix gmail.com> writes:

Le 24/05/2012 15:49, Steven Schveighoffer a écrit :
 You actually need a finalizer if you want to have resources that aren't
 GC allocated.

 -Steve

Indeed. But java's way of doing it is very poor.

"Mehrdad" <wfunction hotmail.com> writes:

On Thursday, 24 May 2012 at 13:49:38 UTC, Steven Schveighoffer 
wrote:
 You actually need a finalizer if you want to have resources 
 that aren't GC allocated.


Does that ring a bell? ;)