www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Destroying structs (literally)

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Dear community, are you ready for this?

https://issues.dlang.org/show_bug.cgi?id=2834
https://github.com/D-Programming-Language/druntime/pull/864

We must do it, and the way I see it the earlier the better. Shall we do 
it in 2.067?

This is a significant change of behavior. Should we provide a temporary 
flag or attribute to disable it?


Thanks,

Andrei
Aug 28 2014
next sibling parent Orvid King <blah38621 gmail.com> writes:
On 8/28/2014 9:21 PM, Andrei Alexandrescu wrote:

 Dear community, are you ready for this?

 https://issues.dlang.org/show_bug.cgi?id=2834
 https://github.com/D-Programming-Language/druntime/pull/864

 We must do it, and the way I see it the earlier the better. Shall we do
 it in 2.067?

 This is a significant change of behavior. Should we provide a temporary
 flag or attribute to disable it?


 Thanks,

 Andrei


There should be no breaking behavior, provided the code was previously 
calling destroy, and not simply manually calling the destructor directly 
for a heap allocated struct, as I've had to make sure that it's not 
called more than once, otherwise the existing unittests wouldn't pass.

The only potentially breaking change introduced in this, which was 
previously present in one of the struct destructor tests, is that 
allocating in the heap allocated struct's destructor isn't allowed.
Aug 28 2014
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Aug 28, 2014 at 07:21:04PM -0700, Andrei Alexandrescu via Digitalmars-d
wrote:

 Dear community, are you ready for this?
 
 https://issues.dlang.org/show_bug.cgi?id=2834
 https://github.com/D-Programming-Language/druntime/pull/864
 
 We must do it, and the way I see it the earlier the better. Shall we
 do it in 2.067?


Let's do it. Issue 2834 was filed in 2009. It's almost winning the
longest-open-bug award. It's about time we did something.



 This is a significant change of behavior. Should we provide a
 temporary flag or attribute to disable it?


[...]

Maybe a more relevant question might be, is there any existing code that
*isn't* broken by structs not being destructed? (D-structed, har har.)


T

-- 
If I were two-faced, would I be wearing this one? -- Abraham Lincoln
Aug 28 2014
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 29 August 2014 at 02:38:54 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 Maybe a more relevant question might be, is there any existing 
 code that
 *isn't* broken by structs not being destructed? (D-structed, 
 har har.)


Well, this new change *could* greatly increase the amount of 
"allocation during destruction" errors we are getting. I've seen 
a fair share of these in learn, whereas a class destructor 
allocates.

Structs will now also be more vulnerable to this problem too. I 
wouldn't be surprised if this pull instantaneously introduced a 
fair amount of breakage in client code.
Aug 28 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Friday, 29 August 2014 at 06:39:20 UTC, monarch_dodra wrote:

 On Friday, 29 August 2014 at 02:38:54 UTC, H. S. Teoh via 
 Digitalmars-d wrote:

 Maybe a more relevant question might be, is there any existing 
 code that
 *isn't* broken by structs not being destructed? (D-structed, 
 har har.)


 Well, this new change *could* greatly increase the amount of 
 "allocation during destruction" errors we are getting. I've 
 seen a fair share of these in learn, whereas a class destructor 
 allocates.

 Structs will now also be more vulnerable to this problem too. I 
 wouldn't be surprised if this pull instantaneously introduced a 
 fair amount of breakage in client code.


Jacob Carlborg just recently brought this up in another thread. 
Isn't it kind of consensus that calling a destructor from the GC 
is not a good idea because of the restrictions that apply in this 
context? Andrei even wanted to deprecate destructors for classes 
because of this. Maybe a better direction would be to separate 
the concepts of destruction and finalization, and introduce two 
kinds of "destructors" for them.
Aug 29 2014
next sibling parent Jacob Carlborg <doob me.com> writes:
On 29/08/14 12:53, "Marc Schütz" <schuetzm gmx.net>" wrote:


 Jacob Carlborg just recently brought this up in another thread. Isn't it
 kind of consensus that calling a destructor from the GC is not a good
 idea because of the restrictions that apply in this context? Andrei even
 wanted to deprecate destructors for classes because of this. Maybe a
 better direction would be to separate the concepts of destruction and
 finalization, and introduce two kinds of "destructors" for them.


Yeah, Tango for D1 added a new method, "dispose", to Object. It's called 
when "delete" or "scope" is used.

-- 
/Jacob Carlborg
Aug 29 2014
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/29/14, 3:53 AM, "Marc Schütz" <schuetzm gmx.net>" wrote:

 Jacob Carlborg just recently brought this up in another thread. Isn't it
 kind of consensus that calling a destructor from the GC is not a good
 idea because of the restrictions that apply in this context? Andrei even
 wanted to deprecate destructors for classes because of this. Maybe a
 better direction would be to separate the concepts of destruction and
 finalization, and introduce two kinds of "destructors" for them.


I think we need to stay with what we have. Adding a distinct kind of 
destructor might be interesting. -- Andrei
Aug 29 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Friday, 29 August 2014 at 19:01:51 UTC, Andrei Alexandrescu 
wrote:

 On 8/29/14, 3:53 AM, "Marc Schütz" <schuetzm gmx.net>" wrote:

 Jacob Carlborg just recently brought this up in another 
 thread. Isn't it
 kind of consensus that calling a destructor from the GC is not 
 a good
 idea because of the restrictions that apply in this context? 
 Andrei even
 wanted to deprecate destructors for classes because of this. 
 Maybe a
 better direction would be to separate the concepts of 
 destruction and
 finalization, and introduce two kinds of "destructors" for 
 them.


 I think we need to stay with what we have. Adding a distinct 
 kind of destructor might be interesting. -- Andrei


Our idea was that an additional destructor (let's call it a 
finalizer) would be helpful because it is backward compatible. 
The compiler could make some validity checks on it, at the least 
make it nothrow, maybe  nogc (but I believe we can relax this 
restriction), pure (?). Disallowing access to references (because 
they could pointer to already destroyed objects) is unfortunately 
not feasible, because we can't distinguish GC pointers from other 
ones. To avoid the need for code duplication, finalizers could 
always be called implicitly by destructors (assuming everything 
that is allowed in finalizers is also permitted in destructors).

Calling destructors from the GC could later be phased out. It is 
technically not a breaking change, because there never was a 
guarantee that they'd be called anyway.
Aug 29 2014
parent reply Orvid King <blah38621 gmail.com> writes:
On 8/29/2014 2:52 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:

 On Friday, 29 August 2014 at 19:01:51 UTC, Andrei Alexandrescu wrote:

 On 8/29/14, 3:53 AM, "Marc Schütz" <schuetzm gmx.net>" wrote:

 Jacob Carlborg just recently brought this up in another thread. Isn't it
 kind of consensus that calling a destructor from the GC is not a good
 idea because of the restrictions that apply in this context? Andrei even
 wanted to deprecate destructors for classes because of this. Maybe a
 better direction would be to separate the concepts of destruction and
 finalization, and introduce two kinds of "destructors" for them.


 I think we need to stay with what we have. Adding a distinct kind of
 destructor might be interesting. -- Andrei


 Our idea was that an additional destructor (let's call it a finalizer)
 would be helpful because it is backward compatible. The compiler could
 make some validity checks on it, at the least make it nothrow, maybe
  nogc (but I believe we can relax this restriction), pure (?).
 Disallowing access to references (because they could pointer to already
 destroyed objects) is unfortunately not feasible, because we can't
 distinguish GC pointers from other ones. To avoid the need for code
 duplication, finalizers could always be called implicitly by destructors
 (assuming everything that is allowed in finalizers is also permitted in
 destructors).

 Calling destructors from the GC could later be phased out. It is
 technically not a breaking change, because there never was a guarantee
 that they'd be called anyway.


I would say that all of those restrictions, except for nothrow, are 
dependent on the current GC implementation. It is possible to write the 
GC in such a way that you can do GC allocations in a destructor, as well 
as access any GC references you want. The only thing with the GC 
references is that there's no way to guarantee that the referenced 
objects won't have already had their destructor called when the current 
destructor is being called.
Aug 29 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 30 August 2014 at 03:54:41 UTC, Orvid King wrote:

 On 8/29/2014 2:52 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:

 On Friday, 29 August 2014 at 19:01:51 UTC, Andrei Alexandrescu 
 wrote:

 On 8/29/14, 3:53 AM, "Marc Schütz" <schuetzm gmx.net>" wrote:

 Jacob Carlborg just recently brought this up in another 
 thread. Isn't it
 kind of consensus that calling a destructor from the GC is 
 not a good
 idea because of the restrictions that apply in this context? 
 Andrei even
 wanted to deprecate destructors for classes because of this. 
 Maybe a
 better direction would be to separate the concepts of 
 destruction and
 finalization, and introduce two kinds of "destructors" for 
 them.


 I think we need to stay with what we have. Adding a distinct 
 kind of
 destructor might be interesting. -- Andrei


 Our idea was that an additional destructor (let's call it a 
 finalizer)
 would be helpful because it is backward compatible. The 
 compiler could
 make some validity checks on it, at the least make it nothrow, 
 maybe
  nogc (but I believe we can relax this restriction), pure (?).
 Disallowing access to references (because they could pointer 
 to already
 destroyed objects) is unfortunately not feasible, because we 
 can't
 distinguish GC pointers from other ones. To avoid the need for 
 code
 duplication, finalizers could always be called implicitly by 
 destructors
 (assuming everything that is allowed in finalizers is also 
 permitted in
 destructors).

 Calling destructors from the GC could later be phased out. It 
 is
 technically not a breaking change, because there never was a 
 guarantee
 that they'd be called anyway.


 I would say that all of those restrictions, except for nothrow, 
 are dependent on the current GC implementation. It is possible 
 to write the GC in such a way that you can do GC allocations in 
 a destructor, as well as access any GC references you want. The 
 only thing with the GC references is that there's no way to 
 guarantee that the referenced objects won't have already had 
 their destructor called when the current destructor is being 
 called.


Hmmm... could the GC zero those references that it already 
destroyed, before calling the finalizer? Don't know how this 
would affect performance, but it would only be necessary if a 
finalizer exists (could even be restricted to those references 
that are accessible from non-trivial finalizers, i.e. if a struct 
has GCed pointers and an embedded struct member with a finalizer, 
but no finalizer of its own, the compiler would probably generate 
one that only calls the member's finalizer, but this would have 
no access to its parent's pointers).

You're right that many of the restrictions are only necessary 
because of the current GC implementation. Even the fact that 
garbage collection can happen in any thread could theoretically 
be changed. Even more complicated: I can imagine that with the 
upcoming allocator work there could be several different GC 
implementations, even used in parallel in the same program, each 
with different capabilities and restrictions. It's clear that 
this requires coordination.
Aug 30 2014
parent reply Orvid King <blah38621 gmail.com> writes:
On 8/30/2014 4:22 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:

 On Saturday, 30 August 2014 at 03:54:41 UTC, Orvid King wrote:

 On 8/29/2014 2:52 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?=
 <schuetzm gmx.net>" wrote:

 On Friday, 29 August 2014 at 19:01:51 UTC, Andrei Alexandrescu wrote:

 On 8/29/14, 3:53 AM, "Marc Schütz" <schuetzm gmx.net>" wrote:

 Jacob Carlborg just recently brought this up in another thread.
 Isn't it
 kind of consensus that calling a destructor from the GC is not a good
 idea because of the restrictions that apply in this context? Andrei
 even
 wanted to deprecate destructors for classes because of this. Maybe a
 better direction would be to separate the concepts of destruction and
 finalization, and introduce two kinds of "destructors" for them.


 I think we need to stay with what we have. Adding a distinct kind of
 destructor might be interesting. -- Andrei


 Our idea was that an additional destructor (let's call it a finalizer)
 would be helpful because it is backward compatible. The compiler could
 make some validity checks on it, at the least make it nothrow, maybe
  nogc (but I believe we can relax this restriction), pure (?).
 Disallowing access to references (because they could pointer to already
 destroyed objects) is unfortunately not feasible, because we can't
 distinguish GC pointers from other ones. To avoid the need for code
 duplication, finalizers could always be called implicitly by destructors
 (assuming everything that is allowed in finalizers is also permitted in
 destructors).

 Calling destructors from the GC could later be phased out. It is
 technically not a breaking change, because there never was a guarantee
 that they'd be called anyway.


 I would say that all of those restrictions, except for nothrow, are
 dependent on the current GC implementation. It is possible to write
 the GC in such a way that you can do GC allocations in a destructor,
 as well as access any GC references you want. The only thing with the
 GC references is that there's no way to guarantee that the referenced
 objects won't have already had their destructor called when the
 current destructor is being called.


 Hmmm... could the GC zero those references that it already destroyed,
 before calling the finalizer? Don't know how this would affect
 performance, but it would only be necessary if a finalizer exists (could
 even be restricted to those references that are accessible from
 non-trivial finalizers, i.e. if a struct has GCed pointers and an
 embedded struct member with a finalizer, but no finalizer of its own,
 the compiler would probably generate one that only calls the member's
 finalizer, but this would have no access to its parent's pointers).

 You're right that many of the restrictions are only necessary because of
 the current GC implementation. Even the fact that garbage collection can
 happen in any thread could theoretically be changed. Even more
 complicated: I can imagine that with the upcoming allocator work there
 could be several different GC implementations, even used in parallel in
 the same program, each with different capabilities and restrictions.
 It's clear that this requires coordination.


The references issue can be gotten around by marking an allocation that 
needs finalization as if it were alive. It does mean however that 
finalizable allocations will live through more than one collection. I 
believe this is how .Net currently handles them, as I don't remember 
anything in the spec about restrictions on what's referenced in 
destructors, nor have I had issues referencing otherwise dead 
allocations in them.
Aug 30 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 30 August 2014 at 15:18:52 UTC, Orvid King wrote:

 On 8/30/2014 4:22 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:

 On Saturday, 30 August 2014 at 03:54:41 UTC, Orvid King wrote:

 On 8/29/2014 2:52 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?=
 <schuetzm gmx.net>" wrote:

 On Friday, 29 August 2014 at 19:01:51 UTC, Andrei 
 Alexandrescu wrote:

 On 8/29/14, 3:53 AM, "Marc Schütz" <schuetzm gmx.net>" 
 wrote:

 Jacob Carlborg just recently brought this up in another 
 thread.
 Isn't it
 kind of consensus that calling a destructor from the GC is 
 not a good
 idea because of the restrictions that apply in this 
 context? Andrei
 even
 wanted to deprecate destructors for classes because of 
 this. Maybe a
 better direction would be to separate the concepts of 
 destruction and
 finalization, and introduce two kinds of "destructors" for 
 them.


 I think we need to stay with what we have. Adding a 
 distinct kind of
 destructor might be interesting. -- Andrei


 Our idea was that an additional destructor (let's call it a 
 finalizer)
 would be helpful because it is backward compatible. The 
 compiler could
 make some validity checks on it, at the least make it 
 nothrow, maybe
  nogc (but I believe we can relax this restriction), pure 
 (?).
 Disallowing access to references (because they could pointer 
 to already
 destroyed objects) is unfortunately not feasible, because we 
 can't
 distinguish GC pointers from other ones. To avoid the need 
 for code
 duplication, finalizers could always be called implicitly by 
 destructors
 (assuming everything that is allowed in finalizers is also 
 permitted in
 destructors).

 Calling destructors from the GC could later be phased out. 
 It is
 technically not a breaking change, because there never was a 
 guarantee
 that they'd be called anyway.


 I would say that all of those restrictions, except for 
 nothrow, are
 dependent on the current GC implementation. It is possible to 
 write
 the GC in such a way that you can do GC allocations in a 
 destructor,
 as well as access any GC references you want. The only thing 
 with the
 GC references is that there's no way to guarantee that the 
 referenced
 objects won't have already had their destructor called when 
 the
 current destructor is being called.


 Hmmm... could the GC zero those references that it already 
 destroyed,
 before calling the finalizer? Don't know how this would affect
 performance, but it would only be necessary if a finalizer 
 exists (could
 even be restricted to those references that are accessible from
 non-trivial finalizers, i.e. if a struct has GCed pointers and 
 an
 embedded struct member with a finalizer, but no finalizer of 
 its own,
 the compiler would probably generate one that only calls the 
 member's
 finalizer, but this would have no access to its parent's 
 pointers).

 You're right that many of the restrictions are only necessary 
 because of
 the current GC implementation. Even the fact that garbage 
 collection can
 happen in any thread could theoretically be changed. Even more
 complicated: I can imagine that with the upcoming allocator 
 work there
 could be several different GC implementations, even used in 
 parallel in
 the same program, each with different capabilities and 
 restrictions.
 It's clear that this requires coordination.


 The references issue can be gotten around by marking an 
 allocation that needs finalization as if it were alive. It does 
 mean however that finalizable allocations will live through 
 more than one collection. I believe this is how .Net currently 
 handles them, as I don't remember anything in the spec about 
 restrictions on what's referenced in destructors, nor have I 
 had issues referencing otherwise dead allocations in them.


The problem is not only dereferencing those pointers (though 
depending on how the GC works even this might be racy, i.e. the 
memory location could have been reused already), but that a 
destructor/finalizer already ran on the referenced object. It is 
thus potentially in an invalid state. Even copying it is 
dangerous, because you're then creating a live object from that 
invalid state which will itself be destroyed again at some point. 
This might lead to double frees of depending manually allocated 
objects, or potentially other problems. Then there's the 
possibility to "resurrect" such an object by storing a reference 
to it somewhere else during finalization.

Doing the scanning and finalization separately like this is 
_probably_ safe:

1. Marking phase as usual.
2. Select the objects that have a finalizer, and clear all 
references in them that point to objects that are now 
unreachable. (This requires a precise GC.)
3. Call the finalizers.

Not sure what to do about things that may or may not be 
references.
Aug 30 2014
parent Orvid King <blah38621 gmail.com> writes:
On 8/30/2014 10:35 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
<schuetzm gmx.net>" wrote:

 The problem is not only dereferencing those pointers (though depending
 on how the GC works even this might be racy, i.e. the memory location
 could have been reused already), but that a destructor/finalizer already
 ran on the referenced object. It is thus potentially in an invalid
 state. Even copying it is dangerous, because you're then creating a live
 object from that invalid state which will itself be destroyed again at
 some point. This might lead to double frees of depending manually
 allocated objects, or potentially other problems. Then there's the
 possibility to "resurrect" such an object by storing a reference to it
 somewhere else during finalization.

 Doing the scanning and finalization separately like this is _probably_
 safe:

 1. Marking phase as usual.
 2. Select the objects that have a finalizer, and clear all references in
 them that point to objects that are now unreachable. (This requires a
 precise GC.)
 3. Call the finalizers.

 Not sure what to do about things that may or may not be references.


It's perfectly fine to not be able to guarantee the order of the 
finalization, but we can guarantee that the values referenced by the 
allocation being finalized are still allocated. It's up to the user to 
understand that the finalizable object they are trying to reference in 
their destructor may already have been finalized.
Aug 30 2014
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu
wrote:

 Dear community, are you ready for this?

 https://issues.dlang.org/show_bug.cgi?id=2834
 https://github.com/D-Programming-Language/druntime/pull/864

 We must do it, and the way I see it the earlier the better. 
 Shall we do it in 2.067?

 This is a significant change of behavior. Should we provide a 
 temporary flag or attribute to disable it?


 Thanks,

 Andrei


I have no idea what the decision is.
Aug 28 2014
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu 
wrote:

 Dear community, are you ready for this?

 https://issues.dlang.org/show_bug.cgi?id=2834
 https://github.com/D-Programming-Language/druntime/pull/864

 We must do it, and the way I see it the earlier the better. 
 Shall we do it in 2.067?

 This is a significant change of behavior. Should we provide a 
 temporary flag or attribute to disable it?


It will _fix_ a lot of code, I expect, whereas it will probably 
break very little. The main thing that it would break would be if 
there's any GC heap allocation in a struct's destructor, but that 
will only matter if the struct is on the GC heap, and I think 
that it's something that we're pretty much stuck with. Certainly, 
that's not much of a loss in comparison to all of the bugs 
created by stuff like an array of std.stdio.Files. Maybe there's 
a problem if someone was manually destroying stuff? But if so, I 
don't see how we can avoid breaking their code. And they'll 
probably be much happier with this change just so long as they're 
aware of it, and hopefull the changelog is enough for that.

So, I'd say to put it in 2.067. I don't think that waiting buys 
us anything.

- Jonathan M Davis
Aug 28 2014
prev sibling next sibling parent Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 8/28/2014 7:21 PM, Andrei Alexandrescu via Digitalmars-d wrote:

 Dear community, are you ready for this?

 https://issues.dlang.org/show_bug.cgi?id=2834
 https://github.com/D-Programming-Language/druntime/pull/864

 We must do it, and the way I see it the earlier the better. Shall we do
 it in 2.067?

 This is a significant change of behavior. Should we provide a temporary
 flag or attribute to disable it?


 Thanks,

 Andrei


I didn't study the changes, except to note that the number of tests 
seems rather considering the nature of what's changing.
Aug 28 2014
prev sibling next sibling parent Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 8/28/2014 7:54 PM, Brad Roberts via Digitalmars-d wrote:

 I didn't study the changes, except to note that the number of tests
 seems rather considering the nature of what's changing.


Er: rather LOW considering...
Aug 28 2014
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu 
wrote:

 Dear community, are you ready for this?

 https://issues.dlang.org/show_bug.cgi?id=2834
 https://github.com/D-Programming-Language/druntime/pull/864

 We must do it, and the way I see it the earlier the better. 
 Shall we do it in 2.067?

 This is a significant change of behavior. Should we provide a 
 temporary flag or attribute to disable it?


 Thanks,

 Andrei


I think providing rollback transition flag is simply a matter of 
politeness even if in this specific case it makes no sense to 
rely on existing broken behavior.
Aug 28 2014
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 29 August 2014 at 02:59:07 UTC, Dicebot wrote:

 On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu 
 wrote:

 Dear community, are you ready for this?

 https://issues.dlang.org/show_bug.cgi?id=2834
 https://github.com/D-Programming-Language/druntime/pull/864

 We must do it, and the way I see it the earlier the better. 
 Shall we do it in 2.067?

 This is a significant change of behavior. Should we provide a 
 temporary flag or attribute to disable it?


 Thanks,

 Andrei


 I think providing rollback transition flag is simply a matter 
 of politeness even if in this specific case it makes no sense 
 to rely on existing broken behavior.


Since this is in the library, not the compiler, I'm not sure how 
you'd do that (have the compiler specific a version identifier 
for it?), but considering how broken the behavior was before and 
that we don't normally provide flags to revert changes in 
behavior, I'd argue that anyone who has problems with it should 
just use the previous version of the compiler until they're ready 
to update their code. It's not like we want to support the old 
behavior for any real period of time anyway. It's just plain 
broken.

- Jonathan M Davis
Aug 28 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/28/14, 8:07 PM, Jonathan M Davis wrote:

 Since this is in the library, not the compiler, I'm not sure how you'd
 do that (have the compiler specific a version identifier for it?), but
 considering how broken the behavior was before and that we don't
 normally provide flags to revert changes in behavior, I'd argue that
 anyone who has problems with it should just use the previous version of
 the compiler until they're ready to update their code.


I'd say the impact is too large to leave things at that. How about a 
function call callStructDestructorsDuringGC(bool) - yes, that long - 
that people can place in main() if they're having trouble?

Andrei
Aug 28 2014
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 29 August 2014 at 03:32:45 UTC, Andrei Alexandrescu 
wrote:

 On 8/28/14, 8:07 PM, Jonathan M Davis wrote:

 Since this is in the library, not the compiler, I'm not sure 
 how you'd
 do that (have the compiler specific a version identifier for 
 it?), but
 considering how broken the behavior was before and that we 
 don't
 normally provide flags to revert changes in behavior, I'd 
 argue that
 anyone who has problems with it should just use the previous 
 version of
 the compiler until they're ready to update their code.


 I'd say the impact is too large to leave things at that. How 
 about a function call callStructDestructorsDuringGC(bool) - 
 yes, that long - that people can place in main() if they're 
 having trouble?


I'm okay with that, though given that this is almost exclusively 
fixing bugs and isn't particularly likely to create many, I'm not 
at all convinced that it's that big a deal.

- Jonathan M Davis
Aug 28 2014
prev sibling parent "Dicebot" <public dicebot.lv> writes:
On Friday, 29 August 2014 at 03:07:06 UTC, Jonathan M Davis wrote:

 and that we don't normally provide flags to revert changes in 
 behavior


Exactly the culture I am trying to change here.

Either companion flag in compiler that prints places where 
structs are going to be "destructored" now or runtime switch for 
druntime that disable new behavior if set. Pretty much anything 
that can show users that we care.
Aug 28 2014
prev sibling next sibling parent "Brian Schott" <briancschott gmail.com> writes:
On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu 
wrote:

 Dear community, are you ready for this?


Yes.
Aug 28 2014
prev sibling next sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu 
wrote:

 Dear community, are you ready for this?

 https://issues.dlang.org/show_bug.cgi?id=2834
 https://github.com/D-Programming-Language/druntime/pull/864

 We must do it, and the way I see it the earlier the better. 
 Shall we do it in 2.067?

 This is a significant change of behavior. Should we provide a 
 temporary flag or attribute to disable it?


 Thanks,

 Andrei


Questions:
- Can and will this work for arrays of structs?
- When doing manual GC allocations (for whatever reason), how can 
we later tell the GC what destructor to call?

These questions combined are really aimed at Appender: I'm 
curious at if and how any changes will have to be made to it.

Also question: Will this play nice wit exiting code that manually 
destroys GC allocated structs?
Aug 28 2014
parent reply Orvid King <blah38621 gmail.com> writes:
On 8/29/2014 12:41 AM, monarch_dodra wrote:

 Questions:
 - Can and will this work for arrays of structs?
 - When doing manual GC allocations (for whatever reason), how can we
 later tell the GC what destructor to call?


Yes, this does work for arrays of structs. Provided that you've passed 
in the type info for the struct when doing the manual allocation, it 
should call the destructor without anything extra needing to be done on 
the user's part.


 These questions combined are really aimed at Appender: I'm curious at if
 and how any changes will have to be made to it.


Appender already uses the type info's destroy, so it shouldn't have any 
issues, as I already had to account for that.


 Also question: Will this play nice wit exiting code that manually
 destroys GC allocated structs?


As I mentioned elsewhere, as long as the existing code is calling 
destroy, and not calling the finalizer directly, then yes, it will play 
nice, and only call the finalizer once.
Aug 29 2014
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 29 August 2014 at 18:12:04 UTC, Orvid King wrote:

 On 8/29/2014 12:41 AM, monarch_dodra wrote:

 Questions:
 - Can and will this work for arrays of structs?
 - When doing manual GC allocations (for whatever reason), how 
 can we
 later tell the GC what destructor to call?


 Yes, this does work for arrays of structs. Provided that you've 
 passed in the type info for the struct when doing the manual 
 allocation, it should call the destructor without anything 
 extra needing to be done on the user's part.


Hum... by "manual" memory allocation. I meant this:

GC.qalloc(newlen * T.sizeof, blockAttribute!T);

That's what Appender does. Unless I'm mistaken, the Type info is 
not passed here? Furthermore, the Type info *can't* be passed...?


 These questions combined are really aimed at Appender: I'm


 curious at if

 and how any changes will have to be made to it.


 Appender already uses the type info's destroy, so it shouldn't 
 have any issues, as I already had to account for that.


This is news to me. Appender does not destroy anything. It makes 
no call to delete/destroy/release or whatnot. It simply just 
keeps allocating away.


 Also question: Will this play nice wit exiting code that


 manually

 destroys GC allocated structs?


 As I mentioned elsewhere, as long as the existing code is 
 calling destroy, and not calling the finalizer directly, then 
 yes, it will play nice, and only call the finalizer once.


OK. Nice. Thanks.

BTW: If Appender is "broken", that's OK (in the sense that it 
won't be any more "broken" than before). I just want as much 
information as possible about what I (we) will need to do to 
update it.

In particular, Appender has an optimization that skips postblit 
to "relocate" when possible. If destructions start happening, 
then we'll need to make sure we first reset to "T.init", or we'll 
risk destroying a non-postblitted copy (which could be 
catastrophic in the case of RC'ed structs).
Aug 29 2014
prev sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 29 August 2014 at 18:12:04 UTC, Orvid King wrote:

 On 8/29/2014 12:41 AM, monarch_dodra wrote:

 Questions:
 - Can and will this work for arrays of structs?
 - When doing manual GC allocations (for whatever reason), how 
 can we
 later tell the GC what destructor to call?


 Yes, this does work for arrays of structs. Provided that you've 
 passed in the type info for the struct when doing the manual 
 allocation, it should call the destructor without anything 
 extra needing to be done on the user's part.


Hum... by "manual" memory allocation. I meant this:

GC.qalloc(newlen * T.sizeof, blockAttribute!T);

That's what Appender does. Unless I'm mistaken, the Type info is 
not passed here? Furthermore, the Type info *can't* be passed...?


 These questions combined are really aimed at Appender: I'm


 curious at if

 and how any changes will have to be made to it.


 Appender already uses the type info's destroy, so it shouldn't 
 have any issues, as I already had to account for that.


This is news to me. Appender does not destroy anything. It makes 
no call to delete/destroy/release or whatnot. It simply just 
keeps allocating away.


 Also question: Will this play nice wit exiting code that


 manually

 destroys GC allocated structs?


 As I mentioned elsewhere, as long as the existing code is 
 calling destroy, and not calling the finalizer directly, then 
 yes, it will play nice, and only call the finalizer once.


OK. Nice. Thanks.

BTW: If Appender is "broken", that's OK (in the sense that it 
won't be any more "broken" than before). I just want as much 
information as possible about what I (we) will need to do to 
update it.

In particular, Appender has an optimization that skips postblit 
to "relocate" when possible. If destructions start happening, 
then we'll need to make sure we first reset to "T.init", or we'll 
risk destroying a non-postblitted copy (which could be 
catastrophic in the case of RC'ed structs).
Aug 29 2014
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 29/08/14 04:21, Andrei Alexandrescu wrote:

 Dear community, are you ready for this?

 https://issues.dlang.org/show_bug.cgi?id=2834
 https://github.com/D-Programming-Language/druntime/pull/864


Does this has the same issues as destructors for classes, i.e. not 
guaranteed to run?

-- 
/Jacob Carlborg
Aug 28 2014
prev sibling next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thu, 28 Aug 2014 19:21:04 -0700
Andrei Alexandrescu via Digitalmars-d <digitalmars-d puremagic.com>
wrote:


 Dear community, are you ready for this?


yes, yes and yes!
Aug 28 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/28/14, 11:13 PM, ketmar via Digitalmars-d wrote:

 On Thu, 28 Aug 2014 19:21:04 -0700
 Andrei Alexandrescu via Digitalmars-d <digitalmars-d puremagic.com>
 wrote:


 Dear community, are you ready for this?


 yes, yes and yes!


You forgot the Oxford comma :o). -- Andrei
Aug 29 2014
parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Fri, 29 Aug 2014 09:49:07 -0700
Andrei Alexandrescu via Digitalmars-d <digitalmars-d puremagic.com>
wrote:


 yes, yes and yes!


 You forgot the Oxford comma :o). -- Andrei


yes, yes, and yes!

fixed, ready to another round of reviews. ;-)
Aug 29 2014
prev sibling next sibling parent reply "ponce" <contact gam3sfrommars.fr> writes:
On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu 
wrote:

 Dear community, are you ready for this?


Yes! Whatever needs be done.
Aug 29 2014
parent reply Andrej Mitrovic via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 8/29/14, ponce via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu
 wrote:

 Dear community, are you ready for this?


 Yes! Whatever needs be done.


Yeah destructors are a sore pain when they're unreliable. "May or may
not be called" is just an awful semantic.
Aug 29 2014
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 29 August 2014 at 09:08:07 UTC, Andrej Mitrovic via 
Digitalmars-d wrote:

 On 8/29/14, ponce via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu
 wrote:

 Dear community, are you ready for this?


 Yes! Whatever needs be done.


 Yeah destructors are a sore pain when they're unreliable. "May 
 or may
 not be called" is just an awful semantic.


That won't really change though, will it? AFAIK, it'll become: 
"will eventually be called at some unspecified point it time. The 
program may terminate before that happens, at which point, the 
destructor will never be called."

...right?
Aug 29 2014
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Friday, 29 August 2014 at 09:16:36 UTC, monarch_dodra wrote:

 On Friday, 29 August 2014 at 09:08:07 UTC, Andrej Mitrovic via 
 Digitalmars-d wrote:

 On 8/29/14, ponce via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu
 wrote:

 Dear community, are you ready for this?


 Yes! Whatever needs be done.


 Yeah destructors are a sore pain when they're unreliable. "May 
 or may
 not be called" is just an awful semantic.


 That won't really change though, will it? AFAIK, it'll become: 
 "will eventually be called at some unspecified point it time. 
 The program may terminate before that happens, at which point, 
 the destructor will never be called."

 ...right?


Yes, there's no way around this with a conservative GC.
Aug 29 2014
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/29/14, 2:16 AM, monarch_dodra wrote:

 On Friday, 29 August 2014 at 09:08:07 UTC, Andrej Mitrovic via
 Digitalmars-d wrote:

 On 8/29/14, ponce via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu
 wrote:

 Dear community, are you ready for this?


 Yes! Whatever needs be done.


 Yeah destructors are a sore pain when they're unreliable. "May or may
 not be called" is just an awful semantic.


 That won't really change though, will it? AFAIK, it'll become: "will
 eventually be called at some unspecified point it time. The program may
 terminate before that happens, at which point, the destructor will never
 be called."

 ....right?


Right, and reasonable. -- Andrei
Aug 29 2014
prev sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu 
wrote:

 Dear community, are you ready for this?


Yes.


 This is a significant change of behavior. Should we provide a 
 temporary flag or attribute to disable it?


I don't think so, it will just hinder adoption. If people don't 
want it they can stay with 2.066.
Aug 29 2014
parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Friday, 29 August 2014 at 14:24:34 UTC, Peter Alexander wrote:

 On Friday, 29 August 2014 at 02:21:07 UTC, Andrei Alexandrescu 
 wrote:

 Dear community, are you ready for this?


 Yes.


 This is a significant change of behavior. Should we provide a 
 temporary flag or attribute to disable it?


 I don't think so, it will just hinder adoption. If people don't 
 want it they can stay with 2.066.


Especially seeing as now we are doing patch releases.
Aug 29 2014