www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Confusion regarding struct lifecycle

reply Matt Elkins <notreal fake.com> writes:
I've been bitten again by my lack of understanding of the D 
struct lifecycle :-/. I managed to reduce my buggy program to the 
following example:

[code]
import std.stdio;

struct Foo
{
      disable this();
      disable this(this);

     this(int valueIn) {value = valueIn;}
     ~this() {writeln("Foo being destroyed: ", value);}

     int value;
}

struct FooList
{
      disable this();
      disable this(this);

     this(int)
     {
         writeln("Before 8");
         foos[0] = Foo(8);
         writeln("Before 1");
         foos[1] = Foo(1);
         writeln("Before 2");
         foos[2] = Foo(2);
         writeln("Before 3");
         foos[3] = Foo(3);
         writeln("After Foo construction");
     }

     Foo[5] foos;
}

unittest
{
     auto fooList = FooList(0);
     writeln("About to lose scope");
}
[/code]

[output]
Before 8
Before 1
Foo being destroyed: 0
Before 2
Foo being destroyed: 0
Before 3
Foo being destroyed: 0
After Foo construction
About to lose scope
Foo being destroyed: 0
Foo being destroyed: 3
Foo being destroyed: 2
Foo being destroyed: 1
Foo being destroyed: 8
[/output]

There are a few things which confuse me about this:
* Why does this code compile? In particular, I would have 
expected that with Foo[5] but initialization for only Foos 0 .. 3 
and with the  disabled constructors in Foo, there would be a 
compiler error.
* Where do those first three destroyed Foos come from? I thought 
there should have been no Foos existing since default 
construction is  disabled...
* Even if somehow the Foos are being created despite the 
 disabled default constructor, why are only three Foos being 
destroyed before the scope is lost?

So I guess what I'm wondering is:
* If I  disable a default constructor on a struct, does the 
language guarantee that I won't have default-constructed 
instances of that struct? If not, what is the point of  disable 
for default constructors? If so, is the above situation a 
compiler bug or something I am missing?
* Is the below the right general syntax for creating an instance 
of a struct so as to avoid creating more than one copy? If not, 
what is?

Stack variable: auto foo = Foo(5);
Member variable: Foo m_foo; this(/* args */) {m_foo = Foo(5);}

Thanks!
Feb 15 2016
next sibling parent reply maik klein <maikklein googlemail.com> writes:
On Tuesday, 16 February 2016 at 02:09:15 UTC, Matt Elkins wrote:

 I've been bitten again by my lack of understanding of the D 
 struct lifecycle :-/. I managed to reduce my buggy program to 
 the following example:

 [...]


In D you can always call Foo.init even with  disable this(), The 
first 3 destructor calls are from the 3 Foo.inits in your static 
array. I guess because you disabled the copy constructor, the 
Foo's will be moved and then they also need to call the 
destructor of the Foo.inits. Just like std::move does.
Feb 15 2016
parent reply Matt Elkins <notreal fake.com> writes:
On Tuesday, 16 February 2016 at 03:31:51 UTC, maik klein wrote:

 In D you can always call Foo.init even with  disable this(),


Foo.init can be called implicitly (not just explicitly)? If so, 
why even have  disable this(), if it offers no guarantees?


 The first 3 destructor calls are from the 3 Foo.inits in your 
 static array.


But why only 3? There are 5 Foos in the array, and 4 were 
explicitly overwritten...
Feb 15 2016
parent reply Mike Parker <aldacron gmail.com> writes:
On Tuesday, 16 February 2016 at 03:39:00 UTC, Matt Elkins wrote:

 On Tuesday, 16 February 2016 at 03:31:51 UTC, maik klein wrote:

 In D you can always call Foo.init even with  disable this(),


 Foo.init can be called implicitly (not just explicitly)? If so, 
 why even have  disable this(), if it offers no guarantees?


IMO, this is a bug. It should have to be explicit, just as it is 
with a single struct instance.
Feb 15 2016
parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Tuesday, 16 February 2016 at 04:00:27 UTC, Mike Parker wrote:

 On Tuesday, 16 February 2016 at 03:39:00 UTC, Matt Elkins wrote:

 On Tuesday, 16 February 2016 at 03:31:51 UTC, maik klein wrote:

 In D you can always call Foo.init even with  disable this(),


 Foo.init can be called implicitly (not just explicitly)? If 
 so, why even have  disable this(), if it offers no guarantees?


 IMO, this is a bug. It should have to be explicit, just as it 
 is with a single struct instance.


There is likely some bug here. In the example, though, the 
elements _are_ constructed explicitly (except foos[4]). This is 
legitimate, as the first assignment of an element in a construct 
counts as construction.
Feb 16 2016
parent reply Matt Elkins <notreal fake.com> writes:
On Tuesday, 16 February 2016 at 10:45:09 UTC, Marc Schütz wrote:

 On Tuesday, 16 February 2016 at 04:00:27 UTC, Mike Parker wrote:

 On Tuesday, 16 February 2016 at 03:39:00 UTC, Matt Elkins 
 wrote:

 On Tuesday, 16 February 2016 at 03:31:51 UTC, maik klein 
 wrote:

 In D you can always call Foo.init even with  disable this(),


 Foo.init can be called implicitly (not just explicitly)? If 
 so, why even have  disable this(), if it offers no guarantees?


 IMO, this is a bug. It should have to be explicit, just as it 
 is with a single struct instance.


 There is likely some bug here. In the example, though, the 
 elements _are_ constructed explicitly (except foos[4]). This is 
 legitimate, as the first assignment of an element in a 
 construct counts as construction.


The elements are not constructed explicitly with Foo.init; they 
are constructed explicitly with a user-defined Foo constructor. 
Since default construction leads to a semantically-invalid object 
in the non-reduced case, I was expecting that a  disabled default 
constructor would cause the compiler to complain on attempts to 
default-construct the struct. Preferably this would be on any 
attempt to do so, including explicit calls to Foo.init, but at a 
minimum I would want it to complain on attempts to do so 
implicitly. Otherwise there don't appear to be any useful 
guarantees offered by  disable this().
Feb 16 2016
parent Matt Elkins <notreal fake.com> writes:
After some more time spent on (the non-reduced version of) this, 
I think there is a decent chance I am really just experiencing 
another manifestation of a bug I reported a little bit ago: 
https://issues.dlang.org/show_bug.cgi?id=15661

The good news is that this is now marked as resolved, so 
hopefully the next build will have the patch and maybe my 
problems will go away :).
Feb 16 2016
prev sibling next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 02/15/2016 06:09 PM, Matt Elkins wrote:


 * Where do those first three destroyed Foos come from?


I've printed some more information to come up with the following guess. 
I am not sure whether it is correct or intended. Just a guess...

When a temporary Foo object is moved into the array, the temporary 
object is set to Foo.init. This temporary object lives on the stack. In 
fact, all temporary Foo objects of Foo.this(int) live at the same location.

After Foo(8) is moved into the array and set to Foo.init, now Foo(1) is 
constructed on top of it. For that to happen, first the destructor is 
executed for the first life of the temporary, and so on...

There is one less Foo.init destruction because conceptually the initial 
temporary was not constructed on top of an existing Foo.init but raw memory.

The reason it makes sense to me is the fact that all Foo.init 
destructions happen on memory that is outside of the array:

import std.stdio;

struct Foo
{
      disable this();
      disable this(this);

     this(int valueIn) {
         value = valueIn;
         writefln("Foo being constructed: %s at %s", value, &this);
     }
     ~this() {writefln("Foo being destroyed  : %s at %s", value, &this);}

     int value;
}

struct FooList
{
      disable this();
      disable this(this);

     this(int)
     {
         writeln("foos.ptr: ", foos.ptr);
         writeln("Before 8");
         foos[0] = Foo(8);
         writeln("Before 1");
         foos[1] = Foo(1);
         writeln("Before 2");
         foos[2] = Foo(2);
         writeln("Before 3");
         foos[3] = Foo(3);
         writeln("Before 4");
         foos[4] = Foo(4);            // <-- Added by Ali
         writeln("After Foo construction");
     }

     Foo[10] foos;
}

unittest
{
     auto fooList = FooList(0);
     writeln("About to lose scope");
}

void main() {
}

Output:

foos.ptr: 7FFF15C01740
Before 8
Foo being constructed: 8 at 7FFF15C01740
Before 1
Foo being constructed: 1 at 7FFF15C016E0
Foo being destroyed  : 0 at 7FFF15C016A8  <-- temporary Foo.init
Before 2
Foo being constructed: 2 at 7FFF15C016E4
Foo being destroyed  : 0 at 7FFF15C016A8  <-- ditto
Before 3
Foo being constructed: 3 at 7FFF15C016E8
Foo being destroyed  : 0 at 7FFF15C016A8  <-- ditto
Before 4
Foo being constructed: 4 at 7FFF15C016EC  <-- Added by Ali
Foo being destroyed  : 0 at 7FFF15C016A8  <-- ditto
After Foo construction
About to lose scope
Foo being destroyed  : 0 at 7FFF15C01764
Foo being destroyed  : 0 at 7FFF15C01760
Foo being destroyed  : 0 at 7FFF15C0175C
Foo being destroyed  : 0 at 7FFF15C01758
Foo being destroyed  : 0 at 7FFF15C01754
Foo being destroyed  : 4 at 7FFF15C01750
Foo being destroyed  : 3 at 7FFF15C0174C
Foo being destroyed  : 2 at 7FFF15C01748
Foo being destroyed  : 1 at 7FFF15C01744
Foo being destroyed  : 8 at 7FFF15C01740

Ali
Feb 16 2016
parent reply Matt Elkins <notreal fake.com> writes:
On Tuesday, 16 February 2016 at 08:18:51 UTC, Ali Çehreli wrote:

 When a temporary Foo object is moved into the array, the 
 temporary object is set to Foo.init. This temporary object 
 lives on the stack. In fact, all temporary Foo objects of 
 Foo.this(int) live at the same location.

 After Foo(8) is moved into the array and set to Foo.init, now 
 Foo(1) is constructed on top of it. For that to happen, first 
 the destructor is executed for the first life of the temporary, 
 and so on...

 There is one less Foo.init destruction because conceptually the 
 initial temporary was not constructed on top of an existing 
 Foo.init but raw memory.


I guess that makes sense. But doesn't it imply that a copy is 
happening, despite the  disabled post-blit? The desired behavior 
was to construct the Foos in place, like with a C++ initializer 
list.
Feb 16 2016
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 02/16/2016 05:50 PM, Matt Elkins wrote:

 On Tuesday, 16 February 2016 at 08:18:51 UTC, Ali Çehreli wrote:

 When a temporary Foo object is moved into the array, the temporary
 object is set to Foo.init. This temporary object lives on the stack.
 In fact, all temporary Foo objects of Foo.this(int) live at the same
 location.

 After Foo(8) is moved into the array and set to Foo.init, now Foo(1)
 is constructed on top of it. For that to happen, first the destructor
 is executed for the first life of the temporary, and so on...

 There is one less Foo.init destruction because conceptually the
 initial temporary was not constructed on top of an existing Foo.init
 but raw memory.


 I guess that makes sense. But doesn't it imply that a copy is happening,
 despite the  disabled post-blit? The desired behavior was to construct
 the Foos in place, like with a C++ initializer list.


I changed my mind! :) This seems to be how D is being clever in 
constructors. I've just learned something (thank you!), which may very 
well be a bug rather than a feature.

The compiler analyzes the body of the constructor to differentiate 
between first use versus first initialization. For example, unlike C++, 
you can call super() anywhere in the constructor.

So, although this(this) is disabled, moving objects into a static-array 
member in a constructor must be allowed because otherwise we could not 
initialize such members. Since a static array must consist of .init 
values to begin with, every move into its members must also trigger its 
destructor if the type has elaborate destructor.

In this example, the compiler is being smart and skips one of the 
"move+destructor" operations. I felt this behavior in my other post 
where I had alluded to "raw memory".

This is what I've discovered: Try printing the static array at the 
beginning of the constructor:

     this(int)
     {
         writefln("Initial foos:");
         foreach (ref f; foos[]) {
             writeln(" ", f.value);
         }
         writeln("starting to assign");

         // ...
     }

Error:  field 'foos' initialization is not allowed in loops or after labels

So, this tells us (in a cryptic way) that the syntax foos[] at that 
point in the constructor is understood as "initialization" not as 
slicing. To contrast, now move that code below the Foo(8) assignment:


struct Foo
{
     // ...
     int value = 7;    // <-- Do this as well
}

     this(int)
     {
         writeln("Before 8");
         foos[0] = Foo(8);         // <-- FIRST ASSIGNMENT

         writefln("Initial foos:");
         foreach (ref f; foos[]) {
             writeln(" ", f.value);
         }
         writeln("starting to assign");

         // ...
     }

Now the code compiles and prints the contents of the array *without* any 
Foo destructor. (I think this is because Foo(8) is emplaced, rather than 
assigned.)

Before 8
Initial foos:
  8
  7
  7
  7
  7
starting to assign
Before 1
[...]

Only after that point we have an array of valid Foos where further 
assignments trigger destructors.

Ali
Feb 16 2016
parent reply Matt Elkins <notreal fake.com> writes:
On Wednesday, 17 February 2016 at 02:23:52 UTC, Ali Çehreli wrote:

 Since a static array must consist of .init values to begin 
 with, every move into its members must also trigger its 
 destructor if the type has elaborate destructor.


Oof. This strikes me as a "gotcha", that this happens even with 
 disable this() as opposed to a compiler error. Is this only for 
static arrays, or are there other places  disable this() is 
silently ignored?


 This is what I've discovered:


Ok, I think that pretty much explains the behavior I was seeing 
in the reduced case. Thanks -- that's helpful to know!

The downside is that it really indicates that I didn't reduce my 
buggy program properly. I'll hold out for the 
live-object-destructor-call fix to see whether that corrects my 
problem; I can just leak resources until then :).
Feb 16 2016
parent reply ZombineDev <valid_email he.re> writes:
On Wednesday, 17 February 2016 at 02:44:04 UTC, Matt Elkins wrote:

 On Wednesday, 17 February 2016 at 02:23:52 UTC, Ali Çehreli 
 wrote:

 [...]


 Oof. This strikes me as a "gotcha", that this happens even with 
  disable this() as opposed to a compiler error. Is this only 
 for static arrays, or are there other places  disable this() is 
 silently ignored?


 [...]


 Ok, I think that pretty much explains the behavior I was seeing 
 in the reduced case. Thanks -- that's helpful to know!

 The downside is that it really indicates that I didn't reduce 
 my buggy program properly. I'll hold out for the 
 live-object-destructor-call fix to see whether that corrects my 
 problem; I can just leak resources until then :).


BTW, you can try the nighly build which should include the bug 
fix:
http://dlang.org/download.html
Feb 16 2016
parent reply Matt Elkins <notreal fake.com> writes:
On Wednesday, 17 February 2016 at 07:10:15 UTC, ZombineDev wrote:

 The downside is that it really indicates that I didn't reduce 
 my buggy program properly. I'll hold out for the 
 live-object-destructor-call fix to see whether that corrects 
 my problem; I can just leak resources until then :).




So I've reduced it more properly now, and am 98% sure I've just 
got another manifestation of that same bug.


 BTW, you can try the nighly build which should include the bug 
 fix:
 http://dlang.org/download.html


I tried this, and got the same issue. Actually, I was still able 
to reproduce the original reported issue as well, which leads me 
to believe either the bug was not actually fixed or (and this is 
more likely) I screwed something up with my install. Do I need to 
do anything special to install a nightly build (I'm on Windows)? 
My "install" procedure was to move my dmd2 folder, verify that 
compilation now fails since there is no compiler installed, and 
then copy in the nightly build's dmd2 folder over the original 
path. Compilation then succeeds (but the bug still manifests).

That was running through my IDE (IntelliJ with the D language 
plugin), and then in turn through DUB. To remove variables, I 
also tried running dmd directly on the test file: dmd.exe test.d 
-unittest -main. This compiled but also reproduced the bug.

I guess the other possibility is that a patch was made and the 
issue marked resolved, but the patch hasn't been folded into the 
nightly build. I have no idea how dmd's patch/build/CI process 
works, so I don't know whether that is a real possibility.
Feb 17 2016
next sibling parent reply anonymous <anonymous example.com> writes:
On 17.02.2016 17:36, Matt Elkins wrote:

 I tried this, and got the same issue. Actually, I was still able to
 reproduce the original reported issue as well, which leads me to believe
 either the bug was not actually fixed or (and this is more likely) I
 screwed something up with my install. Do I need to do anything special
 to install a nightly build (I'm on Windows)?


For reference, link to the issue:
https://issues.dlang.org/show_bug.cgi?id=15661

As you can see on the bugzilla page, the fix was pushed to the 'stable' 
branch. But nightlies are probably built from 'master'. So the fix isn't 
in the nightlies until 'stable' gets merged into 'master'. At the 
latest, that should happen again with the 2.071 release.

You can build a 'stable' dmd from git. I just did that and I can say 
that the test code for issue 15661 now prints

----
HERE1a
HERE1b
HERE2
----

which seems to be the correct output, indicating that the issue got fixed.

However, the code from this thread still prints

----
Before 8
Before 1
Foo being destroyed: 0
Before 2
Foo being destroyed: 0
Before 3
Foo being destroyed: 0
After Foo construction
About to lose scope
Foo being destroyed: 0
Foo being destroyed: 3
Foo being destroyed: 2
Foo being destroyed: 1 

Foo being destroyed: 8
----

So, different issue?
Feb 17 2016
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 02/17/2016 11:37 AM, anonymous wrote:


 However, the code from this thread still prints

 ----
 Before 8
 Before 1
 Foo being destroyed: 0
 Before 2
 Foo being destroyed: 0
 Before 3
 Foo being destroyed: 0
 After Foo construction
 About to lose scope
 Foo being destroyed: 0
 Foo being destroyed: 3
 Foo being destroyed: 2
 Foo being destroyed: 1
 Foo being destroyed: 8
 ----

 So, different issue?


I thought so and opened the following yesterday:

   https://issues.dlang.org/show_bug.cgi?id=15694

Ali
Feb 17 2016
prev sibling parent ZombineDev <valid_email he.re> writes:
On Wednesday, 17 February 2016 at 16:36:35 UTC, Matt Elkins wrote:

 On Wednesday, 17 February 2016 at 07:10:15 UTC, ZombineDev 
 wrote:

 The downside is that it really indicates that I didn't reduce 
 my buggy program properly. I'll hold out for the 
 live-object-destructor-call fix to see whether that corrects 
 my problem; I can just leak resources until then :).




 So I've reduced it more properly now, and am 98% sure I've just 
 got another manifestation of that same bug.


 BTW, you can try the nighly build which should include the bug 
 fix:
 http://dlang.org/download.html


 I tried this, and got the same issue. Actually, I was still 
 able to reproduce the original reported issue as well, which 
 leads me to believe either the bug was not actually fixed or 
 (and this is more likely) I screwed something up with my 
 install. Do I need to do anything special to install a nightly 
 build (I'm on Windows)? My "install" procedure was to move my 
 dmd2 folder, verify that compilation now fails since there is 
 no compiler installed, and then copy in the nightly build's 
 dmd2 folder over the original path. Compilation then succeeds 
 (but the bug still manifests).

 That was running through my IDE (IntelliJ with the D language 
 plugin), and then in turn through DUB. To remove variables, I 
 also tried running dmd directly on the test file: dmd.exe 
 test.d -unittest -main. This compiled but also reproduced the 
 bug.

 I guess the other possibility is that a patch was made and the 
 issue marked resolved, but the patch hasn't been folded into 
 the nightly build. I have no idea how dmd's patch/build/CI 
 process works, so I don't know whether that is a real 
 possibility.


Sorry for the confusion. The bug was indeed fixed in the stable 
branch, however the stable branch is not yet merged into master, 
from which the nightly builds are packaged.
You should be able to test it with a nightly build probably 
within 1 or 2 days after you get a "Commits pushed to master" 
notification on the bugzilla issue [1].

[1]: https://issues.dlang.org/show_bug.cgi?id=15661
Feb 17 2016
prev sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
dmd's behaviour is too confusing to accept. :) Filed:

   https://issues.dlang.org/show_bug.cgi?id=15694

Ali
Feb 17 2016