www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Blogpost about the T.init problem

reply FeepingCreature <feepingcreature gmail.com> writes:
I've written up a short blogpost about the T.init issue.

It is not very enthusiastic.

https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43

Related links:

https://github.com/dlang/phobos/pull/6594 problem with T.init and 
toString

https://github.com/dlang/phobos/pull/6619 Nullable can't work 
with types where T.init violates invariants

https://github.com/dlang/dmd/pull/8462 A somewhat sketchy PR to 
disable invariant on struct ~this
Jul 10
next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:
 I've written up a short blogpost about the T.init issue.
I believe that whoever wrote that spec meant that the invariant WOULD not need to hold if MyDomainData.init WAS called, but that MyDomainData.init must not be called if this is the case. It definitely needs clarification if I understood it's intent right.
Jul 10
parent Dukc <ajieskola gmail.com> writes:
On Tuesday, 10 July 2018 at 15:10:21 UTC, Dukc wrote:
 It definitely needs clarification if I understood it's intent 
 right.
https://github.com/dlang/dlang.org/pull/2418
Jul 11
prev sibling next sibling parent Meta <jared771 gmail.com> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:
 I've written up a short blogpost about the T.init issue.

 It is not very enthusiastic.

 https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43

 Related links:

 https://github.com/dlang/phobos/pull/6594 problem with T.init 
 and toString

 https://github.com/dlang/phobos/pull/6619 Nullable can't work 
 with types where T.init violates invariants

 https://github.com/dlang/dmd/pull/8462 A somewhat sketchy PR to 
 disable invariant on struct ~this
D's contract programming features have been around since D1 and haven't really been updated to work with newer D2 features (this is no excuse IMO; these issues should have been addressed long ago). My opinion is that it's a vicious circle; D's contract programming features are underutilized, thus nobody cares enough to put effort into ironing out the bugs, thus contract programming in D is buggy and interacts poorly with other language features, thus D's contract programming features are underutilized. IMO, if you don't have the knowledge, desire, or time to fix them, the next best thing to do is write articles like these bringing some attention to the various defects of contract programming in D, and/or write DIPs to propose ways to fix it.
Jul 10
prev sibling next sibling parent reply Cym13 <cpicard openmailbox.org> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:
 I've written up a short blogpost about the T.init issue.

 It is not very enthusiastic.

 https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43

 Related links:

 https://github.com/dlang/phobos/pull/6594 problem with T.init 
 and toString

 https://github.com/dlang/phobos/pull/6619 Nullable can't work 
 with types where T.init violates invariants

 https://github.com/dlang/dmd/pull/8462 A somewhat sketchy PR to 
 disable invariant on struct ~this
First of all I must point that I would very much like to have seen a code actually producing an error in that article. Contrary to what is hinted just taking the struct and putting using it with Nullable or format() caused no error for me and worked as expected. Taking .init explicitely was the only thing that actually caused an error. I'm not saying you didn't experience these issues, but if you want to demonstrate a problem then please demonstrate it. That said, I may be missing something obvious but what prevents you from overloading the init field? struct MyDomainData { string username; disable this(); // don't make a MyDomainData() by accident! this(string username) in(!username.empty) // only non-empty usernames please! do { this.username = username; } // let's formalise the restriction. invariant { assert(!username.empty); } string toString() { ... } static property MyDomainData init() { return MyDomainData("uninitialized"); } ... } auto test = MyDomainData.init; // There, no error Of course that value means nothing but .init isn't meant to actually mean something anyway, it's just a valid value and that's what that init is proposing, so it shouldn't cause any more bugs than empty .init in a normal case.
Jul 10
parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Tuesday, 10 July 2018 at 21:08:32 UTC, Cym13 wrote:
 First of all I must point that I would very much like to have 
 seen a code actually producing an error in that article. 
 Contrary to what is hinted just taking the struct and putting 
 using it with Nullable or format() caused no error for me and 
 worked as expected.
To reproduce the format issue, try to print the struct with writefln!"%s"(MyDomainType()). To reproduce the Nullable issue, you need to slightly modify the struct. In Phobos, Nullable will (due to an abundance of caution) refuse to initialize the struct if the default constructor is disabled; also you need a destructor. However, for this it is enough to use any type that has a destructor, so that an implicit struct destructor is generated. For verbosity, I'll write it out: struct MyDomainData { string username; this(string username) safe in(!username.empty) // only non-empty usernames please! do { this.username = username; } // let's formalise the restriction. invariant { assert(!username.empty); } string toString() { return null; } ~this() safe { } } Then just stick it in a Nullable. No explicit .init needed.
 That said, I may be missing something obvious but what prevents 
 you from overloading the init field?

     struct MyDomainData {
         string username;
          disable this(); // don't make a MyDomainData() by 
 accident!
         this(string username)
         in(!username.empty) // only non-empty usernames please!
         do { this.username = username; }
         // let's formalise the restriction.
         invariant { assert(!username.empty); }
         string toString() { ... }

         static  property MyDomainData init() {
             return MyDomainData("uninitialized");
         }

         ...
     }

     auto test = MyDomainData.init;  // There, no error

 Of course that value means nothing but .init isn't meant to 
 actually mean something anyway, it's just a valid value and 
 that's what that init is proposing, so it shouldn't cause any 
 more bugs than empty .init in a normal case.
That would work, it's just a really horrible hack and I hate it. We're constructing a fictitious domain value that passes our invariants while having zero correspondence to the real world, *just to pass our invariants*. It's an obvious sign of a language issue.
Jul 11
next sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Wednesday, 11 July 2018 at 07:30:59 UTC, FeepingCreature wrote:
 Then just stick it in a Nullable. No explicit .init needed.
To clarify this point some more, since on reflection it's ambiguous: you might well say that "well yeah, the default constructor returns an invalid value, no shit it breaks." The semantics of Nullable are weird here though - Nullable!S constructs an S while pretending to not contain an S. The deeper problem is that there is straight up *no way* to implement Nullable correctly in a way that lets it handle types with disabled this(); without using pointers there's no way to bypass S's destructor, and any implementation of Nullable that uses T.init explicitly dies when D tries to destruct it.
 That would work, it's just a really horrible hack and I hate 
 it. We're constructing a fictitious domain value that passes 
 our invariants while having zero correspondence to the real 
 world, *just to pass our invariants*. It's an obvious sign of a 
 language issue.
Furthermore, note that this limits the amount of invariants we can define. For instance, we can't define an invariant that says that a value has to be registered in a central table somewhere - since our fictional value obviously won't be. Better to do bool isInitialized = false, and that's already crappy.
Jul 11
next sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
Destructors are not called for fields embedded in unions.

On the one hand this is a horrible, horrible hack. On the other, 
whee!
Jul 11
parent reply Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Wednesday, 11 July 2018 at 10:43:40 UTC, FeepingCreature wrote:
 Destructors are not called for fields embedded in unions.

 On the one hand this is a horrible, horrible hack. On the 
 other, whee!
Not only a horrible hack, but a bug, and something you really can't rely on working in the future. Especially since this behavior is only observed with named unions, not with anonymous ones. I'll also mention section 14.19 2 on the Structs, Unions page (https://dlang.org/spec/struct.html): 2. Unions may not have fields that have destructors. Whether that means it's undefined behavior or the compiler should statically disallow it is up for debate, I guess. -- Simen
Jul 12
parent FeepingCreature <feepingcreature gmail.com> writes:
On Thursday, 12 July 2018 at 08:54:17 UTC, Simen Kjærås wrote:
 Whether that means it's undefined behavior or the compiler 
 should statically disallow it is up for debate, I guess.

 --
   Simen
Honestly, half the reason I'm using it so enthusiastically is that I want to emphasize that this is an important usecase that can't be neglected, so that if unions are ever fixed they give us some other way to manually handle struct destruction, like managed or nodestroy.
Jul 12
prev sibling parent reply Meta <jared771 gmail.com> writes:
On Wednesday, 11 July 2018 at 07:35:24 UTC, FeepingCreature wrote:
 On Wednesday, 11 July 2018 at 07:30:59 UTC, FeepingCreature 
 wrote:
 Then just stick it in a Nullable. No explicit .init needed.
To clarify this point some more, since on reflection it's ambiguous: you might well say that "well yeah, the default constructor returns an invalid value, no shit it breaks." The semantics of Nullable are weird here though - Nullable!S constructs an S while pretending to not contain an S. The deeper problem is that there is straight up *no way* to implement Nullable correctly in a way that lets it handle types with disabled this();
I hate to say I told you so, but... https://github.com/dlang/phobos/pull/5855#issuecomment-345783238 Just joking, of course =) Nullable has needed to be completely overhauled for a long time because it was only really designed with POD types in mind.
Jul 11
parent FeepingCreature <feepingcreature gmail.de> writes:
On Wednesday, 11 July 2018 at 20:10:17 UTC, Meta wrote:
 I hate to say I told you so, but... 
 https://github.com/dlang/phobos/pull/5855#issuecomment-345783238

 Just joking, of course =)

 Nullable has needed to be completely overhauled for a long time 
 because it was only really designed with POD types in mind.
Good news is the union hack seems to be working actually ( https://github.com/dlang/phobos/pull/6619 ). See the Turducken post in General ( https://forum.dlang.org/thread/ekbxqxhnttihkoszzvxl forum.dlang.org ); I didn't use the full Turducken in the PR for Phobos because I don't think Nullable is specced to be able to handle reassigning const data anyway, so the union assign in moveEmplace seems sufficient.
Jul 11
prev sibling next sibling parent reply Cym13 <cpicard openmailbox.org> writes:
On Wednesday, 11 July 2018 at 07:30:59 UTC, FeepingCreature wrote:
 That would work, it's just a really horrible hack and I hate it.
Bastiaan's solution to simply change the default value slipped my mind but is really cleaner and in the same line of thought.
 We're constructing a fictitious domain value that passes our 
 invariants while having zero correspondence to the real world, 
 *just to pass our invariants*. It's an obvious sign of a 
 language issue.
I'm not sure I understand, that's what T.init is: a fictitious domain value that just happens to be the default value. It doesn't have to have any meaning and shouldn't be used that way. It's just a value until it has a value. If it happens to be conveniently a useful value, all right, but that's not its first goal IIUC. To present things the other way: you are defining constraints on a type while also defining the default value of that type as not meeting these contraints. No matter how you look at it the default value of a type should be a valid value. How is that not an issue with your own code? Just change the default so that it is within the constraints. Furthermore, while changing the default field value directly is less of a hack the solution to redefine init() entirely actually allows you to do things like making sure the struct is registered in a table somewhere. So I think you do have the option to meet your invariants.
Jul 12
parent FeepingCreature <feepingcreature gmail.com> writes:
On Friday, 13 July 2018 at 01:26:28 UTC, Cym13 wrote:
 I'm not sure I understand, that's what T.init is: a fictitious 
 domain value that just happens to be the default value. It 
 doesn't have to have any meaning and shouldn't be used that 
 way. It's just a value until it has a value. If it happens to 
 be conveniently a useful value, all right, but that's not its 
 first goal IIUC.
T.init is a fictitious but valid instance of the type domain, but it isn't necessarily a valid instance of the *data* domain that the value represents; that's why we can disable this() and invariant in the first place, to impose additional restrictions that are not modeled by the typesystem.
 To present things the other way: you are defining constraints 
 on a type while also defining the default value of that type as 
 not meeting these contraints. No matter how you look at it the 
 default value of a type should be a valid value. How is that 
 not an issue with your own code? Just change the default so 
 that it is within the constraints.
It's impossible to have a fixed value that is a valid value of a type that's semantically embedded in a dynamic data structure. Null will never be a valid class instance.
 Furthermore, while changing the default field value directly is 
 less of a hack the solution to redefine init() entirely 
 actually allows you to do things like making sure the struct is 
 registered in a table somewhere.
At which point, not content to ruin our type with invalid "valid" data, we've ruined the rest of our runtime state as well!
 So I think you do have the option to meet your invariants.
Sure, at the cost of making them worthless.
Jul 13
prev sibling next sibling parent reply Nick Treleaven <nick geany.org> writes:
On Wednesday, 11 July 2018 at 07:30:59 UTC, FeepingCreature wrote:
 To reproduce the format issue, try to print the struct with 
 writefln!"%s"(MyDomainType()).
I implemented the compile time format string checking by evaluating `format(fmtStr, Args.init)` at compile time and seeing if an exception was thrown. So the above problem could be worked around by guarding the invariant test with a check that CTFE is not active.
Jul 23
parent FeepingCreature <feepingcreature gmail.com> writes:
On Monday, 23 July 2018 at 14:26:17 UTC, Nick Treleaven wrote:
 On Wednesday, 11 July 2018 at 07:30:59 UTC, FeepingCreature 
 wrote:
 To reproduce the format issue, try to print the struct with 
 writefln!"%s"(MyDomainType()).
I implemented the compile time format string checking by evaluating `format(fmtStr, Args.init)` at compile time and seeing if an exception was thrown. So the above problem could be worked around by guarding the invariant test with a check that CTFE is not active.
Note that I've gotten a fix for this merged that doesn't break format use in ctfe: the compile time format string check simply writes into a NoOpSink, and the formattedWriter detects the NoOpSink and skips the toString. So that's good now.
Jul 27
prev sibling parent Jim Balter <Jim Balter.name> writes:
On Wednesday, 11 July 2018 at 07:30:59 UTC, FeepingCreature wrote:
 On Tuesday, 10 July 2018 at 21:08:32 UTC, Cym13 wrote:
 First of all I must point that I would very much like to have 
 seen a code actually producing an error in that article. 
 Contrary to what is hinted just taking the struct and putting 
 using it with Nullable or format() caused no error for me and 
 worked as expected.
To reproduce the format issue, try to print the struct with writefln!"%s"(MyDomainType()). To reproduce the Nullable issue, you need to slightly modify the struct. In Phobos, Nullable will (due to an abundance of caution) refuse to initialize the struct if the default constructor is disabled; also you need a destructor. However, for this it is enough to use any type that has a destructor, so that an implicit struct destructor is generated. For verbosity, I'll write it out: struct MyDomainData { string username; this(string username) safe in(!username.empty) // only non-empty usernames please! do { this.username = username; } // let's formalise the restriction. invariant { assert(!username.empty); } string toString() { return null; } ~this() safe { } } Then just stick it in a Nullable. No explicit .init needed.
 That said, I may be missing something obvious but what 
 prevents you from overloading the init field?

     struct MyDomainData {
         string username;
          disable this(); // don't make a MyDomainData() by 
 accident!
         this(string username)
         in(!username.empty) // only non-empty usernames please!
         do { this.username = username; }
         // let's formalise the restriction.
         invariant { assert(!username.empty); }
         string toString() { ... }

         static  property MyDomainData init() {
             return MyDomainData("uninitialized");
         }

         ...
     }

     auto test = MyDomainData.init;  // There, no error

 Of course that value means nothing but .init isn't meant to 
 actually mean something anyway, it's just a valid value and 
 that's what that init is proposing, so it shouldn't cause any 
 more bugs than empty .init in a normal case.
That would work, it's just a really horrible hack and I hate it. We're constructing a fictitious domain value that passes our invariants while having zero correspondence to the real world, *just to pass our invariants*. It's an obvious sign of a language issue.
If so, then the only language solution is to remove either invariants or .init, because as long as .init can be called but cannot be made to conform to your invariant, then your design is beyond the scope of the language and you're in a pickle. But the fact is that it's not a language issue and there are several ways in user code to guarantee that .init satisfies the invariant. In fact that very statement suggests a solution that Timothy Cour suggested earlier: invariant { if(this is typeof(this).init) return; assert(!username.empty); }
Jul 29
prev sibling next sibling parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:
 I've written up a short blogpost about the T.init issue.

 It is not very enthusiastic.

 https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43
Have you tried giving your invariants a valid initial value? Change
   string username;
into
   string username = "noname";
-Bastiaan
Jul 10
prev sibling next sibling parent reply Ali <fakeemail example.com> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:
 I've written up a short blogpost about the T.init issue.

 It is not very enthusiastic.

 https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43
Somehow, this is the type of problem, i thought point 1 in the vision document is aimed to solve https://wiki.dlang.org/Vision/2018H1 "1. Lock down the language definition: D is a powerful language but its definition is not precise enough. A number of subtleties can only be assessed only by running the compiler, not by perusing the specification. This semester we are pushing for a better, more precise, and more complete specification of the D language." ensuring that the language features are coherent together
Jul 10
parent reply =?UTF-8?B?THXDrXM=?= Marques <luis luismarques.eu> writes:
On Wednesday, 11 July 2018 at 03:00:48 UTC, Ali wrote:
 Somehow, this is the type of problem, i thought point 1 in the 
 vision document is aimed to solve
 https://wiki.dlang.org/Vision/2018H1

 "1. Lock down the language definition: D is a powerful language 
 but its definition is not precise enough. A number of 
 subtleties can only be assessed only by running the compiler, 
 not by perusing the specification. This semester we are pushing 
 for a better, more precise, and more complete specification of 
 the D language."

 ensuring that the language features are coherent together
I think that point isn't supposed to be so ambitious. I think it only refers to the fact that the documentation isn't particularly formal/precise/complete regarding some language details. Making the language features more coherent and compatible among themselves is a whole other goal, independent of that one, IMO.
Jul 12
parent Timothee Cour <thelastmammoth gmail.com> writes:
the following seems like a easy enough workaround:
just add

` if(this is typeof(this).init) return;` at 1st line of your invariant:

```d
import std.typecons;
import std.range;

struct MyDomainData {
     string username;

     this(string username)  safe
     in(!username.empty)
     do { this.username = username; }

     invariant {
      if(this is typeof(this).init) return;
      assert(!username.empty);
    }

     string toString() { return null; }

     ~this()  safe { }
}

void main(){
  Nullable!MyDomainData b;
}
```
On Thu, Jul 12, 2018 at 9:00 AM Luís Marques via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:
 On Wednesday, 11 July 2018 at 03:00:48 UTC, Ali wrote:
 Somehow, this is the type of problem, i thought point 1 in the
 vision document is aimed to solve
 https://wiki.dlang.org/Vision/2018H1

 "1. Lock down the language definition: D is a powerful language
 but its definition is not precise enough. A number of
 subtleties can only be assessed only by running the compiler,
 not by perusing the specification. This semester we are pushing
 for a better, more precise, and more complete specification of
 the D language."

 ensuring that the language features are coherent together
I think that point isn't supposed to be so ambitious. I think it only refers to the fact that the documentation isn't particularly formal/precise/complete regarding some language details. Making the language features more coherent and compatible among themselves is a whole other goal, independent of that one, IMO.
Jul 12
prev sibling next sibling parent reply Greatsam4sure <greatsam4sure gmail.com> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:
 I've written up a short blogpost about the T.init issue.

 It is not very enthusiastic.

 https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43

 Related links:

 https://github.com/dlang/phobos/pull/6594 problem with T.init 
 and toString

 https://github.com/dlang/phobos/pull/6619 Nullable can't work 
 with types where T.init violates invariants

 https://github.com/dlang/dmd/pull/8462 A somewhat sketchy PR to 
 disable invariant on struct ~this
Sincerely speaking D language does not merit all these criticism. The magnitude of criticism on D language does not really make sense to me. I am yet to see a language so user friendly as D with such power and strength.I trust one day the world will see and know that D is a language build for programmers for great productivity and not just another money making machine This article justified your great love for D. Thanks for your great love for D
Jul 11
parent Jim Balter <Jim Balter.name> writes:
On Wednesday, 11 July 2018 at 16:54:18 UTC, Greatsam4sure wrote:
 On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:
 [...]
Sincerely speaking D language does not merit all these criticism. The magnitude of criticism on D language does not really make sense to me. I am yet to see a language so user friendly as D with such power and strength.I trust one day the world will see and know that D is a language build for programmers for great productivity and not just another money making machine
Yeah, D is great so let's not even have a bug database, eh?
Jul 29
prev sibling parent reply Greatsam4sure <greatsam4sure gmail.com> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:
 I've written up a short blogpost about the T.init issue.

 It is not very enthusiastic.

 https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43

 Related links:

 https://github.com/dlang/phobos/pull/6594 problem with T.init 
 and toString

 https://github.com/dlang/phobos/pull/6619 Nullable can't work 
 with types where T.init violates invariants

 https://github.com/dlang/dmd/pull/8462 A somewhat sketchy PR to 
 disable invariant on struct ~this
Every language is plague with one bug or the order. For those will great love for the language they lend a helping hand to fixed the bug. I expect you to help also in whatsoever capacity you can. I am just learning D but I am thoroughly satisfy with the language. For me it is truly joy.
Jul 11
parent Jim Balter <Jim Balter.name> writes:
On Wednesday, 11 July 2018 at 16:58:53 UTC, Greatsam4sure wrote:
 On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:
 [...]
Every language is plague with one bug or the order. For those will great love for the language they lend a helping hand to fixed the bug. I expect you to help also in whatsoever capacity you can.
What do you think pull requests are?
 I am just learning D but I am thoroughly satisfy with the 
 language.  For me it is truly joy.
Goody for you, but how does that help?
Jul 29