www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Static inline field initialization

reply Jonas Mminnberg <sasq64 gmail.com> writes:
Because of D's static initialization of members, this assert 
fails:

class Test {
     ubyte[] buf = new ubyte[1000];
}

void main() {
     auto a = new Test();
     auto b = new Test();
     assert(a.buf.ptr != b.buf.ptr);
}

This is bad, since;
* It is not how C++ works
* It introduces silent sharing of data
* It's usually not what you want

Shouldn't this at least generate a warning, or ideally not be 
allowed?
Aug 22
next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 22 August 2017 at 11:50:50 UTC, Jonas Mminnberg wrote:
 Because of D's static initialization of members, this assert 
 fails:

 class Test {
     ubyte[] buf = new ubyte[1000];
 }

 void main() {
     auto a = new Test();
     auto b = new Test();
     assert(a.buf.ptr != b.buf.ptr);
 }

 This is bad, since;
 * It is not how C++ works
 * It introduces silent sharing of data
 * It's usually not what you want

 Shouldn't this at least generate a warning, or ideally not be 
 allowed?
I agree that it can be confusing if you try to read it with C++ semantics [1]; the solution, however, imho is not to change D semantics or throw warnings [2], but to refer to the D spec [3], which states that static initialization of class members is used instead of default initialization (before any constructors are run). To me that means a statically initialized class field shall have the same value in all class instances, i.e. it's not silent sharing, you explicitly requested the sharing. If that doesn't mean the same to others, the D spec wording should be updated to be clearer on the subject. If you don't want instances to share a field value, instead of static initialization you can use constructor initialization [4]: ---- class Test { ubyte[] buf; this() { buf = new ubyte[1000]; } } void main() { auto a = new Test(); auto b = new Test(); assert(a.buf.ptr != b.buf.ptr); } ---- [1] D is not C++ and you shouldn't expect similar looking things to behave the same [2] Compiler warnings are (in my experience) ignored by people, anyway [3] https://dlang.org/spec/class.html#constructors [4] https://dlang.org/spec/class.html#field-init
Aug 22
next sibling parent reply Jonas Mminnberg <sasq64 gmail.com> writes:
On Tuesday, 22 August 2017 at 12:20:45 UTC, Moritz Maxeiner wrote:

 I agree that it can be confusing if you try to read it with C++ 
 semantics [1]; the solution, however, imho is not to change D 
 semantics or throw warnings [2], but to refer to the D spec 
 [3], which states that static initialization of class members 
 is used instead of default initialization (before any 
 constructors are run). To me that means a statically 
 initialized class field shall have the same value in all class 
 instances, i.e. it's not silent sharing, you explicitly 
 requested the sharing. If that doesn't mean the same to others, 
 the D spec wording should be updated to be clearer on the 
 subject.
 If you don't want instances to share a field value, instead of 
 static initialization you can use constructor initialization 
 [4]:

 ----
 class Test
 {
     ubyte[] buf;
     this()
     {
         buf = new ubyte[1000];
     }
  }

 void main()
 {
     auto a = new Test();
     auto b = new Test();
     assert(a.buf.ptr != b.buf.ptr);
 }
I know that it is according to the standard but since D has gone out of it's way to make sure sharing doesn't occur otherwise, by defaulting to TLS storage etc, I feel this breaks the "no surprises" rule. I took me a long time to find this out and when I mentioned it to other casual D programmers they also had no idea this was how it worked.
Aug 22
parent Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 22 August 2017 at 12:38:50 UTC, Jonas Mminnberg wrote:
 On Tuesday, 22 August 2017 at 12:20:45 UTC, Moritz Maxeiner 
 wrote:

 I agree that it can be confusing if you try to read it with 
 C++ semantics [1]; the solution, however, imho is not to 
 change D semantics or throw warnings [2], but to refer to the 
 D spec [3], which states that static initialization of class 
 members is used instead of default initialization (before any 
 constructors are run). To me that means a statically 
 initialized class field shall have the same value in all class 
 instances, i.e. it's not silent sharing, you explicitly 
 requested the sharing. If that doesn't mean the same to 
 others, the D spec wording should be updated to be clearer on 
 the subject.
 If you don't want instances to share a field value, instead of 
 static initialization you can use constructor initialization 
 [4]:

 ----
 class Test
 {
     ubyte[] buf;
     this()
     {
         buf = new ubyte[1000];
     }
  }

 void main()
 {
     auto a = new Test();
     auto b = new Test();
     assert(a.buf.ptr != b.buf.ptr);
 }
I know that it is according to the standard but since D has gone out of it's way to make sure sharing doesn't occur otherwise, by defaulting to TLS storage etc,
While I can understand the sentiment, there is a difference between sharing data between threads and sharing data between class instances in the same thread, the latter of which is occurring here. There is a bug with static field initializers (as others have pointed out), but it's about the fact that the array is stored in global storage and not in TLS and doesn't apply in the example above.
 I feel this breaks the "no surprises" rule.
 I took me a long time to find this out and when I mentioned it 
 to other casual D programmers they also had no idea this was 
 how it worked.
While I don't find how it works surprising personally (it's consistent with how static initialization works everywhere else in D) - in contrast to other subtleties in D - it might make be sensible to include this in the Dlang tour.
Aug 22
prev sibling next sibling parent Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, Aug 22, 2017 at 2:20 PM, Moritz Maxeiner via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 22 August 2017 at 11:50:50 UTC, Jonas Mminnberg wrote:

 ...
I agree that it can be confusing if you try to read it with C++ semantics [1]; the solution, however, imho is not to change D semantics or throw warnings [2], but to refer to the D spec [3], which states that static initialization of class members is used instead of default initialization (before any constructors are run). To me that means a statically initialized class field shall have the same value in all class instances, i.e. it's not silent sharing, you explicitly requested the sharing.
What? D spec does say nothing about sharing, its only speak about order, nothing else. So it is a buf from my POV.
Aug 22
prev sibling parent reply Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
s/buf/bug/

On Tue, Aug 22, 2017 at 3:52 PM, Daniel Kozak <kozzi11 gmail.com> wrote:

 On Tue, Aug 22, 2017 at 2:20 PM, Moritz Maxeiner via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:

 On Tuesday, 22 August 2017 at 11:50:50 UTC, Jonas Mminnberg wrote:

 ...
I agree that it can be confusing if you try to read it with C++ semantics [1]; the solution, however, imho is not to change D semantics or throw warnings [2], but to refer to the D spec [3], which states that static initialization of class members is used instead of default initialization (before any constructors are run). To me that means a statically initialized class field shall have the same value in all class instances, i.e. it's not silent sharing, you explicitly requested the sharing.
What? D spec does say nothing about sharing, its only speak about order, nothing else. So it is a buf from my POV.
Aug 22
parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 22 August 2017 at 13:53:05 UTC, Daniel Kozak wrote:
 s/buf/bug/

 On Tue, Aug 22, 2017 at 3:52 PM, Daniel Kozak 
 <kozzi11 gmail.com> wrote:

 On Tue, Aug 22, 2017 at 2:20 PM, Moritz Maxeiner via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Tuesday, 22 August 2017 at 11:50:50 UTC, Jonas Mminnberg 
 wrote:

 ...
I agree that it can be confusing if you try to read it with C++ semantics [1]; the solution, however, imho is not to change D semantics or throw warnings [2], but to refer to the D spec [3], which states that static initialization of class members is used instead of default initialization (before any constructors are run). To me that means a statically initialized class field shall have the same value in all class instances, i.e. it's not silent sharing, you explicitly requested the sharing.
What? D spec does say nothing about sharing, its only speak about order, nothing else. So it is a buf from my POV.
It doesn't have to and it's not a bug (though considering the replies in this thread the wording should be changed to include that): D's builtin arrays consist of a pointer and a length; the example thus *explicitly* initializes both the pointer and the length to the same (static) value for all instances (as per spec), sharing them. There is a bug [1] - as others have pointed out - that the static array isn't stored in TLS, but in global storage, however, but that doesn't apply in this single threaded case. [1] https://issues.dlang.org/show_bug.cgi?id=2947
Aug 22
parent reply Kagamin <spam here.lot> writes:
On Tuesday, 22 August 2017 at 14:53:21 UTC, Moritz Maxeiner wrote:
 There is a bug [1] - as others have pointed out - that the 
 static array isn't stored in TLS, but in global storage, 
 however, but that doesn't apply in this single threaded case.
The initializer is copied from typeinfo, that can't refer to TLS data.
Aug 22
parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 22 August 2017 at 15:52:48 UTC, Kagamin wrote:
 On Tuesday, 22 August 2017 at 14:53:21 UTC, Moritz Maxeiner 
 wrote:
 There is a bug [1] - as others have pointed out - that the 
 static array isn't stored in TLS, but in global storage, 
 however, but that doesn't apply in this single threaded case.
The initializer is copied from typeinfo, that can't refer to TLS data.
Which means it isn't easily fixable (or even feasible). I'd still consider it a loophole in the type system right now as it allows declaring global data mutably shared between threads without `shared` or `__gshared` (the latter of which couldn't be applied here, though). I'd argue that - as new in this case doesn't allocate on the heap, but in the resulting application's appropriate segments) - it should work like this (from a language semantics standpoint): --- class Test { shared(ubyte)[] buf = new shared(ubyte)[1000]; // classic global storage, instances in all threads refer to the same static array } class Test { ubyte[] buf = new ubyte[1000]; // thread local storage, instances in the same thread refer to the same static array } ---
Aug 22
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Aug 22, 2017 at 04:28:43PM +0000, Moritz Maxeiner via Digitalmars-d
wrote:
[...]
 I'd argue that - as new in this case doesn't allocate on the heap, but
 in the resulting application's appropriate segments) - it should work
 like this (from a language semantics standpoint):
 
 ---
 
 class Test
 {
     shared(ubyte)[] buf = new shared(ubyte)[1000]; // classic global
 storage, instances in all threads refer to the same static array
 }
 
 class Test
 {
     ubyte[] buf = new ubyte[1000]; // thread local storage, instances in the
 same thread refer to the same static array
 }
 ---
Sounds like a good idea. Please file this in bugzilla (if it isn't already) so that it doesn't get lost in the ether. T -- Two wrongs don't make a right; but three rights do make a left...
Aug 22
prev sibling parent reply Kagamin <spam here.lot> writes:
On Tuesday, 22 August 2017 at 16:28:43 UTC, Moritz Maxeiner wrote:
 class Test
 {
     ubyte[] buf = new ubyte[1000]; // thread local storage, 
 instances in the same thread refer to the same static array
 }
Dynamic initialization is done by constructor: class Test { static ubyte[1000] s; ubyte[] buf; this() { buf=s; } } It's also unambiguous as to how it works.
Aug 23
parent Moritz Maxeiner <moritz ucworks.org> writes:
On Wednesday, 23 August 2017 at 09:12:19 UTC, Kagamin wrote:
 On Tuesday, 22 August 2017 at 16:28:43 UTC, Moritz Maxeiner 
 wrote:
 class Test
 {
     ubyte[] buf = new ubyte[1000]; // thread local storage, 
 instances in the same thread refer to the same static array
 }
Dynamic initialization is done by constructor: [...] It's also unambiguous as to how it works.
I am aware, as I have pointed out the same in the above, but the post you quote is explicitly not about dynamic initialization (i.e. the result of new points into the heap), but about static initialization (the result of new points into sections of the binary format). Specifically, it's about static initialization done for variables put into classic global storage (e.g. ELF sections .data/.bss) and variables put into thread local storage (e.g. ELF sections .tdata/.tbss).
Aug 23
prev sibling next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 8/22/17 7:50 AM, Jonas Mminnberg wrote:
 Because of D's static initialization of members, this assert fails:
 
 class Test {
      ubyte[] buf = new ubyte[1000];
 }
 
 void main() {
      auto a = new Test();
      auto b = new Test();
      assert(a.buf.ptr != b.buf.ptr);
 }
 
 This is bad, since;
 * It is not how C++ works
 * It introduces silent sharing of data
 * It's usually not what you want
 
 Shouldn't this at least generate a warning, or ideally not be allowed?
 
https://issues.dlang.org/show_bug.cgi?id=2947 -Steve
Aug 22
prev sibling parent Kagamin <spam here.lot> writes:
https://issues.dlang.org/show_bug.cgi?id=2947
Aug 22