www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)

reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 5/17/13, Walter Bright <walter digitalmars.com> wrote:
 I oppose this. D has a lot of nice features because of the .init property.
 Default constructors wreck that.

Would they? I'm thinking the process would be: struct S { int x; int y = void; this() // hypothetical { // x would already be initialized to int.init here assert(x == int.init); // y is left uninitialized here } } Maybe that's already clear. But why is .init actually such a big problem? If it becomes arbitrarily expensive to call .init of a struct, well it's because it has to be - if the user really provided an expensive default ctor. But it's entirely the user's responsibility. So then .init can even throw, but throwing exceptions isn't a big deal. Is there some other problem? A custom default ctor in a struct is one of the most asked for features. Just yesterday we spent several hours explaining to a C++ user why a default ctor doesn't work, and what .init is for. The whole conversation could have been avoided if D had support for custom default ctors for structs. This topic comes up very often in IRC and the forums.
May 17 2013
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On 5/17/13, Walter Bright <walter digitalmars.com> wrote:
 I oppose this. D has a lot of nice features because of the 
 .init property.
 Default constructors wreck that.

So much great feature like : - null all over the place. - Having NUllable and NonNullable in phobos. BTW, NonNullable is unable to ensure that it if effectively non nullable. - Having to check for .init state all over the place (which have a runtime cost, in addition to be error prone). RefCounted implementation is emblematic of that.
May 18 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 12:08 AM, deadalnix wrote:
 On 5/17/13, Walter Bright <walter digitalmars.com> wrote:
 I oppose this. D has a lot of nice features because of the .init property.
 Default constructors wreck that.

So much great feature like : - null all over the place. - Having NUllable and NonNullable in phobos. BTW, NonNullable is unable to ensure that it if effectively non nullable. - Having to check for .init state all over the place (which have a runtime cost, in addition to be error prone). RefCounted implementation is emblematic of that.

What default would you use for non-null pointers?
May 18 2013
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/18/13 7:13 AM, Simen Kjaeraas wrote:
 On Sat, 18 May 2013 09:14:30 +0200, Walter Bright
 <newshound2 digitalmars.com> wrote:

 On 5/18/2013 12:08 AM, deadalnix wrote:
 On 5/17/13, Walter Bright <walter digitalmars.com> wrote:
 I oppose this. D has a lot of nice features because of the .init
 property.
 Default constructors wreck that.

So much great feature like : - null all over the place. - Having NUllable and NonNullable in phobos. BTW, NonNullable is unable to ensure that it if effectively non nullable. - Having to check for .init state all over the place (which have a runtime cost, in addition to be error prone). RefCounted implementation is emblematic of that.

What default would you use for non-null pointers?

Damnit, I thought we'd gotten through to you. non-null pointers have no default, and it is a compile-time error not to initialize them.

This is what I call the "low potential energy mental block". Very hard to get out of. Andrei
May 18 2013
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/18/13 10:30 AM, deadalnix wrote:
 On Saturday, 18 May 2013 at 14:22:44 UTC, Andrei Alexandrescu wrote:
 This is what I call the "low potential energy mental block". Very hard
 to get out of.

Can you explain this please ?

Happens to me a lot, and am fighting it for dear life. Occurs when a person: 1. Doesn't understand something 2. Doesn't understand (s)he doesn't understand that something 3. Consequently forms a belief there is no lack of understanding 4. Thus has little motivation to change opinion in the matter and see things differently, since there's no need for that (i.e. enters a low potential energy that's difficult to get out of) 5. Whenever discussing the matter, amasses further misunderstood pieces of evidence due to confirmation bias, therefore furthering the depth of the potential hole Such phenomena are widespread and used most visibly in sitcoms in the following classic pattern: two people discuss a matter in which they make different assumptions about a context (e.g. one thinks it's about a person, the other thinks it's about a bottle). During the discussion, both use pieces and tidbits of ambiguous language from the other to further consolidate their belief the topic is indeed what they thought it would be. The watcher, knowing the context of both characters, derives enjoyment figuring how the same conversation may be viewed in completely different ways. Consider the following classics that Walter, myself, or sometimes both have been going through: - virtual machines are a crock - non-null pointers are meh - generic is better than OOP - can't/shouldn't check for arithmetic overflow Some past ones that we fixed: - string lambdas are just fine - can't have many integral types without unsafe casts - can't fix a < b < c to stay C-compatible and meaningful Andrei
May 18 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 12:30 AM, deadalnix wrote:
 On Saturday, 18 May 2013 at 07:14:29 UTC, Walter Bright wrote:
 What default would you use for non-null pointers?

Compile time error as a default sound like a nice option. Probably too late now, but that is where most language are going now, and having done quite a lot of java myself, I can guarantee you that it make sense.

D has that: disable this(); which is not the same thing as allowing default constructors. See: http://d.puremagic.com/issues/show_bug.cgi?id=10102
May 18 2013
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 1:05 PM, Maxim Fomin wrote:
  disable this(); commitments are cheap.

http://d.puremagic.com/issues/show_bug.cgi?id=10115
May 18 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 1:12 PM, deadalnix wrote:
 I played with that. It ensure nothing, convey intent at best.

Please file bug reports for any holes in it.
May 18 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 1:22 PM, deadalnix wrote:
 Many are, but I think that isn't the point we are discussing here.

 Removing all holes in  disable this will require the same sacrifices at the
ends
 than default constructor would. For isntance, what should happen in this case :

 S[] ss;
 ss.length = 42;

 if S has  disable this ?

Already reported: http://d.puremagic.com/issues/show_bug.cgi?id=10115
May 18 2013
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 1:39 PM, Walter Bright wrote:
 Already reported:

 http://d.puremagic.com/issues/show_bug.cgi?id=10115

And Kenji has already posted a fix! What can I say, other than Awesome!
May 18 2013
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 12:13 PM, Maxim Fomin wrote:
 Since in system language there is plenty of ways to deliberately pass invalid
 data to the place where some validity assumptions were made,  disable is a
 broken feature.

Please report all such holes to bugzilla.
May 19 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 4:13 AM, Simen Kjaeraas wrote:
 On Sat, 18 May 2013 09:14:30 +0200, Walter Bright <newshound2 digitalmars.com>
 wrote:

 On 5/18/2013 12:08 AM, deadalnix wrote:
 On 5/17/13, Walter Bright <walter digitalmars.com> wrote:
 I oppose this. D has a lot of nice features because of the .init property.
 Default constructors wreck that.

So much great feature like : - null all over the place. - Having NUllable and NonNullable in phobos. BTW, NonNullable is unable to ensure that it if effectively non nullable. - Having to check for .init state all over the place (which have a runtime cost, in addition to be error prone). RefCounted implementation is emblematic of that.

What default would you use for non-null pointers?

Damnit, I thought we'd gotten through to you. non-null pointers have no default, and it is a compile-time error not to initialize them.

See my reply to deadalnix.
May 18 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 5:04 PM, deadalnix wrote:
 On Saturday, 18 May 2013 at 20:59:41 UTC, Simen Kjaeraas wrote:
  disable this is awesome, really. And you're right that it's even better than
 simple non-nullable pointers. Lastly, it's great that it's getting fixes. It's
 been one of my favorite non-working features. :p

The subject here is default constructor, not really nullable pointer. disable this() cost as much as default constructor, but provide less. I don't see any objective reason to limit default constructor as disabled but not enabled. I'd even argue that it actually cost more as it introduce yet another special case.

this() { assert(0); } makes for a runtime check, not a compile time one. Compile time checks are more efficient in both programmer time and run time. Note that C++11 8.4.3 has disable as well, in the form: S() = delete;
 In a way, I fear that we'll end up like C++, with bare pointers/references
 being considered experts-only and 'special use', and everyone will use smart
 pointers instead.

No what will happen it that we will have null all over the place with missing check, and no stack trace when it fails, because NullPointerError have been decided to be bad. regular references/pointer are used way more than in C++ because you don't need to do the manual memory management that goes with it in D. We will have more bugs and slower programs due to null checks all over the place, and unexpressed assumption about what can be null and what cannot.

With NotNull, the runtime check only happens upon assignment and initialization from a pointer type. Dereferencing, copying, etc., of NotNull entails zero runtime overhead and no checks.
May 18 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 5:23 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 00:14:26 UTC, Walter Bright wrote:
     this() { assert(0); }

 makes for a runtime check, not a compile time one. Compile time checks are
 more efficient in both programmer time and run time.

 Note that C++11 8.4.3 has  disable as well, in the form:

    S() = delete;

D has disable. If default constructor are allowed, default constructor can be disabled, as any other declared symbol.

I understand that. But the rationale you gave for having a default constructor was to be able to disable default construction.
 With NotNull, the runtime check only happens upon assignment and
 initialization from a pointer type. Dereferencing, copying, etc., of NotNull
 entails zero runtime overhead and no checks.

People go for the shortest path. You end up having nullable everywhere. Experience have been a countless amount of time in bazillion programming languages. But it is probably too late to fix this in D anyway.

I know that some languages have a special syntax for non-null pointers. I disliked this solution for D because non-nullable pointers are just one instance of creating a type with a limited set of values. Why not go for a general solution? Why not create a mechanism where a type can be defined that can only consist of prime numbers, for example? Why not ranged integer types?
May 18 2013
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05/19/2013 03:20 AM, Walter Bright wrote:
 On 5/18/2013 5:23 PM, deadalnix wrote:
 ...
 People go for the shortest path. You end up having nullable everywhere.
 Experience have been a countless amount of time in bazillion programming
 languages. But it is probably too late to fix this in D anyway.

I know that some languages have a special syntax for non-null pointers.

The saner ones have special syntax for _nullable_ types.
 I disliked this solution for D because non-nullable pointers are just
 one instance of creating a type with a limited set of values.

I think this is the wrong direction of approaching the problem. D-style nullable references are an example of adding a value to a type whose typing judgements are invalid for that value: class C{ void foo(){ ... } void main(){ typeof(null) a; a.foo(); // compile-time error C c = a; c.foo(); // segmentation fault } null is not a C, as it does not provide a C.foo method.
 Why not go for a general solution?

You may want a type that has more than one special value. Why have language support for exactly one special value? In any case, why have unsound type checking for the sake of special values?
 Why not create a mechanism where a type can be defined that can only consist
of prime numbers, for example?

There are programming languages that allow this.
 Why not ranged integer types?

May 18 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 9:06 PM, Jonathan M Davis wrote:
 The closest that there is is
 C++'s references, which aren't rebindable and in many ways act more like
 aliases than pointers.

You can trivially create null references in C++: int* p = NULL; int& r = *p;
May 18 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 9:42 PM, Jonathan M Davis wrote:
 On Saturday, May 18, 2013 21:30:57 Walter Bright wrote:
 On 5/18/2013 9:06 PM, Jonathan M Davis wrote:
 The closest that there is is
 C++'s references, which aren't rebindable and in many ways act more like
 aliases than pointers.

You can trivially create null references in C++: int* p = NULL; int& r = *p;

Yes, but they're designed with the idea that they're non-nullable. You can't assign NULL to them or check if they're NULL. It's just that it's possible to make them NULL by the trick that you just showed.

I don't even think it's a trick, as it can easily happen unintentionally.
 Really, it's an example of
 how C++ references are more like aliases than pointers (much as they're
 pointers underneath the hood). All around, they're a bad example of a non-
 nullable pointer even though that's kind of what they're supposed to be in
 principle.

 - Jonathan M Davis

May 18 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 8:54 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 01:20:31 UTC, Walter Bright wrote:
 I understand that. But the rationale you gave for having a default constructor
 was to be able to disable default construction.

RAII or construction based on template parameters.

I know what default constructors are used for in C++. That wasn't what I asked, though. I asked for compelling rationale.
 I know that some languages have a special syntax for non-null pointers. I
 disliked this solution for D because non-nullable pointers are just one
 instance of creating a type with a limited set of values. Why not go for a
 general solution? Why not create a mechanism where a type can be defined that
 can only consist of prime numbers, for example? Why not ranged integer types?

Actually most languages does the reverse. They have non nullable type, and provide Nullable/Option/Maybe/whatever to allow for nullable type. It allow to ensure that null don't pops up in unexpected places, and that null must be handled when it is an option.

Can you list some of those languages?
 The current solution is to rely on faith, and I remember someone talking about
 that at DConf recently.

Rely on what faith?
 Now that what other languages does is cleared, let's do some consideration on
null.

 A pointer point on something. For instance, an int* point on an integer. null
 doesn't point on a integer. Non nullable pointer aren't a restricted set of
 values, as, by definition, null isn't a value that point to an int. That
doesn't
 stand either.

By definition? Pointer semantics are what we choose it to mean.
 D already have thing like Nullable in the standard lib. Introducing Maybe is
 also pretty easy. Adding NonNullable in addition to Nullable sound like
 something is not quite right.

Nullable is something different - it exists to give a 'null' value to things that don't have a null representation, like an int. C#, for example, has a Nullable type constructor, but it still has object references that can be null. C# has no standard way to produce a non-nullable object reference. There's no non-null wrapper in Java, either.
 The benefit we get from null, ie having a default initialization for
everything,
 is moot if  disable this() is introduced, so it is questionable at this point
 how useful it really is except as make up over current implementation
deficiencies.

It's not fundamentally different from other schemes to prevent any default initialization.
 Finally it has to be added that null cause holes in  safe in ways that are
 difficult to impossible to solve.

I presume you are talking about objects bigger than 64k. It is a problem, and we'll have to deal with it, but a rare one.
May 18 2013
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05/19/2013 06:57 AM, Walter Bright wrote:
 D already have thing like Nullable in the standard lib. Introducing
 Maybe is
 also pretty easy. Adding NonNullable in addition to Nullable sound like
 something is not quite right.

Nullable is something different - it exists to give a 'null' value to things that don't have a null representation, like an int.

Type Object ought not to have a "null representation" either.
 C#, for example, has a Nullable type constructor, but it still has
 object references that can be null. C# has no standard way to produce a
 non-nullable object reference.

 There's no non-null wrapper in Java, either.

This limits those languages' static type safety as much as D's.
May 19 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 5:02 AM, Simen Kjaeraas wrote:
 By definition? Pointer semantics are what we choose it to mean.


For many types, it is extremely useful to have some sort of "invalid" value for it. null fills that role nicely for pointers, just as nan does for floating point types, and 0xFF does for UTF-8. There's not anything insane about it. The Nullable type constructor even exists in order to provide such an invalid state for types (like int) which normally do not have one. Yes, I do understand there's a role for pointers which cannot hold the invalid value.
May 19 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 12:31 PM, Minas Mina wrote:
 On Sunday, 19 May 2013 at 18:30:09 UTC, deadalnix wrote:
 void buzz(Foo f) {
     f.foo(); // Rely in faith. It is invalid and way easier to write than the
 valid code, which is THE recipe for it to spread.
 }

Shouldn't this throw a NullPointerSomething?

It throws a seg fault at runtime. It *is* checked for by the hardware.
May 19 2013
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05/19/2013 09:42 PM, Walter Bright wrote:
 On 5/19/2013 12:31 PM, Minas Mina wrote:
 On Sunday, 19 May 2013 at 18:30:09 UTC, deadalnix wrote:
 void buzz(Foo f) {
     f.foo(); // Rely in faith. It is invalid and way easier to write
 than the
 valid code, which is THE recipe for it to spread.
 }

Shouldn't this throw a NullPointerSomething?

It throws a seg fault at runtime. It *is* checked for by the hardware.

Yes, but this code looks like it calls method 'foo', which is probably its intention. Hence it is buggy. D's current answer is the following: void buzz(Foo f)in{assert(!!f);}body{ f.foo(); }
May 19 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 1:03 PM, Maxim Fomin wrote:
 I think there is difference between catching exception and saving
 data which you have typed for some period and letting harware
 "check" the exception for you meanwile loosing your work.

You can catch seg faults. It's easier on Windows, but it's doable on Linux.
May 19 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 5:23 PM, John Colvin wrote:
 On Sunday, 19 May 2013 at 20:45:39 UTC, Walter Bright wrote:
 On 5/19/2013 1:03 PM, Maxim Fomin wrote:
 I think there is difference between catching exception and saving
 data which you have typed for some period and letting harware
 "check" the exception for you meanwile loosing your work.

You can catch seg faults. It's easier on Windows, but it's doable on Linux.

What's the rational for not doing this by default in D? Wouldn't a MemoryAccessError or similar be better than crashing out with SIGSEGV ?

Writing a seg fault handler under Linux has a large number of weird constraints.
 I have
 no idea about the consequences of this (other than tempting people to catch a
 segfault when they shouldn't, which is pretty much always).

At some point, all the scaffolding and workarounds to try to prevent programmers from having to deal with the underlying reality of how the system works is not appropriate for a systems programming language.
May 19 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 10:41 AM, deadalnix wrote:
 I have bunch of code that goes like :

 auto oldVar = var;
 scope(exit) var = oldVar;

 This is begging for a RAII solution where I pass var as template parameter but
 would require default constructor. This is an actual problem I have right now
as
 all save/restore are harder and harder to keep in sync for no reason and
 generate a lot of boilerplate.

 This is a problem I have right now that default constructor would solve, and
 this isn't the first time I hit that need.

oldVar isn't being default constructed in your example, nor can I see why you'd need a default constructor in order to use RAII for save/restore.
May 19 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 11:37 AM, deadalnix wrote:
 On Sunday, 19 May 2013 at 18:27:08 UTC, Walter Bright wrote:
 oldVar isn't being default constructed in your example, nor can I see why
 you'd need a default constructor in order to use RAII for save/restore.

I need to save the value at construction and restore at destruction. I don't need any runtime parameter at construction.

The saved value is initialized with the value to be saved. This is not default construction.
May 19 2013
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-05-19 19:41, deadalnix wrote:

 I have bunch of code that goes like :

 auto oldVar = var;
 scope(exit) var = oldVar;

 This is begging for a RAII solution where I pass var as template
 parameter but would require default constructor. This is an actual
 problem I have right now as all save/restore are harder and harder to
 keep in sync for no reason and generate a lot of boilerplate.

 This is a problem I have right now that default constructor would solve,
 and this isn't the first time I hit that need.

You can do something like this: void restore (alias value, alias dg) () { auto tmp = value; scope (exit) value = tmp; dg(); } int a; void foo () { a = 4 }; void main () { a = 3; restore!(a, { foo(); }); } The syntax isn't that pretty but it should work. I wish D had better syntax for this, something like: restore(a) { foo(); } -- /Jacob Carlborg
May 19 2013
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 1:41 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 04:57:15 UTC, Walter Bright wrote:
 On 5/18/2013 8:54 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 01:20:31 UTC, Walter Bright wrote:
 I understand that. But the rationale you gave for having a default
 constructor
 was to be able to disable default construction.

RAII or construction based on template parameters.

I know what default constructors are used for in C++. That wasn't what I asked, though. I asked for compelling rationale.

I have bunch of code that goes like : auto oldVar = var; scope(exit) var = oldVar; This is begging for a RAII solution where I pass var as template parameter but would require default constructor.

No need for a default constructor. You pass the current value as a constructor parameter. Andrei
May 19 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, May 18, 2013 00:14:30 Walter Bright wrote:
 What default would you use for non-null pointers?

It would basically have to disable init, because you can't have a default for non-nullable pointers. It just wouldn't make sense. Pretty much the whole point of them is to guarantee that they've been initialized with a valid value. - Jonathan M Davis
May 18 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 18 May 2013 at 07:14:29 UTC, Walter Bright wrote:
 What default would you use for non-null pointers?

Compile time error as a default sound like a nice option. Probably too late now, but that is where most language are going now, and having done quite a lot of java myself, I can guarantee you that it make sense.
May 18 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Sat, 18 May 2013 09:14:30 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 5/18/2013 12:08 AM, deadalnix wrote:
 On 5/17/13, Walter Bright <walter digitalmars.com> wrote:
 I oppose this. D has a lot of nice features because of the .init  
 property.
 Default constructors wreck that.

So much great feature like : - null all over the place. - Having NUllable and NonNullable in phobos. BTW, NonNullable is unable to ensure that it if effectively non nullable. - Having to check for .init state all over the place (which have a runtime cost, in addition to be error prone). RefCounted implementation is emblematic of that.

What default would you use for non-null pointers?

Damnit, I thought we'd gotten through to you. non-null pointers have no default, and it is a compile-time error not to initialize them. -- Simen
May 18 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 18 May 2013 at 14:22:44 UTC, Andrei Alexandrescu 
wrote:
 This is what I call the "low potential energy mental block". 
 Very hard to get out of.

Can you explain this please ?
May 18 2013
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 5/18/13, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
 This is what I call the "low potential energy mental block". Very hard
 to get out of.

Well in any case, I'd like to gather all intel on why it's bad to allow a default ctor so we can put it in the documentation as a rationale. It will avoid endless discussions.
May 18 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 18 May 2013 at 14:43:30 UTC, Andrej Mitrovic wrote:
 On 5/18/13, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> 
 wrote:
 This is what I call the "low potential energy mental block". 
 Very hard
 to get out of.

Well in any case, I'd like to gather all intel on why it's bad to allow a default ctor so we can put it in the documentation as a rationale. It will avoid endless discussions.

I think when you collect arguments why it is bad to have a default ctor, you will found how easily they are beaten by contraarguments. Endless discussions comes from community disagree with Walter, not because of endless flow of people asking same question, getting same answer and quiting discussion repeteadly.
May 18 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 18 May 2013 at 18:41:49 UTC, Walter Bright wrote:
 On 5/18/2013 12:30 AM, deadalnix wrote:
 On Saturday, 18 May 2013 at 07:14:29 UTC, Walter Bright wrote:
 What default would you use for non-null pointers?

Compile time error as a default sound like a nice option. Probably too late now, but that is where most language are going now, and having done quite a lot of java myself, I can guarantee you that it make sense.

D has that: disable this(); which is not the same thing as allowing default constructors. See: http://d.puremagic.com/issues/show_bug.cgi?id=10102

struct S { int a; disable this(); this(int) { a = 1; } ~this() { assert(a !is 0); } } enum E : S { A = S.init } union U { S s; this(this) { assert (s.a !is 0); } ~this() { assert (s.a !is 0); } } void foo(out S s, out E e, out U u) { } void main() { S[] arr; arr.length = 5; // compiles E[] e; e.length = 5; // compiles S[1] x = (S[1]).init; // compiles U[] u; u.length = 5; //compiles foo(arr[0], e[0], u[0]); // compiles } disable this(); commitments are cheap.
May 18 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 18 May 2013 at 18:41:49 UTC, Walter Bright wrote:
 On 5/18/2013 12:30 AM, deadalnix wrote:
 On Saturday, 18 May 2013 at 07:14:29 UTC, Walter Bright wrote:
 What default would you use for non-null pointers?

Compile time error as a default sound like a nice option. Probably too late now, but that is where most language are going now, and having done quite a lot of java myself, I can guarantee you that it make sense.

D has that: disable this(); which is not the same thing as allowing default constructors. See: http://d.puremagic.com/issues/show_bug.cgi?id=10102

I played with that. It ensure nothing, convey intent at best.
May 18 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 18 May 2013 at 20:19:11 UTC, Walter Bright wrote:
 On 5/18/2013 1:12 PM, deadalnix wrote:
 I played with that. It ensure nothing, convey intent at best.

Please file bug reports for any holes in it.

Many are, but I think that isn't the point we are discussing here. Removing all holes in disable this will require the same sacrifices at the ends than default constructor would. For isntance, what should happen in this case : S[] ss; ss.length = 42; if S has disable this ?
May 18 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Sat, 18 May 2013 20:42:32 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 5/18/2013 4:13 AM, Simen Kjaeraas wrote:

 On Sat, 18 May 2013 09:14:30 +0200, Walter Bright  
 <newshound2 digitalmars.com>
 wrote:

 What default would you use for non-null pointers?

Damnit, I thought we'd gotten through to you. non-null pointers have no default, and it is a compile-time error not to initialize them.

See my reply to deadalnix.

On Sat, 18 May 2013 20:41:48 +0200, Walter Bright <newshound2 digitalmars.com> wrote:
 D has that:

      disable this();

 which is not the same thing as allowing default constructors. See:

 http://d.puremagic.com/issues/show_bug.cgi?id=10102

Oh, absolutely. I'm not at all claiming they are (Nor do I think others claim that). It's just it was so hard to make you see the value of non-nullable pointers, and I feared you'd regressed. disable this is awesome, really. And you're right that it's even better than simple non-nullable pointers. Lastly, it's great that it's getting fixes. It's been one of my favorite non-working features. :p In a way, I fear that we'll end up like C++, with bare pointers/references being considered experts-only and 'special use', and everyone will use smart pointers instead. -- Simen
May 18 2013
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 18 May 2013 at 20:59:41 UTC, Simen Kjaeraas wrote:
  disable this is awesome, really. And you're right that it's 
 even better than
 simple non-nullable pointers. Lastly, it's great that it's 
 getting fixes. It's
 been one of my favorite non-working features. :p

The subject here is default constructor, not really nullable pointer. disable this() cost as much as default constructor, but provide less. I don't see any objective reason to limit default constructor as disabled but not enabled. I'd even argue that it actually cost more as it introduce yet another special case.
 In a way, I fear that we'll end up like C++, with bare 
 pointers/references
 being considered experts-only and 'special use', and everyone 
 will use smart
 pointers instead.

No what will happen it that we will have null all over the place with missing check, and no stack trace when it fails, because NullPointerError have been decided to be bad. regular references/pointer are used way more than in C++ because you don't need to do the manual memory management that goes with it in D. We will have more bugs and slower programs due to null checks all over the place, and unexpressed assumption about what can be null and what cannot.
May 18 2013
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05/19/2013 02:32 AM, Jonathan M Davis wrote:
 On Sunday, May 19, 2013 02:22:43 Simen Kjaeraas wrote:
 Or... possibly, the current holes in  disable are fixed, and NonNull!T
 becomes the default, because we tell people to always use them, rather
 than flail our arms and behave like idiots. ("regular pointers are
 broken, use NonNull!T" is a pretty good argument if it's true)

I've never understood why so many people feel that nullable pointers are a problem. Clearly, many people do, but personally, I've rarely had problems with them, and there are plenty of cases where not being to make a pointer null would really suck (which is why we're forced to have std.typecons.Nullable for non-reference types). I'm not arguing against having non-nullable pointers, but I'd probably almost never use them myself, as I really don't think that they'd be buying me much. In my experince, problems with null pointers are extremely rare and easily caught. - Jonathan M Davis

I don't write a lot of buggy code for myself either. You are arguing for leaving out cheap code documentation. Also, in a sane system you'd be forced to use types without a null value in many cases, as potentially null references ought not be dereferenced. i.e. potentially null would mean they _really_ can be null, because otherwise an actual reference to a value type could be used.
May 18 2013
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 8:55 AM, deadalnix wrote:
 On Sunday, 19 May 2013 at 12:06:01 UTC, Simen Kjaeraas wrote:
 My experience is the complete opposite - I think maybe 20% of bugs at
 my job
 are caused by null references. But as you say, they are very easily
 fixed.

Sometime they are super freaking hard. I have a horror story debugging Apache Cayenne : http://cayenne.apache.org/ because it was throwing NPE on a race condition that would never show up once the code is instrumented. A reference was null for a short moment and then set to something. Due to concurrency, in extreme cases it could be seen as null where it was assumed everywhere to be set.

Sounds like a race problem unrelated to null. With non-null objects the race would have manifested itself in a different way, perhaps even more pernicious. Anyhow, this discussion should have finality. We could go on forever arguing the usefulness or lack thereof of non-nullable references. They didn't catch up in some languages and did in some. My personal opinion is "nice to have, but not greatly compelling". It's reasonable to assume no major language changes will accommodate non-null references, so the next best thing would be to focus on a library solution. As Walter said, a library solution has the perk of allowing other interesting restricted types. Andrei
May 19 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 9:11 AM, deadalnix wrote:
 On Sunday, 19 May 2013 at 13:08:53 UTC, Andrei Alexandrescu wrote:
 Sounds like a race problem unrelated to null. With non-null objects
 the race would have manifested itself in a different way, perhaps even
 more pernicious.

It is both a race condition and a null problem.

No, it's just a race condition.
 And having non nullable
 type would have been a compile time error instead of days of debugging.

No, the race condition would have stayed. Andrei
May 19 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 10:35 AM, deadalnix wrote:
 On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote:
 No, the race condition would have stayed.

That is ridiculous. non nullable would have made the bug non existent, and even without race condition the problem would exists. a reference is null, it container shared, then set to something else. You can put barriers all over the place to make that sequentially consistent that it wouldn't change anything and the bug would still arise.

I agree with Andrei that eliminating null is not going to make a race condition into a thread safe one. You've got a serious bug that you may succeed in hiding by eliminating the null, but it's still there. From your description, I suspect the code is suffering from the double-checked locking bug (which can appear in very subtle forms).
 You also never provided any convincing solution to the safety hole. We can't
 even add check only on some edges cases as D also have values types. The only
 solution we are left with that is really safe is to null check every
dereference
 or give up on  safe.

Please don't make us guess what exactly you mean by this.
May 19 2013
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 3:10 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 18:46:31 UTC, Walter Bright wrote:
 You also never provided any convincing solution to the safety hole.
 We can't
 even add check only on some edges cases as D also have values types.
 The only
 solution we are left with that is really safe is to null check every
 dereference
 or give up on  safe.

Please don't make us guess what exactly you mean by this.

This isn't new and I discussed that again and again. When you dereference null, you hit the first plage, which is protected on most systems. But if you access an element with sufficient offset you bypass all protections provided by the type system and you are back in unsafe world.

Oh, the good old "object of sufficient size". We know how to fix that.
 And no, putting nullcheck on access of field of sufficient offset (as
 propose dby Andrei) isn't enough because we have value types. Consider :

 S[BIG_NUMBER]* a;
 auto s = &(*a[SLIGHTLY_BELLOW_CHECK_OFFSET]);
 s.fieldAccess; // May not have enough offset to trigget null check, but
 still can be usnafe

 See bug reports :
 http://d.puremagic.com/issues/show_bug.cgi?id=3677
 http://d.puremagic.com/issues/show_bug.cgi?id=5176

All of the above are variations on the "sufficiently large object" theme. Andrei
May 19 2013
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 3:41 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 19:15:47 UTC, Andrei Alexandrescu wrote:
 Oh, the good old "object of sufficient size". We know how to fix that.

 And no, putting nullcheck on access of field of sufficient offset (as
 propose dby Andrei) isn't enough because we have value types. Consider :

 S[BIG_NUMBER]* a;
 auto s = &(*a[SLIGHTLY_BELLOW_CHECK_OFFSET]);
 s.fieldAccess; // May not have enough offset to trigget null check, but
 still can be usnafe

 See bug reports :
 http://d.puremagic.com/issues/show_bug.cgi?id=3677
 http://d.puremagic.com/issues/show_bug.cgi?id=5176

All of the above are variations on the "sufficiently large object" theme. Andrei

The code above never access a field with a sufficient offset to trigger "sufficiently large runtime check".

It does, when the pointer to the large static array is dereferenced. Andrei
May 19 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 12:10 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 18:46:31 UTC, Walter Bright wrote:
 You also never provided any convincing solution to the safety hole. We can't
 even add check only on some edges cases as D also have values types. The only
 solution we are left with that is really safe is to null check every
dereference
 or give up on  safe.

Please don't make us guess what exactly you mean by this.

This isn't new and I discussed that again and again. When you dereference null, you hit the first plage, which is protected on most systems. But if you access an element with sufficient offset you bypass all protections provided by the type system and you are back in unsafe world.

And we've replied to this before. But when you say "give up on safe", that implies a far more serious issue, so I want to make sure what you're talking about. I agree that we need to deal with the issue. But on a practical note, if we solve 99% of the safe issues, and fail at 1%, that doesn't mean there is no value to safe and we should give up on it.
May 19 2013
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 4:05 PM, Walter Bright wrote:
 On 5/19/2013 12:10 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 18:46:31 UTC, Walter Bright wrote:
 You also never provided any convincing solution to the safety hole.
 We can't
 even add check only on some edges cases as D also have values types.
 The only
 solution we are left with that is really safe is to null check every
 dereference
 or give up on  safe.

Please don't make us guess what exactly you mean by this.

This isn't new and I discussed that again and again. When you dereference null, you hit the first plage, which is protected on most systems. But if you access an element with sufficient offset you bypass all protections provided by the type system and you are back in unsafe world.

And we've replied to this before. But when you say "give up on safe", that implies a far more serious issue, so I want to make sure what you're talking about. I agree that we need to deal with the issue. But on a practical note, if we solve 99% of the safe issues, and fail at 1%, that doesn't mean there is no value to safe and we should give up on it.

Almost safe == almost pregnant. safe must be 100% safe. Andrei
May 19 2013
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 1:35 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote:
 On 5/19/13 9:11 AM, deadalnix wrote:
 It is both a race condition and a null problem.

No, it's just a race condition.
 And having non nullable
 type would have been a compile time error instead of days of debugging.

No, the race condition would have stayed.

That is ridiculous. non nullable would have made the bug non existent, and even without race condition the problem would exists. a reference is null, it container shared, then set to something else. You can put barriers all over the place to make that sequentially consistent that it wouldn't change anything and the bug would still arise.

No, your argument is ridiculous. You make a yarn with precious little detail that describes for everything everyone knows a textbook race condition, essentially ask that you are taking by your word that non-null would miraculously solve it, and, to add insult to injury, and when we don't buy it, you put the burden of proof on us. This is quite a trick, my hat is off to you.
 You also never provided any convincing solution to the safety hole.

What's the safety hole? Objects of large static size?
 We
 can't even add check only on some edges cases as D also have values
 types. The only solution we are left with that is really safe is to null
 check every dereference or give up on  safe.

How about using NonNull. We won't change the language at this point to make non-nullable references by default. Even you acknowledged that that's not practical. So now you contradict your own affirmation. What exactly do you sustain, and what are you asking for?
 I encourage you to look at this :
 http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare

I read it. I don't buy it. Yeah, it's a point, but it's largely exaggerated for dramatic purposes.
 Most new languages removed nullable by default, or limited its uses
 (scala for instance, allow for null for limited scope).

So what do you realistically think we should do, seeing that we're aiming at stability?
 I once again want to get attention on the fact that GC change everything
 in regard to reference, and that the C++ situation is a bad example.

I don't understand this. Andrei
May 19 2013
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05/19/2013 09:10 PM, Andrei Alexandrescu wrote:
 On 5/19/13 1:35 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote:
 On 5/19/13 9:11 AM, deadalnix wrote:
 It is both a race condition and a null problem.

No, it's just a race condition.
 And having non nullable
 type would have been a compile time error instead of days of debugging.

No, the race condition would have stayed.

That is ridiculous. non nullable would have made the bug non existent, and even without race condition the problem would exists. a reference is null, it container shared, then set to something else. You can put barriers all over the place to make that sequentially consistent that it wouldn't change anything and the bug would still arise.

No, your argument is ridiculous. You make a yarn with precious little detail that describes for everything everyone knows a textbook race condition, essentially ask that you are taking by your word that non-null would miraculously solve it, and, to add insult to injury, and when we don't buy it, you put the burden of proof on us. This is quite a trick, my hat is off to you. ...

It is easy to buy that the buggy code would have been rejected by a stronger type system whereas the fixed code would have passed type checking. Especially given that it manifested itself as a NPE, which would not even exist.
May 19 2013
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 05/19/2013 09:10 PM, Andrei Alexandrescu wrote:
 I encourage you to look at this :
 http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare

I read it. I don't buy it. Yeah, it's a point, but it's largely exaggerated for dramatic purposes.

(It is a talk.)
May 19 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 3:58 PM, Timon Gehr wrote:
 On 05/19/2013 09:10 PM, Andrei Alexandrescu wrote:
 I encourage you to look at this :
 http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare

I read it. I don't buy it. Yeah, it's a point, but it's largely exaggerated for dramatic purposes.

(It is a talk.)

(I read a transcript.) Andrei
May 19 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 1:06 PM, Andrei Alexandrescu wrote:
 On 5/19/13 3:58 PM, Timon Gehr wrote:
 (It is a talk.)

(I read a transcript.)

I wish more talks had transcripts available. I can read an hour talk presentation in 5 minutes.
May 19 2013
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 3:36 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 19:10:28 UTC, Andrei Alexandrescu wrote:
 No, your argument is ridiculous. You make a yarn with precious little
 detail that describes for everything everyone knows a textbook race
 condition, essentially ask that you are taking by your word that
 non-null would miraculously solve it, and, to add insult to injury,
 and when we don't buy it, you put the burden of proof on us. This is
 quite a trick, my hat is off to you.

I described a very usual null bug : something is set to null, then to a specific value. It is assumed not to be null. In a specific case it is null and everything explode. The concurrent context here made it especially hard to debug, but isn't the cause of the bug. Additionally, if you don't have enough information to understand what I'm saying, you are perfectly allowed to ask for additional details This isn't a shame.

Your argument has been destroyed so no need to ask details about it. Replace "null" with "invalid state" and it's the same race in any system. Let's move on.
 What's the safety hole? Objects of large static size?

Limiting object size isn't going to cut it. Or must be super restrictive (the protection is 4kb on some systems).

Well you got to do what you got to do. Field accesses for objects larger than 4KB would have to be checked in safe code.
 We
 can't even add check only on some edges cases as D also have values
 types. The only solution we are left with that is really safe is to null
 check every dereference or give up on  safe.

How about using NonNull. We won't change the language at this point to make non-nullable references by default. Even you acknowledged that that's not practical. So now you contradict your own affirmation. What exactly do you sustain, and what are you asking for?

1/ NonNull do not work.

You made the argument that although it does work, people will not use it because it's not the default. That's not quite "does not work". This also ruins your point because if people don't find it worth writing NonNull!T instead of T, it means non-null doesn't buy them anything worthwhile.
 2/ It isn't because it is too late to solve a problem that it magically
 isn't a problem anymore.

You are blowing it out of proportion. Null references are hardly even on the radar of the bug classes I'm encountering in the style of programming of the three groups I worked in at Facebook, and also my previous employers. People I meet at conferences and consulting gigs never mention null references as a real problem, although I very often ask about problems. I find it difficult to agree with you just to be nice.
 Most new languages removed nullable by default, or limited its uses
 (scala for instance, allow for null for limited scope).

So what do you realistically think we should do, seeing that we're aiming at stability?

Acknowledge it was a mistake and move on.

I'd give it more thought if we designed D from scratch. I think it's safe to move on. At any rate, I'd love if we got NonNull working nicely so we accumulate more experience with it. Andrei
May 19 2013
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 1:08 PM, Mr. Anonymous wrote:
 On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:
 Well you got to do what you got to do. Field accesses for objects larger than
 4KB would have to be checked in  safe code.

Isn't the solution as easy as doing: OR PTR:[address], 0 the same way it's done for the stack? The offset it known at compile time in most cases, so the command would be required only if both: * The object is larger than target OS' guard page size. * The position is greater than target OS' guard page size, OR is unknown at compile time.

Not a bad idea.
May 19 2013
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 4:30 PM, Peter Alexander wrote:
 On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:
 You are blowing it out of proportion. Null references are hardly even
 on the radar of the bug classes I'm encountering in the style of
 programming of the three groups I worked in at Facebook, and also my
 previous employers. People I meet at conferences and consulting gigs
 never mention null references as a real problem, although I very often
 ask about problems. I find it difficult to agree with you just to be
 nice.

Just because people don't mention them as a problem doesn't mean it isn't a problem. For what it's worth, null pointers are a real problem in the code I work on (games). I don't know exactly what you work on, but I find that they are more of a problem in highly stateful, interactive applications. Things like generic libraries, utility programs, compilers, etc. probably won't see the same problems because they aren't very stateful or interactive. In my experience, null pointers are easy to fix, but the risk of them causes people to litter their code with if (ptr) tests, often with poor handling of the failure case, which can cause subtle bugs (no crash, but unintended code path). Just my 2c.

OK, this is sensible. One question - would you be willing to type symbols as NullType!T instead of T to avoid these issues? Thanks, Andrei
May 19 2013
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/20/13 11:19 AM, Byron Heads wrote:
 On Sun, 19 May 2013 17:36:17 -0400, Andrei Alexandrescu wrote:
 OK, this is sensible. One question - would you be willing to type
 symbols as NullType!T instead of T to avoid these issues?

 Thanks,

 Andrei

More boiler plate code for functions that take pointers. void foo(T)(T t) if(isPointer!T) { static if(isNullable!T) if(!t) throw .... }

But this goes both ways. Regardless of the default, you'd sometimes need to distinguish between cases. You either hurt one half of your cases or the other half. In fact you are now showing the case that assumes non-nullable being the default.
 May also introduce then need to check for objects in the init state
 (default state)



 outside of NonNull in stdlib, an attribute maybe a possible answer.

 void bar( nonnull SomeClass o)
 {
 	o = foo(); // if foo returns  nonnull, then check is not needed,
 else needs a check added during assignment
 }

 there are a few compile time checks that can be done to prove o is not
 null, if not the compiler adds runtime checks.

I think a parameterized type is a better match for non-null because it attaches to the type, not the symbol.
 But really, checking for valid input is part of programming, might be
 better to have lint integration that can help find these types of problems

Agreed. Andrei
May 20 2013
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 5:52 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:
 On 5/19/13 3:36 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 19:10:28 UTC, Andrei Alexandrescu wrote:
 No, your argument is ridiculous. You make a yarn with precious little
 detail that describes for everything everyone knows a textbook race
 condition, essentially ask that you are taking by your word that
 non-null would miraculously solve it, and, to add insult to injury,
 and when we don't buy it, you put the burden of proof on us. This is
 quite a trick, my hat is off to you.

I described a very usual null bug : something is set to null, then to a specific value. It is assumed not to be null. In a specific case it is null and everything explode. The concurrent context here made it especially hard to debug, but isn't the cause of the bug. Additionally, if you don't have enough information to understand what I'm saying, you are perfectly allowed to ask for additional details This isn't a shame.

Your argument has been destroyed so no need to ask details about it. Replace "null" with "invalid state" and it's the same race in any system. Let's move on.

"I don't want t to understand because I know I'm right. The fact you solved that issue and I didn't is irrelevant, I know better."

Nobody knows what the issue is. It's all unstated assumptions leading to vague claims. You have been challenged to explain it so as to count as an anecdote in favor of non-null pointers - which it may as well be! -, and failed to raise to it. Andrei
May 19 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 3:04 PM, deadalnix wrote:
 Same argument Walter like to make about very rare failure cases apply here.

1. rare as in programmers rarely create such a bug 2. rare as in being rare for an existing bug to show itself I was referring to (1), while you are referring to (2).
May 19 2013
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05/20/2013 02:33 AM, deadalnix wrote:
 On Monday, 20 May 2013 at 00:09:23 UTC, Walter Bright wrote:
 On 5/19/2013 3:04 PM, deadalnix wrote:
 Same argument Walter like to make about very rare failure cases apply
 here.

1. rare as in programmers rarely create such a bug 2. rare as in being rare for an existing bug to show itself I was referring to (1), while you are referring to (2).

When you talk about UNIX utilities not handling properly a full filesystem for instance, you are referring to 1.

You mean 2.
May 19 2013
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 8:46 PM, Steven Schveighoffer wrote:
 I just wanted to chime in with this understanding of the bug that I am
 reading from deadalnix's descriptions:

 SomeObj obj;
 shareTheObj(&obj); // goes to other threads
 obj = new SomeObj; // initialize obj

 This is likely simpler than the actual problem, but I think this is the
 gist of it.

 A Non-Nullable solution WOULD solve the race:

 SomeObj obj; // Compiler: nope, can't do that, must initialize it.

 Now, with an "invalid state" not available like null, is the developer
 more likely to go through the trouble of building an invalid state for
 obj, in order to keep the racy behavior? No. He's just going to move the
 initialization:

 SomeObj obj = new SomeObj;
 shareTheObj(&obj);

 In essence the point of the anecdote is that a Non-Nullable reference
 would have PROMOTED avoiding the race condition -- it would have been
 harder to keep the racy behavior.

One can only assume the entire point was to delay initialization of the object to its first use/sharing point, so the changed pattern simply initializes the object eagerly, thus missing the benefits of the pattern. Otherwise that's an instance of the old adage "initialize variables at the point of definition". That, granted, non-null references do help with. Andrei
May 20 2013
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 12:36 PM, deadalnix wrote:
 I C or C++ you are doomed to manage reference as you need to for memory
 management purpose. In garbage collected languages, you ends up with way more
 unmanaged references, because the GC take care of them. Doing so you multiply
 the surface area where null bug can strike.

I still don't see the connection. I've had many more null pointer bugs in non-gc code than in gc code, but I attribute that to my being a better programmer in gc code because I did that later and was more aware of issues.
May 19 2013
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 1:12 PM, Idan Arye wrote:
 On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote:
 On 5/19/13 9:11 AM, deadalnix wrote:
 On Sunday, 19 May 2013 at 13:08:53 UTC, Andrei Alexandrescu wrote:
 Sounds like a race problem unrelated to null. With non-null objects
 the race would have manifested itself in a different way, perhaps even
 more pernicious.

It is both a race condition and a null problem.

No, it's just a race condition.
 And having non nullable
 type would have been a compile time error instead of days of debugging.

No, the race condition would have stayed. Andrei

I believe this claim requires an explanation: It's a good practice to initialize references(and all other types of variables) as soon as possible - and if possible, right away in the declaration. If that reference started as null, it's safe to assume it was not possible to initialized it at declaration, so it was intentionally initialized with null(if there was no initialization Java would scream at you). Now, let's assume that reference was non-nullable. It is safe to assume that this change would not remove the obstacle that prevented that reference from being initialized right away in the declaration - so you still need to initialize it to something else - let's call that something `Nil`. Nil is an object that tells you that the reference has not yet been initialized. So, in the original bug the reference "could be seen as null where it was assumed everywhere to be set.". But now we don't have null - so that piece of code that thought the reference is null would now think that it is... what? The initialization value? No! Because we didn't switch from initializing the reference with null to initializing it with the later initialization value - we couldn't do it. Instead, we had to use Nil. So now, the reference 'could be seen as Nil where it was assumed everywhere to be set'... Now, if you are lucky and your Nil is the better-kind-of-null that is used in dynamic object oriented languages, you'll get a nil exception - which is just as good as null exception. But if you are not so lucky, and you had to declare Nil as a blank object of the type of that reference, you are going to have logical bug, which is far worse than exceptions...

So the race would have manifested it just the same, except under the name of Nil instead of null. Andrei
May 19 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 11:22 AM, deadalnix wrote:
 The damn thing should have been initialized in the
 first place to avoid the bug.

Sounds like you have the double-checked locking bug. Using a different value to initialize it won't fix it. If you have a global value accessible from multiple threads, you must use synchronization. There is no way around that. If you use some other global state to check for initialization in order to avoid synchronization, you have the double checked locking bug. Yes, you do.
May 19 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 2:02 PM, Walter Bright wrote:
 If you have a global value accessible from multiple threads, you must use
 synchronization. There is no way around that. If you use some other global
state
 to check for initialization in order to avoid synchronization, you have the
 double checked locking bug. Yes, you do.

BTW, a few years ago, I presented my clever solution to the double checked locking bug to Scott Meyers. I was very proud of it. Scott handed me my ss (in a nice way), but I got my comeuppance. It reminded me of the hours I spent in high school determined to show that I could trisect an angle with a compass and a straightedge. There was always some tiny flaw :-)
May 19 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 5:11 PM, Walter Bright wrote:
 It reminded me of the hours I spent in high school determined to show
 that I could trisect an angle with a compass and a straightedge. There
 was always some tiny flaw :-)

Yah, sounds familiar. Did you prove the parallel postulate, too? Andrei
May 19 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 3:30 PM, Andrei Alexandrescu wrote:
 On 5/19/13 5:11 PM, Walter Bright wrote:
 It reminded me of the hours I spent in high school determined to show
 that I could trisect an angle with a compass and a straightedge. There
 was always some tiny flaw :-)

Yah, sounds familiar. Did you prove the parallel postulate, too?

No. That one was intuitively obvious!
May 19 2013
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 5:56 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 21:02:57 UTC, Walter Bright wrote:
 On 5/19/2013 11:22 AM, deadalnix wrote:
 The damn thing should have been initialized in the
 first place to avoid the bug.

Sounds like you have the double-checked locking bug. Using a different value to initialize it won't fix it.

No it sound like initalizing something to null, then initialize it properly, assume all over the place that it is initialized to something else, and in some rare code path it blows up. The fact that this occurs in a multithreaded environment made it super hard to debug, but the whole thing was properly synchronized.

How was there a bug if everything was properly synchronized? You either describe the matter with sufficient detail, or acknowledge the destruction of your anecdote. This is going nowhere. Andrei
May 19 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 4:06 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote:
 How was there a bug if everything was properly synchronized? You either
 describe the matter with sufficient detail, or acknowledge the destruction of
 your anecdote. This is going nowhere.

I explained over and over. A field is initialized to null, while the object lock is owned, and later to its value, while it is locked. In the meantime, another thread access the object, owning the lock, assuming the field is always initialized.

so, you have: ========================== Thread A Thread B lock p = null unlock lock *p = ... unlock lock p = new ... unlock ========================== Although you are using locks, it still isn't properly synchronized. Changing the p=null to p=someothernonnullvalue will not fix it.
May 19 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 5:28 PM, deadalnix wrote:
 The error lie in improper
 initialization of p in the first place, which should never has been null. The
 example looks dumb as this, you have to imagine the pattern hidden in thousands
 of LOC.

I would find a design that declared a variable in one place, then initialized it in another, while releasing the lock in between as a bad design pattern to begin with. What other default initialized types could be there? What about an int default initialized to 0, yet code in another thread expects it to be some other value? I suspect there'd be a lot more bugs in it than just null pointer initializations. It might be time to engineer a new pattern so you don't have to inspect thousands of LOC to manually verify correctness.
May 19 2013
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/19/13 7:06 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote:
 How was there a bug if everything was properly synchronized? You
 either describe the matter with sufficient detail, or acknowledge the
 destruction of your anecdote. This is going nowhere.

I explained over and over. A field is initialized to null, while the object lock is owned, and later to its value, while it is locked. In the meantime, another thread access the object, owning the lock, assuming the field is always initialized.

How does another thread thread accesses the object "owning the lock" when the assignment occurs under lock? How would non-null fix this? Would the object have type Maybe? Andrei
May 19 2013
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05/20/2013 01:39 AM, Andrei Alexandrescu wrote:
 On 5/19/13 7:06 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote:
 How was there a bug if everything was properly synchronized? You
 either describe the matter with sufficient detail, or acknowledge the
 destruction of your anecdote. This is going nowhere.

I explained over and over. A field is initialized to null, while the object lock is owned, and later to its value, while it is locked. In the meantime, another thread access the object, owning the lock, assuming the field is always initialized.

How does another thread thread accesses the object "owning the lock" when the assignment occurs under lock?

lock{ initialize to null. } lock{ in the meantime assume correctly initialized. } lock{ initialize correctly. } This is nothing new. I think he has been pretty clear about what the issue is from the beginning.
 How would non-null fix this? Would the object have type Maybe?

This is one possibility. In this case, the type system would have prevented the null dereference. In the other case, the type system would have caught the invalid initialization.
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 00:14:26 UTC, Walter Bright wrote:
     this() { assert(0); }

 makes for a runtime check, not a compile time one. Compile time 
 checks are more efficient in both programmer time and run time.

 Note that C++11 8.4.3 has  disable as well, in the form:

    S() = delete;

D has disable. If default constructor are allowed, default constructor can be disabled, as any other declared symbol.
 With NotNull, the runtime check only happens upon assignment 
 and initialization from a pointer type. Dereferencing, copying, 
 etc., of NotNull entails zero runtime overhead and no checks.

People go for the shortest path. You end up having nullable everywhere. Experience have been a countless amount of time in bazillion programming languages. But it is probably too late to fix this in D anyway.
May 18 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, May 19, 2013 02:22:43 Simen Kjaeraas wrote:
 Or... possibly, the current holes in  disable are fixed, and NonNull!T
 becomes the default, because we tell people to always use them, rather
 than flail our arms and behave like idiots. ("regular pointers are
 broken, use NonNull!T" is a pretty good argument if it's true)

I've never understood why so many people feel that nullable pointers are a problem. Clearly, many people do, but personally, I've rarely had problems with them, and there are plenty of cases where not being to make a pointer null would really suck (which is why we're forced to have std.typecons.Nullable for non-reference types). I'm not arguing against having non-nullable pointers, but I'd probably almost never use them myself, as I really don't think that they'd be buying me much. In my experince, problems with null pointers are extremely rare and easily caught. - Jonathan M Davis
May 18 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 01:20:31 UTC, Walter Bright wrote:
 I understand that. But the rationale you gave for having a 
 default constructor was to be able to disable default 
 construction.

RAII or construction based on template parameters.
 I know that some languages have a special syntax for non-null 
 pointers. I disliked this solution for D because non-nullable 
 pointers are just one instance of creating a type with a 
 limited set of values. Why not go for a general solution? Why 
 not create a mechanism where a type can be defined that can 
 only consist of prime numbers, for example? Why not ranged 
 integer types?

Actually most languages does the reverse. They have non nullable type, and provide Nullable/Option/Maybe/whatever to allow for nullable type. It allow to ensure that null don't pops up in unexpected places, and that null must be handled when it is an option. The current solution is to rely on faith, and I remember someone talking about that at DConf recently. Now that what other languages does is cleared, let's do some consideration on null. A pointer point on something. For instance, an int* point on an integer. null doesn't point on a integer. Non nullable pointer aren't a restricted set of values, as, by definition, null isn't a value that point to an int. That doesn't stand either. D already have thing like Nullable in the standard lib. Introducing Maybe is also pretty easy. Adding NonNullable in addition to Nullable sound like something is not quite right. The benefit we get from null, ie having a default initialization for everything, is moot if disable this() is introduced, so it is questionable at this point how useful it really is except as make up over current implementation deficiencies. Finally it has to be added that null cause holes in safe in ways that are difficult to impossible to solve.
May 18 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, May 19, 2013 05:54:46 deadalnix wrote:
 Actually most languages does the reverse. They have non nullable
 type, and provide Nullable/Option/Maybe/whatever to allow for
 nullable type. It allow to ensure that null don't pops up in
 unexpected places, and that null must be handled when it is an
 option.

Most languages? I'm not aware of even _one_ C-based language that doesn't have null as the default for pointer/references types. They may have non-nullable reference types in addition to the nullable ones (though that's still rare in my experience), but they were all added later. The closest that there is is C++'s references, which aren't rebindable and in many ways act more like aliases than pointers. Functional languages do less with nullable, but given D's ancestory, it's not at all surprising that nullable is the default, and it would be very bizarre if it weren't. - Jonathan M Davis
May 18 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, May 18, 2013 21:30:57 Walter Bright wrote:
 On 5/18/2013 9:06 PM, Jonathan M Davis wrote:
 The closest that there is is
 C++'s references, which aren't rebindable and in many ways act more like
 aliases than pointers.

You can trivially create null references in C++: int* p = NULL; int& r = *p;

Yes, but they're designed with the idea that they're non-nullable. You can't assign NULL to them or check if they're NULL. It's just that it's possible to make them NULL by the trick that you just showed. Really, it's an example of how C++ references are more like aliases than pointers (much as they're pointers underneath the hood). All around, they're a bad example of a non- nullable pointer even though that's kind of what they're supposed to be in principle. - Jonathan M Davis
May 18 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, May 18, 2013 22:04:08 Walter Bright wrote:
 On 5/18/2013 9:42 PM, Jonathan M Davis wrote:
 On Saturday, May 18, 2013 21:30:57 Walter Bright wrote:
 On 5/18/2013 9:06 PM, Jonathan M Davis wrote:
 The closest that there is is
 C++'s references, which aren't rebindable and in many ways act more like
 aliases than pointers.

You can trivially create null references in C++: int* p = NULL; int& r = *p;

Yes, but they're designed with the idea that they're non-nullable. You can't assign NULL to them or check if they're NULL. It's just that it's possible to make them NULL by the trick that you just showed.

I don't even think it's a trick, as it can easily happen unintentionally.

Yes, but it's not something that would be done intentionally, and it's something that surprises most people. I expect that the vast majority of C++ programmers would think that it's impossible before it was explained to them. C++ references are usually sold as being non-nullable, and this is arguably a hole in their design. - Jonathan M Davis
May 18 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Sun, 19 May 2013 06:57:16 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:

 The current solution is to rely on faith, and I remember someone  
 talking about
 that at DConf recently.

Rely on what faith?

void foo(int* p) {} // p must never be null void foo(NotNull!(int*) p) {} One of these is tested at compile time, *and* includes valuable documentation in the signature. The other is either less performant or buggy.
 Now that what other languages does is cleared, let's do some  
 consideration on null.

 A pointer point on something. For instance, an int* point on an  
 integer. null
 doesn't point on a integer. Non nullable pointer aren't a restricted  
 set of
 values, as, by definition, null isn't a value that point to an int.  
 That doesn't
 stand either.

By definition? Pointer semantics are what we choose it to mean.

Of course. But which definition is saner: "T* is either a valid pointer to a T, or a value that blows up when used in certain ways (but not others)." or "T* is a valid pointer to T." Of course, the latter also requires something like Maybe!T: "Maybe!T is either a valid pointer to a T, or a value on which no operations may be performed. In order to gain access to the T, both cases have to be handled." -- Simen
May 19 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Sun, 19 May 2013 02:32:49 +0200, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Sunday, May 19, 2013 02:22:43 Simen Kjaeraas wrote:
 Or... possibly, the current holes in  disable are fixed, and NonNull!T
 becomes the default, because we tell people to always use them, rather
 than flail our arms and behave like idiots. ("regular pointers are
 broken, use NonNull!T" is a pretty good argument if it's true)

I've never understood why so many people feel that nullable pointers are a problem. Clearly, many people do, but personally, I've rarely had problems with them, and there are plenty of cases where not being to make a pointer null would really suck (which is why we're forced to have std.typecons.Nullable for non-reference types). I'm not arguing against having non-nullable pointers, but I'd probably almost never use them myself, as I really don't think that they'd be buying me much. In my experince, problems with null pointers are extremely rare and easily caught.

My experience is the complete opposite - I think maybe 20% of bugs at my job are caused by null references. But as you say, they are very easily fixed. That said, two things to consider: How many of the functions you write actually need to accept nullable pointers/references? If non-nullable was the default, how often would you explicitly ask for a nullable pointer/reference? -- Simen
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 12:06:01 UTC, Simen Kjaeraas wrote:
 My experience is the complete opposite - I think maybe 20% of 
 bugs at my job
 are caused by null references. But as you say, they are very 
 easily fixed.

Sometime they are super freaking hard. I have a horror story debugging Apache Cayenne : http://cayenne.apache.org/ because it was throwing NPE on a race condition that would never show up once the code is instrumented. A reference was null for a short moment and then set to something. Due to concurrency, in extreme cases it could be seen as null where it was assumed everywhere to be set.
 That said, two things to consider:

 How many of the functions you write actually need to accept 
 nullable
 pointers/references?

 If non-nullable was the default, how often would you explicitly 
 ask for a
 nullable pointer/reference?

Not that much and I'd rather be warned when it is the case.
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 13:08:53 UTC, Andrei Alexandrescu wrote:
 Sounds like a race problem unrelated to null. With non-null 
 objects the race would have manifested itself in a different 
 way, perhaps even more pernicious.

It is both a race condition and a null problem. And having non nullable type would have been a compile time error instead of days of debugging.
May 19 2013
prev sibling next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote:
 On 5/19/13 9:11 AM, deadalnix wrote:
 On Sunday, 19 May 2013 at 13:08:53 UTC, Andrei Alexandrescu 
 wrote:
 Sounds like a race problem unrelated to null. With non-null 
 objects
 the race would have manifested itself in a different way, 
 perhaps even
 more pernicious.

It is both a race condition and a null problem.

No, it's just a race condition.
 And having non nullable
 type would have been a compile time error instead of days of 
 debugging.

No, the race condition would have stayed. Andrei

I believe this claim requires an explanation: It's a good practice to initialize references(and all other types of variables) as soon as possible - and if possible, right away in the declaration. If that reference started as null, it's safe to assume it was not possible to initialized it at declaration, so it was intentionally initialized with null(if there was no initialization Java would scream at you). Now, let's assume that reference was non-nullable. It is safe to assume that this change would not remove the obstacle that prevented that reference from being initialized right away in the declaration - so you still need to initialize it to something else - let's call that something `Nil`. Nil is an object that tells you that the reference has not yet been initialized. So, in the original bug the reference "could be seen as null where it was assumed everywhere to be set.". But now we don't have null - so that piece of code that thought the reference is null would now think that it is... what? The initialization value? No! Because we didn't switch from initializing the reference with null to initializing it with the later initialization value - we couldn't do it. Instead, we had to use Nil. So now, the reference 'could be seen as Nil where it was assumed everywhere to be set'... Now, if you are lucky and your Nil is the better-kind-of-null that is used in dynamic object oriented languages, you'll get a nil exception - which is just as good as null exception. But if you are not so lucky, and you had to declare Nil as a blank object of the type of that reference, you are going to have logical bug, which is far worse than exceptions...
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote:
 On 5/19/13 9:11 AM, deadalnix wrote:
 It is both a race condition and a null problem.

No, it's just a race condition.
 And having non nullable
 type would have been a compile time error instead of days of 
 debugging.

No, the race condition would have stayed.

That is ridiculous. non nullable would have made the bug non existent, and even without race condition the problem would exists. a reference is null, it container shared, then set to something else. You can put barriers all over the place to make that sequentially consistent that it wouldn't change anything and the bug would still arise. You also never provided any convincing solution to the safety hole. We can't even add check only on some edges cases as D also have values types. The only solution we are left with that is really safe is to null check every dereference or give up on safe. I encourage you to look at this : http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare Most new languages removed nullable by default, or limited its uses (scala for instance, allow for null for limited scope). I once again want to get attention on the fact that GC change everything in regard to reference, and that the C++ situation is a bad example. Idan Arye > Nil proposal make no sens in a statically typed language. And you'll find no better kind of null. We have all tools we need in D to work around null in library.
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 04:57:15 UTC, Walter Bright wrote:
 On 5/18/2013 8:54 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 01:20:31 UTC, Walter Bright wrote:
 I understand that. But the rationale you gave for having a 
 default constructor
 was to be able to disable default construction.

RAII or construction based on template parameters.

I know what default constructors are used for in C++. That wasn't what I asked, though. I asked for compelling rationale.

I have bunch of code that goes like : auto oldVar = var; scope(exit) var = oldVar; This is begging for a RAII solution where I pass var as template parameter but would require default constructor. This is an actual problem I have right now as all save/restore are harder and harder to keep in sync for no reason and generate a lot of boilerplate. This is a problem I have right now that default constructor would solve, and this isn't the first time I hit that need.
May 19 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Sun, 19 May 2013 19:12:15 +0200, Idan Arye <GenericNPC gmail.com> wrote:

 It's a good practice to initialize references(and all other types of  
 variables) as soon as possible - and if possible, right away in the  
 declaration. If that reference started as null, it's safe to assume it  
 was not possible to initialized it at declaration, so it was  
 intentionally initialized with null(if there was no initialization Java  
 would scream at you).

 Now, let's assume that reference was non-nullable. It is safe to assume  
 that this change would not remove the obstacle that prevented that  
 reference from being initialized right away in the declaration - so you  
 still need to initialize it to something else - let's call that  
 something `Nil`. Nil is an object that tells you that the reference has  
 not yet been initialized.

Uhm, no. Nononono. No. This is a complete and utter fallacy. What you have just done is define Nullable!(NonNullable!T). I should not have to explain too closely why this is a bad thing and should not be done. -- Simen
May 19 2013
prev sibling next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Sunday, 19 May 2013 at 17:35:43 UTC, deadalnix wrote:
 Idan Arye > Nil proposal make no sens in a statically typed 
 language. And you'll find no better kind of null. We have all 
 tools we need in D to work around null in library.

That's not the point. The point is that if you couldn't initialize this reference to null, you had to initialize it to something else, so you would still get your race condition.
May 19 2013
prev sibling next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Sunday, 19 May 2013 at 17:46:13 UTC, Simen Kjaeraas wrote:
 On Sun, 19 May 2013 19:12:15 +0200, Idan Arye 
 <GenericNPC gmail.com> wrote:

 It's a good practice to initialize references(and all other 
 types of variables) as soon as possible - and if possible, 
 right away in the declaration. If that reference started as 
 null, it's safe to assume it was not possible to initialized 
 it at declaration, so it was intentionally initialized with 
 null(if there was no initialization Java would scream at you).

 Now, let's assume that reference was non-nullable. It is safe 
 to assume that this change would not remove the obstacle that 
 prevented that reference from being initialized right away in 
 the declaration - so you still need to initialize it to 
 something else - let's call that something `Nil`. Nil is an 
 object that tells you that the reference has not yet been 
 initialized.

Uhm, no. Nononono. No. This is a complete and utter fallacy. What you have just done is define Nullable!(NonNullable!T). I should not have to explain too closely why this is a bad thing and should not be done.

These are the assumptions I'm working with: - We can't use a nullable reference - We can't initialize the reference upon declaration to it's real value. The first assumption is required because we want to describe how the bug scenario deadalnix brought up would look like if references were non-nullable. The second assumption is required because if we could initialize the reference upon declaration to it's real value, we should have just done it in the first place and avoid the whole race hazard. Now, I'm not saying the solution I presented is good - I'm trying to show that given those two assumptions, we are forced to use this bad solution.
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 17:51:36 UTC, Idan Arye wrote:
 On Sunday, 19 May 2013 at 17:35:43 UTC, deadalnix wrote:
 Idan Arye > Nil proposal make no sens in a statically typed 
 language. And you'll find no better kind of null. We have all 
 tools we need in D to work around null in library.

That's not the point. The point is that if you couldn't initialize this reference to null, you had to initialize it to something else, so you would still get your race condition.

Note that it had to be initialized, and my patch to Cayenne was simply to reorder that initialization. The patch itself was ~5lines, and would have impossible to do with a nonnullable (except with your Nil stuff, which once again don't make any sens in a strongly type language)
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 18:05:03 UTC, Idan Arye wrote:
 These are the assumptions I'm working with:
  - We can't use a nullable reference
  - We can't initialize the reference upon declaration to it's 
 real value.

If you can't initialize the value, you got to assume when you use it that it may not have been initialized and handle that case. You need either an Option (where you have to be explicit about what you do when the thing is null) or a Maybe (where null is ignored and Maybe "contaminate" every result depending on a maybe value).
 The first assumption is required because we want to describe 
 how the bug scenario deadalnix brought up would look like if 
 references were non-nullable. The second assumption is required 
 because if we could initialize the reference upon declaration 
 to it's real value, we should have just done it in the first 
 place and avoid the whole race hazard.

But that is the whole point ! The damn thing should have been initialized in the first place to avoid the bug. And this should have been caught at compile time with any sane type system. And this is the exact problem with nullable by default : plenty of stuff ends up be null is some weird situation that almost never occurs when they are assumed not to be and the program crashes. NullPointerException now return 4 millions result on google, which is probably around once per java developers.
 Now, I'm not saying the solution I presented is good - I'm 
 trying to show that given those two assumptions, we are forced 
 to use this bad solution.

This solution is complex, do not make any sense in a strongly typed language and don't even solve the presented case.
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 18:23:22 UTC, Walter Bright wrote:
 On 5/19/2013 5:02 AM, Simen Kjaeraas wrote:
 By definition? Pointer semantics are what we choose it to 
 mean.


For many types, it is extremely useful to have some sort of "invalid" value for it. null fills that role nicely for pointers, just as nan does for floating point types, and 0xFF does for UTF-8.

I don't wanted to bring that up because I thought it would confuse people, but yes, 0xFF for char is the exact same problem and I argue in the same direction : require explicit initialization.
 There's not anything insane about it. The Nullable type 
 constructor even exists in order to provide such an invalid 
 state for types (like int) which normally do not have one.

If something can be null, you MUST do something to handle specifically the null case. D completely fail to ensure that. void buzz(Foo f) { f.foo(); // Rely in faith. It is invalid and way easier to write than the valid code, which is THE recipe for it to spread. }
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 18:27:08 UTC, Walter Bright wrote:
 oldVar isn't being default constructed in your example, nor can 
 I see why you'd need a default constructor in order to use RAII 
 for save/restore.

I need to save the value at construction and restore at destruction. I don't need any runtime parameter at construction.
May 19 2013
prev sibling next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Sunday, 19 May 2013 at 18:22:16 UTC, deadalnix wrote:
 On Sunday, 19 May 2013 at 18:05:03 UTC, Idan Arye wrote:
 These are the assumptions I'm working with:
 - We can't use a nullable reference
 - We can't initialize the reference upon declaration to it's 
 real value.

If you can't initialize the value, you got to assume when you use it that it may not have been initialized and handle that case. You need either an Option (where you have to be explicit about what you do when the thing is null) or a Maybe (where null is ignored and Maybe "contaminate" every result depending on a maybe value).

I don't see how Option and Maybe would have helped your bug. The problem was that somewhere in the code the reference was perceived as null while in fact it wasn't - so now it will be perceived as `None`, and you will have the same problem.
 The first assumption is required because we want to describe 
 how the bug scenario deadalnix brought up would look like if 
 references were non-nullable. The second assumption is 
 required because if we could initialize the reference upon 
 declaration to it's real value, we should have just done it in 
 the first place and avoid the whole race hazard.

But that is the whole point ! The damn thing should have been initialized in the first place to avoid the bug. And this should have been caught at compile time with any sane type system. And this is the exact problem with nullable by default : plenty of stuff ends up be null is some weird situation that almost never occurs when they are assumed not to be and the program crashes. NullPointerException now return 4 millions result on google, which is probably around once per java developers.

This is not a problem with "nullable by default" - it is a problem with implicit default values. null(or Nil, or None) are the only sane default values for reference types - I think you would agree that having to construct a new blank object as default value for every reference variable would be far worse than null...
 Now, I'm not saying the solution I presented is good - I'm 
 trying to show that given those two assumptions, we are forced 
 to use this bad solution.

This solution is complex, do not make any sense in a strongly typed language and don't even solve the presented case.

It does not solve the bug - it is something you HAVE to do given the assumptions. If the reference is not nullable, and you can't set it to it's real value until later in the code, then you have to initialize it to some temporary value.
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 18:46:31 UTC, Walter Bright wrote:
 You also never provided any convincing solution to the safety 
 hole. We can't
 even add check only on some edges cases as D also have values 
 types. The only
 solution we are left with that is really safe is to null check 
 every dereference
 or give up on  safe.

Please don't make us guess what exactly you mean by this.

This isn't new and I discussed that again and again. When you dereference null, you hit the first plage, which is protected on most systems. But if you access an element with sufficient offset you bypass all protections provided by the type system and you are back in unsafe world. And no, putting nullcheck on access of field of sufficient offset (as propose dby Andrei) isn't enough because we have value types. Consider : S[BIG_NUMBER]* a; auto s = &(*a[SLIGHTLY_BELLOW_CHECK_OFFSET]); s.fieldAccess; // May not have enough offset to trigget null check, but still can be usnafe See bug reports : http://d.puremagic.com/issues/show_bug.cgi?id=3677 http://d.puremagic.com/issues/show_bug.cgi?id=5176
May 19 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 18 May 2013 at 20:39:29 UTC, Walter Bright wrote:
 On 5/18/2013 1:22 PM, deadalnix wrote:
 Many are, but I think that isn't the point we are discussing 
 here.

 Removing all holes in  disable this will require the same 
 sacrifices at the ends
 than default constructor would. For isntance, what should 
 happen in this case :

 S[] ss;
 ss.length = 42;

 if S has  disable this ?

Already reported: http://d.puremagic.com/issues/show_bug.cgi?id=10115

New case, will report it: struct S { int a; disable this(); this(int) { a = 1; } ~this() { assert(a !is 0); } alias a this; int opCall() { return a; } } void main() { switch (S.init()) { case 0: assert(0); //oops default: } } By the way, here is another bug. I think there is disagreement about disable reliability and usefulness and similar issues ( safe reliability too) due to different attitude to the problem: - As a language designer I care about whether some feature is claimed to solve some problem - and that all, I put it on a slide as lang advantage; - As a programmer who writes medium importance code I care whether the feature stops me from making bugs unintentionally. If it does, than I consider that the feature works. - As a programmer who writes critical code I care whether feature prevents from problem, even made deliberately, and if it doesn't, than the feature isn't reliable. It doesn't mean that it is totally useless, but it does mean that its reliability commitments are cheap. Since in system language there is plenty of ways to deliberately pass invalid data to the place where some validity assumptions were made, disable is a broken feature.
May 19 2013
prev sibling next sibling parent "Minas Mina" <minas_mina1990 hotmail.co.uk> writes:
On Sunday, 19 May 2013 at 18:30:09 UTC, deadalnix wrote:
 void buzz(Foo f) {
     f.foo(); // Rely in faith. It is invalid and way easier to 
 write than the valid code, which is THE recipe for it to spread.
 }

Shouldn't this throw a NullPointerSomething?
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 19:10:28 UTC, Andrei Alexandrescu wrote:
 No, your argument is ridiculous. You make a yarn with precious 
 little detail that describes for everything everyone knows a 
 textbook race condition, essentially ask that you are taking by 
 your word that non-null would miraculously solve it, and, to 
 add insult to injury, and when we don't buy it, you put the 
 burden of proof on us. This is quite a trick, my hat is off to 
 you.

I described a very usual null bug : something is set to null, then to a specific value. It is assumed not to be null. In a specific case it is null and everything explode. The concurrent context here made it especially hard to debug, but isn't the cause of the bug. Additionally, if you don't have enough information to understand what I'm saying, you are perfectly allowed to ask for additional details This isn't a shame.
 You also never provided any convincing solution to the safety 
 hole.

What's the safety hole? Objects of large static size?

Limiting object size isn't going to cut it. Or must be super restrictive (the protection is 4kb on some systems).
 We
 can't even add check only on some edges cases as D also have 
 values
 types. The only solution we are left with that is really safe 
 is to null
 check every dereference or give up on  safe.

How about using NonNull. We won't change the language at this point to make non-nullable references by default. Even you acknowledged that that's not practical. So now you contradict your own affirmation. What exactly do you sustain, and what are you asking for?

1/ NonNull do not work. 2/ It isn't because it is too late to solve a problem that it magically isn't a problem anymore.
 Most new languages removed nullable by default, or limited its 
 uses
 (scala for instance, allow for null for limited scope).

So what do you realistically think we should do, seeing that we're aiming at stability?

Acknowledge it was a mistake and move on. Use the analysis that need to be done to track down disable this issue to warn about uninitialized null stuffs.
 I once again want to get attention on the fact that GC change 
 everything
 in regard to reference, and that the C++ situation is a bad 
 example.

I don't understand this.

I C or C++ you are doomed to manage reference as you need to for memory management purpose. In garbage collected languages, you ends up with way more unmanaged references, because the GC take care of them. Doing so you multiply the surface area where null bug can strike.
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 19:15:47 UTC, Andrei Alexandrescu wrote:
 Oh, the good old "object of sufficient size". We know how to 
 fix that.

 And no, putting nullcheck on access of field of sufficient 
 offset (as
 propose dby Andrei) isn't enough because we have value types. 
 Consider :

 S[BIG_NUMBER]* a;
 auto s = &(*a[SLIGHTLY_BELLOW_CHECK_OFFSET]);
 s.fieldAccess; // May not have enough offset to trigget null 
 check, but
 still can be usnafe

 See bug reports :
 http://d.puremagic.com/issues/show_bug.cgi?id=3677
 http://d.puremagic.com/issues/show_bug.cgi?id=5176

All of the above are variations on the "sufficiently large object" theme. Andrei

The code above never access a field with a sufficient offset to trigger "sufficiently large runtime check". Obviously, in the presented code the bug is trivial, but if the dereferences occurs across several functions, this is doomed to fail. The solutions are : prevent any conglomerate of value type to be bigger than 4kb (the protection on OSX is 4kb) or put a null check on every dereference in safe code.
May 19 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Sunday, 19 May 2013 at 19:42:59 UTC, Walter Bright wrote:
 On 5/19/2013 12:31 PM, Minas Mina wrote:
 On Sunday, 19 May 2013 at 18:30:09 UTC, deadalnix wrote:
 void buzz(Foo f) {
    f.foo(); // Rely in faith. It is invalid and way easier to 
 write than the
 valid code, which is THE recipe for it to spread.
 }

Shouldn't this throw a NullPointerSomething?

It throws a seg fault at runtime. It *is* checked for by the hardware.

I think there is difference between catching exception and saving data which you have typed for some period and letting harware "check" the exception for you meanwile loosing your work.
May 19 2013
prev sibling next sibling parent "Mr. Anonymous" <mailnew4ster gmail.com> writes:
On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:
 Well you got to do what you got to do. Field accesses for 
 objects larger than 4KB would have to be checked in  safe code.

Isn't the solution as easy as doing: OR PTR:[address], 0 the same way it's done for the stack? The offset it known at compile time in most cases, so the command would be required only if both: * The object is larger than target OS' guard page size. * The position is greater than target OS' guard page size, OR is unknown at compile time.
May 19 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:
 You are blowing it out of proportion. Null references are 
 hardly even on the radar of the bug classes I'm encountering in 
 the style of programming of the three groups I worked in at 
 Facebook, and also my previous employers. People I meet at 
 conferences and consulting gigs never mention null references 
 as a real problem, although I very often ask about problems. I 
 find it difficult to agree with you just to be nice.

Just because people don't mention them as a problem doesn't mean it isn't a problem. For what it's worth, null pointers are a real problem in the code I work on (games). I don't know exactly what you work on, but I find that they are more of a problem in highly stateful, interactive applications. Things like generic libraries, utility programs, compilers, etc. probably won't see the same problems because they aren't very stateful or interactive. In my experience, null pointers are easy to fix, but the risk of them causes people to litter their code with if (ptr) tests, often with poor handling of the failure case, which can cause subtle bugs (no crash, but unintended code path). Just my 2c.
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:
 On 5/19/13 3:36 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 19:10:28 UTC, Andrei Alexandrescu 
 wrote:
 No, your argument is ridiculous. You make a yarn with 
 precious little
 detail that describes for everything everyone knows a 
 textbook race
 condition, essentially ask that you are taking by your word 
 that
 non-null would miraculously solve it, and, to add insult to 
 injury,
 and when we don't buy it, you put the burden of proof on us. 
 This is
 quite a trick, my hat is off to you.

I described a very usual null bug : something is set to null, then to a specific value. It is assumed not to be null. In a specific case it is null and everything explode. The concurrent context here made it especially hard to debug, but isn't the cause of the bug. Additionally, if you don't have enough information to understand what I'm saying, you are perfectly allowed to ask for additional details This isn't a shame.

Your argument has been destroyed so no need to ask details about it. Replace "null" with "invalid state" and it's the same race in any system. Let's move on.

"I don't want t to understand because I know I'm right. The fact you solved that issue and I didn't is irrelevant, I know better."
 You are blowing it out of proportion. Null references are 
 hardly even on the radar of the bug classes I'm encountering in 
 the style of programming of the three groups I worked in at 
 Facebook, and also my previous employers. People I meet at 
 conferences and consulting gigs never mention null references 
 as a real problem, although I very often ask about problems. I 
 find it difficult to agree with you just to be nice.

Hiphop type annotations are non null by default. Just saying.
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 21:02:57 UTC, Walter Bright wrote:
 On 5/19/2013 11:22 AM, deadalnix wrote:
 The damn thing should have been initialized in the
 first place to avoid the bug.

Sounds like you have the double-checked locking bug. Using a different value to initialize it won't fix it.

No it sound like initalizing something to null, then initialize it properly, assume all over the place that it is initialized to something else, and in some rare code path it blows up. The fact that this occurs in a multithreaded environment made it super hard to debug, but the whole thing was properly synchronized. Don't assume that I do not understand what the problem with double check locking is : http://d.puremagic.com/issues/show_bug.cgi?id=6607
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 20:30:28 UTC, Peter Alexander wrote:
 On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu 
 wrote:
 You are blowing it out of proportion. Null references are 
 hardly even on the radar of the bug classes I'm encountering 
 in the style of programming of the three groups I worked in at 
 Facebook, and also my previous employers. People I meet at 
 conferences and consulting gigs never mention null references 
 as a real problem, although I very often ask about problems. I 
 find it difficult to agree with you just to be nice.

Just because people don't mention them as a problem doesn't mean it isn't a problem. For what it's worth, null pointers are a real problem in the code I work on (games). I don't know exactly what you work on, but I find that they are more of a problem in highly stateful, interactive applications. Things like generic libraries, utility programs, compilers, etc. probably won't see the same problems because they aren't very stateful or interactive. In my experience, null pointers are easy to fix, but the risk of them causes people to litter their code with if (ptr) tests, often with poor handling of the failure case, which can cause subtle bugs (no crash, but unintended code path). Just my 2c.

Exactly ! Most of time null issue are easy to solve and is just the kind of crap that slow you down, and once in a while this ends up being an horrible mess. It is also true that rare path are often badly tested (and sometime you have no clue if the thing can be null or not, so you don't even know how to test it). Same argument Walter like to make about very rare failure cases apply here.
May 19 2013
prev sibling next sibling parent "w0rp" <devw0rp gmail.com> writes:
On Sunday, 19 May 2013 at 21:36:17 UTC, Andrei Alexandrescu wrote:
 On 5/19/13 4:30 PM, Peter Alexander wrot
 OK, this is sensible. One question - would you be willing to 
 type symbols as NullType!T instead of T to avoid these issues?

 Thanks,

 Andrei

Trying to come up with some once-and-for-all safe way to deal with null in languages which allow null references by default is something I wonder about now and then. This is due to my desire to adopt nice patterns for making my time spent working with a language safer and easier. Option/Maybe I think only really works when the language has references as non-null by default. Otherwise, you're writing verbose code for Option/Maybe and lies the rest of the time. (I'm looking at Scala.) For D, I decided that my way to do it, to be applied almost all of the time, is to write contracts. void func(Klass value) in { assert(value !is null); } body { } This stops null from being passed through too many functions, so that alleviates a lot of problems. I think contract programming is a good solution for this, and it applies more generally to other kinds of invalid values. This can include values other than the default values.
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote:
 How was there a bug if everything was properly synchronized? 
 You either describe the matter with sufficient detail, or 
 acknowledge the destruction of your anecdote. This is going 
 nowhere.

I explained over and over. A field is initialized to null, while the object lock is owned, and later to its value, while it is locked. In the meantime, another thread access the object, owning the lock, assuming the field is always initialized. The exact same problem arise quite often in the single threaded world : a reference is set to null, the dev try to be clever when initializing it, in a rare case it isn't, and everything blows up when it occurs. It is simply easier to reproduce when things are single threaded, and are often quickly debugged in that case. As explained, the multithreaded environment makes it super hard to debug, not the primary cause of the issue. The simply consistent in moving the initialization where it was set to null in the first place. It is an instance of the very classic something may be null and code use it assuming it is never null.
May 19 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, May 19, 2013 09:08:53 Andrei Alexandrescu wrote:
 Anyhow, this discussion should have finality. We could go on forever
 arguing the usefulness or lack thereof of non-nullable references. They
 didn't catch up in some languages and did in some. My personal opinion
 is "nice to have, but not greatly compelling".
 
 It's reasonable to assume no major language changes will accommodate
 non-null references, so the next best thing would be to focus on a
 library solution. As Walter said, a library solution has the perk of
 allowing other interesting restricted types.

Wow, this thread really expanded since I looked at it last night. Yeah, I thought that it was clear that NonNullable or NotNull or whatever we want to call it was going to go in std.typecons and that that was going to be our solution to non-nullable references. This is one of those discussions that I should probably just stay out of, since it never seems to go anywhere useful. We've already decided on a solution. We just haven't gotten it into the standard library yet. - Jonathan M Davis
May 19 2013
prev sibling next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Sunday, 19 May 2013 at 23:07:00 UTC, deadalnix wrote:
 On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu 
 wrote:
 How was there a bug if everything was properly synchronized? 
 You either describe the matter with sufficient detail, or 
 acknowledge the destruction of your anecdote. This is going 
 nowhere.

I explained over and over. A field is initialized to null, while the object lock is owned, and later to its value, while it is locked. In the meantime, another thread access the object, owning the lock, assuming the field is always initialized. The exact same problem arise quite often in the single threaded world : a reference is set to null, the dev try to be clever when initializing it, in a rare case it isn't, and everything blows up when it occurs. It is simply easier to reproduce when things are single threaded, and are often quickly debugged in that case. As explained, the multithreaded environment makes it super hard to debug, not the primary cause of the issue. The simply consistent in moving the initialization where it was set to null in the first place. It is an instance of the very classic something may be null and code use it assuming it is never null.

So this is not a problem of nullableness - rather this is a problem of mutability.
May 19 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Sunday, 19 May 2013 at 20:45:39 UTC, Walter Bright wrote:
 On 5/19/2013 1:03 PM, Maxim Fomin wrote:
 I think there is difference between catching exception and 
 saving
 data which you have typed for some period and letting harware
 "check" the exception for you meanwile loosing your work.

You can catch seg faults. It's easier on Windows, but it's doable on Linux.

What's the rational for not doing this by default in D? Wouldn't a MemoryAccessError or similar be better than crashing out with SIGSEGV ? I have no idea about the consequences of this (other than tempting people to catch a segfault when they shouldn't, which is pretty much always).
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 19 May 2013 at 23:29:53 UTC, Walter Bright wrote:
 On 5/19/2013 4:06 PM, deadalnix wrote:
 On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu 
 wrote:
 How was there a bug if everything was properly synchronized? 
 You either
 describe the matter with sufficient detail, or acknowledge 
 the destruction of
 your anecdote. This is going nowhere.

I explained over and over. A field is initialized to null, while the object lock is owned, and later to its value, while it is locked. In the meantime, another thread access the object, owning the lock, assuming the field is always initialized.

so, you have: ========================== Thread A Thread B lock p = null

Here p = null is implicit, this is part of the fun. The initialisation is still properly synchronized.
 unlock
              lock
                 *p = ...

It was in java, so more somethign like p.foo(); But yes.
              unlock
 lock
    p = new ...
 unlock
 ==========================
 Although you are using locks, it still isn't properly 
 synchronized. Changing the p=null to p=someothernonnullvalue 
 will not fix it.

No race condition exists in that program. The error lie in improper initialization of p in the first place, which should never has been null. The example looks dumb as this, you have to imagine the pattern hidden in thousands of LOC. The code is bugguy, but you'll find no undefined threading effect? What happen is perfectly defined here and no thread access shared data without owning the lock.
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 20 May 2013 at 00:23:59 UTC, John Colvin wrote:
 On Sunday, 19 May 2013 at 20:45:39 UTC, Walter Bright wrote:
 On 5/19/2013 1:03 PM, Maxim Fomin wrote:
 I think there is difference between catching exception and 
 saving
 data which you have typed for some period and letting harware
 "check" the exception for you meanwile loosing your work.

You can catch seg faults. It's easier on Windows, but it's doable on Linux.

What's the rational for not doing this by default in D? Wouldn't a MemoryAccessError or similar be better than crashing out with SIGSEGV ? I have no idea about the consequences of this (other than tempting people to catch a segfault when they shouldn't, which is pretty much always).

https://github.com/D-Programming-Language/druntime/blob/master/src/etc/linux/memoryerror.d Still you can have weird effect when the code above throw in a middle of a C routine or something. As C don't know about D exception, it is definitively something to be aware of.
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 20 May 2013 at 00:09:23 UTC, Walter Bright wrote:
 On 5/19/2013 3:04 PM, deadalnix wrote:
 Same argument Walter like to make about very rare failure 
 cases apply here.

1. rare as in programmers rarely create such a bug 2. rare as in being rare for an existing bug to show itself I was referring to (1), while you are referring to (2).

When you talk about UNIX utilities not handling properly a full filesystem for instance, you are referring to 1.
May 19 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 19 May 2013 16:03:24 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 5/19/13 3:36 PM, deadalnix wrote:
 I described a very usual null bug : something is set to null, then to a
 specific value. It is assumed not to be null. In a specific case it is
 null and everything explode.

 The concurrent context here made it especially hard to debug, but isn't
 the cause of the bug.

 Additionally, if you don't have enough information to understand what
 I'm saying, you are perfectly allowed to ask for additional details This
 isn't a shame.

Your argument has been destroyed so no need to ask details about it. Replace "null" with "invalid state" and it's the same race in any system. Let's move on.

I just wanted to chime in with this understanding of the bug that I am reading from deadalnix's descriptions: SomeObj obj; shareTheObj(&obj); // goes to other threads obj = new SomeObj; // initialize obj This is likely simpler than the actual problem, but I think this is the gist of it. A Non-Nullable solution WOULD solve the race: SomeObj obj; // Compiler: nope, can't do that, must initialize it. Now, with an "invalid state" not available like null, is the developer more likely to go through the trouble of building an invalid state for obj, in order to keep the racy behavior? No. He's just going to move the initialization: SomeObj obj = new SomeObj; shareTheObj(&obj); In essence the point of the anecdote is that a Non-Nullable reference would have PROMOTED avoiding the race condition -- it would have been harder to keep the racy behavior. I'm not saying that I think we need NN references as a compiler-supported type, or that it needs to be the default, or that NN references ALWAYS solve race conditions. I'm just pointing out what I see is an obvious misinterpretation of the underlying story. -Steve
May 19 2013
prev sibling next sibling parent Kenji Hara <k.hara.pg gmail.com> writes:
--001a11c25e4c50d17604dd1bc83f
Content-Type: text/plain; charset=UTF-8

Unfortunately this is currently not a bug.
T.init provides "default initialized" object image, and it *does not*
provide "default constructed" object. The difference is important.

That is already documented in lanugage reference.
http://dlang.org/property#init

 Note: .init produces a default initialized object, not default

 1. If T is a nested struct, the context pointer in T.init is null.
 2. If T is a struct which has  disable this();, T.init might return a

Kenji Hara 2013/5/20 Maxim Fomin <maxim maxim-fomin.ru>
 On Saturday, 18 May 2013 at 20:39:29 UTC, Walter Bright wrote:

 On 5/18/2013 1:22 PM, deadalnix wrote:

 Many are, but I think that isn't the point we are discussing here.

 Removing all holes in  disable this will require the same sacrifices at
 the ends
 than default constructor would. For isntance, what should happen in this
 case :

 S[] ss;
 ss.length = 42;

 if S has  disable this ?

Already reported: http://d.puremagic.com/issues/**show_bug.cgi?id=10115<http://d.puremagic.com/issues/show_bug.cgi?id=10115>

New case, will report it: struct S { int a; disable this(); this(int) { a = 1; } ~this() { assert(a !is 0); } alias a this; int opCall() { return a; } } void main() { switch (S.init()) { case 0: assert(0); //oops default: } } By the way, here is another bug. I think there is disagreement about disable reliability and usefulness and similar issues ( safe reliability too) due to different attitude to the problem: - As a language designer I care about whether some feature is claimed to solve some problem - and that all, I put it on a slide as lang advantage; - As a programmer who writes medium importance code I care whether the feature stops me from making bugs unintentionally. If it does, than I consider that the feature works. - As a programmer who writes critical code I care whether feature prevents from problem, even made deliberately, and if it doesn't, than the feature isn't reliable. It doesn't mean that it is totally useless, but it does mean that its reliability commitments are cheap. Since in system language there is plenty of ways to deliberately pass invalid data to the place where some validity assumptions were made, disable is a broken feature.

--001a11c25e4c50d17604dd1bc83f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra">Unfortunately this is currently= not a bug.</div><div class=3D"gmail_extra">T.init provides &quot;default i= nitialized&quot; object image, and it *does not* provide &quot;default cons= tructed&quot; object. The difference is important.<br> </div><div class=3D"gmail_extra"><div class=3D"gmail_extra"><br></div><div = class=3D"gmail_extra">That is already documented in lanugage reference.</di= v><div class=3D"gmail_extra"><a href=3D"http://dlang.org/property#init">htt= p://dlang.org/property#init</a></div> <div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">&gt; Note: = .init produces a default initialized object, not default constructed. That = means using .init is sometimes incorrect.</div><div class=3D"gmail_extra">&= gt; 1. If T is a nested struct, the context pointer in T.init is null.</div=

();, T.init might return a logically incorrect object.</div><div class=3D"g= mail_extra"><br></div><div class=3D"gmail_extra">Kenji Hara</div></div><div= class=3D"gmail_extra"> <br><div class=3D"gmail_quote">2013/5/20 Maxim Fomin <span dir=3D"ltr">&lt;= <a href=3D"mailto:maxim maxim-fomin.ru" target=3D"_blank">maxim maxim-fomin= .ru</a>&gt;</span><br><blockquote class=3D"gmail_quote" style=3D"margin-top= :0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width= :1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-lef= t:1ex"> <div class=3D""><div class=3D"h5">On Saturday, 18 May 2013 at 20:39:29 UTC,= Walter Bright wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin-top:0px;margin-right:0px;= margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color= :rgb(204,204,204);border-left-style:solid;padding-left:1ex"> On 5/18/2013 1:22 PM, deadalnix wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin-top:0px;margin-right:0px;= margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color= :rgb(204,204,204);border-left-style:solid;padding-left:1ex"> Many are, but I think that isn&#39;t the point we are discussing here.<br> <br> Removing all holes in disable this will require the same sacrifices at the= ends<br> than default constructor would. For isntance, what should happen in this ca= se :<br> <br> S[] ss;<br> ss.length =3D 42;<br> <br> if S has disable this ?<br> </blockquote> <br> Already reported:<br> <br> <a href=3D"http://d.puremagic.com/issues/show_bug.cgi?id=3D10115" target=3D= "_blank">http://d.puremagic.com/issues/<u></u>show_bug.cgi?id=3D10115</a><b= r> </blockquote> <br></div></div> New case, will report it:<div class=3D"im"><br> <br> struct S<br> {<br> =C2=A0 =C2=A0int a;<br> =C2=A0 =C2=A0 disable this();<br> =C2=A0 =C2=A0this(int) { a =3D 1; }<br> =C2=A0 =C2=A0~this() { assert(a !is 0); }<br></div> =C2=A0 =C2=A0alias a this;<br> =C2=A0 =C2=A0int opCall() { return a; }<br> }<br> <br> void main()<br> {<br> =C2=A0 =C2=A0switch (S.init())<br> =C2=A0 =C2=A0{<br> =C2=A0 =C2=A0 =C2=A0 case 0:<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0assert(0); //oops<br> =C2=A0 =C2=A0 =C2=A0 default:<br> =C2=A0 =C2=A0}<br> }<br> <br> By the way, here is another bug.<br> <br> I think there is disagreement about disable reliability and usefulness and= similar issues ( safe reliability too) due to different attitude to =C2=A0= the problem:<br> - As a language designer I care about whether some feature is claimed to so= lve some problem - and that all, I put it on a slide as lang advantage;<br> - As a programmer who writes medium importance code I care whether the feat= ure stops me from making bugs unintentionally. If it does, than I consider = that the feature works.<br> - As a programmer who writes critical code I care whether feature prevents = from problem, even made deliberately, and if it doesn&#39;t, than the featu= re isn&#39;t reliable. It doesn&#39;t mean that it is totally useless, but = it does mean that its reliability commitments are cheap.<br> <br> Since in system language there is plenty of ways to deliberately pass inval= id data to the place where some validity assumptions were made, disable is= a broken feature.<br> </blockquote></div><br></div></div> --001a11c25e4c50d17604dd1bc83f--
May 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 20 May 2013 at 01:08:16 UTC, Walter Bright wrote:
 On 5/19/2013 5:28 PM, deadalnix wrote:
 The error lie in improper
 initialization of p in the first place, which should never has 
 been null. The
 example looks dumb as this, you have to imagine the pattern 
 hidden in thousands
 of LOC.

I would find a design that declared a variable in one place, then initialized it in another, while releasing the lock in between as a bad design pattern to begin with.

I cannot agree more. This is what made tracking the cause of the bug super hard.
 What other default initialized types could be there? What about 
 an int default initialized to 0, yet code in another thread 
 expects it to be some other value? I suspect there'd be a lot 
 more bugs in it than just null pointer initializations.

 It might be time to engineer a new pattern so you don't have to 
 inspect thousands of LOC to manually verify correctness.

I didn't programed Apache Cayenne, int he first place. But I had to patch it anyway.
May 19 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Monday, 20 May 2013 at 00:55:14 UTC, Kenji Hara wrote:
 Unfortunately this is currently not a bug.
 T.init provides "default initialized" object image, and it 
 *does not*
 provide "default constructed" object. The difference is 
 important.

 That is already documented in lanugage reference.
 http://dlang.org/property#init

 Note: .init produces a default initialized object, not default

 1. If T is a nested struct, the context pointer in T.init is 
 null.
 2. If T is a struct which has  disable this();, T.init might 
 return a

Kenji Hara

I think this should be fixed otherwise disable this() is compromised. What is rationale behind allowing .init?
May 19 2013
prev sibling next sibling parent Kenji Hara <k.hara.pg gmail.com> writes:
--001a11c2690c5a350204dd202fd3
Content-Type: text/plain; charset=UTF-8

I know at least two cases which T.init is commonly used.

1. Inside predicate template for type T.

template isSomething(T) {
    enum isSomething = is(typeof({
        //T t1;    // not good if T is nested struct, or has  disable this()
        //T t2 = void; auto x = t2;  // not good if T is non-mutable type
        T t = T.init;   // avoid default construct check
        ...use t...
    }));
}

2. Some library utilities that treats object state directly, e.g.
std.conv.emplace

Kenji Hara

2013/5/20 Maxim Fomin <maxim maxim-fomin.ru>

 On Monday, 20 May 2013 at 00:55:14 UTC, Kenji Hara wrote:

 Unfortunately this is currently not a bug.
 T.init provides "default initialized" object image, and it *does not*
 provide "default constructed" object. The difference is important.

 That is already documented in lanugage reference.
 http://dlang.org/property#init

  Note: .init produces a default initialized object, not default

 1. If T is a nested struct, the context pointer in T.init is null.
 2. If T is a struct which has  disable this();, T.init might return a

Kenji Hara

What is rationale behind allowing .init?

--001a11c2690c5a350204dd202fd3 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">I know at least two cases which T.init is commonly used.<d= iv><br></div><div>1. Inside predicate template for type T.</div><div><br></= div><div>template isSomething(T) {</div><div>=C2=A0 =C2=A0 enum isSomething= =3D is(typeof({</div> <div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 //T t1; =C2=A0 =C2=A0// not good if T is n= ested struct, or has disable this()</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 = //T t2 =3D void;=C2=A0auto x =3D t2; =C2=A0// not good if T is non-mutable = type</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 T t =3D T.init; =C2=A0 // avoid = default construct check</div> <div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 ...use t...</div><div>=C2=A0 =C2=A0 }));</= div><div><div class=3D"gmail_extra">}</div><div class=3D"gmail_extra"><br><= /div><div class=3D"gmail_extra">2. Some library utilities that treats objec= t state directly, e.g. std.conv.emplace<br> <div class=3D"gmail_quote"><br></div><div class=3D"gmail_quote">Kenji Hara<= /div><div class=3D"gmail_quote"><br></div><div class=3D"gmail_quote">2013/5= /20 Maxim Fomin <span dir=3D"ltr">&lt;<a href=3D"mailto:maxim maxim-fomin.r= u" target=3D"_blank">maxim maxim-fomin.ru</a>&gt;</span><br> <blockquote class=3D"gmail_quote" style=3D"margin-top:0px;margin-right:0px;= margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color= :rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class=3D"i= m"> On Monday, 20 May 2013 at 00:55:14 UTC, Kenji Hara wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin-top:0px;margin-right:0px;= margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color= :rgb(204,204,204);border-left-style:solid;padding-left:1ex"> Unfortunately this is currently not a bug.<br> T.init provides &quot;default initialized&quot; object image, and it *does = not*<br> provide &quot;default constructed&quot; object. The difference is important= .<br> <br> That is already documented in lanugage reference.<br> <a href=3D"http://dlang.org/property#init" target=3D"_blank">http://dlang.o= rg/property#init</a><br> <br> <blockquote class=3D"gmail_quote" style=3D"margin-top:0px;margin-right:0px;= margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color= :rgb(204,204,204);border-left-style:solid;padding-left:1ex"> Note: .init produces a default initialized object, not default<br> </blockquote> constructed. That means using .init is sometimes incorrect.<br> <blockquote class=3D"gmail_quote" style=3D"margin-top:0px;margin-right:0px;= margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color= :rgb(204,204,204);border-left-style:solid;padding-left:1ex"> 1. If T is a nested struct, the context pointer in T.init is null.<br> 2. If T is a struct which has disable this();, T.init might return a<br> </blockquote> logically incorrect object.<br> <br> Kenji Hara<br> <br> </blockquote> <br></div> I think this should be fixed otherwise disable this() is compromised. What= is rationale behind allowing .init?<br> <br> </blockquote></div><br></div></div></div> --001a11c2690c5a350204dd202fd3--
May 19 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Monday, 20 May 2013 at 06:10:22 UTC, Kenji Hara wrote:
 I know at least two cases which T.init is commonly used.

 1. Inside predicate template for type T.

 template isSomething(T) {
     enum isSomething = is(typeof({
         //T t1;    // not good if T is nested struct, or has 
  disable this()
         //T t2 = void; auto x = t2;  // not good if T is 
 non-mutable type
         T t = T.init;   // avoid default construct check
         ...use t...
     }));
 }

 2. Some library utilities that treats object state directly, 
 e.g.
 std.conv.emplace

 Kenji Hara

I see. But unfortunately this undermines disable and defeats arguments for using it. disable is another feature (like ref and safe) which cannot be fixed be design.
May 19 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Sunday, 19 May 2013 at 21:36:17 UTC, Andrei Alexandrescu wrote:
 OK, this is sensible. One question - would you be willing to 
 type symbols as NullType!T instead of T to avoid these issues?

Good question. Probably not. I think it's one of those things were the awkwardness of it not being the default would lead to lack of use (in the same way usage of pure suffers from not being default).
May 20 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Sun, 19 May 2013 21:02:11 +0200, Idan Arye <GenericNPC gmail.com> wrote:

 I don't see how Option and Maybe would have helped your bug. The problem  
 was that somewhere in the code the reference was perceived as null while  
 in fact it wasn't

What does that even mean?
 - so now it will be perceived as `None`, and you will have the same  
 problem.

Except that now the code would be forced to handle the None case. In a way, having nullable by default is like having a stringly typed system: function foo( s ) { return (s + 4) * 2; // Works great when s == "16", falls dead on its // back when s == "goobers". } function bar( int i ) { return (i + 4) * 2; } These functions look very different. That's because they are. One of them only takes valid parameters, the other takes any old garbage and barfs when the wrong garbage is given to it. Of course, if you have a string, and you want to call bar, you need to convert the string to an int. So you end up with this: function baz( string s ) { return s.parseInt( i => bar(i), { alert("error"); }); } Notice how the parseInt function takes two delegates? One of these (the first) is only called when the string is valid. The other is only called if the string is invalid. That way, we can be sure that the failure case is handled. Exactly the same would be the case for non-nullable pointers - if you want to convert a nullable pointer to non-nullable, you *have* to handle the failure case. No two ways about it. Now, the same example with class references: int foo(A a) { return a.qux(); // Works great when a == new A(), falls dead on its // back when a == null. } int bar(NonNull!A a) { return a.qux(); } See how one of these does not blow up in your face (unless you do something stupid like create a special Nil value that will do exactly that)? Now, for baz: int baz(A a) { return a.match( (NonNull!A a) => bar(a), (None) => -1 ); }
 And this is the exact problem with nullable by default : plenty of  
 stuff ends up be null is some weird situation that almost never occurs  
 when they are assumed not to be and the program crashes.  
 NullPointerException now return 4 millions result on google, which is  
 probably around once per java developers.

This is not a problem with "nullable by default" - it is a problem with implicit default values. null(or Nil, or None) are the only sane default values for reference types - I think you would agree that having to construct a new blank object as default value for every reference variable would be far worse than null...

If you absolutely cannot initialize the pointer to something sensible, then use a nullable pointer. But if non-nullable pointers are not available, or are harder to use than nullable pointer, then people will use nullable pointers even where non-nullable would have been a much more fitting choice.
 It does not solve the bug - it is something you HAVE to do given the  
 assumptions. If the reference is not nullable, and you can't set it to  
 it's real value until later in the code, then you have to initialize it  
 to some temporary value.

I don't know enough about the bug to say such things for sure, but I will say this: If deadalnix solved the bug, he is likely in a much better position to say anything about what would solve the problem than the rest of us. -- Simen
May 20 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Sun, 19 May 2013 20:05:02 +0200, Idan Arye <GenericNPC gmail.com> wrote:

 These are the assumptions I'm working with:
   - We can't use a nullable reference
   - We can't initialize the reference upon declaration to it's real  
 value.

Indeed, if that's the case, then what you're doing is fairly sensible. But if #2 is true, then #1 should never be true. -- Simen
May 20 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Sun, 19 May 2013 20:23:21 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 5/19/2013 5:02 AM, Simen Kjaeraas wrote:
 By definition? Pointer semantics are what we choose it to mean.


For many types, it is extremely useful to have some sort of "invalid" value for it. null fills that role nicely for pointers, just as nan does for floating point types, and 0xFF does for UTF-8. There's not anything insane about it. The Nullable type constructor even exists in order to provide such an invalid state for types (like int) which normally do not have one. Yes, I do understand there's a role for pointers which cannot hold the invalid value.

I contend that not only is there a role for them, but that most pointers should never be null. Here's two questions that convinced me: 1. How many functions that take a pointer or class reference make sense to call with null in that pointer/reference? 2. If pointers were non-nullable by default, how often would you need to reach for the nullable one? I argued in another post that nullable by default is analogous to using a string instead of an int - any number representable in an int is representable in a string, *and* the string can represent error states. But if you only want valid ints, there's no reason to use a string. -- Simen
May 20 2013
prev sibling next sibling parent "estew" <estewh gmail.com> writes:
I'm surprised people still have problems with null pointers. I 
for one am glad D has null by default makes life easy coming from 
C++ and Java.

I may have missed something but what happens with the following 
code if I could not have a null pointer?

int*[] pntrs = new int*[10];

Would I need to write something like?

Null!(int*)[] pntrs = new Null!(int*)[10];

Personally, I'd rather have null by default as I find it less 
noisy and I don't need it spelled out in the code, it is implied.

 No it sound like initalizing something to null, then 
 initialize it properly, assume all over the place that it is 
 initialized to something else, and in some rare code path it 
 blows up.


OK, so the D gurus kindly introduce for us NotNull!T, Maybe!T, Option!T and SegFault!T (just for me). Now I want to access a pointer, write code using it etc. But I need to manually track at development time whether it is NotNull!T, Null!T, Maybe!T, Option!T or whatever. I cannot just have a pointer anymore, knowing it's initialised to null. Now I realise it needs to change from NotNull!T to Maybe!T...great yet more refactoring. Ok refactoring done (yay sed!) but you know what, now I need to find every access to that pointer and check for null. More error prone than this: If you are in doubt (i.e. most multi-threaded apps) then check if null, with the added comfort that D has initialised all pointers to NULL for you. If still in doubt, don't use pointers. If you want non-null pointers (please no) then it is all or nothing. Allowing some pointers null and others not, via Nullable!T or NotNull!T, immediately adds another layer of complexity. I don't want to hear: D pointers cannot be null...well ok, they can sometimes, it depends, you'll have to read the code. But don't worry, D is very easy to read... My 1 cent. Disregard if I have totally misunderstood the thread, possible as it is very late! :-) Cheers, Stewart
May 20 2013
prev sibling next sibling parent "Dicebot" <m.strashun gmail.com> writes:
On Monday, 20 May 2013 at 14:49:32 UTC, estew wrote:
 Now I want to access a pointer, write code using it etc. But I 
 need to manually track at development time whether it is 
 NotNull!T, Null!T, Maybe!T, Option!T or whatever. I cannot just 
 have a pointer anymore, knowing it's initialised to null.

Yes and this is awesome. This is correctness enforced by type system. Because if you _don't_ track this, you have a possible error in your code. Only difference between nullable and non-nullable pointers by default is that latter _force_ you to write correct code. Which is good.
May 20 2013
prev sibling next sibling parent Byron Heads <byron.heads gmail.com> writes:
On Sun, 19 May 2013 17:36:17 -0400, Andrei Alexandrescu wrote:

 On 5/19/13 4:30 PM, Peter Alexander wrote:
 On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:
 You are blowing it out of proportion. Null references are hardly even
 on the radar of the bug classes I'm encountering in the style of
 programming of the three groups I worked in at Facebook, and also my
 previous employers. People I meet at conferences and consulting gigs
 never mention null references as a real problem, although I very often
 ask about problems. I find it difficult to agree with you just to be
 nice.

Just because people don't mention them as a problem doesn't mean it isn't a problem. For what it's worth, null pointers are a real problem in the code I work on (games). I don't know exactly what you work on, but I find that they are more of a problem in highly stateful, interactive applications. Things like generic libraries, utility programs, compilers, etc. probably won't see the same problems because they aren't very stateful or interactive. In my experience, null pointers are easy to fix, but the risk of them causes people to litter their code with if (ptr) tests, often with poor handling of the failure case, which can cause subtle bugs (no crash, but unintended code path). Just my 2c.

OK, this is sensible. One question - would you be willing to type symbols as NullType!T instead of T to avoid these issues? Thanks, Andrei

More boiler plate code for functions that take pointers. void foo(T)(T t) if(isPointer!T) { static if(isNullable!T) if(!t) throw .... } May also introduce then need to check for objects in the init state (default state) outside of NonNull in stdlib, an attribute maybe a possible answer. void bar( nonnull SomeClass o) { o = foo(); // if foo returns nonnull, then check is not needed, else needs a check added during assignment } there are a few compile time checks that can be done to prove o is not null, if not the compiler adds runtime checks. But really, checking for valid input is part of programming, might be better to have lint integration that can help find these types of problems
May 20 2013
prev sibling next sibling parent Byron Heads <byron.heads gmail.com> writes:
On Mon, 20 May 2013 11:43:35 -0400, Andrei Alexandrescu wrote:

 On 5/20/13 11:19 AM, Byron Heads wrote:
 On Sun, 19 May 2013 17:36:17 -0400, Andrei Alexandrescu wrote:
 OK, this is sensible. One question - would you be willing to type
 symbols as NullType!T instead of T to avoid these issues?

 Thanks,

 Andrei

More boiler plate code for functions that take pointers. void foo(T)(T t) if(isPointer!T) { static if(isNullable!T) if(!t) throw .... }

But this goes both ways. Regardless of the default, you'd sometimes need to distinguish between cases. You either hurt one half of your cases or the other half. In fact you are now showing the case that assumes non-nullable being the default.
 May also introduce then need to check for objects in the init state
 (default state)



 outside of NonNull in stdlib, an attribute maybe a possible answer.

 void bar( nonnull SomeClass o)
 {
 	o = foo(); // if foo returns  nonnull, then check is not needed,
 else needs a check added during assignment }

 there are a few compile time checks that can be done to prove o is not
 null, if not the compiler adds runtime checks.

I think a parameterized type is a better match for non-null because it attaches to the type, not the symbol.
 But really, checking for valid input is part of programming, might be
 better to have lint integration that can help find these types of
 problems

Agreed. Andrei

What about dealing with externs you want to protect? extern(C) void foo( nonnull int* x); other then that I think the library solution is fine nonnull Bar* b <=> NotNull!(NotNull!Bar) b
May 20 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Sun, 19 May 2013 05:17:57 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 5/18/2013 1:39 PM, Walter Bright wrote:
 Already reported:

 http://d.puremagic.com/issues/show_bug.cgi?id=10115

And Kenji has already posted a fix! What can I say, other than Awesome!

Great! One more: http://d.puremagic.com/issues/show_bug.cgi?id=1528 -- Simen
May 20 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Mon, 20 May 2013 18:05:18 +0200, Byron Heads <byron.heads gmail.com>  
wrote:

 What about dealing with externs you want to protect?

 extern(C) void foo( nonnull int* x);

There is nothing stopping you from declaring that with this signature: extern(C) void foo(NonNull!(int*) x); -- Simen
May 20 2013
prev sibling next sibling parent "Bottled Gin" <gin bottled.com> writes:
On Monday, 20 May 2013 at 00:55:14 UTC, Kenji Hara wrote:
 Unfortunately this is currently not a bug.
 T.init provides "default initialized" object image, and it 
 *does not*
 provide "default constructed" object. The difference is 
 important.

 That is already documented in lanugage reference.
 http://dlang.org/property#init

 Note: .init produces a default initialized object, not default

 1. If T is a nested struct, the context pointer in T.init is 
 null.
 2. If T is a struct which has  disable this();, T.init might 
 return a

Kenji Hara

In that case, kindly let me understand why it is not possible to allow explicit default constructor for structs given that: 1. The constructor gets called at run time. 2. Is not considered for evaluating S.init (compile time).
May 20 2013
prev sibling next sibling parent "estew" <estewh gmail.com> writes:
On Monday, 20 May 2013 at 14:57:17 UTC, Dicebot wrote:
 On Monday, 20 May 2013 at 14:49:32 UTC, estew wrote:
 Now I want to access a pointer, write code using it etc. But I 
 need to manually track at development time whether it is 
 NotNull!T, Null!T, Maybe!T, Option!T or whatever. I cannot 
 just have a pointer anymore, knowing it's initialised to null.

Yes and this is awesome. This is correctness enforced by type system. Because if you _don't_ track this, you have a possible error in your code. Only difference between nullable and non-nullable pointers by default is that latter _force_ you to write correct code. Which is good.

True I grant you that, it was late when I posted :-) I've actually come around a bit on this after sleeping on it and rereading some of the posts. I am starting to like NotNull!T idea but I'm a bit hesitant still with Maybe!T, Option!T. I cannot remember the last time our team had NULL pointer issues. We have 102 devs on four integrated products. Big enough to not know context your code might be used in nor the implementation details of all libraries. The only pointer troubles we see are forget to init to NULL or reset to NULL after freeing resources. All devs know raw pointers are initialised to NULL. We are C/C++. So non-null pointers wouldn't make much difference to us here, although it may make the code more readable which is always a good thing. D needs nullable pointers though, of some form. But I'm not convinced it would cost us less to have NotNull!T and Nullable!T. I feel it is cheaper to mindlessly write "if(a is null) {}" when using pointers than to worry at design time what the behaviour of a pointer should be. Design time is the second most expensive developer time for us. The most expensive dev. time is changing a design that turned out to be incorrect, or is now outdated for whatever reason. Moving pointer behaviour to be a design time issue rather than "knowing it could be NULL so check it" could increase the probability of redesign bugs creeping in. Still, I am loving the discussion in this thread it's very interesting from both sides. Stewart
May 20 2013
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 21 May 2013 at 01:34:29 UTC, estew wrote:
 But I'm not convinced it would cost us less to have NotNull!T 
 and Nullable!T. I feel it is cheaper to mindlessly write "if(a 
 is null) {}" when using pointers than to worry at design time 
 what the behaviour of a pointer should be.

As a matter of fact, most reference are never null, or are assumed never to be null. For instance, in DMD2.062, e2ir.c line 869 it is assumed that irs->sclosure can't be null, when in fact it can and that lead to an ICE (and that isn't the first one, which kind of mitigate the strength of the arguement that this rarely happens and is easy to fix).
 Design time is the second most expensive developer time for us. 
 The most expensive dev. time is changing a design that turned 
 out to be incorrect, or is now outdated for whatever reason. 
 Moving pointer behaviour to be a design time issue rather than 
 "knowing it could be NULL so check it" could increase the 
 probability of redesign bugs creeping in.

You may not know if a reference will be nullable or not when you write you code at first. With current model, you start writing code as if it can't be null, and then later, when you see you in fact need null, you now can have surprise breakage anywhere. With a Nullable, you'll have code breakage that force you to handle the null case. This enforce correctness instead of relying on faith.
May 24 2013