www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Is the other-kind-of-null really necessary in Nullable and Variant?

reply "Idan Arye" <GenericNPC gmail.com> writes:
When you use `std.typecons.Nullable` with a type that already 
accept `null` values, you get two types of nulls - the 
`Nullable`'s null state the the regular type's `null`:

     Nullable!string a;
     writeln(a.isNull()); //prints "true"
     a = null;
     writeln(a.isNull()); //prints "false"
     a.nullify();
     writeln(a.isNull()); //prints "true"

There is another version of  Nullable where instead of a boolean 
specifying if the value is null or not, you set a value to act as 
the null value. If we set that null value to D's `null`, we can 
get the previous example to work as expected:

     Nullable!(string, null) a;
     writeln(a.isNull()); //prints "true"
     a = null;
     writeln(a.isNull()); //prints "true"
     a.nullify();
     writeln(a.isNull()); //prints "true"

What I suggest is to check if the type passes to `Nullable` 
accepts `null`, and if so - alias it to the second template with 
`null` as it's null value.

As an added benefit, this will allow us to make null assignment 
to `Nullable` possible. The way Phobos is now it's tricky - if I 
write:
     Nullable!string a = null;
Should `a` be D's `null` or a nullified `Nullable`? This is too 
confusing, and a uniform null state would solve this conflict.


`std.variant.Variant` is more tricky:

     Variant a;
     Variant b = cast(string) null;
     Variant c = cast(Object) null;
     writeln(a == b); //prints "false"
     writeln(a == c); //prints "false"
     writeln(b == c); //prints "false"
     writeln(b.convertsTo!Object()); //prints "false"
     writeln(c.convertsTo!string()); //prints "false"
     writeln(a.convertsTo!string()); //throws VariantException

And even more surprising:

     writeln(c == c); //prints "false"

Although:

     writeln(a == a); //prints "true"
     writeln(b == b); //prints "true"

And even:

     Object o1=null;
     Object o2=null;
     writeln(o1 == o2); //prints "true"


Again, there is no reason to have all those different types of 
no-value - a null is a null is a null. a `Variant` should have a 
single null value, which is also the default value, and whenever 
it is assigned to `null` it should change back to that value 
without caring about the type. That also mean that a `Variant` 
storing a null value should implicitly convert to *any* type that 
accept `null`(when using `get`, `coerce` and `convertsTo`).


I can probably implement this myself, but I want to hear the 
community's opinion before I start hacking.
Apr 28 2013
next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Idan Arye:

 When you use `std.typecons.Nullable` with a type that already 
 accept `null` values, you get two types of nulls - the 
 `Nullable`'s null state the the regular type's `null`:

     Nullable!string a;
     writeln(a.isNull()); //prints "true"
     a = null;
     writeln(a.isNull()); //prints "false"

Originally D dynamic arrays were almost conflated with regular pointers (they were seen as fat pointers). This was recently partially fixed (so assigning a pointer to a dynamic array is now forbidden), but accepting "null" for their empty literal is one left part of that original suboptimal design. Time ago I have proposed to forbid "null" as literal for an empty dynamic array literal, an empty associative literal, or an empty string, and to accept only [] "" [:] (D already has the first two literals and the third looks natural). See also: http://d.puremagic.com/issues/show_bug.cgi?id=3889 http://d.puremagic.com/issues/show_bug.cgi?id=5788 http://d.puremagic.com/issues/show_bug.cgi?id=7227 Bye, bearophile
Apr 28 2013
prev sibling next sibling parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Sunday, 28 April 2013 at 16:33:19 UTC, Idan Arye wrote:
 I can probably implement this myself, but I want to hear the 
 community's opinion before I start hacking.

I would like to see the non-null null value to be removed. I believe there is an improved variant somewhere, which may or may not do something with this.
Apr 28 2013
prev sibling next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Sunday, 28 April 2013 at 17:02:57 UTC, bearophile wrote:
 Idan Arye:

 When you use `std.typecons.Nullable` with a type that already 
 accept `null` values, you get two types of nulls - the 
 `Nullable`'s null state the the regular type's `null`:

    Nullable!string a;
    writeln(a.isNull()); //prints "true"
    a = null;
    writeln(a.isNull()); //prints "false"

Originally D dynamic arrays were almost conflated with regular pointers (they were seen as fat pointers). This was recently partially fixed (so assigning a pointer to a dynamic array is now forbidden), but accepting "null" for their empty literal is one left part of that original suboptimal design. Time ago I have proposed to forbid "null" as literal for an empty dynamic array literal, an empty associative literal, or an empty string, and to accept only [] "" [:] (D already has the first two literals and the third looks natural). See also: http://d.puremagic.com/issues/show_bug.cgi?id=3889 http://d.puremagic.com/issues/show_bug.cgi?id=5788 http://d.puremagic.com/issues/show_bug.cgi?id=7227 Bye, bearophile

I used `Nullable!string` for the example, but it could just as easily be `Nullable!Object` or `Nullable!(int*)`.
Apr 28 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 28 April 2013 at 16:33:19 UTC, Idan Arye wrote:
 When you use `std.typecons.Nullable` with a type that already 
 accept `null` values, you get two types of nulls - the 
 `Nullable`'s null state the the regular type's `null`:

     Nullable!string a;
     writeln(a.isNull()); //prints "true"
     a = null;
     writeln(a.isNull()); //prints "false"
     a.nullify();
     writeln(a.isNull()); //prints "true"

All types should be non nullable. Problem solved.
Apr 29 2013
prev sibling next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Monday, 29 April 2013 at 12:23:04 UTC, deadalnix wrote:
 On Sunday, 28 April 2013 at 16:33:19 UTC, Idan Arye wrote:
 When you use `std.typecons.Nullable` with a type that already 
 accept `null` values, you get two types of nulls - the 
 `Nullable`'s null state the the regular type's `null`:

    Nullable!string a;
    writeln(a.isNull()); //prints "true"
    a = null;
    writeln(a.isNull()); //prints "false"
    a.nullify();
    writeln(a.isNull()); //prints "true"

All types should be non nullable. Problem solved.

*All* types? Even object references and pointers?
Apr 29 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On 2013-04-29, 17:34, Idan Arye wrote:

 On Monday, 29 April 2013 at 12:23:04 UTC, deadalnix wrote:
 On Sunday, 28 April 2013 at 16:33:19 UTC, Idan Arye wrote:
 When you use `std.typecons.Nullable` with a type that already accept  
 `null` values, you get two types of nulls - the `Nullable`'s null  
 state the the regular type's `null`:

    Nullable!string a;
    writeln(a.isNull()); //prints "true"
    a = null;
    writeln(a.isNull()); //prints "false"
    a.nullify();
    writeln(a.isNull()); //prints "true"

All types should be non nullable. Problem solved.

*All* types? Even object references and pointers?

That would be nice, yes. -- Simen
Apr 29 2013
prev sibling next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Monday, 29 April 2013 at 15:39:47 UTC, Simen Kjaeraas wrote:
 On 2013-04-29, 17:34, Idan Arye wrote:

 On Monday, 29 April 2013 at 12:23:04 UTC, deadalnix wrote:
 On Sunday, 28 April 2013 at 16:33:19 UTC, Idan Arye wrote:
 When you use `std.typecons.Nullable` with a type that 
 already accept `null` values, you get two types of nulls - 
 the `Nullable`'s null state the the regular type's `null`:

   Nullable!string a;
   writeln(a.isNull()); //prints "true"
   a = null;
   writeln(a.isNull()); //prints "false"
   a.nullify();
   writeln(a.isNull()); //prints "true"

All types should be non nullable. Problem solved.

*All* types? Even object references and pointers?

That would be nice, yes.

And what would they be initialized to? When you write: Object obj; what will `obj` refer to? Also, what about the C&C++ interface? Without null values, how can you use an extern function that accepts or returns pointers?
Apr 29 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 29 April 2013 at 15:34:30 UTC, Idan Arye wrote:
 On Monday, 29 April 2013 at 12:23:04 UTC, deadalnix wrote:
 On Sunday, 28 April 2013 at 16:33:19 UTC, Idan Arye wrote:
 When you use `std.typecons.Nullable` with a type that already 
 accept `null` values, you get two types of nulls - the 
 `Nullable`'s null state the the regular type's `null`:

   Nullable!string a;
   writeln(a.isNull()); //prints "true"
   a = null;
   writeln(a.isNull()); //prints "false"
   a.nullify();
   writeln(a.isNull()); //prints "true"

All types should be non nullable. Problem solved.

*All* types? Even object references and pointers?

Especially object references and pointers.
Apr 29 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 29 April 2013 at 16:02:11 UTC, Idan Arye wrote:
 On Monday, 29 April 2013 at 15:39:47 UTC, Simen Kjaeraas wrote:
 On 2013-04-29, 17:34, Idan Arye wrote:

 On Monday, 29 April 2013 at 12:23:04 UTC, deadalnix wrote:
 On Sunday, 28 April 2013 at 16:33:19 UTC, Idan Arye wrote:
 When you use `std.typecons.Nullable` with a type that 
 already accept `null` values, you get two types of nulls - 
 the `Nullable`'s null state the the regular type's `null`:

  Nullable!string a;
  writeln(a.isNull()); //prints "true"
  a = null;
  writeln(a.isNull()); //prints "false"
  a.nullify();
  writeln(a.isNull()); //prints "true"

All types should be non nullable. Problem solved.

*All* types? Even object references and pointers?

That would be nice, yes.

And what would they be initialized to? When you write: Object obj; what will `obj` refer to? Also, what about the C&C++ interface? Without null values, how can you use an extern function that accepts or returns pointers?

Data flow analysis can smash your face if you try to use that before initializing it. In fact, this is already done in many languages.
Apr 29 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On 2013-04-29, 18:02, Idan Arye wrote:

 On Monday, 29 April 2013 at 15:39:47 UTC, Simen Kjaeraas wrote:
 On 2013-04-29, 17:34, Idan Arye wrote:

 On Monday, 29 April 2013 at 12:23:04 UTC, deadalnix wrote:
 On Sunday, 28 April 2013 at 16:33:19 UTC, Idan Arye wrote:
 When you use `std.typecons.Nullable` with a type that already accept  
 `null` values, you get two types of nulls - the `Nullable`'s null  
 state the the regular type's `null`:

   Nullable!string a;
   writeln(a.isNull()); //prints "true"
   a = null;
   writeln(a.isNull()); //prints "false"
   a.nullify();
   writeln(a.isNull()); //prints "true"

All types should be non nullable. Problem solved.

*All* types? Even object references and pointers?

That would be nice, yes.

And what would they be initialized to? When you write: Object obj; what will `obj` refer to?

It won't. That would be a compile-time error: 'Variable obj needs an initializer'. We have some of this already in disable this(). However, a true non-nullable reference is, I believe, not possible in D today. -- Simen
Apr 29 2013
prev sibling next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Monday, 29 April 2013 at 16:14:02 UTC, deadalnix wrote:
 On Monday, 29 April 2013 at 16:02:11 UTC, Idan Arye wrote:
 On Monday, 29 April 2013 at 15:39:47 UTC, Simen Kjaeraas wrote:
 On 2013-04-29, 17:34, Idan Arye wrote:

 On Monday, 29 April 2013 at 12:23:04 UTC, deadalnix wrote:
 On Sunday, 28 April 2013 at 16:33:19 UTC, Idan Arye wrote:
 When you use `std.typecons.Nullable` with a type that 
 already accept `null` values, you get two types of nulls - 
 the `Nullable`'s null state the the regular type's `null`:

 Nullable!string a;
 writeln(a.isNull()); //prints "true"
 a = null;
 writeln(a.isNull()); //prints "false"
 a.nullify();
 writeln(a.isNull()); //prints "true"

All types should be non nullable. Problem solved.

*All* types? Even object references and pointers?

That would be nice, yes.

And what would they be initialized to? When you write: Object obj; what will `obj` refer to? Also, what about the C&C++ interface? Without null values, how can you use an extern function that accepts or returns pointers?

Data flow analysis can smash your face if you try to use that before initializing it. In fact, this is already done in many languages.

The point is that you are forcing ref variables to point to data, even in paths where there is no data for them to point to. Let's say we have something like: MyClass myObject; auto myVar1=DEFAULT_VALUE_FOR_MY_VAR_1; bool objectCreated=false; if(myCondition()){ myObject=new MyClass(/*some arguments*/); objectCreated=true; myVar1=myObject.calculateSomething(); } auto myVar2=calculateSomethingElse(myVar1); if(objectCreated){ myObject.doSomething(myVar2); } doSomethingElse(myVar2); It would be nice to create `myObject` in the second `if`, right before we use it, but this is not possible, since if `myCondition` is `true` and we do allocate `myObject`, we need to use it to calculate `myVar2`. And we can't pull the call for `myObject.doSomething` to the first `if` either - because we need to calculate `myVar2` before we use it. So, why not bring the calculation of `myVar2` into the first `if` as well? Because we need it for `doSomethingElse`, which should be invoked whether or not we allocate `myObject`! As humans, we can easily see that if we didn't allocate `myObject` that will also mean that `objectCreated` remains false, and therefore the program will not enter the second `if`. I would like to see a data flow analysis mechanism that figures that out - and that's a simple case! If `myObject` was `int` it would be easy - we could initialize it to `0` at declaration. But object references are not that simple - if you take away `null`, what will you init `myObject` to? Remember - it needs to be an instance of `MyClass` or of a subclass of `MyClass`. That means that: a) You have to allocate memory for an object you will not use. b) You need to call a constructor, that might have side-effects. c) You end up with an object that has a meaningless state. Now, let's look at that meaningless object we created. What if `MyClass` has fields which are object references themselves? They can't be in the "uninitialized state" - I would like to see the data flow analysis mechanism that can follow that! So, that means we need to set them to meaningless objects as well. Now, consider implementing a linked list.
Apr 29 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On 2013-04-29, 19:57, Idan Arye wrote:

 On Monday, 29 April 2013 at 16:14:02 UTC, deadalnix wrote:
 On Monday, 29 April 2013 at 16:02:11 UTC, Idan Arye wrote:
 On Monday, 29 April 2013 at 15:39:47 UTC, Simen Kjaeraas wrote:
 On 2013-04-29, 17:34, Idan Arye wrote:

 On Monday, 29 April 2013 at 12:23:04 UTC, deadalnix wrote:
 On Sunday, 28 April 2013 at 16:33:19 UTC, Idan Arye wrote:
 When you use `std.typecons.Nullable` with a type that already  
 accept `null` values, you get two types of nulls - the  
 `Nullable`'s null state the the regular type's `null`:

 Nullable!string a;
 writeln(a.isNull()); //prints "true"
 a = null;
 writeln(a.isNull()); //prints "false"
 a.nullify();
 writeln(a.isNull()); //prints "true"

All types should be non nullable. Problem solved.

*All* types? Even object references and pointers?

That would be nice, yes.

And what would they be initialized to? When you write: Object obj; what will `obj` refer to? Also, what about the C&C++ interface? Without null values, how can you use an extern function that accepts or returns pointers?

Data flow analysis can smash your face if you try to use that before initializing it. In fact, this is already done in many languages.

The point is that you are forcing ref variables to point to data, even in paths where there is no data for them to point to. Let's say we have something like: MyClass myObject; auto myVar1=DEFAULT_VALUE_FOR_MY_VAR_1; bool objectCreated=false; if(myCondition()){ myObject=new MyClass(/*some arguments*/); objectCreated=true; myVar1=myObject.calculateSomething(); } auto myVar2=calculateSomethingElse(myVar1); if(objectCreated){ myObject.doSomething(myVar2); } doSomethingElse(myVar2); It would be nice to create `myObject` in the second `if`, right before we use it, but this is not possible, since if `myCondition` is `true` and we do allocate `myObject`, we need to use it to calculate `myVar2`. And we can't pull the call for `myObject.doSomething` to the first `if` either - because we need to calculate `myVar2` before we use it. So, why not bring the calculation of `myVar2` into the first `if` as well? Because we need it for `doSomethingElse`, which should be invoked whether or not we allocate `myObject`! As humans, we can easily see that if we didn't allocate `myObject` that will also mean that `objectCreated` remains false, and therefore the program will not enter the second `if`. I would like to see a data flow analysis mechanism that figures that out - and that's a simple case! If `myObject` was `int` it would be easy - we could initialize it to `0` at declaration. But object references are not that simple - if you take away `null`, what will you init `myObject` to? Remember - it needs to be an instance of `MyClass` or of a subclass of `MyClass`. That means that: a) You have to allocate memory for an object you will not use. b) You need to call a constructor, that might have side-effects. c) You end up with an object that has a meaningless state. Now, let's look at that meaningless object we created. What if `MyClass` has fields which are object references themselves? They can't be in the "uninitialized state" - I would like to see the data flow analysis mechanism that can follow that! So, that means we need to set them to meaningless objects as well. Now, consider implementing a linked list.

Now, consider the fact we have Nullable in Phobos. -- Simen
Apr 29 2013
prev sibling next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Monday, 29 April 2013 at 18:20:35 UTC, Simen Kjaeraas wrote:
 On 2013-04-29, 19:57, Idan Arye wrote:

 Now, consider the fact we have Nullable in Phobos.

Yes, we have `Nullable` in Phobos. It works by having two member fields - `_value`, which stores the value, and `_isNull`, which specifies if the `Nullable` is null or not. Let's implement a bare bones Linked List: class Node(T){ T value; Nullable!(Node!T) next; this(){} this(T value){ this.value=value; } this(T value,Node!T next){ this.value=value; this.next=next; } } Now lets create an instance: auto myList=new Node("Hello"); What will happen? We create a new `Node!string`. it's `value` will be set to "hello", and since the one-argument constructor does not modify `next`, it will it will remain as the init value of `Nullable!(Node!string)`. What is the init value of `Nullable!(Node!string)`? The init value of `Nullable!(Node!string)` is an object with two member fields - `value` of type `string` and `next` of type `Nullable!(Node!string)`. The default constructor modifies neither, so they will both remain with their initial values. The init value of `string` is an empty string. What is the init value of `Nullable!(Node!string)`? To find the answer, return to the beginning of this paragraph and read it again. You are not supposed to be reading this paragraph. You are supposed to be stuck in an infinite recursion in the previous paragraph. As a human, you can cheat and escape it, but the computer is not so lucky - it'll allocate more and more objects until it'll crash with a stack overflow. Actually - that won't happen, since the compiler is smart enough to detect type recursion and emit e compile time error. But the problem remains - we are left unable to implement any type of recursive structure.
Apr 29 2013
prev sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Monday, 29 April 2013 at 20:00:32 UTC, Idan Arye wrote:
 The init value of `Nullable!(Node!string)` is an object with 
 two member fields - `value` of type `string` and `next` of type 
 `Nullable!(Node!string)`. The default constructor modifies 
 neither, so they will both remain with their initial values. 
 The init value of `string` is an empty string. What is the init 
 value of `Nullable!(Node!string)`? To find the answer, return 
 to the beginning of this paragraph and read it again.

I made a mistake here - `Nullable!(Node!string)` is a struct with a boolean member `_isNull` and a `Node!string` member named `_value` which is an object as described in this paragraph. Still, the point remains - even if `_isNull` is true, `_value` still need to refer to a `Node!string` object.
Apr 29 2013