www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Pluggable type sytems

reply bearophile <bearophileHUGS lycos.com> writes:
I think pluggable type systems will become more common in the following years
(see also the optional annotations of Python3 that are designed for that too).
This is more or less related:

http://bartoszmilewski.wordpress.com/2009/01/18/java-pluggable-types/

(but nonnullability is so basic that it's better inside the language, and not
left out to a plug-in type system).

Bye,
bearophile
Jan 18 2009
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2009-01-18 21:29:02 -0500, bearophile <bearophileHUGS lycos.com> said:

 I think pluggable type systems will become more common in the following 
 years (see also the optional annotations of Python3 that are designed 
 for that too). This is more or less related:
 
 http://bartoszmilewski.wordpress.com/2009/01/18/java-pluggable-types/

Nice post.
 (but nonnullability is so basic that it's better inside the language, 
 and not left out to a plug-in type system).

I agree for non-nullability. In fact, I'd even argue that non-nullability should be the default for pointers and class references, because it is either safer (if the compiler doesn't do the null check for you) or less troublesome to use (if the compiler forces you to check for null everytime). Another reason being consistency: value-types can't be null by default and so should be class references and pointers. And making pointers non-nullable by default can only be done at the language level, so obviously it's better in the language than as a user-defined type modifier. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Jan 19 2009
next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Mon, 19 Jan 2009 16:26:47 +0300, Michel Fortin <michel.fortin michelf.com>
wrote:

 On 2009-01-18 21:29:02 -0500, bearophile <bearophileHUGS lycos.com> said:

 I think pluggable type systems will become more common in the following  
 years (see also the optional annotations of Python3 that are designed  
 for that too). This is more or less related:
  http://bartoszmilewski.wordpress.com/2009/01/18/java-pluggable-types/

Nice post.
 (but nonnullability is so basic that it's better inside the language,  
 and not left out to a plug-in type system).

I agree for non-nullability. In fact, I'd even argue that non-nullability should be the default for pointers and class references, because it is either safer (if the compiler doesn't do the null check for you) or less troublesome to use (if the compiler forces you to check for null everytime). Another reason being consistency: value-types can't be null by default and so should be class references and pointers. And making pointers non-nullable by default can only be done at the language level, so obviously it's better in the language than as a user-defined type modifier.

Agree.
Jan 20 2009
prev sibling next sibling parent reply Bartosz Milewski <bartosz relisoft.com> writes:
I also believe that non-null should be the default. Most references in a
typical program are assumed to be non-null. 

Like for every default, there should be an escape mechanism from it. However,
adding "nullable" to the type system might be controversial, especially because
it requires special treatment of constructors and polymorphism (hence
additional type modifiers, raw and poly-null, and I haven't even mentioned
wildcard annotations).

So some time ago I came up with a more modest proposal. The compiler should
catch nullable references at their source. It should enforce what I call
"declaration is initialization" (DII). For instance, the following is correct:

Foo foo = new Foo;

and the following is not:

Foo foo; // error
...
foo = new Foo;

Obviously this rule is too strict, so there must be a way to override it.
Here's the simple syntax:

Foo foo = null; // not an error!

Notice that the programmer explicitly states that one should be careful when
using foo, because it might be null. 

Since this solution doesn't involve the type system, the nullability doesn't
propagate across function calls. But it might be a tradeoff worth taking, if
only for the simplicity.

Constructors still need some special treatment. Every class member variable
must be initialized in the constructor. Again, the escape is to _explicitly_
initialize it to null. This might seem redundant, since the variable is already
default initialized to null, but the compiler can easily eliminate such
redundancies. 

I also found out that Java's editor, Eclipse, enforces DII--a very useful
feature that paid off for me in the first 100 lines of code I wrote after a
long period of inactivity in Java ;-) 

 Michel Fortin Wrote:

 In fact, I'd even argue that non-nullability should be the default for 
 pointers and class references, because it is either safer (if the 
 compiler doesn't do the null check for you) or less troublesome to use 
 (if the compiler forces you to check for null everytime). Another 
 reason being consistency: value-types can't be null by default and so 
 should be class references and pointers. And making pointers 
 non-nullable by default can only be done at the language level, so 
 obviously it's better in the language than as a user-defined type 
 modifier.

Feb 08 2009
next sibling parent reply Robert Fraser <fraserofthenight gmail.com> writes:
Bartosz Milewski wrote:
 I also found out that Java's editor, Eclipse, enforces DII--a very useful
feature that paid off for me in the first 100 lines of code I wrote after a
long period of inactivity in Java ;-) 

The Java spec does, actually. Eclipse JDT just reports compiler errors as you type.
Feb 08 2009
parent Bartosz Milewski <bartosz relisoft.com> writes:
Shows you what a beginner in Java I am. Again and again I keep reinventing the
wheel. That settles it--we have to have it in D!


Robert Fraser Wrote:

 Bartosz Milewski wrote:
 I also found out that Java's editor, Eclipse, enforces DII--a very useful
feature that paid off for me in the first 100 lines of code I wrote after a
long period of inactivity in Java ;-) 

The Java spec does, actually. Eclipse JDT just reports compiler errors as you type.

Feb 08 2009
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Bartosz Milewski:
 I also believe that non-null should be the default.

That' probably right. And you can add ? to the things that can be null. Better to let the type system deal with this. Bye, bearophile
Feb 08 2009
prev sibling parent Bartosz Milewski <bartosz relisoft.com> writes:
I just assume that any extension of the already complex D type system will be
met with a lot of resistance. I remember the all-out wars about const and
immutable (a.k.a invariant). Even those extensions are still half-assed: the
construction of immutable objects and const polymorphism issues remain. 

My impression is that theoreticians and very advanced programmers love
elaborate type systems. Nine-to-five programmers, which are in the majority,
prefer simplicity even if the cost is reliability. Just look at the comments to
my blog post. 

I know we have to do something about null references in D, but I'm still on the
fence about how we should accomplish that.

Denis Koroskin Wrote:

 On Mon, 09 Feb 2009 01:19:55 +0300, Bartosz Milewski <bartosz relisoft.com>
wrote:
 
 I also believe that non-null should be the default. Most references in a  
 typical program are assumed to be non-null.

Oh, that's true!
 Like for every default, there should be an escape mechanism from it.  
 However, adding "nullable" to the type system might be controversial,  
 especially because it requires special treatment of constructors and  
 polymorphism (hence additional type modifiers, raw and poly-null, and I  
 haven't even mentioned wildcard annotations).

Common opinion is that this is worth the trouble these days. More and more people discuss, write articles on non-nullable types and incorporate them into languages. Main C# architects has stated that absence of non-nullable types by default is his greatest mistake that is too late to be fixed.
 So some time ago I came up with a more modest proposal. The compiler  
 should catch nullable references at their source. It should enforce what  
 I call "declaration is initialization" (DII). For instance, the  
 following is correct:

 Foo foo = new Foo;

 and the following is not:

 Foo foo; // error
 ...
 foo = new Foo;

 Obviously this rule is too strict, so there must be a way to override  
 it. Here's the simple syntax:

 Foo foo = null; // not an error!

And what about value types? Why special treatment for reference types only? I don't like rules like that in a language: Foo foo; // error int i; // okay? If not, and this is an error, too, then this defeats the whole purpose of having T.init in D, because it will never be used per your proposal: int i = 0; // initialized explicitly char c = char.init; // would you use that?
 Notice that the programmer explicitly states that one should be careful  
 when using foo, because it might be null.

 Since this solution doesn't involve the type system, the nullability  
 doesn't propagate across function calls. But it might be a tradeoff  
 worth taking, if only for the simplicity.

Naah.. The whole purpose of having non-nullable types is that given a pointer you are sure it is not null: string toString(Object o) { // is it null or not? Should I check for null or not? What if it is null? // Should I throw? What kind of exception? // But was going to make it nothrow! So what, no checks? // Just fail with access violation? Is this the best D can do? return o.toString(); } As an end user, you neglect and avoid null checks in your code ("I'm sure I won't pass null here"). And then you run your program half an hour, it segfaults with access violation and you don't have any clue where did null dereference take place. I have came across this situation so many times that I hate it. As a library designer, you end up having null checks everywhere in your code which is some kind of a paranoia because in 99% of cases the pointer will be not null. But why put so many special case and checks if it is *disallowed* to run code with null pointer in first place? So people decided - let's enforce that at compile time! That's the whole point of the feature - disallow invalid use cases at compile time. You get cleaner code (no unnecessary null-checks and throws) and small performance improvement as a bonus. I would like to write code as this as be sure that it will never fail: string toString(Object o) // non-nullability is enforced at compile time { return o.toString(); } This is D style - do as much as possible at compile time - and I believe non-nullable types fit D type system very well.
 Constructors still need some special treatment. Every class member  
 variable must be initialized in the constructor. Again, the escape is to  
 _explicitly_ initialize it to null. This might seem redundant, since the  
 variable is already default initialized to null, but the compiler can  
 easily eliminate such redundancies.

 I also found out that Java's editor, Eclipse, enforces DII--a very  
 useful feature that paid off for me in the first 100 lines of code I  
 wrote after a long period of inactivity in Java ;-)

 Michel Fortin Wrote:

 In fact, I'd even argue that non-nullability should be the default for
 pointers and class references, because it is either safer (if the
 compiler doesn't do the null check for you) or less troublesome to use
 (if the compiler forces you to check for null everytime). Another
 reason being consistency: value-types can't be null by default and so
 should be class references and pointers. And making pointers
 non-nullable by default can only be done at the language level, so
 obviously it's better in the language than as a user-defined type
 modifier.



Feb 08 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Mon, 09 Feb 2009 01:19:55 +0300, Bartosz Milewski <bartosz relisoft.com>
wrote:

 I also believe that non-null should be the default. Most references in a  
 typical program are assumed to be non-null.

Oh, that's true!
 Like for every default, there should be an escape mechanism from it.  
 However, adding "nullable" to the type system might be controversial,  
 especially because it requires special treatment of constructors and  
 polymorphism (hence additional type modifiers, raw and poly-null, and I  
 haven't even mentioned wildcard annotations).

Common opinion is that this is worth the trouble these days. More and more people discuss, write articles on non-nullable types and incorporate them into languages. Main C# architects has stated that absence of non-nullable types by default is his greatest mistake that is too late to be fixed.
 So some time ago I came up with a more modest proposal. The compiler  
 should catch nullable references at their source. It should enforce what  
 I call "declaration is initialization" (DII). For instance, the  
 following is correct:

 Foo foo = new Foo;

 and the following is not:

 Foo foo; // error
 ...
 foo = new Foo;

 Obviously this rule is too strict, so there must be a way to override  
 it. Here's the simple syntax:

 Foo foo = null; // not an error!

And what about value types? Why special treatment for reference types only? I don't like rules like that in a language: Foo foo; // error int i; // okay? If not, and this is an error, too, then this defeats the whole purpose of having T.init in D, because it will never be used per your proposal: int i = 0; // initialized explicitly char c = char.init; // would you use that?
 Notice that the programmer explicitly states that one should be careful  
 when using foo, because it might be null.

 Since this solution doesn't involve the type system, the nullability  
 doesn't propagate across function calls. But it might be a tradeoff  
 worth taking, if only for the simplicity.

Naah.. The whole purpose of having non-nullable types is that given a pointer you are sure it is not null: string toString(Object o) { // is it null or not? Should I check for null or not? What if it is null? // Should I throw? What kind of exception? // But was going to make it nothrow! So what, no checks? // Just fail with access violation? Is this the best D can do? return o.toString(); } As an end user, you neglect and avoid null checks in your code ("I'm sure I won't pass null here"). And then you run your program half an hour, it segfaults with access violation and you don't have any clue where did null dereference take place. I have came across this situation so many times that I hate it. As a library designer, you end up having null checks everywhere in your code which is some kind of a paranoia because in 99% of cases the pointer will be not null. But why put so many special case and checks if it is *disallowed* to run code with null pointer in first place? So people decided - let's enforce that at compile time! That's the whole point of the feature - disallow invalid use cases at compile time. You get cleaner code (no unnecessary null-checks and throws) and small performance improvement as a bonus. I would like to write code as this as be sure that it will never fail: string toString(Object o) // non-nullability is enforced at compile time { return o.toString(); } This is D style - do as much as possible at compile time - and I believe non-nullable types fit D type system very well.
 Constructors still need some special treatment. Every class member  
 variable must be initialized in the constructor. Again, the escape is to  
 _explicitly_ initialize it to null. This might seem redundant, since the  
 variable is already default initialized to null, but the compiler can  
 easily eliminate such redundancies.

 I also found out that Java's editor, Eclipse, enforces DII--a very  
 useful feature that paid off for me in the first 100 lines of code I  
 wrote after a long period of inactivity in Java ;-)

 Michel Fortin Wrote:

 In fact, I'd even argue that non-nullability should be the default for
 pointers and class references, because it is either safer (if the
 compiler doesn't do the null check for you) or less troublesome to use
 (if the compiler forces you to check for null everytime). Another
 reason being consistency: value-types can't be null by default and so
 should be class references and pointers. And making pointers
 non-nullable by default can only be done at the language level, so
 obviously it's better in the language than as a user-defined type
 modifier.


Feb 08 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Mon, 09 Feb 2009 04:09:26 +0300, Bartosz Milewski <bartosz relisoft.com>
wrote:

 I just assume that any extension of the already complex D type system  
 will be met with a lot of resistance. I remember the all-out wars about  
 const and immutable (a.k.a invariant). Even those extensions are still  
 half-assed: the construction of immutable objects and const polymorphism  
 issues remain.

 My impression is that theoreticians and very advanced programmers love  
 elaborate type systems. Nine-to-five programmers, which are in the  
 majority, prefer simplicity even if the cost is reliability. Just look  
 at the comments to my blog post.

 I know we have to do something about null references in D, but I'm still  
 on the fence about how we should accomplish that.

So, let's ask the community: Would you like to see nullable types in D? http://www.micropoll.com/akira/mpview/539369-138652 (please, don't abuse by voting multiple time) Explain your reasoning in newsgroups. Thank you.
Feb 08 2009
prev sibling parent "Danny Wilson" <bluezenix gmail.com> writes:
Op Mon, 09 Feb 2009 02:25:55 +0100 schreef Denis Koroskin  
<2korden gmail.com>:

 On Mon, 09 Feb 2009 04:09:26 +0300, Bartosz Milewski  
 <bartosz relisoft.com> wrote:

 ...

 My impression is that theoreticians and very advanced programmers love  
 elaborate type systems. Nine-to-five programmers, which are in the  
 majority, prefer simplicity even if the cost is reliability. Just look  
 at the comments to my blog post.

 I know we have to do something about null references in D, but I'm  
 still on the fence about how we should accomplish that.

So, let's ask the community: Would you like to see nullable types in D? http://www.micropoll.com/akira/mpview/539369-138652 (please, don't abuse by voting multiple time) Explain your reasoning in newsgroups. Thank you.

I voted yes. There have been far too many NullPointerExceptions in various languages, for the 9-5 programmer to understand in 15 minutes what non-nullable types are :-)
Feb 09 2009
prev sibling parent "Nick Sabalausky" <a a.a> writes:
"bearophile" <bearophileHUGS lycos.com> wrote in message 
news:gl0ohe$1gda$1 digitalmars.com...
I think pluggable type systems will become more common in the following 
years (see also the optional annotations of Python3 that are designed for 
that too). This is more or less related:

 http://bartoszmilewski.wordpress.com/2009/01/18/java-pluggable-types/

 (but nonnullability is so basic that it's better inside the language, and 
 not left out to a plug-in type system).

 Bye,
 bearophile

Interesting article. Sounds a lot like D2 but with that flow-analysis nice-ity, and PolyNull (which has been proposed here...often). Although I wonder...maybe PolyNull should be the default (wherever applicable) instead of Nullable or NonNull? (Or maybe I'm overlooking some obvious problem with that.) I see pluggable type sytems as potentially being a specific form of a more general thing I've been observing an increasing need for/trend towards: customizable languages. Ex: A language that doesn't say "Semicolons mark end-of-statement" or "Newline marks end-of-statement and lines can be split with a backslash", but provides both as mutually-exclusive programmer-selectable options. Or a choice of C-style curly-braces vs Python-style indentation. Etc. (Side note: This stuff could really cut down on religious language debates. Not eliminate, of course, but reduce.) Also, a tool could then be used to translate code between the various options, maybe even integrated into the IDE, so that people working on multi-person projects could each set up their own environments and use their own "customized language" even while working on the same line of code. The only project-wide standard would be what settings are used for checked-in code, just like tab/space settings and EOL markers (but ideally performed automatically (and reliably) by the check-in/check-out process). Eventually this might even be generalizable to the point where most languages can be reduced to nothing more than a collection of language settings. Which I think is the direction we appear to be headed anyway, since it seems like everyone and their grandmother is designing their own language (or two) these days, and many of these languages are more-or-less "just like language X, but with A, B and C changed to be more like language Y." Obviously this is all much easier said than done, but it's something I've been thinking a lot about lately, and I think may be worth exploring.
Jan 22 2009