www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - NotNull pointers

reply Walter Bright <newshound2 digitalmars.com> writes:
For the latest dmd, 
https://github.com/D-Programming-Language/dmd/commit/1193f7828b444056c943
42daae0a5ccf262272e 
,
  I've implemented the ability to disable default initialization. This makes it 
practical to implement a library based "NotNull" type without a special syntax 
for it. The rationale for this is (rather than making it builtin) one can now 
build any type that is a restricted subset of another type. I suspect that a 
non-null type is just scratching the surface of this ability.

Here's a draft prototype of the NotNull type:

import std.c.stdio;

struct NotNull(P) if (is(typeof({P p = null;})))
{
     P p;

     this()  disable;

     this(P q)
     {
         assert(q);
         p = q;
     }

     NotNull opAssign(P q)
     {
         assert(q);
         p = q;
         return this;
     }

     alias p this;
}

void main()
{
     int x;
     NotNull!(int*) s = &x;
     *s = 3;
     printf("test1 %d\n", *s);
     s++;
     printf("test2 %d\n", s[-1]);
}

What it does is:
1. disallows instantiation with any type that null doesn't implicitly convert to
2. disables default construction (this is the new feature)
3. intercepts construction and assignment to enforce not-nullness
4. uses alias this to forward the rest to the wrapped pointer type

For example, try this:

void main()
{
     NotNull!(int*) s;
// Error: variable test.main.s initializer required for type NotNull!(int*)
}

I do think that the "this()  disable;" is an ugly syntax, and I cringe when 
seeing it. But I can't think of anything better. It does make logical sense 
given the existence of default construction syntax and the  disable, so in a 
sense it is just connecting existing dots, which has a compelling value.
Aug 29 2011
next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Walter Bright (newshound2 digitalmars.com)'s article
 For the latest dmd,

 ,
   I've implemented the ability to disable default initialization. This makes it
 practical to implement a library based "NotNull" type without a special syntax
 for it.

How does it work for member variables of classes and structs? E.g.: // Is this legal or not? struct Foo { NotNull!(int*) notNull; this(int* ptr) { notNull = ptr; } }
Aug 29 2011
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/29/2011 1:32 PM, dsimcha wrote:
 How does it work for member variables of classes and structs?  E.g.:

 // Is this legal or not?
 struct Foo
 {
      NotNull!(int*) notNull;

      this(int* ptr)
      {
          notNull = ptr;
      }
 }

Try it and see! (It works as you'd expect.)
Aug 29 2011
parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Walter Bright (newshound2 digitalmars.com)'s article
 On 8/29/2011 1:32 PM, dsimcha wrote:
 How does it work for member variables of classes and structs?  E.g.:

 // Is this legal or not?
 struct Foo
 {
      NotNull!(int*) notNull;

      this(int* ptr)
      {
          notNull = ptr;
      }
 }


http://d.puremagic.com/test-results/ DMD doesn't even build right now.
Aug 29 2011
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/29/2011 2:09 PM, dsimcha wrote:
 DMD doesn't even build right now.

Dang ships passing in the night. Fixed.
Aug 29 2011
prev sibling next sibling parent reply "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Mon, 29 Aug 2011 22:22:52 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:

 For the latest dmd,  
 https://github.com/D-Programming-Language/dmd/commit/1193f7828b444056c943
42daae0a5ccf262272e  
 ,
   I've implemented the ability to disable default initialization. This  
 makes it practical to implement a library based "NotNull" type without a  
 special syntax for it. The rationale for this is (rather than making it  
 builtin) one can now build any type that is a restricted subset of  
 another type. I suspect that a non-null type is just scratching the  
 surface of this ability.

Awesome! How does it deal with arrays whose elements have no default constructor? I believe increasing the length of such an array should be an error (which likely also excludes decreasing it except for slices).
 I do think that the "this()  disable;" is an ugly syntax, and I cringe  
 when seeing it. But I can't think of anything better. It does make  
 logical sense given the existence of default construction syntax and the  
  disable, so in a sense it is just connecting existing dots, which has a  
 compelling value.

I think it's perfect, you just need the words to trade places: disable this(); One could argue that disabled would work better, but that's a can of worms I'm not touching. Really looking forward to trying this out. -- Simen
Aug 29 2011
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/30/2011 1:06 PM, Simen Kjaeraas wrote:
 Testing concludes that changing the length of the array indeed appends
 structs to it, with no compile-time error. Should I file this in BugZilla?

Not yet. I'm not sure that using bugzilla for things under development is that good an idea. Once it is released, then sure.
Aug 30 2011
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/29/11 3:22 PM, Walter Bright wrote:
 For the latest dmd,
 https://github.com/D-Programming-Language/dmd/commit/1193f7828b444056c943742daae0a5ccf262272e
 ,
 I've implemented the ability to disable default initialization. This
 makes it practical to implement a library based "NotNull" type without a
 special syntax for it. The rationale for this is (rather than making it
 builtin) one can now build any type that is a restricted subset of
 another type. I suspect that a non-null type is just scratching the
 surface of this ability.

 Here's a draft prototype of the NotNull type:

 import std.c.stdio;

 struct NotNull(P) if (is(typeof({P p = null;})))
 {
 P p;

 this()  disable;

 this(P q)
 {
 assert(q);
 p = q;
 }

 NotNull opAssign(P q)
 {
 assert(q);
 p = q;
 return this;
 }

 alias p this;
 }

This is very compelling. Great work! FWIW the implementation could be refined to never expose the held reference as an lvalue: struct NotNull(P) if (is(typeof({P p = null;}))) { private P p; this() disable; this(P q) { assert(q); p = q; } NotNull opAssign(P q) { assert(q); p = q; return this; } property P get() { return p; } alias get this; // force rvalue access } Andrei
Aug 29 2011
prev sibling next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
Oh, one more thing:  What about this:

NotNull!(int*) notNull = void;
Aug 29 2011
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/29/2011 1:56 PM, dsimcha wrote:
 Oh, one more thing:  What about this:

 NotNull!(int*) notNull = void;

It's accepted, unless it's in safe code.
Aug 29 2011
prev sibling next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 30.08.2011 0:22, Walter Bright wrote:
 For the latest dmd,
 https://github.com/D-Programming-Language/dmd/commit/1193f7828b444056c943742daae0a5ccf262272e
 ,
 I've implemented the ability to disable default initialization. This
 makes it practical to implement a library based "NotNull" type without a
 special syntax for it. The rationale for this is (rather than making it
 builtin) one can now build any type that is a restricted subset of
 another type. I suspect that a non-null type is just scratching the
 surface of this ability.

Nice! Clean RAII with structs comes to mind, before you'd have to have .init to be as an invalid state that is always checked later on, e.g.: File f; f.write("blah");//will fail enforce inside write I suspect it can get a bit tricky to deal with arrays of such types though. -- Dmitry Olshansky
Aug 29 2011
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter:

   I've implemented the ability to disable default initialization. This makes
it 
 practical to implement a library based "NotNull" type without a special syntax 
 for it. The rationale for this is (rather than making it builtin)

If it works well enough, a syntax is allowed to come later, as it's happening with tuple unpacking now (dmd pull 341). I have tried the feature, but what's the way to use it? import core.stdc.stdio; struct NotNull(P) if (__traits(compiles, {P p = null;})) { private P p; this() disable; this(P q) { assert(q); p = q; } NotNull opAssign(P q) { assert(q); p = q; return this; } property P get() { return p; } alias get this; // force rvalue access } struct Foo { int x; } alias NotNull!(Foo*) NFooP; void bar(NFooP foop) { // Error: variable test2.bar.foop initializer required for type NotNull!(Foo*) printf("d\n", foop.x); } void main() { NFooP p = new Foo; bar(p); } Bye, bearophile
Aug 29 2011
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/29/2011 4:52 PM, bearophile wrote:
 alias NotNull!(Foo*) NFooP;

 void bar(NFooP foop) { // Error: variable test2.bar.foop initializer required
for type NotNull!(Foo*)

Looks like you found a bug. I'll take care of it.
Aug 29 2011
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/29/2011 5:11 PM, Walter Bright wrote:
 On 8/29/2011 4:52 PM, bearophile wrote:
 alias NotNull!(Foo*) NFooP;

 void bar(NFooP foop) { // Error: variable test2.bar.foop initializer required
 for type NotNull!(Foo*)

Looks like you found a bug. I'll take care of it.

Fixed. Keep 'em coming! https://github.com/D-Programming-Language/dmd/commit/883ccdb8b2d423de0f028522edd0c91b5992113f
Aug 29 2011
parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter:

 Fixed. Keep 'em coming!

You are quick :-) What's the way to solve this problem? import core.stdc.stdio; struct NotNull(P) if (__traits(compiles, {P p = null;})) { private P p; disable this(); this(P q) { assert(q); p = q; } NotNull opAssign(P q) { assert(q); p = q; return this; } property P get() { return p; } alias get this; // force rvalue access } class Foo { int x; this(int x_) { this.x = x_; } } alias NotNull!(Foo) nFoo; // if nFoo is a good enough nonnull type, then this // function should _never_ cause an Access Violation void bar(nFoo foo) { printf("%d\n", foo.x); } void main() { auto a = new nFoo[5]; bar(a[0]); } Bye, bearophile
Aug 29 2011
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
 What's the way to solve this problem?

This gives you some ideas: http://research.microsoft.com/pubs/67461/non-null.pdf Bye, bearophile
Aug 29 2011
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/29/11 7:49 PM, bearophile wrote:
[snip]

 void main() {
      auto a = new nFoo[5];
      bar(a[0]);
 }

Ah, that reminds me. The introduction of disable requires the introduction of array-with-constructor syntax that has long been proposed to both C++ and D: new Type[n](argument1, argument2, ..., argumentn) creates an array of Type of size n. Each element is constructed with argument list (argument1, argument2, ..., argumentn). Andrei
Aug 30 2011
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 new Type[n](argument1, argument2, ..., argumentn)

I'd like those constructors to know what's their array index. But too much magic (like using $) is better left to Perl language. Bye, bearophile
Aug 30 2011
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 new Type[n](argument1, argument2, ..., argumentn)

More musings about this. A generic lazy/eager array comprehension syntax seems better, like (for the eager one): [Baz(x, x*y) foreach(x; 0 .. 10) if (isGood(x))] Currently to create an array of structs like this Foo you need to use array append (if you want to keep the x fields mutable): struct Foo { int x; const int y; } void main() { Foo[] foos; foos.reserve(10); foreach (i; 0 .. 10) foos ~= Foo(i, i*10); } With an array comp the compiler doesn't need to use an an array append, just array assign (when there is no if condition), that in D is quite faster: struct Foo { int x; const int y; } void main() { auto a = [Foo(i, i*10) foreach(i; 0 .. 10)]; } I think a good compiler is able to optimize even the following code, removing the array append, but this is less simple to do if the body of foreach becomes more complex, with gotos, etc: foos.reserve(10); foreach (i; 0 .. 10) foos ~= Foo(i, i*10); Bye, bearophile
Aug 30 2011
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, August 30, 2011 09:35 Andrei Alexandrescu wrote:
 On 8/29/11 7:49 PM, bearophile wrote:
 [snip]
 
 void main() {
 
 auto a = new nFoo[5];
 bar(a[0]);
 
 }

Ah, that reminds me. The introduction of disable requires the introduction of array-with-constructor syntax that has long been proposed to both C++ and D: new Type[n](argument1, argument2, ..., argumentn) creates an array of Type of size n. Each element is constructed with argument list (argument1, argument2, ..., argumentn).

That particular syntax does kind of fly in the face of discusssions to remove new Type[n] in favor of requiring new Type[](n). And even if that's not considered a problem, how does that interact with multi-dimensional arrays? e.g. new Type[][](4, 5)? Or does it just not make sense with multi-dimensional arrays? I suppose that we could just say that a second set of parens is required. So, new Type[n](args) works as long as we have new Type[n](args), and new Type[] (n)(args) works for the case where you put the size in the parens (as arguably should be required). Then if it supported multi-dimensional arrays, it would be something like new Type[][](4, 5)(args), though I'm not quite sure how you'd really support it in multidimensional arrays except perhaps something like new Type[][](4, 5)((args1), (args2), (args3), (args4)) - where each argsX is a full arguments list for each inner array. That's arguably pretty ugly though. However, I don't see any other way to support multi-dimensional arrays if we want to. Regardless, the issue of how to properly distinguish between the current new Type[](n) and the the array-with-constructor syntax. And I'd _love_ to see new Type[n] die, given the confusion that it causes when going to multi- dimensional arrays. But assuming that we can't do that, we should at least make sure that there's no ambiguity with new Type[](n). - Jonathan M Davis
Aug 30 2011
prev sibling next sibling parent reply "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Wed, 31 Aug 2011 01:11:20 +0200, bearophile <bearophileHUGS lycos.com>  
wrote:

 Andrei Alexandrescu:

 new Type[n](argument1, argument2, ..., argumentn)

More musings about this. A generic lazy/eager array comprehension syntax seems better, like (for the eager one): [Baz(x, x*y) foreach(x; 0 .. 10) if (isGood(x))]

I think this warrants discussion. Earlier, I have implemented rudimentary range comprehensions that look like this: list!"2 * a" | iota(10) & where!"a & 1" It works, for simple cases. In my opinion, any D array/list/range comprehension should work on ranges as its base, so foreach is out. Instead, a range should be the base for the comprehension. A simple solution would thus be: [(x){return 2 * x;}; iota(10); (x){return x & 1;}] This could even be implemented in a library: list((int x){return 2 * x;}, iota(10), (int x){return x & 1;}) However, as anyone can see, this is crap. It uses delegates, is waay too verbose, and hurts my eyes. (It could be somewhat better with the functions passed as template alias parameters, but that would mess up the order of parameters, and on the whole not help that much) Next iteration: [x; 2 * x; iota(10); x & 1] This is a syntax I could accept. I would like to have the x be more closely associated with the range, something more akin to this: [2 * x; x <= iota(10); x & 1] I am however at a loss as to what combination of squiggles could be used in place of the <= (which is of course not a good choice). The following variations should be allowed: [x; ; iota(10); x & 1]; // (or [; x <= iota(10); x & 1]) [x; 2 * x; iota(10); ]; // (or [2 * x; x <= iota(10); ]) With the first being simply a filter, the second simply a map. -- Simen
Aug 31 2011
parent bearophile <bearophileHUGS lycos.com> writes:
Simen Kjaeraas:

 Next iteration:
 
      [x; 2 * x; iota(10); x & 1]

If you don't like the syntax I've shown with foreach, then use the Python syntax, it's readable and short and good: [2 * x for x in iota(10) if x & 1] Or: [2 * x for x in 0 .. 10 if x & 1] Bye, bearophile
Aug 31 2011
prev sibling next sibling parent reply "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Wed, 31 Aug 2011 18:57:01 +0200, bearophile <bearophileHUGS lycos.com>  
wrote:

 Simen Kjaeraas:

 Next iteration:

      [x; 2 * x; iota(10); x & 1]

If you don't like the syntax I've shown with foreach, then use the Python syntax, it's readable and short and good: [2 * x for x in iota(10) if x & 1] Or: [2 * x for x in 0 .. 10 if x & 1]

I feel it is too much a departure from the style otherwise present in D. It is important not only that the syntax is good in and of itself, but also in the context of D. -- Simen
Aug 31 2011
parent reply bearophile <bearophileHUGS lycos.com> writes:
Simen Kjaeraas:

 I feel it is too much a departure from the style otherwise present in D.

This is why I have shown my original syntax with [foreach() if()].
 It is important not only that the syntax is good in and of itself,
 but also in the context of D.

The syntax you have shown is noisy and hard to read. Array/seq comps are all (or most) a matter of syntax sugar. So if their syntax is bad, they miss their main point by a mile. Bye, bearophile
Aug 31 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/31/2011 11:10 PM, Simen Kjaeraas wrote:
 On Wed, 31 Aug 2011 21:16:37 +0200, bearophile
 <bearophileHUGS lycos.com> wrote:

 Simen Kjaeraas:

 I feel it is too much a departure from the style otherwise present in D.

This is why I have shown my original syntax with [foreach() if()].
 It is important not only that the syntax is good in and of itself,
 but also in the context of D.

The syntax you have shown is noisy and hard to read. Array/seq comps are all (or most) a matter of syntax sugar. So if their syntax is bad, they miss their main point by a mile.

And this is where we disagree. I think the syntax is pretty good, and that the pythonesque syntax sticks out like a sore thumb. Instead of conflating the concepts of filter, map and source data, I aimed for a solution that is familiar to those accustomed to D, with clear distinctions and the benefits that brings. My design is meant to be somewhat similar to for-loops, with the semicolon-separated expressions, and to set builder notation. I kinda wish 'in' was used in foreach loops (foreach(x in foo){}), as that would be a perfect fit for the <= in the proposed syntax: My notation: [2 * x; x in iota(10); x*x > 4] Set builder notation: {2 · x | x ∈ ℕ, x² > 4} Personally, I find set builder notation to be very clear and understandable, and thus worth striving to imitate. In D however, the curly brackets and comma operator already have other meanings that we should try not to interfere with.

The meaning of the comma is already quite overloaded, so, if it is better readable, I think using comma would be fine.
Aug 31 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/31/2011 11:33 PM, Simen Kjaeraas wrote:
 On Wed, 31 Aug 2011 23:16:26 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 My design is meant to be somewhat similar to for-loops, with the
 semicolon-separated expressions, and to set builder notation. I kinda
 wish 'in'
 was used in foreach loops (foreach(x in foo){}), as that would be a
 perfect fit
 for the <= in the proposed syntax:

 My notation:
 [2 * x; x in iota(10); x*x > 4]

 Set builder notation:
 {2 · x | x ∈ ℕ, x² > 4}

 Personally, I find set builder notation to be very clear and
 understandable, and
 thus worth striving to imitate. In D however, the curly brackets and
 comma
 operator already have other meanings that we should try not to interfere
 with.

The meaning of the comma is already quite overloaded, so, if it is better readable, I think using comma would be fine.

I'm afraid that's not really an option in this case. It would conflict with both array literals and the comma operator. e.g. [2 * x, x in iota(10), x*x > 4] would be seen by the compiler as an attempt to create an array containing three different types. also, [2 * x; x in iota(10); log(x), x*x > 4] would be more complex to write ([2 * x; x in iota(10); (a){ log(a); return a*a > 4;}(x)], possibly?)

[2 * x ; x <- iota(10), log(x), x*x > 4] or, in a library: compr!q {2 * x ; x <- iota(10), log(x), x*x > 4};
Aug 31 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/31/2011 11:42 PM, Timon Gehr wrote:
 On 08/31/2011 11:33 PM, Simen Kjaeraas wrote:
 On Wed, 31 Aug 2011 23:16:26 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 My design is meant to be somewhat similar to for-loops, with the
 semicolon-separated expressions, and to set builder notation. I kinda
 wish 'in'
 was used in foreach loops (foreach(x in foo){}), as that would be a
 perfect fit
 for the <= in the proposed syntax:

 My notation:
 [2 * x; x in iota(10); x*x > 4]

 Set builder notation:
 {2 · x | x ∈ ℕ, x² > 4}

 Personally, I find set builder notation to be very clear and
 understandable, and
 thus worth striving to imitate. In D however, the curly brackets and
 comma
 operator already have other meanings that we should try not to
 interfere
 with.

The meaning of the comma is already quite overloaded, so, if it is better readable, I think using comma would be fine.

I'm afraid that's not really an option in this case. It would conflict with both array literals and the comma operator. e.g. [2 * x, x in iota(10), x*x > 4] would be seen by the compiler as an attempt to create an array containing three different types. also, [2 * x; x in iota(10); log(x), x*x > 4] would be more complex to write ([2 * x; x in iota(10); (a){ log(a); return a*a > 4;}(x)], possibly?)

[2 * x ; x <- iota(10), log(x), x*x > 4] or, in a library: compr!q {2 * x ; x <- iota(10), log(x), x*x > 4};

compr!q{2 * x ; x <- iota(10), log(x), x*x > 4};
Aug 31 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/01/2011 12:26 AM, Simen Kjaeraas wrote:
 On Wed, 31 Aug 2011 23:54:04 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 [2 * x ; x <- iota(10), log(x), x*x > 4]

 or, in a library:

 compr!q {2 * x ; x <- iota(10), log(x), x*x > 4};

compr!q{2 * x ; x <- iota(10), log(x), x*x > 4};

The library solution has problems with scope variables.

I know, that is why I have not implemented this yet. Basically it should be possible to implicitly pass a whole scope to a template, or at least, to have local templates.
 Example:

 auto foo(Range)(int n, Range r) {
 return compr!q{x * n; x <- arr, log(x), x*x > 4};
 }

Workaround: auto foo(Range)(int n, Range r) { mixin Compr!q{x * n ; x <- arr, log(x), x*x > 4}; return compr; }
 More importantly, IMO, is that the notation does not clearly
 mark what is what. is that interpreted (iota(10), log(x))
 or (log(x), x*x > 4)?

It actually does. The comma is a mere separator, and imho it looks very clean. It is just like the set builder syntax you like. (it even has the curly brackets!)
Aug 31 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/01/2011 01:03 AM, Timon Gehr wrote:
 On 09/01/2011 12:26 AM, Simen Kjaeraas wrote:
 On Wed, 31 Aug 2011 23:54:04 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 [2 * x ; x <- iota(10), log(x), x*x > 4]

 or, in a library:

 compr!q {2 * x ; x <- iota(10), log(x), x*x > 4};

compr!q{2 * x ; x <- iota(10), log(x), x*x > 4};

The library solution has problems with scope variables.

I know, that is why I have not implemented this yet. Basically it should be possible to implicitly pass a whole scope to a template, or at least, to have local templates.
 Example:

 auto foo(Range)(int n, Range r) {
 return compr!q{x * n; x <- arr, log(x), x*x > 4};
 }

Workaround: auto foo(Range)(int n, Range r) { mixin Compr!q{x * n ; x <- arr, log(x), x*x > 4}; return compr; }

but auto foo(Range)(int n, Range r) { return mixin(compr!q{x * n ; x <- arr, log(x), x*x > 4}); } is clearly better.
Aug 31 2011
next sibling parent reply kennytm <kennytm gmail.com> writes:
Timon Gehr <timon.gehr gmx.ch> wrote:
 On 09/01/2011 01:03 AM, Timon Gehr wrote:
 On 09/01/2011 12:26 AM, Simen Kjaeraas wrote:
 On Wed, 31 Aug 2011 23:54:04 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:
 
 [2 * x ; x <- iota(10), log(x), x*x > 4]
 
 or, in a library:
 
 compr!q {2 * x ; x <- iota(10), log(x), x*x > 4};
 
 

compr!q{2 * x ; x <- iota(10), log(x), x*x > 4};

The library solution has problems with scope variables.

I know, that is why I have not implemented this yet. Basically it should be possible to implicitly pass a whole scope to a template, or at least, to have local templates.
 Example:
 
 auto foo(Range)(int n, Range r) {
 return compr!q{x * n; x <- arr, log(x), x*x > 4};
 }

Workaround: auto foo(Range)(int n, Range r) { mixin Compr!q{x * n ; x <- arr, log(x), x*x > 4}; return compr; }

but auto foo(Range)(int n, Range r) { return mixin(compr!q{x * n ; x <- arr, log(x), x*x > 4}); } is clearly better.

Hey guys, the expression 'x <- arr' always means "is x less than negation of arr?" in D. Please choose another symbol.
Aug 31 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/01/2011 05:02 AM, kennytm wrote:
 Timon Gehr<timon.gehr gmx.ch>  wrote:
 On 09/01/2011 01:03 AM, Timon Gehr wrote:
 On 09/01/2011 12:26 AM, Simen Kjaeraas wrote:
 On Wed, 31 Aug 2011 23:54:04 +0200, Timon Gehr<timon.gehr gmx.ch>  wrote:

 [2 * x ; x<- iota(10), log(x), x*x>  4]

 or, in a library:

 compr!q {2 * x ; x<- iota(10), log(x), x*x>  4};

compr!q{2 * x ; x<- iota(10), log(x), x*x> 4};

The library solution has problems with scope variables.

I know, that is why I have not implemented this yet. Basically it should be possible to implicitly pass a whole scope to a template, or at least, to have local templates.
 Example:

 auto foo(Range)(int n, Range r) {
 return compr!q{x * n; x<- arr, log(x), x*x>  4};
 }

Workaround: auto foo(Range)(int n, Range r) { mixin Compr!q{x * n ; x<- arr, log(x), x*x> 4}; return compr; }

but auto foo(Range)(int n, Range r) { return mixin(compr!q{x * n ; x<- arr, log(x), x*x> 4}); } is clearly better.

Hey guys, the expression 'x<- arr' always means "is x less than negation of arr?" in D. Please choose another symbol.

Yes, unless it is tokenized in a different way (which is very feasible). Any suggestions? ∈ ?
Sep 01 2011
prev sibling next sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
On Thu, 01 Sep 2011 13:33:20 +0200, Timon Gehr <timon.gehr gmx.ch> wrote=
:

 On 09/01/2011 05:02 AM, kennytm wrote:
 Timon Gehr<timon.gehr gmx.ch>  wrote:
 On 09/01/2011 01:03 AM, Timon Gehr wrote:
 On 09/01/2011 12:26 AM, Simen Kjaeraas wrote:
 On Wed, 31 Aug 2011 23:54:04 +0200, Timon Gehr<timon.gehr gmx.ch> =





 wrote:

 [2 * x ; x<- iota(10), log(x), x*x>  4]

 or, in a library:

 compr!q {2 * x ; x<- iota(10), log(x), x*x>  4};

compr!q{2 * x ; x<- iota(10), log(x), x*x> 4};

The library solution has problems with scope variables.

I know, that is why I have not implemented this yet. Basically it =




 should
 be possible to implicitly pass a whole scope to a template, or at  =




 least,
 to have local templates.

 Example:

 auto foo(Range)(int n, Range r) {
 return compr!q{x * n; x<- arr, log(x), x*x>  4};
 }

Workaround: auto foo(Range)(int n, Range r) { mixin Compr!q{x * n ; x<- arr, log(x), x*x> 4}; return compr; }

but auto foo(Range)(int n, Range r) { return mixin(compr!q{x * n ; x<- arr, log(x), x*x> 4}); } is clearly better.

Hey guys, the expression 'x<- arr' always means "is x less than negat=


 of arr?" in D. Please choose another symbol.

Yes, unless it is tokenized in a different way (which is very feasible=

 Any suggestions? =E2=88=88 ?

You should try to split condition and production rules from element = feeding. Sequence allows for any state = http://www.digitalmars.com/d/2.0/phobos/std_range.html#sequence and has a special symbol 'n'. This idea proved to be efficient as in binaryF= un. Comp!(q{2 * a + b}, q{a * b > 4})(range, range2) would offer the same = degree of freedom as do Haskell's list comprehensions. And even more important you restrict scope issues to only two functions,= = that can use usual predicates. P.S.: Implementation was simpler than I'd thought. https://gist.github.com/1186982#file_list_comprehension.d Sorrowly there is no phobos function to create permutations and = std.functional doesn't extend to n-ary functions. martin
Sep 01 2011
prev sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Thu, 01 Sep 2011 21:11:43 +0200, Martin Nowak <dawg dawgfoto.de> wrote:

 Implementation was simpler than I'd thought.
 https://gist.github.com/1186982#file_list_comprehension.d
 Sorrowly there is no phobos function to create permutations and  
 std.functional doesn't extend to n-ary functions.

That's why Philippe Sigaud has created naryFun: http://svn.dsource.org/projects/dranges/trunk/dranges/docs/functional.html -- Simen
Sep 01 2011
prev sibling next sibling parent "Marco Leise" <Marco.Leise gmx.de> writes:
Am 31.08.2011, 19:36 Uhr, schrieb Simen Kjaeraas <simen.kjaras gmail.com>:

 On Wed, 31 Aug 2011 18:57:01 +0200, bearophile  
 <bearophileHUGS lycos.com> wrote:

 Simen Kjaeraas:

 Next iteration:

      [x; 2 * x; iota(10); x & 1]

If you don't like the syntax I've shown with foreach, then use the Python syntax, it's readable and short and good: [2 * x for x in iota(10) if x & 1] Or: [2 * x for x in 0 .. 10 if x & 1]

I feel it is too much a departure from the style otherwise present in D. It is important not only that the syntax is good in and of itself, but also in the context of D.

I agree with Simen. Use semicolons where you separate logical parts of the expression like in for(each) and commas when you have an argument list or enumerations where all elements have the same context or meaning (hard to explain). So the Python syntax is [<calculation> for <counter> in <range> if <filter>]. I don't know if the filter is actually required, but it sounds like a neat idea. How about multidimensional arrays though?
Aug 31 2011
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Wed, 31 Aug 2011 21:16:37 +0200, bearophile <bearophileHUGS lycos.com=
  =

wrote:
 Simen Kjaeraas:

 I feel it is too much a departure from the style otherwise present in=


 This is why I have shown my original syntax with [foreach() if()].


 It is important not only that the syntax is good in and of itself,
 but also in the context of D.

The syntax you have shown is noisy and hard to read. Array/seq comps are all (or most) a matter of syntax sugar. So if thei=

 syntax is bad, they miss their main point by a mile.

And this is where we disagree. I think the syntax is pretty good, and th= at = the pythonesque syntax sticks out like a sore thumb. Instead of conflating t= he concepts of filter, map and source data, I aimed for a solution that is = = familiar to those accustomed to D, with clear distinctions and the benefits that = = brings. My design is meant to be somewhat similar to for-loops, with the semicolon-separated expressions, and to set builder notation. I kinda wi= sh = 'in' was used in foreach loops (foreach(x in foo){}), as that would be a = perfect fit for the <=3D in the proposed syntax: My notation: [2 * x; x in iota(10); x*x > 4] Set builder notation: {2 =C2=B7 x | x =E2=88=88 =E2=84=95, x=C2=B2 > 4} Personally, I find set builder notation to be very clear and = understandable, and thus worth striving to imitate. In D however, the curly brackets and com= ma operator already have other meanings that we should try not to interfere= = with. -- = Simen
Aug 31 2011
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Wed, 31 Aug 2011 23:16:26 +0200, Timon Gehr <timon.gehr gmx.ch> wrote=
:

 My design is meant to be somewhat similar to for-loops, with the
 semicolon-separated expressions, and to set builder notation. I kinda=


 wish 'in'
 was used in foreach loops (foreach(x in foo){}), as that would be a
 perfect fit
 for the <=3D in the proposed syntax:

 My notation:
 [2 * x; x in iota(10); x*x > 4]

 Set builder notation:
 {2 =C2=B7 x | x =E2=88=88 =E2=84=95, x=C2=B2 > 4}

 Personally, I find set builder notation to be very clear and
 understandable, and
 thus worth striving to imitate. In D however, the curly brackets and =


 comma
 operator already have other meanings that we should try not to interf=


 with.

The meaning of the comma is already quite overloaded, so, if it is =

 better readable, I think using comma would be fine.

I'm afraid that's not really an option in this case. It would conflict w= ith both array literals and the comma operator. e.g. [2 * x, x in iota(10), x*x > 4] would be seen by the compiler as an= attempt to create an array containing three different types. also, [2 * x; x in iota(10); log(x), x*x > 4] would be more complex to = write ([2 * x; x in iota(10); (a){ log(a); return a*a > 4;}(x)], possibly?) -- = Simen
Aug 31 2011
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Wed, 31 Aug 2011 23:54:04 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 [2 * x ; x <- iota(10), log(x), x*x > 4]

 or, in a library:

 compr!q {2 * x ; x <- iota(10), log(x), x*x > 4};

compr!q{2 * x ; x <- iota(10), log(x), x*x > 4};

The library solution has problems with scope variables. Example: auto foo(Range)(int n, Range r) { return compr!q{x * n; x <- arr, log(x), x*x > 4}; } More importantly, IMO, is that the notation does not clearly mark what is what. is that interpreted (iota(10), log(x)) or (log(x), x*x > 4)? -- Simen
Aug 31 2011
prev sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Thu, 01 Sep 2011 01:21:12 +0200, Timon Gehr <timon.gehr gmx.ch> wrote=
:

 auto foo(Range)(int n, Range r) {
      return mixin(compr!q{x * n ; x <- arr, log(x), x*x > 4});
 }

This is one of the reasons I feel mixin templates should require 'mixin'= = only when defined, not when instantiated. Sure, it would be possible to = massively abuse that to make things completely unreadable, but I like to believe programmers other than me are also capable of making some decisions on = their own. ;p I still really don't like the <-. Like Kenny=E2=84=A2 said, it already h= as a = defined meaning that would make (some) sense in that location. <=3D really is no= = better, and those are the two best squiggle combinations for this, I think. Perhaps <: or :=3D could be used. Those have no other meaning in D, and = kinda conveys what we want. Given a good IDE and keyboard, I'd love to use =E2= =88=88, but it's not really feasible.
 More importantly, IMO, is that the notation does not clearly
 mark what is what. is that interpreted (iota(10), log(x))
 or (log(x), x*x > 4)?

It actually does. The comma is a mere separator, and imho it looks ver=

 clean.

D allows you to string expressions together using the comma operator. Wh= at = does this mean: [x; x <: iota(10), iota(9), true] Is that iota(10) to be simply discarded, or is the iota(9) of value as = something other than a range for x to take elements from?
 It is just like the set builder syntax you like. (it even has the curl=

 brackets!)

Well yes, I like that syntax. For D though, the square brackets indicate= array-like behavior, so I would like to use those. Lastly, I feel that a library solution is suboptimal for this, even if = 'mixin' were elidible. Compare: compr!q{2 * x; x <: iota(10), x*x > 4}; [2 * x; x <: iota(10); x*x > 4] -- = Simen
Sep 01 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Both:
this()  disable;

and
 disable this();

will work, right? I don't think the syntax is that bad.

Anyway this is a great addition. I've seen numerous use-cases for
disabling a struct's default ctor.
Aug 29 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 29 Aug 2011 16:41:19 -0400, Simen Kjaeraas  
<simen.kjaras gmail.com> wrote:

 On Mon, 29 Aug 2011 22:22:52 +0200, Walter Bright  
 <newshound2 digitalmars.com> wrote:
 I do think that the "this()  disable;" is an ugly syntax, and I cringe  
 when seeing it. But I can't think of anything better. It does make  
 logical sense given the existence of default construction syntax and  
 the  disable, so in a sense it is just connecting existing dots, which  
 has a compelling value.

I think it's perfect, you just need the words to trade places: disable this(); One could argue that disabled would work better, but that's a can of worms I'm not touching.

I think it works on both sides. And I agree, it's not that cringeworthy, especially if you put it on the left. -Steve
Aug 30 2011
prev sibling next sibling parent reply =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <xtzgzorex gmail.com> writes:
On 29-08-2011 22:22, Walter Bright wrote:
 For the latest dmd,
 https://github.com/D-Programming-Language/dmd/commit/1193f7828b444056c943742daae0a5ccf262272e
 ,
 I've implemented the ability to disable default initialization. This
 makes it practical to implement a library based "NotNull" type without a
 special syntax for it. The rationale for this is (rather than making it
 builtin) one can now build any type that is a restricted subset of
 another type. I suspect that a non-null type is just scratching the
 surface of this ability.

 Here's a draft prototype of the NotNull type:

 import std.c.stdio;

 struct NotNull(P) if (is(typeof({P p = null;})))
 {
 P p;

 this()  disable;

 this(P q)
 {
 assert(q);
 p = q;
 }

 NotNull opAssign(P q)
 {
 assert(q);
 p = q;
 return this;
 }

 alias p this;
 }

 void main()
 {
 int x;
 NotNull!(int*) s = &x;
 *s = 3;
 printf("test1 %d\n", *s);
 s++;
 printf("test2 %d\n", s[-1]);
 }

 What it does is:
 1. disallows instantiation with any type that null doesn't implicitly
 convert to
 2. disables default construction (this is the new feature)
 3. intercepts construction and assignment to enforce not-nullness
 4. uses alias this to forward the rest to the wrapped pointer type

 For example, try this:

 void main()
 {
 NotNull!(int*) s;
 // Error: variable test.main.s initializer required for type NotNull!(int*)
 }

 I do think that the "this()  disable;" is an ugly syntax, and I cringe
 when seeing it. But I can't think of anything better. It does make
 logical sense given the existence of default construction syntax and the
  disable, so in a sense it is just connecting existing dots, which has a
 compelling value.

This is great! I've often had to use classes where I wanted to use structs because the default state would result in bad behavior. Thanks! - Alex
Aug 30 2011
parent bearophile <bearophileHUGS lycos.com> writes:
Alex Rnne Petersen:

 This is great! I've often had to use classes where I wanted to use 
 structs because the default state would result in bad behavior.

Currently NotNull is broken, it's a trap. It doesn't really do what I want, and maybe it does something else that I don't need. Let's see if Walter will improve it a lot. Bye, bearophile
Aug 30 2011
prev sibling next sibling parent reply Christian Kamm <kamm-incasoftware removethis.de> writes:
I want to point out the following bug in the example, because it's *such* a 
common problem and, in my opinion, one of D's warts:

Walter Bright wrote:
      this(P q)
      {
          assert(q);
          p = q;
      }

Try auto d = NotNull!Object(null); you'll get a segmentation fault and not an assert. That's because assert(someclass) does not check if someclass is non-null at all, it verifies its invariant instead! All the asserts should be assert(q !is null);.
Aug 30 2011
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/30/2011 04:58 PM, Christian Kamm wrote:
 I want to point out the following bug in the example, because it's *such* a
 common problem and, in my opinion, one of D's warts:

 Walter Bright wrote:
       this(P q)
       {
           assert(q);
           p = q;
       }

Try auto d = NotNull!Object(null); you'll get a segmentation fault and not an assert. That's because assert(someclass) does not check if someclass is non-null at all, it verifies its invariant instead! All the asserts should be assert(q !is null);.

Imho cast(bool)someclass should be changed to mean someclass !is null && someclass.__invariant() immediately, if that is even how Andrei and Walter expect it to work.
Aug 30 2011
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/30/2011 7:58 AM, Christian Kamm wrote:
 I want to point out the following bug in the example, because it's *such* a
 common problem and, in my opinion, one of D's warts:

 Walter Bright wrote:
       this(P q)
       {
           assert(q);
           p = q;
       }

Try auto d = NotNull!Object(null); you'll get a segmentation fault and not an assert. That's because assert(someclass) does not check if someclass is non-null at all, it verifies its invariant instead! All the asserts should be assert(q !is null);.

Is it really a bug? If you compile in release mode, and q is null, and you use !is null, you'll get a processor hardware halt. Halt or seg fault - there's not much practical difference. As I've said many times, I really don't understand the fear of seg faults. The important thing is that the program stop running as near as possible to the source of the error. Whether it stops because of a software detected error or a hardware detected error is simply not relevant.
Aug 30 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/30/2011 11:07 PM, Walter Bright wrote:
 On 8/30/2011 7:58 AM, Christian Kamm wrote:
 I want to point out the following bug in the example, because it's
 *such* a
 common problem and, in my opinion, one of D's warts:

 Walter Bright wrote:
 this(P q)
 {
 assert(q);
 p = q;
 }

Try auto d = NotNull!Object(null); you'll get a segmentation fault and not an assert. That's because assert(someclass) does not check if someclass is non-null at all, it verifies its invariant instead! All the asserts should be assert(q !is null);.

Is it really a bug? If you compile in release mode, and q is null, and you use !is null, you'll get a processor hardware halt. Halt or seg fault - there's not much practical difference.

You mean, in release mode the assert gets compiled out, and it seg faults in the code sequence that depends on the assertion, right?
 As I've said many times, I really don't understand the fear of seg
 faults. The important thing is that the program stop running as near as
 possible to the source of the error. Whether it stops because of a
 software detected error or a hardware detected error is simply not
 relevant.

I am not afraid of seg faults so much, but in debug mode it is certainly an annoyance, because you don't get the source line and stack trace. I'd say it would be really nice if assert(classreference); would never seg fault during debugging. Is there any practical use for manually running the class invariant?
Aug 30 2011
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/30/2011 4:07 PM, Timon Gehr wrote:
 You mean, in release mode the assert gets compiled out, and it seg faults in
the
 code sequence that depends on the assertion, right?

Yes. Any dereference of a null class ref would cause a seg fault.
 I am not afraid of seg faults so much, but in debug mode it is certainly an
 annoyance, because you don't get the source line and stack trace. I'd say it
 would be really nice if assert(classreference); would never seg fault during
 debugging.

Why? I rely on that for debugging. I run it under the debugger, seg fault, bing the debugger shows where it faulted and a stack trace. It's about 98% of what a debugger is good for. Andrei will reply that there are some environments where you cannot use a debugger, and he's right. But there are other workarounds for that - D doesn't *prevent* you from doing soft debugging.
 Is there any practical use for manually running the class invariant?

Looking for corruption of the data.
Aug 30 2011
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/31/2011 01:19 AM, Walter Bright wrote:
 On 8/30/2011 4:07 PM, Timon Gehr wrote:
 You mean, in release mode the assert gets compiled out, and it seg
 faults in the
 code sequence that depends on the assertion, right?

Yes. Any dereference of a null class ref would cause a seg fault.
 I am not afraid of seg faults so much, but in debug mode it is
 certainly an
 annoyance, because you don't get the source line and stack trace. I'd
 say it
 would be really nice if assert(classreference); would never seg fault
 during
 debugging.

Why? I rely on that for debugging. I run it under the debugger, seg fault, bing the debugger shows where it faulted and a stack trace. It's about 98% of what a debugger is good for.

I don't usually run my programs in a debugger, and with a few assertions now and then this is practical. For me, a seg fault is always more of a productivity blocker than an AssertError.
 Andrei will reply that there are some environments where you cannot use
 a debugger, and he's right. But there are other workarounds for that - D
 doesn't *prevent* you from doing soft debugging.

It does prevent me from writing assert(classreference); for no apparent reason. :)
 Is there any practical use for manually running the class invariant?

Looking for corruption of the data.

But is there any reason why assert(classreference); should not be changed to assert(classreference !is null && classreference.__invariant()); ? Basically, all it does is making debugging more pleasant if you don't run a debugger, and save you some key strokes. (It is also more consistent, because assert(classreference&&1); will never segfault either.)
Aug 30 2011
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:

 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.
Aug 30 2011
parent reply Brad Roberts <braddr slice-2.puremagic.com> writes:
On Tue, 30 Aug 2011, Walter Bright wrote:

 On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:
 
 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.

The frequency with which this comes up and the lack of converts to that point of view ought to tell you something here. :) Would you entertain a pull request with this fairly simple change?
Aug 30 2011
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/30/2011 6:28 PM, Brad Roberts wrote:
 On Tue, 30 Aug 2011, Walter Bright wrote:

 On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:

 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.

The frequency with which this comes up and the lack of converts to that point of view ought to tell you something here. :)

I am simply baffled by it.
 Would you entertain a pull request with this fairly simple change?

It'll add a lot of bloat.
Aug 30 2011
next sibling parent reply Iain Buclaw <ibuclaw ubuntu.com> writes:
== Quote from Brad Roberts (braddr puremagic.com)'s article
 On 8/30/2011 10:35 PM, Walter Bright wrote:
 On 8/30/2011 6:28 PM, Brad Roberts wrote:
 On Tue, 30 Aug 2011, Walter Bright wrote:

 On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:

 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.

The frequency with which this comes up and the lack of converts to that point of view ought to tell you something here. :)

I am simply baffled by it.
 Would you entertain a pull request with this fairly simple change?

It'll add a lot of bloat.


   https://github.com/D-Programming-Language/dmd/pull/358
 (I haven't tried it myself.)
 Later,
 Brad

You should be able to do it with less changes than that. :~) In GDC I settled with assert(e1 != null ? e1.invariant() : _d_assert(...)) I agree, it can be a bit of a nuance if code were to ICE in the runtime library and it takes away several minutes just to find that and pinpoint the source of the problems. Regards
Aug 31 2011
parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Iain Buclaw" <ibuclaw ubuntu.com> wrote in message 
news:j3kmih$2g4m$1 digitalmars.com...
 You should be able to do it with less changes than that. :~)

It's really about 8 lines of code. The rest is all whitespace/alignment changes from making the assert/message generation unconditional.
 In GDC I settled with assert(e1 != null ? e1.invariant() : _d_assert(...))

__assert(msg), tmp.__invariant"
Aug 31 2011
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/31/11 6:46 AM, Steven Schveighoffer wrote:
 On Wed, 31 Aug 2011 01:35:51 -0400, Walter Bright
 <newshound2 digitalmars.com> wrote:

 On 8/30/2011 6:28 PM, Brad Roberts wrote:
 On Tue, 30 Aug 2011, Walter Bright wrote:

 On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:

 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.

The frequency with which this comes up and the lack of converts to that point of view ought to tell you something here. :)

I am simply baffled by it.

Seg faults are not as useful as asserts. It's a fact. If you have a seg fault, you must reproduce the error while in a debugger, or generate a core dump. Reproducing not be possible, or might take considerable time. Any argument against this is revisionist history. Yes, if I go back in time and run it in a debugger for that execution, it would be useful. Yes, if I go back in time and change my shell options to generate a core dump, it would be useful. If you have an assert, you get a stack trace, no need to reproduce the assert in a debugger, or enable non-default settings in your shell. It just gives you the information you need.

Good. I'm equally baffled by Walter's neglect of these fairly obvious arguments :o). Andrei
Aug 31 2011
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/31/2011 4:46 AM, Steven Schveighoffer wrote:
 Seg faults are not as useful as asserts. It's a fact.

You might be surprised, then, that I'll often temporarily replace an assert with a HLT instruction so I can use a debugger on it :-)
 If you have a seg fault,
 you must reproduce the error while in a debugger, or generate a core dump.
 Reproducing not be possible, or might take considerable time. Any argument
 against this is revisionist history. Yes, if I go back in time and run it in a
 debugger for that execution, it would be useful. Yes, if I go back in time and
 change my shell options to generate a core dump, it would be useful. If you
have
 an assert, you get a stack trace, no need to reproduce the assert in a
debugger,
 or enable non-default settings in your shell. It just gives you the information
 you need.

It's also possible for the program to have its own seg fault handler that reads its own symbolic debug info and generates a line number and stack trace. There was a patch to Phobos that did this a while back. Optlink's register dump on a seg fault is not Windows' doing, it installs a seg fault handler which does this.
 So 4 instructions per assert of a class reference (which is arguably not 

15 bytes. And yes, this stuff adds up surprisingly quickly, especially if you're the type that wants to leave the asserts on even in release mode.
Aug 31 2011
next sibling parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 31/08/2011 21:02, Walter Bright wrote:
 On 8/31/2011 4:46 AM, Steven Schveighoffer wrote:
 Seg faults are not as useful as asserts. It's a fact.

You might be surprised, then, that I'll often temporarily replace an assert with a HLT instruction so I can use a debugger on it :-)

What's wrong with 'b _d_assert'? (for gdb at least) You still get back traces, the ability to inspect variables etc. -- Robert http://octarineparrot.com/
Aug 31 2011
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/31/2011 1:38 PM, Robert Clipsham wrote:
 On 31/08/2011 21:02, Walter Bright wrote:
 On 8/31/2011 4:46 AM, Steven Schveighoffer wrote:
 Seg faults are not as useful as asserts. It's a fact.

You might be surprised, then, that I'll often temporarily replace an assert with a HLT instruction so I can use a debugger on it :-)

What's wrong with 'b _d_assert'? (for gdb at least)

Nothing. Just more than one way to do things.
Aug 31 2011
prev sibling next sibling parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 31/08/2011 21:19, Steven Schveighoffer wrote:
 It's also possible for the program to have its own seg fault handler
 that reads its own symbolic debug info and generates a line number and
 stack trace. There was a patch to Phobos that did this a while back.

This would also be a valid option. I have no idea why that wasn't included. Do you?

It's not easy to catch SIGSEGV on linux, there's a library that'll do it but it's GPL (iirc), an equivalent would need writing from scratch if it were to be included in druntime/phobos. -- Robert http://octarineparrot.com/
Aug 31 2011
parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 31/08/2011 22:01, Steven Schveighoffer wrote:
 You can catch sigsegv on linux. It takes one call to signal to register
 a handler.

Are you sure? I could have sworn it didn't work? If it does work, what is the purpose of this library: http://libsigsegv.sourceforge.net/ -- Robert http://octarineparrot.com/
Aug 31 2011
parent "Eric Poggel (JoeCoder)" <dnewsgroup2 yage3d.net> writes:
On 8/31/2011 5:40 PM, Steven Schveighoffer wrote:
 But this program works swimmingly:

Wow. Is there any reason to not use that as the default sigsegv handler?
Aug 31 2011
prev sibling next sibling parent reply kennytm <kennytm gmail.com> writes:
Walter Bright <newshound2 digitalmars.com> wrote:

 You might be surprised, then, that I'll often temporarily replace an
 assert with a HLT instruction so I can use a debugger on it :-)

 It's also possible for the program to have its own seg fault handler that
 reads its own symbolic debug info and generates a line number and stack
 trace. There was a patch to Phobos that did this a while back.
 
 Optlink's register dump on a seg fault is not Windows' doing, it installs
 a seg fault handler which does this.

Please stop caring only about Windows. In Mac OS X because of the broken dSYM and the ancient gdb (having no D support) supplied by Xcode, it's almost impossible to get any valuable information - in particular the __FILE__ and __LINE__ - from the debugger.
 15 bytes. And yes, this stuff adds up surprisingly quickly, especially if
 you're the type that wants to leave the asserts on even in release mode.

Then you should just kill D's RTTI, the TypeInfos from the unused template instances pile up at a much faster rate. 15 bytes mean nothing then the executable size is already in megabytes. Also, the proposal only affects asserts of the form assert(classObj); all other asserts won't have those extra 15 bytes. I believe you're strongly exaggerating, and I'd challenge you to compile Phobos with and without the patch and compare if the size is really bloated a lot.
Aug 31 2011
parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"kennytm" <kennytm gmail.com> wrote in message
 Also, the proposal only affects
 asserts of the form

    assert(classObj);

And assert(ptrToStructWithInvariant);
Aug 31 2011
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/31/2011 1:19 PM, Steven Schveighoffer wrote:
 It's also possible for the program to have its own seg fault handler that
 reads its own symbolic debug info and generates a line number and stack trace.
 There was a patch to Phobos that did this a while back.

This would also be a valid option. I have no idea why that wasn't included. Do you?

Nobody spent the time to do it, mainly because it takes a fairly advanced low level Windows programmer to understand and do it correctly.
 Optlink's register dump on a seg fault is not Windows' doing, it installs a
 seg fault handler which does this.

I predominantly use Linux, so Optlink/Windows tools really don't help. D is supposed to support all OSes not just be useful on Windows.

Sure. It's just that since debuggers work on Linux, there should be no technical reason why this cannot be made to work. Of course it will be very different code than for Windows, but the user wouldn't see that.
 On linux, the shell options are what determine whether the OS generates a core
 dump, and by default they are off. It also does not hook seg faults to start a
 debugger by default, it just says "Segmentation Fault" on the command line and
 gives you a prompt. Useless.

 So 4 instructions per assert of a class reference (which is arguably not

15 bytes. And yes, this stuff adds up surprisingly quickly, especially if you're the type that wants to leave the asserts on even in release mode.

How is this possible? I thought release mode removed asserts?

I misspoke. -release does remove the asserts, but some developers do not use -release because they wish to leave the asserts on in their release builds.
 Currently, the empty main() program is 256k. Add in writeln("hello, world") and
 it adds 500k. Even if I had 10,000 asserts in there for objects, that adds
150k.

 I think bloat can't possibly be a valid concern here.

You'd be surprised. I was surprised to discover that all the asserts in std.datetime caused the unittest build of phobos to exceed all available memory on my (old) FreeBSD box. (Jonathan used his own version of assert that uses more runtime memory, he's got like 7500 in there.) Minimizing the footprint of assert's reduces the resistance people have to using them. Bloat can get to be a big problem if you're using templates, the templates contain asserts, and those templates get inlined. People have regularly complained about the size of D binaries.
 Besides, you ignored my comment about how we can eliminate the bloat. Care to
 respond?

I haven't thought about it.
Aug 31 2011
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, August 31, 2011 14:18 Steven Schveighoffer wrote:
 On Wed, 31 Aug 2011 16:54:50 -0400, Walter Bright
 <newshound2 digitalmars.com> wrote:
 On 8/31/2011 1:19 PM, Steven Schveighoffer wrote:
 I think bloat can't possibly be a valid concern here.

You'd be surprised. I was surprised to discover that all the asserts in std.datetime caused the unittest build of phobos to exceed all available memory on my (old) FreeBSD box. (Jonathan used his own version of assert that uses more runtime memory, he's got like 7500 in there.) Minimizing the footprint of assert's reduces the resistance people have to using them.

I think that: 1. this is likely a combination of code generation issues in dmd and proliferation of asserts with templates/inlining as you say (I still feel there is something wrong with dmd, but I have no evidence). 2. Are more than 10% of those 7500 asserts of object references? I'd be surprised if that were true.

I don't think that there's even one assert on an object reference in there (if nothing else, there are very few classes in std.datetime), and none of them have invariants (some of the structs do but the few classes have very little state). The issue with std.datetime is almost certainly related to the number of templates (including _assertPred, which is definitely templatized). Given the general proliferation of templates in D (and Phobos in particular), dmd definitely needs to be efficient when it comes to compiling templates, and that doesn't generally seem to be the case with regards to memory consumption. That does remind me though that I need to get back to cleaning up std.datetime's unit tests soon. - Jonathan M Davis
Aug 31 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/01/2011 12:58 AM, Jonathan M Davis wrote:
 On Wednesday, August 31, 2011 14:18 Steven Schveighoffer wrote:
 On Wed, 31 Aug 2011 16:54:50 -0400, Walter Bright
 <newshound2 digitalmars.com>  wrote:
 On 8/31/2011 1:19 PM, Steven Schveighoffer wrote:
 I think bloat can't possibly be a valid concern here.

You'd be surprised. I was surprised to discover that all the asserts in std.datetime caused the unittest build of phobos to exceed all available memory on my (old) FreeBSD box. (Jonathan used his own version of assert that uses more runtime memory, he's got like 7500 in there.) Minimizing the footprint of assert's reduces the resistance people have to using them.

I think that: 1. this is likely a combination of code generation issues in dmd and proliferation of asserts with templates/inlining as you say (I still feel there is something wrong with dmd, but I have no evidence). 2. Are more than 10% of those 7500 asserts of object references? I'd be surprised if that were true.

I don't think that there's even one assert on an object reference in there (if nothing else, there are very few classes in std.datetime), and none of them have invariants (some of the structs do but the few classes have very little state). The issue with std.datetime is almost certainly related to the number of templates (including _assertPred, which is definitely templatized). Given the general proliferation of templates in D (and Phobos in particular), dmd definitely needs to be efficient when it comes to compiling templates, and that doesn't generally seem to be the case with regards to memory consumption.

Probably that is because dmd leaks most memory it allocates. I seem to recall that is because of some bugs with interplay of the GC, which were resolved by just turning off the GC. What bugs were those? I consider this an important issue, because I often cause the OS to swap heavily when compiling my Metaprogramming heavy code on my 2GB Ram machine, and I get one or two complete meltdowns each week.
Aug 31 2011
prev sibling next sibling parent reply travert phare.normalesup.org (Christophe) writes:
Maybe think assert should not be rewritten, but opCast!(bool) should 
test if the class reference is null, which would be consistent with 
opEquals
Aug 31 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/31/2011 11:24 PM, Christophe wrote:
 Maybe think assert should not be rewritten, but opCast!(bool) should
 test if the class reference is null, which would be consistent with
 opEquals

It actually does: class C; C x; // assert(x&&1); // no segfault assert(x); // segfault The problem is that assert(classref); is special cased to segfault on invalid class references.
Aug 31 2011
parent travert phare.normalesup.org (Christophe) writes:
OK, so basically there is a confusing special case just to save few bits 
on debug programs...
Sep 01 2011
prev sibling parent David Nadlinger <see klickverbot.at> writes:
On 8/31/11 10:02 PM, Walter Bright wrote:
 On 8/31/2011 4:46 AM, Steven Schveighoffer wrote:
 Seg faults are not as useful as asserts. It's a fact.

You might be surprised, then, that I'll often temporarily replace an assert with a HLT instruction so I can use a debugger on it :-)

I am sure you have your reasons, because, well, you are Walter Bright, but wouldn't this rather be a testament of the inadequacy of the assert() implementation than an argument for – well, what is it exactly you are proposing? Using an assert solely for the segfault it causes as a side effect while the condition is evaluated? All C++ implementations of assert() I have encountered so far provide a way of breaking on failing assertions if the program is being debugged: MSVC-compiled binaries display the »Abort/Retry/Ignore« dialog, and on Posix systems, the SIGABRT raised by abort() is usually caught by the debugger, causing it to break. Have you considered following a similar approach for D? This could be implemented via a custom assert handler, but setAssertHandler has been deprecated in druntime commit fbbd2a7 (the commit message unfortunately doesn't mention a reason, IIRC because DMD doesn't generate a normal stack). David
Aug 31 2011
prev sibling next sibling parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
One thing that hasn't been mentioned is that asserts do not always terminate 
the program.  Should this program segfault?

(Not a great example, because while it does segfault inside A.fun's 
contract, the contract handler's catch-all ignores it and contines, but 
that's a different issue)

class X
{
  invariant() {}
}

class A
{
  void fun(X x)
  in
  {
    assert(x);
  }
  body
  {
  }
}

class B : A
{
  void fun(X x)
  in
  {
  }
  body
  {
  }
}

void main()
{
  auto b = new B();
  b.fun(null);
}
Aug 31 2011
prev sibling next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/01/2011 04:02 PM, Lars T. Kyllingstad wrote:
 On Tue, 30 Aug 2011 22:35:51 -0700, Walter Bright wrote:

 On 8/30/2011 6:28 PM, Brad Roberts wrote:
 On Tue, 30 Aug 2011, Walter Bright wrote:

 On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:

 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.

The frequency with which this comes up and the lack of converts to that point of view ought to tell you something here. :)

I am simply baffled by it.

Walter, I don't think there is a single person in this community, at least not among the vocal ones, who agrees with you on this. It is a case where the language completely fails to do what 99.9% of users expect of it. One question which I don't think has been raised yet is: Why the inconsistency with assert(pointer), which always tests for null? I fear that when D gets broader adoption and people start writing down coding conventions, every one of them will have an item saying "Do NOT use assert(classref), use assert(classref !is null&& classref) instead". -Lars

Actually, to test both for non-null and the invariant, it would have to read version(release){}else{if(classref) assert(classref);else assert(0);} or similar. cast(bool)classref will not check the invariant. To check just for null, assert(classref&&1); is the most concise variant.
Sep 01 2011
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/01/2011 09:02 AM, Lars T. Kyllingstad wrote:
 On Tue, 30 Aug 2011 22:35:51 -0700, Walter Bright wrote:

 On 8/30/2011 6:28 PM, Brad Roberts wrote:
 On Tue, 30 Aug 2011, Walter Bright wrote:

 On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:

 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.

The frequency with which this comes up and the lack of converts to that point of view ought to tell you something here. :)

I am simply baffled by it.

Walter, I don't think there is a single person in this community, at least not among the vocal ones, who agrees with you on this. It is a case where the language completely fails to do what 99.9% of users expect of it.

I think 99.9% would be an underestimation. Who'd be part of the 0.1%? Andrei
Sep 01 2011
next sibling parent reply David Nadlinger <see klickverbot.at> writes:
On 9/1/11 7:17 PM, Martin Nowak wrote:
 Well I agree that segfaults are way more informative than throwing an
 AssertError.
 It does have less issues with incorrect stack unwinding and the core
 file still contains
 register data. This is biased somewhat by using FreeBSD. I never get
 stack traces for exceptions
 but always a core dump.

This behavior could also be enabled by providing a way to provide a custom trace handler. By default, an AssertError would be thrown (or whatever deemed appropriate), but druntime could include an implementation which does the OS-specific things to invoke a debugger/cause it to break. I don't know the details about the aforementioned assertion call stack »issue« though (druntime's setTraceHandler is currently disabled because of it) – Walter, Sean, et al., could you comment on whether a trace handler like described above would be feasible? David
Sep 01 2011
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/1/2011 10:46 AM, Andrej Mitrovic wrote:
 Err, DMD crashes on every error, even this:

 int z = "bla";

 I guess a due to a bad commit?

It works when I try it.
Sep 01 2011
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/1/2011 11:33 AM, Steven Schveighoffer wrote:
 On Thu, 01 Sep 2011 14:27:36 -0400, Daniel Murphy <yebblies nospamgmail.com>
wrote:
 Walter Bright wrote:

 You might be surprised, then, that I'll often temporarily replace an assert
 with a HLT instruction so I can use a debugger on it :-)

Now that's karma :)

I'll often use: *(char*)0=0; in C++ code :-) Very handy.
Sep 01 2011
parent David Nadlinger <see klickverbot.at> writes:
On 9/1/11 9:03 PM, Walter Bright wrote:
 I'll often use:

 *(char*)0=0;

 in C++ code :-)

 Very handy.

The result might not quite be what you expect with all C++ compilers, though: ——— $ clang -Wall test.cpp test.cpp:2:5: warning: indirection of non-volatile null pointer will be deleted, not trap [-Wnull-dereference] *(char*)0=0; ^~~~~~~~~ test.cpp:2:5: note: consider using __builtin_trap() or qualifying pointer with 'volatile' 1 warning generated. ——— David
Sep 01 2011
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-09-01 16:40, Andrei Alexandrescu wrote:
 On 09/01/2011 09:02 AM, Lars T. Kyllingstad wrote:
 Walter, I don't think there is a single person in this community, at
 least not among the vocal ones, who agrees with you on this. It is a
 case where the language completely fails to do what 99.9% of users expect
 of it.

I think 99.9% would be an underestimation. Who'd be part of the 0.1%? Andrei

Depending on how many we are in the community, only Walter :) -- /Jacob Carlborg
Sep 01 2011
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/1/2011 8:52 AM, Jonathan M Davis wrote:
 that
 should already be verified just by calling its functions) rather than something
 like every time that a reference is used,

If you've ever had the joy of trying to track down a memory corruption bug, being able to verify the state of an object without calling member functions on it comes in handy.
Sep 01 2011
prev sibling next sibling parent reply "Eric Poggel (JoeCoder)" <dnewsgroup2 yage3d.net> writes:
On 8/30/2011 7:19 PM, Walter Bright wrote:
 Why? I rely on that for debugging. I run it under the debugger, seg
 fault, bing the debugger shows where it faulted and a stack trace. It's
 about 98% of what a debugger is good for.

What debugger do you use? It's been years, but the last time I investigated debuggers on Windows was with dmd1. The ones I tried (I don't remember which) either added about 30 seconds to my startup time (otherwise 1-2 seconds) or failed to run my program at all. That was the main reason I switched to Tango, since it always prints a stack trace on any error--greatly improved my productivity. But D2 is looking very sweet and I want to switch.
Aug 30 2011
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/30/2011 8:41 PM, Eric Poggel (JoeCoder) wrote:
 What debugger do you use? It's been years, but the last time I investigated
 debuggers on Windows was with dmd1. The ones I tried (I don't remember which)
 either added about 30 seconds to my startup time (otherwise 1-2 seconds) or
 failed to run my program at all.

 That was the main reason I switched to Tango, since it always prints a stack
 trace on any error--greatly improved my productivity. But D2 is looking very
 sweet and I want to switch.

\dmd\windows\bin\windbg.exe
Aug 30 2011
prev sibling next sibling parent Ary Manzana <ary esperanto.org.ar> writes:
On 8/30/11 8:19 PM, Walter Bright wrote:
 On 8/30/2011 4:07 PM, Timon Gehr wrote:
 You mean, in release mode the assert gets compiled out, and it seg
 faults in the
 code sequence that depends on the assertion, right?

Yes. Any dereference of a null class ref would cause a seg fault.
 I am not afraid of seg faults so much, but in debug mode it is
 certainly an
 annoyance, because you don't get the source line and stack trace. I'd
 say it
 would be really nice if assert(classreference); would never seg fault
 during
 debugging.

Why? I rely on that for debugging. I run it under the debugger, seg fault, bing the debugger shows where it faulted and a stack trace. It's about 98% of what a debugger is good for. Andrei will reply that there are some environments where you cannot use a debugger, and he's right. But there are other workarounds for that - D doesn't *prevent* you from doing soft debugging.
 Is there any practical use for manually running the class invariant?

Looking for corruption of the data.

You loose time by having to run it under the debugger. If it just showed you the line of the null pointer it would be much faster to develop.
Aug 31 2011
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/31/2011 7:14 AM, Sean Kelly wrote:
 It's worth mentioning that for some reason, Solaris doesn't protect the
 entire first page of memory--only the zero address. For accesses the where
 the compiler pre-computes the offset and reads that location directly, you
 won't get a segfault if the pointer is null (at least with GCC--haven't
 tested with Sun's compiler). This makes no sense to me and it's been the
 source of a ton of problems in C apps I've found. I'm just mentioning this
 because it's made me leery of relying on the hardware to flag null accesses.

Solaris is dead anyway :-) Also, the virtual function table pointer is at offset 0 in the class object, and that is the most likely one to be referenced. And lastly, different code could be generated for Solaris targets.
Aug 31 2011
parent Iain Buclaw <ibuclaw ubuntu.com> writes:
== Quote from Walter Bright (newshound2 digitalmars.com)'s article
 On 8/31/2011 7:14 AM, Sean Kelly wrote:
 It's worth mentioning that for some reason, Solaris doesn't protect the
 entire first page of memory--only the zero address. For accesses the where
 the compiler pre-computes the offset and reads that location directly, you
 won't get a segfault if the pointer is null (at least with GCC--haven't
 tested with Sun's compiler). This makes no sense to me and it's been the
 source of a ton of problems in C apps I've found. I'm just mentioning this
 because it's made me leery of relying on the hardware to flag null accesses.


Long live openIndiana! :)
Aug 31 2011
prev sibling next sibling parent Bernard Helyer <b.helyer gmail.com> writes:
On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:

 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?
Aug 30 2011
prev sibling next sibling parent Brad Roberts <braddr puremagic.com> writes:
On 8/30/2011 10:35 PM, Walter Bright wrote:
 On 8/30/2011 6:28 PM, Brad Roberts wrote:
 On Tue, 30 Aug 2011, Walter Bright wrote:

 On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:

 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.

The frequency with which this comes up and the lack of converts to that point of view ought to tell you something here. :)

I am simply baffled by it.
 Would you entertain a pull request with this fairly simple change?

It'll add a lot of bloat.

Well, for those that would like it.. yebblies created a pull request with the appropriate changes: https://github.com/D-Programming-Language/dmd/pull/358 (I haven't tried it myself.) Later, Brad
Aug 30 2011
prev sibling next sibling parent Brad Roberts <braddr puremagic.com> writes:
On Wednesday, August 31, 2011 12:06:57 AM, Iain Buclaw wrote:
 == Quote from Brad Roberts (braddr puremagic.com)'s article
 On 8/30/2011 10:35 PM, Walter Bright wrote:
 On 8/30/2011 6:28 PM, Brad Roberts wrote:
 On Tue, 30 Aug 2011, Walter Bright wrote:

 On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:

 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.

The frequency with which this comes up and the lack of converts to that point of view ought to tell you something here. :)

I am simply baffled by it.
 Would you entertain a pull request with this fairly simple change?

It'll add a lot of bloat.



   https://github.com/D-Programming-Language/dmd/pull/358
 (I haven't tried it myself.)
 Later,
 Brad

You should be able to do it with less changes than that. :~) In GDC I settled with assert(e1 != null ? e1.invariant() : _d_assert(...)) I agree, it can be a bit of a nuance if code were to ICE in the runtime library and it takes away several minutes just to find that and pinpoint the source of the problems. Regards

I didn't look at the impl, just the existence. Please comment on the pull request.
Aug 31 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 31 Aug 2011 01:35:51 -0400, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 8/30/2011 6:28 PM, Brad Roberts wrote:
 On Tue, 30 Aug 2011, Walter Bright wrote:

 On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:

 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.

The frequency with which this comes up and the lack of converts to that point of view ought to tell you something here. :)

I am simply baffled by it.

Seg faults are not as useful as asserts. It's a fact. If you have a seg fault, you must reproduce the error while in a debugger, or generate a core dump. Reproducing not be possible, or might take considerable time. Any argument against this is revisionist history. Yes, if I go back in time and run it in a debugger for that execution, it would be useful. Yes, if I go back in time and change my shell options to generate a core dump, it would be useful. If you have an assert, you get a stack trace, no need to reproduce the assert in a debugger, or enable non-default settings in your shell. It just gives you the information you need.
 Would you entertain a pull request with this fairly simple change?

It'll add a lot of bloat.

Define "a lot"... Running a test: class C {} void foo(C c) { int x = 1; version(bloat) { if(c !is null) assert(c); } else { assert(c); } } diff bloat.asm nobloat.asm < mov -4[EBP],EAX < cmp dword ptr -4[EBP],0 < je L17 < mov EAX,-4[EBP] So 4 instructions per assert of a class reference (which is arguably not common) is a lot of bloat? And let's not forget that we are not in release mode here, the bloat does not affect release code. In other words, people are willing to have 4 extra instructions per function in order to avoid needless seg faults. You can actually *DO AWAY* with all this bloat by instead calling a global runtime assertObject function, which first checks that the object is not null, then calls object.__invariant. This is a similar approach that opEquals has. -Steve
Aug 31 2011
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
It's worth mentioning that for some reason, Solaris doesn't protect the enti=
re first page of memory--only the zero address. For accesses the where the c=
ompiler pre-computes the offset and reads that location directly, you won't g=
et a segfault if the pointer is null (at least with GCC--haven't tested with=
 Sun's compiler). This makes no sense to me and it's been the source of a to=
n of problems in C apps I've found. I'm just mentioning this because it's ma=
de me leery of relying on the hardware to flag null accesses.=20

Sent from my iPhone

On Aug 30, 2011, at 4:19 PM, Walter Bright <newshound2 digitalmars.com> wrot=
e:

 On 8/30/2011 4:07 PM, Timon Gehr wrote:
 You mean, in release mode the assert gets compiled out, and it seg faults=


 code sequence that depends on the assertion, right?

Yes. Any dereference of a null class ref would cause a seg fault. =20
 I am not afraid of seg faults so much, but in debug mode it is certainly a=


 annoyance, because you don't get the source line and stack trace. I'd say=


 would be really nice if assert(classreference); would never seg fault dur=


 debugging.

Why? I rely on that for debugging. I run it under the debugger, seg fault,=

f what a debugger is good for.
=20
 Andrei will reply that there are some environments where you cannot use a d=

t *prevent* you from doing soft debugging.
=20
=20
 Is there any practical use for manually running the class invariant?

Looking for corruption of the data. =20

Aug 31 2011
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 31 Aug 2011 16:02:56 -0400, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 8/31/2011 4:46 AM, Steven Schveighoffer wrote:
 Seg faults are not as useful as asserts. It's a fact.

You might be surprised, then, that I'll often temporarily replace an assert with a HLT instruction so I can use a debugger on it :-)

Not at all. If you prefer that mode of debugging, that's fine. But if I have 150 systems running a program I wrote, and one of them fails every week (random which one, and length of time), you'd better bet I'm going to prefer a stack trace to a seg fault. I don't want to wait another week + install debuggers on each of those systems. When I said Seg faults are not as useful as asserts, I meant *unplanned* seg faults. If you plan for a seg fault to occur, (or even an assert) you can use a debugger to capture it.
 If you have a seg fault,
 you must reproduce the error while in a debugger, or generate a core  
 dump.
 Reproducing not be possible, or might take considerable time. Any  
 argument
 against this is revisionist history. Yes, if I go back in time and run  
 it in a
 debugger for that execution, it would be useful. Yes, if I go back in  
 time and
 change my shell options to generate a core dump, it would be useful. If  
 you have
 an assert, you get a stack trace, no need to reproduce the assert in a  
 debugger,
 or enable non-default settings in your shell. It just gives you the  
 information
 you need.

It's also possible for the program to have its own seg fault handler that reads its own symbolic debug info and generates a line number and stack trace. There was a patch to Phobos that did this a while back.

This would also be a valid option. I have no idea why that wasn't included. Do you?
 Optlink's register dump on a seg fault is not Windows' doing, it  
 installs a seg fault handler which does this.

I predominantly use Linux, so Optlink/Windows tools really don't help. D is supposed to support all OSes not just be useful on Windows. On linux, the shell options are what determine whether the OS generates a core dump, and by default they are off. It also does not hook seg faults to start a debugger by default, it just says "Segmentation Fault" on the command line and gives you a prompt. Useless.
  > So 4 instructions per assert of a class reference (which is arguably  
 not common) is a lot of bloat?

 15 bytes. And yes, this stuff adds up surprisingly quickly, especially  
 if you're the type that wants to leave the asserts on even in release  
 mode.

How is this possible? I thought release mode removed asserts? Currently, the empty main() program is 256k. Add in writeln("hello, world") and it adds 500k. Even if I had 10,000 asserts in there for objects, that adds 150k. I think bloat can't possibly be a valid concern here. Besides, you ignored my comment about how we can eliminate the bloat. Care to respond? -Steve
Aug 31 2011
parent bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:

 Currently, the empty main() program is 256k.  Add in writeln("hello,  
 world") and it adds 500k. Even if I had 10,000 asserts in there for  
 objects, that adds 150k.
 
 I think bloat can't possibly be a valid concern here.

Bytes are not enough to measure "bloat". You also need to keep in account where the bloat is. If the bloat means functions that never get called and are physically collocated far from the hot function, then this bloat is not hurting much. If your bloat is instead spread inside hot function and close enough to their hot paths, this bloat will cause problems, and slowdown of the program. Bye, bearophile
Aug 31 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 31 Aug 2011 16:40:02 -0400, Robert Clipsham  
<robert octarineparrot.com> wrote:

 On 31/08/2011 21:19, Steven Schveighoffer wrote:
 It's also possible for the program to have its own seg fault handler
 that reads its own symbolic debug info and generates a line number and
 stack trace. There was a patch to Phobos that did this a while back.

This would also be a valid option. I have no idea why that wasn't included. Do you?

It's not easy to catch SIGSEGV on linux, there's a library that'll do it but it's GPL (iirc), an equivalent would need writing from scratch if it were to be included in druntime/phobos.

You can catch sigsegv on linux. It takes one call to signal to register a handler. The difficult part is unwinding the stack. You can't throw an exception from a signal handler (or reliably do it anyways). Also, the stack may be in some other language (e.g. C). But I do remember the patch and discussion, I thought all these problems were solved, no? Why wasn't this adopted? -Steve
Aug 31 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 31 Aug 2011 16:54:50 -0400, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 8/31/2011 1:19 PM, Steven Schveighoffer wrote:
 It's also possible for the program to have its own seg fault handler  
 that
 reads its own symbolic debug info and generates a line number and  
 stack trace.
 There was a patch to Phobos that did this a while back.

This would also be a valid option. I have no idea why that wasn't included. Do you?

Nobody spent the time to do it, mainly because it takes a fairly advanced low level Windows programmer to understand and do it correctly.

You said there was a patch...? Was it not complete?
 Optlink's register dump on a seg fault is not Windows' doing, it  
 installs a
 seg fault handler which does this.

I predominantly use Linux, so Optlink/Windows tools really don't help. D is supposed to support all OSes not just be useful on Windows.

Sure. It's just that since debuggers work on Linux, there should be no technical reason why this cannot be made to work. Of course it will be very different code than for Windows, but the user wouldn't see that.

Again, starting a debugger because you *know* an assert/segfault is about to occur is vastly different than diagnosing an *unexpected* assert in the field or on a machine that has no debugger. Not all seg faults occur in tightly controlled scenarios. Please stop suggesting debuggers are the only tool we should be using, it's just not always possible in the real world. Otherwise, why have logging, stack traces, etc.
 So 4 instructions per assert of a class reference (which is arguably  

common) is a lot of bloat? 15 bytes. And yes, this stuff adds up surprisingly quickly, especially if you're the type that wants to leave the asserts on even in release mode.

How is this possible? I thought release mode removed asserts?

I misspoke. -release does remove the asserts, but some developers do not use -release because they wish to leave the asserts on in their release builds.

And that leaves in all bounds checking and every method call on any object calls the invariant (whether the derived object defines an invariant or not) without inlining. Let me know who these people are so I never use their products! There are much better tools for this, e.g. enforce.
 Currently, the empty main() program is 256k. Add in writeln("hello,  
 world") and
 it adds 500k. Even if I had 10,000 asserts in there for objects, that  
 adds 150k.

 I think bloat can't possibly be a valid concern here.

You'd be surprised. I was surprised to discover that all the asserts in std.datetime caused the unittest build of phobos to exceed all available memory on my (old) FreeBSD box. (Jonathan used his own version of assert that uses more runtime memory, he's got like 7500 in there.) Minimizing the footprint of assert's reduces the resistance people have to using them.

I think that: 1. this is likely a combination of code generation issues in dmd and proliferation of asserts with templates/inlining as you say (I still feel there is something wrong with dmd, but I have no evidence). 2. Are more than 10% of those 7500 asserts of object references? I'd be surprised if that were true.
 Bloat can get to be a big problem if you're using templates, the  
 templates contain asserts, and those templates get inlined.

 People have regularly complained about the size of D binaries.

Bloat is something to be combated, yes. I am still of the school of thought that smaller exes are much better than bloatier ones, even when we have oodles of memory at our disposal. But we are talking about *debug* builds. I don't think your "people like to ship code compiled in non-release mode" argument holds water like a strainer. And I think people *expect* bloat when they enable debugging information. 15 extra bytes per assert is nothing. writeln adds 500k of "bloat". Do people who use console output debugging (a.k.a. printf debugging) complain about bloat because they added in writeln? I know I don't, and that's my main form of debugging. -Steve
Aug 31 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 31 Aug 2011 17:20:04 -0400, Robert Clipsham  
<robert octarineparrot.com> wrote:

 On 31/08/2011 22:01, Steven Schveighoffer wrote:
 You can catch sigsegv on linux. It takes one call to signal to register
 a handler.

Are you sure? I could have sworn it didn't work? If it does work, what is the purpose of this library: http://libsigsegv.sourceforge.net/

I think that may be a way to override the OS's page fault handling. Here's how a page fault (which is *NOT* a segmentation fault necessarily) works (this might be really badly described, it's been a while since I had to know this): The cpu requests a certain memory address. If the memory isn't in the CPU's cache, it requests the page from the MMU (memory management unit). If the page isn't in memory, it triggers an interrupt to the OS for a page fault. If the OS sees the page is cached on disk in the swap file, it loads the page from disk, and then gets it from the MMU. If the page is not on disk, then the process is sent the SIGSEGV signal. Maybe the library hooks the signal to do it's own retrieval of memory from some other cache. No clue. But this program works swimmingly: import core.stdc.signal; import std.stdio; extern(C) void sigsegvhandler(int sig) { writeln("in signal handler!"); assert(0); } void main() { signal(SIGSEGV, &sigsegvhandler); int * i; *i = 5; } output: in signal handler! core.exception.AssertError testsignal(7): Assertion failure ---------------- ./testsignal(onAssertError+0x2e) [0x8086b9e] ./testsignal(_d_assertm+0x16) [0x80844b6] ./testsignal() [0x8081e96] ./testsignal(sigsegvhandler+0x1e) [0x8081de2] [0x8bf400] ./testsignal(_D2rt6dmain24mainUiPPaZi7runMainMFZv+0x1a) [0x808495e] ./testsignal(_D2rt6dmain24mainUiPPaZi7tryExecMFMDFZvZv+0x20) [0x80845f8] ./testsignal(_D2rt6dmain24mainUiPPaZi6runAllMFZv+0x32) [0x80849a2] ./testsignal(_D2rt6dmain24mainUiPPaZi7tryExecMFMDFZvZv+0x20) [0x80845f8] ./testsignal(main+0x94) [0x80845a4] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0xa79e37] ./testsignal() [0x8081d11] ---------------- -Steve
Aug 31 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 01 Sep 2011 00:18:16 -0400, bearophile <bearophileHUGS lycos.com>  
wrote:

 Steven Schveighoffer:

 Currently, the empty main() program is 256k.  Add in writeln("hello,
 world") and it adds 500k. Even if I had 10,000 asserts in there for
 objects, that adds 150k.

 I think bloat can't possibly be a valid concern here.

Bytes are not enough to measure "bloat". You also need to keep in account where the bloat is. If the bloat means functions that never get called and are physically collocated far from the hot function, then this bloat is not hurting much. If your bloat is instead spread inside hot function and close enough to their hot paths, this bloat will cause problems, and slowdown of the program.

Yeah but, this is in non-release mode. Performance is not as important as finding bugs. -Steve
Sep 01 2011
prev sibling next sibling parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Tue, 30 Aug 2011 22:35:51 -0700, Walter Bright wrote:

 On 8/30/2011 6:28 PM, Brad Roberts wrote:
 On Tue, 30 Aug 2011, Walter Bright wrote:

 On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:

 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.

The frequency with which this comes up and the lack of converts to that point of view ought to tell you something here. :)

I am simply baffled by it.

Walter, I don't think there is a single person in this community, at least not among the vocal ones, who agrees with you on this. It is a case where the language completely fails to do what 99.9% of users expect of it. One question which I don't think has been raised yet is: Why the inconsistency with assert(pointer), which always tests for null? I fear that when D gets broader adoption and people start writing down coding conventions, every one of them will have an item saying "Do NOT use assert(classref), use assert(classref !is null && classref) instead". -Lars
Sep 01 2011
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, September 01, 2011 09:40:13 Andrei Alexandrescu wrote:
 On 09/01/2011 09:02 AM, Lars T. Kyllingstad wrote:
 On Tue, 30 Aug 2011 22:35:51 -0700, Walter Bright wrote:
 On 8/30/2011 6:28 PM, Brad Roberts wrote:
 On Tue, 30 Aug 2011, Walter Bright wrote:
 On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:
 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.

The frequency with which this comes up and the lack of converts to that point of view ought to tell you something here. :)

I am simply baffled by it.

Walter, I don't think there is a single person in this community, at least not among the vocal ones, who agrees with you on this. It is a case where the language completely fails to do what 99.9% of users expect of it.

I think 99.9% would be an underestimation. Who'd be part of the 0.1%?

Someone's big toe agrees with him? ;) Personally, I think that the fact that asserting on a class calls the invariant instead of checking for null is incredibly surprising. It's potentially useful, but it's one of those things that I think suprises pretty much everyone when they first find out. So, having it test for null and _then_ test for the invariant instead of just testing the invariant just makes good sense. And since this debate only seems to be about when people assert on a reference (which can't be all _that_ common honestly - _especially_ if people actually expect it to call the invariant instead of checking for null; that should already be verified just by calling its functions) rather than something like every time that a reference is used, I don't see how Walter has a foot to stand on in this case. Now, if we were debating whether we were going to introduce a general check for null similar to what Java does to get its NullPointerExceptions, _then_ I could see a definite debate. But just for assertions on object references? I just don't see it. - Jonathan M Davis
Sep 01 2011
prev sibling next sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
On Thu, 01 Sep 2011 16:40:13 +0200, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 09/01/2011 09:02 AM, Lars T. Kyllingstad wrote:
 On Tue, 30 Aug 2011 22:35:51 -0700, Walter Bright wrote:

 On 8/30/2011 6:28 PM, Brad Roberts wrote:
 On Tue, 30 Aug 2011, Walter Bright wrote:

 On 8/30/2011 5:08 PM, Bernard Helyer wrote:
 On Tue, 30 Aug 2011 16:19:00 -0700, Walter Bright wrote:

 Looking for corruption of the data.

Why doesn't it check for null, and pass if no invariant is defined?

Because the hardware does the null check for you, which is what a seg fault is.

The frequency with which this comes up and the lack of converts to that point of view ought to tell you something here. :)

I am simply baffled by it.

Walter, I don't think there is a single person in this community, at least not among the vocal ones, who agrees with you on this. It is a case where the language completely fails to do what 99.9% of users expect of it.

I think 99.9% would be an underestimation. Who'd be part of the 0.1%? Andrei

Well I agree that segfaults are way more informative than throwing an AssertError. It does have less issues with incorrect stack unwinding and the core file still contains register data. This is biased somewhat by using FreeBSD. I never get stack traces for exceptions but always a core dump. martin
Sep 01 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Ouch, I've found a naughty bug:

---------------------------
dmd.exe - Application Error
---------------------------
The instruction at "0x0040235d" referenced memory at "0x00000000". The
memory could not be "written".

Code:
void main()
{
    Foo foo;
}

struct Foos
{
     disable this();
}

Notice I've accidentally named my struct "Foos" instead of "Foo". The
console does output the error message:
"Error: undefined identifier Foo, did you mean struct Foos?"

But DMD crashing can't be good. Built from commit
34f2b4ddfafcea4d561e3c1ac8318db79f713638
Sep 01 2011
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Err, DMD crashes on every error, even this:

int z = "bla";

I guess a due to a bad commit?
Sep 01 2011
parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Andrej Mitrovic" <andrej.mitrovich gmail.com> wrote in message 
news:mailman.2630.1314899211.14074.digitalmars-d puremagic.com...
 Err, DMD crashes on every error, even this:

 int z = "bla";

 I guess a due to a bad commit?

Walter left a halt in verror. He does this occasionally.
Sep 01 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 01 Sep 2011 14:27:36 -0400, Daniel Murphy  
<yebblies nospamgmail.com> wrote:

 "Andrej Mitrovic" <andrej.mitrovich gmail.com> wrote in message
 news:mailman.2630.1314899211.14074.digitalmars-d puremagic.com...
 Err, DMD crashes on every error, even this:

 int z = "bla";

 I guess a due to a bad commit?

Walter left a halt in verror. He does this occasionally.

Yesterday in defense of using segfaults instead of asserts: Walter Bright wrote:
 You might be surprised, then, that I'll often temporarily replace an  
 assert with a HLT instruction so I can use a debugger on it :-)

Now that's karma :) -Steve
Sep 01 2011
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Mon, 29 Aug 2011 22:41:19 +0200, Simen Kjaeraas  
<simen.kjaras gmail.com> wrote:

 On Mon, 29 Aug 2011 22:22:52 +0200, Walter Bright  
 <newshound2 digitalmars.com> wrote:

 For the latest dmd,  
 https://github.com/D-Programming-Language/dmd/commit/1193f7828b444056c943
42daae0a5ccf262272e  
 ,
   I've implemented the ability to disable default initialization. This  
 makes it practical to implement a library based "NotNull" type without  
 a special syntax for it. The rationale for this is (rather than making  
 it builtin) one can now build any type that is a restricted subset of  
 another type. I suspect that a non-null type is just scratching the  
 surface of this ability.

Awesome! How does it deal with arrays whose elements have no default constructor? I believe increasing the length of such an array should be an error (which likely also excludes decreasing it except for slices).

Testing concludes that changing the length of the array indeed appends structs to it, with no compile-time error. Should I file this in BugZilla? -- Simen
Aug 30 2011
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, August 31, 2011 16:11 Timon Gehr wrote:
 On 09/01/2011 12:58 AM, Jonathan M Davis wrote:
 On Wednesday, August 31, 2011 14:18 Steven Schveighoffer wrote:
 On Wed, 31 Aug 2011 16:54:50 -0400, Walter Bright
 
 <newshound2 digitalmars.com> wrote:
 On 8/31/2011 1:19 PM, Steven Schveighoffer wrote:
 I think bloat can't possibly be a valid concern here.

You'd be surprised. I was surprised to discover that all the asserts in std.datetime caused the unittest build of phobos to exceed all available memory on my (old) FreeBSD box. (Jonathan used his own version of assert that uses more runtime memory, he's got like 7500 in there.) Minimizing the footprint of assert's reduces the resistance people have to using them.

I think that: 1. this is likely a combination of code generation issues in dmd and proliferation of asserts with templates/inlining as you say (I still feel there is something wrong with dmd, but I have no evidence). 2. Are more than 10% of those 7500 asserts of object references? I'd be surprised if that were true.

I don't think that there's even one assert on an object reference in there (if nothing else, there are very few classes in std.datetime), and none of them have invariants (some of the structs do but the few classes have very little state). The issue with std.datetime is almost certainly related to the number of templates (including _assertPred, which is definitely templatized). Given the general proliferation of templates in D (and Phobos in particular), dmd definitely needs to be efficient when it comes to compiling templates, and that doesn't generally seem to be the case with regards to memory consumption.

Probably that is because dmd leaks most memory it allocates. I seem to recall that is because of some bugs with interplay of the GC, which were resolved by just turning off the GC. What bugs were those? I consider this an important issue, because I often cause the OS to swap heavily when compiling my Metaprogramming heavy code on my 2GB Ram machine, and I get one or two complete meltdowns each week.

LOL. The std.datetime unit tests have been disabled in Windows for some time now, because dmd ran out of memory compiling them. I think that it might be okay now, since the Windows makefile seems to have been changed so that modules are compiled and tested in blocks instead of just using one file, and various improvements have been made to CTFE and the like of late which should reduced memory consumption, but this is definitely a problem. As I understand it, with regards to CTFE at least, the compiler doesn't collect _any_ memory until it's done compiling (since the memory management is easier that way). I don't know if templates have the same problem or not. There are several bugs which have been reported on issues with dmd running out of memory. It doesn't appear to be a high priority issue though, since it hasn't been fixed yet. I do think that there were some recent improvements to it though (for instance, it used to be insanely easy to get startsWith to cause the compiler to run out of memory - since it's a recursive template - but that was supposedly fixed with the last release). - Jonathan M Davis
Aug 31 2011
parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.2616.1314834798.14074.digitalmars-d puremagic.com...
 As I understand
 it, with regards to CTFE at least, the compiler doesn't collect _any_ 
 memory
 until it's done compiling (since the memory management is easier that 
 way). I
 don't know if templates have the same problem or not.

generally involves duplicating the entire expression tree.
 There are several bugs
 which have been reported on issues with dmd running out of memory. It 
 doesn't
 appear to be a high priority issue though, since it hasn't been fixed yet. 
 I
 do think that there were some recent improvements to it though (for 
 instance,
 it used to be insanely easy to get startsWith to cause the compiler to run 
 out
 of memory - since it's a recursive template - but that was supposedly 
 fixed
 with the last release).

This one was fixed, but not by improving memory management in dmd. There was a bug that caused templates without type specializations to be instantiated more than once, and this coupled with a recursive constraint caused something like n! templates to be instantiated (and allocated).
Aug 31 2011