www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Automatic invariant generation

reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
The compiler seems to inset an `assert(this !is null, "null 
this");` into my struct.
which is for all intents and purposes.
struct Foo {
     Bar b;
}

struct Bar {
     void* ptr;
}

I tried disabling the invariant but it complained that invariant 
requires a function body.
Is there a way to disable the implicit generations of invariants 
for my code other than -release? Like for a particular subset of 
files (separate invocations of the compiler is not an acceptable 
approach.

The reason being that I do not support global variables (of any 
kind) at the moment in dcompute and the insertion of the string 
literal to the assert breaks that.
Jul 07 2017
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 7 July 2017 at 08:21:50 UTC, Nicholas Wilson wrote:

 The compiler seems to inset an `assert(this !is null, "null 
 this");` into my struct.
 which is for all intents and purposes.
 struct Foo {
     Bar b;
 }

 struct Bar {
     void* ptr;
 }

 I tried disabling the invariant but it complained that 
 invariant requires a function body.
 Is there a way to disable the implicit generations of 
 invariants for my code other than -release? Like for a 
 particular subset of files (separate invocations of the 
 compiler is not an acceptable approach.

 The reason being that I do not support global variables (of any 
 kind) at the moment in dcompute and the insertion of the string 
 literal to the assert breaks that.


Looks like you'd need to do your own hack to disable this 
particular assert.
When it's used in a dcompute-context.
Jul 07 2017
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 7 July 2017 at 08:24:22 UTC, Stefan Koch wrote:

 On Friday, 7 July 2017 at 08:21:50 UTC, Nicholas Wilson wrote:

 [...]


 Looks like you'd need to do your own hack to disable this 
 particular assert.
 When it's used in a dcompute-context.


Problem is it gets decided waaaay before.
Jul 07 2017
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 7 July 2017 at 08:35:28 UTC, Nicholas Wilson wrote:

 On Friday, 7 July 2017 at 08:24:22 UTC, Stefan Koch wrote:

 On Friday, 7 July 2017 at 08:21:50 UTC, Nicholas Wilson wrote:

 [...]


 Looks like you'd need to do your own hack to disable this 
 particular assert.
 When it's used in a dcompute-context.


 Problem is it gets decided waaaay before.


You should be able to hack around it.
LDC should be able to given you a hit when it hits this.
It's the first thing that gets put in the method.
So it has a fixed location.
Hence you can treat is it specially.
Jul 07 2017
prev sibling next sibling parent Nemanja Boric <4burgos gmail.com> writes:
On Friday, 7 July 2017 at 08:21:50 UTC, Nicholas Wilson wrote:

 The compiler seems to inset an `assert(this !is null, "null 
 this");` into my struct.
 which is for all intents and purposes.
 struct Foo {
     Bar b;
 }

 struct Bar {
     void* ptr;
 }

 I tried disabling the invariant but it complained that 
 invariant requires a function body.
 Is there a way to disable the implicit generations of 
 invariants for my code other than -release? Like for a 
 particular subset of files (separate invocations of the 
 compiler is not an acceptable approach.

 The reason being that I do not support global variables (of any 
 kind) at the moment in dcompute and the insertion of the string 
 literal to the assert breaks that.


Related: https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md
Jul 07 2017
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/7/17 4:21 AM, Nicholas Wilson wrote:

 The compiler seems to inset an `assert(this !is null, "null this");` 
 into my struct.
 which is for all intents and purposes.
 struct Foo {
      Bar b;
 }
 
 struct Bar {
      void* ptr;
 }


What? When is this invariant called? I've never heard of a hidden 
invariant being added to structs, structs are supposed to be free of 
such things.

I would call such a thing a bug.

-Steve
Jul 07 2017
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 7 July 2017 at 13:34:20 UTC, Steven Schveighoffer 
wrote:

 On 7/7/17 4:21 AM, Nicholas Wilson wrote:

 The compiler seems to inset an `assert(this !is null, "null 
 this");` into my struct.
 which is for all intents and purposes.
 struct Foo {
      Bar b;
 }
 
 struct Bar {
      void* ptr;
 }


 What? When is this invariant called? I've never heard of a 
 hidden invariant being added to structs, structs are supposed 
 to be free of such things.

 I would call such a thing a bug.

 -Steve


It was added because someone VIP demanded it I guess.
you can see the assert being added using -vcg-ast ;)
Jul 07 2017
next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Friday, July 7, 2017 1:38:13 PM MDT Stefan Koch via Digitalmars-d wrote:

 On Friday, 7 July 2017 at 13:34:20 UTC, Steven Schveighoffer

 wrote:

 On 7/7/17 4:21 AM, Nicholas Wilson wrote:

 The compiler seems to inset an `assert(this !is null, "null
 this");` into my struct.
 which is for all intents and purposes.
 struct Foo {

      Bar b;

 }

 struct Bar {

      void* ptr;

 }


 What? When is this invariant called? I've never heard of a
 hidden invariant being added to structs, structs are supposed
 to be free of such things.

 I would call such a thing a bug.

 -Steve


 It was added because someone VIP demanded it I guess.
 you can see the assert being added using -vcg-ast ;)


What does it even do? I don't see how it makes any sense for _anything_ to
have an invariant if it's not explicitly declared. And honestly, I'm of the
opinion that invariants with structs are borderline useless, because they're
run even before opAssign, meaning that if you ever need to use = void; or
use emplace, then you're screwed if you have an invariant, because it's
bound to fail due to the object not having been initialized previously.
Unfortunately, I couldn't get Walter to agree that it made sense to not call
the invariant prior to opAssign being called - which is why SysTime no
longer has an invariant (it was blowing up in people's code due to emplace
IIRC). As such, it seems that much more stupid for structs to get any kind
fo invariant automatically.

- Jonathan M Davis
Jul 07 2017
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 7 July 2017 at 14:17:34 UTC, Jonathan M Davis wrote:

 What does it even do?


asserts that the this pointer is not null, apparently which is 
annoying because you'd crash anyway. I suppose you might get a 
nicer error message but it doesn't add much.


 I don't see how it makes any sense for _anything_ to have an 
 invariant if it's not explicitly declared.


Worse I can't even  disable it because thats a syntax error.


 And honestly, I'm of the opinion that invariants with structs 
 are borderline useless, because they're run even before 
 opAssign, meaning that if you ever need to use = void; or use 
 emplace, then you're screwed if you have an invariant, because 
 it's bound to fail due to the object not having been 
 initialized previously.


Huh, I didn't know that. That does seems to be purpose defeating 
zealotry.


Unfortunately, I couldn't get Walter to
 agree that it made sense to not call the invariant prior to 
 opAssign being called - which is why SysTime no longer has an 
 invariant (it was blowing up in people's code due to emplace 
 IIRC). As such, it seems that much more stupid for structs to 
 get any kind fo invariant automatically.

 - Jonathan M Davis
Jul 07 2017
next sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Friday, July 7, 2017 2:31:42 PM MDT Nicholas Wilson via Digitalmars-d 
wrote:

 On Friday, 7 July 2017 at 14:17:34 UTC, Jonathan M Davis wrote:

 What does it even do?


 asserts that the this pointer is not null, apparently which is
 annoying because you'd crash anyway. I suppose you might get a
 nicer error message but it doesn't add much.


 I don't see how it makes any sense for _anything_ to have an
 invariant if it's not explicitly declared.


 Worse I can't even  disable it because thats a syntax error.


 And honestly, I'm of the opinion that invariants with structs
 are borderline useless, because they're run even before
 opAssign, meaning that if you ever need to use = void; or use
 emplace, then you're screwed if you have an invariant, because
 it's bound to fail due to the object not having been
 initialized previously.


 Huh, I didn't know that.


I was not pleased to find out about it either, and the result is that I tend
to think that invariants have no business being in structs, much as it would
be desriable for them to be there.


 That does seems to be purpose defeating zealotry.


It's desirable when the object is supposed to be in a good state, because
then you know that when you do the assignment, you'll catch if something
broke the invariant before the assignment. But it's completely undesirable
when the object was purposely uninitialized, and since you can't choose
whether the invariant is run or not, IMHO, _not_ running it would be better,
but I was not persuasive enough:

https://issues.dlang.org/show_bug.cgi?id=5058

I was discussing this issue with someone at dconf, and as a result of that
conversation, I've considered writing a DIP that would allow you to
explicitly skip calling an invariant on a specific assignment (since in
theory, you should know when you're assigning to an unitialized object), but
I haven't had the time to think it through completely, let alone put
together a DIP that might actually be persuasive.

- Jonathan M Davis
Jul 07 2017
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/7/2017 7:31 AM, Nicholas Wilson wrote:

 asserts that the this pointer is not null, apparently which is annoying
because 
 you'd crash anyway. I suppose you might get a nicer error message but it
doesn't 
 add much.


If I recall correctly, the nicer message was the reason. (It was a long time
ago.)

This kind of thing has been asked for, a lot. The current thread entitled:

"All asserts need to have messages attached! Explicit as possible!"

is representative.
Jul 09 2017
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/9/17 5:14 AM, Walter Bright wrote:

 On 7/7/2017 7:31 AM, Nicholas Wilson wrote:

 asserts that the this pointer is not null, apparently which is 
 annoying because you'd crash anyway. I suppose you might get a nicer 
 error message but it doesn't add much.


 
 If I recall correctly, the nicer message was the reason. (It was a long 
 time ago.)
 
 This kind of thing has been asked for, a lot. The current thread entitled:
 
 "All asserts need to have messages attached! Explicit as possible!"
 
 is representative.


Wait, you have stated many many times, a segfault is good enough, it's 
not worth the added cost to do null pointer exceptions (a position I'm 
completely in agreement with). Yet, here is an example of where we have 
effectively added a null pointer exception. At the very least, this 
should be eliminated on Linux and just use the signal handling null 
pointer error mechanism!

Note that there is a significant difference between this situation 
(where you are *adding* an extra check), and the argument to add 
messages to asserts (where you are *already* asserting). For the record, 
I also don't agree that all asserts need messages.

Also noted, even if you inline, the assert is still there. Those who 
want to keep asserts (particularly for safety reasons), will pay this 
penalty.

In things like smart pointer wrappers or converters, many things are 
properties. inlining these is supposed to boil down to simply accessing 
the right field, with zero added cost. That is also a lie.

Or factoring out pieces of a function into more modular member 
functions. Now you are compounding the asserts. When calling another 
member function from within one, there is no reason to re-assert `this 
!is null`.

We need to get rid of this feature, or at least make it optional (and by 
optional, I mean opt-in). And no, having -release remove all asserts is 
not the same as having the ability to eliminate asserts I never wrote or 
wanted. At least the assert shouldn't appear in virtual functions (which 
will never fail because the vlookup will segfault before it ever gets 
there).

I've been using D for 10 years, and have never triggered this assert. 
But I've apparently paid for it that entire time.

-Steve
Jul 09 2017
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 9 July 2017 at 10:37:53 UTC, Steven Schveighoffer 
wrote:

 On 7/9/17 5:14 AM, Walter Bright wrote:

 On 7/7/2017 7:31 AM, Nicholas Wilson wrote:

 asserts that the this pointer is not null, apparently which 
 is annoying because you'd crash anyway. I suppose you might 
 get a nicer error message but it doesn't add much.


 
 If I recall correctly, the nicer message was the reason. (It 
 was a long time ago.)
 
 This kind of thing has been asked for, a lot. The current 
 thread entitled:
 
 "All asserts need to have messages attached! Explicit as 
 possible!"
 
 is representative.


 Wait, you have stated many many times, a segfault is good 
 enough, it's not worth the added cost to do null pointer 
 exceptions (a position I'm completely in agreement with). Yet, 
 here is an example of where we have effectively added a null 
 pointer exception. At the very least, this should be eliminated 
 on Linux and just use the signal handling null pointer error 
 mechanism!

 Note that there is a significant difference between this 
 situation (where you are *adding* an extra check), and the 
 argument to add messages to asserts (where you are *already* 
 asserting). For the record, I also don't agree that all asserts 
 need messages.

 Also noted, even if you inline, the assert is still there. 
 Those who want to keep asserts (particularly for safety 
 reasons), will pay this penalty.

 In things like smart pointer wrappers or converters, many 
 things are properties. inlining these is supposed to boil down 
 to simply accessing the right field, with zero added cost. That 
 is also a lie.

 Or factoring out pieces of a function into more modular member 
 functions. Now you are compounding the asserts. When calling 
 another member function from within one, there is no reason to 
 re-assert `this !is null`.

 We need to get rid of this feature, or at least make it 
 optional (and by optional, I mean opt-in). And no, having 
 -release remove all asserts is not the same as having the 
 ability to eliminate asserts I never wrote or wanted. At least 
 the assert shouldn't appear in virtual functions (which will 
 never fail because the vlookup will segfault before it ever 
 gets there).

 I've been using D for 10 years, and have never triggered this 
 assert. But I've apparently paid for it that entire time.

 -Steve


In C++, but I recently ran across this problem debugging LLVM and 
wasted a good while trying to figure out what was causing the 
crash. This would have been a useful feature (actually if i had 
been comping in debug mode I would have hit that informative 
assert but I was too stubborn to recompile), so it does have its 
place but I think on by default is not it and certainly if it is 
not under my control.

The most annoying part is that the struct will _never_ be passed 
by pointer, its a struct that contains a struct that contains a 
pointer, POD if ever there was. In the older models of OpenCL you 
can't have pointers to pointers _at all_ so by definition the 
assert will never be triggered.
Jul 09 2017
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/9/2017 3:37 AM, Steven Schveighoffer wrote:

 Wait, you have stated many many times, a segfault is good enough, it's not
worth 
 the added cost to do null pointer exceptions (a position I'm completely in 
 agreement with).


That's right.



 Yet, here is an example of where we have effectively added a 
 null pointer exception. > At the very least, this should be eliminated on Linux
 and just use the signal handling null pointer error mechanism!


You're a few years late, as pretty much nobody agreed with me that the
operating 
system handling of it was plenty.



 Note that there is a significant difference between this situation (where you 
 are *adding* an extra check), and the argument to add messages to asserts
(where 
 you are *already* asserting).


It's not really different. It's the desire for ever more messages. I've long 
advocated that a file/line is quite sufficient, but I seem to be in a tiny 
minority of 1. Now 2. :-)



 Also noted, even if you inline, the assert is still there. Those who want to 
 keep asserts (particularly for safety reasons), will pay this penalty.


Yup. Though at one point I advocated an option to replace the assert fails with 
a HLT instruction.



 I've been using D for 10 years, and have never triggered this assert. But I've 
 apparently paid for it that entire time.


It's always worth looking at the assembler output now and then.

---
The thing is, the bloat from all these messages and checks has caused DMD to be 
compiled with -release. Oops, that let through a few bugs.
Jul 09 2017
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/9/17 7:00 AM, Walter Bright wrote:

 On 7/9/2017 3:37 AM, Steven Schveighoffer wrote:



 Yet, here is an example of where we have effectively added a null 
 pointer exception. > At the very least, this should be eliminated on 
 Linux
 and just use the signal handling null pointer error mechanism!


 
 You're a few years late, as pretty much nobody agreed with me that the 
 operating system handling of it was plenty.


I think you misunderstand, we have etc.linux.memoryerror that can 
actually throw an error on a null pointer using the signal handler.

I have a suggestion: eliminate this feature, and add a -npe switch to 
the compiler that errors on any null pointer usage. Asserts will be 
sprinkled in everywhere, but may be useful to someone debugging a nasty 
null pointer segfault somewhere.

-Steve
Jul 09 2017
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 9 July 2017 at 11:37:55 UTC, Steven Schveighoffer 
wrote:

 On 7/9/17 7:00 AM, Walter Bright wrote:

 On 7/9/2017 3:37 AM, Steven Schveighoffer wrote:



 Yet, here is an example of where we have effectively added a 
 null pointer exception. > At the very least, this should be 
 eliminated on Linux
 and just use the signal handling null pointer error mechanism!


 
 You're a few years late, as pretty much nobody agreed with me 
 that the operating system handling of it was plenty.


 I think you misunderstand, we have etc.linux.memoryerror that 
 can actually throw an error on a null pointer using the signal 
 handler.

 I have a suggestion: eliminate this feature, and add a -npe 
 switch to the compiler that errors on any null pointer usage. 
 Asserts will be sprinkled in everywhere, but may be useful to 
 someone debugging a nasty null pointer segfault somewhere.

 -Steve


I think the generated assert(this !is null) has its place, it is 
useful to catch a null this as early as possible but not by 
default. Perhaps debug mode (as in the compiler switch) or a 
switch of its own.
Jul 09 2017
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/9/17 7:45 AM, Nicholas Wilson wrote:

 On Sunday, 9 July 2017 at 11:37:55 UTC, Steven Schveighoffer wrote:

 On 7/9/17 7:00 AM, Walter Bright wrote:

 On 7/9/2017 3:37 AM, Steven Schveighoffer wrote:



 Yet, here is an example of where we have effectively added a null 
 pointer exception. > At the very least, this should be eliminated on 
 Linux
 and just use the signal handling null pointer error mechanism!


 You're a few years late, as pretty much nobody agreed with me that 
 the operating system handling of it was plenty.


 I think you misunderstand, we have etc.linux.memoryerror that can 
 actually throw an error on a null pointer using the signal handler.

 I have a suggestion: eliminate this feature, and add a -npe switch to 
 the compiler that errors on any null pointer usage. Asserts will be 
 sprinkled in everywhere, but may be useful to someone debugging a 
 nasty null pointer segfault somewhere.


 
 I think the generated assert(this !is null) has its place, it is useful 
 to catch a null this as early as possible but not by default. Perhaps 
 debug mode (as in the compiler switch) or a switch of its own.


I'd argue it's not useful at all. I've seen segfaults many many many 
times when debugging D code. I've never seen this error show up. Even 
when developing RedBlackTree (which is full of null pointers to structs 
on every leaf). And it makes sense why too:

1. Structs are generally allocated on the stack, or an array, or inside 
another type. Very rarely would you have a struct pointer that you 
didn't initialize (and was therefore null).
2. Often times, you are using a struct's data members, so you get a 
segfault before ever trying to call a method on it.
3. Classes are where you might see this issue, as people declare a class 
and try to use it without allocating one all the time. But in this case, 
when you are calling a virtual function, the segfault occurs before the 
assert can ever be used.

That being said, if people depend on it for some reason, switching it to 
an opt-in feature would be fine with me. In that case, I suggest just 
going whole-hog, and instrumenting all pointers.

-Steve
Jul 09 2017
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09.07.2017 13:57, Steven Schveighoffer wrote:

 3. Classes are where you might see this issue, as people declare a class 
 and try to use it without allocating one all the time. But in this case, 
 when you are calling a virtual function, the segfault occurs before the 
 assert can ever be used.


What about final member functions?
Jul 09 2017
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/9/17 8:04 AM, Timon Gehr wrote:

 On 09.07.2017 13:57, Steven Schveighoffer wrote:

 3. Classes are where you might see this issue, as people declare a 
 class and try to use it without allocating one all the time. But in 
 this case, when you are calling a virtual function, the segfault 
 occurs before the assert can ever be used.


 
 What about final member functions?


It's possible one has a final class, and does experience this. But it's 
not likely -- most people don't think about virtual by default, and just 
have virtual functions.

Or the final member functions are called after attempting to access 
members or virtual functions.

Or you just learn not to leave class instances uninitialized (most 
have), and you never see the error.

I'll also note that dmd 2.050 still segfaults even for final functions. 
2.060 doesn't. I didn't test in between. So at least as far back as 
2.050, this was a wasted assert.

-Steve
Jul 09 2017
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/9/2017 4:37 AM, Steven Schveighoffer wrote:

 On 7/9/17 7:00 AM, Walter Bright wrote:

 On 7/9/2017 3:37 AM, Steven Schveighoffer wrote:


 

 Yet, here is an example of where we have effectively added a null pointer 
 exception. > At the very least, this should be eliminated on Linux
 and just use the signal handling null pointer error mechanism!


 You're a few years late, as pretty much nobody agreed with me that the 
 operating system handling of it was plenty.


 
 I think you misunderstand, we have etc.linux.memoryerror that can actually
throw 
 an error on a null pointer using the signal handler.


Windows creates a exception, too, on null seg faults.



 I have a suggestion: eliminate this feature, and add a -npe switch to the 
 compiler that errors on any null pointer usage. Asserts will be sprinkled in 
 everywhere, but may be useful to someone debugging a nasty null pointer
segfault 
 somewhere.


It's just redundant to add these.
Jul 10 2017
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sunday, July 9, 2017 4:00:33 AM MDT Walter Bright via Digitalmars-d 
wrote:

 On 7/9/2017 3:37 AM, Steven Schveighoffer wrote:

 Wait, you have stated many many times, a segfault is good enough, it's
 not worth the added cost to do null pointer exceptions (a position I'm
 completely in agreement with).


 That's right.


 Yet, here is an example of where we have effectively added a
 null pointer exception. > At the very least, this should be eliminated
 on Linux and just use the signal handling null pointer error mechanism!


 You're a few years late, as pretty much nobody agreed with me that the
 operating system handling of it was plenty.


What I don't understand about this is that it's not like we have these sort
of checks in general - just in this weirdly specific case. I could
understand that argument if we were doing null pointer checks in general,
but we're not, and you clearly haven't given in to the push for that.

In _C++_, I have literally only seen a null this pointer inside a member
function twice in my career (and the second time it happened, I had to
explain what was happening to my coworkers - some of them being quite
knowledgeable - because they didn't even think that it was possible to call
a function with a null pointer and not have it blow up at the call site). I
have never seen this problem in D. I would be _very_ surprised if you
couldn't just remove this check, and no one would complain because they hit
this problem and didn't get an assertion.


 Note that there is a significant difference between this situation
 (where you are *adding* an extra check), and the argument to add
 messages to asserts (where you are *already* asserting).


 It's not really different. It's the desire for ever more messages. I've
 long advocated that a file/line is quite sufficient, but I seem to be in
 a tiny minority of 1. Now 2. :-)


For some assertions, having more than a file and line number is nice (e.g.
if you have messages for your pre-condition assertions, then you can make it
so that when it fails, the programmer doesn't even need to look at the
source code), but for many, many assertions, having a message doesn't do
much for you IMHO. You need to look at the code anyway, and if it's
asserting an internal thing rather than DbC, then it's usually really not
the sort of thing where a message is going to help particularly. In such
cases, the only real benefit that I see from having an error message that
does more than tell you where the assertion was and what the call stack was
is that if you have a message, and there are several assertions in the same
area of code, then when an assertion fails, you're less likely to mistake
one assertion for another if the source you're looking at doesn't exactly
match the build that the person reporting the issue was using (and that
mismatch isn't always obvious if you're not just building and running your
code locally).

So, to an extent at least, I agree with you, but a number of folks do seem
to think that messages that add no information are better than no messages
for some reason, and unfortunately, there's now a PR to outright require
messages for all assertions in Phbos:

https://github.com/dlang/phobos/pull/5578

- Jonathan M Davis
Jul 09 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/9/2017 6:37 PM, Jonathan M Davis via Digitalmars-d wrote:

 For some assertions, having more than a file and line number is nice (e.g.
 if you have messages for your pre-condition assertions, then you can make it
 so that when it fails, the programmer doesn't even need to look at the
 source code),


He always needs to look at the source code. Asserts are for BUGS, and to fix 
bugs, you gotta look at the code. If asserts are being used as a substitute for 
input/environment errors, they're misused.

In 40 years of asserts, I can't think of ever not visiting the file/line where 
it failed as the first thing I did to debug it.
Jul 10 2017
parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, July 10, 2017 11:26:43 AM MDT Walter Bright via Digitalmars-d 
wrote:

 On 7/9/2017 6:37 PM, Jonathan M Davis via Digitalmars-d wrote:

 For some assertions, having more than a file and line number is nice
 (e.g. if you have messages for your pre-condition assertions, then you
 can make it so that when it fails, the programmer doesn't even need to
 look at the source code),


 He always needs to look at the source code. Asserts are for BUGS, and to
 fix bugs, you gotta look at the code. If asserts are being used as a
 substitute for input/environment errors, they're misused.

 In 40 years of asserts, I can't think of ever not visiting the file/line
 where it failed as the first thing I did to debug it.


Yes, assertions are used for catching bugs, but when they're used for DbC,
the bug is in the caller, and if it's clear from the message that the caller
violated a pre-condition as well as what that pre-condition was, then there
really is no need for the programmer to look at the source code of the
function being called. They just need to look at their code that's calling
it and figure out how they screwed up and passed a bad value. Essentially,
with pre-conditions, it's really the caller's code that's being tested, not
the code where the assertion itself is.

Assertions which test the code that they're actually in, on the other hand,
pretty much always require that you look at that code to figure out what's
going on.

- Jonathan M Davis
Jul 10 2017
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 07.07.2017 16:17, Jonathan M Davis via Digitalmars-d wrote:

 On Friday, July 7, 2017 1:38:13 PM MDT Stefan Koch via Digitalmars-d wrote:

 On Friday, 7 July 2017 at 13:34:20 UTC, Steven Schveighoffer

 wrote:

 On 7/7/17 4:21 AM, Nicholas Wilson wrote:

 The compiler seems to inset an `assert(this !is null, "null
 this");` into my struct.
 which is for all intents and purposes.
 struct Foo {

       Bar b;

 }

 struct Bar {

       void* ptr;

 }


 What? When is this invariant called? I've never heard of a
 hidden invariant being added to structs, structs are supposed
 to be free of such things.

 I would call such a thing a bug.

 -Steve


 It was added because someone VIP demanded it I guess.
 you can see the assert being added using -vcg-ast ;)


 
 What does it even do? I don't see how it makes any sense for _anything_ to
 have an invariant if it's not explicitly declared.


It is not an implicit invariant, it is an implicit precondition of a 
member function. (The 'this' reference is not part of the state, it is a 
function argument.)



 And honestly, I'm of the
 opinion that invariants with structs are borderline useless, because they're
 run even before opAssign, meaning that if you ever need to use = void;


A struct that has public methods that can accept an uninitialized 
instance does not have an invariant.
That does not mean invariants are useless for structs in general.


 or use emplace,


Why would the invariant be called if you use emplace? There are no 
public member functions involved.


 then you're screwed if you have an invariant, because it's
 bound to fail due to the object not having been initialized previously. >
Unfortunately, I couldn't get Walter to agree that it made sense to 


not call

 the invariant prior to opAssign being called


It does not always make sense, and when it does make sense, it is not 
limited to opAssign, so maybe we can have an explicit way to disable 
invariant calls for member functions that do not rely on the object 
invariant. (For non-operator overloads there is an obvious workaround: 
just forward to a private member function using UFCS.)


 - which is why SysTime no
 longer has an invariant (it was blowing up in people's code due to emplace
 IIRC).


The only way this can be a problem is if they emplace a state that does 
not satisfy the invariant and then call opAssign. Was your invariant 
satisfied in the init state?


 As such, it seems that much more stupid for structs to get any kind
 fo invariant automatically.
 
 - Jonathan M Davis
 


That's not what is happening though.
Jul 08 2017
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/7/17 9:38 AM, Stefan Koch wrote:

 On Friday, 7 July 2017 at 13:34:20 UTC, Steven Schveighoffer wrote:

 On 7/7/17 4:21 AM, Nicholas Wilson wrote:

 The compiler seems to inset an `assert(this !is null, "null this");` 
 into my struct.
 which is for all intents and purposes.
 struct Foo {
      Bar b;
 }

 struct Bar {
      void* ptr;
 }


 What? When is this invariant called? I've never heard of a hidden 
 invariant being added to structs, structs are supposed to be free of 
 such things.

 I would call such a thing a bug.


 
 It was added because someone VIP demanded it I guess.


Nope, it's really REALLY old (version 0.167):

https://github.com/dlang/dmd/commit/43a336d81c38817ae545becf02b7459836025c60


 you can see the assert being added using -vcg-ast ;)
 


Hm... it doesn't look like an invariant, it just looks like an inserted 
assert inside every function.

An incorrect assert, IMO:

struct Foo
{
     int x;
     void foo() {}
}

void main()
{
    Foo *foo;
    foo.foo(); // shouldn't assert, wouldn't crash anyway.
}

And since when did we care about null pointers causing segfaults?

Can anyone vouch for this feature?

-Steve
Jul 07 2017
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 7 July 2017 at 14:26:57 UTC, Steven Schveighoffer 
wrote:

 Hm... it doesn't look like an invariant, it just looks like an 
 inserted assert inside every function.


Ahh, thats why I get duplicate asserts when I add an assert.


 And since when did we care about null pointers causing 
 segfaults?

 Can anyone vouch for this feature?

 -Steve


Not me.
Jul 07 2017
prev sibling next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
Steven Schveighoffer wrote:


 Hm... it doesn't look like an invariant, it just looks like an inserted 
 assert inside every function.

 An incorrect assert, IMO:

 struct Foo
 {
      int x;
      void foo() {}
 }

 void main()
 {
     Foo *foo;
     foo.foo(); // shouldn't assert, wouldn't crash anyway.
 }


yeah, this is annoying. while checking for "null this" in *class* method 
may look hacky, i see nothing wrong in "null this" for struct method. tbh, 
i patched that assert (the whole invariant thingy, actually) away long time 
ago, and only remembered about it recently, when my code spit that error in 
vanilla. real PITA, 'cause adding useless checks for "if this struct 
pointer isn't null, then assign what struct method will assign on null, and 
don't forget to sync it when i'll change method, and no, you cannot assert 
in ternaly without deprecated comma, and... no, that code won't be 
converted to 'normal D'."
Jul 07 2017
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
ketmar wrote:


 yeah, this is annoying. while checking for "null this" in *class* method 
 may look hacky


tbh, i see nothing wrong in checking for "null this" even in class methods, 
but this is a completely different story.
Jul 07 2017
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/7/17 2:27 PM, ketmar wrote:

 ketmar wrote:
 

 yeah, this is annoying. while checking for "null this" in *class* 
 method may look hacky


 
 tbh, i see nothing wrong in checking for "null this" even in class 
 methods, but this is a completely different story.


In *final* methods maybe. Virtual methods are going to crash anyway 
before they get to that point.

e.g.:

class C
{
     void foo() {}
     final void bar() {}
}
void main()
{
     C c;
     version(segfault) c.foo();
     version(asserts) c.bar();
}

using -vcg-ast (BTW, I really like this feature!) shows that the assert 
is still put inside foo, even though it will never trigger!

In older versions of dmd, both segfault, I think maybe because the 
virtual invariant is attempted before calling either.

-Steve
Jul 07 2017
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
Steven Schveighoffer wrote:


 On 7/7/17 2:27 PM, ketmar wrote:

 ketmar wrote:
 

 yeah, this is annoying. while checking for "null this" in *class* 
 method may look hacky


 tbh, i see nothing wrong in checking for "null this" even in class 
 methods, but this is a completely different story.


 In *final* methods maybe. Virtual methods are going to crash anyway 
 before they get to that point.


i meant "in manual checking", i.e. "i think that compiler-inserted `assert` 
is not necessary at all". sorry for writing indecipherable engrish. ;-)
Jul 07 2017
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/7/17 4:26 PM, ketmar wrote:

 Steven Schveighoffer wrote:
 

 On 7/7/17 2:27 PM, ketmar wrote:

 ketmar wrote:


 yeah, this is annoying. while checking for "null this" in *class* 
 method may look hacky


 tbh, i see nothing wrong in checking for "null this" even in class 
 methods, but this is a completely different story.


 In *final* methods maybe. Virtual methods are going to crash anyway 
 before they get to that point.


 
 i meant "in manual checking", i.e. "i think that compiler-inserted 
 `assert` is not necessary at all". sorry for writing indecipherable 
 engrish. ;-)


My statement still applies ;)

-Steve
Jul 07 2017
parent ketmar <ketmar ketmar.no-ip.org> writes:
Steven Schveighoffer wrote:


 On 7/7/17 4:26 PM, ketmar wrote:

 Steven Schveighoffer wrote:
 

 On 7/7/17 2:27 PM, ketmar wrote:

 ketmar wrote:


 yeah, this is annoying. while checking for "null this" in *class* 
 method may look hacky


 tbh, i see nothing wrong in checking for "null this" even in class 
 methods, but this is a completely different story.


 In *final* methods maybe. Virtual methods are going to crash anyway 
 before they get to that point.


 i meant "in manual checking", i.e. "i think that compiler-inserted 
 `assert` is not necessary at all". sorry for writing indecipherable 
 engrish. ;-)


 My statement still applies ;)


yeah ;-)
Jul 07 2017
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/7/2017 7:26 AM, Steven Schveighoffer wrote:

 And since when did we care about null pointers causing segfaults?


Remember Tony Hoare's "The Billion Dollar Mistake"? That added a lot of fuel to 
the fire that a null pointer seg fault is supposed to be avoided at all costs, 
leading to wanting a softer, friendlier assert message instead.

I strongly disagree with Hoare - the billion dollar C mistake is having arrays 
relentlessly decay to pointers, leading to endless buffer overflow bugs. Seg 
faults aren't malware vectors.



 Can anyone vouch for this feature?


I'm sure if you're willing to spend a while searching this n.g. database,
you'll 
find a lot.
Jul 09 2017
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/9/17 7:10 AM, Walter Bright wrote:

 On 7/7/2017 7:26 AM, Steven Schveighoffer wrote:

 And since when did we care about null pointers causing segfaults?


 
 Remember Tony Hoare's "The Billion Dollar Mistake"? That added a lot of 
 fuel to the fire that a null pointer seg fault is supposed to be avoided 
 at all costs, leading to wanting a softer, friendlier assert message 
 instead.
 
 I strongly disagree with Hoare - the billion dollar C mistake is having 
 arrays relentlessly decay to pointers, leading to endless buffer 
 overflow bugs. Seg faults aren't malware vectors.


But this isn't that. This is asserting a certain type of pointer (the 
this pointer), which is almost NEVER null, isn't null. It's so 
ineffective, I've never seen it trigger in 10 years. It's basically the 
worst possible place to deviate from the rule of "we don't do null 
pointer exceptions".


 Can anyone vouch for this feature?


 
 I'm sure if you're willing to spend a while searching this n.g. 
 database, you'll find a lot.


I've seen a lot of people argue on the n.g. that null pointer checks 
should be added for every pointer dereference. I've NEVER seen anyone 
argue that upon every member function call, the compiler should verify 
`this` isn't null. Of course, why would they? it's already there :P

I'm seeing a large swath of well-known people arguing in this thread 
that it shouldn't work this way, and 0 people defending it.

-Steve
Jul 09 2017
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 7 July 2017 at 13:34:20 UTC, Steven Schveighoffer 
wrote:

 On 7/7/17 4:21 AM, Nicholas Wilson wrote:

 The compiler seems to inset an `assert(this !is null, "null 
 this");` into my struct.
 which is for all intents and purposes.
 struct Foo {
      Bar b;
 }
 
 struct Bar {
      void* ptr;
 }


 What? When is this invariant called? I've never heard of a 
 hidden invariant being added to structs, structs are supposed 
 to be free of such things.

 I would call such a thing a bug.

 -Steve


I am missing a couple of methods on Foo in that example that in 
turn call the invariant.
That at least can be disable with -release, which while certainly 
not desirable is not a blocker _for me at the moment_. It is 
certainly unacceptable in the long run to demand that if users 
wish to use DCompute that they can't have asserts _in the code 
running in the host_.

However at the moment I'm have more trouble with 
https://forum.dlang.org/thread/ayanaomqklqvknzrljid forum.dlang.org any help
appreciated.
Jul 07 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/7/2017 7:08 AM, Nicholas Wilson wrote:

 It is certainly unacceptable in the 
 long run to demand that if users wish to use DCompute that they can't have 
 asserts _in the code running in the host_.


One thing you can do is replace:

     assert(i > 3);

with:

     if (!(i > 3)) assert(0);

which won't be removed with -release, and the only bloat will be a lovely HLT 
instruction.
Jul 09 2017
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 9 July 2017 at 11:13:02 UTC, Walter Bright wrote:

 On 7/7/2017 7:08 AM, Nicholas Wilson wrote:

 It is certainly unacceptable in the long run to demand that if 
 users wish to use DCompute that they can't have asserts _in 
 the code running in the host_.


 One thing you can do is replace:

     assert(i > 3);

 with:

     if (!(i > 3)) assert(0);

 which won't be removed with -release, and the only bloat will 
 be a lovely HLT instruction.


Of course I could do that for _my_ code , but I am not every end 
user, and even if I was doing the above is not a good use of my 
time.

And this wouldn't be so bad if it were code that I or someone 
else had written, but the fact that the compiler inserts it and I 
didn't ask for it and, save from hacking the compiler, I can't 
make it do otherwise.
Jul 09 2017
prev sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
https://github.com/dlang/dmd/pull/6982
Jul 10 2017