www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Helping the compiler with assumptions while using exceptions

reply simendsjo <simen.endsjo pandavre.com> writes:
I'm having some problems trying to get the best of both worlds here.

void f(Class c) {
   assert(c != null);
   // use c
}

In this example, we tell the compiler that c is never able to be null. 
The compiler can use assertions like this for optimizations (not sure if 
dmd does this though).

But assert is only a debugging tool.
Say we wanted to have this check at runtime too - just in case - so we 
can fail where the problem is.

So we do this:

void f(Class c) {
   enforce(c != null);
   // use c
}

But now the compiler has no idea c will never be null later on (or does 
it...?).

We could always do this:

void f(Class c) {
   assert(c != null);
   enforce(c != null);
   // use c
}

But this is overly verbose.

Or is this not a problem at all? E.g. Use enforce for runtime checks - 
the compiler understands them/won't use asserts for optimizations anyway?
May 30 2011
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
simendsjo:

 void f(Class c) {
    assert(c != null);
    // use c
 }
 
 In this example, we tell the compiler that c is never able to be null. 
 The compiler can use assertions like this for optimizations (not sure if 
 dmd does this though).

I think currently DMD is not using this information, and probably it will keep not using it for some more time. Note: generally asserts go into Contracts. Here it goes in the precondition of f: void f(Class c) in { assert(c != null); } body { // use c }
 void f(Class c) {
    assert(c != null);
    enforce(c != null);
    // use c
 }

This is not meaningful. assert and enforce have different purposes, you don't use both of them at the same time. If f is a inner function/method of your system, then you add asserts inside Contracts to assert your code contains no bugs. If your f is a function/method that's on an "interface" border, so it's used by other users, then you may want to use enforce instead. Bye, bearophile
May 30 2011
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-30 12:49, simendsjo wrote:
 I'm having some problems trying to get the best of both worlds here.
 
 void f(Class c) {
    assert(c != null);
    // use c
 }
 
 In this example, we tell the compiler that c is never able to be null.
 The compiler can use assertions like this for optimizations (not sure if
 dmd does this though).
 
 But assert is only a debugging tool.
 Say we wanted to have this check at runtime too - just in case - so we
 can fail where the problem is.
 
 So we do this:
 
 void f(Class c) {
    enforce(c != null);
    // use c
 }
 
 But now the compiler has no idea c will never be null later on (or does
 it...?).
 
 We could always do this:
 
 void f(Class c) {
    assert(c != null);
    enforce(c != null);
    // use c
 }
 
 But this is overly verbose.
 
 Or is this not a problem at all? E.g. Use enforce for runtime checks -
 the compiler understands them/won't use asserts for optimizations anyway?

assert and enforce serve two very different purposes. assert is used for code verification. You can't assume that it's always be there, because it might be compiled out. As such, the compiler is not going to use it for any optimizations. If an assertion fails, there's a bug in your program. It's used to detect bugs. enforce and exceptions, on the other hand, are _not_ intended for bug detection. They are for error handling. They're generally used in cases where you want to ensure that something is true but where you can't guarantee it (such as when dealing with user input or the file system). You throw an exception if the condition isn't true, and then you deal with it however is appropriate. Generally-speaking, an exception does _not_ indicate a bug. It indicates an error, but that doesn't mean that there's anything wrong with your program. Rather, there was a problem with its input. As such, if you were to use assert in the above code, then you're saying that it's a bug in your program if c is null and having that check go away in release mode is fine, because you're done looking for bugs. It's assumed that the code is correct enough that c will never be null. If, you use enforce, on the other hand, you're saying that it could be perfectly valid for c to be null, but you don't want your f function to ever execute with a null c, so you throw if c is null. Then whatever calls f can handle it in whatever way is appropriate, possibly resulting in an error being printed to the user or another attempt to call f with another value for c which isn't null or whatever is appropriate for your program. By using enforce, you're saying that it's not a bug for c to be null but that it _is_ an error (likely caused by input in some manner) if it occurs. What assert and enforce are doing here are completely different. You shouldn't be choosing between assert and enforce based on whether they'll be around in release mode or not. They're two completely different types of error handling. Now, if it's a bug in your code if c is null, and you want to be absolutely certain that c is never null - even in release mode - then you can use assert(0). e.g. if(c is null) assert(0, "c cannot be null!"); assert(0) is translated to a HLT instruction in release mode. So, it'll kill your program if it's hit. It's intended for cases which should _never_ happen and you want to be absolutely sure that they never do (such as a default case statement or else clause which should never be hit as long as your logic is correct). It's still for bug detection like normal assert, but it's going the extra step of saying that if this line of code is ever reached in release mode, kill the program. It's still completely different from enforce and exceptions which are runtime errors which are potentially handleable. - Jonathan M Davis
May 30 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
Jonathan M Davis wrote:
 On 2011-05-30 12:49, simendsjo wrote:
 I'm having some problems trying to get the best of both worlds here.

 void f(Class c) {
    assert(c != null);
    // use c
 }

 In this example, we tell the compiler that c is never able to be null.
 The compiler can use assertions like this for optimizations (not sure if
 dmd does this though).

 But assert is only a debugging tool.
 Say we wanted to have this check at runtime too - just in case - so we
 can fail where the problem is.

 So we do this:

 void f(Class c) {
    enforce(c != null);
    // use c
 }

 But now the compiler has no idea c will never be null later on (or does
 it...?).

 We could always do this:

 void f(Class c) {
    assert(c != null);
    enforce(c != null);
    // use c
 }

 But this is overly verbose.


Actually, what you have written does not make sense. Correct would be: void f(Class c) { enforce(c != null); assert(c != null); //if c is null, control can never reach this instruction. // use c } You cannot move assert into the contract in your use case, because it is not a contract. It will be valid to call your function with a null reference, it will just throw an exception.
 Or is this not a problem at all? E.g. Use enforce for runtime checks -
 the compiler understands them/won't use asserts for optimizations anyway?


Currently, DMD does not use assertions or exceptions for optimization of code, but it could.
 assert and enforce serve two very different purposes. assert is used for code
 verification. You can't assume that it's always be there, because it might be
 compiled out. As such, the compiler is not going to use it for any
 optimizations. If an assertion fails, there's a bug in your program. It's used
 to detect bugs.

And to give hints to the compiler. This is what his question was about.
 enforce and exceptions, on the other hand, are _not_ intended for bug
 detection. They are for error handling. They're generally used in cases where
 you want to ensure that something is true but where you can't guarantee it
 (such as when dealing with user input or the file system). You throw an
 exception if the condition isn't true, and then you deal with it however is
 appropriate. Generally-speaking, an exception does _not_ indicate a bug. It
 indicates an error, but that doesn't mean that there's anything wrong with
 your program. Rather, there was a problem with its input.
 [snip.]

I think he understands most of that. His question was whether or not the compiler can use enforcements to optimize code. Eg: (silly example) int foo(int x){ enforce(x==0); return x*extremely_costly_pure_function_call(); } A smart compiler could optimize the function to int foo(int x){ enforce(x==0); return 0; } The answer is yes, theoretically it could. (It would either have to have some very advanced code analysis caps, or would just have to treat enforce specially.) DMD does not currently use either enforcements or assertions to generate optimized code, so using an enforcement is sufficient. Timon
May 30 2011
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Timon Gehr:

 The answer is yes, theoretically it could. (It would either have to have some
very
 advanced code analysis caps, or would just have to treat enforce specially.)

Id's not so advanced stuff. Bye, bearophile
May 30 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
 Timon Gehr:

 The answer is yes, theoretically it could. (It would either have to have some
very
 advanced code analysis caps, or would just have to treat enforce specially.)

Id's not so advanced stuff. Bye, bearophile

You are saying that analyzing a function for thrown exceptions and using the conditions on which these exceptions are thrown in a meaningful way across function calls is not advanced? Why don't we have it in that case? Jonathan M Davis wrote:
 A valid point. But as long as enforce isn't inlineable, it can't be optimized.
 And with assert compiled out, I wouldn't expect it to have any affect on
 optimizations. If it did, depending on what was optimized, it could make bugs
 caused by values that would have made the assertion fail were it compiled in
 that much worse. Generally, assert is supposed to have no effect beyond
 throwing if it's compiled in and fails. Now, maybe Walter has a somewhat
 different take on that and maybe assertions could be used for optimizations,
 but the fact that they can be compiled out makes it a bit iffy.

 But sure, as long as it all gets reduced down to if's and flow control, then
 the optimizer can have a field day. But at present, that can't happen with
 enforce (if it ever will), and I question that it can ever be done with assert
 thanks to the guarantees that are associated with it.

assert does not give any guarantees whatsoever. That would be reverse. It takes guarantees from the user. If an assertion fails, the program is buggy no matter if it has been compiled out or not. The behavior of the program is undefined if a compiled out assertion would fail. It is like array bounds. If the check is disabled, the compiler can still assume that all accesses are valid. (a smart compiler could even optimize based on that fact ;)) "It could make bugs so much worse." A bug is a bug, no matter what. Actually, the optimizations based on assert could also make the bug less bad, that depends on the case. And if the bug slipped through when testing with assertions enabled, I'd usually like it to be bad enough to crash the program when it occurs. Timon
May 30 2011
prev sibling parent Brad Roberts <braddr puremagic.com> writes:
On 5/30/2011 2:55 PM, Jonathan M Davis wrote:

 I'd be very surprised to see the compiler ever optimize code based on assert 
 or enforce statement. It's unlikely to do so based on assert simply because 
 the assertion is going to be compiled out. I think that there's a high chance 
 that optimizing based on an assertion that is removed in release mode would 
 actually be violating the guarantees of assert, since then the code without it 
 wouldn't act the same. But you'd have to ask Walter on that one. As for 
 enforce, it'll never happen because enforce is a library function, and the 
 compiler doesn't know anything about it. So, I wouldn't expect to ever seen 
 any compiler optimzations based on assert or enforce.
 
 - Jonathan M Davis

I think you're underselling the possibilities. The only thing preventing enforce from helping optimize in DMD is the lack of inlining of it. That will be improved at some point. Once it's inlined, the rest of the optimizations can easily take advantage of it. There's nothing particularly magic about either assert or enforce. They're just if's and flow control which are an optimizer's bread and butter. Later, Brad
May 30 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-30 14:39, Timon Gehr wrote:
 Jonathan M Davis wrote:
 On 2011-05-30 12:49, simendsjo wrote:
 I'm having some problems trying to get the best of both worlds here.
 
 void f(Class c) {
 
    assert(c != null);
    // use c
 
 }
 
 In this example, we tell the compiler that c is never able to be null.
 The compiler can use assertions like this for optimizations (not sure if
 dmd does this though).
 
 But assert is only a debugging tool.
 Say we wanted to have this check at runtime too - just in case - so we
 can fail where the problem is.
 
 So we do this:
 
 void f(Class c) {
 
    enforce(c != null);
    // use c
 
 }
 
 But now the compiler has no idea c will never be null later on (or does
 it...?).
 
 We could always do this:
 
 void f(Class c) {
 
    assert(c != null);
    enforce(c != null);
    // use c
 
 }
 
 But this is overly verbose.


Actually, what you have written does not make sense. Correct would be: void f(Class c) { enforce(c != null); assert(c != null); //if c is null, control can never reach this instruction. // use c } You cannot move assert into the contract in your use case, because it is not a contract. It will be valid to call your function with a null reference, it will just throw an exception.
 Or is this not a problem at all? E.g. Use enforce for runtime checks -
 the compiler understands them/won't use asserts for optimizations
 anyway?


Currently, DMD does not use assertions or exceptions for optimization of code, but it could.
 assert and enforce serve two very different purposes. assert is used for
 code verification. You can't assume that it's always be there, because
 it might be compiled out. As such, the compiler is not going to use it
 for any optimizations. If an assertion fails, there's a bug in your
 program. It's used to detect bugs.

And to give hints to the compiler. This is what his question was about.
 enforce and exceptions, on the other hand, are _not_ intended for bug
 detection. They are for error handling. They're generally used in cases
 where you want to ensure that something is true but where you can't
 guarantee it (such as when dealing with user input or the file system).
 You throw an exception if the condition isn't true, and then you deal
 with it however is appropriate. Generally-speaking, an exception does
 _not_ indicate a bug. It indicates an error, but that doesn't mean that
 there's anything wrong with your program. Rather, there was a problem
 with its input.
 [snip.]

I think he understands most of that.

The fact that he was talking about essentially swapping out enforce for assert implied that he doesn't.
 His question was whether or not the
 compiler can use enforcements to optimize code. Eg: (silly example)
 
 int foo(int x){
     enforce(x==0);
     return x*extremely_costly_pure_function_call();
 }
 
 A smart compiler could optimize the function to
 
 int foo(int x){
     enforce(x==0);
     return 0;
 }
 
 The answer is yes, theoretically it could. (It would either have to have
 some very advanced code analysis caps, or would just have to treat enforce
 specially.) DMD does not currently use either enforcements or assertions
 to generate optimized code, so using an enforcement is sufficient.

I'd be very surprised to see the compiler ever optimize code based on assert or enforce statement. It's unlikely to do so based on assert simply because the assertion is going to be compiled out. I think that there's a high chance that optimizing based on an assertion that is removed in release mode would actually be violating the guarantees of assert, since then the code without it wouldn't act the same. But you'd have to ask Walter on that one. As for enforce, it'll never happen because enforce is a library function, and the compiler doesn't know anything about it. So, I wouldn't expect to ever seen any compiler optimzations based on assert or enforce. - Jonathan M Davis
May 30 2011
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-30 15:03, Brad Roberts wrote:
 On 5/30/2011 2:55 PM, Jonathan M Davis wrote:
 I'd be very surprised to see the compiler ever optimize code based on
 assert or enforce statement. It's unlikely to do so based on assert
 simply because the assertion is going to be compiled out. I think that
 there's a high chance that optimizing based on an assertion that is
 removed in release mode would actually be violating the guarantees of
 assert, since then the code without it wouldn't act the same. But you'd
 have to ask Walter on that one. As for enforce, it'll never happen
 because enforce is a library function, and the compiler doesn't know
 anything about it. So, I wouldn't expect to ever seen any compiler
 optimzations based on assert or enforce.
 
 - Jonathan M Davis

I think you're underselling the possibilities. The only thing preventing enforce from helping optimize in DMD is the lack of inlining of it. That will be improved at some point. Once it's inlined, the rest of the optimizations can easily take advantage of it. There's nothing particularly magic about either assert or enforce. They're just if's and flow control which are an optimizer's bread and butter.

A valid point. But as long as enforce isn't inlineable, it can't be optimized. And with assert compiled out, I wouldn't expect it to have any affect on optimizations. If it did, depending on what was optimized, it could make bugs caused by values that would have made the assertion fail were it compiled in that much worse. Generally, assert is supposed to have no effect beyond throwing if it's compiled in and fails. Now, maybe Walter has a somewhat different take on that and maybe assertions could be used for optimizations, but the fact that they can be compiled out makes it a bit iffy. But sure, as long as it all gets reduced down to if's and flow control, then the optimizer can have a field day. But at present, that can't happen with enforce (if it ever will), and I question that it can ever be done with assert thanks to the guarantees that are associated with it. - Jonathan M Davis
May 30 2011