www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Program logic bugs vs input/environmental errors

reply Walter Bright <newshound2 digitalmars.com> writes:
This issue comes up over and over, in various guises. I feel like Yosemite Sam
here:

     https://www.youtube.com/watch?v=hBhlQgvHmQ0

In that vein, Exceptions are for either being able to recover from 
input/environmental errors, or report them to the user of the application.

When I say "They are NOT for debugging programs", I mean they are NOT for 
debugging programs.

assert()s and contracts are for debugging programs.

After all, what would you think of a compiler that spewed out messages like
this:

    > dmd test.d
    test.d(15) Error: missing } thrown from dmd/src/parse.c(283)

?

See:

     https://issues.dlang.org/show_bug.cgi?id=13543

As for the programmer wanting to know where the message "missing }" came from,

     grep -r dmd/src/*.c "missing }"

works nicely. I do that sort of thing all the time. It really isn't a problem.
Sep 27 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:

 As for the programmer wanting to know where the message 
 "missing }" came from,

     grep -r dmd/src/*.c "missing }"

 works nicely. I do that sort of thing all the time. It really 
 isn't a problem.
grep is not useful for the purposes explained in issue 13543 because the file name is often inside a string variable, that is initialized elsewhere or generated in some way. So the exception is useful to know where's the instruction in user code that has tried the failed I/O action, as I've explained in that issue. Bye, bearophile
Sep 27 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/27/2014 4:33 PM, bearophile wrote:
 Walter Bright:

 As for the programmer wanting to know where the message "missing }" came from,

     grep -r dmd/src/*.c "missing }"

 works nicely. I do that sort of thing all the time. It really isn't a problem.
grep is not useful for the purposes explained in issue 13543 because the file name is often inside a string variable, that is initialized elsewhere or generated in some way. So the exception is useful to know where's the instruction in user code that has tried the failed I/O action, as I've explained in that issue.
Even if that is what you wanted, you won't get that from FileException, as it will only show file/lines emanating from calls inside std.file, not from higher level callers. Besides, take a bit of care when formulating a string for exceptions, and you won't have any trouble grepping for it. This isn't rocket science. Presenting internal debugging data to users for input/environmental errors is just bad programming practice. We shouldn't be enshrining it in Phobos and presenting it as a professional way to code.
Sep 27 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Sep 27, 2014 at 04:42:18PM -0700, Walter Bright via Digitalmars-d wrote:
 On 9/27/2014 4:33 PM, bearophile wrote:
Walter Bright:

As for the programmer wanting to know where the message "missing }"
came from,

    grep -r dmd/src/*.c "missing }"

works nicely. I do that sort of thing all the time. It really isn't
a problem.
grep is not useful for the purposes explained in issue 13543 because the file name is often inside a string variable, that is initialized elsewhere or generated in some way. So the exception is useful to know where's the instruction in user code that has tried the failed I/O action, as I've explained in that issue.
Even if that is what you wanted, you won't get that from FileException, as it will only show file/lines emanating from calls inside std.file, not from higher level callers. Besides, take a bit of care when formulating a string for exceptions, and you won't have any trouble grepping for it. This isn't rocket science. Presenting internal debugging data to users for input/environmental errors is just bad programming practice. We shouldn't be enshrining it in Phobos and presenting it as a professional way to code.
My take on this, is that uncaught exceptions are a program bug. Any messages displayed to the user ought to come from a catch block that not only prints the exception message (*without* things like line numbers and stack traces, btw), but also provides context (e.g., "Error in configuration file section 'abc': illegal field value" instead of just "illegal field value" with no context of where it might have been triggered). Uncaught exceptions (which ideally should only be Errors, not Exceptions) are a program bug that ought to be fixed. In the case that somehow one managed to elude your catch blocks, the full debug infodump (source file, line number, stack trace) is useful for users to hand back to you in a bug report, so that you can track down the problem. The user should not be expected to understand the infodump from an uncaught exception, whereas a message printed from a catch block ought to be user-understandable (like "can't open 'myphoto.jpg': file not found", not "internal error on line 12345" which makes no sense to a user). T -- Laissez-faire is a French term commonly interpreted by Conservatives to mean 'lazy fairy,' which is the belief that if governments are lazy enough, the Good Fairy will come down from heaven and do all their work for them.
Sep 27 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/27/2014 4:55 PM, H. S. Teoh via Digitalmars-d wrote:
 My take on this, is that uncaught exceptions are a program bug.
Not to me. Custom messages would be better, but the exception message should be serviceable.
 Uncaught exceptions (which ideally should only be Errors, not
 Exceptions) are a program bug that ought to be fixed. In the case that
 somehow one managed to elude your catch blocks, the full debug infodump
 (source file, line number, stack trace) is useful for users to hand back
 to you in a bug report, so that you can track down the problem. The user
 should not be expected to understand the infodump from an uncaught
 exception, whereas a message printed from a catch block ought to be
 user-understandable (like "can't open 'myphoto.jpg': file not found",
 not "internal error on line 12345" which makes no sense to a user).
Whoa, Camel! You're again thinking of Exceptions as a debugging tool.
Sep 27 2014
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/28/2014 02:40 AM, Walter Bright wrote:
 On 9/27/2014 4:55 PM, H. S. Teoh via Digitalmars-d wrote:
 My take on this, is that uncaught exceptions are a program bug.
Not to me. ...
It is not worth fixing if a program terminates with a stack trace?
Sep 27 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/27/2014 5:54 PM, Timon Gehr wrote:
 On 09/28/2014 02:40 AM, Walter Bright wrote:
 On 9/27/2014 4:55 PM, H. S. Teoh via Digitalmars-d wrote:
 My take on this, is that uncaught exceptions are a program bug.
Not to me. ...
It is not worth fixing if a program terminates with a stack trace?
I never was in favor of adding the stack trace output, either, for the same reason. Exceptions are not programming bugs. For Errors, a case can be made for them.
Sep 27 2014
prev sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Sunday, 28 September 2014 at 00:40:26 UTC, Walter Bright wrote:
 Whoa, Camel! You're again thinking of Exceptions as a debugging 
 tool.
They can be. What if an API you're using throws an exception you didn't expect, and therefore don't handle? This might be considered a logic error if the exception is recoverable and you don't intend the program to abort from that operation.
Sep 28 2014
next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Sunday, 28 September 2014 at 16:16:09 UTC, Sean Kelly wrote:
 On Sunday, 28 September 2014 at 00:40:26 UTC, Walter Bright 
 wrote:
 Whoa, Camel! You're again thinking of Exceptions as a 
 debugging tool.
They can be. What if an API you're using throws an exception you didn't expect, and therefore don't handle? This might be considered a logic error if the exception is recoverable and you don't intend the program to abort from that operation.
Also, I think the idea that a program is created and shipped to an end user is overly simplistic. In the server/cloud programming world, when an error occurs, the client who submitted the request will get a response appropriate for them and the system will also generate log information intended for people working on the system. So things like stack traces and assertion failure information is useful even for production software. Same with any critical system, as I'm sure you're aware. The systems are designed to handle failures in specific ways, but they also have to leave a breadcrumb trail so the underlying problem can be diagnosed and fixed. Internal testing is never perfect, and achieving a high coverage percentage is nearly impossible if the system wasn't designed from the ground up to be testable in such a way (mock frameworks and such).
Sep 28 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 9:23 AM, Sean Kelly wrote:
 Also, I think the idea that a program is created and shipped to an end user is
 overly simplistic.  In the server/cloud programming world, when an error
occurs,
 the client who submitted the request will get a response appropriate for them
 and the system will also generate log information intended for people working
on
 the system.  So things like stack traces and assertion failure information is
 useful even for production software.  Same with any critical system, as I'm
sure
 you're aware.  The systems are designed to handle failures in specific ways,
but
 they also have to leave a breadcrumb trail so the underlying problem can be
 diagnosed and fixed.  Internal testing is never perfect, and achieving a high
 coverage percentage is nearly impossible if the system wasn't designed from the
 ground up to be testable in such a way (mock frameworks and such).
Then use assert(). That's just what it's for.
Sep 28 2014
next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Sunday, 28 September 2014 at 17:33:38 UTC, Walter Bright wrote:
 On 9/28/2014 9:23 AM, Sean Kelly wrote:
 Also, I think the idea that a program is created and shipped 
 to an end user is overly simplistic.  In the server/cloud 
 programming world, when an error occurs, the client who 
 submitted the request will get a response appropriate for them 
 and the system will also generate log information intended for 
 people working on the system.  So things like stack traces and 
 assertion failure information is useful even for production 
 software.  Same with any critical system, as I'm sure you're 
 aware.  The systems are designed to handle failures in 
 specific ways, but they also have to leave a breadcrumb trail 
 so the underlying problem can be diagnosed and fixed.  
 Internal testing is never perfect, and achieving a high 
 coverage percentage is nearly impossible if the system wasn't 
 designed from the ground up to be testable in such a way (mock 
 frameworks and such).
Then use assert(). That's just what it's for.
What if I don't want to be forced to abort the program in the event of such an error?
Sep 28 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 12:33 PM, Sean Kelly wrote:
 Then use assert(). That's just what it's for.
What if I don't want to be forced to abort the program in the event of such an error?
Then we are back to the discussion about can a program continue after a logic error is uncovered, or not. In any program, the programmer must decide if an error is a bug or not, before shipping it. Trying to avoid making this decision leads to confusion and using the wrong techniques to deal with it. A program bug is, by definition, unknown and unanticipated. The idea that one can "recover" from it is fundamentally wrong. Of course, in D one can try and recover from them anyway, but you're on your own trying that, just as you're on your own when casting integers to pointers. On the other hand, input/environmental errors must be anticipated and can often be recovered from. But presenting debug traces to the users for these implies at the very least a sloppily engineered product, in my not so humble opinion :-)
Sep 28 2014
next sibling parent reply Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 28/09/14 22:13, Walter Bright via Digitalmars-d wrote:
 On 9/28/2014 12:33 PM, Sean Kelly wrote:
 Then use assert(). That's just what it's for.
What if I don't want to be forced to abort the program in the event of such an error?
Then we are back to the discussion about can a program continue after a logic error is uncovered, or not. In any program, the programmer must decide if an error is a bug or not, before shipping it. Trying to avoid making this decision leads to confusion and using the wrong techniques to deal with it. A program bug is, by definition, unknown and unanticipated. The idea that one can "recover" from it is fundamentally wrong. Of course, in D one can try and recover from them anyway, but you're on your own trying that, just as you're on your own when casting integers to pointers.
Allowing for your "you can try ..." remarks, I still feel this doesn't really cover the practical realities of how some applications need to behave. Put it this way: suppose we're writing the software for a telephone exchange, which is handling thousands of simultaneous calls. If an Error is thrown inside the part of the code handling one single call, is it correct to bring down everyone else's call too? I appreciate that you might tell me "You need to find a different means of error handling that can distinguish errors that are recoverable", but the bottom line is, in such a scenario it's not possible to completely rule out an Error being thrown (an obvious cause would be an assert that gets triggered because the programmer forgot to put a corresponding enforce() statement at a higher level in the code). However, it's clearly very desirable in this use-case for the application to keep going if at all possible and for any problem, even an Error, to be contained in its local context if we can do so. (By "local context", in practice this probably means a thread or fiber or some other similar programming construct.) Sean's touched on this in the current thread with his reference to Erlang, and I remember that he and Dicebot brought the issue up in an earlier discussion on the Error vs. Exception question, but I don't recall that discussion having any firm conclusion, and I think it's important to address; we can't simply take "An Error is unrecoverable" as a point of principle for every application. (Related note: If I recall right, an Error or uncaught Exception thrown within a thread or fiber will not actually bring the application down, only cause that thread/fiber to hang, without printing any indication of anything going wrong. So on a purely practical basis, it can be essential for the top-level code of a thread or fiber to have a catch {} block for both Errors and Exceptions, just in order to be able to report what has happened effectively.)
Sep 28 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 3:51 PM, Joseph Rushton Wakeling via Digitalmars-d wrote:
 However, it's clearly very desirable in this use-case for the application to
 keep going if at all possible and for any problem, even an Error, to be
 contained in its local context if we can do so.  (By "local context", in
 practice this probably means a thread or fiber or some other similar
programming
 construct.)
If the program has entered an unknown state, its behavior from then on cannot be predictable. There's nothing I or D can do about that. D cannot officially endorse such a practice, though D being a systems programming language it will let you do what you want. I would not even consider such a practice for a program that is in charge of anything that could result in injury, death, property damage, security breaches, etc.
Sep 28 2014
next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 29 September 2014 at 00:09:59 UTC, Walter Bright wrote:
 On 9/28/2014 3:51 PM, Joseph Rushton Wakeling via Digitalmars-d 
 wrote:
 However, it's clearly very desirable in this use-case for the 
 application to
 keep going if at all possible and for any problem, even an 
 Error, to be
 contained in its local context if we can do so.  (By "local 
 context", in
 practice this probably means a thread or fiber or some other 
 similar programming
 construct.)
If the program has entered an unknown state, its behavior from then on cannot be predictable. There's nothing I or D can do about that. D cannot officially endorse such a practice, though D being a systems programming language it will let you do what you want. I would not even consider such a practice for a program that is in charge of anything that could result in injury, death, property damage, security breaches, etc.
Well... suppose you design a system with redundancy such that an error in a specific process isn't enough to bring down the system. Say it's a quorum method or whatever. In the instance that a process goes crazy, I would argue that the system is in an undefined state but a state that it's designed specifically to handle, even if that state can't be explicitly defined at design time. Now if enough things go wrong at once the whole system will still fail, but it's about building systems with the expectation that errors will occur. They may even be logic errors--I think it's kind of irrelevant at that point. Even a network of communicating processes, one getting in a bad state can theoretically poison the entire system and you're often not in a position to simply shut down the whole thing and wait for a repairman. And simply rebooting the system if it's a bad sensor that's causing the problem just means a pause before another failure cascade. I think any modern program designed to run continuously (increasingly the typical case) must be designed with some degree of resiliency or self-healing in mind. And that means planning for and limiting the scope of undefined behavior.
Sep 28 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 6:39 PM, Sean Kelly wrote:
 Well... suppose you design a system with redundancy such that an error in a
 specific process isn't enough to bring down the system.  Say it's a quorum
 method or whatever.  In the instance that a process goes crazy, I would argue
 that the system is in an undefined state but a state that it's designed
 specifically to handle, even if that state can't be explicitly defined at
design
 time.  Now if enough things go wrong at once the whole system will still fail,
 but it's about building systems with the expectation that errors will occur.
 They may even be logic errors--I think it's kind of irrelevant at that point.

 Even a network of communicating processes, one getting in a bad state can
 theoretically poison the entire system and you're often not in a position to
 simply shut down the whole thing and wait for a repairman.  And simply
rebooting
 the system if it's a bad sensor that's causing the problem just means a pause
 before another failure cascade.  I think any modern program designed to run
 continuously (increasingly the typical case) must be designed with some degree
 of resiliency or self-healing in mind.  And that means planning for and
limiting
 the scope of undefined behavior.
I've said that processes are different, because the scope of the effects is limited by the hardware. If a system with threads that share memory cannot be restarted, there are serious problems with the design of it, because a crash and the necessary restart are going to happen sooner or later, probably sooner. I don't believe that the way to get 6 sigma reliability is by ignoring errors and hoping. Airplane software is most certainly not done that way. I recall Toyota got into trouble with their computer controlled cars because of their idea of handling of inevitable bugs and errors. It was one process that controlled everything. When something unexpected went wrong, it kept right on operating, any unknown and unintended consequences be damned. The way to get reliable systems is to design to accommodate errors, not pretend they didn't happen, or hope that nothing else got affected, etc. In critical software systems, that means shut down and restart the offending system, or engage the backup. There's no other way that works.
Sep 28 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 29 September 2014 at 02:57:03 UTC, Walter Bright wrote:
 I've said that processes are different, because the scope of 
 the effects is limited by the hardware.

 If a system with threads that share memory cannot be restarted, 
 there are serious problems with the design of it, because a 
 crash and the necessary restart are going to happen sooner or 
 later, probably sooner.
Right. But if the condition that caused the restart persists, the process can end up in a cascading restart scenario. Simply restarting on error isn't necessarily enough.
 I don't believe that the way to get 6 sigma reliability is by 
 ignoring errors and hoping. Airplane software is most certainly 
 not done that way.
I believe I was arguing the opposite. More to the point, I think it's necessary to expect undefined behavior to occur and to plan for it. I think we're on the same page here and just miscommunicating.
 I recall Toyota got into trouble with their computer controlled 
 cars because of their idea of handling of inevitable bugs and 
 errors. It was one process that controlled everything. When 
 something unexpected went wrong, it kept right on operating, 
 any unknown and unintended consequences be damned.

 The way to get reliable systems is to design to accommodate 
 errors, not pretend they didn't happen, or hope that nothing 
 else got affected, etc. In critical software systems, that 
 means shut down and restart the offending system, or engage the 
 backup.
My point was that it's often more complicated than that. There have been papers written on self-repairing systems, for example, and ways to design systems that are inherently durable when it comes to even internal errors. I think what I'm trying to say is that simply aborting on error is too brittle in some cases, because it only deals with one vector--memory corruption that is unlikely to reoccur. But I've watched always-on systems fall apart from some unexpected ongoing situation, and simply restarting doesn't actually help.
Sep 28 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 9:03 PM, Sean Kelly wrote:
 On Monday, 29 September 2014 at 02:57:03 UTC, Walter Bright wrote:
 Right.  But if the condition that caused the restart persists, the process can
 end up in a cascading restart scenario.  Simply restarting on error isn't
 necessarily enough.
When it isn't enough, use the "engage the backup" technique.
 I don't believe that the way to get 6 sigma reliability is by ignoring errors
 and hoping. Airplane software is most certainly not done that way.
I believe I was arguing the opposite. More to the point, I think it's necessary to expect undefined behavior to occur and to plan for it. I think we're on the same page here and just miscommunicating.
Assuming that the program bug couldn't have affected other threads is relying on hope. Bugs happen when the program went into an unknown and unanticipated state. You cannot know, until after you debug it, what other damage the fault caused, or what other damage caused the detected fault.
 My point was that it's often more complicated than that.  There have been
papers
 written on self-repairing systems, for example, and ways to design systems that
 are inherently durable when it comes to even internal errors.
I confess much skepticism about such things when it comes to software. I do know how reliable avionics software is done, and that stuff does work even in the face of all kinds of bugs, damage, and errors. I'll be betting my life on that tomorrow :-) Would you bet your life on software that had random divide by 0 bugs in it that were just ignored in the hope that they weren't serious? Keep in mind that software is rather unique in that a single bit error in a billion bytes can render the software utterly demented. Remember the Apollo 11 lunar landing, when the descent computer software started showing self-detected faults? Armstrong turned it off and landed manually. He wasn't going to bet his ass that the faults could be ignored. You and I wouldn't, either.
 I think what I'm
 trying to say is that simply aborting on error is too brittle in some cases,
 because it only deals with one vector--memory corruption that is unlikely to
 reoccur.  But I've watched always-on systems fall apart from some unexpected
 ongoing situation, and simply restarting doesn't actually help.
In such a situation, ignoring the error seems hardly likely to do any better.
Sep 28 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 29 September 2014 at 05:15:14 UTC, Walter Bright wrote:
 I confess much skepticism about such things when it comes to 
 software. I do know how reliable avionics software is done, and 
 that stuff does work even in the face of all kinds of bugs, 
 damage, and errors. I'll be betting my life on that tomorrow :-)

 Would you bet your life on software that had random divide by 0 
 bugs in it that were just ignored in the hope that they weren't 
 serious? Keep in mind that software is rather unique in that a 
 single bit error in a billion bytes can render the software 
 utterly demented.
I'm not saying the errors should be ignored, but rather that there are other approaches to handling errors besides (or in addition to) terminating the process. For me, the single most important thing is detecting errors as soon as possible so corrective action can be taken before things go too far south (so hooray for contracts!). From there, the proper response depends on the error detected and the type of system I'm working on. Like with persistent stateful systems, even if a restart occurs can you assume that the persisted state is valid? With a mesh of communicating systems, if one node goes insane, what impact might it have on other nodes in the network? I think the definition of what constitutes an interdependent system is application defined. And yes, I know all about tiny bugs creating insane problems. With event-based asynchronous programming, the most common serious bugs I encounter memory corruption problems from dangling pointers, and the only way to find and fix these is by analyzing gigabytes worth of log files to try and unpack what happened after the fact. Spending a day looking at the collateral damage from what ultimately turns out to be a backwards conditional expression in an error handler somewhere gives a pretty healthy respect for the brittleness of memory unsafe code. This is one area where having a GC is an enormous win.
 Remember the Apollo 11 lunar landing, when the descent computer 
 software started showing self-detected faults? Armstrong turned 
 it off and landed manually. He wasn't going to bet his ass that 
 the faults could be ignored. You and I wouldn't, either.
And this is great if there's a human available to take over. But what if this were a space probe?
 I think what I'm trying to say is that simply aborting on 
 error is too brittle in some cases, because it only deals with 
 one vector--memory corruption that is unlikely to reoccur.  
 But I've watched always-on systems fall apart from some 
 unexpected ongoing situation, and simply restarting doesn't 
 actually help.
In such a situation, ignoring the error seems hardly likely to do any better.
Again, not ignoring, but rather that a restart may not be the appropriate response to the problem. Or it may be a part of the appropriate response, but other things need to happen as well.
Sep 29 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/29/2014 12:09 PM, Sean Kelly wrote:
 I'm not saying the errors should be ignored, but rather that
 there are other approaches to handling errors besides (or in
 addition to) terminating the process.
Those "other methods" are not robust and are not acceptable for a system claiming to be robust.
 Remember the Apollo 11 lunar landing, when the descent computer software
 started showing self-detected faults? Armstrong turned it off and landed
 manually. He wasn't going to bet his ass that the faults could be ignored. You
 and I wouldn't, either.
And this is great if there's a human available to take over. But what if this were a space probe?
A space probe would have either: 1. an independent backup system. -- or -- 2. a "can't fail" system that fails and the probe will be lost http://www.drdobbs.com/architecture-and-design/safe-systems-from-unreliable-parts/228701716 Any system designed around software that "cannot fail" is doomed from the start. You cannot write such software, and nobody else can, either. Attempting to do so will only reap expensive and bitter lessons :-(
Oct 04 2014
prev sibling next sibling parent reply "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Monday, 29 September 2014 at 04:03:37 UTC, Sean Kelly wrote:
 On Monday, 29 September 2014 at 02:57:03 UTC, Walter Bright

 Right.  But if the condition that caused the restart persists, 
 the process can end up in a cascading restart scenario.  Simply 
 restarting on error isn't necessarily enough.
This can be mitigated: a cascade reboot would occur if the problem affects the reboot sequence itself. --- /Paolo
Sep 29 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 29 September 2014 at 07:52:33 UTC, Paolo Invernizzi
wrote:
 On Monday, 29 September 2014 at 04:03:37 UTC, Sean Kelly wrote:
 On Monday, 29 September 2014 at 02:57:03 UTC, Walter Bright

 Right.  But if the condition that caused the restart persists, 
 the process can end up in a cascading restart scenario.  
 Simply restarting on error isn't necessarily enough.
This can be mitigated: a cascade reboot would occur if the problem affects the reboot sequence itself.
Or if an ongoing situation causes the problem to rapidly reoccur. Look at most MMO game launches for example. Production load hits and some process falls over in a weird way, which increases load because everyone goes nuts trying to log back in, and when the system comes back up it immediately falls over again. Rinse, repeat.
Sep 29 2014
parent reply "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Monday, 29 September 2014 at 19:23:28 UTC, Sean Kelly wrote:
 On Monday, 29 September 2014 at 07:52:33 UTC, Paolo Invernizzi
 wrote:
 On Monday, 29 September 2014 at 04:03:37 UTC, Sean Kelly wrote:
 On Monday, 29 September 2014 at 02:57:03 UTC, Walter Bright

 Right.  But if the condition that caused the restart 
 persists, the process can end up in a cascading restart 
 scenario.  Simply restarting on error isn't necessarily 
 enough.
This can be mitigated: a cascade reboot would occur if the problem affects the reboot sequence itself.
Or if an ongoing situation causes the problem to rapidly reoccur. Look at most MMO game launches for example. Production load hits and some process falls over in a weird way, which increases load because everyone goes nuts trying to log back in, and when the system comes back up it immediately falls over again. Rinse, repeat.
Is it not better to throttle down the connection volumes before it reach processes not being able to handle an overload in in a correct way? --- /Paolo
Sep 29 2014
parent "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 29 September 2014 at 21:49:44 UTC, Paolo Invernizzi
wrote:
 Is it not better to throttle down the connection volumes before 
 it reach processes not being able to handle an overload in in a 
 correct way?
Well, it many cases it isn't the pure load numbers that are the problem but the interaction of the behavior of actual real users. Like it isn't overly difficult to simulate a large volume of traffic on a system, but it's extremely difficult to simulate realistic behavior patters. I think in many cases the problem ends up being unexpected interactions of different behaviors that cause problems rather than simply high volume.
Sep 29 2014
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 29/09/2014 05:03, Sean Kelly wrote:
 I recall Toyota got into trouble with their computer controlled cars
 because of their idea of handling of inevitable bugs and errors. It
 was one process that controlled everything. When something unexpected
 went wrong, it kept right on operating, any unknown and unintended
 consequences be damned.

 The way to get reliable systems is to design to accommodate errors,
 not pretend they didn't happen, or hope that nothing else got
 affected, etc. In critical software systems, that means shut down and
 restart the offending system, or engage the backup.
My point was that it's often more complicated than that. There have been papers written on self-repairing systems, for example, and ways to design systems that are inherently durable when it comes to even internal errors. I think what I'm trying to say is that simply aborting on error is too brittle in some cases, because it only deals with one vector--memory corruption that is unlikely to reoccur. But I've watched always-on systems fall apart from some unexpected ongoing situation, and simply restarting doesn't actually help.
Sean, I fully agree with the points you have been making so far. But if Walter is fixated on thinking that all the practical uses of D will be critical systems, or simple (ie, single-use, non-interactive) command-line applications, it will be hard for him to comprehend the whole point that "simply aborting on error is too brittle in some cases". PS: Walter, what browser to you use? -- Bruno Medeiros https://twitter.com/brunodomedeiros
Oct 01 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Bruno Medeiros:

 But if Walter is fixated on thinking that all the practical 
 uses of D will be critical systems, or simple (ie, single-use, 
 non-interactive) command-line applications,
There's still some of way to go for D design to make it well fit for high integrity systems (some people even use a restricted subset of C for such purposes, but it's a bad language for it). Bye, bearophile
Oct 01 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/1/2014 7:32 AM, bearophile wrote:
 Bruno Medeiros:

 But if Walter is fixated on thinking that all the practical uses of D will be
 critical systems, or simple (ie, single-use, non-interactive) command-line
 applications,
There's still some of way to go for D design to make it well fit for high integrity systems (some people even use a restricted subset of C for such purposes, but it's a bad language for it).
No matter what language is used, a high integrity system cannot be constructed from continuing after the program has entered an unknown state.
Oct 04 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/1/2014 7:17 AM, Bruno Medeiros wrote:
 Sean, I fully agree with the points you have been making so far.
 But if Walter is fixated on thinking that all the practical uses of D will be
 critical systems, or simple (ie, single-use, non-interactive) command-line
 applications, it will be hard for him to comprehend the whole point that
"simply
 aborting on error is too brittle in some cases".
Airplane avionics systems all abort on error, yet the airplanes don't fall out of the sky. I've explained why and how this works many times, here it is again: http://www.drdobbs.com/architecture-and-design/safe-systems-from-unreliable-parts/228701716
Oct 04 2014
next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Saturday, 4 October 2014 at 09:04:58 UTC, Walter Bright wrote:
 Airplane avionics systems all abort on error, yet the airplanes 
 don't fall out of the sky.
To be fair, the's a function of aerodynamics more than system design. But I see what you're getting at.
Oct 04 2014
prev sibling parent Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 04/10/2014 10:05, Walter Bright wrote:
 On 10/1/2014 7:17 AM, Bruno Medeiros wrote:
 Sean, I fully agree with the points you have been making so far.
 But if Walter is fixated on thinking that all the practical uses of D
 will be
 critical systems, or simple (ie, single-use, non-interactive)
 command-line
 applications, it will be hard for him to comprehend the whole point
 that "simply
 aborting on error is too brittle in some cases".
Airplane avionics systems all abort on error, yet the airplanes don't fall out of the sky. I've explained why and how this works many times, here it is again: http://www.drdobbs.com/architecture-and-design/safe-systems-from-unreliable-parts/228701716
That's completely irrelevant to the "simply aborting on error is too brittle in some cases" point above, because I wasn't talking about avionics systems, or any kind of mission critical systems at all. In fact, the opposite (non critical systems). -- Bruno Medeiros https://twitter.com/brunodomedeiros
Oct 08 2014
prev sibling next sibling parent reply Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 29/09/14 02:09, Walter Bright via Digitalmars-d wrote:
 If the program has entered an unknown state, its behavior from then on cannot
be
 predictable. There's nothing I or D can do about that. D cannot officially
 endorse such a practice, though D being a systems programming language it will
 let you do what you want.

 I would not even consider such a practice for a program that is in charge of
 anything that could result in injury, death, property damage, security
breaches,
 etc.
I think I should clarify that I'm not asking you to say "I endorse catching Errors". Your point about systems responsible for the safety of people or property is very well made, and I'm fully in agreement with you about this. What I'm asking you to consider is a use-case, one that I picked quite carefully. Without assuming anything about how the system is architected, if we have a telephone exchange, and an Error occurs in the handling of a single call, it seems to me fairly unarguable that it's essential to avoid this bringing down everyone else's call with it. That's not simply a matter of convenience -- it's a matter of safety, because those calls might include emergency calls, urgent business communications, or any number of other circumstances where dropping someone's call might have severe negative consequences. As I'm sure you realize, I also picked that particular use-case because it's one where there is a well-known technological solution -- Erlang -- which has as a key feature its ability to isolate different parts of the program, and to deal with errors by bringing down the local process where the error occurred, rather than the whole system. This is an approach which is seriously battle-tested in production. As I said, I'm not asking you to endorse catching Errors in threads, or other gross simplifications of Erlang's approach. What I'm interested in are your thoughts on how we might approach resolving the requirement for this kind of stability and localization of error-handling with the tools that D provides. I don't mind if you say to me "That's your problem" (which it certainly is:-), but I'd like it to be clear that it _is_ a problem, and one that it's important for D to address, given its strong standing in the development of super-high-connectivity server applications.
Oct 03 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/3/2014 10:00 AM, Joseph Rushton Wakeling via Digitalmars-d wrote:
 What I'm asking you to consider is a use-case, one that I picked quite
 carefully.  Without assuming anything about how the system is architected, if
we
 have a telephone exchange, and an Error occurs in the handling of a single
call,
 it seems to me fairly unarguable that it's essential to avoid this bringing
down
 everyone else's call with it.  That's not simply a matter of convenience --
it's
 a matter of safety, because those calls might include emergency calls, urgent
 business communications, or any number of other circumstances where dropping
 someone's call might have severe negative consequences.
What you're doing is attempting to write a program with the requirement that the program cannot fail. It's impossible. If that's your requirement, the system needs to be redesigned so that it can accommodate the failure of the program. (Ignoring bugs in the program is not accommodating failure, it's pretending that the program cannot fail.)
 As I'm sure you realize, I also picked that particular use-case because it's
one
 where there is a well-known technological solution -- Erlang -- which has as a
 key feature its ability to isolate different parts of the program, and to deal
 with errors by bringing down the local process where the error occurred, rather
 than the whole system.  This is an approach which is seriously battle-tested in
 production.
As I (and Brad) has stated before, process isolation, shutting down the failed process, and restarting the process, is acceptable, because processes are isolated from each other. Threads are not isolated from each other. They are not. Not. Not.
 As I said, I'm not asking you to endorse catching Errors in threads, or other
 gross simplifications of Erlang's approach.  What I'm interested in are your
 thoughts on how we might approach resolving the requirement for this kind of
 stability and localization of error-handling with the tools that D provides.

 I don't mind if you say to me "That's your problem" (which it certainly is:-),
 but I'd like it to be clear that it _is_ a problem, and one that it's important
 for D to address, given its strong standing in the development of
 super-high-connectivity server applications.
The only way to have super high uptime is to design the system so that failure is isolated, and the failed process can be quickly restarted or replaced. Ignoring bugs is not isolation, and hoping that bugs in one thread doesn't affected memory shared by other threads doesn't work.
Oct 04 2014
next sibling parent reply Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 04/10/14 11:18, Walter Bright via Digitalmars-d wrote:
 What you're doing is attempting to write a program with the requirement that
the
 program cannot fail.

 It's impossible.
No, I'm attempting to discuss how to approach the problem that the program _can_ fail, and how to isolate that failure appropriately. I'm asking for discussion of how to handle a use-case, not trying to advocate for particular solutions. You seem to be convinced that I don't understand the principles you are advocating of isolation, backup, and so forth. What I've been trying (but obviously failing) to communicate to you is, "OK, I agree on these principles, let's talk about how to achieve them in a practical sense with D."
 If that's your requirement, the system needs to be redesigned so that it can
 accommodate the failure of the program.

 (Ignoring bugs in the program is not accommodating failure, it's pretending
that
 the program cannot fail.)
Indeed.
 As I'm sure you realize, I also picked that particular use-case because it's
one
 where there is a well-known technological solution -- Erlang -- which has as a
 key feature its ability to isolate different parts of the program, and to deal
 with errors by bringing down the local process where the error occurred, rather
 than the whole system.  This is an approach which is seriously battle-tested in
 production.
As I (and Brad) has stated before, process isolation, shutting down the failed process, and restarting the process, is acceptable, because processes are isolated from each other. Threads are not isolated from each other. They are not. Not. Not.
I will repeat what I said in my previous email: "Without assuming anything about how the system is architected". I realize that in my earlier remark:
 However, it's clearly very desirable in this use-case for the application to
keep going if at all possible and for any problem, even an Error, to be
contained in its local context if we can do so.  (By "local context", in
practice this probably means a thread or fiber or some other similar
programming construct.)
... I probably conveyed the idea that I was seeking to contain Errors inside threads or fibers. I was already anticipating that the answer here would be a definitive "You can't under any circumstances", and hence why I wrote, "or other similar programming construct", by which I was thinking of Erlang-style processes. Actually, a large part of my reason for continuing this discussion is because where high-connectivity server applications are concerned, I'm keen to ensure that their developers _avoid_ the dangerous solution that is, "Spawn lots of threads and fibers, and localize Errors by catching them and throwing away the thread rather than the application." However, unless there is an alternative in a practical sense, that is probably what people are going to do, because the trade-offs of their use-case make it seem the least bad option. I think that's a crying shame and that we can and should do better.
 The only way to have super high uptime is to design the system so that failure
 is isolated, and the failed process can be quickly restarted or replaced.
 Ignoring bugs is not isolation, and hoping that bugs in one thread doesn't
 affected memory shared by other threads doesn't work.
Right. Which is why I'd like to move the discussion over to "How can we achieve this in D?"
Oct 04 2014
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 4 October 2014 at 11:19:10 UTC, Joseph Rushton
Wakeling via Digitalmars-d wrote:
 On 04/10/14 11:18, Walter Bright via Digitalmars-d wrote:
 What you're doing is attempting to write a program with the 
 requirement that the
 program cannot fail.
 The only way to have super high uptime is to design the system 
 so that failure
 is isolated, and the failed process can be quickly restarted 
 or replaced.
 Ignoring bugs is not isolation, and hoping that bugs in one 
 thread doesn't
 affected memory shared by other threads doesn't work.
Right. Which is why I'd like to move the discussion over to "How can we achieve this in D?"
I see two things that are in the way (aside from the obvious things like non- safe code): Casting away shared, and implicitly shared immutable data. The former can be checked statically, but the latter is harder to work around in the current language.
Oct 04 2014
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/4/2014 4:19 AM, Joseph Rushton Wakeling via Digitalmars-d wrote:
 On 04/10/14 11:18, Walter Bright via Digitalmars-d wrote:
 You seem to be convinced that I don't understand the principles you are
 advocating of isolation, backup, and so forth.  What I've been trying (but
 obviously failing) to communicate to you is, "OK, I agree on these principles,
 let's talk about how to achieve them in a practical sense with D."
Ok, I understand. My apologies for misunderstanding you. I would suggest the best way to achieve that is to use the process isolation abilities provided by the operating system. Separate the system into processes that communicate via some messaging system provided by the operating system (not shared memory). I read that the Chrome browser was done this way, so if one part of Chrome crashed, the failed part could be restarted without restarting the rest of Chrome. Note that such a solution has little to do with D in particular, or C or C++. It's more to do with what the operating system provides for process isolation and interprocess communication.
 Right.  Which is why I'd like to move the discussion over to "How can we
achieve
 this in D?"
D provides a lot of ability to make a single process more robust, such as pure functions, immutable data structures, unit testing, safe, etc., so bugs are less likely. And my personal experience with developing D programs is they come up faster and are less buggy than my C++ ones. But once a bug is detected, we're back to chucking the process.
Oct 04 2014
prev sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Saturday, 4 October 2014 at 09:18:41 UTC, Walter Bright wrote:
 Threads are not isolated from each other. They are not. Not. 
 Not.
Neither are programs that communicate in some fashion. I'll grant that the possibility of memory corruption doesn't exist in this case (a problem unique to systems languages like D), but system corruption still does. And I absolutely agree with you that if memory corruption is ever even suspected, the process must immediately halt. In that case I wouldn't even throw an Error, I'd call exit(1).
Oct 04 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/4/2014 9:16 AM, Sean Kelly wrote:
 On Saturday, 4 October 2014 at 09:18:41 UTC, Walter Bright wrote:
 Threads are not isolated from each other. They are not. Not. Not.
Neither are programs that communicate in some fashion.
Operating systems typically provide methods of interprocess communication that are robust against corruption, such as pipes, message passing, etc. The receiving process should regard such input as "user/environmental input", and must validate it. Corruption in it would not be regarded as a logic bug in the receiving process (unless it failed to check for it). Interprocess shared memory, though, is not robust.
 I'll grant that the
 possibility of memory corruption doesn't exist in this case (a problem unique
to
 systems languages like D), but system corruption still does.  And I absolutely
 agree with you that if memory corruption is ever even suspected, the process
 must immediately halt.  In that case I wouldn't even throw an Error, I'd call
 exit(1).
System corruption is indeed a problem with this type of setup. We're relying here on the operating system not having such bugs in it, and indeed OS vendors work very hard at preventing an errant program from corrupting the system. We all know, of course, that this sort of thing happens anyway. An even more robust system design will need a way to deal with that, and failure of the hardware, and failure of the data center, etc. All components of a reliable system are unreliable, and a robust system needs to be able to recover from the inevitable failure of any component. This kind of thinking needs to pervade the initial system design from the ground up, it's hard to tack it on later.
Oct 04 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Saturday, 4 October 2014 at 19:36:02 UTC, Walter Bright wrote:
 On 10/4/2014 9:16 AM, Sean Kelly wrote:
 On Saturday, 4 October 2014 at 09:18:41 UTC, Walter Bright 
 wrote:
 Threads are not isolated from each other. They are not. Not. 
 Not.
Neither are programs that communicate in some fashion.
Operating systems typically provide methods of interprocess communication that are robust against corruption, such as pipes, message passing, etc. The receiving process should regard such input as "user/environmental input", and must validate it. Corruption in it would not be regarded as a logic bug in the receiving process (unless it failed to check for it). Interprocess shared memory, though, is not robust.
 I'll grant that the
 possibility of memory corruption doesn't exist in this case (a 
 problem unique to
 systems languages like D), but system corruption still does.  
 And I absolutely
 agree with you that if memory corruption is ever even 
 suspected, the process
 must immediately halt.  In that case I wouldn't even throw an 
 Error, I'd call
 exit(1).
System corruption is indeed a problem with this type of setup. We're relying here on the operating system not having such bugs in it, and indeed OS vendors work very hard at preventing an errant program from corrupting the system. We all know, of course, that this sort of thing happens anyway. An even more robust system design will need a way to deal with that, and failure of the hardware, and failure of the data center, etc. All components of a reliable system are unreliable, and a robust system needs to be able to recover from the inevitable failure of any component. This kind of thinking needs to pervade the initial system design from the ground up, it's hard to tack it on later.
This is not different from fiber or thread based approach. If one uses only immutable data for inter-thread communications (== does not use inter-process shared memory) same gurantees and reasoning come. And such design allows for many data optimizations hard or impossible to do with process-based approach. There is no magic solution that does not allow to screw up in 100% of cases whatever programmer does. Killing process is pragmatical default but not a pragmatical silver bullet and from pure theoretical point of few it has no advantages over killing the thread/fiber - it is all about chances of failure, not preventing it. Same in Erlang - some failure warrant killing the runtime, some only specific process. It is all about the context and programmer should decide what is best approach for any specific program. I am fine with non-default being hard but I want it to be still possible within legal language restricions.
Oct 05 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/5/2014 8:35 AM, Dicebot wrote:
 I am fine with non-default being hard but I
 want it to be still possible within legal language restricions.
D being a systems language, you can without much difficulty do whatever works for you. People do look to us for guidance, however. The levels of programming mastery: newbie: follow the rules because you're told to master: follow the rules because you understand them guru: break the rules because you understand their limitations I'd be doing our users an injustice by not making sure they understand the rules before trying to break them.
Oct 05 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Sunday, 5 October 2014 at 20:41:44 UTC, Walter Bright wrote:
 On 10/5/2014 8:35 AM, Dicebot wrote:
 I am fine with non-default being hard but I
 want it to be still possible within legal language restricions.
D being a systems language, you can without much difficulty do whatever works for you.
Yes but it shouldn't be in undefined behaviour domain. In other words there needs to be a confidence that some new compiler optimization will not break the application completely. Right now Throwable/Error docs heavily suggest catching it is "shoot yourself in the foot" thing and new compiler release can possibly change its behaviour without notice. I'd like to have a bit more specific documentation about what can and what can't be expected. Experimental observations are that one shouldn't rely on any cleanup code (RAII / scope(exit)) to happen but other than that it is OK to consume Error if execution context for it (fiber in our case) gets terminated. As D1 compiler does not change it is good enough observation for practical means. But for D2 it would be nice to have some official clarification. I think this is the only important concern I have as long as power user stuff remains possible without re-implementing whole exception system from scratch.
Oct 05 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/5/2014 2:51 PM, Dicebot wrote:
 On Sunday, 5 October 2014 at 20:41:44 UTC, Walter Bright wrote:
 On 10/5/2014 8:35 AM, Dicebot wrote:
 I am fine with non-default being hard but I
 want it to be still possible within legal language restricions.
D being a systems language, you can without much difficulty do whatever works for you.
Yes but it shouldn't be in undefined behaviour domain. In other words there needs to be a confidence that some new compiler optimization will not break the application completely.
Relying on program state after entering an unknown state is undefined by definition. I don't see how a language can make a statement like "it's probably ok".
 Right now Throwable/Error docs heavily suggest catching it is "shoot yourself
in
 the foot" thing and new compiler release can possibly change its behaviour
 without notice. I'd like to have a bit more specific documentation about what
 can and what can't be expected. Experimental observations are that one
shouldn't
 rely on any cleanup code (RAII / scope(exit)) to happen but other than that it
 is OK to consume Error if execution context for it (fiber in our case) gets
 terminated. As D1 compiler does not change it is good enough observation for
 practical means. But for D2 it would be nice to have some official
clarification.
Definitely unwinding may or may not happen from Error throws, "nothrow" functions may throw Errors, and optimizers need not account for Errors being thrown. Attempting to unwind the stack when an Error is thrown may cause further corruption (if the Error was thrown because of corruption), another reason for the language not to try to do it. An Error is, by definition, unrecoverable.
 I think this is the only important concern I have as long as power user stuff
 remains possible without re-implementing whole exception system from scratch.
You can catch an Error. But what is done from there is up to you - and to do more than just log the error, engage the backup, and exit, I cannot recommend. To do more, use an Exception. But to throw an Exception when a logic bug has been detected, then try and continue based on it "probably" being ok, is something I cannot recommend and D certainly cannot guarantee anything. If the program does anything that matters, that is.
Oct 05 2014
next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Sunday, 5 October 2014 at 23:01:48 UTC, Walter Bright wrote:
 Definitely unwinding may or may not happen from Error throws, 
 "nothrow" functions may throw Errors, and optimizers need not 
 account for Errors being thrown.
This is the real concern. If an Error is thrown out of a nothrow function that contains a synchronized block, for example, the mutex might still be locked. So the only viable option is to terminate, even for something theoretically recoverable like a divide by zero or an OOME.
Oct 05 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/5/2014 4:28 PM, Sean Kelly wrote:
 On Sunday, 5 October 2014 at 23:01:48 UTC, Walter Bright wrote:
 Definitely unwinding may or may not happen from Error throws, "nothrow"
 functions may throw Errors, and optimizers need not account for Errors being
 thrown.
This is the real concern. If an Error is thrown out of a nothrow function that contains a synchronized block, for example, the mutex might still be locked. So the only viable option is to terminate, even for something theoretically recoverable like a divide by zero or an OOME.
Divide by zero is not recoverable since you don't know why it occurred. It could be the result of overflowing a buffer with 0s. Until a human debugs it and figures out why it happened, it not recoverable. Because it could be the result of corruption like buffer overflows, the less code that is executed between the detection of the bug and terminating the program, the safer the program is. Continuing execution may mess up user data, may execute injected malware, etc.
Oct 05 2014
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Sunday, 5 October 2014 at 23:01:48 UTC, Walter Bright wrote:
 On 10/5/2014 2:51 PM, Dicebot wrote:
 On Sunday, 5 October 2014 at 20:41:44 UTC, Walter Bright wrote:
 On 10/5/2014 8:35 AM, Dicebot wrote:
 I am fine with non-default being hard but I
 want it to be still possible within legal language 
 restricions.
D being a systems language, you can without much difficulty do whatever works for you.
Yes but it shouldn't be in undefined behaviour domain. In other words there needs to be a confidence that some new compiler optimization will not break the application completely.
Relying on program state after entering an unknown state is undefined by definition. I don't see how a language can make a statement like "it's probably ok".
It is only in undefined state because language handles Errors that way. At the point of throwing the Error state was perfectly defined and 100% recoverable. This is the typical case for assertion failure in contract - it detects some program flaw like inability to handle specific data combination from other process but it does not mean memory is corrupted or program is inherently broken. Just killing the fiber and continuing with other requests (which don't trigger that unexpected code path) is absolutely fine unless compiler kicks in and optimizes something away in surprising fashion. If destructors are ignored those must be always ignored and defined in spec. Same for scope(exit) or any similar affected feature. Currently it may or may not attempt cleanup and that is the problem when trying to circumvent the semantics.
 I think this is the only important concern I have as long as 
 power user stuff
 remains possible without re-implementing whole exception 
 system from scratch.
You can catch an Error. But what is done from there is up to you - and to do more than just log the error, engage the backup, and exit, I cannot recommend.
Killing whole process is unacceptable in many cases, it will effectively shut down the whole service if faulty request happens at least one in a few seconds.
 To do more, use an Exception. But to throw an Exception when a 
 logic bug has been detected, then try and continue based on it 
 "probably" being ok, is something I cannot recommend and D 
 certainly cannot guarantee anything. If the program does 
 anything that matters, that is.
Assertions / contracts use Error. Do you think it is a better approach to prohibit using `assert` and throw custom exceptions from contracts?
Oct 06 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/6/2014 10:09 AM, Dicebot wrote:
 It is only in undefined state because language handles Errors that way. At the
 point of throwing the Error state was perfectly defined and 100% recoverable.
 This is the typical case for assertion failure in contract - it detects some
 program flaw like inability to handle specific data combination from other
 process but it does not mean memory is corrupted or program is inherently
 broken. Just killing the fiber and continuing with other requests (which don't
 trigger that unexpected code path) is absolutely fine unless compiler kicks in
 and optimizes something away in surprising fashion.
What you're describing sounds like using asserts to validate input data. This is not what asserts are for.
Oct 06 2014
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Mon, Oct 06, 2014 at 07:55:23PM -0700, Walter Bright via Digitalmars-d wrote:
 On 10/6/2014 10:09 AM, Dicebot wrote:
It is only in undefined state because language handles Errors that
way. At the point of throwing the Error state was perfectly defined
and 100% recoverable.  This is the typical case for assertion failure
in contract - it detects some program flaw like inability to handle
specific data combination from other process but it does not mean
memory is corrupted or program is inherently broken. Just killing the
fiber and continuing with other requests (which don't trigger that
unexpected code path) is absolutely fine unless compiler kicks in and
optimizes something away in surprising fashion.
What you're describing sounds like using asserts to validate input data. This is not what asserts are for.
Using assertions in contracts in D currently has some design issues, one of the foremost being that in-contracts of derived classes are allowed to be more relaxed than base classes, which means that the effective contract is (baseClassContract || derivedClassContract). However, since in-contracts are allowed to be arbitrary statements, the only way to implement this is to catch AssertErrors and continue running if at least one of the contracts didn't assert. But that means we're technically in an undefined state after that point. :-( (If the in-contract calls a nothrow function that asserts, for example, dtors may have been skipped, cleanups not performed, etc., and yet we blindly barge on because the other contract didn't assert.) T -- LINUX = Lousy Interface for Nefarious Unix Xenophobes.
Oct 06 2014
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/06/2014 01:01 AM, Walter Bright wrote:
 On 10/5/2014 2:51 PM, Dicebot wrote:
 On Sunday, 5 October 2014 at 20:41:44 UTC, Walter Bright wrote:
 On 10/5/2014 8:35 AM, Dicebot wrote:
 I am fine with non-default being hard but I
 want it to be still possible within legal language restricions.
D being a systems language, you can without much difficulty do whatever works for you.
Yes but it shouldn't be in undefined behaviour domain. In other words there needs to be a confidence that some new compiler optimization will not break the application completely.
Relying on program state after entering an unknown state is undefined by definition.
What definition?
 I don't see how a language can make a statement like "it's
 probably ok".
E.g. type safety.
Oct 07 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/7/2014 6:56 AM, Timon Gehr wrote:
 On 10/06/2014 01:01 AM, Walter Bright wrote:
 Relying on program state after entering an unknown state is undefined by
 definition.
What definition?
How can one define the behavior of an unknown state?
Oct 07 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/07/2014 09:26 PM, Walter Bright wrote:
 On 10/7/2014 6:56 AM, Timon Gehr wrote:
 On 10/06/2014 01:01 AM, Walter Bright wrote:
 Relying on program state after entering an unknown state is undefined by
 definition.
What definition?
How can one define the behavior of an unknown state?
Well, how do you define the behaviour of a program that will be fed an unknown input? That way. I don't really understand what this question is trying to get at. Just define the language semantics appropriately. Your reasoning usually goes like a certain kind of event you assume to be bad -> bug -> unknown state -> undefined behaviour. Why does this apply to D and not to e.g. Java?
Oct 07 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/7/2014 12:44 PM, Timon Gehr wrote:
 On 10/07/2014 09:26 PM, Walter Bright wrote:
 On 10/7/2014 6:56 AM, Timon Gehr wrote:
 On 10/06/2014 01:01 AM, Walter Bright wrote:
 Relying on program state after entering an unknown state is undefined by
 definition.
What definition?
How can one define the behavior of an unknown state?
Well, how do you define the behaviour of a program that will be fed an unknown input? That way. I don't really understand what this question is trying to get at. Just define the language semantics appropriately. Your reasoning usually goes like a certain kind of event you assume to be bad -> bug -> unknown state -> undefined behaviour.
What defined behavior would you suggest would be possible after an overflow bug is detected?
Oct 07 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/07/2014 10:09 PM, Walter Bright wrote:
 On 10/7/2014 12:44 PM, Timon Gehr wrote:
 On 10/07/2014 09:26 PM, Walter Bright wrote:
 On 10/7/2014 6:56 AM, Timon Gehr wrote:
 On 10/06/2014 01:01 AM, Walter Bright wrote:
 Relying on program state after entering an unknown state is
 undefined by
 definition.
What definition?
How can one define the behavior of an unknown state?
Well, how do you define the behaviour of a program that will be fed an unknown input? That way. I don't really understand what this question is trying to get at. Just define the language semantics appropriately. Your reasoning usually goes like a certain kind of event you assume to be bad -> bug -> unknown state -> undefined behaviour.
What defined behavior would you suggest would be possible after an overflow bug is detected?
At the language level, there are many possibilities. Just look at what type safe languages do. It is not true that this must lead to UB by a "definition" commonly agreed upon by participants in this thread.
Oct 07 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/7/2014 2:12 PM, Timon Gehr wrote:
 On 10/07/2014 10:09 PM, Walter Bright wrote:
 What defined behavior would you suggest would be possible after an
 overflow bug is detected?
At the language level, there are many possibilities. Just look at what type safe languages do. It is not true that this must lead to UB by a "definition" commonly agreed upon by participants in this thread.
And even in a safe language, how would you know that a bug in the runtime didn't lead to corruption which put your program into the unknown state? Your assertion rests on some assumptions: 1. the "safe" language doesn't have bugs in its proof or specification 2. the "safe" language doesn't have bugs in its implementation 3. that it is knowable what caused a bug without ever having debugged it 4. that program state couldn't have been corrupted due to hardware failures 5. that it's possible to write a perfect system all of which are false. I.e. it is not possible to define the state of a program after it has entered an unknown state that was defined to never happen.
Oct 07 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/08/2014 02:27 AM, Walter Bright wrote:
 On 10/7/2014 2:12 PM, Timon Gehr wrote:
 On 10/07/2014 10:09 PM, Walter Bright wrote:
 What defined behavior would you suggest would be possible after an
 overflow bug is detected?
At the language level, there are many possibilities. Just look at what type safe languages do. It is not true that this must lead to UB by a "definition" commonly agreed upon by participants in this thread.
And even in a safe language, how would you know that a bug in the runtime didn't lead to corruption which put your program into the unknown state? Your assertion
Which assertion? That there are languages that call themselves type safe?
 rests on some assumptions:

 1. the "safe" language doesn't have bugs in its proof or specification
So what? I can report these if present. That's not undefined behaviour, it is a wrong specification or a bug in the automated proof checker. (In my experience however, the developers might not actually acknowledge that the specification violates type safety. UB in safe code is a joke. But I am diverting.) Not specific to our situation where we get an overflow.
 2. the "safe" language doesn't have bugs in its implementation
So what? I can report these if present. That's not undefined behaviour, it is wrong behaviour. Not specific to our situation where we get an overflow.
 3. that it is knowable what caused a bug without ever having debugged it
Why would I need to assume this to make my point? Not specific to our situation where we get an overflow
 4. that program state couldn't have been corrupted due to hardware failures
Not specific to our situation where we detect the problem.
 5. that it's possible to write a perfect system
You cannot disprove this one, and no, I am not assuming this, but it would be extraordinarily silly to write into the official language specification: "a program may do anything at any time, because a conforming implementation might contain bugs". Also: Not specific to our situation where we detect the problem.
 all of which are false.


 I.e.
Why "I.e."?
 it is not possible to define the state of a program after it has
 entered an unknown state that was defined to never happen.
By assuming your 5 postulates are false, and filling in the justification for the "i.e." you left out, one will quickly reach the conclusion that it is not possible to define the behaviour of a program at all. Therefore, if we describe programs, our words are meaningless, because this is not "possible". This seems to quickly become a great example of the kind of black/white thinking you warned against in another post in this thread. It has to be allowed to use idealised language, otherwise you cannot say or think anything. What is _undefined behaviour_ depends on the specification alone, and as flawed and ambiguous as that specification may be, in practice it will still be an invaluable tool for communication among language users/developers. Can we at least agree that Dicebot's request for having the behaviour of inadvisable constructs defined such that an implementation cannot randomly change behaviour and then have the developers close down the corresponding bugzilla issue because it was the user's fault anyway is not unreasonable by definition because the system will not reach a perfect state anyway, and then retire this discussion?
Oct 07 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/7/2014 6:18 PM, Timon Gehr wrote:
 I can report these if present.
Writing a strongly worded letter to the White Star Line isn't going to help you when the ship is sinking in the middle of the North Atlantic. What will help is minimizing the damage that a detected fault may cause. You cannot rely on the specification when a fault has been detected. "This can't be happening!" are likely the last words of more than a few people.
 Can we at least agree that Dicebot's request for having the behaviour of
 inadvisable constructs defined such that an implementation cannot randomly
 change behaviour and then have the developers close down the corresponding
 bugzilla issue because it was the user's fault anyway is not unreasonable by
 definition because the system will not reach a perfect state anyway, and then
 retire this discussion?
I've been working with Dicebot behind the scenes to help resolve the particular issues with the code he's responsible for. As for D, D cannot offer any guarantees about behavior after a program crash. Nor can any other language.
Oct 07 2014
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 10/08/2014 05:19 AM, Walter Bright wrote:
 On 10/7/2014 6:18 PM, Timon Gehr wrote:
  > I can report these if present.

 Writing a strongly worded letter to the White Star Line isn't going to
 help you when the ship is sinking in the middle of the North Atlantic.
 ...
Maybe it is going to help the next guy whose ship will not be sinking due to that report.
 What will help is minimizing the damage that a detected fault may cause.
 You cannot rely on the specification when a fault has been detected.
 "This can't be happening!" are likely the last words of more than a few
 people.
Sure, I agree. Just note that if some programmer is checking for overflow after the fact using the following idiom: int x=y*z; if(x/y!=z) assert(0); Then the language can be defined such that e.g.: 0. The overflow will throw on its own. 1. Overflow is undefined, i.e. the optimizer is allowed to remove the check and avoid the detection of the bug. 2. Guaranteed wrap-around behaviour makes the code valid and the bug is detected by the assert. 3. Arbitrary-precision integers. 4. ... Code is simply less likely to run as intended or else abort if possibility 1 is consciously taken. The language implementation may still be buggy, but if it may even sink your ship when it generated code according to the specification, it likely sinks in more cases. Of course you can say that the programmer is at fault for checking for overflow in the wrong fashion, but this does not matter at the point where your ship is sinking. One may still see this choice as the right trade-off, but it is not the only possibility 'by definition'.
Oct 08 2014
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Wednesday, 8 October 2014 at 03:20:21 UTC, Walter Bright wrote:
 Can we at least agree that Dicebot's request for having the 
 behaviour of
 inadvisable constructs defined such that an implementation 
 cannot randomly
 change behaviour and then have the developers close down the 
 corresponding
 bugzilla issue because it was the user's fault anyway is not 
 unreasonable by
 definition because the system will not reach a perfect state 
 anyway, and then
 retire this discussion?
I've been working with Dicebot behind the scenes to help resolve the particular issues with the code he's responsible for. As for D, D cannot offer any guarantees about behavior after a program crash. Nor can any other language.
Just wanted to point out that resulting solution (== manually switching many of contracts to exceptions from asserts) to me is an unhappy workaround to deal with overly opinionated language and not actually a solution. I still consider this a problem.
Oct 09 2014
parent reply Johannes Pfau <nospam example.com> writes:
Am Thu, 09 Oct 2014 13:10:34 +0000
schrieb "Dicebot" <public dicebot.lv>:

 On Wednesday, 8 October 2014 at 03:20:21 UTC, Walter Bright wrote:
 Can we at least agree that Dicebot's request for having the 
 behaviour of
 inadvisable constructs defined such that an implementation 
 cannot randomly
 change behaviour and then have the developers close down the 
 corresponding
 bugzilla issue because it was the user's fault anyway is not 
 unreasonable by
 definition because the system will not reach a perfect state 
 anyway, and then
 retire this discussion?
I've been working with Dicebot behind the scenes to help resolve the particular issues with the code he's responsible for. As for D, D cannot offer any guarantees about behavior after a program crash. Nor can any other language.
Just wanted to point out that resulting solution (== manually switching many of contracts to exceptions from asserts) to me is an unhappy workaround to deal with overly opinionated language and not actually a solution. I still consider this a problem.
A point which hasn't been discussed yet: Errors and therefore assert can be used in nothrow functions. This is a pita for compilers because it now can't do certain optimizations. When porting GDC to ARM we started to see problems because of that (can't unwind from nothrow functions on ARM, program just aborts). And now we therefore have to worsen the codegen for nothrow functions because of this. I think Walter sometimes suggested that it would be valid for a compiler to not unwind Errors at all (in release mode), but simply kill the program and dump a error message. This would finally allow us to optimize nothrow functions.
Oct 09 2014
next sibling parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 9 October 2014 17:33, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Am Thu, 09 Oct 2014 13:10:34 +0000
 schrieb "Dicebot" <public dicebot.lv>:

 On Wednesday, 8 October 2014 at 03:20:21 UTC, Walter Bright wrote:
 Can we at least agree that Dicebot's request for having the
 behaviour of
 inadvisable constructs defined such that an implementation
 cannot randomly
 change behaviour and then have the developers close down the
 corresponding
 bugzilla issue because it was the user's fault anyway is not
 unreasonable by
 definition because the system will not reach a perfect state
 anyway, and then
 retire this discussion?
I've been working with Dicebot behind the scenes to help resolve the particular issues with the code he's responsible for. As for D, D cannot offer any guarantees about behavior after a program crash. Nor can any other language.
Just wanted to point out that resulting solution (== manually switching many of contracts to exceptions from asserts) to me is an unhappy workaround to deal with overly opinionated language and not actually a solution. I still consider this a problem.
A point which hasn't been discussed yet: Errors and therefore assert can be used in nothrow functions. This is a pita for compilers because it now can't do certain optimizations. When porting GDC to ARM we started to see problems because of that (can't unwind from nothrow functions on ARM, program just aborts). And now we therefore have to worsen the codegen for nothrow functions because of this. I think Walter sometimes suggested that it would be valid for a compiler to not unwind Errors at all (in release mode), but simply kill the program and dump a error message. This would finally allow us to optimize nothrow functions.
This behaviour was agreed at the bar at DConf. We'd have to put it in the spec to give it the official stamp of approval though. Iain.
Oct 09 2014
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 9 October 2014 at 16:33:53 UTC, Johannes Pfau wrote:
 I think Walter sometimes suggested that it would be valid for a
 compiler to not unwind Errors at all (in release mode), but 
 simply kill
 the program and dump a error message. This would finally allow 
 us to
 optimize nothrow functions.
I think this is reasonable in general but as long as assert throws Error and assert is encouraged to be used in unittest blocks such implementation would mark compiler as unusable for me. We may need to have another look at what is truly an Error and what is not before going that path.
Oct 09 2014
next sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Thursday, 9 October 2014 at 17:31:32 UTC, Dicebot wrote:
 On Thursday, 9 October 2014 at 16:33:53 UTC, Johannes Pfau 
 wrote:
 I think Walter sometimes suggested that it would be valid for a
 compiler to not unwind Errors at all (in release mode), but 
 simply kill
 the program and dump a error message. This would finally allow 
 us to
 optimize nothrow functions.
I think this is reasonable in general but as long as assert throws Error and assert is encouraged to be used in unittest blocks such implementation would mark compiler as unusable for me.
Can it simply skip unwinding up to the next not-nothrow function? I.e. destructors of objects (and finally/scope(exit)) inside `nothrow` functions will not be executed, but unwinding will continue as normal at the first function up the call stack that supports it? Would this work for both GDC and LDC? If yes, your unittest framework will probably continue to work as is.
Oct 09 2014
parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 9 October 2014 at 18:32:55 UTC, Marc Schütz wrote:
 Can it simply skip unwinding up to the next not-nothrow 
 function? I.e. destructors of objects (and finally/scope(exit)) 
 inside `nothrow` functions will not be executed, but unwinding 
 will continue as normal at the first function up the call stack 
 that supports it?

 Would this work for both GDC and LDC? If yes, your unittest 
 framework will probably continue to work as is.
Funny that it happens other way around right now with dmd (http://dpaste.dzfl.pl/e685c0c32b0d) : struct S { string id; ~this() nothrow { import core.stdc.stdio; printf("%s\n", id.ptr); } } void foo() nothrow { auto s = S("foo"); throw new Error("hmm"); } void main() { auto s = S("main"); foo(); } ==== foo object.Error: hmm ---------------- ./f57(_Dmain+0x23) [0x4171cf] ./f57(void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll().void __lambda1()+0x18) [0x417f40] ./f57(void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate())+0x2a) [0x417e9a] ./f57(void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll()+0x30) [0x417f00] ./f57(void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate())+0x2a) [0x417e9a] ./f57(_d_run_main+0x1a3) [0x417e1b] ./f57(main+0x17) [0x4171f3] /usr/lib/libc.so.6(__libc_start_main+0xf5) [0x40967a15]
Oct 09 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/9/2014 10:31 AM, Dicebot wrote:
 On Thursday, 9 October 2014 at 16:33:53 UTC, Johannes Pfau wrote:
 I think Walter sometimes suggested that it would be valid for a
 compiler to not unwind Errors at all (in release mode), but simply kill
 the program and dump a error message. This would finally allow us to
 optimize nothrow functions.
I think this is reasonable in general but as long as assert throws Error and assert is encouraged to be used in unittest blocks such implementation would mark compiler as unusable for me.
All assert actually does is call a function in druntime. You can override and insert your own assert handling function, and have it do as you need. It was deliberately designed that way.
 We may need to have another look at what is truly an Error and what is not
 before going that path.
This involves making some hard decisions, but is worthwhile.
Oct 11 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Saturday, 11 October 2014 at 22:06:38 UTC, Walter Bright wrote:
 All assert actually does is call a function in druntime. You 
 can override and insert your own assert handling function, and 
 have it do as you need. It was deliberately designed that way.
A while ago I have been a looking for a way to throw Exception instead of Error for failing assertions inside unittest blocks but druntime didn't seem to provide needed context. Do you think this can be a worthy addition?
Oct 14 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/14/2014 8:03 PM, Dicebot wrote:
 On Saturday, 11 October 2014 at 22:06:38 UTC, Walter Bright wrote:
 All assert actually does is call a function in druntime. You can override and
 insert your own assert handling function, and have it do as you need. It was
 deliberately designed that way.
A while ago I have been a looking for a way to throw Exception instead of Error for failing assertions inside unittest blocks but druntime didn't seem to provide needed context. Do you think this can be a worthy addition?
assert() in a unittest calls: __d_unittest(string file, uint line) which calls: onUnittestErrorMsg(string file, size_t line, string msg) which calls: extern (C) void onAssertError( string file = __FILE__, size_t line = __LINE__ ) nothrow { if( _assertHandler is null ) throw new AssertError( file, line ); _assertHandler( file, line, null); } You can use setAssertHandler() to set _assertHandler to do whatever you wish. However, the compiler is still going to regard the assert() as nothrow, so the unwinding from an Exception won't happen until up stack a throwing function is encountered.
Oct 14 2014
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Wednesday, 15 October 2014 at 03:18:31 UTC, Walter Bright 
wrote:
 However, the compiler is still going to regard the assert() as 
 nothrow, so the unwinding from an Exception won't happen until 
 up stack a throwing function is encountered.
This makes impossible to have non-fatal unittests and the very reason I was looking for a replacement.
Oct 14 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/14/2014 8:36 PM, Dicebot wrote:
 On Wednesday, 15 October 2014 at 03:18:31 UTC, Walter Bright wrote:
 However, the compiler is still going to regard the assert() as nothrow, so the
 unwinding from an Exception won't happen until up stack a throwing function is
 encountered.
This makes impossible to have non-fatal unittests and the very reason I was looking for a replacement.
I don't really understand the issue. Unittests are usually run with a separate build of the app, not in the main app. When they are run in the main app, they are run before the app even gets to main(). Why do you need non-fatal unittests?
Oct 14 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 2014-10-15 07:57, Walter Bright wrote:

 Why do you need non-fatal unittests?
I don't know if this would cause problems with the current approach. But most unit test frameworks don't NOT stop on the first failure, like D does. It catches the exception, continues with the next test and in the end prints a final report. -- /Jacob Carlborg
Oct 14 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/14/2014 11:23 PM, Jacob Carlborg wrote:
 On 2014-10-15 07:57, Walter Bright wrote:

 Why do you need non-fatal unittests?
I don't know if this would cause problems with the current approach. But most unit test frameworks don't NOT stop on the first failure, like D does. It catches the exception, continues with the next test and in the end prints a final report.
I understand that, but I don't think that is what Dicebot is looking for. He's looking to recover from unittests, not just continue.
Oct 15 2014
next sibling parent Ary Borenszweig <ary esperanto.org.ar> writes:
On 10/15/14, 4:25 AM, Walter Bright wrote:
 On 10/14/2014 11:23 PM, Jacob Carlborg wrote:
 On 2014-10-15 07:57, Walter Bright wrote:

 Why do you need non-fatal unittests?
I don't know if this would cause problems with the current approach. But most unit test frameworks don't NOT stop on the first failure, like D does. It catches the exception, continues with the next test and in the end prints a final report.
I understand that, but I don't think that is what Dicebot is looking for. He's looking to recover from unittests, not just continue.
I think this means you can't get stack traces for exceptions thrown in unit tests, right?
Oct 15 2014
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Wednesday, 15 October 2014 at 07:26:03 UTC, Walter Bright 
wrote:
 On 10/14/2014 11:23 PM, Jacob Carlborg wrote:
 On 2014-10-15 07:57, Walter Bright wrote:

 Why do you need non-fatal unittests?
I don't know if this would cause problems with the current approach. But most unit test frameworks don't NOT stop on the first failure, like D does. It catches the exception, continues with the next test and in the end prints a final report.
I understand that, but I don't think that is what Dicebot is looking for. He's looking to recover from unittests, not just continue.
How can one continue without recovering? This will result in any kind of environment not being cleaned and false failures of other tests that share it.
Oct 15 2014
next sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 15 October 2014 at 14:25:43 UTC, Dicebot wrote:
 How can one continue without recovering? This will result in 
 any kind of environment not being cleaned and false failures of 
 other tests that share it.
fork()?
Oct 15 2014
next sibling parent "eles" <eles215 gzk.dot> writes:
On Wednesday, 15 October 2014 at 14:47:33 UTC, Ola Fosheim 
Grøstad wrote:
 On Wednesday, 15 October 2014 at 14:25:43 UTC, Dicebot wrote:
 How can one continue without recovering? This will result in 
 any kind of environment not being cleaned and false failures 
 of other tests that share it.
fork()?
http://check.sourceforge.net/doc/check_html/check_2.html "Writing a framework for C requires solving some special problems that frameworks for Smalltalk, Java or Python don’t have to face. In all of those language, the worst that a unit test can do is fail miserably, throwing an exception of some sort. In C, a unit test is just as likely to trash its address space as it is to fail to meet its test requirements, and if the test framework sits in the same address space, goodbye test framework. To solve this problem, Check uses the fork() system call to create a new address space in which to run each unit test, and then uses message queues to send information on the testing process back to the test framework. That way, your unit test can do all sorts of nasty things with pointers, and throw a segmentation fault, and the test framework will happily note a unit test error, and chug along. "
Oct 15 2014
prev sibling parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
"Ola Fosheim "Grøstad\"" <ola.fosheim.grostad+dlang gmail.com> writes:

 On Wednesday, 15 October 2014 at 14:25:43 UTC, Dicebot wrote:
 How can one continue without recovering? This will result in any
 kind of environment not being cleaned and false failures of other
 tests that share it.
fork()?
Forking each unittest sounds like a good solution. -- dano
Oct 16 2014
parent reply "monnoroch" <monnoroch gmail.com> writes:
Hi all!
I've read the topic and I am really surprised so many engeneers
arguing for so long and not having systematic approach to the
problem.

As I see it, Walter states that there are eenvironmental errors
and program bugs, which are non-recoverable. So use exceptions
(enforce) for first ones and asserts for the latter.

Other folks argue, that you might want to recover from a program
bug or not recover from invalid input. Also,  exception might be
itself a program bug. This is a valid point too.

So if both are true, that clearly means that the right solution
would be to introduce four categories: a cross product of the
obove:

- bugs, that are recoverable
- bugs, that are unrecoverable
- input errors, that are recoverable
- input errors, that are not

Given that, makes sence to use exceptions for recoverable errors,
doesn't matter whether they are bugs or environmental errors, and
use asserts, if you can't recover.

So, the programmer decides, if his program can recover and puts
an assert or an enforce call in his code.

The problem is, as always, with libraries. The library writer
cannot possibly decide, is some unexpected condition recoverable
or not, so he just can't put both assert and enforce into his
library function and the caller must check the arguments before
calling the function. Yes, this is annoying, but it it the only
correct way.

But what if he didn't? This brings us to error codes. Yes, they
are the best for library error handling, imo, of course, in a
form of Maby and Error monads. They are as clear as asking the
ccaller to decide, what to do with the error. But I realize, that
you, guys, are all against error codes of any kind, so...

I would say, that since the caller didn't check the arguments
himself the bug becomes unrecoverable by default and there should
be an assert, which gives stack trace, so the programmer would
insert appropriate enforces before the function call.

Finally, this brings me to the conclusion: you don't need a stack
trace in the exception, it is never a bug.
Oct 16 2014
parent "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 16 October 2014 at 18:53:22 UTC, monnoroch wrote:
 So if both are true, that clearly means that the right solution
 would be to introduce four categories: a cross product of the
 obove:

 - bugs, that are recoverable
 - bugs, that are unrecoverable
 - input errors, that are recoverable
 - input errors, that are not
Yes, I've already started a thread for this: http://forum.dlang.org/thread/zwnycclpgvfsfaactcyl forum.dlang.org but almost no one replied.
Oct 16 2014
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-10-15 16:25, Dicebot wrote:

 How can one continue without recovering? This will result in any kind of
 environment not being cleaned and false failures of other tests that
 share it.
I will probably use something else than "assert" in my unit tests. Something like assertEq, assertNotEq and so on. It's more flexible, can give better error message and I can have it throw an exception instead of an error. But there's still the problem with asserts in contracts and other parts of the code. -- /Jacob Carlborg
Oct 15 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 16 October 2014 at 06:11:46 UTC, Jacob Carlborg 
wrote:
 On 2014-10-15 16:25, Dicebot wrote:

 How can one continue without recovering? This will result in 
 any kind of
 environment not being cleaned and false failures of other 
 tests that
 share it.
I will probably use something else than "assert" in my unit tests. Something like assertEq, assertNotEq and so on. It's more flexible, can give better error message and I can have it throw an exception instead of an error. But there's still the problem with asserts in contracts and other parts of the code.
This is what we are using right now: public void test ( char[] op, T1, T2 ) ( T1 a, T2 b, char[] file = __FILE__, size_t line = __LINE__ ) { enforce!(op, TestException)(a, b, file, line); } but it won't work well with 3d-party libraries that use assertions.
Oct 16 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/16/2014 12:21 PM, Dicebot wrote:
 On Thursday, 16 October 2014 at 06:11:46 UTC, Jacob Carlborg wrote:
 On 2014-10-15 16:25, Dicebot wrote:

 How can one continue without recovering? This will result in any kind of
 environment not being cleaned and false failures of other tests that
 share it.
I will probably use something else than "assert" in my unit tests. Something like assertEq, assertNotEq and so on. It's more flexible, can give better error message and I can have it throw an exception instead of an error. But there's still the problem with asserts in contracts and other parts of the code.
This is what we are using right now: public void test ( char[] op, T1, T2 ) ( T1 a, T2 b, char[] file = __FILE__, size_t line = __LINE__ ) { enforce!(op, TestException)(a, b, file, line); } but it won't work well with 3d-party libraries that use assertions.
Ok, but why would 3rd party library unittests be a concern? They shouldn't have shipped it if their own unittests fail - that's the whole point of having unittests.
Oct 16 2014
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 16 October 2014 at 19:35:40 UTC, Walter Bright wrote:
 Ok, but why would 3rd party library unittests be a concern? 
 They shouldn't have shipped it if their own unittests fail - 
 that's the whole point of having unittests.
Libraries tend to be forked and modified. Libraries aren't always tested in environment similar to specific production case. At the same time not being able to use same test runner in all Continious Integration jobs greatly reduces the value of having standard unittest blocks in the very first place.
Oct 16 2014
next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 16 October 2014 at 19:56:57 UTC, Dicebot wrote:
 Libraries tend to be forked and modified. Libraries aren't 
 always tested in environment similar to specific production 
 case.
This seems relevant: http://www.tele-task.de/archive/video/flash/16130/
Oct 16 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/16/2014 12:56 PM, Dicebot wrote:
 On Thursday, 16 October 2014 at 19:35:40 UTC, Walter Bright wrote:
 Ok, but why would 3rd party library unittests be a concern? They shouldn't
 have shipped it if their own unittests fail - that's the whole point of having
 unittests.
Libraries tend to be forked and modified.
If you're willing to go that far, then yes, you do wind up owning the unittests, in which case s/assert/myassert/ should do it.
 Libraries aren't always tested in
 environment similar to specific production case.
Unittests should not be testing their environment. They should be testing the function's logic, and should mock up input for them as required.
 At the same time not being able
 to use same test runner in all Continious Integration jobs greatly reduces the
 value of having standard unittest blocks in the very first place.
I understand that, but wouldn't you be modifying the unittests anyway if using an external test runner tool?
Oct 16 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 16 October 2014 at 20:18:04 UTC, Walter Bright wrote:
 On 10/16/2014 12:56 PM, Dicebot wrote:
 On Thursday, 16 October 2014 at 19:35:40 UTC, Walter Bright 
 wrote:
 Ok, but why would 3rd party library unittests be a concern? 
 They shouldn't
 have shipped it if their own unittests fail - that's the 
 whole point of having
 unittests.
Libraries tend to be forked and modified.
If you're willing to go that far, then yes, you do wind up owning the unittests, in which case s/assert/myassert/ should do it.
Which means changing almost all sources and resolving conflicts upon each merge. Forking library for few tweaks is not "going that far", it is an absolutely minor routine. It also complicates propagating changes back upstream because all tests need to be re-adjusted back to original style.
 Libraries aren't always tested in
 environment similar to specific production case.
Unittests should not be testing their environment. They should be testing the function's logic, and should mock up input for them as required.
compiler version, libc version, kernel version - all it can affect behaviour or pretty self-contained function. Perfectly tested library is as much reality as program with 0 bugs.
 At the same time not being able
 to use same test runner in all Continious Integration jobs 
 greatly reduces the
 value of having standard unittest blocks in the very first 
 place.
I understand that, but wouldn't you be modifying the unittests anyway if using an external test runner tool?
No, right now one can affect the way tests are run by simply replacing the runner to a custom one and it will work for any amount of modules compiled in. Beauty of `unittest` block approach is that it is simply a bunch of functions that are somewhat easy to discover from the combined sources of the program - custom runner can do pretty much anything with those. Or it could if not the issue with AssertError and cleanup.
Oct 16 2014
parent reply "Atila Neves" <atila.neves gmail.com> writes:
 No, right now one can affect the way tests are run by simply 
 replacing the runner to a custom one and it will work for any 
 amount of modules compiled in. Beauty of `unittest` block 
 approach is that it is simply a bunch of functions that are 
 somewhat easy to discover from the combined sources of the 
 program - custom runner can do pretty much anything with those. 
 Or it could if not the issue with AssertError and cleanup.
Is cleaning up in a unittest build a problem? I'd say no, if it the tests fail it doesn't make much sense to clean up unless it affects the reporting of tests failing. I catch assertion errors in unit-threaded exactly to support the standard unittest blocks and can't see why I'd care about clean-up. At least in practice it hasn't been an issue, although to be fair I haven't used that functionality a lot (of using unit-threaded to run unittest blocks). Atila
Oct 17 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 2014-10-17 10:26, Atila Neves wrote:

 Is cleaning up in a unittest build a problem? I'd say no, if it the
 tests fail it doesn't make much sense to clean up unless it affects the
 reporting of tests failing.
I have used files in some of my unit tests. I would certainly like those to be properly closed if a tests failed (for whatever reason). Now, some of you will argue that one shouldn't use files in unit tests. But that would only work in a ideal and perfect world, which we don't live in. -- /Jacob Carlborg
Oct 17 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/17/2014 9:10 AM, Jacob Carlborg wrote:
 On 2014-10-17 10:26, Atila Neves wrote:

 Is cleaning up in a unittest build a problem? I'd say no, if it the
 tests fail it doesn't make much sense to clean up unless it affects the
 reporting of tests failing.
I have used files in some of my unit tests. I would certainly like those to be properly closed if a tests failed (for whatever reason). Now, some of you will argue that one shouldn't use files in unit tests. But that would only work in a ideal and perfect world, which we don't live in.
This should be fairly straightforward to deal with: 1. Write functions to input/output from/to ranges instead of files. Then, have the unittests "mock up" input to drive them that does not come from files. I've used this technique very successfully in Warp. 2. If (1) cannot be done, then write the unittests like: { openfile(); scope (exit) closefile(); scope (failure) assert(0); ... use enforce() instead of assert() ... } 3. In a script that compiles/runs the unittests, have the script delete any extraneous generated files.
Oct 17 2014
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Saturday, 18 October 2014 at 05:22:54 UTC, Walter Bright wrote:
 2. If (1) cannot be done, then write the unittests like:

   {
     openfile();
     scope (exit) closefile();
     scope (failure) assert(0);
     ... use enforce() instead of assert() ...
   }

 3. In a script that compiles/runs the unittests, have the 
 script delete any extraneous generated files.
This is bad, it means: - I risk having my filesystem ruined by running unit-tests through the compiler. - The test environment changes between runs. Built in unit tests should have no side effects. Something along these lines would be a better setup: 1. Load a filesystem from a read-only file to a virtual driver. 2. Run a special initializer for unit tests to set up the in-memory test environment. 3. Create N forks (N=number of cores): 4. Fork the filesystem/program before running a single unit test. 5. Mount the virtual filesystem (from 1) 6. Run the unit test 7. Collect result from child process and print result. 8. goto 4 But just banning writing to resources would be more suitable. D unit tests are only suitable for testing simple library code anyway.
Oct 17 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-10-16 21:35, Walter Bright wrote:

 Ok, but why would 3rd party library unittests be a concern? They
 shouldn't have shipped it if their own unittests fail - that's the whole
 point of having unittests.
They will have asserts in contracts and other parts of that code that is not unit tests. -- /Jacob Carlborg
Oct 17 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/17/2014 9:05 AM, Jacob Carlborg wrote:
 On 2014-10-16 21:35, Walter Bright wrote:

 Ok, but why would 3rd party library unittests be a concern? They
 shouldn't have shipped it if their own unittests fail - that's the whole
 point of having unittests.
They will have asserts in contracts and other parts of that code that is not unit tests.
This particular subthread is about unittests.
Oct 17 2014
parent Jacob Carlborg <doob me.com> writes:
On 2014-10-18 06:36, Walter Bright wrote:

 This particular subthread is about unittests.
That doesn't make the problem go away. -- /Jacob Carlborg
Oct 18 2014
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/15/2014 7:25 AM, Dicebot wrote:
 How can one continue without recovering? This will result in any kind of
 environment not being cleaned and false failures of other tests that share it.
Unittest asserts are top level - they shouldn't need recovering from (i.e. unwinding). Just continuing.
Oct 16 2014
prev sibling parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
Walter Bright <newshound2 digitalmars.com> writes:

 On 10/14/2014 11:23 PM, Jacob Carlborg wrote:
 On 2014-10-15 07:57, Walter Bright wrote:

 Why do you need non-fatal unittests?
I don't know if this would cause problems with the current approach. But most unit test frameworks don't NOT stop on the first failure, like D does. It catches the exception, continues with the next test and in the end prints a final report.
I understand that, but I don't think that is what Dicebot is looking for. He's looking to recover from unittests, not just continue.
That is what I am looking for, just being able to continue from a failed assert in a unittest. On iOS, it is easier to build one app with all unit tests. This is because I don't know of a way to automate the download/run of a bunch of smaller unittests. The test driver catches Throwables, records failure, then goes on to the next test. After catching up on this thread, I feel like unittests should throw an Exceptions.
Oct 15 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/15/2014 7:35 AM, Dan Olson wrote:
 That is what I am looking for, just being able to continue from a failed
 assert in a unittest.
Just use enforce() or something similar instead of assert(). Nothing says you have to use assert() in a unittest.
Oct 16 2014
parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
Walter Bright <newshound2 digitalmars.com> writes:

 On 10/15/2014 7:35 AM, Dan Olson wrote:
 That is what I am looking for, just being able to continue from a failed
 assert in a unittest.
Just use enforce() or something similar instead of assert(). Nothing says you have to use assert() in a unittest.
Makes sense. However it is druntime and phobos unittests that already use assert. I have convinced myself that catching Throwable is just fine in my case because at worst, unittests that follow an Error might be tainted, but only a perfect score of passing all tests really counts. -- dano
Oct 16 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/16/2014 8:36 AM, Dan Olson wrote:
 Walter Bright <newshound2 digitalmars.com> writes:

 On 10/15/2014 7:35 AM, Dan Olson wrote:
 That is what I am looking for, just being able to continue from a failed
 assert in a unittest.
Just use enforce() or something similar instead of assert(). Nothing says you have to use assert() in a unittest.
Makes sense. However it is druntime and phobos unittests that already use assert. I have convinced myself that catching Throwable is just fine in my case because at worst, unittests that follow an Error might be tainted, but only a perfect score of passing all tests really counts.
I don't understand why unittests in druntime/phobos are an issue for users. We don't release a DMD unless they all pass - it should be moot for users.
Oct 16 2014
next sibling parent Dan Olson <zans.is.for.cans yahoo.com> writes:
Walter Bright <newshound2 digitalmars.com> writes:
 I don't understand why unittests in druntime/phobos are an issue for
 users. We don't release a DMD unless they all pass - it should be moot
 for users.
I think some context was lost. This is different. I am making mods to LDC, druntime, and phobos to target iPhones and iPads (ARM-iOS). I also can't claim victory until all unittests pass.
Oct 16 2014
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2014-10-16 20:50, Walter Bright wrote:

 I don't understand why unittests in druntime/phobos are an issue for
 users. We don't release a DMD unless they all pass - it should be moot
 for users.
There are asserts elsewhere in the code. -- /Jacob Carlborg
Oct 17 2014
prev sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Wednesday, 15 October 2014 at 03:18:31 UTC, Walter Bright 
wrote:
 However, the compiler is still going to regard the assert() as 
 nothrow, so the unwinding from an Exception won't happen until 
 up stack a throwing function is encountered.
I hate to say it, but I'm inclined to treat nothrow the same as in C++, which is to basically pretend it's not a part of the language. The efficiency is nice, but not if it means that throwing an Error will cause the program to be invalid. Please tell me there's no plan to change the unwinding behavior when Error is thrown in standard (ie not nothrow) code. I touched on all this in my "on errors" thread that seems to have died. I suppose I could write a DIP but I was hoping for discussion.
Oct 15 2014
next sibling parent reply "Elena" <elena.works gmail.com> writes:
Hi,every one!
How can I realize mathematical rounding (not banking) for Double 
in D
0.4 -> 0
0.5 -> 1
1.5 -> 2
2.5 -> 3
3.5 -> 4  ?

Thank you.
Oct 15 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/15/14, 8:11 PM, Elena wrote:
 Hi,every one!
 How can I realize mathematical rounding (not banking) for Double in D
 0.4 -> 0
 0.5 -> 1
 1.5 -> 2
 2.5 -> 3
 3.5 -> 4  ?

 Thank you.
Add 0.5 and then cast to integral. -- Andrei
Oct 15 2014
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Thursday, 16 October 2014 at 05:24:23 UTC, Andrei Alexandrescu 
wrote:
 On 10/15/14, 8:11 PM, Elena wrote:
 Hi,every one!
 How can I realize mathematical rounding (not banking) for 
 Double in D
 0.4 -> 0
 0.5 -> 1
 1.5 -> 2
 2.5 -> 3
 3.5 -> 4  ?

 Thank you.
Add 0.5 and then cast to integral. -- Andrei
This would be the "round half up" mode that produce 0 for -0.5: floor(x+0.5) Whereas "round nearest" should produce -1 for -0.5: floor(abs(x)+0.5)*sgn(x)
Oct 16 2014
prev sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Thursday, 16 October 2014 at 03:11:57 UTC, Elena wrote:
 Hi,every one!
 How can I realize mathematical rounding (not banking) for
Never used it, but: http://dlang.org/library/std/math/FloatingPointControl.html phobos/math.d lists the enums: roundToNearest, roundDown, roundUp, roundToZero
Oct 15 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/15/2014 6:54 PM, Sean Kelly wrote:
 I hate to say it, but I'm inclined to treat nothrow the same as in C++, which
is
 to basically pretend it's not a part of the language. The efficiency is nice,
 but not if it means that throwing an Error will cause the program to be
 invalid.  Please tell me there's no plan to change the unwinding behavior when
 Error is thrown in standard (ie not nothrow) code.
Don't throw Errors when you need to unwind. Throw Exceptions. I.e. use enforce() instead of assert().
Oct 16 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 16 October 2014 at 07:44:37 UTC, Walter Bright wrote:
 On 10/15/2014 6:54 PM, Sean Kelly wrote:
 I hate to say it, but I'm inclined to treat nothrow the same 
 as in C++, which is
 to basically pretend it's not a part of the language. The 
 efficiency is nice,
 but not if it means that throwing an Error will cause the 
 program to be
 invalid.  Please tell me there's no plan to change the 
 unwinding behavior when
 Error is thrown in standard (ie not nothrow) code.
Don't throw Errors when you need to unwind. Throw Exceptions. I.e. use enforce() instead of assert().
I'm more concerned about Phobos. If it uses nothrow and asserts in preconditions then the decision has been made for me.
Oct 16 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/16/2014 6:46 AM, Sean Kelly wrote:
 On Thursday, 16 October 2014 at 07:44:37 UTC, Walter Bright wrote:
 Don't throw Errors when you need to unwind. Throw Exceptions. I.e. use
 enforce() instead of assert().
I'm more concerned about Phobos. If it uses nothrow and asserts in preconditions then the decision has been made for me.
Which function(s) in particular?
Oct 16 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 16 October 2014 at 18:49:13 UTC, Walter Bright wrote:
 On 10/16/2014 6:46 AM, Sean Kelly wrote:
 On Thursday, 16 October 2014 at 07:44:37 UTC, Walter Bright 
 wrote:
 Don't throw Errors when you need to unwind. Throw Exceptions. 
 I.e. use enforce() instead of assert().
I'm more concerned about Phobos. If it uses nothrow and asserts in preconditions then the decision has been made for me.
Which function(s) in particular?
Nothing specifically... which is kind of the problem. If I call an impure nothrow function, it's possible the function accesses shared state that will not be properly cleaned up in the event of a thrown Error--say it contains a synchronized block, for example. So even if I can be sure that the problem that resulted in an Error being thrown did not corrupt program state, I can't be sure that the failure to unwind did not as well. That said, I'm inclined to say that this is only a problem because of how many things are classified as Errors at this point. If contracts used some checking mechanism other than assert, perhaps this would be enough. Again I'll refer to my "on errors" post that gets into this a bit. Using two broad categories: exceptions and errors, is unduly limiting.
Oct 16 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/16/2014 12:08 PM, Sean Kelly wrote:
 On Thursday, 16 October 2014 at 18:49:13 UTC, Walter Bright wrote:
 On 10/16/2014 6:46 AM, Sean Kelly wrote:
 On Thursday, 16 October 2014 at 07:44:37 UTC, Walter Bright wrote:
 Don't throw Errors when you need to unwind. Throw Exceptions. I.e. use
 enforce() instead of assert().
I'm more concerned about Phobos. If it uses nothrow and asserts in preconditions then the decision has been made for me.
Which function(s) in particular?
Nothing specifically... which is kind of the problem. If I call an impure nothrow function, it's possible the function accesses shared state that will not be properly cleaned up in the event of a thrown Error--say it contains a synchronized block, for example. So even if I can be sure that the problem that resulted in an Error being thrown did not corrupt program state, I can't be sure that the failure to unwind did not as well.
Contract errors in Phobos/Druntime should be limited to having passed it invalid arguments, which should be documented, or simply that the function has a bug in it, or that it ran out of memory (which is generally not recoverable anyway). I.e. I'm not seeing where this is a practical problem.
 That said, I'm inclined to say that this is only a problem
 because of how many things are classified as Errors at this
 point.  If contracts used some checking mechanism other than
 assert, perhaps this would be enough.  Again I'll refer to my "on
 errors" post that gets into this a bit.  Using two broad
 categories: exceptions and errors, is unduly limiting.
My initial impression is that there's so much confusion about what should be an Error and what should be an Exception, that adding a third category will not improve things.
Oct 16 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 2014-10-16 21:31, Walter Bright wrote:

 Contract errors in Phobos/Druntime should be limited to having passed it
 invalid arguments, which should be documented
That doesn't mean it won't happen. -- /Jacob Carlborg
Oct 17 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/17/2014 9:13 AM, Jacob Carlborg wrote:
 On 2014-10-16 21:31, Walter Bright wrote:

 Contract errors in Phobos/Druntime should be limited to having passed it
 invalid arguments, which should be documented
That doesn't mean it won't happen.
Which means they'll be program bugs, not environmental errors. It is of great value to distinguish between program bugs and input/environmental errors, and to treat them entirely differently. It makes code easier to understand, more robust, and better/faster code can be generated. Using asserts to detect input/environmental errors is a bad practice - something like enforce() should be used instead. I understand that some have to work with poorly written libraries that incorrectly use assert. If that's the only issue with those libraries, you're probably lucky :-) Short term, I suggest editing the code of those libraries, and pressuring the authors of them. Longer term, we need to establish a culture of using assert/enforce correctly. This is not as pie-in-the-sky as it sounds. Over the years, a lot of formerly popular bad practices in C and C++ have been relentlessly driven out of existence by getting the influential members of the communities to endorse and advocate proper best practices. ---------------------- I do my best to practice what I preach. In the DMD source code, an assert tripping always, by definition, means it's a compiler bug. It is never used to signal errors in code being compiled or environmental errors. If a badly formed .d file causes dmd to assert, it is always a BUG in dmd.
Oct 17 2014
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-10-18 07:09, Walter Bright wrote:

 Which means they'll be program bugs, not environmental errors.
Yes, but just because I made a mistake in using a function (hitting an assert) doesn't mean I want to have undefined behavior. -- /Jacob Carlborg
Oct 18 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/18/2014 8:21 AM, Jacob Carlborg wrote:
 On 2014-10-18 07:09, Walter Bright wrote:

 Which means they'll be program bugs, not environmental errors.
Yes, but just because I made a mistake in using a function (hitting an assert) doesn't mean I want to have undefined behavior.
As I've said before, tripping an assert by definition means the program has entered an unknown state. I don't believe it is possible for any language to make guarantees beyond that point. Now, if it is a "known" unknown state, and you want to recover, the solution is straightforward - use enforce(). enforce() offers the guarantees you're asking for. Using assert() when you mean enforce() is like pulling the fire alarm but not wanting the fire dept. to show up.
Oct 18 2014
next sibling parent reply "eles" <eles215 gzk.dot> writes:
On Saturday, 18 October 2014 at 17:40:43 UTC, Walter Bright wrote:
 On 10/18/2014 8:21 AM, Jacob Carlborg wrote:
 On 2014-10-18 07:09, Walter Bright wrote:

 Which means they'll be program bugs, not environmental errors.
Yes, but just because I made a mistake in using a function (hitting an assert) doesn't mean I want to have undefined behavior.
As I've said before, tripping an assert by definition means the program has entered an unknown state. I don't believe it is possible for any language to make guarantees beyond that point.
What about using the contracts of a fucntion to optimize? They are mainly asserts, after all.
Oct 20 2014
parent reply "rst256" <ussr.24 yandex.ru> writes:
On Monday, 20 October 2014 at 20:36:58 UTC, eles wrote:
 On Saturday, 18 October 2014 at 17:40:43 UTC, Walter Bright 
 wrote:
 On 10/18/2014 8:21 AM, Jacob Carlborg wrote:
 On 2014-10-18 07:09, Walter Bright wrote:

 Which means they'll be program bugs, not environmental 
 errors.
Yes, but just because I made a mistake in using a function (hitting an assert) doesn't mean I want to have undefined behavior.
As I've said before, tripping an assert by definition means the program has entered an unknown state. I don't believe it is possible for any language to make guarantees beyond that point.
What about using the contracts of a fucntion to optimize? They are mainly asserts, after all.
this(errnoEnforce(.fopen(name, stdioOpenmode), text("Cannot open file `", name, "' in mode `", stdioOpenmode, "'")), name); making a couple instances of classes not knowing whether they are necessary at all, the performance did not cry. And why do you have all type of cars. Is it really you are so good compiler?
 What about using the contracts of a fucntion to optimize? They
Its linking time.
 is possible for any language to make guarantees beyond that
Of cous no, I will explain later 2-3 hour/ Sorry bisnes offtop: string noexist_file_name = "bag_file_global"; {writefln("------ begin scope: after"); auto fobj = File(noexist_file_name); scope(failure) writefln("test1.failure"); scope(exit) writefln("test1.exit"); } std.exception.ErrnoException std\stdio.d(362): Cannot open file `bag_... --------------- 5 line with only a memory addr 0x7C81077 in RegisterWaitForInputIdle i think you need stoped after first errmsg see exception.d: in constructor or class ErrnoException : Exception
Oct 20 2014
next sibling parent "rst256" <ussr.24 yandex.ru> writes:
On Tuesday, 21 October 2014 at 03:25:55 UTC, rst256 wrote:
In this post i forgot make correction machine translation.
I am so sorry!
 On Monday, 20 October 2014 at 20:36:58 UTC, eles wrote:
 On Saturday, 18 October 2014 at 17:40:43 UTC, Walter Bright 
 wrote:
 On 10/18/2014 8:21 AM, Jacob Carlborg wrote:
 On 2014-10-18 07:09, Walter Bright wrote:

 Which means they'll be program bugs, not environmental 
 errors.
Yes, but just because I made a mistake in using a function (hitting an assert) doesn't mean I want to have undefined behavior.
As I've said before, tripping an assert by definition means the program has entered an unknown state. I don't believe it is possible for any language to make guarantees beyond that point.
What about using the contracts of a fucntion to optimize? They are mainly asserts, after all.
this(errnoEnforce(.fopen(name, stdioOpenmode), text("Cannot open file `", name, "' in mode `", stdioOpenmode, "'")), name); making a couple instances of classes not knowing whether they are necessary at all, the performance did not cry. And why do you have all type of cars. Is it really you are so good compiler?
 What about using the contracts of a fucntion to optimize? They
Its linking time.
 is possible for any language to make guarantees beyond that
Of cous no, I will explain later 2-3 hour/ Sorry bisnes offtop: string noexist_file_name = "bag_file_global"; {writefln("------ begin scope: after"); auto fobj = File(noexist_file_name); scope(failure) writefln("test1.failure"); scope(exit) writefln("test1.exit"); } std.exception.ErrnoException std\stdio.d(362): Cannot open file `bag_... --------------- 5 line with only a memory addr 0x7C81077 in RegisterWaitForInputIdle i think you need stoped after first errmsg see exception.d: in constructor or class ErrnoException : Exception
Oct 21 2014
prev sibling parent "eles" <eles eles.com> writes:
On Tuesday, 21 October 2014 at 03:25:55 UTC, rst256 wrote:
 On Monday, 20 October 2014 at 20:36:58 UTC, eles wrote:
 On Saturday, 18 October 2014 at 17:40:43 UTC, Walter Bright 
 wrote:
 On 10/18/2014 8:21 AM, Jacob Carlborg wrote:
 On 2014-10-18 07:09, Walter Bright wrote:
 Its linking time.
No, it's not. At least if you move them in the generated .di files along with function prototypes. Basically, you would pull more source from .d files into .di files.
Oct 22 2014
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/18/2014 07:40 PM, Walter Bright wrote:
 As I've said before, tripping an assert by definition means the program
 has entered an unknown state. I don't believe it is possible for any
 language to make guarantees beyond that point.
What about the guarantee that your compiler didn't _intentionally_ screw them completely?
Oct 20 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2014 1:54 PM, Timon Gehr wrote:
 On 10/18/2014 07:40 PM, Walter Bright wrote:
 As I've said before, tripping an assert by definition means the program
 has entered an unknown state. I don't believe it is possible for any
 language to make guarantees beyond that point.
What about the guarantee that your compiler didn't _intentionally_ screw them completely?
What does that mean?
Oct 20 2014
parent "rst256" <ussr.24 yandex.ru> writes:
in the extreme case, if you want to change anything.
or as a way to shut problem quickly
add to debugging symbols generation algorithm info about file and 
line
- Walter?
Oct 23 2014
prev sibling next sibling parent reply "w0rp" <devw0rp gmail.com> writes:
On Saturday, 18 October 2014 at 17:40:43 UTC, Walter Bright wrote:
 On 10/18/2014 8:21 AM, Jacob Carlborg wrote:
 On 2014-10-18 07:09, Walter Bright wrote:

 Which means they'll be program bugs, not environmental errors.
Yes, but just because I made a mistake in using a function (hitting an assert) doesn't mean I want to have undefined behavior.
As I've said before, tripping an assert by definition means the program has entered an unknown state. I don't believe it is possible for any language to make guarantees beyond that point. Now, if it is a "known" unknown state, and you want to recover, the solution is straightforward - use enforce(). enforce() offers the guarantees you're asking for. Using assert() when you mean enforce() is like pulling the fire alarm but not wanting the fire dept. to show up.
I agree with you on this. I've only ever used assert() for expressing, "This should never happen." There's a difference between "this might happen if the environment goes wrong," which is like a tire being popped in car, and "this should never happen," which is like a car turning into King Kong and flying away. My most common assert in D is typically assert(x !is null) for demanding that objects are initialised.
Oct 22 2014
parent "eles" <eles eles.com> writes:
On Wednesday, 22 October 2014 at 15:05:58 UTC, w0rp wrote:
 On Saturday, 18 October 2014 at 17:40:43 UTC, Walter Bright 
 wrote:
 On 10/18/2014 8:21 AM, Jacob Carlborg wrote:
 On 2014-10-18 07:09, Walter Bright wrote:
 never happen," which is like a car turning into King Kong and
It depends on the environment: http://i.dailymail.co.uk/i/pix/2011/03/27/article-1370559-0B499D2E00000578-931_634x470.jpg
Oct 22 2014
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 18/10/2014 18:40, Walter Bright wrote:
 As I've said before, tripping an assert by definition means the program
 has entered an unknown state. I don't believe it is possible for any
 language to make guarantees beyond that point.
The guarantees (if any), would not be made by the language, but by the programmer. The language cannot know if a program is totally broken and undefined when an assert fails, but a programmer can, for each particular assert, make some assumptions about which fault domains (like Sean put it) can be affected and which are not. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Oct 29 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/29/2014 5:37 AM, Bruno Medeiros wrote:
 On 18/10/2014 18:40, Walter Bright wrote:
 As I've said before, tripping an assert by definition means the program
 has entered an unknown state. I don't believe it is possible for any
 language to make guarantees beyond that point.
The guarantees (if any), would not be made by the language, but by the programmer. The language cannot know if a program is totally broken and undefined when an assert fails, but a programmer can, for each particular assert, make some assumptions about which fault domains (like Sean put it) can be affected and which are not.
Assumptions are not guarantees. In any case, if the programmer knows than assert error is restricted to a particular domain, and is recoverable, and wants to recover from it, use enforce(), not assert().
Oct 29 2014
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2014-10-29 22:22, Walter Bright wrote:

 Assumptions are not guarantees.

 In any case, if the programmer knows than assert error is restricted to
 a particular domain, and is recoverable, and wants to recover from it,
 use enforce(), not assert().
I really don't like "enforce". It encourage the use of plain Exception instead of a subclass. -- /Jacob Carlborg
Oct 30 2014
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 29 October 2014 at 21:23:00 UTC, Walter Bright 
wrote:
 In any case, if the programmer knows than assert error is 
 restricted to a particular domain, and is recoverable, and 
 wants to recover from it, use enforce(), not assert().
But all that does is working around the assert's behavior to ignore cleanups. Maybe, when it's known, that a failure is not restricted, some different way of failure reporting should be used?
Nov 01 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/1/2014 4:14 AM, Kagamin wrote:
 On Wednesday, 29 October 2014 at 21:23:00 UTC, Walter Bright wrote:
 In any case, if the programmer knows than assert error is restricted to a
 particular domain, and is recoverable, and wants to recover from it, use
 enforce(), not assert().
But all that does is working around the assert's behavior to ignore cleanups.
It is not "working around" anything unless you're trying to use a screwdriver as a hammer. Cleanups are not appropriate after a program has entered an unknown state.
 Maybe, when it's known, that a failure is not restricted, some different way of
 failure reporting should be used?
assert() and enforce() both work as designed.
Nov 01 2014
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 29/10/2014 21:22, Walter Bright wrote:
 On 10/29/2014 5:37 AM, Bruno Medeiros wrote:
 On 18/10/2014 18:40, Walter Bright wrote:
 As I've said before, tripping an assert by definition means the program
 has entered an unknown state. I don't believe it is possible for any
 language to make guarantees beyond that point.
The guarantees (if any), would not be made by the language, but by the programmer. The language cannot know if a program is totally broken and undefined when an assert fails, but a programmer can, for each particular assert, make some assumptions about which fault domains (like Sean put it) can be affected and which are not.
Assumptions are not guarantees.
Let me give an example: double sqrt(double num) { assert(num >= 0); ... With just this, then purely from a compiler/language viewpoint, if the assert is triggered the *language* doesn't know if the whole program is corrupted (formatting the hard disk, etc.), or if the fault is localized there, and an error/exception can be thrown cleanly (clean in the sense that other parts of the program are not corrupted). So the language doesn't know, but the *programmer* can make a reasoning in each particular assert of which domains/components of the program are affected by that assertion failure. In the sqrt() case above, the programmer can easily state that the math library that sqrt is part of is not corrupted, and its state is not totally unknown (as in, it's not deadlocked, nor is it formatting the hard disk!). That being the case, sqrt() can be made to throw an exception, and then that "assertion" failure can be recovered cleanly. Which leads to what you say next:
 In any case, if the programmer knows than assert error is restricted to
 a particular domain, and is recoverable, and wants to recover from it,
 use enforce(), not assert().
Very well then. But then we'll get to the point where enforce() will become much more popular than assert to check for contract conditions. assert() will be relegated to niche and rare situations where the program cant really know how to continue/recover cleanly (memory corruption for example). That idiom is fine with me actually - but then the documentation for assert should reflect that. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Nov 07 2014
next sibling parent "Dicebot" <public dicebot.lv> writes:
On Friday, 7 November 2014 at 15:00:35 UTC, Bruno Medeiros wrote:
 Very well then. But then we'll get to the point where enforce() 
 will become much more popular than assert to check for contract 
 conditions. assert() will be relegated to niche and rare 
 situations where the program cant really know how to 
 continue/recover cleanly (memory corruption for example).

 That idiom is fine with me actually - but then the 
 documentation for assert should reflect that.
This looks like a only practical solution for me right now - but it is complicated by the fact that assert is not the only Error in the library code.
Nov 09 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/7/2014 7:00 AM, Bruno Medeiros wrote:
 Let me give an example:

 double sqrt(double num) {
    assert(num >= 0);
    ...

 With just this, then purely from a compiler/language viewpoint, if the assert
is
 triggered the *language* doesn't know if the whole program is corrupted
 (formatting the hard disk, etc.), or if the fault is localized there, and an
 error/exception can be thrown cleanly (clean in the sense that other parts of
 the program are not corrupted).

 So the language doesn't know, but the *programmer* can make a reasoning in each
 particular assert of which domains/components of the program are affected by
 that assertion failure. In the sqrt() case above, the programmer can easily
 state that the math library that sqrt is part of is not corrupted, and its
state
 is not totally unknown (as in, it's not deadlocked, nor is it formatting the
 hard disk!).
Making such an assumption presumes that the programmer knows the SOURCE of the bug. He does not. The source could be a buffer overflow, a wild pointer, any sort of corruption.
 Very well then. But then we'll get to the point where enforce() will become
much
 more popular than assert to check for contract conditions. assert() will be
 relegated to niche and rare situations where the program cant really know how
to
 continue/recover cleanly (memory corruption for example).

 That idiom is fine with me actually - but then the documentation for assert
 should reflect that.
I created this thread because it is an extremely important topic. It has come up again and again for my entire career. There is no such thing as knowing in advance what caused a bug, and that the bug is "safe" to continue from. If you know in advance what caused it, then it becomes expected program behavior, and is not a bug. assert() is for bug detection, detecting state that should have never happened. By definition you cannot know it is "safe", you cannot know what caused it. enforce() is for dealing with known, expected conditions.
Nov 09 2014
next sibling parent reply "eles" <eles eles.com> writes:
On Sunday, 9 November 2014 at 21:34:05 UTC, Walter Bright wrote:
 On 11/7/2014 7:00 AM, Bruno Medeiros wrote:
 assert() is for bug detection, detecting state that should have 
 never happened. By definition you cannot know it is "safe", you 
 cannot know what caused it.

 enforce() is for dealing with known, expected conditions.
This is clear. The missing piece is a way to make the compile enforce that use on the user. Code review alone does not work.
Nov 09 2014
parent "eles" <eles eles.com> writes:
On Sunday, 9 November 2014 at 21:59:19 UTC, eles wrote:
 On Sunday, 9 November 2014 at 21:34:05 UTC, Walter Bright wrote:
 On 11/7/2014 7:00 AM, Bruno Medeiros wrote:
 assert() is for bug detection, detecting state that should 
 have never happened. By definition you cannot know it is 
 "safe", you cannot know what caused it.

 enforce() is for dealing with known, expected conditions.
This is clear. The missing piece is a way to make the compile enforce that use on the user. Code review alone does not work.
This is clear. The missing piece is a way to make the compiler enforce that separate use on the user. Code review alone does not work.
Nov 09 2014
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 09/11/2014 21:33, Walter Bright wrote:
 On 11/7/2014 7:00 AM, Bruno Medeiros wrote:
 Let me give an example:

 double sqrt(double num) {
    assert(num >= 0);
    ...

 With just this, then purely from a compiler/language viewpoint, if the
 assert is
 triggered the *language* doesn't know if the whole program is corrupted
 (formatting the hard disk, etc.), or if the fault is localized there,
 and an
 error/exception can be thrown cleanly (clean in the sense that other
 parts of
 the program are not corrupted).

 So the language doesn't know, but the *programmer* can make a
 reasoning in each
 particular assert of which domains/components of the program are
 affected by
 that assertion failure. In the sqrt() case above, the programmer can
 easily
 state that the math library that sqrt is part of is not corrupted, and
 its state
 is not totally unknown (as in, it's not deadlocked, nor is it
 formatting the
 hard disk!).
Making such an assumption presumes that the programmer knows the SOURCE of the bug. He does not. The source could be a buffer overflow, a wild pointer, any sort of corruption.
 Very well then. But then we'll get to the point where enforce() will
 become much
 more popular than assert to check for contract conditions. assert()
 will be
 relegated to niche and rare situations where the program cant really
 know how to
 continue/recover cleanly (memory corruption for example).

 That idiom is fine with me actually - but then the documentation for
 assert
 should reflect that.
I created this thread because it is an extremely important topic. It has come up again and again for my entire career. There is no such thing as knowing in advance what caused a bug, and that the bug is "safe" to continue from. If you know in advance what caused it, then it becomes expected program behavior, and is not a bug. assert() is for bug detection, detecting state that should have never happened. By definition you cannot know it is "safe", you cannot know what caused it. enforce() is for dealing with known, expected conditions.
As I mentioned before, it's not about knowing exactly what caused it, nor knowing for sure if it is "safe" (this is an imprecise term anyways, in this context). It's about making an educated guess about what will provide a better user experience when an assertion is triggered: halting the program, or ignoring the bug and continuing the program (even if admittedly the program will be in a buggy state). I've already mentioned several examples of situations where I think the later is preferable. Just to add another one, one that I recently came across while coding, was an assertion check that I put, which, if it where to fail, would only cause a redundant use of memory (but no NPEs or access violations or invalid state, etc.). -- Bruno Medeiros https://twitter.com/brunodomedeiros
Nov 19 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/19/2014 3:59 AM, Bruno Medeiros wrote:
 Just to add another one, one that I recently came across while coding, was an
 assertion check that I put, which, if it where to fail, would only cause a
 redundant use of memory (but no NPEs or access violations or invalid state,
etc.).
If you're comfortable with that, then you should be using enforce(), not assert().
Nov 19 2014
prev sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Saturday, 18 October 2014 at 05:10:20 UTC, Walter Bright wrote:
 I understand that some have to work with poorly written 
 libraries that incorrectly use assert. If that's the only issue 
 with those libraries, you're probably lucky :-) Short term, I 
 suggest editing the code of those libraries, and pressuring the 
 authors of them. Longer term, we need to establish a culture of 
 using assert/enforce correctly.
So you consider the library interface to be user input? What about calls that are used internally but also exposed as part of the library interface?
Oct 18 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/18/2014 9:01 AM, Sean Kelly wrote:
 So you consider the library interface to be user input?
The library designer has to make that decision, not the language.
 What about calls that are used internally but also exposed as part of the
library interface?
The library designer has to make a decision about who's job it is to validate the parameters to an interface, and thereby deciding what are input/environmental errors and what are programming bugs. Avoiding making this decision means the API is underspecified, and we all know how poorly that works. Consider: /****************************** * foo() does magical things. * Parameters: * x a value that must be greater than 0 and less than 8 * y a positive integer * Throws: * Exception if y is negative * Returns: * magic value */ int foo(int x, int y) in { assert(x > 0 && x < 8); } body { enforce(y >= 0, "silly rabbit, y should be positive"); ... return ...; }
Oct 18 2014
parent "rst256" <ussr.24 yandex.ru> writes:
On Saturday, 18 October 2014 at 17:55:04 UTC, Walter Bright wrote:
 On 10/18/2014 9:01 AM, Sean Kelly wrote:
 So you consider the library interface to be user input?
The library designer has to make that decision, not the language.
 What about calls that are used internally but also exposed as 
 part of the library interface?
The library designer has to make a decision about who's job it is to validate the parameters to an interface, and thereby deciding what are input/environmental errors and what are programming bugs. Avoiding making this decision means the API is underspecified, and we all know how poorly that works. Consider: /****************************** * foo() does magical things. * Parameters: * x a value that must be greater than 0 and less than 8 * y a positive integer * Throws: * Exception if y is negative * Returns: * magic value */ int foo(int x, int y) in { assert(x > 0 && x < 8); } body { enforce(y >= 0, "silly rabbit, y should be positive"); ... return ...; }
Contract Programming. Contract-rider list all required for this item api conditions. In that case is a list of exseption handler on client side. Epic sample of first post, may be like this: in {rider(except, "this class may trowing io exception, you must defined ...") }
Oct 20 2014
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/9/2014 9:33 AM, Johannes Pfau wrote:
 A point which hasn't been discussed yet:

 Errors and therefore assert can be used in nothrow functions. This is a
 pita for compilers because it now can't do certain optimizations. When
 porting GDC to ARM we started to see problems because of that (can't
 unwind from nothrow functions on ARM, program just aborts). And now we
 therefore have to worsen the codegen for nothrow functions because of
 this.

 I think Walter sometimes suggested that it would be valid for a
 compiler to not unwind Errors at all (in release mode), but simply kill
 the program and dump a error message. This would finally allow us to
 optimize nothrow functions.
Currently, Errors can be caught, but intervening finally blocks are not necessarily run. The reasons for this are: 1. better code generation 2. since after an Error the program is likely in an unknown state, the less code that is run after an Error, the better, because it may make things worse
Oct 11 2014
prev sibling next sibling parent reply Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 10/3/2014 10:00 AM, Joseph Rushton Wakeling via Digitalmars-d wrote:
 On 29/09/14 02:09, Walter Bright via Digitalmars-d wrote:
 If the program has entered an unknown state, its behavior from then on
 cannot be
 predictable. There's nothing I or D can do about that. D cannot
 officially
 endorse such a practice, though D being a systems programming language
 it will
 let you do what you want.

 I would not even consider such a practice for a program that is in
 charge of
 anything that could result in injury, death, property damage, security
 breaches,
 etc.
I think I should clarify that I'm not asking you to say "I endorse catching Errors". Your point about systems responsible for the safety of people or property is very well made, and I'm fully in agreement with you about this. What I'm asking you to consider is a use-case, one that I picked quite carefully. Without assuming anything about how the system is architected, if we have a telephone exchange, and an Error occurs in the handling of a single call, it seems to me fairly unarguable that it's essential to avoid this bringing down everyone else's call with it. That's not simply a matter of convenience -- it's a matter of safety, because those calls might include emergency calls, urgent business communications, or any number of other circumstances where dropping someone's call might have severe negative consequences. As I'm sure you realize, I also picked that particular use-case because it's one where there is a well-known technological solution -- Erlang -- which has as a key feature its ability to isolate different parts of the program, and to deal with errors by bringing down the local process where the error occurred, rather than the whole system. This is an approach which is seriously battle-tested in production. As I said, I'm not asking you to endorse catching Errors in threads, or other gross simplifications of Erlang's approach. What I'm interested in are your thoughts on how we might approach resolving the requirement for this kind of stability and localization of error-handling with the tools that D provides. I don't mind if you say to me "That's your problem" (which it certainly is:-), but I'd like it to be clear that it _is_ a problem, and one that it's important for D to address, given its strong standing in the development of super-high-connectivity server applications.
The part of Walter's point that is either deliberately overlooked or somewhat misunderstood here is the notion of a fault domain. In a typical unix or windows based environment, it's a process. A fault within the process yields the aborting of the process but not all processes. Erlang introduces within it's execution model a concept of a process within the higher level notion of the os level process. Within the erlang runtime it's individual processes run independently and can each fail independently. The erlang runtime guarantees a higher level of separation than a typical threaded java or c++ application. An error within the erlang runtime itself would justifiably cause the entire system to be halted. Just as within an airplane, to use Walter's favorite analogy, the seat entertainment system is physically and logically separated from flight control systems thus a fault within the former has no impact on the latter. So, where you have domains which must not impact each other, you reach for tools that allow complete separation such that faults within one CANNOT impact the other. You don't leave room for 'might not'. Later, Brad
Oct 03 2014
next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Friday, 3 October 2014 at 17:38:40 UTC, Brad Roberts via
Digitalmars-d wrote:
 The part of Walter's point that is either deliberately 
 overlooked or somewhat misunderstood here is the notion of a 
 fault domain.  In a typical unix or windows based environment, 
 it's a process.  A fault within the process yields the aborting 
 of the process but not all processes.  Erlang introduces within 
 it's execution model a concept of a process within the higher 
 level notion of the os level process.  Within the erlang 
 runtime it's individual processes run independently and can 
 each fail independently.  The erlang runtime guarantees a 
 higher level of separation than a typical threaded java or c++ 
 application.  An error within the erlang runtime itself would 
 justifiably cause the entire system to be halted.  Just as 
 within an airplane, to use Walter's favorite analogy, the seat 
 entertainment system is physically and logically separated from 
 flight control systems thus a fault within the former has no 
 impact on the latter.
Yep. And I think it's a fair assertion that the default fault domain in a D program is at the process level, since D is not inherently memory safe. But I don't think the language should necessarily make that assertion to the degree that no other definition is possible.
Oct 03 2014
parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 03/10/2014 19:20, Sean Kelly wrote:
 On Friday, 3 October 2014 at 17:38:40 UTC, Brad Roberts via
 Digitalmars-d wrote:
 The part of Walter's point that is either deliberately overlooked or
 somewhat misunderstood here is the notion of a fault domain.  In a
 typical unix or windows based environment, it's a process.  A fault
 within the process yields the aborting of the process but not all
 processes.  Erlang introduces within it's execution model a concept of
 a process within the higher level notion of the os level process.
 Within the erlang runtime it's individual processes run independently
 and can each fail independently.  The erlang runtime guarantees a
 higher level of separation than a typical threaded java or c++
 application.  An error within the erlang runtime itself would
 justifiably cause the entire system to be halted.  Just as within an
 airplane, to use Walter's favorite analogy, the seat entertainment
 system is physically and logically separated from flight control
 systems thus a fault within the former has no impact on the latter.
Yep. And I think it's a fair assertion that the default fault domain in a D program is at the process level, since D is not inherently memory safe. But I don't think the language should necessarily make that assertion to the degree that no other definition is possible.
Yes to Brad, and then yes to Sean. That nailed the point. To that I would only add that, when encountering a fault in a process, even an estimation (that is, not a 100% certainty) that such fault only affects a certain domain of the process, that would still be useful to certain kinds of systems and applications. I don't think memory-safety is at the core of the issue. Java is memory-safe, yet if you encounter a null pointer exception, you're still not sure if your whole application is now in an unusable state, or if the NPE was just confined to say, the operation the user just tried to do, or some other component of the application. There are no guarantees. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Oct 08 2014
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Wed, 08 Oct 2014 16:30:19 +0100
schrieb Bruno Medeiros <bruno.do.medeiros+dng gmail.com>:

 I don't think memory-safety is at the core of the issue. Java is 
 memory-safe, yet if you encounter a null pointer exception, you're still 
 not sure if your whole application is now in an unusable state, or if 
 the NPE was just confined to say, the operation the user just tried to 
 do, or some other component of the application. There are no guarantees.
I know a Karaoke software written in .NET, that doesn't check if a list view is empty when trying to select the first element. It results in a popup message about the Exception and you can continue from there. It is a logic error (aka assertion), but being tailored towards user interfaces, .NET doesn't kill the app. Heck, if Mono-D closed down on every NPE I wouldn't be using it any more. ;) -- Marco
Oct 10 2014
parent reply Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 10/10/2014 2:26 AM, Marco Leise via Digitalmars-d wrote:
 Am Wed, 08 Oct 2014 16:30:19 +0100
 schrieb Bruno Medeiros <bruno.do.medeiros+dng gmail.com>:

 I don't think memory-safety is at the core of the issue. Java is
 memory-safe, yet if you encounter a null pointer exception, you're still
 not sure if your whole application is now in an unusable state, or if
 the NPE was just confined to say, the operation the user just tried to
 do, or some other component of the application. There are no guarantees.
I know a Karaoke software written in .NET, that doesn't check if a list view is empty when trying to select the first element. It results in a popup message about the Exception and you can continue from there. It is a logic error (aka assertion), but being tailored towards user interfaces, .NET doesn't kill the app. Heck, if Mono-D closed down on every NPE I wouldn't be using it any more. ;)
How much you want to bet that if it did exit the app, that the bug would actually have been addressed rather than still remain broken?
Oct 10 2014
parent "Dicebot" <public dicebot.lv> writes:
On Friday, 10 October 2014 at 18:53:46 UTC, Brad Roberts via 
Digitalmars-d wrote:
 How much you want to bet that if it did exit the app, that the 
 bug would actually have been addressed rather than still remain 
 broken?
It more likely would cause maintainer to abandon the project after encountering raging users he can't help. Most of such Mono-D issues simply are not reproducable on Alex side and remain unfixed because of that. I can't remember a single case when he remained willingly ignorant of an unhandled exception.
Oct 10 2014
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/3/2014 10:38 AM, Brad Roberts via Digitalmars-d wrote:
 The part of Walter's point that is either deliberately overlooked or somewhat
 misunderstood here is the notion of a fault domain.  In a typical unix or
 windows based environment, it's a process.  A fault within the process yields
 the aborting of the process but not all processes.  Erlang introduces within
 it's execution model a concept of a process within the higher level notion of
 the os level process.  Within the erlang runtime it's individual processes run
 independently and can each fail independently.  The erlang runtime guarantees a
 higher level of separation than a typical threaded java or c++ application.  An
 error within the erlang runtime itself would justifiably cause the entire
system
 to be halted.  Just as within an airplane, to use Walter's favorite analogy,
the
 seat entertainment system is physically and logically separated from flight
 control systems thus a fault within the former has no impact on the latter.

 So, where you have domains which must not impact each other, you reach for
tools
 that allow complete separation such that faults within one CANNOT impact the
 other.  You don't leave room for 'might not'.
Thanks, Brad, that is a correct formulation.
Oct 04 2014
prev sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Fri, 03 Oct 2014 10:38:21 -0700
schrieb Brad Roberts via Digitalmars-d
<digitalmars-d puremagic.com>:

 Just as within an airplane, to use Walter's 
 favorite analogy, the seat entertainment system is physically and 
 logically separated from flight control systems thus a fault within the 
 former has no impact on the latter.
And just like the Erlang runtime, the electrical components could be faulty or operating beyond their limits and cause the aircraft to shutdown. http://www.dailymail.co.uk/news/article-2519705/Blaze-scare-BA-Jumbo-Serious-electrical-sparked-planes-flight-entertainment-system.html http://en.wikipedia.org/wiki/Swissair_Flight_111 http://www.iasa.com.au/folders/sr111/747IFE-fire.htm -- Marco
Oct 05 2014
prev sibling next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 28 Sep 2014 17:09:57 -0700
Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 If the program has entered an unknown state, its behavior from then
 on cannot be predictable.
and D compiler itself contradicts this principle. why it tries to "recover" from parsing/compiling errors? it should stop on the first encountered error and not trying to "recover" itself from unknown state. hate this. and it's inconsistent with your words.
Oct 03 2014
parent reply "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Saturday, 4 October 2014 at 01:52:41 UTC, ketmar via 
Digitalmars-d wrote:
 On Sun, 28 Sep 2014 17:09:57 -0700
 Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:

 If the program has entered an unknown state, its behavior from 
 then
 on cannot be predictable.
and D compiler itself contradicts this principle. why it tries to "recover" from parsing/compiling errors? it should stop on the first encountered error and not trying to "recover" itself from unknown state. hate this. and it's inconsistent with your words.
I think that there's a big confusion about terms: there's nothing unknown in the parser state when it reach an error in the grammar. --- /Paolo
Oct 04 2014
parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 04 Oct 2014 08:26:53 +0000
Paolo Invernizzi via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I think that there's a big confusion about terms: there's nothing=20
 unknown in the parser state when it reach an error in the grammar.
just like nothing unknown in program state when it gets garbage input. yet there is no sense in processing garbage input to produce garbage output. the best thing the program can do is to complain and shut down (if this is not a special recovery program which tries to get all possible bits of info).
Oct 04 2014
prev sibling next sibling parent Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 10/3/2014 6:52 PM, ketmar via Digitalmars-d wrote:
 On Sun, 28 Sep 2014 17:09:57 -0700
 Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 If the program has entered an unknown state, its behavior from then
 on cannot be predictable.
and D compiler itself contradicts this principle. why it tries to "recover" from parsing/compiling errors? it should stop on the first encountered error and not trying to "recover" itself from unknown state. hate this. and it's inconsistent with your words.
Where's the contradiction? The compilers state hasn't been corrupted just because it encounters errors in the text file. In fact, it's explicitly built to detect and handle them. There's not even a contradiction in making assumptions about what that input could have been and attempting to continue based on those assumptions. At no time in there is the compilers internal state corrupted. And in direct affirmation of the principle, the compiler has numerous asserts scattered around that _do_ abort compilation should an unexpected and invalid state be detected.
Oct 03 2014
prev sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Fri, 03 Oct 2014 19:25:53 -0700
Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 Where's the contradiction?  The compilers state hasn't been corrupted=20
 just because it encounters errors in the text file.
but compiler is in unknown state. it can't do telepathy, and it's tries are annoying. there is no reason to guess what code programmer meant to write, it's just a source of mystic/nonsensical error messages ("garbage", in other words). the original source of "trying to recover and continue analysis" was slow compilation times, it was really painful to restart compiler after each error. but D compilation times are good enough to stop this "guess-and-miss" nonsence. and we have good IDEs that can analyse code on background and highligh errors, so there are virtually no reasons for telepathy left. yet many compilers (including D) still tries to do telepathy (and fails). 'cmon, it's best to improve compiling times, not attempting to guess. PL/1 fails at this, and all other compilers since then fails too. it's strange to me that Walter telling us that program should stop once it enters unknown state, but forcing D compiler to make uneducated guesses when D compiler enters unknown state. something is very wrong with one of this things.
Oct 03 2014
next sibling parent reply "eles" <eles215 gzk.dot> writes:
On Saturday, 4 October 2014 at 03:16:08 UTC, ketmar via 
Digitalmars-d wrote:
 On Fri, 03 Oct 2014 19:25:53 -0700
 Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:

 Where's the contradiction?  The compilers state hasn't been 
 corrupted just because it encounters errors in the text file.
but compiler is in unknown state.
It's not. It just detected that another system would enter in an unknown state if built. The compiler is like the engineer that examines the design of a project and discovers an error on it, so it refuses to build the product. Is the engineer in an unknown state? No, it is, just like the compiler is, outside its/his/her normal execution flow, wich is "take the design and build the product". It is a known state of the compiler/engineer, namely the error processing path. While on the error processing path you allow yourself to be slower and take time to guess. This is important. Just as for the engineer, this will educate the designers and will not come again so often with the same design mistake. Then, working/building fast is measured on the normal execution path, not on the error recovery path since the latters is assumed to occur quite rarely (in any case, its seen like an exceptional issue). Of course, the engineer has to handle the work conflict with the designer in a polite form. For the compiler this means printing a nice error message to the user. Guessing might not be good, but it is nice effort to do. Do you really miss the super-cryptic C (let's not even talk about C++) error messages that you sometimes receive?
Oct 03 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 04 Oct 2014 04:10:49 +0000
eles via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 Where's the contradiction?  The compilers state hasn't been=20
 corrupted just because it encounters errors in the text file.
but compiler is in unknown state.
It's not. It just detected that another system would enter in an=20 unknown state if built. The compiler is like the engineer that=20 examines the design of a project and discovers an error on it, so=20 it refuses to build the product. Is the engineer in an unknown=20 state?
sorry. i meant that compiler WILL be in unknown state if it will continute to processing invalid source. that's why it should stop right after the first error.
 Guessing might not be good, but it is nice effort to do. Do you=20
 really miss the super-cryptic C (let's not even talk about C++)=20
 error messages that you sometimes receive?
yes. DMD attempts to 'guess' what identifier i mistyped drives me crazy. just shut up and stop after "unknown identifier", you robot, don't try to show me your artificial idiocity!
Oct 03 2014
next sibling parent reply "eles" <eles215 gzk.dot> writes:
On Saturday, 4 October 2014 at 04:26:45 UTC, ketmar via 
Digitalmars-d wrote:
 On Sat, 04 Oct 2014 04:10:49 +0000
 eles via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 Where's the contradiction?  The compilers state hasn't been 
 corrupted just because it encounters errors in the text 
 file.
but compiler is in unknown state.
It's not. It just detected that another system would enter in an unknown state if built. The compiler is like the engineer that examines the design of a project and discovers an error on it, so it refuses to build the product. Is the engineer in an unknown state?
sorry. i meant that compiler WILL be in unknown state if it will continute to processing invalid source. that's why it should stop right after the first error.
No. It might produce an invalid product. Just like a real engineer could produce a flawed product on the basis of wrong designs. Yes, you might reach a state where you are no longer be able to continue because physically impossible. "Build a round square". Both the engineer and the compiler will bark when they see this.
 Guessing might not be good, but it is nice effort to do. Do 
 you really miss the super-cryptic C (let's not even talk about 
 C++) error messages that you sometimes receive?
yes. DMD attempts to 'guess' what identifier i mistyped drives me crazy. just shut up and stop after "unknown identifier", you robot, don't try to show me your artificial idiocity!
Could we add a flag to the compiler for that? -cassandramode=[yes/no/whatever] Joke :) Anyway, the Cassandra name for the compiler is just perfect: it might be right, but you won't believe!
Oct 03 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 04 Oct 2014 04:36:48 +0000
eles via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 sorry. i meant that compiler WILL be in unknown state if it will
 continute to processing invalid source. that's why it should=20
 stop right
 after the first error.
No. It might produce an invalid product. Just like a real=20 engineer could produce a flawed product on the basis of wrong=20 designs.
i can't see any sane reason to process garbage data, 'cause result is known to be garbage too.
 Could we add a flag to the compiler for that?
=20
 -cassandramode=3D[yes/no/whatever]
=20
 Joke :)
but this will be fine. let beginners get suggestions about importing 'std.stdio' on unknown 'writeln', but don't spit that on me. i tend to read compiler messages, and additional noise is annoying.
Oct 03 2014
parent reply "eles" <eles215 gzk.dot> writes:
On Saturday, 4 October 2014 at 05:02:04 UTC, ketmar via 
Digitalmars-d wrote:
 On Sat, 04 Oct 2014 04:36:48 +0000
 eles via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 sorry. i meant that compiler WILL be in unknown state if it 
 will
 continute to processing invalid source. that's why it should 
 stop right
 after the first error.
No. It might produce an invalid product. Just like a real engineer could produce a flawed product on the basis of wrong designs.
i can't see any sane reason to process garbage data, 'cause result is known to be garbage too.
The same reason why a schoolteacher will process garbage exam sheets.
Oct 03 2014
parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 04 Oct 2014 05:19:32 +0000
eles via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 i can't see any sane reason to process garbage data, 'cause=20
 result is
 known to be garbage too.
The same reason why a schoolteacher will process garbage exam=20 sheets.
it can be true if DMD will has advanced AI someday -- to clearly understand what programmer wants to write. but then there will be no reason to emit error messages. and to writing code manually too.
Oct 03 2014
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/3/14, 9:26 PM, ketmar via Digitalmars-d wrote:
 yes. DMD attempts to 'guess' what identifier i mistyped drives me
 crazy. just shut up and stop after "unknown identifier", you robot,
 don't try to show me your artificial idiocity!
awesome feature -- Andrei
Oct 04 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/4/2014 2:45 PM, Andrei Alexandrescu wrote:
 On 10/3/14, 9:26 PM, ketmar via Digitalmars-d wrote:
 yes. DMD attempts to 'guess' what identifier i mistyped drives me
 crazy. just shut up and stop after "unknown identifier", you robot,
 don't try to show me your artificial idiocity!
awesome feature -- Andrei
I agree, I like it very much.
Oct 04 2014
parent reply "Cliff" <cliff.s.hudson gmail.com> writes:
On Sunday, 5 October 2014 at 03:34:31 UTC, Walter Bright wrote:
 On 10/4/2014 2:45 PM, Andrei Alexandrescu wrote:
 On 10/3/14, 9:26 PM, ketmar via Digitalmars-d wrote:
 yes. DMD attempts to 'guess' what identifier i mistyped 
 drives me
 crazy. just shut up and stop after "unknown identifier", you 
 robot,
 don't try to show me your artificial idiocity!
awesome feature -- Andrei
I agree, I like it very much.
This is a great feature where we lack a really solid IDE experience (which would have intellisense and auto-completion that could be accurate and prevent such errors from occurring in the first place.) Otherwise it would probably be redundant.
Oct 04 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 05 Oct 2014 03:47:31 +0000
Cliff via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 This is a great feature where we lack a really solid IDE=20
 experience (which would have intellisense and auto-completion=20
 that could be accurate and prevent such errors from occurring in=20
 the first place.)  Otherwise it would probably be redundant.
i'm not using IDEs for more than a decade (heh, i'm using mcedit to write code). yet this feature drives me mad: it trashes my terminal with useless garbage output. it was *never* in help, there were no moment when i looked at suggested identifier and thinked: "aha, THAT is the bug!" but virtually each time i see usggestion i'm thinking: "oh, well, i know. 'cmon, why don't you just shut up?!" it's like colorizing the output, yet colorizing can be turned off, and suggestions can't.
Oct 04 2014
parent reply "Cliff" <cliff.s.hudson gmail.com> writes:
On Sunday, 5 October 2014 at 05:46:56 UTC, ketmar via 
Digitalmars-d wrote:
 On Sun, 05 Oct 2014 03:47:31 +0000
 Cliff via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 This is a great feature where we lack a really solid IDE 
 experience (which would have intellisense and auto-completion 
 that could be accurate and prevent such errors from occurring 
 in the first place.)  Otherwise it would probably be redundant.
i'm not using IDEs for more than a decade (heh, i'm using mcedit to write code). yet this feature drives me mad: it trashes my terminal with useless garbage output. it was *never* in help, there were no moment when i looked at suggested identifier and thinked: "aha, THAT is the bug!" but virtually each time i see usggestion i'm thinking: "oh, well, i know. 'cmon, why don't you just shut up?!" it's like colorizing the output, yet colorizing can be turned off, and suggestions can't.
That you even make the bug at all which triggers the error is an indication the developer workflow you use is fundamentally flawed. This is something which should be caught much earlier - when you are at the point the typo was made - not after you have committed a change to disk and presented it to the compiler, where your train of thought may be significantly different. I'd much rather energy be directed at the prevention of mistakes, not the suppression of help in fixing them - if I had to choose. But I wouldn't object to having a switch to turn off the help if it bothers you that much. Seems like a very small thing to add.
Oct 04 2014
parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 05 Oct 2014 05:55:37 +0000
Cliff via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 That you even make the bug at all which triggers the error is an=20
 indication the developer workflow you use is fundamentally=20
 flawed.
what we are talking about here? sorry, i'm lost.
  This is something which should be caught much earlier -=20
 when you are at the point the typo was made - not after you have=20
 committed a change to disk and presented it to the compiler,=20
 where your train of thought may be significantly different.
i'm writing code in big pieces, and fixing typos, etc. then works as a "background thread" in my brain, while "main thread" is still thinking about the overall picture. no IDE helps me to improve this, and i don't need all their tools and their bloat. and i'm not making alot of typos. ;-)
 But I wouldn't object to having a switch to turn off the help if=20
 it bothers you that much.  Seems like a very small thing to add.
this is a spin-off ;-) of the actual discussion. what i'm talking about originally is that compiler should stop after first error it encountered, not trying to parse/analyze code further. and there i was talking about IDEs which helps to fix alot of typos and other grammar bugs even before compilation starts. and what i'm talking about is that trying to make sense from garbage input is not a sign of robust software, if that software is not a special case software which was designed to parse garbage. and that's why i'm against any "warnings" in compilers: all warnings should be errors.
Oct 04 2014
prev sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 04 Oct 2014 14:45:42 -0700
Andrei Alexandrescu via Digitalmars-d <digitalmars-d puremagic.com>
wrote:

 On 10/3/14, 9:26 PM, ketmar via Digitalmars-d wrote:
 yes. DMD attempts to 'guess' what identifier i mistyped drives me
 crazy. just shut up and stop after "unknown identifier", you robot,
 don't try to show me your artificial idiocity!
awesome feature -- Andrei
the only bad thing with this future is that it is useless. a simple typo is not worth of additional noise, and in complex cases feature is not working (i.e. it can't guess method name from another module, especially if it's not imported). all in all this adds noise for nothing. ah, and for some fake coolness.
Oct 04 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/3/2014 8:15 PM, ketmar via Digitalmars-d wrote:
 there is no reason to guess what code programmer meant to write,
It doesn't do that anymore (at least mostly not). The compiler only continues to issue error messages for code that is not dependent on the code that caused an error message.
Oct 04 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 04 Oct 2014 01:06:14 -0700
Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 It doesn't do that anymore (at least mostly not). The compiler only
 continues to issue error messages for code that is not dependent on
 the code that caused an error message.
but first it trues to guess what to skip to "get in sync" again. that's exactly the guessing i'm against. got error? stop parsing/analyzing and die.
Oct 04 2014
parent reply "eles" <eles215 gzk.dot> writes:
On Saturday, 4 October 2014 at 08:15:57 UTC, ketmar via 
Digitalmars-d wrote:
 On Sat, 04 Oct 2014 01:06:14 -0700
 Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:

 It doesn't do that anymore (at least mostly not). The compiler 
 only
 continues to issue error messages for code that is not 
 dependent on
 the code that caused an error message.
but first it trues to guess what to skip to "get in sync" again. that's exactly the guessing i'm against. got error? stop parsing/analyzing and die.
The compiler or the parser is not guessing "what I should do", but "what the user wanted to do". They are very different things. The parser itself is as sound as ever. It is on its user input error processing path but, hey, it was designed to do that. It knows what it is doing.
Oct 04 2014
parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 04 Oct 2014 09:14:03 +0000
eles via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 It knows what it is doing.
yes. processing garbage to generate more garbage.
Oct 04 2014
prev sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 09/28/2014 04:13 PM, Walter Bright wrote:
 A program bug is, by definition, unknown and unanticipated. The idea
 that one can "recover" from it is fundamentally wrong.
In many cases you can. Don't forget, enterprise code aside, we're not in the days of spaghetti code where everything's directly intermingled: Things are encapsulated. If I call on a component, and that component fails, chances are the problems exist purely within that component. So yes, recovery is most definitely possible. Not *always*, but often. I imagine you're about to jump on that "chances are" and "not always" part...and you're right: For safety-critical software, relying on that "*chances are*" the bug exists purely within that component is, of course, completely inappropriate. I won't dispute that. But a LOT of software is *NOT* safety-critical. In those cases, going through the trouble of process-separated components or complete shutdowns on every minor compartmentalized hiccup, just on the off-chance that it might be symptomatic of something bigger (it usually isn't) usually amounts to swatting a fly with a bazooka. If tetris, or "semi-interactive movie of the year", or a music library manager, or an online poll, or youtube comments suddenly goes haywire, it's really not a big deal. And process separation and permissions still limit the collateral damage anyway. It's unprofessional, sure, all bugs and crashes are, but it's certainly no less professional than a program that shuts down whenever the digital wind shifts. Also, on a somewhat separate note: As other have indicted, suppose you *do* strictly limit Exceptions to bad user inputs and other expected non-logic-errors. What happens when you forget to handle one of those Exceptions? That's a logic error that manifests as an uncaught Exception. Code that handles and recovers from bad user input and other such "Exception" purposes is every bit as subject to logic errors as any other code. Therefore it is *impossible* to reliably ensure that Exception *only* indicates non-logic-errors such as bad user input. So the whole notion of strict separation falls apart right there. Exceptions are *going to* occasionally indicate logic errors, period. Therefore, they must carry enough information to help the developer diagnose and fix. Is showing that diagnostic information to the user upon a crash unprofessional? Sure. But *all* crashes are unprofessional, so there's really no avoiding that. The *least* unprofessional crashes are the ones that 1. Don't get triggered more often than actually necessary (considering the given program domain.) and 2. Contain enough information to actually get fixed.
Oct 04 2014
prev sibling next sibling parent reply Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 28/09/14 19:33, Walter Bright via Digitalmars-d wrote:
 On 9/28/2014 9:23 AM, Sean Kelly wrote:
 Also, I think the idea that a program is created and shipped to an end user is
 overly simplistic.  In the server/cloud programming world, when an error
occurs,
 the client who submitted the request will get a response appropriate for them
 and the system will also generate log information intended for people working
on
 the system.  So things like stack traces and assertion failure information is
 useful even for production software.  Same with any critical system, as I'm
sure
 you're aware.  The systems are designed to handle failures in specific ways,
but
 they also have to leave a breadcrumb trail so the underlying problem can be
 diagnosed and fixed.  Internal testing is never perfect, and achieving a high
 coverage percentage is nearly impossible if the system wasn't designed from the
 ground up to be testable in such a way (mock frameworks and such).
Then use assert(). That's just what it's for.
I don't follow this point. How can this approach work with programs that are built with the -release switch? Moreover, Sean's points here are absolutely on the money -- there are cases where the "users" of a program may indeed want to see traces even for anticipated errors. And even if you design a nice structure of throwing and catching exceptions so that the simple error message _always_ gives good enough context to understand what went wrong, you still have the other issue that Sean raised -- of an exception accidentally escaping its intended scope, because you forgot to handle it -- when a trace may be extremely useful. Put it another way -- I think you make a good case that stack traces for exceptions should be turned off by default (possibly just in -release mode?), but if that happens I think there's also a good case for a build flag that ensures stack traces _are_ shown for Exceptions as well as Errors.
Sep 28 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 4:18 PM, Joseph Rushton Wakeling via Digitalmars-d wrote:
 I don't follow this point.  How can this approach work with programs that are
 built with the -release switch?
All -release does is not generate code for assert()s. To leave the asserts in, do not use -release. If you still want the asserts to be in even with -release, if (condition) assert(0);
 Moreover, Sean's points here are absolutely on the money -- there are cases
 where the "users" of a program may indeed want to see traces even for
 anticipated errors.
 And even if you design a nice structure of throwing and
 catching exceptions so that the simple error message _always_ gives good enough
 context to understand what went wrong, you still have the other issue that Sean
 raised -- of an exception accidentally escaping its intended scope, because you
 forgot to handle it -- when a trace may be extremely useful.

 Put it another way -- I think you make a good case that stack traces for
 exceptions should be turned off by default (possibly just in -release mode?),
 but if that happens I think there's also a good case for a build flag that
 ensures stack traces _are_ shown for Exceptions as well as Errors.
The -g switch should take care of that. It's what I use when I need a stack trace, as there are many ways a program can fail (not just Errors).
Sep 28 2014
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/29/2014 02:47 AM, Walter Bright wrote:
 On 9/28/2014 4:18 PM, Joseph Rushton Wakeling via Digitalmars-d wrote:
 I don't follow this point.  How can this approach work with programs
 that are
 built with the -release switch?
All -release does is not generate code for assert()s. ...
(Euphemism for undefined behaviour.)
Sep 28 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/29/2014 06:06 AM, Timon Gehr wrote:
 On 09/29/2014 02:47 AM, Walter Bright wrote:
 On 9/28/2014 4:18 PM, Joseph Rushton Wakeling via Digitalmars-d wrote:
 I don't follow this point.  How can this approach work with programs
 that are
 built with the -release switch?
All -release does is not generate code for assert()s. ...
(Euphemism for undefined behaviour.)
Also, -release additionally removes contracts, in particular invariant calls, and enables version(assert).
Sep 28 2014
prev sibling next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Sun, 28 Sep 2014 17:47:56 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 9/28/2014 4:18 PM, Joseph Rushton Wakeling via Digitalmars-d wrote:
 I don't follow this point.  How can this approach work with
 programs that are built with the -release switch?
All -release does is not generate code for assert()s. To leave the asserts in, do not use -release. If you still want the asserts to be in even with -release, if (condition) assert(0);
Right now, but some time ago there was a huge debate whether it should be valid for the compiler to optimize based on asserts. I wonder if these 'use asserts for stack traces' and 'an assert is always supposed to pass, so it's valid to assume the condition holds (in release)' notions can go together. I guess it might at least lead to programs that are unusable when compiled with -release.
Sep 29 2014
next sibling parent "Daniel N" <ufo orbiting.us> writes:
On Monday, 29 September 2014 at 08:18:29 UTC, Johannes Pfau wrote:
 Right now, but some time ago there was a huge debate whether it 
 should
 be valid for the compiler to optimize based on asserts.

 I wonder if these 'use asserts for stack traces' and 'an assert 
 is
 always supposed to pass, so it's valid to assume the condition 
 holds
 (in release)' notions can go together. I guess it might at 
 least lead
 to programs that are unusable when compiled with -release.
For this reason I think it makes more sense to use abort() if you plan to use "-release".
Sep 29 2014
prev sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 29 Sep 2014 10:18:27 +0200
Johannes Pfau via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I wonder if these 'use asserts for stack traces' and 'an assert is
 always supposed to pass, so it's valid to assume the condition holds
 (in release)' notions can go together. I guess it might at least lead
 to programs that are unusable when compiled with -release.
compiler never optimizes away `assert(0)`, AFAIR. `assert(0)` is a special thing, it means `abort()`.
Sep 29 2014
prev sibling parent reply "Joseph Rushton Wakeling" <joseph.wakeling webdrake.net> writes:
On Monday, 29 September 2014 at 00:47:58 UTC, Walter Bright wrote:
 On 9/28/2014 4:18 PM, Joseph Rushton Wakeling via Digitalmars-d 
 wrote:
 I don't follow this point.  How can this approach work with 
 programs that are
 built with the -release switch?
All -release does is not generate code for assert()s. To leave the asserts in, do not use -release. If you still want the asserts to be in even with -release, if (condition) assert(0);
The reason I queried your approach here is because I feel you're conflating two things: * the _definition_ of an Exception vs. an Error, on which we 100% agree: the former as an anticipated possibility which a program is committed to try and handle, the latter a failure which is fundamentally wrong and should not happen under any conditions. * the way in which a program should report these different kinds of error. You seem to be advocating that, by definition, Exceptions and Errors should be reported differently (one without, and one with, a trace). I don't at all object to that as a sensible default, but I think that the ultimate decision on how Exceptions and Errors report themselves should be in the hands of the program developer, depending on the use-case of the application.
Sep 29 2014
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Monday, 29 September 2014 at 15:39:08 UTC, Joseph Rushton 
Wakeling wrote:
 with, a trace).  I don't at all object to that as a sensible 
 default, but I think that the ultimate decision on how 
 Exceptions and Errors report themselves should be in the hands 
 of the program developer, depending on the use-case of the 
 application.
That's a very sensible argument. From a pragmatic point of view I am usually not interested in the whole stack trace. I am primarily interested in where it was thrown, where it was turned into an error and where it was logged + "function call input" at that point. Maybe a trade off can be found that is less costly than building a full stack trace.
Sep 29 2014
prev sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 29 Sep 2014 01:18:02 +0200
Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com>
wrote:

 Then use assert(). That's just what it's for.
=20 I don't follow this point. How can this approach work with programs that are built with the -release switch?
don't use "-release" switch. the whole concept of "release version" is broken by design. ship what you debugged, not what you think you debugged.
Sep 28 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 9:16 AM, Sean Kelly wrote:
 On Sunday, 28 September 2014 at 00:40:26 UTC, Walter Bright wrote:
 Whoa, Camel! You're again thinking of Exceptions as a debugging tool.
They can be.
Of course they can be. But it's inappropriate to use them that way, and we should not be eschewing such in the library.
 What if an API you're using throws an exception you didn't expect,
 and therefore don't handle?
Then the app user sees the error message. This is one of the cool things about D - I can write small apps with NO error handling logic in it, and I still get appropriate and friendly messages when things go wrong like missing files. That is, until recently, when I get a bunch of debug stack traces and internal file/line messages, which are of no use at all to an app user and look awful.
 This might be considered a logic error if the
 exception is recoverable and you don't intend the program to abort from that
 operation.
Adding file/line to all exceptions implies that they are all bugs, and encourages them to be thought of as bugs and debugging tools, when they are NOT. Exceptions are for: 1. enabling recovery from input/environmental errors 2. reporting input/environmental errors to the app user 3. making input/environmental errors not ignorable by default They are not for detecting logic errors. Assert is designed for that.
Sep 28 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sun, Sep 28, 2014 at 10:32:14AM -0700, Walter Bright via Digitalmars-d wrote:
 On 9/28/2014 9:16 AM, Sean Kelly wrote:
[...]
What if an API you're using throws an exception you didn't expect,
and therefore don't handle?
Then the app user sees the error message. This is one of the cool things about D - I can write small apps with NO error handling logic in it, and I still get appropriate and friendly messages when things go wrong like missing files. That is, until recently, when I get a bunch of debug stack traces and internal file/line messages, which are of no use at all to an app user and look awful.
It looks even more awful when the person who wrote the library code is Russian, and the user speaks English, and when an uncaught exception terminates the program, you get a completely incomprehensible message in a language you don't know. Not much different from a line number and filename that has no meaning for a user. That's why I said, an uncaught exception is a BUG. The only place where user-readable messages can be output is in a catch block where you actually have the chance to localize the error string. But if no catch block catches it, then by definition it's a bug, and you might as while print some useful info with it that your users can send back to you, rather than unhelpful bug reports of the form "the program crashed with error message 'internal error'". Good luck finding where in the code that is. (And no, grepping does not work -- the string 'internal error' could have come from a system call or C library error code translated by a generic code-to-message function, which could've been called from *anywhere*.)
This might be considered a logic error if the exception is
recoverable and you don't intend the program to abort from that
operation.
Adding file/line to all exceptions implies that they are all bugs, and encourages them to be thought of as bugs and debugging tools, when they are NOT. Exceptions are for: 1. enabling recovery from input/environmental errors 2. reporting input/environmental errors to the app user 3. making input/environmental errors not ignorable by default They are not for detecting logic errors. Assert is designed for that.
I do not condone adding file/line to exception *messages*. Catch blocks can print / translate those messages, which can be made user-friendly, but if the program failed to catch an exception, you're already screwed anyway so why not provide more info rather than less? Unless, of course, you're suggesting that we put this around every main() function: void main() { try { ... } catch(Exception e) { assert(0, "Unhandled exception: I screwed up"); } } T -- I think Debian's doing something wrong, `apt-get install pesticide', doesn't seem to remove the bugs on my system! -- Mike Dresser
Sep 28 2014
next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Sunday, 28 September 2014 at 20:58:20 UTC, H. S. Teoh via 
Digitalmars-d wrote:
 That's why I said, an uncaught exception is a BUG. The only  
 place where
 user-readable messages can be output is in a catch block where  
 you
 actually have the chance to localize the error string. But if  
 no catch
 block catches it, then by definition it's a bug, and you might  
 as while
 print some useful info with it that your users can send back to 
  you,
 rather than unhelpful bug reports of the form "the program 
 crashed with
 error message 'internal error'".
Pretty much every system should generate a localized error message for the user and a detailed log of the problem for the programmer. Ideally, the user message will indicate how to provide the detailed information to the developer so the problem can be fixed. The one case where uncaught exceptions aren't really a bug is with programs that aren't being used outside the group that developed them. In these cases, the default behavior is pretty much exactly what's desired--a message, file/line info, and a stack trace. Which is why it's there. The vast bulk of today's shipping code doesn't run from the command line anyway, so the default exception handler should be practically irrelevant.
Sep 28 2014
prev sibling next sibling parent "Cliff" <cliff.s.hudson gmail.com> writes:
On Sunday, 28 September 2014 at 20:58:20 UTC, H. S. Teoh via 
Digitalmars-d wrote:
 I do not condone adding file/line to exception *messages*. 
 Catch blocks
 can print / translate those messages, which can be made 
 user-friendly,
 but if the program failed to catch an exception, you're already 
 screwed
 anyway so why not provide more info rather than less?

 Unless, of course, you're suggesting that we put this around 
 every
 main() function:

 	void main() {
 		try {
 			...
 		} catch(Exception e) {
 			assert(0, "Unhandled exception: I screwed up");
 		}
 	}
In our production C# code, we had a few practices which might be applicable here: 1. main() definitely had a top-level try/catch handler to produce useful output messages. Because throwing an uncaught exception out to the user *is* a bug, we naturally want to not just toss out a stack trace but information on what to do with it should a user encounter it. Even better if there is additional runtime information which can be provided for a bug report. 2. We also registered a top-level unhandled exception handler on the AppDomain (equivalent to a process in .NET, except that multiple AppDomains may exist within a single OS process), which allows the catching to exceptions which would otherwise escape background threads. Depending on the nature of the application, these could be logged to some repository to which the user could be directed. It's hard to strictly automate this because exactly what you can do with an exception which escapes a thread will be application dependent. In our case, these exceptions were considered bugs, were considered to be unrecoverable and resulted in a program abort with a user message indicating where to find the relevant log outputs and how to contact us. 3. For some cases, throwing an exception would also trigger an application dump suitable for post-mortem debugging from the point the exception was about to be thrown. This functionality is, of course, OS-specific, but helped us on more than a few occasions by eliminating the need to try to pre-determine which information was important and which was not so the exception could be usefully populated. I'm not a fan of eliminating the stack from exceptions. While exceptions should not be used to catch logic errors, an uncaught exception is itself a logic error (that is, one has omitted some required conditions in their code) and thus the context of the error needs to be made available somehow.
Sep 28 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 1:56 PM, H. S. Teoh via Digitalmars-d wrote:
 It looks even more awful when the person who wrote the library code is
 Russian, and the user speaks English, and when an uncaught exception
 terminates the program, you get a completely incomprehensible message in
 a language you don't know. Not much different from a line number and
 filename that has no meaning for a user.
I cannot buy into the logic that since Russian error messages are incomprehensible to me, that therefore incomprehensible messages are ok.
 That's why I said, an uncaught exception is a BUG.
It's a valid opinion, but is not the way D is designed to work.
 The only place where
 user-readable messages can be output is in a catch block where you
 actually have the chance to localize the error string. But if no catch
 block catches it, then by definition it's a bug, and you might as while
 print some useful info with it that your users can send back to you,
 rather than unhelpful bug reports of the form "the program crashed with
 error message 'internal error'".
If anyone is writing code that throws an Exception with "internal error", then they are MISUSING exceptions to throw on logic bugs. I've been arguing this all along.
 if the program failed to catch an exception, you're already screwed
 anyway
This is simply not true. One can write utilities with no caught exceptions at all, and yet have the program emit user friendly messages about "disk full" and stuff like that.
 so why not provide more info rather than less?
Because having an internal stack dump presented to the app user for when he, say, puts in invalid command line arguments, is quite inappropriate.
 Unless, of course, you're suggesting that we put this around every
 main() function:

 	void main() {
 		try {
 			...
 		} catch(Exception e) {
 			assert(0, "Unhandled exception: I screwed up");
 		}
 	}
I'm not suggesting that Exceptions are to be thrown on programmer screwups - I suggest the OPPOSITE.
Sep 28 2014
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/29/2014 12:59 AM, Walter Bright wrote:
 ...

 Unless, of course, you're suggesting that we put this around every
 main() function:

     void main() {
         try {
             ...
         } catch(Exception e) {
             assert(0, "Unhandled exception: I screwed up");
         }
     }
I'm not suggesting that Exceptions are to be thrown on programmer screwups - I suggest the OPPOSITE.
He does not suggest that Exceptions are to be thrown on programmer screw-ups, but rather that the thrown exception itself is the screw-up, with a possibly complex cause. It is not: if(screwedUp()) throw Exception(""); It is rather: void foo(int x){ if(!test(x)) throw Exception(""); // this may be an expected code path for some callers } void bar(){ // ... int y=screwUp(); foo(y); // yet it is unexpected here }
Sep 28 2014
prev sibling next sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Sunday, 28 September 2014 at 22:59:46 UTC, Walter Bright wrote:
 If anyone is writing code that throws an Exception with 
 "internal error", then they are MISUSING exceptions to throw on 
 logic bugs. I've been arguing this all along.
Nothing wrong with it. Quite common and useful for a non-critical web service to log the exception, then re-throw something like "internal error", catch the internal error at the root and returning the appropriate 5xx HTTP response, then keep going. You are arguing as if it is impossible to know whether the logic error is local to the handler, or not, with a reasonable probability. "Division by zero" is usually not a big deal, but it is a logic error. No need to shut down the service.
 I'm not suggesting that Exceptions are to be thrown on 
 programmer screwups - I suggest the OPPOSITE.
It is impossible to verify what the source is. It might be a bug in a boolean expression leading to a throw when the system is ok. assert()s should also not be left in production code. They are not for catching runtime errors, but for testing at the expense of performance. Uncaught exceptions should be re-thrown higher up in the call chain to a different error level based on the possible impact on the system. Getting an unexpected mismatch exception in a form-validator is not a big deal. Getting out-of-bounds error in main storage is a big deal. Whether it is a big deal can only be decided at the higher level. It is no doubt useful to be able to obtain a stack trace so that you can log it when an exception turns out to fall into the "big deal" category and therefore should be re-thrown as a critical failture. The deciding factor should be performance.
Sep 28 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 9:31 PM, "Ola Fosheim Grøstad" 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 Nothing wrong with it. Quite common and useful for a non-critical web service
to
 log the exception, then re-throw something like "internal error",  catch the
 internal error at the root and returning the appropriate 5xx HTTP response,
then
 keep going.
Lots of bad practices are commonplace.
 You are arguing as if it is impossible to know whether the logic error is local
 to the handler, or not, with a reasonable probability.
You're claiming to know that a program in an unknown and unanticipated state is really in a known state. It isn't.
 assert()s should also not be left in production code. They are not for catching
 runtime errors, but for testing at the expense of performance.
Are you really suggesting that asserts should be replaced by thrown exceptions? I suspect we have little common ground here.
 Uncaught exceptions should be re-thrown higher up in the call chain to a
 different error level based on the possible impact on the system. Getting an
 unexpected mismatch exception in a form-validator is not a big deal. Getting
 out-of-bounds error in main storage is a big deal. Whether it is a big deal can
 only be decided at the higher level.
A vast assumption here that you know in advance what bugs you're going to have and what causes them.
 It is no doubt useful to be able to obtain a stack trace so that you can log it
 when an exception turns out to fall into the "big deal" category and therefore
 should be re-thrown as a critical failture. The deciding factor should be
 performance.
You're using exceptions as a program bug reporting mechanism. Whoa camel, indeed!
Sep 28 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Monday, 29 September 2014 at 04:57:45 UTC, Walter Bright wrote:
 Lots of bad practices are commonplace.
This is not an argument, it is a postulate.
 You are arguing as if it is impossible to know whether the 
 logic error is local
 to the handler, or not, with a reasonable probability.
You're claiming to know that a program in an unknown and unanticipated state is really in a known state. It isn't.
It does not have to be known, it is sufficient that it is isolated or that it is improbable to be global or that it is of low impact to long term integrity.
 Are you really suggesting that asserts should be replaced by 
 thrown exceptions? I suspect we have little common ground here.
No, regular asserts should not be caught except for mailing the error log to the developer. They are for testing only. Pre/postconditions between subsystems are on a different level though. They should not be conflated with regular asserts.
 A vast assumption here that you know in advance what bugs 
 you're going to have and what causes them.
I know in advance that a "divison-by-zero" error is of limited scope with high probability or that an error in a strictly pure validator is of low impact with high probability. I also know that any sign of a flaw in a transaction engine is a critical error that warrants a shutdown. We know in advance that all programs above low complexity will contain bugs, most of them innocent and they are not a good excuse of shutting down the entire service for many services. If you have memory safety, reasonable isolation and well tested global data-structures it is most desirable to keep the system running if it is incapable of corrupting a critical database.
 You're using exceptions as a program bug reporting mechanism.
Uncaught exceptions are bugs and should be logged as such. If a form validator throws an unexpected exception then it is a bug. It makes the validation questionable, but does not affect the rest of the system. It is a non-critical bug that needs attention.
 Whoa camel, indeed!
By your line of reasoning no software should ever be shipped, without a formal proof, because they most certainly will be buggy and contain unspecified undetected state. Keep in mind that a working program, in the real world, is a program that provides reasonable output for reasonable input. Total correctness is a pipe dream, it is not within reach for most real programs. Not even with formal proofs.
Sep 28 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 11:08 PM, "Ola Fosheim Grøstad" 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 It does not have to be known, it is sufficient that it is isolated or that it
is
 improbable to be global or that it is of low impact to long term integrity.
You cannot make such presumptions and then pretend you've written robust software.
 By your line of reasoning no software should ever be shipped, without a formal
 proof, because they most certainly will be buggy and contain unspecified
 undetected state.
You're utterly misunderstanding my point. Perhaps this will help: http://www.drdobbs.com/architecture-and-design/safe-systems-from-unreliable-parts/228701716
Oct 04 2014
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Sun, 28 Sep 2014 15:59:45 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 if the program failed to catch an exception, you're already screwed
 anyway  
This is simply not true. One can write utilities with no caught exceptions at all, and yet have the program emit user friendly messages about "disk full" and stuff like that.
You're always thinking of simple console apps but this is the only place where the default 'print exception to console' strategy works. In a daemon which logs to syslog or in a GUI application or a game an uncaught 'disk full exception' would go completely unnoticed and that's definitely a bug.
Sep 29 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/29/2014 1:27 AM, Johannes Pfau wrote:
 In a daemon which logs to syslog or in a GUI application or a game an
 uncaught 'disk full exception' would go completely unnoticed and that's
 definitely a bug.
Failure to respond properly to an input/environmental error is a bug. But the input/environmental error is not a bug. If it was, then the program should assert on the error, not throw.
Sep 29 2014
parent Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 29/09/2014 10:06, Walter Bright wrote:
 On 9/29/2014 1:27 AM, Johannes Pfau wrote:
 In a daemon which logs to syslog or in a GUI application or a game an
 uncaught 'disk full exception' would go completely unnoticed and that's
 definitely a bug.
Failure to respond properly to an input/environmental error is a bug. But the input/environmental error is not a bug. If it was, then the program should assert on the error, not throw.
I agree. And isn't that exactly what Teoh said then: "That's why I said, an uncaught exception is a BUG. " I think people should be more careful with the term "uncaught exception" because it's not very precise. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Oct 01 2014
prev sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:

 Even if that is what you wanted, you won't get that from 
 FileException, as it will only show file/lines emanating from 
 calls inside std.file, not from higher level callers.
Can't we use the template arguments like __LINE__ to offer the line of code in the IO function in user code?
 Besides, take a bit of care when formulating a string for 
 exceptions, and you won't have any trouble grepping for it. 
 This isn't rocket science.
The exception is generated inside library code, not in user code. There's nothing to grep in the user code. The D script often looks like this: void main() { import std.stdio; auto file_name = "some_file; // some code here auto file_name1 = file_name ~ "1.txt"; auto f1 = File(file_name1); // some code here auto file_name2 = file_name ~ "2.txt"; auto f2 = File(file_name2, "w"); // some code here }
 Presenting internal debugging data to users for 
 input/environmental errors is just bad programming practice.7 
 We shouldn't be enshrining it in Phobos and presenting it as a 
 professional way to code.
The file/line of code is not meant to replace serious practices for dealing with I/O failures in serious D programs. Bye, bearophile
Sep 27 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/27/2014 4:59 PM, bearophile wrote:
 Walter Bright:

 Even if that is what you wanted, you won't get that from FileException, as it
 will only show file/lines emanating from calls inside std.file, not from
 higher level callers.
Can't we use the template arguments like __LINE__ to offer the line of code in the IO function in user code?
We could, but again, WHOA CAMEL! Also, one should be careful in using __LINE__ in a template, as it will be forced to generate a new instantiation with every use.
 Besides, take a bit of care when formulating a string for exceptions, and you
 won't have any trouble grepping for it. This isn't rocket science.
The exception is generated inside library code, not in user code. There's nothing to grep in the user code. The D script often looks like this: void main() { import std.stdio; auto file_name = "some_file; // some code here auto file_name1 = file_name ~ "1.txt"; auto f1 = File(file_name1); // some code here auto file_name2 = file_name ~ "2.txt"; auto f2 = File(file_name2, "w"); // some code here }
Come on. What is hard about "Cannot open file `xxxx` in mode `xxxx`", then you grep for "Cannot open file" ? Let's try it: > grep "Cannot open file" *.d stdio.d: text("Cannot open file `", name, "' in mode `", stdio.d:// new StdioException("Cannot open file `"~fName~"' for reading")); stdio.d:// new StdioException("Cannot open file `"~fName~"' for reading")); Besides, File doesn't throw a FileException and doesn't add caller's __FILE__ and __LINE__. If you're seriously proposing adding __FILE__ and __LINE__ to every function call, then D is the wrong language. This would have disastrous bloat and performance problems. You mentioned Python and Ruby, both of which are well known for miserable performance. Building the equivalent of a symbolic debugger into RELEASED D code is just not acceptable.
 Presenting internal debugging data to users for input/environmental errors is
 just bad programming practice.7 We shouldn't be enshrining it in Phobos and
 presenting it as a professional way to code.
The file/line of code is not meant to replace serious practices for dealing with I/O failures in serious D programs.
That's exactly what you're doing with it.
Sep 27 2014
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 9/27/14 7:15 PM, Walter Bright wrote:

 When I say "They are NOT for debugging programs", I mean they are NOT
 for debugging programs.
Library code often cannot make that choice. The issue with exceptions vs. errors is that often you don't know where the input comes from. e.g.: auto f = File(someInternalStringThatIsCorrupted) -> error auto f = File(argv[1]) -> exception How does File know what it's target file name came from? -Steve
Sep 27 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/27/2014 6:24 PM, Steven Schveighoffer wrote:
 On 9/27/14 7:15 PM, Walter Bright wrote:

 When I say "They are NOT for debugging programs", I mean they are NOT
 for debugging programs.
Library code often cannot make that choice. The issue with exceptions vs. errors is that often you don't know where the input comes from. e.g.: auto f = File(someInternalStringThatIsCorrupted) -> error auto f = File(argv[1]) -> exception How does File know what it's target file name came from?
If the app is concerned about invalid filenames as bugs, you should scrub the filenames first. I.e. the interface is defined improperly if the code confuses a programming bug with input errors.
Sep 27 2014
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 9/27/14 9:52 PM, Walter Bright wrote:
 On 9/27/2014 6:24 PM, Steven Schveighoffer wrote:
 On 9/27/14 7:15 PM, Walter Bright wrote:

 When I say "They are NOT for debugging programs", I mean they are NOT
 for debugging programs.
Library code often cannot make that choice. The issue with exceptions vs. errors is that often you don't know where the input comes from. e.g.: auto f = File(someInternalStringThatIsCorrupted) -> error auto f = File(argv[1]) -> exception How does File know what it's target file name came from?
If the app is concerned about invalid filenames as bugs, you should scrub the filenames first. I.e. the interface is defined improperly if the code confuses a programming bug with input errors.
OK, so if you want to avoid improperly having errors/exceptions, don't put bugs into your code. A simple plan! -Steve
Sep 29 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/29/2014 4:47 AM, Steven Schveighoffer wrote:
 On 9/27/14 9:52 PM, Walter Bright wrote:
 On 9/27/2014 6:24 PM, Steven Schveighoffer wrote:
 On 9/27/14 7:15 PM, Walter Bright wrote:

 When I say "They are NOT for debugging programs", I mean they are NOT
 for debugging programs.
Library code often cannot make that choice. The issue with exceptions vs. errors is that often you don't know where the input comes from. e.g.: auto f = File(someInternalStringThatIsCorrupted) -> error auto f = File(argv[1]) -> exception How does File know what it's target file name came from?
If the app is concerned about invalid filenames as bugs, you should scrub the filenames first. I.e. the interface is defined improperly if the code confuses a programming bug with input errors.
OK, so if you want to avoid improperly having errors/exceptions, don't put bugs into your code. A simple plan!
Validating user input is not the same thing as removing all the logic bugs from the program.
Sep 29 2014
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 9/29/14 7:53 AM, Walter Bright wrote:
 On 9/29/2014 4:47 AM, Steven Schveighoffer wrote:
 On 9/27/14 9:52 PM, Walter Bright wrote:
 If the app is concerned about invalid filenames as bugs, you should
 scrub the filenames first. I.e. the interface is defined improperly if
 the code confuses a programming bug with input errors.
OK, so if you want to avoid improperly having errors/exceptions, don't put bugs into your code. A simple plan!
Validating user input is not the same thing as removing all the logic bugs from the program.
What if it's not user input? -Steve
Sep 29 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-09-28 03:24, Steven Schveighoffer wrote:

 Library code often cannot make that choice. The issue with exceptions
 vs. errors is that often you don't know where the input comes from.

 e.g.:

 auto f = File(someInternalStringThatIsCorrupted) -> error
 auto f = File(argv[1]) -> exception

 How does File know what it's target file name came from?
Both of theses should throw an exception. Most stuff related to file operations should throw an exception, not an error. -- /Jacob Carlborg
Sep 28 2014
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 9/28/14 5:01 AM, Jacob Carlborg wrote:
 On 2014-09-28 03:24, Steven Schveighoffer wrote:

 Library code often cannot make that choice. The issue with exceptions
 vs. errors is that often you don't know where the input comes from.

 e.g.:

 auto f = File(someInternalStringThatIsCorrupted) -> error
 auto f = File(argv[1]) -> exception

 How does File know what it's target file name came from?
Both of theses should throw an exception. Most stuff related to file operations should throw an exception, not an error.
That makes no sense. The opening of the file is subject to issues with the filesystem, which means they may be environmental or user errors, not programming errors. But that doesn't mean the opening of the file failed because the file doesn't exist, it could be an error in how you construct the file name. What about: File f; f.open(null); Is that an environmental error or User error? -Steve
Sep 29 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/29/2014 4:51 AM, Steven Schveighoffer wrote:
 What about:

 File f;
 f.open(null);

 Is that an environmental error or User error?
Passing invalid arguments to a function is a programming bug.
Sep 29 2014
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 9/29/14 7:54 AM, Walter Bright wrote:
 On 9/29/2014 4:51 AM, Steven Schveighoffer wrote:
 What about:

 File f;
 f.open(null);

 Is that an environmental error or User error?
Passing invalid arguments to a function is a programming bug.
That throws an exception. My point exactly. -Steve
Sep 29 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 29/09/14 13:51, Steven Schveighoffer wrote:

 That makes no sense. The opening of the file is subject to issues with
 the filesystem, which means they may be environmental or user errors,
 not programming errors. But that doesn't mean the opening of the file
 failed because the file doesn't exist, it could be an error in how you
 construct the file name.

 What about:

 File f;
 f.open(null);

 Is that an environmental error or User error?
That depends on what "open" expect from its argument. In this particular case, in D, "null" is the same as the empty string. I don't see why that technically shouldn't be allowed. Of course, you can specify that "open" expects a string argument with the length that is greater then 0, in that case it's a bug by the programmer that uses the "open" function. -- /Jacob Carlborg
Sep 29 2014
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 9/29/14 10:48 AM, Jacob Carlborg wrote:
 On 29/09/14 13:51, Steven Schveighoffer wrote:

 That makes no sense. The opening of the file is subject to issues with
 the filesystem, which means they may be environmental or user errors,
 not programming errors. But that doesn't mean the opening of the file
 failed because the file doesn't exist, it could be an error in how you
 construct the file name.

 What about:

 File f;
 f.open(null);

 Is that an environmental error or User error?
That depends on what "open" expect from its argument. In this particular case, in D, "null" is the same as the empty string. I don't see why that technically shouldn't be allowed.
My entire point is, it doesn't matter what is expected or what is treated as "correct." what matters is where the input CAME from. To a library function, it has no idea. There is no extra type info saying "this parameter comes from user input."
 Of course, you can specify that "open" expects a string argument with
 the length that is greater then 0, in that case it's a bug by the
 programmer that uses the "open" function.
Is it? I can think of cases where it's programmer error, and cases where it's user error. There are also better example functions, but I'm focusing on File because that's what this thread is about. The largest issue I see with this whole scheme is that exceptions can be turned into errors, but not the reverse. Once an error is thrown, it's pretty much game over. So defensive coding would suggest when you don't know the answer, throw an exception, and something higher up would say "Oh, that is really a program error, rethrowing" But expecting developers to do this at EVERY CALL is really impossible. My expectation is that an exception is really an error unless caught. It would be nice to postpone generating the stack trace unless the exception is caught outside the main function. I don't know enough about how exceptions work to know if this is possible or not. -Steve
Sep 29 2014
next sibling parent reply Jeremy Powers via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, Sep 29, 2014 at 8:13 AM, Steven Schveighoffer via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 My entire point is, it doesn't matter what is expected or what is treated
 as "correct." what matters is where the input CAME from. To a library
 function, it has no idea. There is no extra type info saying "this
 parameter comes from user input."
From the method's view, parameters passed in are user input.  Full stop.
One thing that seems to be talked around a bit here is the separation/encapsulation of things. It is perfectly fine, and expected, for a library method to throw exceptions on bad input to the method - even if this input turns out to be a programming bug elsewhere. From the standpoint of the method, it does not know (and does not care) where the thing ultimately came from - all it knows is that it is input here, and it is wrong. If you call a method with bad input, and fail to catch the resulting exception, then _that_ is a bug, not the method throwing. It may be perfectly recoverable to ignore/retry/whatever, or it may be a symptom of something that should abort the program. But the method throwing does not know or care.
Sep 29 2014
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 9/29/14 2:43 PM, Jeremy Powers via Digitalmars-d wrote:
 On Mon, Sep 29, 2014 at 8:13 AM, Steven Schveighoffer via Digitalmars-d
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:

     My entire point is, it doesn't matter what is expected or what is
     treated as "correct." what matters is where the input CAME from. To
     a library function, it has no idea. There is no extra type info
     saying "this parameter comes from user input."


  From the method's view, parameters passed in are user input.  Full stop.
This is missing the point of an exception. An uncaught exception is an error which crashes the program. If you catch the exception, you can handle it, but if you don't expect it, then it's a bug. Any uncaught exceptions are BY DEFINITION programming errors.
 One thing that seems to be talked around a bit here is the
 separation/encapsulation of things.  It is perfectly fine, and expected,
 for a library method to throw exceptions on bad input to the method -
 even if this input turns out to be a programming bug elsewhere.  From
 the standpoint of the method, it does not know (and does not care) where
 the thing ultimately came from - all it knows is that it is input here,
 and it is wrong.
What is being discussed here is removing the stack trace and printout when an exception is thrown. Imagine this error message: myprompt> ./mycoolprogram (1 hour later) FileException: Error opening file xgghfsnbuer myprompt> Now what? xgghfsnbuer may not even be in the code anywhere. There may be no hints at all as to what caused it to happen. You don't even know which line of code YOU wrote that was causing the issue! You have to examine every File open, and every call that may have opened a file, and see if possibly that file name was passed into it. Whereas if you get a trace, you can at least see where the exception occurred, and start from there. Now, RELYING on this printout to be your interface to the user, that is incorrect design, I will agree. But one cannot possibly be expected to handle every possible exception at every possible call so one can throw an error in the cases where it actually is an error. D doesn't even require listing the exceptions that may be thrown on the API (and no, I'm not suggesting it should).
 If you call a method with bad input, and fail to catch the resulting
 exception, then _that_ is a bug, not the method throwing.  It may be
 perfectly recoverable to ignore/retry/whatever, or it may be a symptom
 of something that should abort the program.  But the method throwing
 does not know or care.
Sure, but it doesn't happen. Just like people do not check return values from syscalls. The benefit of the exception printing is at least you get a trace of where things went wrong when you didn't expect them to. Ignoring a call's return value doesn't give any notion something is wrong until much later. -Steve
Sep 29 2014
next sibling parent Jeremy Powers via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, Sep 29, 2014 at 11:58 AM, Steven Schveighoffer via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 9/29/14 2:43 PM, Jeremy Powers via Digitalmars-d wrote:

 On Mon, Sep 29, 2014 at 8:13 AM, Steven Schveighoffer via Digitalmars-d
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:

     My entire point is, it doesn't matter what is expected or what is
     treated as "correct." what matters is where the input CAME from. To
     a library function, it has no idea. There is no extra type info
     saying "this parameter comes from user input."


  From the method's view, parameters passed in are user input.  Full stop.
This is missing the point of an exception. An uncaught exception is an error which crashes the program. If you catch the exception, you can handle it, but if you don't expect it, then it's a bug. Any uncaught exceptions are BY DEFINITION programming errors.
I agree (except the bit about missing the point). The point I wanted to make was that encapsulation means what is a fatal error to one part of a program may be easily handled by the containing part. Just because an exception is thrown somewhere does not mean the program is broken - it is the failure to handle the exception (explicit or inadvertent) that indicates an error. What is being discussed here is removing the stack trace and printout when
 an exception is thrown.
 ....
Sure, but it doesn't happen. Just like people do not check return values
 from syscalls.

 The benefit of the exception printing is at least you get a trace of where
 things went wrong when you didn't expect them to. Ignoring a call's return
 value doesn't give any notion something is wrong until much later.

 -Steve
I absolutely do not want a removal of stack trace information. If an uncaught exception bubbles up and terminates the program, this is a bug and I sure as hell want to know as much about it as possible. If having such information presented to the end user is unacceptable, then wrap and spew something better. Ignoring an error return value is like ignoring an exception - bad news, and indicative of a broken program.
Sep 29 2014
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 29/09/2014 19:58, Steven Schveighoffer wrote:
 Any uncaught exceptions are BY DEFINITION programming errors.
Not necessarily. For some applications (for example simple console apps), you can consider the D runtime's default exception handler to be an appropriate way to respond to the exception. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Oct 01 2014
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/1/14 9:47 AM, Bruno Medeiros wrote:
 On 29/09/2014 19:58, Steven Schveighoffer wrote:
 Any uncaught exceptions are BY DEFINITION programming errors.
Not necessarily. For some applications (for example simple console apps), you can consider the D runtime's default exception handler to be an appropriate way to respond to the exception.
No, this is lazy/incorrect coding. You don't want your user to see an indecipherable stack trace on purpose. -Steve
Oct 01 2014
next sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 01/10/2014 14:55, Steven Schveighoffer wrote:
 On 10/1/14 9:47 AM, Bruno Medeiros wrote:
 On 29/09/2014 19:58, Steven Schveighoffer wrote:
 Any uncaught exceptions are BY DEFINITION programming errors.
Not necessarily. For some applications (for example simple console apps), you can consider the D runtime's default exception handler to be an appropriate way to respond to the exception.
No, this is lazy/incorrect coding. You don't want your user to see an indecipherable stack trace on purpose. -Steve
Well, at the very least it's bad UI design for sure (textual UI is still UI). But it's only a *bug* if it's not the behavior the programmer intended. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Oct 01 2014
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/1/14 10:36 AM, Bruno Medeiros wrote:
 On 01/10/2014 14:55, Steven Schveighoffer wrote:
 On 10/1/14 9:47 AM, Bruno Medeiros wrote:
 On 29/09/2014 19:58, Steven Schveighoffer wrote:
 Any uncaught exceptions are BY DEFINITION programming errors.
Not necessarily. For some applications (for example simple console apps), you can consider the D runtime's default exception handler to be an appropriate way to respond to the exception.
No, this is lazy/incorrect coding. You don't want your user to see an indecipherable stack trace on purpose. -Steve
Well, at the very least it's bad UI design for sure (textual UI is still UI). But it's only a *bug* if it's not the behavior the programmer intended.
Sure, one could also halt a program by reading a null pointer on purpose. This is a grey area that I think reasonable people can correctly call a bug if they so wish, despite the intentions of the developer. -Steve
Oct 01 2014
parent reply "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Wednesday, 1 October 2014 at 14:46:50 UTC, Steven 
Schveighoffer wrote:
 On 10/1/14 10:36 AM, Bruno Medeiros wrote:

This is a grey area that I think reasonable people
 can correctly call a bug if they so wish, despite the 
 intentions of the developer.
Correctly? In a discussion, It's amazing how difficult it is to agree also on simple words meaning: an _intentional programmer behaviour_ a bug? Whah ;-P --- /Paolo
Oct 01 2014
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/1/14 3:24 PM, Paolo Invernizzi wrote:
 On Wednesday, 1 October 2014 at 14:46:50 UTC, Steven Schveighoffer wrote:
 On 10/1/14 10:36 AM, Bruno Medeiros wrote:

 This is a grey area that I think reasonable people
 can correctly call a bug if they so wish, despite the intentions of
 the developer.
Correctly? In a discussion, It's amazing how difficult it is to agree also on simple words meaning: an _intentional programmer behaviour_ a bug?
More appropriately, it's not a bug but an incorrect design. I still would call it a bug, and I'm 99% sure the users would report it as a bug :) -Steve
Oct 01 2014
prev sibling parent reply Andrej Mitrovic via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 10/1/14, Steven Schveighoffer via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 No, this is lazy/incorrect coding. You don't want your user to see an
 indecipherable stack trace on purpose.
So when they file a bug report are you going to also ask them to run the debugger so they capture the stack trace and file that to you? Come on.
Oct 01 2014
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/1/14 11:00 AM, Andrej Mitrovic via Digitalmars-d wrote:
 On 10/1/14, Steven Schveighoffer via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 No, this is lazy/incorrect coding. You don't want your user to see an
 indecipherable stack trace on purpose.
So when they file a bug report are you going to also ask them to run the debugger so they capture the stack trace and file that to you? Come on.
No what I mean is: ./niftyapp badfilename.txt Result should be: Error: Could not open badfilename.txt, please check and make sure the file exists and is readable. Not: std.exception.ErrnoException std/stdio.d(345): Cannot open file `badfilename.txt' in mode `rb' (No such file or directory) ---------------- 5 testexception 0x0000000104fad02d ref std.stdio.File std.stdio.File.__ctor(immutable(char)[], const(char[])) + 97 6 testexception 0x0000000104f8d735 _Dmain + 69 7 testexception 0x0000000104f9f771 void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll().void __lambda1() + 33 8 testexception 0x0000000104f9f6bd void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate()) + 45 9 testexception 0x0000000104f9f71d void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() + 45 10 testexception 0x0000000104f9f6bd void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate()) + 45 11 testexception 0x0000000104f9f639 _d_run_main + 449 12 testexception 0x0000000104f8d75c main + 20 13 libdyld.dylib 0x00007fff8fb2a5fd start + 1 14 ??? 0x0000000000000001 0x0 + 1 If it's an error due to *user input*, you should not rely on the exception handling of the runtime, you should have a more user-friendly message. Obviously, if you fail to handle it, the full trace happens, and then you must fix that in your code. It's for your benefit too :) This way you get less nuisance troubleshooting calls since the error message is clearer. -Steve
Oct 01 2014
prev sibling next sibling parent reply Jeremy Powers via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, Sep 29, 2014 at 8:13 AM, Steven Schveighoffer via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 The largest issue I see with this whole scheme is that exceptions can be
 turned into errors, but not the reverse. Once an error is thrown, it's
 pretty much game over. So defensive coding would suggest when you don't
 know the answer, throw an exception, and something higher up would say "Oh,
 that is really a program error, rethrowing"

 But expecting developers to do this at EVERY CALL is really impossible.
And this is an argument for checked exceptions - being able to explicitly state 'these are known fatal cases for this component, you should deal with them appropriately' when defining a method. Cuts down the catch/check to just the common cases, and makes such cases explicit to the caller. Anything not a checked exception falls into the 'error, abort!' path. (Memory corruption etc. being abort scenarios) If I really needed to write a robust program in D right now, I would (attempt) to wrap every call in a try/catch, and check if the thrown exception was of a handleable type. But knowing which types for which methods would lead me to basically hacking up some documentation-enforced checked exceptions, and being entirely unmaintainable.
Sep 29 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 29 September 2014 at 18:59:59 UTC, Jeremy Powers via
Digitalmars-d wrote:
 And this is an argument for checked exceptions - being able to 
 explicitly state 'these are known fatal cases for this 
 component, you should deal with them appropriately' when 
 defining a method.  Cuts down the catch/check to just the 
 common cases, and makes such cases explicit to the caller. 
 Anything not a checked exception falls into the 'error, abort!' 
 path.  (Memory corruption etc. being abort scenarios)
Checked exceptions are good in theory but they failed utterly in Java. I'm not interested in seeing them in D.
Sep 29 2014
next sibling parent reply Jeremy Powers via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, Sep 29, 2014 at 12:28 PM, Sean Kelly via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 Checked exceptions are good in theory but they failed utterly in
 Java.  I'm not interested in seeing them in D.
I've heard this before, but have not seen a reasonable argument as to why they are a failure. Last time this was discussed a link to a blog was provided, with lots of discussion there - which as far as I could tell boiled down to 'catching exceptions is ugly, and people just do the wrong thing anyway which is ugly when you have checked exceptions.' I am unlucky enough to write Java all day, and from my standpoint checked exceptions are a huge win. There are certain edges which can catch you, but they are immensely useful in developing robust programs. Basically checked exceptions -> recoverable problems, unchecked -> unrecoverable/programming errors (like asserts or memory errors). Note I am not advocating adding checked exceptions to D (though I would like it). Point is to acknowledge that there are different kinds of exceptions, and an exception for one part of the code may not be a problem for the bit that invokes it.
Sep 29 2014
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 9/29/14 3:44 PM, Jeremy Powers via Digitalmars-d wrote:
 On Mon, Sep 29, 2014 at 12:28 PM, Sean Kelly via Digitalmars-d
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:

     Checked exceptions are good in theory but they failed utterly in
     Java.  I'm not interested in seeing them in D.


 I've heard this before, but have not seen a reasonable argument as to
 why they are a failure.  Last time this was discussed a link to a blog
 was provided, with lots of discussion there - which as far as I could
 tell boiled down to 'catching exceptions is ugly, and people just do the
 wrong thing anyway which is ugly when you have checked exceptions.'

 I am unlucky enough to write Java all day, and from my standpoint
 checked exceptions are a huge win.  There are certain edges which can
 catch you, but they are immensely useful in developing robust programs.
 Basically checked exceptions -> recoverable problems, unchecked ->
 unrecoverable/programming errors (like asserts or memory errors).
Well, the failure comes from the effort to effect a certain behavior. Sun was looking to make programmers more diligent about handling errors. However, humans are lazy worthless creatures. What ends up happening is, the compiler complains they aren't handling an exception. They can't see any reason why the exception would occur, so they simply catch and ignore it to shut the compiler up. In 90% of cases, they are right -- the exception will not occur. But because they have been "trained" to simply discard exceptions, it ends up defeating the purpose for the 10% of the time that they are wrong. If you have been able to resist that temptation and handle every exception, then I think you are in the minority. But I have no evidence to back this up, it's just a belief.
 Note I am not advocating adding checked exceptions to D (though I would
 like it).  Point is to acknowledge that there are different kinds of
 exceptions, and an exception for one part of the code may not be a
 problem for the bit that invokes it.
I think this is appropriate for a lint tool for those out there like yourself who want that information. But requiring checked exceptions is I think a futile attempt to outlaw natural human behavior. -Steve
Sep 30 2014
parent Jeremy Powers via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, Sep 30, 2014 at 5:43 AM, Steven Schveighoffer via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 ...
 Well, the failure comes from the effort to effect a certain behavior.

 Sun was looking to make programmers more diligent about handling errors.
 However, humans are lazy worthless creatures. What ends up happening is,
 the compiler complains they aren't handling an exception. They can't see
 any reason why the exception would occur, so they simply catch and ignore
 it to shut the compiler up.

 In 90% of cases, they are right -- the exception will not occur. But
 because they have been "trained" to simply discard exceptions, it ends up
 defeating the purpose for the 10% of the time that they are wrong.
That's the argument, but it doesn't seem valid to me. Without checked exceptions, you will always be ignoring exceptions. With checked exceptions, you have to explicitly ignore (some) exceptions, and when you do it is immediately obvious in the code. You go from everyone ignoring exceptions all the time, to some people ignoring them - and being able to easily notice and call out such. Anyone 'trained' to ignore checked exceptions are simply shooting themselves in the foot - same as if there were no checked exceptions, but with more verbosity. This is not a failure of checked exceptions, but a failure of people to use a language feature properly. (Which, yeah, meta is a failure of the feature... not going to go there)
 If you have been able to resist that temptation and handle every
 exception, then I think you are in the minority. But I have no evidence to
 back this up, it's just a belief.
In my world of professional java, ignoring exceptions is an immediate, obvious indicator of bad code. You will be called on it, and chastised appropriately. So from my standpoint, Sun was successful in making programmers more diligent about handling errors.
  Note I am not advocating adding checked exceptions to D (though I would
 like it).  Point is to acknowledge that there are different kinds of
 exceptions, and an exception for one part of the code may not be a
 problem for the bit that invokes it.
I think this is appropriate for a lint tool for those out there like yourself who want that information. But requiring checked exceptions is I think a futile attempt to outlaw natural human behavior.
Perhaps I shouldn't have mentioned checked exceptions at all, seem to be distracting from what I wanted to say. The important bit I wanted to bring to the discussion is that not all exceptions are the same, and different sections of code have their own ideas of what is a breaking problem. A module/library/component/whatever treats any input into itself as its input, and thus appropriately throws exceptions on bad input. But code using that whatever may be perfectly fine handling exceptions coming from there. Exceptions need to be appropriate to the given abstraction, and dealt with by the user of that abstraction.
Sep 30 2014
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 29/09/2014 20:28, Sean Kelly wrote:
 Checked exceptions are good in theory but they failed utterly in
 Java.  I'm not interested in seeing them in D.
That is the conventional theory, the established wisdom. But the more I become experienced with Java, over the years, I've become convinced otherwise. What has failed is not the concept of checked exceptions per se, but mostly, the failure of Java programmers to use checked exceptions effectively, and properly design their code around this paradigm. Like Jeremy mentioned, if one puts catch blocks right around the function that throws an exception, and just swallow/forget it there without doing anything else, then it's totally the programmers fault for being lazy. If one is annoyed that often, adding a throws clause in a function will require adding the same throws clause function to several other functions, well, that is editing work you have to accept for the sake of more correctness. But also one should understand there are ways to mitigate this editing work: First point is that in a lot of code, is better to have a function throw just one generic (but checked) exception, that can wrap any other specific errors/exceptions. If you are doing an operation that can throw File-Not-Found, Invalid-Path, No-Permissions, IO-Exception, etc., then often all of these will be handled in the same user-reporting code, so they could be wrapped under a single exception that would be used in the throws clause. And so the whole function call chain doesn't need to be modified every time a new exception is added or removed. If you're thinking that means adding a "throws Exception" to such functions in Java, then no. Because this will catch RuntimeExceptions too (the unchecked exceptions of Java), and these you often want to handle elsewhere than where you handle the checked exceptions. In this regard, Java does have a design fault, IMO, which is that there is no common superclass for checked Exceptions. (there is only for unchecked exceptions) The second point, is that even adding (or modifying) the throws clause of function signatures cause be made much easier with an IDE, and in particular Eclipse JDT helps a lot. If you have an error in the editor about a checked exception that is not caught or thrown, you can just press Ctrl-1 to automatically add either a throws clause, or a surrounding try-catch block. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Oct 01 2014
parent reply Jeremy Powers via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, Oct 1, 2014 at 7:24 AM, Bruno Medeiros via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 What has failed is not the concept of checked exceptions per se, but
 mostly, the failure of Java programmers to use checked exceptions
 effectively, and properly design their code around this paradigm.
This. I have seen many java programs (and their programmers) fail utterly at using exceptions. Like OO, the design of the system has to leverage it properly, and there are places where people can easily get tripped up - but when used well, can be immensely useful. Error handling is part of an API, and exceptions are error handling, so should be considered when designing API. Checked exceptions are a language-supported way to do this. For those that consider checked exceptions a failure: what other feature(s) would work instead? NB: If you see "throws Exception" in java code chances are the code is broken, same as if you see "catch (Exception" - this tells you nothing about the exception that happened, and hence you can do nothing with it. So you either swallow (and silently break in many cases) or rethrow (and break for things you needn't have). As mentioned, the standard way to avoid this is to have a parent exception type appropriate to the abstraction in the API, and throw subtypes in the implementation. Among other things, this means you can change the implementation to throw different exceptions without breaking any users (who will already be catching the parent exception). Adding/modifying a throws clause is an API-breaking change, so should be avoided however easy it is in the IDE. (Yes, I'm biased to writing libraries consumed by others) NNB: Retrofitting a program to use proper exception handling is much harder than it is to design it the right way from scratch. I'm going to compare to OO again: don't consider OO broken because people use inheritance when they want ownership, and it is painful to fix later.
Oct 01 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Wednesday, 1 October 2014 at 22:42:27 UTC, Jeremy Powers via
Digitalmars-d wrote:
 If you see "throws Exception" in java code chances are the code 
 is broken,
 same as if you see "catch (Exception" - this tells you nothing 
 about the
 exception that happened, and hence you can do nothing with it.  
 So you
 either swallow (and silently break in many cases) or rethrow 
 (and break for things you needn't have).  As mentioned, the 
 standard way to avoid this is to have a parent exception type 
 appropriate to the abstraction in the API, and throw subtypes 
 in the implementation.  Among other things, this means you can 
 change the implementation to throw different exceptions without 
 breaking any users (who will already be catching the parent 
 exception).
...while in Phobos, most of the subtyped exceptions were eliminated a while back in favor of just always throwing Exception.
Oct 01 2014
next sibling parent reply "David Nadlinger" <code klickverbot.at> writes:
On Wednesday, 1 October 2014 at 23:00:41 UTC, Sean Kelly wrote:
 ...while in Phobos, most of the subtyped exceptions were
 eliminated a while back in favor of just always throwing
 Exception.
What are you referring to specifically? Compared to Tango, yes, Phobos might have a lot fewer concrete exception types. But I don't recall actually eliminating existing ones. David
Oct 01 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 02/10/14 01:19, David Nadlinger wrote:

 What are you referring to specifically? Compared to Tango, yes, Phobos
 might have a lot fewer concrete exception types. But I don't recall
 actually eliminating existing ones.
It happens implicitly when using "enforce". By default it will throw an instance of Exception. In most cases I don't think the developer bothers with specifying a specific exception type. I really, really hate this. It makes it basically impossible to do any form of error handling correctly. I think Exception should be an interface or abstract class. -- /Jacob Carlborg
Oct 01 2014
parent reply "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 2 October 2014 at 06:39:17 UTC, Jacob Carlborg wrote:
 On 02/10/14 01:19, David Nadlinger wrote:

 What are you referring to specifically? Compared to Tango, 
 yes, Phobos
 might have a lot fewer concrete exception types. But I don't 
 recall actually eliminating existing ones.
It happens implicitly when using "enforce".
I'm well aware of that. But as On Wednesday, 1 October 2014 at 23:00:41 UTC, Sean Kelly wrote:
 ...while in Phobos, most of the subtyped exceptions were
 eliminated a while back in favor of just always throwing
 Exception.
you are saying that specific exceptions were replaced by enforce? I can't recall something like this happening. David
Oct 03 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 2014-10-03 14:36, David Nadlinger wrote:

 you are saying that specific exceptions were replaced by enforce? I
 can't recall something like this happening.
I have no idea about this but I know there are a lot of "enforce" in Phobos and it sees to be encouraged to use it. Would be really sad if specific exceptions were deliberately replaced with less specific exceptions. -- /Jacob Carlborg
Oct 03 2014
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Fri, 03 Oct 2014 21:35:01 +0200
schrieb Jacob Carlborg <doob me.com>:

 On 2014-10-03 14:36, David Nadlinger wrote:
 
 you are saying that specific exceptions were replaced by enforce? I
 can't recall something like this happening.
I have no idea about this but I know there are a lot of "enforce" in Phobos and it sees to be encouraged to use it. Would be really sad if specific exceptions were deliberately replaced with less specific exceptions.
Nice, finally someone who actually wants to discern Exception types. I'm always at a loss as to what warrants its own exception type. E.g. when looking at network protocols, would a 503 be a NetworkException, a HTTPException or a HTTPInternalServerErrorException ? Where do *you* wish libraries would differentiate? Or does it really come down to categories like "illegal argument", "division by zero", "null pointer", "out of memory" for you? -- Marco
Oct 05 2014
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/5/14, 8:08 AM, Marco Leise wrote:
 Nice, finally someone who actually wants to discern Exception
 types. I'm always at a loss as to what warrants its own
 exception type. E.g. when looking at network protocols, would
 a 503 be a NetworkException, a HTTPException or a
 HTTPInternalServerErrorException ?
 Where do*you*  wish libraries would differentiate?
 Or does it really come down to categories like "illegal
 argument", "division by zero", "null pointer", "out of memory"
 for you?
That reflects my misgivings about using exception hierarchies for error kinds as well. -- Andrei
Oct 05 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-10-05 17:08, Marco Leise wrote:

 Nice, finally someone who actually wants to discern Exception
 types. I'm always at a loss as to what warrants its own
 exception type.
Yeah, that can be quite difficult. In the end, if you have a good exception hierarchy I don't think it will hurt to have many different exception types.
 E.g. when looking at network protocols, would
 a 503 be a NetworkException, a HTTPException or a
 HTTPInternalServerErrorException ?
For this specific case I would probably have one general exception for all HTTP error codes.
 Where do *you* wish libraries would differentiate?
I would like to have as specific exception type as possible. Also a nice hierarchy of exception when catching a specific exception is not interesting. Instead of just a FileException there could be FileNotFoundException, PermissionDeniedExcepton and so on.
 Or does it really come down to categories like "illegal
 argument", "division by zero", "null pointer", "out of memory"
 for you?
-- /Jacob Carlborg
Oct 05 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/5/14, 8:56 AM, Jacob Carlborg wrote:
 I would like to have as specific exception type as possible. Also a nice
 hierarchy of exception when catching a specific exception is not
 interesting. Instead of just a FileException there could be
 FileNotFoundException, PermissionDeniedExcepton and so on.
Exceptions are all about centralized error handling. How, and how often, would you handle FileNotFoundException differently than PermissionDeniedException? Andrei
Oct 05 2014
next sibling parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 5 October 2014 at 16:18:33 UTC, Andrei Alexandrescu 
wrote:
 On 10/5/14, 8:56 AM, Jacob Carlborg wrote:
 I would like to have as specific exception type as possible. 
 Also a nice
 hierarchy of exception when catching a specific exception is 
 not
 interesting. Instead of just a FileException there could be
 FileNotFoundException, PermissionDeniedExcepton and so on.
Exceptions are all about centralized error handling. How, and how often, would you handle FileNotFoundException differently than PermissionDeniedException? Andrei
While precise formalization of the principle is hard I think this comment nails it in general. When defining exceptions hierarchies it makes sense to think about it in terms of "what code is likely to catch it and why?" and not "what this code should throw?". Dedicated exception type only makes sense if it is a common to catch it separately, any additional details can be stored as runtime field (like status code for HTTPStatusException)
Oct 05 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/5/2014 9:18 AM, Andrei Alexandrescu wrote:
 Exceptions are all about centralized error handling. How, and how often, would
 you handle FileNotFoundException differently than PermissionDeniedException?
You would handle it differently if there was extra data attached to that particular exception, specific to that sort of error.
Oct 05 2014
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/5/14, 2:27 PM, Walter Bright wrote:
 On 10/5/2014 9:18 AM, Andrei Alexandrescu wrote:
 Exceptions are all about centralized error handling. How, and how
 often, would
 you handle FileNotFoundException differently than
 PermissionDeniedException?
You would handle it differently if there was extra data attached to that particular exception, specific to that sort of error.
Indeed. Very few in Phobos do. -- Andrei
Oct 05 2014
prev sibling next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
05-Oct-2014 20:18, Andrei Alexandrescu пишет:
 On 10/5/14, 8:56 AM, Jacob Carlborg wrote:
 I would like to have as specific exception type as possible. Also a nice
 hierarchy of exception when catching a specific exception is not
 interesting. Instead of just a FileException there could be
 FileNotFoundException, PermissionDeniedExcepton and so on.
Exceptions are all about centralized error handling. How, and how often, would you handle FileNotFoundException differently than PermissionDeniedException?
Seems like it should be possible to define multiple interfaces for exceptions, and then catch by that (and/or combinations of such). Each of interface would be interested in a particular property of exception. Then catching by: FileException with PermissionException would mean OS-level permission viloated and it was during file access, while ProcessException with PermissionException would mean process manipulation was forbiden, etc. Of course, some code may be interested only in PermissionException side of things, while other code may want to contain anything related to files, and the catch-all-sensible-ones inside of the main function. -- Dmitry Olshansky
Oct 05 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 05/10/14 23:50, Dmitry Olshansky wrote:

 Seems like it should be possible to define multiple interfaces for
 exceptions, and then catch by that (and/or combinations of such).

 Each of interface would be interested in a particular property of
 exception. Then catching by:

 FileException with PermissionException

 would mean OS-level permission viloated and it was during file access,

 while

 ProcessException with PermissionException would mean process
 manipulation was forbiden, etc.

 Of course, some code may be interested only in PermissionException side
 of things, while other code may want to contain anything related to
 files, and the catch-all-sensible-ones inside of the main function.
Why not just define two different PermissionException? -- /Jacob Carlborg
Oct 05 2014
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
06-Oct-2014 10:33, Jacob Carlborg пишет:
 On 05/10/14 23:50, Dmitry Olshansky wrote:

 Seems like it should be possible to define multiple interfaces for
 exceptions, and then catch by that (and/or combinations of such).

 Each of interface would be interested in a particular property of
 exception. Then catching by:

 FileException with PermissionException

 would mean OS-level permission viloated and it was during file access,

 while

 ProcessException with PermissionException would mean process
 manipulation was forbiden, etc.

 Of course, some code may be interested only in PermissionException side
 of things, while other code may want to contain anything related to
 files, and the catch-all-sensible-ones inside of the main function.
Why not just define two different PermissionException?
Rather N permission exceptions. It gets worse with more problem domains to cover. Again the trick is being able to catch all PermissionExceptions, including for exception type you may not know in advance. It's obvious to me that one hierarchy is way to limiting for exceptions, simply because there could be many ways to categorize the same set of error conditions. -- Dmitry Olshansky
Oct 05 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/5/14, 11:39 PM, Dmitry Olshansky wrote:
 It's obvious to me that one hierarchy is way to limiting for exceptions,
 simply because there could be many ways to categorize the same set of
 error conditions.
Well put. -- Andrei
Oct 06 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Mon, Oct 06, 2014 at 06:46:31AM -0700, Andrei Alexandrescu via Digitalmars-d
wrote:
 On 10/5/14, 11:39 PM, Dmitry Olshansky wrote:
It's obvious to me that one hierarchy is way to limiting for
exceptions, simply because there could be many ways to categorize the
same set of error conditions.
Well put. -- Andrei
What's the alternative, though? I can't think of any solution that (1) isn't far more complicated than the current state of things and (2) is easier to use. T -- "Holy war is an oxymoron." -- Lazarus Long
Oct 06 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/6/14, 7:01 AM, H. S. Teoh via Digitalmars-d wrote:
 On Mon, Oct 06, 2014 at 06:46:31AM -0700, Andrei Alexandrescu via
Digitalmars-d wrote:
 On 10/5/14, 11:39 PM, Dmitry Olshansky wrote:
 It's obvious to me that one hierarchy is way to limiting for
 exceptions, simply because there could be many ways to categorize the
 same set of error conditions.
Well put. -- Andrei
What's the alternative, though? I can't think of any solution that (1) isn't far more complicated than the current state of things and (2) is easier to use.
I'm thinking a simple key-value store Variant[string] would accommodate any state needed for differentiating among exception kinds whenever that's necessary. It's commonly accepted that the usability scope of OOP has gotten significantly narrower since its heydays. However, surprisingly, the larger community hasn't gotten to the point to scrutinize object-oriented error handling, which as far as I can tell has never delivered. Andrei
Oct 06 2014
parent reply Jeremy Powers via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, Oct 6, 2014 at 7:50 AM, Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 10/6/14, 7:01 AM, H. S. Teoh via Digitalmars-d wrote:

 On Mon, Oct 06, 2014 at 06:46:31AM -0700, Andrei Alexandrescu via
 Digitalmars-d wrote:

 On 10/5/14, 11:39 PM, Dmitry Olshansky wrote:

 It's obvious to me that one hierarchy is way to limiting for
 exceptions, simply because there could be many ways to categorize the
 same set of error conditions.
Well put. -- Andrei
What's the alternative, though? I can't think of any solution that (1) isn't far more complicated than the current state of things and (2) is easier to use.
I'm thinking a simple key-value store Variant[string] would accommodate any state needed for differentiating among exception kinds whenever that's necessary.
And 'kinds' is a synonym for 'types' - You can have different kinds of problems, so you raise them with different kinds of exceptions. s/kind/type/g and the question is: why not leverage the type system? For a consumer-of-something-that-throws, having different types of exceptions for different things with different data makes sense. You have to switch on something to determine what data you can get from the exception anyway.
 It's commonly accepted that the usability scope of OOP has gotten
 significantly narrower since its heydays. However, surprisingly, the larger
 community hasn't gotten to the point to scrutinize object-oriented error
 handling, which as far as I can tell has never delivered.
Maybe, but what fits better? Errors/Exceptions have an inherent hierarchy, which maps well to a hierarchy of types. When catching an Exception, you want to guarantee you only catch the kinds (types) of things you are looking for, and nothing else.
Oct 06 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/6/14, 4:46 PM, Jeremy Powers via Digitalmars-d wrote:
 On Mon, Oct 6, 2014 at 7:50 AM, Andrei Alexandrescu via Digitalmars-d
     I'm thinking a simple key-value store Variant[string] would
     accommodate any state needed for differentiating among exception
     kinds whenever that's necessary.


 And 'kinds' is a synonym for 'types' - You can have different kinds of
 problems, so you raise them with different kinds of exceptions.

 s/kind/type/g and the question is: why not leverage the type system?
I've used "kinds" intentionally there. My basic thesis here is I haven't seen any systematic and successful use of exception hierarchies in 20 years. In rare scattered cases I've seen a couple of multiple "catch"es, and even those could have been helped by the use of a more flat handling. You'd think in 20 years some good systematic use of the feature would come forward. It's probably time to put exception hierarchies in the "emperor's clothes" bin.
 For a consumer-of-something-that-throws, having different types of
 exceptions for different things with different data makes sense.  You
 have to switch on something to determine what data you can get from the
 exception anyway.
Oh yah I know the theory. It's beautiful.
     It's commonly accepted that the usability scope of OOP has gotten
     significantly narrower since its heydays. However, surprisingly, the
     larger community hasn't gotten to the point to scrutinize
     object-oriented error handling, which as far as I can tell has never
     delivered.


 Maybe, but what fits better?  Errors/Exceptions have an inherent
 hierarchy, which maps well to a hierarchy of types.  When catching an
 Exception, you want to guarantee you only catch the kinds (types) of
 things you are looking for, and nothing else.
Yah, it's just that most/virtually all of the time I'm looking for all. And nothing else :o). Andrei
Oct 06 2014
next sibling parent Jeremy Powers via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, Oct 6, 2014 at 6:19 PM, Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 10/6/14, 4:46 PM, Jeremy Powers via Digitalmars-d wrote:

 On Mon, Oct 6, 2014 at 7:50 AM, Andrei Alexandrescu via Digitalmars-d
     I'm thinking a simple key-value store Variant[string] would
     accommodate any state needed for differentiating among exception
     kinds whenever that's necessary.


 And 'kinds' is a synonym for 'types' - You can have different kinds of
 problems, so you raise them with different kinds of exceptions.

 s/kind/type/g and the question is: why not leverage the type system?
I've used "kinds" intentionally there. My basic thesis here is I haven't seen any systematic and successful use of exception hierarchies in 20 years. In rare scattered cases I've seen a couple of multiple "catch"es, and even those could have been helped by the use of a more flat handling. You'd think in 20 years some good systematic use of the feature would come forward. It's probably time to put exception hierarchies in the "emperor's clothes" bin. For a consumer-of-something-that-throws, having different types of
 exceptions for different things with different data makes sense.  You
 have to switch on something to determine what data you can get from the
 exception anyway.
Oh yah I know the theory. It's beautiful.
I'm not talking theory (exclusively). From a practical standpoint, if I ever need information from an exception I need to know what information I can get. If different exceptions have different information, how do I tell what I can get? Types fits this as well as/better than anything I can think of.
      It's commonly accepted that the usability scope of OOP has gotten
     significantly narrower since its heydays. However, surprisingly, the
     larger community hasn't gotten to the point to scrutinize
     object-oriented error handling, which as far as I can tell has never
     delivered.


 Maybe, but what fits better?  Errors/Exceptions have an inherent
 hierarchy, which maps well to a hierarchy of types.  When catching an
 Exception, you want to guarantee you only catch the kinds (types) of
 things you are looking for, and nothing else.
Yah, it's just that most/virtually all of the time I'm looking for all. And nothing else :o).
Most/virtually all of the time I am looking only for the kind of exceptions I expect and can handle. If I catch an exception that I was not expecting, this is a program bug (and may result in undefined behavior, memory corruption, etc). Catching all is almost _never_ what I want. I have not found a whole lot of use for deep exception hierarchies, but some organization of types/kinds of exceptions is needed. At the very least you need to know if it is the kind of exception you know how to handle - and without a hierarchy, you need to know every single specific kind of exception anything you call throws. Which is not tenable.
Oct 07 2014
prev sibling parent Jeremy Powers via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, Oct 6, 2014 at 6:19 PM, Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 10/6/14, 4:46 PM, Jeremy Powers via Digitalmars-d wrote:

 On Mon, Oct 6, 2014 at 7:50 AM, Andrei Alexandrescu via Digitalmars-d
     I'm thinking a simple key-value store Variant[string] would
     accommodate any state needed for differentiating among exception
     kinds whenever that's necessary.


 And 'kinds' is a synonym for 'types' - You can have different kinds of
 problems, so you raise them with different kinds of exceptions.

 s/kind/type/g and the question is: why not leverage the type system?
I've used "kinds" intentionally there. My basic thesis here is I haven't seen any systematic and successful use of exception hierarchies in 20 years. In rare scattered cases I've seen a couple of multiple "catch"es, and even those could have been helped by the use of a more flat handling. You'd think in 20 years some good systematic use of the feature would come forward. It's probably time to put exception hierarchies in the "emperor's clothes" bin.
Sorry, forgot to respond to this part. As mentioned, I'm not a defender of hierarchies per se - but I've not seen any alternate way to accomplish what they give. I need to know that I am catching exceptions that I can handle, and not catching exceptions I can't/won't handle. Different components and layers of code have different ideas of what can and should be handled. Without particular exception types, how can I know that I am only catching what is appropriate, and not catching and swallowing other problems?
Oct 07 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 05/10/14 18:18, Andrei Alexandrescu wrote:

 Exceptions are all about centralized error handling. How, and how often,
 would you handle FileNotFoundException differently than
 PermissionDeniedException?
Probably not that often. But I don't think it's up to "File" to decide that. I think "File" should throw a specific error as possible containing all details it has. Then it's up to the user of "File" how to handle the exception. If you're not interested in the differences between FileNotFoundException and PermissionDeniedException, then catch the base class instead. The details provided between these two exception could be different as well. FileNotFoundException should contain the path of the file that wasn't found. PermissionDeniedException should contain some information about if it was the source or target that caused the exception to be thrown. Think of a copy or move operation. How would one localize error messages if the specific exception is not known? -- /Jacob Carlborg
Oct 05 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/5/14, 11:31 PM, Jacob Carlborg wrote:
 On 05/10/14 18:18, Andrei Alexandrescu wrote:

 Exceptions are all about centralized error handling. How, and how often,
 would you handle FileNotFoundException differently than
 PermissionDeniedException?
Probably not that often. But I don't think it's up to "File" to decide that. I think "File" should throw a specific error as possible containing all details it has.
Sure, but that can be distinguished by payload, not type.
 Then it's up to the user of "File" how to
 handle the exception. If you're not interested in the differences
 between FileNotFoundException and PermissionDeniedException, then catch
 the base class instead.

 The details provided between these two exception could be different as
 well. FileNotFoundException should contain the path of the file that
 wasn't found. PermissionDeniedException should contain some information
 about if it was the source or target that caused the exception to be
 thrown. Think of a copy or move operation.

 How would one localize error messages if the specific exception is not
 known?
Knowledge doesn't have to be by type; just place data inside the exception. About the only place where multiple "catch" statements are used to make fine distinctions between exception types is in sample code showing how to use multiple "catch" statements :o). This whole notion that different exceptions need different types is as far as I can tell a red herring. Andrei
Oct 06 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 06/10/14 15:45, Andrei Alexandrescu wrote:

 Knowledge doesn't have to be by type; just place data inside the
 exception. About the only place where multiple "catch" statements are
 used to make fine distinctions between exception types is in sample code
 showing how to use multiple "catch" statements :o). This whole notion
 that different exceptions need different types is as far as I can tell a
 red herring.
What do you suggest, error codes? I consider that an ugly hack. -- /Jacob Carlborg
Oct 06 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/6/14, 7:48 AM, Jacob Carlborg wrote:
 On 06/10/14 15:45, Andrei Alexandrescu wrote:

 Knowledge doesn't have to be by type; just place data inside the
 exception. About the only place where multiple "catch" statements are
 used to make fine distinctions between exception types is in sample code
 showing how to use multiple "catch" statements :o). This whole notion
 that different exceptions need different types is as far as I can tell a
 red herring.
What do you suggest, error codes? I consider that an ugly hack.
I don't. On the contrary, I do consider proliferating types to the multiplicity of possible errors an obvious design sin. -- Andrei
Oct 06 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 2014-10-06 17:07, Andrei Alexandrescu wrote:

 I don't. On the contrary, I do consider proliferating types to the
 multiplicity of possible errors an obvious design sin. -- Andrei
You loose the ability to have exception specific data. And no, I don't want to see an associative array of Variants, that's even worse hack then error codes. You'll run in to problems with unique keys and will most likely need to "scope" all keys. -- /Jacob Carlborg
Oct 06 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdan.org> writes:
Jacob Carlborg <doob me.com> wrote:
 On 2014-10-06 17:07, Andrei Alexandrescu wrote:
 
 I don't. On the contrary, I do consider proliferating types to the
 multiplicity of possible errors an obvious design sin. -- Andrei
You loose the ability to have exception specific data. And no, I don't want to see an associative array of Variants, that's even worse hack then error codes. You'll run in to problems with unique keys and will most likely need to "scope" all keys.
Then scope them.
Oct 06 2014
parent Jacob Carlborg <doob me.com> writes:
On 06/10/14 20:26, Andrei Alexandrescu wrote:

 Then scope them.
We already have scoping rules built in to the language. Why should we invent a new way and set of rules for this. -- /Jacob Carlborg
Oct 06 2014
prev sibling parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Mon, 06 Oct 2014 15:48:31 +0100, Jacob Carlborg <doob me.com> wrote:

 On 06/10/14 15:45, Andrei Alexandrescu wrote:

 Knowledge doesn't have to be by type; just place data inside the
 exception. About the only place where multiple "catch" statements are
 used to make fine distinctions between exception types is in sample code
 showing how to use multiple "catch" statements :o). This whole notion
 that different exceptions need different types is as far as I can tell a
 red herring.
What do you suggest, error codes? I consider that an ugly hack.
Why? It gives us the benefits of error code return values: - ability to easily/cheaply check for/compare them using "switch" on code value (vs comparing/casting types) - ability to pass through OS level codes directly Without any of the penalties: - checking for them after every call. - losing the return value "slot" or having to engineer multiple return values in the language. - having to mix error codes in with valid return values (for int() functions). We also get: - no type proliferation. - no arguments about what exception types are needed, or the hierarchy to put them in. Seems like a win to me. Of course.. it would be nicer still if there was a list of OS/platform agnostic error codes which were used throughout phobos and could be re-used by client code. And.. (for example) it would be nice if there was a FileNotFound(string path) function which returned an Exception using the correct code allowing: throw FileNotFound(path); and so on. I do not know a lot about how exceptions are thrown and caught at the compiler/compiled code level, but perhaps there is even a performance benefit to be had if you know that only 2 possible types (Exception and Error) can/will be thrown.. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Oct 06 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 2014-10-06 18:03, Regan Heath wrote:

 Why?

 It gives us the benefits of error code return values:
   - ability to easily/cheaply check for/compare them using "switch" on
 code value (vs comparing/casting types)
   - ability to pass through OS level codes directly

 Without any of the penalties:
   - checking for them after every call.
   - losing the return value "slot" or having to engineer multiple return
 values in the language.
   - having to mix error codes in with valid return values (for int()
 functions).

 We also get:
   - no type proliferation.
   - no arguments about what exception types are needed, or the hierarchy
 to put them in.

 Seems like a win to me.
Then you'll always catch all exceptions. If error code doesn't match you need to rethrow the exception. Or make a language change that allows to catch based on the error code. -- /Jacob Carlborg
Oct 06 2014
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/6/14, 11:13 AM, Jacob Carlborg wrote:
 Then you'll always catch all exceptions. If error code doesn't match you
 need to rethrow the exception. Or make a language change that allows to
 catch based on the error code.
Either solution is fine here because that's the rare case. -- Andrei
Oct 06 2014
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/1/14, 4:00 PM, Sean Kelly wrote:
 On Wednesday, 1 October 2014 at 22:42:27 UTC, Jeremy Powers via
 Digitalmars-d wrote:
 If you see "throws Exception" in java code chances are the code is
 broken,
 same as if you see "catch (Exception" - this tells you nothing about the
 exception that happened, and hence you can do nothing with it. So you
 either swallow (and silently break in many cases) or rethrow (and
 break for things you needn't have).  As mentioned, the standard way to
 avoid this is to have a parent exception type appropriate to the
 abstraction in the API, and throw subtypes in the implementation.
 Among other things, this means you can change the implementation to
 throw different exceptions without breaking any users (who will
 already be catching the parent exception).
....while in Phobos, most of the subtyped exceptions were eliminated a while back in favor of just always throwing Exception.
My recollection is that was only talked about. Anyhow, one thing is clear - as of now there are no clear idioms and successful techniques for handling errors with exceptions (including the use of subtyping). -- Andrei
Oct 01 2014
parent Jacob Carlborg <doob me.com> writes:
On 02/10/14 01:19, Andrei Alexandrescu wrote:

 My recollection is that was only talked about. Anyhow, one thing is
 clear - as of now there are no clear idioms and successful techniques
 for handling errors with exceptions (including the use of subtyping). --
 Andrei
I think most error handling is done with "enforce", this will, by default, throw an instance of Exception. I really, really hate this. I think Exception should be an abstract class or an interface. -- /Jacob Carlborg
Oct 01 2014
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-09-29 17:13, Steven Schveighoffer wrote:

 Is it? I can think of cases where it's programmer error, and cases where
 it's user error.
When would it be a user error? -- /Jacob Carlborg
Oct 01 2014
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/1/14 3:27 PM, Jacob Carlborg wrote:
 On 2014-09-29 17:13, Steven Schveighoffer wrote:

 Is it? I can think of cases where it's programmer error, and cases where
 it's user error.
When would it be a user error?
./progThatExpectsFilename "" -Steve
Oct 01 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 01/10/14 21:57, Steven Schveighoffer wrote:

 ./progThatExpectsFilename ""

 -Steve
It's the developer's responsibility to make sure a value like that never reaches the "File" constructor. That is, the developer of the "progThatExpectsFilename" application that uses "File". Not the developer of "File". Although, I don't see why you shouldn't be able to pass an empty string to "File". You'll just get an exception, "cannot open file ''". This is exactly what happens in Bash: $ echo "asd" > "" -bash: : No such file or directory -- /Jacob Carlborg
Oct 01 2014
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/2/14 2:45 AM, Jacob Carlborg wrote:
 On 01/10/14 21:57, Steven Schveighoffer wrote:

 ./progThatExpectsFilename ""

 -Steve
It's the developer's responsibility to make sure a value like that never reaches the "File" constructor. That is, the developer of the "progThatExpectsFilename" application that uses "File". Not the developer of "File".
Then what is the point of File's constructor throwing an exception? This means, File is checking the filename, and I have to also check the file name.
 Although, I don't see why you shouldn't be able to pass an empty string
 to "File". You'll just get an exception, "cannot open file ''".
Right, that is fine. If you catch the exception and handle the result with a nice message to the user, that is exactly what should happen. If you forget to catch the exception, this is a bug, and the program should crash with an appropriate stack trace. Note 2 things: 1. You should NOT depend on the stack trace/Exception to be your error message. 2. File's ctor has NO IDEA whether throwing an exception is going to be a bug or a handled error condition. I would say, as soon as an exception is thrown and is not caught by user code, for all intents and purposes, it becomes an Error. -Steve
Oct 04 2014
parent Jacob Carlborg <doob me.com> writes:
On 2014-10-04 12:29, Steven Schveighoffer wrote:

 Then what is the point of File's constructor throwing an exception? This
 means, File is checking the filename, and I have to also check the file
 name.
"File" should check if the file exists, can be opened and similar things. These are things that can change from outside your application during a function call between your application and the underlying system. But, if "File" for some reason doesn't accept null as a valid value then that's something the developer of the application that uses "File" is responsible to check. It's not like the value can suddenly change without you knowing it.
 Right, that is fine. If you catch the exception and handle the result
 with a nice message to the user, that is exactly what should happen.

 If you forget to catch the exception, this is a bug, and the program
 should crash with an appropriate stack trace.
Yes, I agree.
 Note 2 things:

 1. You should NOT depend on the stack trace/Exception to be your error
 message.
Agree.
 2. File's ctor has NO IDEA whether throwing an exception is going to be
 a bug or a handled error condition.
Yes, but it's the responsibility of "File" to properly document what exceptions it can throw and during which conditions. Then it's up to the developer that uses "File" to handle these exceptions appropriately.
 I would say, as soon as an exception is thrown and is not caught by user
 code, for all intents and purposes, it becomes an Error.
Sure. In theory you can have an other library that handles these exceptions. Think something like a web framework that turns most exceptions into 500 responses. -- /Jacob Carlborg
Oct 04 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/29/2014 8:13 AM, Steven Schveighoffer wrote:
 I can think of cases where it's programmer error, and cases where it's
 user error.
More carefully design the interfaces if programmer error and input error are conflated.
Oct 04 2014
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/4/14 4:47 AM, Walter Bright wrote:
 On 9/29/2014 8:13 AM, Steven Schveighoffer wrote:
 I can think of cases where it's programmer error, and cases where it's
 user error.
More carefully design the interfaces if programmer error and input error are conflated.
You mean more carefully design File's ctor? How so? -Steve
Oct 04 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/4/2014 3:30 AM, Steven Schveighoffer wrote:
 On 10/4/14 4:47 AM, Walter Bright wrote:
 On 9/29/2014 8:13 AM, Steven Schveighoffer wrote:
 I can think of cases where it's programmer error, and cases where it's
 user error.
More carefully design the interfaces if programmer error and input error are conflated.
You mean more carefully design File's ctor? How so?
You can start with deciding if random binary data passed as a "file name" is legal input to the ctor or not.
Oct 04 2014
next sibling parent =?UTF-8?Q?Tobias=20M=C3=BCller?= <troplin bluewin.ch> writes:
Walter Bright <newshound2 digitalmars.com> wrote:
 On 10/4/2014 3:30 AM, Steven Schveighoffer wrote:
 On 10/4/14 4:47 AM, Walter Bright wrote:
 On 9/29/2014 8:13 AM, Steven Schveighoffer wrote:
 I can think of cases where it's programmer error, and cases where it's
 user error.
More carefully design the interfaces if programmer error and input error are conflated.
You mean more carefully design File's ctor? How so?
You can start with deciding if random binary data passed as a "file name" is legal input to the ctor or not.
I think it helps to see contracts as an informal extension to the type system. Ideally, the function signature would not allow invalid input at all. In practice, that's not always possible and contracts are a less formal way to specify the function signature. But conceptually they are still part of the signature. And of course (as with normal contract-less functions) you are always allowed to provide convenience functions with extended input validation. Those should then be based on the strict version. For example take a constructor for an XML document class. It could take the (unvalidated) file path as string parameter. Or a file (validated that it exists) object. Or a stream (validated that it exists and is already opened). You can provide all three for convenience, but I think it's still good design to provide three _different_ functions. Contracts are not a magical tool to provide you all three variants in one function depending somehow on the needs of the caller. Tobi
Oct 05 2014
prev sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sat, 04 Oct 2014 13:12:43 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 10/4/2014 3:30 AM, Steven Schveighoffer wrote:
 On 10/4/14 4:47 AM, Walter Bright wrote:
 On 9/29/2014 8:13 AM, Steven Schveighoffer wrote:
 I can think of cases where it's programmer error, and cases where it's
 user error.
More carefully design the interfaces if programmer error and input err=
or
 are conflated.
You mean more carefully design File's ctor? How so?
=20 You can start with deciding if random binary data passed as a "file name=
" is=20
 legal input to the ctor or not.
In POSIX speak [1] a file name consisting only of A-Za-z0-9.,- is a "character string" (a portable file name) whereas anything not representable in all locales is just a "string". Locales' charsets are required to be able to represent A-Za-z0-9.,- but may use a different mapping than ASCII for that. Only the slash '/' must have a fixed value of 0x2F. =46rom that I conclude, that File() should open files by ubyte[] exclusively to be POSIX compliant. This is the stuff that's frustrating me much about POSIX. It practically makes it impossible to write correct code. Even Qt and Gtk+ settled for the system locale and UTF-8 respectively as the assumed I/O charset for all file names, although each file system could be mounted in a different charset. E.g. CD-ROMs in ISO charset. Windows does much better by offering Unicode versions on top of the "ANSI" functions. The only fix I see for POSIX is to deprecate all other locales except UTF-8 at some point. [1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag= _03_267 --=20 Marco
Oct 05 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 5 Oct 2014 17:44:31 +0200
Marco Leise via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 From that I conclude, that File() should open files by ubyte[]
 exclusively to be POSIX compliant.
and yet there is currently no way in hell to open file with non-utf8 name with hardcoded name and without ugly hacks with File("..."). and ubyte will not help here, as D has no way to represent non-utf8 string literal without unmaintainable shit like "\xNN". and speaking about utf-8: for strings we at least have a hack, and for shebangs... nothing. nada. =D0=BD=D0=B8=D1=87=D0=B5=D0=B3=D0=BE. locale set= tings? other encodings? who needs that, there CAN'T be non-utf8 shebangs. OS interoperability? it's overhyped, there is only two kinds of OSes: those which are D-compatible and bad. changing your locale to non-utf8 magically turns your OS to bad, D will not try to interoperate with it anymore.
Oct 05 2014
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Sun, 5 Oct 2014 19:04:23 +0300
schrieb ketmar via Digitalmars-d <digitalmars-d puremagic.com>:

 On Sun, 5 Oct 2014 17:44:31 +0200
 Marco Leise via Digitalmars-d <digitalmars-d puremagic.com> wrote:
=20
 From that I conclude, that File() should open files by ubyte[]
 exclusively to be POSIX compliant.
and yet there is currently no way in hell to open file with non-utf8 name with hardcoded name and without ugly hacks with File("..."). and ubyte will not help here, as D has no way to represent non-utf8 string literal without unmaintainable shit like "\xNN". =20 and speaking about utf-8: for strings we at least have a hack, and for shebangs... nothing. nada. =D0=BD=D0=B8=D1=87=D0=B5=D0=B3=D0=BE. locale s=
ettings? other encodings? who
 needs that, there CAN'T be non-utf8 shebangs. OS interoperability? it's
 overhyped, there is only two kinds of OSes: those which are D-compatible
 and bad. changing your locale to non-utf8 magically turns your OS to
 bad, D will not try to interoperate with it anymore.
There comes the day that you have to let Sputnik go and board the ISS. Still I and a others agree with you that Phobos should not assume Unicode locales everywhere, no need to rant. What I find difficult is to define just where in std.stdio the locale transcoding has to happen. Java probably just wraps a stream in a stream in a stream as usual, but in std.stdio it is just a File struct that more or less directly writes to the file descriptor. So my guess is, the transcoding has to happen at an earlier stage. Next, when output is NOT a terminal you typically want to output with no transcoding or set it up depending on your needs. Personally I want transcoding to the system locale when stdout/stderr is a terminal and UTF-8 for everything else (i.e. pipes and files). That would be my defaults. --=20 Marco
Oct 05 2014
prev sibling next sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 27 Sep 2014 16:15:40 -0700
Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 After all, what would you think of a compiler that spewed out
 messages like this:
=20
     > dmd test.d
     test.d(15) Error: missing } thrown from dmd/src/parse.c(283)
=20
 ?
"wow, that's cool! one more pleasing feature!"
Sep 27 2014
prev sibling next sibling parent reply luka8088 <luka8088 owave.net> writes:
On 28.9.2014. 1:15, Walter Bright wrote:
 This issue comes up over and over, in various guises. I feel like
 Yosemite Sam here:

      https://www.youtube.com/watch?v=hBhlQgvHmQ0

 In that vein, Exceptions are for either being able to recover from
 input/environmental errors, or report them to the user of the application.

 When I say "They are NOT for debugging programs", I mean they are NOT
 for debugging programs.

 assert()s and contracts are for debugging programs.

 After all, what would you think of a compiler that spewed out messages
 like this:

     > dmd test.d
     test.d(15) Error: missing } thrown from dmd/src/parse.c(283)

 ?

 See:

      https://issues.dlang.org/show_bug.cgi?id=13543

 As for the programmer wanting to know where the message "missing }" came
 from,
 li
      grep -r dmd/src/*.c "missing }"

 works nicely. I do that sort of thing all the time. It really isn't a
 problem.
We has this issue at work (we are working with php). We outputted a stack trace for both exceptions and asserts but the lines that should be addressed are not always so obvious. I found a solution and it works great for us. All library code is marked appropriately so when stack is outputted it is shadows out (with gray color) all the lines in library code and point out first non-library line from the top of the stack. In 95% of the time it is the line that the programmer should look into. Other 5% is the time when it shows the line where programmer is forwarding a call to the library but turns out to be ok as it turns out to be much more comprehensible than the entire stack. One note worth mentioning is that juniors have much easier time understanding which lines concern them, and from that I can only conclude that such approach is more intuitive. Marking is done on namespace level so it can be easily disabled for entire namespace. I think outputting a stack trace for asserts is a must because of that 5%. And for exceptions I agree completely with your arguments and I think that there is no need for stack. From my experience this has been a good approach and I think is worth considering.
Sep 28 2014
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
luka8088:

 All library code is marked appropriately so when stack is 
 outputted it is shadows out (with gray color) all the lines in 
 library code and point out first non-library line from the top 
 of the stack. In 95% of the time it is the line that the 
 programmer should look into. Other 5% is the time when it shows 
 the line where programmer is forwarding a call to the library 
 but turns out to be ok as it turns out to be much more 
 comprehensible than the entire stack. One note worth mentioning 
 is that juniors have much easier time understanding which lines 
 concern them, and from that I can only conclude that such 
 approach is more intuitive.
This looks like a little enhancement request, to colourize differently the parts of the stack trace of the library code/runtime from the user code.
 And for exceptions I agree completely with your arguments and I 
 think that there is no need for stack.
I think Walter is not suggesting to remove the stack trace for exceptions. Bye, bearophile
Sep 28 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 2:28 AM, bearophile wrote:
 And for exceptions I agree completely with your arguments and I think that
 there is no need for stack.
I think Walter is not suggesting to remove the stack trace for exceptions.
I suggest removal of stack trace for exceptions, but leaving them in for asserts. Asserts are a deliberately designed debugging tool. Exceptions are not.
Sep 28 2014
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/28/14, 10:36 AM, Walter Bright wrote:
 On 9/28/2014 2:28 AM, bearophile wrote:
 And for exceptions I agree completely with your arguments and I think
 that
 there is no need for stack.
I think Walter is not suggesting to remove the stack trace for exceptions.
I suggest removal of stack trace for exceptions, but leaving them in for asserts. Asserts are a deliberately designed debugging tool. Exceptions are not.
I'm fine with that philosophy, with the note that it's customary nowadays to inflict things like the stack trace on the user. -- Andrei
Sep 28 2014
prev sibling next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:

 I suggest removal of stack trace for exceptions, but leaving 
 them in for asserts.
I suggest to keep stack trace for both cases, and improve it with colors :-) Another possibility is to keep the stack trace for exceptions in nonrelease mode only.
 Asserts are a deliberately designed debugging tool. Exceptions 
 are not.
Exceptions are often used to help debugging... We have even allowed exceptions inside D contracts (but I don't know why). Bye, bearophile
Sep 28 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 11:25 AM, bearophile wrote:
 Exceptions are often used to help debugging...
https://www.youtube.com/watch?v=hBhlQgvHmQ0
Sep 28 2014
parent luka8088 <luka8088 owave.net> writes:
On 28.9.2014. 21:32, Walter Bright wrote:
 On 9/28/2014 11:25 AM, bearophile wrote:
 Exceptions are often used to help debugging...
https://www.youtube.com/watch?v=hBhlQgvHmQ0
Example exception messages: Unable to connect to database Invalid argument count Invalid network package format All this messages do not require a stack trace as they do not require code fixes, they indicate an issue outside the program itself. If stack trace is required then assert should have been used instead. Or to better put it: can anyone give an example of exception that would require stack trace?
Sep 28 2014
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-09-28 19:36, Walter Bright wrote:

 I suggest removal of stack trace for exceptions, but leaving them in for
 asserts.
If you don't like the stack track, just wrap the "main" function in a try-catch block, catch all exceptions and print the error message. -- /Jacob Carlborg
Sep 28 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 12:33 PM, Jacob Carlborg wrote:
 On 2014-09-28 19:36, Walter Bright wrote:

 I suggest removal of stack trace for exceptions, but leaving them in for
 asserts.
If you don't like the stack track, just wrap the "main" function in a try-catch block, catch all exceptions and print the error message.
That's what the runtime that calls main() is supposed to do.
Sep 28 2014
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Sun, 28 Sep 2014 13:14:43 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 9/28/2014 12:33 PM, Jacob Carlborg wrote:
 On 2014-09-28 19:36, Walter Bright wrote:

 I suggest removal of stack trace for exceptions, but leaving them in for
 asserts.
If you don't like the stack track, just wrap the "main" function in a try-catch block, catch all exceptions and print the error message.
That's what the runtime that calls main() is supposed to do.
Guys, a druntime flag could settle matters in 10 minutes. But this topic is clearly about the right school of thought. I use contracts to check for logical errors, like when an argument must not be null or a value less than the length of some data structure. I use exceptions to check for invalid input and the return values of external libraries. External libraries can be anything from my own code in the same project to OpenGL from vendor XY. They could error out on valid input (if we leave out of memory aside for now), because of bugs or incorrect assumptions of the implementation. If that happens and all I get is: "Library XY Exception: code 0x13533939 (Invalid argument)." I'm at a loss, where the library might have had a hickup. Did some function internally handle a uint as an int and wrapped around? Maybe with std.logger we will see single line messages on the terminal and multi-line exception traces in the logs (which by default print to stderr as well). And then this discussion can be resolved. -- Marco
Oct 01 2014
prev sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Sunday, 28 September 2014 at 17:36:14 UTC, Walter Bright wrote:
 On 9/28/2014 2:28 AM, bearophile wrote:
 And for exceptions I agree completely with your arguments and 
 I think that
 there is no need for stack.
I think Walter is not suggesting to remove the stack trace for exceptions.
I suggest removal of stack trace for exceptions, but leaving them in for asserts. Asserts are a deliberately designed debugging tool. Exceptions are not.
Fair. So we generate traces for Errors but not Exceptions.
Sep 28 2014
prev sibling next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 9/27/14, 8:15 PM, Walter Bright wrote:
 This issue comes up over and over, in various guises. I feel like
 Yosemite Sam here:

      https://www.youtube.com/watch?v=hBhlQgvHmQ0

 In that vein, Exceptions are for either being able to recover from
 input/environmental errors, or report them to the user of the application.

 When I say "They are NOT for debugging programs", I mean they are NOT
 for debugging programs.

 assert()s and contracts are for debugging programs.
For me, assert is useless. We are developing a language using LLVM as our backend. If you give LLVM something it doesn't like, you get something this: ~~~ Assertion failed: (S1->getType() == S2->getType() && "Cannot create binary operator with two operands of differing type!"), function Create, file Instructions.cpp, line 1850. Abort trap: 6 ~~~ That is what the user gets when there is a bug in the compiler, at least when we are generating invalid LLVM code. And that's one of the good paths, if you compiled LLVM with assertions, because otherwise I guess it's undefined behaviour. What I'd like to do, as a compiler, is to catch those errors and tell the user: "You've found a bug in the app, could you please report it in this URL? Thank you.". We can't: the assert is there and we can't change it. Now, this is when you interface with C++/C code. But inside our language code we always use exceptions so that programmers can choose what to do in case of an error. With assert you loose that possibility. Raising an exception is costly, but that should happen in exceptional cases. Installing an exception handler is cost-free, so I don't see why there is a need for a less powerful construct like assert.
Sep 28 2014
next sibling parent "Xiao Xie" <xiao.xie 163.com> writes:
On Sunday, 28 September 2014 at 15:10:26 UTC, Ary Borenszweig 
wrote:
 What I'd like to do, as a compiler, is to catch those errors 
 and tell the user: "You've found a bug in the app, could you 
 please report it in this URL? Thank you.". We can't: the assert 
 is there and we can't change it.
Why is SIGABRT handler not working for your usecase? print and exit?
Sep 28 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 8:10 AM, Ary Borenszweig wrote:
 For me, assert is useless.

 We are developing a language using LLVM as our backend. If you give LLVM
 something it doesn't like, you get something this:

 ~~~
 Assertion failed: (S1->getType() == S2->getType() && "Cannot create binary
 operator with two operands of differing type!"), function Create, file
 Instructions.cpp, line 1850.

 Abort trap: 6
 ~~~

 That is what the user gets when there is a bug in the compiler, at least when
we
 are generating invalid LLVM code. And that's one of the good paths, if you
 compiled LLVM with assertions, because otherwise I guess it's undefined
behaviour.

 What I'd like to do, as a compiler, is to catch those errors and tell the user:
 "You've found a bug in the app, could you please report it in this URL? Thank
 you.". We can't: the assert is there and we can't change it.
You can hook D's assert and do what you want with it.
 Now, this is when you interface with C++/C code. But inside our language code
we
 always use exceptions so that programmers can choose what to do in case of an
 error. With assert you loose that possibility.
If you want to use Exceptions for debugging in your code, I won't try and stop you. But using them for debugging in official Phobos I strongly object to.
 Installing an exception handler is cost-free,
Take a look at the assembler dump from std.file.copy() that I posted in the other thread.
 so I don't see why there is a need
 for a less powerful construct like assert.
Exceptions are meant for RECOVERABLE errors. If you're using them instead of assert for logic bugs, you're looking at undefined behavior. Logic bugs are not recoverable.
Sep 28 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Sunday, 28 September 2014 at 17:40:49 UTC, Walter Bright wrote:
 You can hook D's assert and do what you want with it.
With the caveat that you must finish by either exiting the app or throwing an exception, since the compiler doesn't generate a stack frame that can be returned from.
 Exceptions are meant for RECOVERABLE errors. If you're using 
 them instead of assert for logic bugs, you're looking at 
 undefined behavior. Logic bugs are not recoverable.
In a multithreaded program, does this mean that the thread must be terminated or the entire process? In a multi-user system, does this mean the transaction or the entire process? The scope of a logic bug can be known to be quite limited. Remember my earlier point about Erlang, where a "process" there is actually just a logical thread in the VM.
Sep 28 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 12:38 PM, Sean Kelly wrote:
 Exceptions are meant for RECOVERABLE errors. If you're using them instead of
 assert for logic bugs, you're looking at undefined behavior. Logic bugs are
 not recoverable.
In a multithreaded program, does this mean that the thread must be terminated or the entire process? In a multi-user system, does this mean the transaction or the entire process? The scope of a logic bug can be known to be quite limited. Remember my earlier point about Erlang, where a "process" there is actually just a logical thread in the VM.
This has been asked many times before. If the threads share memory, the only robust choice is to terminate all the threads and the application. If the thread is in another process, where the memory is not shared, then terminating and possibly restarting that process is quite acceptable.
 The scope of a logic bug can be known to be quite limited.
If you know about the bug, then you'd have fixed it already instead of inserting recovery code for unknown problems. I can't really accept that one has "unknown bugs of known scope".
Sep 28 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Sunday, 28 September 2014 at 20:31:03 UTC, Walter Bright wrote:
 If the threads share memory, the only robust choice is to 
 terminate all the threads and the application.

 If the thread is in another process, where the memory is not 
 shared, then terminating and possibly restarting that process 
 is quite acceptable.

 The scope of a logic bug can be known to be quite limited.
If you know about the bug, then you'd have fixed it already instead of inserting recovery code for unknown problems. I can't really accept that one has "unknown bugs of known scope".
Well, say you're using SafeD or some other system where you know that memory corruption is not possible (pure functional programming, for example). In this case, if you know what data a particular execution flow touches, you know the scope of the potential damage. And if the data touched is all either shared but read-only or generated during the processing of the request, you can be reasonably certain that nothing outside the scope of the transaction has been adversely affected at all.
Sep 28 2014
next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
29-Sep-2014 00:50, Sean Kelly пишет:
 On Sunday, 28 September 2014 at 20:31:03 UTC, Walter Bright wrote:
 If the threads share memory, the only robust choice is to terminate
 all the threads and the application.

 If the thread is in another process, where the memory is not shared,
 then terminating and possibly restarting that process is quite
 acceptable.

 The scope of a logic bug can be known to be quite limited.
If you know about the bug, then you'd have fixed it already instead of inserting recovery code for unknown problems. I can't really accept that one has "unknown bugs of known scope".
Well, say you're using SafeD or some other system where you know that memory corruption is not possible (pure functional programming, for example).
 In this case, if you know what data a particular execution
 flow touches, you know the scope of the potential damage.  And if the
 data touched is all either shared but read-only or generated during the
 processing of the request, you can be reasonably certain that nothing
 outside the scope of the transaction has been adversely affected at all.
not possible / highly unlikely (i.e. bug in VM or said system) But otherwise agreed, dropping the whole process is not always a good idea or it easily becomes a DoS attack vector in a public service. -- Dmitry Olshansky
Sep 28 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Sunday, 28 September 2014 at 21:16:51 UTC, Dmitry Olshansky 
wrote:
 But otherwise agreed, dropping the whole process is not always 
 a good idea or it easily becomes a DoS attack vector in a 
 public service.
What I really want to work towards is the Erlang model where an app is a web of communicating processes (though Erlang processes are effectively equivalent to D objects). Then, killing a process on an error is absolutely correct. It doesn't affect the resilience of the system. But if these processes are actually threads or fibers with memory protection, things get a lot more complicated. I really need to spend some time investigating how modern Linux systems handle tons of processes running on them and try to find a happy medium.
Sep 28 2014
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
29-Sep-2014 01:21, Sean Kelly пишет:
 On Sunday, 28 September 2014 at 21:16:51 UTC, Dmitry Olshansky wrote:
 But otherwise agreed, dropping the whole process is not always a good
 idea or it easily becomes a DoS attack vector in a public service.
What I really want to work towards is the Erlang model where an app is a web of communicating processes (though Erlang processes are effectively equivalent to D objects). Then, killing a process on an error is absolutely correct. It doesn't affect the resilience of the system. But if these processes are actually threads or fibers with memory protection, things get a lot more complicated.
One thing I really appreciated about JVM is exactly the memory safety with ability to handle this pretty much in the same way Erlang does.
 I really need to spend
 some time investigating how modern Linux systems handle tons of
 processes running on them and try to find a happy medium.
Keep us posted. -- Dmitry Olshansky
Sep 28 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 1:50 PM, Sean Kelly wrote:
 On Sunday, 28 September 2014 at 20:31:03 UTC, Walter Bright wrote:
 The scope of a logic bug can be known to be quite limited.
If you know about the bug, then you'd have fixed it already instead of inserting recovery code for unknown problems. I can't really accept that one has "unknown bugs of known scope".
Well, say you're using SafeD or some other system where you know that memory corruption is not possible (pure functional programming, for example). In this case, if you know what data a particular execution flow touches, you know the scope of the potential damage. And if the data touched is all either shared but read-only or generated during the processing of the request, you can be reasonably certain that nothing outside the scope of the transaction has been adversely affected at all.
You may know the error is not a memory corrupting one, but that doesn't mean there aren't non-corrupting changes to the shared memory that would result in additional unexpected failures. Also, the logic bug may be the result of an system part of the code going wrong. You do not know, because YOU DO NOT KNOW the cause the error. And if you knew the cause, you wouldn't need a stack trace to debug it anyway. I.e. despite being 'safe' it does not imply the program is in a predictable or anticipated state. I can't get behind the notion of "reasonably certain". I certainly would not use such techniques in any code that needs to be robust, and we should not be using such cowboy techniques in Phobos nor officially advocate their use.
Sep 28 2014
next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Sunday, 28 September 2014 at 22:00:24 UTC, Walter Bright wrote:
 I can't get behind the notion of "reasonably certain". I 
 certainly would not use such techniques in any code that needs 
 to be robust, and we should not be using such cowboy techniques 
 in Phobos nor officially advocate their use.
I think it's a fair stance not to advocate this approach. But as it is I spend a good portion of my time diagnosing bugs in production systems based entirely on archived log data, and analyzing the potential impact on the system to determine the importance of a hot fix. The industry seems to be moving towards lowering the barrier between engineering and production code (look at what Netflix has done for example), and some of this comes from an isolation model akin to the Erlang approach, but the typical case is still that hot fixing code is incredibly expensive and so you don't want to do it if it isn't necessary. For me, the correct approach may simply be to eschew assert() in favor of enforce() in some cases. But the direction I want to be headed is the one you're encouraging. I simply don't know if it's practical from a performance perspective. This is still developing territory.
Sep 28 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2014 6:17 PM, Sean Kelly wrote:
 On Sunday, 28 September 2014 at 22:00:24 UTC, Walter Bright wrote:
 I can't get behind the notion of "reasonably certain". I certainly would not
 use such techniques in any code that needs to be robust, and we should not be
 using such cowboy techniques in Phobos nor officially advocate their use.
I think it's a fair stance not to advocate this approach. But as it is I spend a good portion of my time diagnosing bugs in production systems based entirely on archived log data, and analyzing the potential impact on the system to determine the importance of a hot fix. The industry seems to be moving towards lowering the barrier between engineering and production code (look at what Netflix has done for example), and some of this comes from an isolation model akin to the Erlang approach, but the typical case is still that hot fixing code is incredibly expensive and so you don't want to do it if it isn't necessary. For me, the correct approach may simply be to eschew assert() in favor of enforce() in some cases. But the direction I want to be headed is the one you're encouraging. I simply don't know if it's practical from a performance perspective. This is still developing territory.
You've clearly got a tough job to do, and I understand you're doing the best you can with it. I know I'm hardcore and uncompromising on this issue, but that's where I came from (the aviation industry). I know what works (airplanes are incredibly safe) and what doesn't work (Toyota's approach was in the news not too long ago). Deepwater Horizon and Fukushima are also prime examples of not dealing properly with modest failures that cascaded into disaster.
Sep 28 2014
parent reply "Kagamin" <spam here.lot> writes:
On Monday, 29 September 2014 at 03:04:11 UTC, Walter Bright wrote:
 You've clearly got a tough job to do, and I understand you're 
 doing the best you can with it. I know I'm hardcore and 
 uncompromising on this issue, but that's where I came from (the 
 aviation industry).

 I know what works (airplanes are incredibly safe) and what 
 doesn't work (Toyota's approach was in the news not too long 
 ago). Deepwater Horizon and Fukushima are also prime examples 
 of not dealing properly with modest failures that cascaded into 
 disaster.
Do you interpret airplane safety right? As I understand, airplanes are safe exactly because they recover from assert failures and continue operation. Your suggestion is when seat 2A creaks, shut down the whole airplane. In reality airplanes continue to operate until there's zero physical resource to operate. Fukushima caused disaster because it didn't try to handle failure. But this is your idea that one can do nothing meaningful on failure, and Fukushima did just that: nothing. Termination of the process is the safe default, especially in the case of client software, but servers should probably terminate failed request, gracefully clean up and continue operation, like airplanes.
Oct 03 2014
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 03/10/14 13:27, Kagamin wrote:

 Do you interpret airplane safety right? As I understand, airplanes are
 safe exactly because they recover from assert failures and continue
 operation. Your suggestion is when seat 2A creaks, shut down the whole
 airplane. In reality airplanes continue to operate until there's zero
 physical resource to operate.
I have no idea of airplane works but I think Walter usual says they have at least three backup systems. If one system fails, shut it down and switch to the backup. -- /Jacob Carlborg
Oct 03 2014
next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Friday, 3 October 2014 at 12:16:30 UTC, Jacob Carlborg wrote:
 On 03/10/14 13:27, Kagamin wrote:

 Do you interpret airplane safety right? As I understand, 
 airplanes are
 safe exactly because they recover from assert failures and 
 continue
 operation. Your suggestion is when seat 2A creaks, shut down 
 the whole airplane. In reality airplanes continue to operate 
 until there's zero physical resource to operate.
I have no idea of airplane works but I think Walter usual says they have at least three backup systems. If one system fails, shut it down and switch to the backup.
My point, and I think Kagamin's as well, is that the entire plane is a system and the redundant internals are subsystems. They may not share memory, but they are wired to the same sensors, servos, displays, etc. Thus the point about shutting down the entire plane as a result of a small failure is fair.
Oct 03 2014
next sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 3 October 2014 at 15:43:59 UTC, Sean Kelly wrote:
 My point, and I think Kagamin's as well, is that the entire 
 plane is a system and the redundant internals are subsystems.  
 They may not share memory, but they are wired to the same 
 sensors, servos, displays, etc.  Thus the point about shutting 
 down the entire plane as a result of a small failure is fair.
An airplane is a bad analogy for a regular server. You have redundant backups everywhere and you are not allowed to take off at the smallest sign of deviation from normal operation. You will never see D in a fighter jet (and you can probably not fly it without the controller in operation either, your only choice is to send the plane into the ocean and escape in a parachute). I think Walter forgets that you ensure integrity of a complex system of servers by utilizing a rock solid proven transaction database/task-scheduler for handling all critical information. If that fails, you probably should shut down everything, roll back to the last backup and reboot. But you don't shut down a restaurant because the waiter forgets to write down an order every once in a while, you shut it down if the kitchen is unsuitable for preparing food. After sanitizing the kitchen you open the restaurant again. You also don't fire the sloppy waiter until you have a better waiter at hand…
Oct 03 2014
next sibling parent reply "Piotrek" <p nonexistent.pl> writes:
On Friday, 3 October 2014 at 16:11:00 UTC, Ola Fosheim Grøstad 
wrote:
 On Friday, 3 October 2014 at 15:43:59 UTC, Sean Kelly wrote:
 My point, and I think Kagamin's as well, is that the entire 
 plane is a system and the redundant internals are subsystems.  
 They may not share memory, but they are wired to the same 
 sensors, servos, displays, etc.  Thus the point about shutting 
 down the entire plane as a result of a small failure is fair.
An airplane is a bad analogy for a regular server. You have redundant backups everywhere and you are not allowed to take off at the smallest sign of deviation from normal operation.
That depends on design (logic). Ever heard of this? http://www.reddit.com/r/programming/comments/1ax0oa/how_kdes_1500_git_repositories_almost_were_lost/
 I think Walter forgets that you ensure integrity of a complex 
 system of servers by utilizing a rock solid proven transaction 
 database/task-scheduler for handling all critical information. 
 If that fails, you probably should shut down everything, roll 
 back to the last backup and reboot.
I agree with Walter wholeheartedly. If I get him correctly he speaks about distinction between the program logic and input errors. Not about recovery strategies/decisions.
 But you don't shut down a restaurant because the waiter forgets 
 to write down an order every once in a while, you shut it down 
 if the kitchen is unsuitable for preparing food. After 
 sanitizing the kitchen you open the restaurant again. You also 
 don't fire the sloppy waiter until you have a better waiter at 
 hand…
Let me play the game of finding analogies ;) IMO, an exception is more suitable for the analogy with waiter and dirty kitchen. A logic error would be a case when you think you are running a garage but suddenly you noticed your stuff is selling meals and is wearing chef's uniforms. Piotrek
Oct 03 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 3 October 2014 at 17:33:33 UTC, Piotrek wrote:
 That depends on design (logic). Ever heard of this?
 http://www.reddit.com/r/programming/comments/1ax0oa/how_kdes_1500_git_repositories_almost_were_lost/
How is not having redundant storage, logging or backup related to this? This is a risk assessment scenario: «What are we willing to loose compared to what it costs to mitigate the risks by investing in additional resources?»
 A logic error would be a case when you think you are running a 
 garage but suddenly you noticed your stuff is selling meals and 
 is wearing chef's uniforms.
But it is a business decision whether it is better to take amazon.com off the network for a week or just let their search engine occasionally serve food instead of books as search results. Not an engineering decision. It is a business decision whether it is better for a game to corrupt 1% of user accounts and let customer support manually build them back up than to take the game off the network until the problem is fixed. You would probably have heavier load on customer support and loose more subscriptions by taking the game off the network than giving those 1% one year of free game play as a compensation. If you have a logic error in a functional routine, it is local. It might not matter, it might be expected. Logic errors do not imply memory corruption. Memory corruption does not imply that exceptions are thrown. Even if memory corruption would lead to exceptions being thrown in 30% of the cases, you'd still have 70% of cases where memory corruption goes undetected. So if that is a concern, you need to focus elsewhere. You have to think about this in probabilistic terms and relate it to business decisions. Defining thresholds for acceptable reliability is not an engineering decision. An engineering decision is to use isolates, Erlang, Haskell etc to achieve the thresholds set as acceptable reliability/quality viewed from a business point of view.
Oct 03 2014
parent reply "Joseph Rushton Wakeling" <joseph.wakeling webdrake.net> writes:
On Friday, 3 October 2014 at 19:05:51 UTC, Ola Fosheim Grøstad 
wrote:
 But it is a business decision whether it is better to take 
 amazon.com off the network for a week or just let their search 
 engine occasionally serve food instead of books as search 
 results. Not an engineering decision.

 It is a business decision whether it is better for a game to 
 corrupt 1% of user accounts and let customer support manually 
 build them back up than to take the game off the network until 
 the problem is fixed. You would probably have heavier load on 
 customer support and loose more subscriptions by taking the 
 game off the network than giving those 1% one year of free game 
 play as a compensation.
The thing is, the privilege to make that kind of business decision is wholly dependent on the fact that there are no meaningful safety issues involved. Compare that to the case of the Ford Pinto. The allegation made was that Ford had preferred to risk paying out lawsuits to injured drivers over fixing a design flaw responsible for those (serious) injuries, because a cost-benefit analysis had shown the payouts were cheaper than rolling out the fix. This allegation was rightly met with outrage, and severe punitive damages in court.
Oct 04 2014
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/4/2014 4:39 AM, Joseph Rushton Wakeling wrote:
 The thing is, the privilege to make that kind of business decision is wholly
 dependent on the fact that there are no meaningful safety issues involved.

 Compare that to the case of the Ford Pinto.  The allegation made was that Ford
 had preferred to risk paying out lawsuits to injured drivers over fixing a
 design flaw responsible for those (serious) injuries, because a cost-benefit
 analysis had shown the payouts were cheaper than rolling out the fix.  This
 allegation was rightly met with outrage, and severe punitive damages in court.
Unfortunately, such business decisions are always made. Nobody can make a 100% safe system, and if one even tried, such a system would be unusable. A car where safety was the overriding priority could not move an inch, nobody could afford to buy one, etc. The best one can do in an imperfect world is set a standard of the maximum probability of a fatal accident. In aviation, this standard is set by regulation, and airframe manufacturers are obliged to prove that the system reliability is greater than that standard, in order to get their designs certified. The debate then is how high can that standard be set and still have affordable, useful products.
Oct 04 2014
prev sibling parent "Ola Fosheim Grostad" <ola.fosheim.grostad+dlang gmail.com> writes:
On Saturday, 4 October 2014 at 11:39:24 UTC, Joseph Rushton 
Wakeling wrote:
 The thing is, the privilege to make that kind of business 
 decision is wholly dependent on the fact that there are no 
 meaningful safety issues involved.
Surgeons can do remote surgery using VR equipment. There are obvious dangerous technical factors there, but you have to weigh that up to getting an emergency operation done by the most exerienced surgeon in the country. So it is a probabilistic calculation. From an absolute safety point of view, you should not do that. The comm link might fail and that could be serious. It is a system that might create extra complications. From a probabilistic point of view it might lead to the highest survial rate and make local surgeons better, so it might be worth the risk when you amortize risk over N operations.
Oct 04 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/3/2014 9:10 AM, "Ola Fosheim Grøstad" 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 I think Walter forgets that you ensure integrity of a complex system of servers
 by utilizing a rock solid proven transaction database/task-scheduler for
 handling all critical information. If that fails, you probably should shut down
 everything, roll back to the last backup and reboot.
You don't ensure integrity of anything by running software after it has entered an unknown and unanticipated state. There's no way you'd bet your life on it.
 rock solid proven
Yeah, right.
 If that fails
"When that fails" FTFY
 I think Walter forgets
I think you forget my background in designing critical flight controls systems. I know what works, and the proof is the incredible safety of airliners. Yeah, I know that's "appeal to authority", but I've backed it up, too.
Oct 04 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Saturday, 4 October 2014 at 08:25:22 UTC, Walter Bright wrote:
 On 10/3/2014 9:10 AM, "Ola Fosheim Grøstad" 
 <ola.fosheim.grostad+dlang gmail.com>" wrote:
 I think Walter forgets that you ensure integrity of a complex 
 system of servers
 by utilizing a rock solid proven transaction 
 database/task-scheduler for
 handling all critical information. If that fails, you probably 
 should shut down
 everything, roll back to the last backup and reboot.
You don't ensure integrity of anything by running software after it has entered an unknown and unanticipated state.
Integrity is ensured by the transaction engine. The world outside of the transaction engine has NO WAY of affecting integrity. D code that is written today belongs outside the transaction engine.
 There's no way you'd bet your life on it.
SAAB Gripen crashed in 1989 and 1993 due to control software, the pilots used their parachutes and sent the plane in a safe direction. Eurofighter is wire controlled, you most likely cannot keep it stable without electronic control. So if it fails, you have to use the parachute. Bye, bye $100.000.000. Anyway, failure should not be due to "asserts", that should be covered by program verification and formal proofs. Failure can still happen if the stabilizing model is inadequate. During peace time fighter jets stay grounded for many days every year due to technical issues, maybe as much as 50%. In war time they would be up fighting… So yes, you bet your life on it when you defend the air base. Your life is worth nothing in certain circumstances. It is contextual.
 I think you forget my background in designing critical flight 
 controls systems. I know what works, and the proof is the 
 incredible safety of airliners. Yeah, I know that's "appeal to 
 authority", but I've backed it up, too.
That's a marginal use scenario and software for critical control systems should not rely on asserts in 2014. Critical software should be formally proven correct.
Oct 04 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/4/2014 1:40 AM, "Ola Fosheim Grøstad" 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 On Saturday, 4 October 2014 at 08:25:22 UTC, Walter Bright wrote:
 On 10/3/2014 9:10 AM, "Ola Fosheim Grøstad"
 <ola.fosheim.grostad+dlang gmail.com>" wrote:
 I think Walter forgets that you ensure integrity of a complex system of servers
 by utilizing a rock solid proven transaction database/task-scheduler for
 handling all critical information. If that fails, you probably should shut down
 everything, roll back to the last backup and reboot.
You don't ensure integrity of anything by running software after it has entered an unknown and unanticipated state.
Integrity is ensured
Sorry, Ola, you've never written bug-free software, and nobody else has, either.
 by the transaction engine. The world outside of the
 transaction engine has NO WAY of affecting integrity.
Hardware fails, too.
 SAAB Gripen crashed in 1989 and 1993 due to control software,
Wikipedia sez these were caused by "pilot induced oscillations". http://en.wikipedia.org/wiki/Accidents_and_incidents_involving_the_JAS_39_Gripen#February_1989 In any case, Fighter aircraft are not built to airliner safety standards.
 Eurofighter is wire
 controlled, you most likely cannot keep it stable without electronic control.
So
 if it fails, you have to use the parachute. Bye, bye $100.000.000.
That doesn't mean there are no backups to the primary flight control computer.
 Anyway, failure should not be due to "asserts", that should be covered by
 program verification and formal proofs.
The assumption that "proof" means the code doesn't have bugs is charming, but still false.
 Failure can still happen if the stabilizing model is inadequate.
It seems we can't escape bugs.
 During peace time fighter jets stay grounded for many days every year due to
 technical issues, maybe as much as 50%. In war time they would be up
fighting…
 So yes, you bet your life on it when you defend the air base. Your life is
worth
 nothing in certain circumstances. It is contextual.
Again, warplanes are not built to airliner safety standards. They have different priorities.
 I think you forget my background in designing critical flight controls
 systems. I know what works, and the proof is the incredible safety of
 airliners. Yeah, I know that's "appeal to authority", but I've backed it up,
too.
That's a marginal use scenario and software for critical control systems should not rely on asserts in 2014. Critical software should be formally proven correct.
Airframe companies are going to continue to rely on things that have a long, successful track record. It's pretty hard to argue with success.
Oct 04 2014
next sibling parent "eles" <eles215 gzk.dot> writes:
On Saturday, 4 October 2014 at 09:40:26 UTC, Walter Bright wrote:
 On 10/4/2014 1:40 AM, "Ola Fosheim Grøstad" 
 <ola.fosheim.grostad+dlang gmail.com>" wrote:
 On Saturday, 4 October 2014 at 08:25:22 UTC, Walter Bright 
 wrote:
 On 10/3/2014 9:10 AM, "Ola Fosheim Grøstad"
 <ola.fosheim.grostad+dlang gmail.com>" wrote:
 In any case, Fighter aircraft are not built to airliner safety 
 standards.
That concerns only the degrees of freedom of pilot's inputs and airplane's angles, but not the standard of the software or other components. It just trusts the pilot for the assetion of the situation. After all a missile is a far more danger than a possible stall.
Oct 04 2014
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Oct 04, 2014 at 02:40:28AM -0700, Walter Bright via Digitalmars-d wrote:
 On 10/4/2014 1:40 AM, "Ola Fosheim Grstad"
 <ola.fosheim.grostad+dlang gmail.com>" wrote:
[...]
Anyway, failure should not be due to "asserts", that should be
covered by program verification and formal proofs.
The assumption that "proof" means the code doesn't have bugs is charming, but still false.
[...] "Beware -- I've only proven that the code is correct, not tested it." -- Donald Knuth. :-) T -- It is not the employer who pays the wages. Employers only handle the money. It is the customer who pays the wages. -- Henry Ford
Oct 04 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/4/2014 7:13 AM, H. S. Teoh via Digitalmars-d wrote:
 "Beware -- I've only proven that the code is correct, not tested it." --
 Donald Knuth.

 :-)
Quotes like that prove (!) what a cool guy Knuth is!
Oct 04 2014
parent reply "Ola Fosheim Grostad" <ola.fosheim.grostad+dlang gmail.com> writes:
On Saturday, 4 October 2014 at 19:49:23 UTC, Walter Bright wrote:
 On 10/4/2014 7:13 AM, H. S. Teoh via Digitalmars-d wrote:
 "Beware -- I've only proven that the code is correct, not 
 tested it." --
 Donald Knuth.

 :-)
Quotes like that prove (!) what a cool guy Knuth is!
Nah, it only proves that he had not tested the code in his memo (and probably not run the proof through a verifier either).
Oct 04 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 10/04/2014 10:37 PM, Ola Fosheim Grostad wrote:
 On Saturday, 4 October 2014 at 19:49:23 UTC, Walter Bright wrote:
 On 10/4/2014 7:13 AM, H. S. Teoh via Digitalmars-d wrote:
 "Beware -- I've only proven that the code is correct, not tested it." --
 Donald Knuth.

 :-)
Quotes like that prove (!) what a cool guy Knuth is!
Nah, it only proves that he had not tested the code in his memo (and probably not run the proof through a verifier either).
It doesn't prove that.
Oct 04 2014
prev sibling parent reply "Ola Fosheim Grostad" <ola.fosheim.grostad+dlang gmail.com> writes:
On Saturday, 4 October 2014 at 09:40:26 UTC, Walter Bright wrote:
 Sorry, Ola, you've never written bug-free software, and nobody 
 else has, either.
I have, but only simple ones. This is also key to making systems robust. Army equipment tend to consist of few parts, robust construction, and as little fancy features as you can get away with. But it us often rather unconvinient and hard to adapt to non-army settings. D is on the other side of the spectrum. Nothing wrong with it, but not really following the principles that would make it suitable for creating simple robust systems.
 by the transaction engine. The world outside of the
 transaction engine has NO WAY of affecting integrity.
Hardware fails, too.
Sure, but the point is to have integrity ensured in a conceptually simple system that has been harnessed at a cost level that exceeds the budget of any single application.
 That doesn't mean there are no backups to the primary flight 
 control computer.
No, but the point is that the operational context matters. Robustness is something you have to reason about on a probabilistic level in relation to the operational context.
 The assumption that "proof" means the code doesn't have bugs is 
 charming, but still false.
A validated correctness proof ensures that the code follows the specification, so no bugs.
 Failure can still happen if the stabilizing model is
inadequate. It seems we can't escape bugs.
An inadeqate specification is not a bug!
 Again, warplanes are not built to airliner safety standards. 
 They have different priorities.
Indeed, so the operational context is what matters, therefore the app should set the priorities, not the language and libraries.
Oct 04 2014
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/04/2014 03:57 PM, Ola Fosheim Grostad wrote:
 A validated correctness proof ensures that the code follows the
 specification, so no bugs.
The day we can guarantee a correctness proof as being perfectly sound and...well...correct is the day we can guarantee software correctness without the formal proof. Proofs are just as subject to mistakes and oversights as actual code. Proofs deal with that via auditing and peer review, but...so does software. It sure as hell doesn't guarantee lack of bugs in software, so what in the world would make anyone think it magically guarantees lack of mistakes in a "proof"? (A "proof" btw, is little more than code which may or may not ever actually get mechanically executed.)
Oct 04 2014
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/04/2014 06:20 PM, Nick Sabalausky wrote:
 On 10/04/2014 03:57 PM, Ola Fosheim Grostad wrote:
 A validated correctness proof ensures that the code follows the
 specification, so no bugs.
The day we can guarantee a correctness proof as being perfectly sound and...well...correct is the day we can guarantee software correctness without the formal proof. Proofs are just as subject to mistakes and oversights as actual code. Proofs deal with that via auditing and peer review, but...so does software. It sure as hell doesn't guarantee lack of bugs in software, so what in the world would make anyone think it magically guarantees lack of mistakes in a "proof"? (A "proof" btw, is little more than code which may or may not ever actually get mechanically executed.)
And the "specification" itself may have flaws as well, so again, there are NO guarantees here whatsoever. The only thing proofs do in an engineering context is decrease the likelihood of problems, just like any other engineering strategy.
Oct 04 2014
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/4/2014 3:24 PM, Nick Sabalausky wrote:
 On 10/04/2014 06:20 PM, Nick Sabalausky wrote:
 The day we can guarantee a correctness proof as being perfectly sound
 and...well...correct is the day we can guarantee software correctness
 without the formal proof.

 Proofs are just as subject to mistakes and oversights as actual code.
 Proofs deal with that via auditing and peer review, but...so does
 software. It sure as hell doesn't guarantee lack of bugs in software, so
 what in the world would make anyone think it magically guarantees lack
 of mistakes in a "proof"? (A "proof" btw, is little more than code which
 may or may not ever actually get mechanically executed.)
And the "specification" itself may have flaws as well, so again, there are NO guarantees here whatsoever. The only thing proofs do in an engineering context is decrease the likelihood of problems, just like any other engineering strategy.
Yup. Proofs are just another tool that helps, not a magic solution.
Oct 04 2014
prev sibling parent reply "Ola Fosheim Grostad" <ola.fosheim.grostad+dlang gmail.com> writes:
On Saturday, 4 October 2014 at 22:24:08 UTC, Nick Sabalausky 
wrote:
 And the "specification" itself may have flaws as well, so 
 again, there are NO guarantees here whatsoever. The only thing 
 proofs do in an engineering context is decrease the likelihood 
 of problems, just like any other engineering strategy.
Machine validated proofs guarantee that there are no bugs in the source code for any reasonable definition of "guarantee". There is no reason for having proper asserts left in the code after that. If the specification the contract is based on is inadequate, then that is not an issue for the contractor. You still implement according to the spec/contract until the contract is changed by the customer. If an architect didn't follow the requirements of the law when drawing a house, then he cannot blame the carpenter for building the house according to the drawings.
Oct 04 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/4/2014 10:24 PM, Ola Fosheim Grostad wrote:
 On Saturday, 4 October 2014 at 22:24:08 UTC, Nick Sabalausky wrote:
 And the "specification" itself may have flaws as well, so again, there are NO
 guarantees here whatsoever. The only thing proofs do in an engineering context
 is decrease the likelihood of problems, just like any other engineering
strategy.
Machine validated proofs guarantee that there are no bugs in the source code for any reasonable definition of "guarantee". There is no reason for having proper asserts left in the code after that. If the specification the contract is based on is inadequate, then that is not an issue for the contractor. You still implement according to the spec/contract until the contract is changed by the customer. If an architect didn't follow the requirements of the law when drawing a house, then he cannot blame the carpenter for building the house according to the drawings.
Carpenters can be liable for building things they know are wrong, regardless of what the spec says.
Oct 04 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Sunday, 5 October 2014 at 06:09:44 UTC, Walter Bright wrote:
 Carpenters can be liable for building things they know are 
 wrong, regardless of what the spec says.
You can be made liable if you don't notify the responsible entity you work for that the solution is unsuitable for the purpose. How it works in my country is that to raise a building you need a qualified entity that is "responsible for the quality of the construction" which could be an architect, a building engineer or a carpenter with some extra certification. They are responsible for ensuring/approving the overall quality. They are required to ensure the quality of the spec/work and are therefore liable vs the customer.
Oct 04 2014
parent reply "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Sunday, 5 October 2014 at 06:55:16 UTC, Ola Fosheim Grøstad 
wrote:
 On Sunday, 5 October 2014 at 06:09:44 UTC, Walter Bright wrote:
 Carpenters can be liable for building things they know are 
 wrong, regardless of what the spec says.
You can be made liable if you don't notify the responsible entity you work for that the solution is unsuitable for the purpose. How it works in my country is that to raise a building you need a qualified entity that is "responsible for the quality of the construction" which could be an architect, a building engineer or a carpenter with some extra certification. They are responsible for ensuring/approving the overall quality. They are required to ensure the quality of the spec/work and are therefore liable vs the customer.
Oh, I think that here in Italy we outperform your country with that, as for sure we are the most bureaucratised country on the hearth. So it happens that we have people "responsible for the quality" for pretty everything, from building houses to doing pizzas. And guess it, here the buildings made by ancient romans are still "up and running", while we have schools building made in the '90 that come down at every earthquake... Guru meditation... ;-P --- /Paolo
Oct 05 2014
next sibling parent "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Sunday, 5 October 2014 at 09:06:45 UTC, Paolo Invernizzi wrote:
 On Sunday, 5 October 2014 at 06:55:16 UTC, Ola Fosheim Grøstad

 Oh, I think that here in Italy we outperform your country with 
 that, as for sure we are the most bureaucratised country on the 
 hearth.
/hearh/earth I must stop writing while doing breakfast *sigh* --- /P
Oct 05 2014
prev sibling next sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Sunday, 5 October 2014 at 09:06:45 UTC, Paolo Invernizzi wrote:
 Oh, I think that here in Italy we outperform your country with 
 that, as for sure we are the most bureaucratised country on the 
 hearth.
Hah! In Norway parents sign evaluation of progress for 6 years old school children every 2 weeks due to a "quality reform". And and all pupils are kept on the same level of progress so that nobody should feel left behind due to "social democracy principles"... In Italy you have Montesorri! Consider yourself lucky! In Norway bureaucracy is a life style and the way of being. In northern Norway the public sector accounts for 38-42% of all employees. In Norway the cost of living is so high that starting a business is a risky proposition unless the public sector is your customer base. :^)
Oct 05 2014
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/05/2014 05:35 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 On Sunday, 5 October 2014 at 09:06:45 UTC, Paolo Invernizzi wrote:
 Oh, I think that here in Italy we outperform your country with that,
 as for sure we are the most bureaucratised country on the hearth.
Hah! In Norway parents sign evaluation of progress for 6 years old school children every 2 weeks due to a "quality reform". And and all pupils are kept on the same level of progress so that nobody should feel left behind due to "social democracy principles"...
Aside from those 2 week evals (ouch!), the US isn't a whole lot different. US schools are still notoriously bureaucracy-heavy (just ask any school employee), and "No child left behind" is a big thing (at least, supposedly) while any advanced kids are capped at the level of the rest of their age group and forbidden from advancing at their own level (thus boring the shit out of them and seeding quite a few additional problems). Partly, that level-capping is done because there's a prevalent (but obviously BS) belief that kids should be kept with others of the same age, rather than with others of the same level of development or even a healthy mix. But also, they call this capping of advanced students "Being fair to the *other* kids". Obviously US teachers have no idea what the word "fair" actually means. But then, in my experience, there's a LOT that US teachers don't know. I blame both the teacher's unions (that's not intended as a statement on unions in general, BTW) and the complete and total lack of "logic" being part of the curriculum *they* were taught as kids (which is still inexcusably absent from modern curriculums).
 In Italy you have
 Montesorri! Consider yourself lucky!
US has a few of those too. They're constantly ridiculed (leave it to the US to blast anything that isn't group-think-compatible), but from what I've seen Montesorri's are at least less god-awful than US public schools. I almost went to one (but backed out since, by that point, it would have only been for one year - actually wound up with one of the best teachers I ever had that year, so it worked out fine in the end).
Oct 05 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Sunday, 5 October 2014 at 20:37:18 UTC, Nick Sabalausky wrote:
 forbidden from advancing at their own level (thus boring the 
 shit out of them and seeding quite a few additional problems).
Yeah, I've always wondered if the motivation for keeping kids locked up in schools is to prevent them from doing mischief in the streets and teach them to accept hours of pointless boredom so that they will eventually accept doing boring pointless stuff when they later join the workforce as bureaucrats. :-/
 Obviously US teachers have no idea what the word "fair" 
 actually means. But then, in my experience, there's a LOT that 
 US teachers don't know.
The status of teaching has gone downhill. It used to be viewed as highly skilled work, then you got some idealistic teachers of the 1968-generation, but after the yuppie period in the 80s it has become the easy path to higher-education when you don't know what you want to do except that you aren't good with theory and you want to work with people. I think the same can be said about psychology, the field you enter if you don't know what to study and find science too difficult… (or am I being unfair?)
 statement on unions in general, BTW) and the complete and total 
 lack of "logic" being part of the curriculum *they* were taught 
 as kids (which is still inexcusably absent from modern 
 curriculums).
"logic" is theory. Theory does not belong in schools. Too difficult. You are only supposed to learn things that you don't have to figure out, otherwise finding qualified teachers will become impossible.
 US has a few of those too. They're constantly ridiculed (leave 
 it to the US to blast anything that isn't 
 group-think-compatible), but from what I've seen Montesorri's 
 are at least less god-awful than US public schools.
We have a few Montesorri and a few more Rudolph Steiner schools. Which I believe are better at motivating kids and at least not killing the fun of learning. Learning that figuring things out and doing things your own way is fun is a very important lesson. The government says that the public school system has changed and is kind of incorporating those methodologies too, but that is not what I see. And how can a single teacher handle 27 kids in a class, with only 2 minutes per kid per hour and a rigid plan for what you are supposed to teach? Btw, did you know that the majority in Norway voted against joining EU, so we did not, but since the majority of politicians were pro EU they got us in the backdoor by signing away our freedom in treaties instead? And even when the treaties do not apply we "harmonize our laws to EU" because we "just have to". So thanks to bureaucratic maneuvers we are now 99% EU regulated, have shittier consumer laws than we used to, have next to no control on the borders and are flooded by romanian beggars and criminals from every corner of Europe and beyond, in return we get no vote in the EU decision making process since we are "independent"… You gotta love democracy… On a positive note: the IOC managed to demand that the norwegian King ought to hold a party for the IOC leaders and additionally demanded that he should pay their drinks. It was part of their 7000 page olympics qualification requirements document. It is so heavily regulated that it explicitly specifies that the personnel in the hotels MUST SMILE to the IOC leaders when they arrive. I kid you not, even games and past times are heavily bureaucratic down to minuscule details these days. So, due pressure from the newspapers/grassroots and the royal insult the politicians eventually had to turn down the ~$10.000.000.000 winter olympics budget proposal. Good riddance. Live monarchy! I am personally looking forward to Beijing hosting the winter olympics 2022. I am sure they will mange to fake a smile after the politicians have demolished their homes to make space for the ski-jumping event. In the meantime Norway should not even think about hosting any sporting event until we can avoid being ranked as the worlds most expensive country: http://www.numbeo.com/cost-of-living/rankings_by_country.jsp 'Cause, we all know that a $10 billion budget turns into at least a $30 billion budget before the games are over. That would have been $6000 per capita. I'd say that royal insult paid off. Was this off-topic?
Oct 06 2014
next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/06/2014 07:06 PM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 On Sunday, 5 October 2014 at 20:37:18 UTC, Nick Sabalausky wrote:
 statement on unions in general, BTW) and the complete and total lack
 of "logic" being part of the curriculum *they* were taught as kids
 (which is still inexcusably absent from modern curriculums).
"logic" is theory. Theory does not belong in schools. Too difficult. You are only supposed to learn things that you don't have to figure out, otherwise finding qualified teachers will become impossible.
Math is theory too. But regardless: Yes, there *is* a theoretical side to logic, but logic is also *extremely* applicable to ordinary everyday life. Even moreso than math, I would argue. Now, I don't necessarily mean things like formal symbolic logic, lambda calculus. Although, my 9th grade math class *did* have a very heavy focus on formal proofs - and it wasn't even remotely one of the hardest math classes I'd taken, even just up to that point. Students can handle theory just fine as long as it isn't the more advanced/complex stuff...Although college students should be *expected* to be capable of handling even that. Now, *cutting edge* theory? Sure, leave that for grad students and independent study. Anyway, when I say "teach logic in schools" I just mean (at the very least) the basic things: Like recognizing and identifying the basic logical fallacies (no need necessarily to dive into the actual latin names - the names aren't nearly as crucial as understanding the concepts themselves), recognizing ambiguity, understanding *why* the fallacies and ambiguity are flaws, and the problems and absurdities that can occur when such things aren't noticed and avoided. This is VERY simple, and crucial, stuff. And yet I see SOOO many grown adults, even ones with advanced graduate degrees, consistently fail completely and uttery at basic logical reasoning in everyday life (and we're talking very, very obvious and basic fallacies), that it's genuinely disturbing.
 I am personally looking forward to Beijing hosting the winter olympics
 2022. I am sure they will mange to fake a smile after the politicians
 have demolished their homes to make space for the ski-jumping event.
Don't know whether this has always been the case and just never got noticed until recent years, but between the last winter olympics and the recent soccer/football match, and what you're saying about 2022, I'm noticing a rather bad trend with these big international sporting events. I get the feeling this'll be something that'll get bigger and bigger until either A. the right people get together and do something about it, or B. things come to a head and the shit *really* starts to hit the fan. (Yes, I like outdated slang ;) ) Nothing good can come from the current trajectory.
 Was this off-topic?
It was off-topic several posts up. :)
Oct 07 2014
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On 10/7/2014 5:19 PM, Nick Sabalausky wrote:

 Anyway, when I say "teach logic in schools" I just mean (at the very
 least) the basic things: Like recognizing and identifying the basic
 logical fallacies (no need necessarily to dive into the actual latin
 names - the names aren't nearly as crucial as understanding the concepts
 themselves), recognizing ambiguity, understanding *why* the fallacies
 and ambiguity are flaws, and the problems and absurdities that can occur
 when such things aren't noticed and avoided.
In other words, critical thinking. This is something that, at least in America, is not at all part of the primary school experience.
 This is VERY simple, and crucial, stuff. And yet I see SOOO many grown
 adults, even ones with advanced graduate degrees, consistently fail
 completely and uttery at basic logical reasoning in everyday life (and
 we're talking very, very obvious and basic fallacies), that it's
 genuinely disturbing.


I've personally seen two university courses offered under different guises that try to correct this problem. One is called "Introduction to Mathematical Thinking" and is taught by Keith Devlin at Stanford. The other is called "Think Again: How to Reason and Argue", headed by alter Sinnott-Armstrong at Duke. Despite the disparity in the course titles and the very different approaches taken by the instructors, the content is directed at the same goal -- pushing students to get past their cognitive biases and critically and logically examine any data presented to them. Sadly, American culture seems to increasingly encourage the opposite of critical thinking. It has almost become a badge of honor among some (rather large) circles to embrace a form of willful ignorance rooted in rejecting logic and hard, cold data in favor of falling victim to confirmation bias. --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com
Oct 07 2014
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/07/2014 04:55 AM, Mike Parker wrote:
 On 10/7/2014 5:19 PM, Nick Sabalausky wrote:

 Anyway, when I say "teach logic in schools" I just mean (at the very
 least) the basic things: Like recognizing and identifying the basic
 logical fallacies (no need necessarily to dive into the actual latin
 names - the names aren't nearly as crucial as understanding the concepts
 themselves), recognizing ambiguity, understanding *why* the fallacies
 and ambiguity are flaws, and the problems and absurdities that can occur
 when such things aren't noticed and avoided.
In other words, critical thinking. This is something that, at least in America, is not at all part of the primary school experience.
Pretty much, yea. In all my years of schooling, I only had one class that actually covered any of that stuff (as an actual stated topic anyway, rather than just as an implied part of another topic): It wasn't until college, *and* it was just an elective. Formal Logic, IIRC, or something along those lines, from the Philosophy dept (aiui, logic *is* considered a branch of philosophy, at least historically. Which does make sense IMO). It was actually a good course (not much new to me though since I was already neck-deep in programming, which basically *IS* applied logic. But it's one of the few courses I've ever actually been impressed with.) The downside of the course, though: Ever since I took it I've been ashamed at society for placing such incredibly minimal emphasis on something so crucially fundamental and important. :/ Just something that make me scream in by head "Yes! Everybody needs to know this!!!"
 This is VERY simple, and crucial, stuff. And yet I see SOOO many grown
 adults, even ones with advanced graduate degrees, consistently fail
 completely and uttery at basic logical reasoning in everyday life (and
 we're talking very, very obvious and basic fallacies), that it's
 genuinely disturbing.


I've personally seen two university courses offered under different guises that try to correct this problem. One is called "Introduction to Mathematical Thinking" and is taught by Keith Devlin at Stanford. The other is called "Think Again: How to Reason and Argue", headed by alter Sinnott-Armstrong at Duke. Despite the disparity in the course titles and the very different approaches taken by the instructors, the content is directed at the same goal -- pushing students to get past their cognitive biases and critically and logically examine any data presented to them.
Personally, I think that not presenting it *as* logic may be somewhat of a mistake. Makes it sounds almost like some self-help or management seminar or something. Less respectable-sounding, and obscures the true core nature of the material: logic. But then again, MANY people seem to be repelled by any mention of logic, whereas I've aways been attracted to it, so maybe that's just my own bias.
 Sadly, American culture seems to increasingly encourage the opposite of
 critical thinking. It has almost become a badge of honor among some
 (rather large) circles to embrace a form of willful ignorance rooted in
 rejecting logic and hard, cold data in favor of falling victim to
 confirmation bias.
Unfortunate, yes. Of course, there have *always* been things that have quite blatantly encouraged people to deliberately *not* think, reason, or question assumptions. So it's naturally not limited to just a modern american culture thing, FWIW.
Oct 07 2014
next sibling parent Mike Parker <aldacron gmail.com> writes:
On 10/7/2014 7:52 PM, Nick Sabalausky wrote:
 On 10/07/2014 04:55 AM, Mike Parker wrote:
 On 10/7/2014 5:19 PM, Nick Sabalausky wrote:
 In all my years of schooling, I only had one class that actually covered
 any of that stuff (as an actual stated topic anyway, rather than just as
 an implied part of another topic): It wasn't until college, *and* it was
 just an elective. Formal Logic, IIRC, or something along those lines,
 from the Philosophy dept (aiui, logic *is* considered a branch of
 philosophy, at least historically. Which does make sense IMO).
Yeah, my "Introduction to Logic" was branded PHIL 170 (I only know that because I just happened to find the syllabus in the front of my copy of Copi & Cohen -- the only book from college somehow managed to hold on to).

 Personally, I think that not presenting it *as* logic may be somewhat of
 a mistake. Makes it sounds almost like some self-help or management
 seminar or something. Less respectable-sounding, and obscures the true
 core nature of the material: logic.

 But then again, MANY people seem to be repelled by any mention of logic,
 whereas I've aways been attracted to it, so maybe that's just my own bias.
Formal logic (which is what is typically taught in courses with "Logic" in the title) and critical thinking aren't quite the same thing, though. Logic as a means of probing arguments is just one tool in the critical thinker's toolbox. A lot of people can pick up how to draw a Venn diagram or call out a fallacy, but they fall short in their ability to actually understand where an opposing argument is coming from, or in expressing themselves in a way appropriate for their target audience, or in getting past their own biases. That takes a lot of hard work that isn't solved by mapping out truth tables. --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com
Oct 07 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/7/2014 3:52 AM, Nick Sabalausky wrote:
 But then again, MANY people seem to be repelled by any mention of logic,
whereas
 I've aways been attracted to it, so maybe that's just my own bias.
Amusingly, Mr. Spock was the most illogical member of the crew.
Oct 07 2014
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/07/2014 03:39 PM, Walter Bright wrote:
 On 10/7/2014 3:52 AM, Nick Sabalausky wrote:
 But then again, MANY people seem to be repelled by any mention of
 logic, whereas
 I've aways been attracted to it, so maybe that's just my own bias.
Amusingly, Mr. Spock was the most illogical member of the crew.
There was one episode in particular that really bugged me with that. Left me thinking "Uhh, I'm not sure the writer actually understood logic very well": Spock and a few crew members were stranded in a shuttle with seemingly no chance for rescue except for a one-in-a-million longshot. Spock objected to actually doing the longshot because he'd inexplicably decided that its chances were really zero (even though logic would *really* dictate the chances were merely very small - which was obviously an improvement, albeit minor, over the "do nothing and guarantee lack of rescue" approach Spock was bizarrely in favor of). Oddly, the episode was clearly *trying* to tell people "logic isn't always right, use your gut"...which was interesting because, uhh, the author's logic wasn't right ;) Left me with a very big "Wait...WTF?!?" That episode's been bugging me ever since!
Oct 07 2014
prev sibling next sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 7 October 2014 at 08:19:15 UTC, Nick Sabalausky wrote:
 But regardless: Yes, there *is* a theoretical side to logic, 
 but logic is also *extremely* applicable to ordinary everyday 
 life. Even moreso than math, I would argue.
Yep, however what the human brain is really bad at is reasoning about probability. I agree that primary school should cover modus ponens, modus tollens and how you can define equivalance in terms of two implications. BUT I think you also need to experiment informally with probability at the same time and experience how intuition clouds our thinking. It is important to avoid the fallacies of black/white reasoning that comes with propositional logic. Actually, one probably should start with teaching "ad hoc" object-oriented modelling in primary schools. Turning what humans are really good at, abstraction, into something structured and visual. That way you also learn that when you argue a point you are biased, you always model certain limited projections of the relations that are present in real world.
 up to that point. Students can handle theory just fine as long 
 as it isn't the more advanced/complex stuff...Although college 
 students should be *expected* to be capable of handling even 
 that. Now, *cutting edge* theory? Sure, leave that for grad 
 students and independent study.
Educational research shows that students can handle theory much better if it they view it as useful. Students have gone from being very bad at math, to doing fine when it was applied to something they cared about (like building something, or predicting the outcome of soccer matches). Internalized motivation is really the key to progress in school, which is why the top-down fixed curriculum approach is underperforming compared to the enormous potential kids have. They are really good at learning stuff they find fun (like games).
 This is VERY simple, and crucial, stuff. And yet I see SOOO 
 many grown adults, even ones with advanced graduate degrees, 
 consistently fail completely and uttery at basic logical 
 reasoning in everyday life (and we're talking very, very 
 obvious and basic fallacies), that it's genuinely disturbing.
Yes, social factors are more important in the real world than optimal decision making, unless you build something that can fall apart in a spectacular way that makes it to the front page of the newspapers. :-)
 big international sporting events. I get the feeling this'll be 
 something that'll get bigger and bigger until either A. the 
 right people get together and do something about it, or B. 
 things come to a head and the shit *really* starts to hit the 
 fan. (Yes, I like outdated slang ;) ) Nothing good can come 
 from the current trajectory.
Yeah, I think the trajectory will keep going upwards until there are no more less-democratic countries willing to pay the price to look civilized. It is probably also the result of it being increasingly hard to be heard in the increased and interactive information flow of media, so being big and loud is viewed as a necessity. The Internet makes it much easier to escape from the events, in the 80s the olympics would be on all media surfaces. I barely noticed the last winter olympics despite the 20 billion price tag.
 Was this off-topic?
It was off-topic several posts up. :)
At some point the forum will split into a developer-section and a end-user-section. It is kind of inevitable :). The current confusion about the roles of developer vs end-user is kind of interesting. Maybe it is a positive thing. Not sure. :-)
Oct 07 2014
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/07/2014 06:47 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 On Tuesday, 7 October 2014 at 08:19:15 UTC, Nick Sabalausky wrote:
 But regardless: Yes, there *is* a theoretical side to logic, but logic
 is also *extremely* applicable to ordinary everyday life. Even moreso
 than math, I would argue.
Yep, however what the human brain is really bad at is reasoning about probability.
Yea, true. Probability can be surprisingly unintuitive even for people well-versed in logic. Ex: A lot of people have trouble understanding that getting "heads" in a coinflip many times in a row does *not* increase the likelihood of the next flip being "tails". And there's a very understandable reason why that's difficult to grasp. I've managed to grok it, but yet even I (try as I may) just cannot truly grok the monty hall problem. I *can* reliably come up with the correct answer, but *never* through an actual mental model of the problem, *only* by very, very carefully thinking through each step of the problem. And that never changes no matter how many times I think it through. That really impressed me about the one student depicted in the "21" movie (the one based around the real-life people who created card counting): Don't know how much of it was hollywood artistic license, but when he demonstrated a crystal-clear *intuitive* understanding of the monty hall problem - that was *impressive*.
 I agree that primary school should cover modus ponens,
 modus tollens and how you can define equivalance in terms of two
 implications. BUT I think you also need to experiment informally with
 probability at the same time and experience how intuition clouds our
 thinking. It is important to avoid the fallacies of black/white
 reasoning that comes with propositional logic.

 Actually, one probably should start with teaching "ad hoc"
 object-oriented modelling in primary schools. Turning what humans are
 really good at, abstraction, into something structured and visual. That
 way you also learn that when you argue a point you are biased, you
 always model certain limited projections of the relations that are
 present in real world.
Interesting points, I hadn't thought of any of that.
 Educational research shows that students can handle theory much better
 if it they view it as useful. Students have gone from being very bad at
 math, to doing fine when it was applied to something they cared about
 (like building something, or predicting the outcome of soccer matches).
Yea, definitely. Self-intimidation has a lot to do with it too. I've talked to several math teachers who say they've had very good success teaching algebra to students who struggled with it *just* by replacing the letter-based variables with empty squares. People are very good at intimidating themselves into refusing to even think. It's not just students, it's people in general, heck I've seen both my parents do it quite a bit: "Nick! Something popped up on my screen! I don't know what to do!!" "What does it say?" "I dunno! I didn't read it!! How do I get rid of it?!?" /facepalm
 Internalized motivation is really the key to progress in school,
This is something I've always felt needed to be, as mandatory, drilled into the heads of every educator. Required knowledge for educators, IMO. Think like "gold stars" are among the worst things you can do. It really drives the point home that it's all tedium and has no *inherent* value. Of course, in the classroom, most of it usually *is* tedium with little inherent value...
 which
 is why the top-down fixed curriculum approach is underperforming
 compared to the enormous potential kids have. They are really good at
 learning stuff they find fun (like games).
Yea, and that really proves just how bad the current approach is. Something I think is an appropriate metaphor for that (and bear with me on this): Are you familiar with the sitcom "It's always sunny in Philadelphia"? Created by a group of young writers/actors who were just getting their start. After the first season, it had impressed Danny DiVito enough (apparently he was a fan of the show) that he joined the cast. In an interview with one of the shows creators (on the Season 1&2 DVDs), this co-creator talked about how star-struck they were about having Danny DiVito on board, and how insecure/panicked he was about writing for DiVito...until he realized: (His words, more or less) "Wait a minute, this is *Danny DiVito* - If we can't make **him** funny, then we really suck!" A school that has trouble teaching kids is like a writer who can't make Danny DiVito funny. Learning is what kids *do*! How much failure does it take to mess *that* up? "Those who make a distinction between education and entertainment don't know the first thing about either."
 Yes, social factors are more important in the real world than optimal
 decision making,
I was quite disillusioned when I finally discovered that as an adult. Intelligence, knowledge and ability don't count for shit 90+% of the time (in fact, frequently it's a liability - people *expect* group-think and get very agitated and self-righteous when you don't conform to group-think). Intelligence/knowledge/ability *should* matter a great deal, and people *think* they do. But they don't.
 unless you build something that can fall apart in a
 spectacular way that makes it to the front page of the newspapers. :-)
I've noticed that people refuse to recognize (let alone fix) problems, even when directly pointed out, until people start dying. And even then it's kind of a crapshoot as to whether or not anything will actually be done.
Oct 07 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/08/2014 12:10 AM, Nick Sabalausky wrote:
 On 10/07/2014 06:47 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?=
 <ola.fosheim.grostad+dlang gmail.com>" wrote:
 On Tuesday, 7 October 2014 at 08:19:15 UTC, Nick Sabalausky wrote:
 But regardless: Yes, there *is* a theoretical side to logic, but logic
 is also *extremely* applicable to ordinary everyday life. Even moreso
 than math, I would argue.
Yep, however what the human brain is really bad at is reasoning about probability.
Yea, true. Probability can be surprisingly unintuitive even for people well-versed in logic. ...
Really?
 Ex: A lot of people have trouble understanding that getting "heads" in a
 coinflip many times in a row does *not* increase the likelihood of the
 next flip being "tails". And there's a very understandable reason why
 that's difficult to grasp.
What is this reason? It would be really spooky if the probability was actually increased in this way. You could win at 'heads or tails' by flipping a coin really many times until you got a sufficiently long run of 'tails', then going to another room and betting that the next flip will be 'heads', and if people didn't intuitively understand that, some would actually try to apply this trick. (Do they?)
 I've managed to grok it, but yet even I (try
 as I may) just cannot truly grok the monty hall problem. I *can*
 reliably come up with the correct answer, but *never* through an actual
 mental model of the problem, *only* by very, very carefully thinking
 through each step of the problem. And that never changes no matter how
 many times I think it through.
It is actually entirely straightforward, but it is popular to present the problem as if it was actually really complicated, and those who like to present it often seem to understand it poorly as well. The stage is usually set up to maximise entertainment, not understanding. The presenter is often trying to impress, by forcing a quick answer, hoping that you will not think at all and get it wrong. Sometimes, the context is even set up that such a quick shot is more likely to be wrong, because of an intended wrong analogy to some other completely obvious question that came just before. Carefully thinking it through step by step multiple times afterwards tends to only serve to confuse oneself into strengthening the belief that something counter-intuitive is going on, and this is aggravated by the fact that there isn't, because therefore the part that is supposedly counter-intuitive can never be pinned down. I.e. I think it is confusing because one approaches the problem with a wrong set of assumptions. That said, it's just: When you first randomly choose the door, you would intuitively rather bet that you guessed wrong. The show master is simply proposing to tell you behind which of the other doors the car is in case you indeed guessed wrong. There's not more to it.
 I agree that primary school should cover modus ponens,
 modus tollens and how you can define equivalance in terms of two
 implications. BUT I think you also need to experiment informally with
 probability at the same time and experience how intuition clouds our
 thinking. It is important to avoid the fallacies of black/white
 reasoning that comes with propositional logic.

 Actually, one probably should start with teaching "ad hoc"
 object-oriented modelling in primary schools. Turning what humans are
 really good at, abstraction, into something structured and visual. That
 way you also learn that when you argue a point you are biased, you
 always model certain limited projections of the relations that are
 present in real world.
Interesting points, I hadn't thought of any of that. ...
I mostly agree, except I wouldn't go object-oriented, but do something else, because it tends to quickly fail at actually capturing relations that are present in the real world in a straightforward fashion.
 Educational research shows that students can handle theory much better
 if it they view it as useful. Students have gone from being very bad at
 math, to doing fine when it was applied to something they cared about
 (like building something, or predicting the outcome of soccer matches).
Yea, definitely. Self-intimidation has a lot to do with it too. I've talked to several math teachers who say they've had very good success teaching algebra to students who struggled with it *just* by replacing the letter-based variables with empty squares. People are very good at intimidating themselves into refusing to even think. It's not just students, it's people in general, heck I've seen both my parents do it quite a bit: "Nick! Something popped up on my screen! I don't know what to do!!" "What does it say?" "I dunno! I didn't read it!! How do I get rid of it?!?" /facepalm
Sounds familiar. I've last run into this on e.g. category theory (especially monads) and the monty hall problem. :-P In fact, I only now realised that those two seem to be rather related phenomena. Thanks!
Oct 07 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On 10/08/2014 12:10 AM, Nick Sabalausky wrote:
[...]
 I've managed to grok it, but yet even I (try as I may) just cannot
 truly grok the monty hall problem. I *can* reliably come up with the
 correct answer, but *never* through an actual mental model of the
 problem, *only* by very, very carefully thinking through each step of
 the problem. And that never changes no matter how many times I think
 it through.
[...] The secret behind the monty hall scenario, is that the host is actually leaking extra information to you about where the car might be. You make a first choice, which has 1/3 chance of being right, then the host opens another door, which is *always* wrong. This last part is where the information leak comes from. The host's choice is *not* fully random, because if your initial choice was the wrong door, then he is *forced* to pick the other wrong door (because he never opens the right door, for obvious reasons), thereby indirectly revealing which is the right door. So we have: 1/3 chance: you picked the right door. Then the host can randomly choose between the 2 remaining doors. In this case, no extra info is revealed. 2/3 chance: you picked the wrong door, and the host has no choice but to pick the other wrong door, thereby indirectly revealing the right door. So if you stick with your initial choice, you have 1/3 chance of winning, but if you switch, you have 2/3 chance of winning, because if your initial choice was wrong, which is 2/3 of the time, the host is effectively leaking out the right answer to you. The supposedly counterintuitive part comes from wrongly assuming that the host has full freedom to pick which door to open, which he does not in the given scenario. Of course, this scenario is also often told in a deliberately misleading way -- the fact that the host *never* opens the right door is often left as an unstated "common sense" assumption, thereby increasing the likelihood that people will overlook this minor but important detail. T -- Written on the window of a clothing store: No shirt, no shoes, no service.
Oct 07 2014
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/08/2014 02:37 AM, H. S. Teoh via Digitalmars-d wrote:
 On 10/08/2014 12:10 AM, Nick Sabalausky wrote:
 [...]
 I've managed to grok it, but yet even I (try as I may) just cannot
 truly grok the monty hall problem. I *can* reliably come up with the
 correct answer, but *never* through an actual mental model of the
 problem, *only* by very, very carefully thinking through each step of
 the problem. And that never changes no matter how many times I think
 it through.
[...] The secret behind the monty hall scenario, is that the host is actually leaking extra information to you about where the car might be. You make a first choice, which has 1/3 chance of being right, then the host opens another door, which is *always* wrong. This last part is where the information leak comes from. The host's choice is *not* fully random, because if your initial choice was the wrong door, then he is *forced* to pick the other wrong door (because he never opens the right door, for obvious reasons), thereby indirectly revealing which is the right door. So we have: 1/3 chance: you picked the right door. Then the host can randomly choose between the 2 remaining doors. In this case, no extra info is revealed. 2/3 chance: you picked the wrong door, and the host has no choice but to pick the other wrong door, thereby indirectly revealing the right door. So if you stick with your initial choice, you have 1/3 chance of winning, but if you switch, you have 2/3 chance of winning, because if your initial choice was wrong, which is 2/3 of the time, the host is effectively leaking out the right answer to you. The supposedly counterintuitive part comes from wrongly assuming that the host has full freedom to pick which door to open, which he does not in the given scenario. Of course, this scenario is also often told in a deliberately misleading way -- the fact that the host *never* opens the right door is often left as an unstated "common sense" assumption, thereby increasing the likelihood that people will overlook this minor but important detail. T
The problem with this explanation is simply that it is too long and calls the overly detailed reasoning a 'secret'. :o) It's like monad tutorials!
Oct 07 2014
parent reply "Dominikus Dittes Scherkl" writes:
On Wednesday, 8 October 2014 at 01:22:49 UTC, Timon Gehr wrote:
 The secret behind the monty hall scenario, is that the host is 
 actually
 leaking extra information to you about where the car might be.

 You make a first choice, which has 1/3 chance of being right, 
 then the
 host opens another door, which is *always* wrong. This last 
 part is
 where the information leak comes from.  The host's choice is 
 *not* fully
 random, because if your initial choice was the wrong door, 
 then he is
 *forced* to pick the other wrong door (because he never opens 
 the right
 door, for obvious reasons), thereby indirectly revealing which 
 is the
 right door.  So we have:

 1/3 chance: you picked the right door. Then the host can 
 randomly choose
 	between the 2 remaining doors. In this case, no extra info is
 	revealed.

 2/3 chance: you picked the wrong door, and the host has no 
 choice but to
 	pick the other wrong door, thereby indirectly revealing the
 	right door.

 So if you stick with your initial choice, you have 1/3 chance 
 of
 winning, but if you switch, you have 2/3 chance of winning, 
 because if
 your initial choice was wrong, which is 2/3 of the time, the 
 host is
 effectively leaking out the right answer to you.

 The supposedly counterintuitive part comes from wrongly 
 assuming that
 the host has full freedom to pick which door to open, which he 
 does not
But yes. He has. It makes no difference. If he would ever open the right door, you would just take it too. So if the win is behind the two doors you did not choose first, you will always get it.
 The problem with this explanation is simply that it is too long 
 and calls the overly detailed reasoning a 'secret'. :o)
So take this shorter explanation: "There are three doors and two of them are opened, one by him and one by you. So the chance to win is two out of three." It doesn't matter if he uses his knowledge to open always a false door. It only matters that you open your door AFTER him, which allows you to react on the result of his door. If you open the door first, your chance is only 1/3.
Oct 08 2014
next sibling parent "eles" <eles eles.com> writes:
On Wednesday, 8 October 2014 at 07:00:38 UTC, Dominikus Dittes 
Scherkl wrote:
 On Wednesday, 8 October 2014 at 01:22:49 UTC, Timon Gehr wrote:
 It doesn't matter if he uses his knowledge to open always a
 false door.
It does. Actually, this is the single most important thing.
 It only matters that you open your door AFTER him,
 which allows you to react on the result of his door. If you
 open the door first, your chance is only 1/3.
If his choice would be completely random, as you seem to suggest above (because, actually, its choice is conditioned by your first choice) then, even if you open a door after him, the only thing you have is the fact that you are now in problem with 50% probability to win. If you remove the above piece of information, you could simply assume that there are only two doors and you are to open one of them. In this case is just 50/50.
Oct 08 2014
prev sibling parent "eles" <eles eles.com> writes:
On Wednesday, 8 October 2014 at 07:00:38 UTC, Dominikus Dittes 
Scherkl wrote:
 On Wednesday, 8 October 2014 at 01:22:49 UTC, Timon Gehr wrote:
 If he would ever open the right door, you would just take it 
 too.
Almost. If he opens the winning door, he gives you another very important information: the correctness of your first choice. If you already know if your first choice is correct or wrong, then having the host opening a door (does not matter which of the remaining two, in this case) solves the problem without ambiguity. But, when you make your second choice, you still not know if your first choice was correct or not. The only thing that you know is that the chance that your first choice was correct is two times less than the chance it was wrong. So you bet that your first choice was wrong, and you move on to the next problem, which, assuming this bet, now becomes a non-ambiguous problem. The key is this: "how would a third person bet on my first choice?" Reasonably, he would bet that the choice is wrong. So why wouldn't I do the same?
Oct 08 2014
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/07/2014 08:37 PM, H. S. Teoh via Digitalmars-d wrote:
 On 10/08/2014 12:10 AM, Nick Sabalausky wrote:
 [...]
 I've managed to grok it, but yet even I (try as I may) just cannot
 truly grok the monty hall problem. I *can* reliably come up with the
 correct answer, but *never* through an actual mental model of the
 problem, *only* by very, very carefully thinking through each step of
 the problem. And that never changes no matter how many times I think
 it through.
[...] The secret behind the monty hall scenario, is that the host is actually leaking extra information to you about where the car might be. [...]
Hmm, yea, that is a good thing to realize about it. I think a big part of what trips me up is applying a broken, contorted version of the "coin toss" reasoning (coin toss == my usual approach to probability). Because of the "coin toss" problem, I'm naturally inclined to see past events as irrelevant. So, initial impression is: I see two closed doors, an irrelevant open door, and a choice: "Closed door A or Closed door B?". Obviously, it's a total fallacy to assume "well, if there's two choices then they must be weighted equally." But, naturally, I figure that all three doors initially have equal changes, and so I'm already thinking "unweighted, even distribution", and then bam, my mind (falsely) sums up: "Two options, uniform distribution, third door isn't a choice so it's just a distraction. Therefore, 50/50". Now yes, I *do* see several fallacies/oversights/mistakes in that, but that's how my mind tries to setup the problem. So then I wind up working backwards from that or forcing myself to abandon it by starting from the beginning and carefully working it through. Another way to look at it, very similar to yours actually, and I think more or less the way the kid presented it in "21" (but in a typical hollywood "We hate exposition with a passion, so just rush through it as hastily as possible, we're only including it because writers expect us to, so who cares if nobody can follow it" style): 1. Three equal choices: 1/3 I'm right, 2/3 I'm wrong. 2. New choice: Bet that I was right (1/3), bet that I was wrong (2/3) 3. "An extra 33% chance for free? Sure, I'll take it." Hmm, looking at it now, I guess the second choice is simply *inverting* you first choice. Ahh, now I get what the kid (and you) was saying much better: Choosing "I'll change my guess" is equivalent to choosing *both* of the other two doors. The fact that he opens one of those other two doors is a complete distraction and totally irrelevent. Makes you think you're only choosing "the other ONE door" when you're really choosing "the other TWO doors". Interesting. See, this is why I love this NG :)
Oct 08 2014
parent "eles" <eles eles.com> writes:
On Wednesday, 8 October 2014 at 08:16:08 UTC, Nick Sabalausky 
wrote:
 On 10/07/2014 08:37 PM, H. S. Teoh via Digitalmars-d wrote:
 On 10/08/2014 12:10 AM, Nick Sabalausky wrote:
 [...]
 equivalent to choosing *both* of the other two doors.
Yeah, I think is the best way to put it.
Oct 08 2014
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/7/2014 4:49 PM, Timon Gehr wrote:
 On 10/08/2014 12:10 AM, Nick Sabalausky wrote:
 On 10/07/2014 06:47 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?=
 <ola.fosheim.grostad+dlang gmail.com>" wrote:
 Yep, however what the human brain is really bad at is reasoning about
 probability.
Yea, true. Probability can be surprisingly unintuitive even for people well-versed in logic. ...
Really?
Yes. See "Thinking, Fast and Slow": http://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555 I'm working my way through it now. Very interesting.
Oct 07 2014
prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/07/2014 07:49 PM, Timon Gehr wrote:
 On 10/08/2014 12:10 AM, Nick Sabalausky wrote:
 Ex: A lot of people have trouble understanding that getting "heads" in a
 coinflip many times in a row does *not* increase the likelihood of the
 next flip being "tails". And there's a very understandable reason why
 that's difficult to grasp.
What is this reason? It would be really spooky if the probability was actually increased in this way. You could win at 'heads or tails' by flipping a coin really many times until you got a sufficiently long run of 'tails', then going to another room and betting that the next flip will be 'heads', and if people didn't intuitively understand that, some would actually try to apply this trick. (Do they?)
I have actually met a lot of people who instinctively believe that getting "tails" many times in a row means that "heads" becomes more and more inevitable. Obviously they're wrong about that, but I think I *do* understand how they get tripped up: What people *do* intuitively understand is that the overall number of "heads" and "tails" are likely to be similar. Moreover, statistically speaking, the more coin tosses there are, the more the overall past results tend to converge towards 50%/50%. (Which is pretty much what's implied by "uniform random distribution".) This much is pretty easy for people to intuitively understand, even if they don't know the mathematical details. As a result, people's mental models will usually involve some general notion of "There's a natural tendency for the 'heads' and 'tails' to even out" Unfortunately, that summary is...well...partly truth but also partly inaccurate. So they take that kinda-shaky and not-entirely-accurate (but still *partially* true) mental summary and are then faced with the coin toss problem: "You've gotten 'tails' 10,000 times in a row." "Wow, really? That many?" So then the questionable mental model kicks in: "...natural tendency to even out..." The inevitable result? "Wow, I must be overdue for a heads!" Fallacious certainly, but also common and somewhat understandable. Another aspect that can mix people up: If you keep flipping the coin, over and over and over, it *is* very likely that at *some* point you'll get a "heads". That much *is* true and surprises nobody. Unfortunately, as true as it is, it's *not* relevant to individual tosses: They're individual likelihoods *always* stay the same: 50%. So we seemingly have a situation where something ("very, very likely to get a heads") is true of the whole *without* being true of *any* of the individual parts. While that does occur, it isn't exactly a super-common thing in normal everyday life, so it can be unintuitive for people. And another confusion: Suppose we rephrase it like this: "If you keep tossing a coin, how likely are you to get 10,000 'tails' in a row AND then get ANOTHER 'tails'?" Not very freaking likely, of course: 1 in 2^10,001. But *after* those first 10,000 'tails' have already occurred, the answer changes completely. What? Seriously? Math that totally changes based on "when"?!? But 2+2 is *always* 4!! All of a sudden, here we have a math where your location on the timeline is *crucially* important[1], and that's gotta trip up some of the people who (like everyone) started out with math just being arithmetic. [1] Or at least time *appears* to be crucially important, depending on your perspective: We could easily say that "time" is nothing more than an irrelevant detail of the hypothetical scenario and the *real* mathematics is just one scenario of "I have 10,001 samples of 50% probability" versus a completely different scenario of "I have 10,000 samples of 100% probability and 1 sample of 50% probability". Of course, deciding which of those problems is the one we're actually looking at involves considering where you are on the timeline.
 That said, it's just: When you first randomly choose the door, you would
 intuitively rather bet that you guessed wrong. The show master is simply
 proposing to tell you behind which of the other doors the car is in case
 you indeed guessed wrong.

 There's not more to it.
Hmm, yea, an interesting way to look at it.
Oct 08 2014
parent reply "eles" <eles eles.com> writes:
On Wednesday, 8 October 2014 at 07:35:25 UTC, Nick Sabalausky 
wrote:
 On 10/07/2014 07:49 PM, Timon Gehr wrote:
 On 10/08/2014 12:10 AM, Nick Sabalausky wrote:
 Ex: A lot of people have trouble understanding that getting 
 "heads" in a
 coinflip many times in a row does *not* increase the 
 likelihood of the
 next flip being "tails". And there's a very understandable
Of course it does not increase the probability to get a "tails". Actually, it increases the probability that you'll get "heads" again. For the simplest explanation, see here: http://batman.wikia.com/wiki/Two-Face's_Coin
Oct 08 2014
parent reply "Joseph Rushton Wakeling" <joseph.wakeling webdrake.net> writes:
On Wednesday, 8 October 2014 at 07:40:14 UTC, eles wrote:
 Of course it does not increase the probability to get a 
 "tails". Actually, it increases the probability that you'll get 
 "heads" again.
Er, no, unless you assume that repeated flips of the coin in some way deform it so as to bias the outcome. An extended run of heads might lead you to conclude that the coin is biased, and so increase your _expectation_ that the next flip will also result in a head, but that doesn't alter the actual underlying probability.
Oct 09 2014
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 10 Oct 2014 01:35, "Joseph Rushton Wakeling via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:
 On Wednesday, 8 October 2014 at 07:40:14 UTC, eles wrote:
 Of course it does not increase the probability to get a "tails".
Actually, it increases the probability that you'll get "heads" again.
 Er, no, unless you assume that repeated flips of the coin in some way
deform it so as to bias the outcome.
 An extended run of heads might lead you to conclude that the coin is
biased, and so increase your _expectation_ that the next flip will also result in a head, but that doesn't alter the actual underlying probability. http://www.wired.com/2012/12/what-does-randomness-look-like/
Oct 09 2014
next sibling parent reply "eles" <eles eles.com> writes:
On Friday, 10 October 2014 at 06:28:06 UTC, Iain Buclaw via 
Digitalmars-d wrote:
 On 10 Oct 2014 01:35, "Joseph Rushton Wakeling via 
 Digitalmars-d" <
 digitalmars-d puremagic.com> wrote:
 On Wednesday, 8 October 2014 at 07:40:14 UTC, eles wrote:
 Of course it does not increase the probability to get a 
 "tails".
Actually, it increases the probability that you'll get "heads" again.
 Er, no, unless you assume that repeated flips of the coin in 
 some way
deform it so as to bias the outcome.
;) It was a joke. The link that I posted was supposed to make things clear. :)
Oct 10 2014
parent "Joseph Rushton Wakeling" <joseph.wakeling webdrake.net> writes:
On Friday, 10 October 2014 at 09:21:37 UTC, eles wrote:
 ;) It was a joke. The link that I posted was supposed to make 
 things clear. :)
You two-faced person, you. ;-)
Oct 10 2014
prev sibling parent reply "Joseph Rushton Wakeling" <joseph.wakeling webdrake.net> writes:
On Friday, 10 October 2014 at 06:28:06 UTC, Iain Buclaw via 
Digitalmars-d wrote:
 http://www.wired.com/2012/12/what-does-randomness-look-like/
... yes, allowing for the reasonable expectations one can have for extended runs of heads in a regular 50/50 coin-flip process. Actually, related to that article, in my very first stats lecture at university, the first slide the lecturer showed us (on the overhead projector...) was, side by side, two patterns of dots each within a rectangular area. He asked: "Do you think these points are distributed at random?" Well, they pretty much looked the same to the naked eye. Then he took another transparency, which placed grids over the two rectangular dot-filled areas. In one, the dots were here, there, with some grid squares containing no dots at all, some containing clusters, whatever. In the other, every single grid square contained exactly one dot. I still think that was one of the single most important lessons in probability that I ever had.
Oct 10 2014
next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/10/2014 05:31 AM, Joseph Rushton Wakeling wrote:
 On Friday, 10 October 2014 at 06:28:06 UTC, Iain Buclaw via
 Digitalmars-d wrote:
 http://www.wired.com/2012/12/what-does-randomness-look-like/
... yes, allowing for the reasonable expectations one can have for extended runs of heads in a regular 50/50 coin-flip process. Actually, related to that article, in my very first stats lecture at university, the first slide the lecturer showed us (on the overhead projector...) was, side by side, two patterns of dots each within a rectangular area. He asked: "Do you think these points are distributed at random?" Well, they pretty much looked the same to the naked eye. Then he took another transparency, which placed grids over the two rectangular dot-filled areas. In one, the dots were here, there, with some grid squares containing no dots at all, some containing clusters, whatever. In the other, every single grid square contained exactly one dot. I still think that was one of the single most important lessons in probability that I ever had.
I like that. I actually have a similar classroom probability story too (involving one of the best teachers I ever had): As part of a probability homework assignment, we were asked to flip a coin 100 times and write down the results. "Uhh, yea, there's no way I'm doing that. I'm just gonna write down a bunch of T's and F's." Having previously played around with PRNG's (using them, not actually creating them), I had noticed that you do tend to get surprisingly long runs of one value missing, or the occasional clustering. I carefully used that knowledge to help me cheat. During the next class, the teacher pointed out that "I can tell, most of you didn't actually flip a coin, did you? You just wrote down T's and F's..." Which turned out to be the whole *point* of the assignment. Deliberately get students to "cheat" and fake randomness - poorly - in order to *really* get them to understand the nature of randomness. Then he turned to me and said, "Uhh, Nick, you actually DID flip a coin didn't you?" Hehe heh heh. "Nope :)" I got a good chuckle out of that.
Oct 11 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/11/2014 1:34 PM, Nick Sabalausky wrote:
 Having previously played around with PRNG's (using them, not actually creating
 them), I had noticed that you do tend to get surprisingly long runs of one
value
 missing, or the occasional clustering. I carefully used that knowledge to help
 me cheat.
Election fraud is often detected by examining the randomness of the least significant digits of the vote totals.
Oct 14 2014
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/10/2014 2:31 AM, Joseph Rushton Wakeling wrote:
 I still think that was one of the single most important lessons in probability
 that I ever had.
Research shows that humans, even trained statisticians, are spectacularly bad at intuitive probability. -- "Thinking, Fast and Slow" by Daniel Kahneman
Nov 01 2014
prev sibling parent "eles" <eles eles.com> writes:
On Friday, 10 October 2014 at 00:32:05 UTC, Joseph Rushton
Wakeling wrote:
 On Wednesday, 8 October 2014 at 07:40:14 UTC, eles wrote:
 Of course it does not increase the probability to get a 
 "tails". Actually, it increases the probability that you'll 
 get "heads" again.
Did you see the link? ;;)
Oct 10 2014
prev sibling next sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 7 Oct 2014 17:37:51 -0700
"H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> wrote:

 told in a deliberately misleading way -- the fact that the host
 *never* opens the right door is often left as an unstated "common
 sense" assumption, thereby increasing the likelihood that people will
 overlook this minor but important detail.
that's why i was always against this "problem". if you giving me the logic problem, give me all information. anything not written clear can't be the part of the problem. that's why the right answer for the problem when i didn't told that the host never opens the right door is "50/50".
Oct 08 2014
prev sibling parent reply "eles" <eles eles.com> writes:
On Tuesday, 7 October 2014 at 23:49:37 UTC, Timon Gehr wrote:
 On 10/08/2014 12:10 AM, Nick Sabalausky wrote:
 On 10/07/2014 06:47 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?=
 <ola.fosheim.grostad+dlang gmail.com>" wrote:
 On Tuesday, 7 October 2014 at 08:19:15 UTC, Nick Sabalausky
 What is this reason?
This one: Result of coin tossing is independent at any current attempt. It does not depend of past or future results. Probability is 0.5 On the other hand, obtaining a series with 100 heads in a row is very small (exactly because of the independence). Obtaining a series with 101 heads in a row is even smaller, so people will assume that the 101 tossing should probably give a "tails". But they forget that the probability of a 101 series where first 100 are heads and the 101 is tails is exactly *the same* as the probability of a 101 series of heads. They compare the probability of a 101 series of heads with the probability of a series of 100 heads instead of comparing it against a series with first 100 heads and the 101rd being a tail. It is the bias choice (we have tendency to compare things that are easier - not more pertinent - to compare).
Oct 08 2014
parent "eles" <eles eles.com> writes:
On Wednesday, 8 October 2014 at 07:51:39 UTC, eles wrote:
 On Tuesday, 7 October 2014 at 23:49:37 UTC, Timon Gehr wrote:
 On 10/08/2014 12:10 AM, Nick Sabalausky wrote:
 On 10/07/2014 06:47 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?=
 <ola.fosheim.grostad+dlang gmail.com>" wrote:
 On Tuesday, 7 October 2014 at 08:19:15 UTC, Nick Sabalausky
 against a series with first 100 heads and the 101rd being a
well, 101st
Oct 08 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/7/2014 1:19 AM, Nick Sabalausky wrote:
 This is VERY simple, and crucial, stuff. And yet I see SOOO many grown adults,
 even ones with advanced graduate degrees, consistently fail completely and
 uttery at basic logical reasoning in everyday life (and we're talking very,
very
 obvious and basic fallacies), that it's genuinely disturbing.
I agree with teaching logical fallacies. I believe one of the most important things we can teach the young is how to separate truth from crap. And this is not done - I'd never really heard of logical fallacies until after college. (I was taught the scientific method, though.) I.e. logical fallacies and the scientific method should be core curriculum. Ironically, I've seen many researchers with PhD's carefully using the scientific method in their research, and promptly lapsing into logical fallacies with everything else. It's like sales techniques. I've read books on sales techniques and the psychology behind them. I don't use or apply them with any skill, but it has enabled me to recognize when those techniques are used on me, and has the effect of immunizing me against them. At least learning the logical fallacies helps immunize one against being fraudulently influenced.
Oct 07 2014
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/07/2014 03:37 PM, Walter Bright wrote:
 I believe one of the most
 important things we can teach the young is how to separate truth from
 crap. And this is not done
Hear, hear!
 I.e. logical fallacies and the scientific method should be core curriculum.
Yes. My high-school (and maybe junior high, IIRC) science classes covered the scientific method at least. So that much is good (at least, where I was anyway).
 Ironically, I've seen many researchers with PhD's carefully using the
 scientific method in their research, and promptly lapsing into logical
 fallacies with everything else.
Yes, people use entirely different mindsets for different topics. Seems to be an inherent part of the mind, and I can certainly see some benefit to that. Unfortunately it can occasionally go wrong, like you describe.
 It's like sales techniques. I've read books on sales techniques and the
 psychology behind them. I don't use or apply them with any skill, but it
 has enabled me to recognize when those techniques are used on me, and
 has the effect of immunizing me against them.

 At least learning the logical fallacies helps immunize one against being
 fraudulently influenced.
Definitely. I can always spot a commissioned (or "bonus"-based) salesman a mile away. A lot of their tactics are incredibly irritating, patronizing, and frankly very transparent. (But my dad's a salesman so maybe that's how I managed to develop a finely-tuned "sales-bullshit detector") It's interesting (read: disturbing) how convinced they are that you're just being rude and difficult when you don't fall hook, line and sinker for their obvious bullshit and their obvious lack of knowledge. Here's another interesting tactic you may not be aware of: I'm not sure how wide-spread this is, but I have direct inside information that it *is* common in car dealerships around my general area. Among themselves, the salesmen have a common saying: "Buyers are liars". It's an interesting (and disturbing) method of ensuring salesmen police themselves and continue to be 100% read-and-willing to abandon ethics and bullshit the crap out of customers. Obviously customers *do* lie of course (and that helps the tactic perpetuate itself), but when a *salesman* says it, it really is an almost hilarious case of "The pot calling the grey paint 'black'." It's a salesman's whole freaking *job* is be a professional liar! (And there's all sorts of tricks to self-rationalizing it and staying on the good side of the law. But their whole professional JOB is to *bullshit*! And they themselves are dumb enough to buy into their *own* "It's the *buyers* who are disonest!" nonsence.) Casinos are similar. Back in college, when my friends and I were all 19 and attending a school only about 2 hours from Canada...well, whaddya expect?...We took a roadtrip up to Casino Windsor! Within minutes of walking through the place I couldn't even *help* myself from counting the seemingly-neverending stream of blatantly-obvious psychological gimmicks. It was just one after another, everywhere you'd look, and they were so SO OBVIOUS it was like walking inside a salesman's brain. The physical manifestation of a direct insult to people's intelligence. It's really unbelievable how stupid a person has to be to fall for those blatant tricks. But then again, slots and video poker aren't exactly my thing anyway. I'm from the 80's: If I plunk coins into a machine I expect to get food, beverage, clean laundry, or *actual gameplay*. Repeatedly purchasing the message "You loose" while the entire building itself is treating me like a complete brain-dead idiot isn't exactly my idea of "addictive". If I want to spend money to watch non-interactive animations and occasionally push a button or two to keep it all going, I'll just buy "The Last of Us".
Oct 07 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/7/2014 3:54 PM, Nick Sabalausky wrote:
 It's a salesman's whole freaking *job* is be a professional liar!
Poor salesmen are liars. But the really, really good ones are ones who are able to match up what a customer needs with the right product for him. There, he is providing a valuable service to the customer. Serve the customer well like that, and you get a repeat customer. I know many salesmen who get my repeat business because of that. The prof who taught me accounting used to sell cars. I asked him how to tell a good dealership from a bad one. He told me the good ones have been in business for more than 5 years, because by then one has run out of new suckers and is relying on repeat business.
 But then again, slots and video poker aren't exactly my thing anyway. I'm from
 the 80's: If I plunk coins into a machine I expect to get food, beverage, clean
 laundry, or *actual gameplay*. Repeatedly purchasing the message "You loose"
 while the entire building itself is treating me like a complete brain-dead
idiot
 isn't exactly my idea of "addictive".
I found gambling to be a painful experience, not entertaining at all.
Oct 07 2014
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/07/2014 11:29 PM, Walter Bright wrote:
 On 10/7/2014 3:54 PM, Nick Sabalausky wrote:
 It's a salesman's whole freaking *job* is be a professional liar!
Poor salesmen are liars. But the really, really good ones are ones who are able to match up what a customer needs with the right product for him. There, he is providing a valuable service to the customer.
Can't say I've personally come across any of the latter (it relies on salesmen knowing what they're talking about and still working sales anyway - which I'm sure does occur for various reasons, but doesn't seem common from what I've seen). But maybe I've just spent far too much time at MicroCenter ;) Great store, but dumb sales staff ;)
 Serve the customer well like that, and you get a repeat customer. I know
 many salesmen who get my repeat business because of that.
Certainly reasonable points, and I'm glad to hear there *are* respectable ones out there.
 The prof who taught me accounting used to sell cars. I asked him how to
 tell a good dealership from a bad one. He told me the good ones have
 been in business for more than 5 years, because by then one has run out
 of new suckers and is relying on repeat business.
That sounds reasonable on the surface, but it relies on several questionable assumptions: 1. Suckers routinely know they've been suckered. 2. Suckers avoid giving repeat business to those who suckered them (not as reliable an assumption as one might expect) 3. The rate of loss on previous suckers overshadows the rate of new suckers. (Ex: No matter how badly people hate working at McDonald's, they're unlikely to run low on fresh applicants without a major birthrate decline - and even then they'd have 16 years to prepare) 4. Good dealerships don't become bad. 5. There *exists* a good one within a reasonable distance. 6. People haven't become disillusioned and given up on trying to find a good one (whether a good one exists or not, the effect here would be the same). 7. The bad ones aren't able to compete/survive through other means. (Cornering a market, mindshare, convincing ads, misc gimmicks, merchandising or other side-revenue streams, anti-competitive practices, etc.) Also, the strategy has a potential self-destruct switch: Even if the strategy works, if enough people follow it then even good dealerships might not be able to survive the initial 5 year trial. Plus, I know of a counter-example around here. From an insider, I've heard horror stories about the shit the managers, finance people, etc would pull. But they're a MAJOR dealer in the area and have been for as long as I can remember.
 But then again, slots and video poker aren't exactly my thing anyway.
 I'm from
 the 80's: If I plunk coins into a machine I expect to get food,
 beverage, clean
 laundry, or *actual gameplay*. Repeatedly purchasing the message "You
 loose"
 while the entire building itself is treating me like a complete
 brain-dead idiot
 isn't exactly my idea of "addictive".
I found gambling to be a painful experience, not entertaining at all.
I actually enjoyed that evening quite a bit: A road trip with friends is always fun, as is seeing new places, and it was certainly a very pretty place inside (for a very good reason, of course). But admittedly, the psychological tricks were somewhat insulting, and by the time I got through the $20 I'd budgeted I had already gotten very, very bored with slot machines and video poker. And blackjack's something I'd gotten plenty of all the way back on the Apple II. If I want to gamble I'll just go buy more insurance ;) Better odds. Or stock market. At least that doesn't have quite as much of a "house" that "always wins", at least not to the same extent.
Oct 08 2014
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/6/2014 4:06 PM, "Ola Fosheim Grøstad" 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 On a positive note: the IOC managed to demand that the norwegian King ought to
 hold a party for the IOC leaders and additionally demanded that he should pay
 their drinks. It was part of their 7000 page olympics qualification
requirements
 document. It is so heavily regulated that it explicitly specifies that the
 personnel in the hotels MUST SMILE to the IOC leaders when they arrive. I kid
 you not, even games and past times are heavily bureaucratic down to minuscule
 details these days. So, due pressure from the newspapers/grassroots and the
 royal insult the politicians eventually had to turn down the ~$10.000.000.000
 winter olympics budget proposal. Good riddance. Live monarchy!
Yay for Norway!
Oct 07 2014
prev sibling parent reply =?UTF-8?Q?Tobias=20M=C3=BCller?= <troplin bluewin.ch> writes:
"Paolo Invernizzi" <paolo.invernizzi no.address> wrote:
 And guess it, here the buildings made by ancient romans are still "up and
 running", while we have schools building made in the '90 that come down
 at every earthquake...
All the bad buildings from the ancient romans already came down while the last 2000 years. The best 1% survived. Tobi
Oct 05 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/5/2014 3:00 AM, Tobias Müller wrote:
 All the bad buildings from the ancient romans already came down while the
 last 2000 years. The best 1% survived.
Yay for survivorship bias!
Oct 05 2014
parent reply "eles" <eles215 gzk.dot> writes:
On Sunday, 5 October 2014 at 20:46:16 UTC, Walter Bright wrote:
 On 10/5/2014 3:00 AM, Tobias Müller wrote:
 All the bad buildings from the ancient romans already came 
 down while the
 last 2000 years. The best 1% survived.
Yay for survivorship bias!
The same happens when asking people if such or such dictatorship was good or bad. You hear "it wasn't so bad, after all". Yes, because they only ask the survivors, they don't go to ask the dead, too. Sorry, my rant about politics.
Oct 05 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/5/2014 2:55 PM, eles wrote:
 The same happens when asking people if such or such dictatorship was good or
 bad. You hear "it wasn't so bad, after all". Yes, because they only ask the
 survivors, they don't go to ask the dead, too.
Yeah, I've often wondered what the dead soldiers would say about wars - whether it was worth it or not. Only the ones who lived get interviewed.
 Sorry, my rant about politics.
Oct 05 2014
prev sibling next sibling parent reply "Piotrek" <p nonexistent.pl> writes:
On Friday, 3 October 2014 at 15:43:59 UTC, Sean Kelly wrote:

 My point, and I think Kagamin's as well, is that the entire 
 plane is a system and the redundant internals are subsystems.  
 They may not share memory, but they are wired to the same 
 sensors, servos, displays, etc.  Thus the point about shutting 
 down the entire plane as a result of a small failure is fair.
This "real life" example: http://en.wikipedia.org/wiki/Air_France_Flight_447 I just pick some interesting statements (there are other factors described as well): "temporary inconsistency between the measured speeds, likely as a result of the obstruction of the pitot tubes by ice crystals, causing autopilot disconnection and reconfiguration to alternate law;" And as I can see it, all subsystems related to the "small failure" was shut down. But what is also important information was not clearly provided to the pilots: "Despite the fact that they were aware that altitude was declining rapidly, the pilots were unable to determine which instruments to trust: it may have appeared to them that all values were incoherent" "the cockpit lacked a clear display of the inconsistencies in airspeed readings identified by the flight computers;" Piotrek
Oct 03 2014
next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Friday, 3 October 2014 at 18:00:58 UTC, Piotrek wrote:
 And as I can see it, all subsystems related to the "small 
 failure" was shut down. But what is also important information 
 was not clearly provided to the pilots:

 "Despite the fact that they were aware that altitude was 
 declining rapidly, the pilots were unable to determine which 
 instruments to trust: it may have appeared to them that all 
 values were incoherent"

 "the cockpit lacked a clear display of the inconsistencies in 
 airspeed readings identified by the flight computers;"
There's a similar issue with nuclear reactors, which is that there are so many blinky lights and such that it can be impossible to spot or prioritize problems in a failure scenario. I know there have been articles written on revisions of user interface design in reactors specifically to deal with this issue, and I suspect the ideas are applicable to error handling in general.
Oct 03 2014
prev sibling next sibling parent reply "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Friday, 3 October 2014 at 18:00:58 UTC, Piotrek wrote:
 On Friday, 3 October 2014 at 15:43:59 UTC, Sean Kelly wrote:

 This "real life" example:

 http://en.wikipedia.org/wiki/Air_France_Flight_447

 I just pick some interesting statements (there are other 
 factors described as well):

 "temporary inconsistency between the measured speeds, likely as 
 a result of the obstruction of the pitot tubes by ice crystals, 
 causing autopilot disconnection and reconfiguration to 
 alternate law;"


 And as I can see it, all subsystems related to the "small 
 failure" was shut down. But what is also important information 
 was not clearly provided to the pilots:

 "Despite the fact that they were aware that altitude was 
 declining rapidly, the pilots were unable to determine which 
 instruments to trust: it may have appeared to them that all 
 values were incoherent"

 "the cockpit lacked a clear display of the inconsistencies in 
 airspeed readings identified by the flight computers;"

 Piotrek
As one that has read the original report integrally, I think that you have taken a bad example: despite the autopilot was disengaged, the stall alarm ringed a pletora of times. There's no real alternative to the disengagement of the autopilot is that fundamental parameter is compromised. It took the captain only a few moment to understand the problem (read the voice-recording transcription), but it was too late... --- /Paolo
Oct 03 2014
next sibling parent reply "Piotrek" <p nonexistent.pl> writes:
On Friday, 3 October 2014 at 20:31:42 UTC, Paolo Invernizzi wrote:

 As one that has read the original report integrally, I think 
 that you have taken a bad example: despite the autopilot was 
 disengaged, the stall alarm ringed a pletora of times.
My point was that the broken speed indicators shut down the autopilot systems. Piotrek
Oct 03 2014
next sibling parent reply "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Friday, 3 October 2014 at 22:27:44 UTC, Piotrek wrote:
 On Friday, 3 October 2014 at 20:31:42 UTC, Paolo Invernizzi 
 wrote:

 As one that has read the original report integrally, I think 
 that you have taken a bad example: despite the autopilot was 
 disengaged, the stall alarm ringed a pletora of times.
My point was that the broken speed indicators shut down the autopilot systems. Piotrek
And that is still the only reasonable thing to do in that case. --- /Paolo
Oct 04 2014
parent "Piotrek" <p nonexistent.pl> writes:
On Saturday, 4 October 2014 at 08:24:40 UTC, Paolo Invernizzi 
wrote:

 And that is still the only reasonable thing to do in that case.

 ---
 /Paolo
And I never said otherwise. See my response to Walter's post. Piotrek
Oct 04 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/3/2014 3:27 PM, Piotrek wrote:
 My point was that the broken speed indicators shut down the autopilot systems.
The alternative is to have the autopilot crash the airplane. The autopilot cannot fly with compromised airspeed data.
Oct 04 2014
parent "Piotrek" <p nonexistent.pl> writes:
On Saturday, 4 October 2014 at 08:30:11 UTC, Walter Bright wrote:
 On 10/3/2014 3:27 PM, Piotrek wrote:
 My point was that the broken speed indicators shut down the 
 autopilot systems.
The alternative is to have the autopilot crash the airplane. The autopilot cannot fly with compromised airspeed data.
Yes, I know. I just provided that example as a response to:
 Do you interpret airplane safety right? As I understand, 
 airplanes are safe
 exactly because they recover from assert failures and continue 
 operation.
And Paulo stated it's a bad example. Maybe it is, but I couldn't find a better one. This accident just sits in my head as the sequence of events shocked me the most from all accident stories I heard. Piotrek
Oct 04 2014
prev sibling parent reply "eles" <eles215 gzk.dot> writes:
On Friday, 3 October 2014 at 20:31:42 UTC, Paolo Invernizzi wrote:
 On Friday, 3 October 2014 at 18:00:58 UTC, Piotrek wrote:
 On Friday, 3 October 2014 at 15:43:59 UTC, Sean Kelly wrote:
 As one that has read the original report integrally, I think 
 that you have taken a bad example: despite the autopilot was 
 disengaged, the stall alarm ringed a pletora of times.

 There's no real alternative to the disengagement of the 
 autopilot is that fundamental parameter is compromised.

 It took the captain only a few moment to understand the problem 
 (read the voice-recording transcription), but it was too late...
For the curious, the flight analysis here: http://www.popularmechanics.com/technology/aviation/crashes/what-really-happened-aboard-air-france-447-6611877 Captain's first error was to leave the cockpit when approaching storm. Second, was to give command to the lower-experienced co-pilots. Not only for quality of the flight, but also for the quality of the team. His third error was to have neglected the fact that the radar was not correctly set up. And his most important (and final) error was to not take commands back while coming back in to the cockpit. And it was airliner's fault to embark on transatlantic flights just one experienced man and two very low experienced copilots. This is a beginner mistake: "the insanity of pulling back on the controls while stalled" and this passage resumes it quite well: "the captain of the flight makes no attempt to physically take control of the airplane. Had Dubois done so, he almost certainly would have understood, as a pilot with many hours flying light airplanes, the insanity of pulling back on the controls while stalled" If you read the analysis you get scared.
Oct 03 2014
parent reply "eles" <eles eles.com> writes:
On Saturday, 4 October 2014 at 05:26:52 UTC, eles wrote:
 On Friday, 3 October 2014 at 20:31:42 UTC, Paolo Invernizzi 
 wrote:
 On Friday, 3 October 2014 at 18:00:58 UTC, Piotrek wrote:
 On Friday, 3 October 2014 at 15:43:59 UTC, Sean Kelly wrote:
 For the curious, the flight analysis here:

 http://www.popularmechanics.com/technology/aviation/crashes/what-really-happened-aboard-air-france-447-6611877
A just-printed new analysis of the same: http://www.vanityfair.com/business/2014/10/air-france-flight-447-crash
Oct 14 2014
parent "eles" <eles215 gzk.dot> writes:
On Tuesday, 14 October 2014 at 15:57:05 UTC, eles wrote:
 On Saturday, 4 October 2014 at 05:26:52 UTC, eles wrote:
 On Friday, 3 October 2014 at 20:31:42 UTC, Paolo Invernizzi 
 wrote:
 On Friday, 3 October 2014 at 18:00:58 UTC, Piotrek wrote:
 On Friday, 3 October 2014 at 15:43:59 UTC, Sean Kelly wrote:
 For the curious, the flight analysis here:

 http://www.popularmechanics.com/technology/aviation/crashes/what-really-happened-aboard-air-france-447-6611877
A just-printed new analysis of the same: http://www.vanityfair.com/business/2014/10/air-france-flight-447-crash
& the movie of it: http://www.almdares.net/vz/youtube_browser.php?do=show&vidid=TsgyBqlFixo
Oct 27 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/3/2014 11:00 AM, Piotrek wrote:
 http://en.wikipedia.org/wiki/Air_France_Flight_447

 I just pick some interesting statements (there are other factors described as
 well):
The overriding failure in that accident was the pilot panicked and persistently did the wrong thing, the opposite of what every pilot is relentlessly trained to do.
Oct 04 2014
parent reply "Paolo Invernizzi" <paolo.invernizzi no.address> writes:
On Saturday, 4 October 2014 at 08:28:40 UTC, Walter Bright wrote:
 On 10/3/2014 11:00 AM, Piotrek wrote:
 http://en.wikipedia.org/wiki/Air_France_Flight_447

 I just pick some interesting statements (there are other 
 factors described as
 well):
The overriding failure in that accident was the pilot panicked and persistently did the wrong thing, the opposite of what every pilot is relentlessly trained to do.
One point that the report stressed: when the co-pilots pulled back repeatedly the joystick to raise the plane, the second one could not see that action, as the two joystick are not connected, and they are not moving in sync in the airbus. Basically, the info-path regarding the joystick between two disconnected systems (the co-pilots) is cut in modern plane, so it's more difficult to check if their "output" (push/pull/etc) is coherent at the end. That, in my opinion, was the biggest problem in that tragedy. --- /Paolo
Oct 04 2014
next sibling parent reply "Piotrek" <p nonexistent.pl> writes:
On Saturday, 4 October 2014 at 08:45:57 UTC, Paolo Invernizzi 
wrote:

 Basically, the info-path regarding the joystick between two 
 disconnected systems (the co-pilots) is cut in modern plane, so 
 it's more difficult to check if their "output" (push/pull/etc) 
 is coherent at the end.

 That, in my opinion, was the biggest problem in that tragedy.
 ---
 /Paolo
Yeah, I wish to know the rationale for asynchronous joysticks. Pioterk
Oct 04 2014
parent "eles" <eles215 gzk.dot> writes:
On Saturday, 4 October 2014 at 09:36:37 UTC, Piotrek wrote:
 On Saturday, 4 October 2014 at 08:45:57 UTC, Paolo Invernizzi 
 wrote:
 Yeah, I wish to know the rationale for asynchronous joysticks.
Not only that, also the computer *averaged* the inputs (I am really at loss with that, in a plane. If there is a mountain in front of the plane, and one pilot pulls left, while the other pilot pulls right, the computer makes the "smart" decision to fly ahead straight throught the mountain.) At least if it would let you know: "the other post is giving me other inputs! I warn you, I am averaging the inputs!". It did not warn. As the right seat has been pulling all back, the left seat saw nothing else that the plane did not respond (or very little) to his own inputs. He, following his inputs, have no feedback about how the plane is reacting. Not knowing the other is pulling back, all that he saw was an iressponsive plane. Anyway, systems might be blamed. But the attitude of that guy who never informed the others that he was pulling back for a quarter of hour is the issue there.
Oct 04 2014
prev sibling parent "eles" <eles215 gzk.dot> writes:
On Saturday, 4 October 2014 at 08:45:57 UTC, Paolo Invernizzi 
wrote:
 On Saturday, 4 October 2014 at 08:28:40 UTC, Walter Bright 
 wrote:
 On 10/3/2014 11:00 AM, Piotrek wrote:
 That, in my opinion, was the biggest problem in that tragedy.
There is also this one: http://www.flightglobal.com/news/articles/stall-warning-controversy-haunts-af447-inquiry-360336/ "It insists the design of the stall warning "misled" the pilots. "Each time they reacted appropriately the alarm triggered inside the cockpit, as though they were reacting wrongly. Conversely each time the pilots pitched up the aircraft, the alarm shut off, preventing a proper diagnosis of the situation." SNPL's argument essentially suggests the on-off alarm might have incorrectly been interpreted by the crew as an indication that the A330 was alternating between being stalled and unstalled, when it was actually alternating between being stalled with valid airspeed data and stalled with invalid airspeed data."
Oct 04 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/3/2014 8:43 AM, Sean Kelly wrote:
 My point, and I think Kagamin's as well, is that the entire plane is a system
 and the redundant internals are subsystems.  They may not share memory, but
they
 are wired to the same sensors, servos, displays, etc.
No, they do not share sensors, servos, etc.
 Thus the point about shutting down the entire plane as a result of a small
failure is fair.
That's a complete misunderstanding. In NO CASE does avionics software do anything after an assert but get shut down and physically isolated from what it controls. I've explained this over and over. It baffles me how twisted up this simple concept becomes when repeated back to me.
Oct 04 2014
next sibling parent "eles" <eles215 gzk.dot> writes:
On Saturday, 4 October 2014 at 08:15:51 UTC, Walter Bright wrote:
 On 10/3/2014 8:43 AM, Sean Kelly wrote:
 In NO CASE does avionics software do anything after an assert 
 but get shut down and physically isolated from what it controls.

 I've explained this over and over. It baffles me how twisted up 
 this simple concept becomes when repeated back to me.
AFAICT, you hold the righ idea. Either the discussion fellows argue about a different thing and this is a communication problem, either they are on a wrong path. Once an inconsistency in its own state is detected, any program quits guessing, and shuts down, sometimes with a minimal logging (if not harmless). An inconsistency is when a software or a system finds intself in a situation for which it wasn't designed to face. It simply breaks the ultimat variants of all that any critical program makes. This invariant is: "I know what I'm doing." Yes, a software might detect inconsistencies in the rest of the system and try to correct those *if it is designed (read: if it is made to know) what is doing. Majority voting, for example, is such case, but there the basic hypothesis is that sensors are not 100% reliable. And, sometimes, even the majority voting is directly wired. But, a critical software (and, for the general case, any software) only goes as far as it (believes) that it knows what's doing. When this is not the case, it does not continue, it stops immediately.
Oct 04 2014
prev sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Saturday, 4 October 2014 at 08:15:51 UTC, Walter Bright wrote:
 On 10/3/2014 8:43 AM, Sean Kelly wrote:
 My point, and I think Kagamin's as well, is that the entire 
 plane is a system
 and the redundant internals are subsystems.  They may not 
 share memory, but they
 are wired to the same sensors, servos, displays, etc.
No, they do not share sensors, servos, etc.
Gotcha. I imagine there are redundant displays in the cockpit as well, which makes sense. Thus the unifying factor in an airplane is the pilot. In a non-mannned system, it would be a control program (or a series of redundant control programs). So the system in this case includes the pilot.
 Thus the point about shutting down the entire plane as a 
 result of a small failure is fair.
That's a complete misunderstanding.
Right. So the system relies on the intelligence and training of the pilot for proper operation. Choosing which systems are in error vs. which are correct, etc. I still think an argument could be made that an entire airplane, pilot included, is analogous to a server infrastructure, or even a memory isolated program (the Erlang example). My only point in all this is that while choosing the OS process is a good default when considering the potential scope of undefined behavior, it's not the only definition. The pilot misinterpreting sensor data and making a bad judgement call is equivalent to the failure of distinct subsystems corrupting the state of the entire system to the point where the whole thing fails. The sensors were communicating confusing information to the pilot, and his programming, as it were, was not up to the task of separating the good information from the bad. Do you have any thoughts concerning my proposal in the "on errors" thread?
Oct 04 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/4/2014 9:09 AM, Sean Kelly wrote:
 On Saturday, 4 October 2014 at 08:15:51 UTC, Walter Bright wrote:
 On 10/3/2014 8:43 AM, Sean Kelly wrote:
 My point, and I think Kagamin's as well, is that the entire plane is a system
 and the redundant internals are subsystems.  They may not share memory, but
they
 are wired to the same sensors, servos, displays, etc.
No, they do not share sensors, servos, etc.
Gotcha. I imagine there are redundant displays in the cockpit as well, which makes sense. Thus the unifying factor in an airplane is the pilot.
Even the pilot has a backup! Next time you go flying, peek in the cockpit. You'll see dual instruments and displays. If you examine the outside, you'll see two (or three) pitot tubes (which measure airspeed).
 Right.  So the system relies on the intelligence and training of the pilot for
 proper operation.  Choosing which systems are in error vs. which are correct,
 etc.
A lot of design revolves around making it obvious which component is the failed one, the classic being a red light on the instrument panel.
 I still think an argument could be made that an entire airplane, pilot
 included, is analogous to a server infrastructure, or even a memory isolated
 program (the Erlang example).
Anyone with little training can fly an airplane. Heck, you can go to any flight school and they'll take you up on an introductory flight and let you try out the controls in flight. Most of a pilot's training consists of learning how to deal with failure.
 My only point in all this is that while choosing the OS process is a good
 default when considering the potential scope of undefined behavior, it's not
the
 only definition.  The pilot misinterpreting sensor data and making a bad
 judgement call is equivalent to the failure of distinct subsystems corrupting
 the state of the entire system to the point where the whole thing fails.  The
 sensors were communicating confusing information to the pilot, and his
 programming, as it were, was not up to the task of separating the good
 information from the bad.
That's true. Many accidents have resulted from the pilot getting confused about the failures being reported to him, and his failure to properly grasp the situation and what to do about it. All of these result in reevaluations of how failures are presented to the pilot, and the pilot's training and procedures. On the other hand, many failures have not resulted in accidents because of the pilot's ability to "think outside the box" and come up with a creative solution on the spot. It's why we need human pilots. These solutions then become part of standard procedure!
 Do you have any thoughts concerning my proposal in the "on errors" thread?
Looks interesting, but haven't gotten to it yet.
Oct 04 2014
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/3/2014 5:16 AM, Jacob Carlborg wrote:
 I have no idea of airplane works but I think Walter usual says they have at
 least three backup systems. If one system fails, shut it down and switch to the
 backup.
It's a "dual path" system, meaning there's one backup. Some systems are in triplicate, such as the hydraulic system.
Oct 04 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/3/2014 4:27 AM, Kagamin wrote:
 Do you interpret airplane safety right? As I understand, airplanes are safe
 exactly because they recover from assert failures and continue operation.
Nope. That's exactly 180 degrees from how it works. Any airplane system that detects a fault shuts itself down and the backup is engaged. No way in hell is software allowed to continue that asserted.
Oct 04 2014
parent reply "Kagamin" <spam here.lot> writes:
On Saturday, 4 October 2014 at 08:08:49 UTC, Walter Bright wrote:
 On 10/3/2014 4:27 AM, Kagamin wrote:
 Do you interpret airplane safety right? As I understand, 
 airplanes are safe
 exactly because they recover from assert failures and continue 
 operation.
Nope. That's exactly 180 degrees from how it works. Any airplane system that detects a fault shuts itself down and the backup is engaged. No way in hell is software allowed to continue that asserted.
Sure, software is one part of an airplane, like a thread is a part of a process. When the part fails, you discard it and continue operation. In software it works by rolling back a failed transaction. An airplane has some tricks to recover from failures, but still it's a "no fail" design you argue against: it shuts down parts one by one when and only when they fail and continues operation no matter what until nothing works and even then it still doesn't fail, just does nothing. The airplane example works against your arguments. The unreliable design you talk about would be committing a failed transaction, but no, nobody suggests that.
Oct 15 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/15/2014 12:19 AM, Kagamin wrote:
 Sure, software is one part of an airplane, like a thread is a part of a
process.
 When the part fails, you discard it and continue operation. In software it
works
 by rolling back a failed transaction. An airplane has some tricks to recover
 from failures, but still it's a "no fail" design you argue against: it shuts
 down parts one by one when and only when they fail and continues operation no
 matter what until nothing works and even then it still doesn't fail, just does
 nothing. The airplane example works against your arguments.
This is a serious misunderstanding of what I'm talking about. Again, on an airplane, no way in hell is a software system going to be allowed to continue operating after it has self-detected a bug. Trying to bend the imprecise language I use into meaning the opposite doesn't change that.
Oct 16 2014
next sibling parent "Kagamin" <spam here.lot> writes:
On Thursday, 16 October 2014 at 19:53:42 UTC, Walter Bright wrote:
 On 10/15/2014 12:19 AM, Kagamin wrote:
 Sure, software is one part of an airplane, like a thread is a 
 part of a process.
 When the part fails, you discard it and continue operation. In 
 software it works
 by rolling back a failed transaction. An airplane has some 
 tricks to recover
 from failures, but still it's a "no fail" design you argue 
 against: it shuts
 down parts one by one when and only when they fail and 
 continues operation no
 matter what until nothing works and even then it still doesn't 
 fail, just does
 nothing. The airplane example works against your arguments.
This is a serious misunderstanding of what I'm talking about. Again, on an airplane, no way in hell is a software system going to be allowed to continue operating after it has self-detected a bug.
Neither does failed transaction. I already approved that:
 When the part fails, you discard it and continue operation. In 
 software it works by rolling back a failed transaction.
 Trying to bend the imprecise language I use into meaning the 
 opposite doesn't change that.
Do you think I question that? I don't. I agree discarding a failed part is ok, and this is what traditional multithreaded server software already do: rollback a failed transaction and continue operation, just like airplane: loosing a part doesn't lose the whole.
Oct 17 2014
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Thursday, 16 October 2014 at 19:53:42 UTC, Walter Bright wrote:
 On 10/15/2014 12:19 AM, Kagamin wrote:
 Sure, software is one part of an airplane, like a thread is a 
 part of a process.
 When the part fails, you discard it and continue operation. In 
 software it works
 by rolling back a failed transaction. An airplane has some 
 tricks to recover
 from failures, but still it's a "no fail" design you argue 
 against: it shuts
 down parts one by one when and only when they fail and 
 continues operation no
 matter what until nothing works and even then it still doesn't 
 fail, just does
 nothing. The airplane example works against your arguments.
This is a serious misunderstanding of what I'm talking about. Again, on an airplane, no way in hell is a software system going to be allowed to continue operating after it has self-detected a bug. Trying to bend the imprecise language I use into meaning the opposite doesn't change that.
To better depict the big picture as I see it: You suggest that a system should shutdown as soon as possible on first sign of failure, which can affect the system. You provide the hospital in a hurricane example. But you don't praise the hospitals, which shutdown on failure, you praise the hospital, which continues to operate in face of an unexpected and uncontrollable disaster in total contradiction with your suggestion to shutdown ASAP. You refer to airplane's ability to not shutdown ASAP and continue operation on unexpected failure as if it corresponds to your suggestion to shutdown ASAP. This makes no sense, you contradict yourself. Why didn't you praise hospital shutdown? Why nobody wants airplanes to dive into ocean on first suspicion? Because that's how unreliable systems work: they often stop working. And reliable systems work in a completely different way, they employ many tricks, but one big objective of these tricks is to have ability to continue operation on failure. All the effort put into airplane design with one reason: to fight against immediate shutdown, defended by you as the only true way of operation. Exactly the way explicitly rejected by real reliable systems design. How an airplane without the tricks would work? It would dive into ocean on first failure (and crash investigation team diagnoses the failure) - exactly as you suggest. That's safe: it could fall on a city or a nuclear reactor. How a real airplane works? Failure happens and it still flies, contrary to your suggestion to shutdown on failure. That's how critical missions are done: they take a risk of a greater disaster to complete the mission, and failures can be diagnosed when appropriate. That's why I think your examples contradict to your proposal.
Oct 31 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Oct 31, 2014 at 08:15:17PM +0000, Kagamin via Digitalmars-d wrote:
 On Thursday, 16 October 2014 at 19:53:42 UTC, Walter Bright wrote:
On 10/15/2014 12:19 AM, Kagamin wrote:
Sure, software is one part of an airplane, like a thread is a part
of a process.  When the part fails, you discard it and continue
operation. In software it works by rolling back a failed
transaction. An airplane has some tricks to recover from failures,
but still it's a "no fail" design you argue against: it shuts down
parts one by one when and only when they fail and continues
operation no matter what until nothing works and even then it still
doesn't fail, just does nothing. The airplane example works against
your arguments.
This is a serious misunderstanding of what I'm talking about. Again, on an airplane, no way in hell is a software system going to be allowed to continue operating after it has self-detected a bug. Trying to bend the imprecise language I use into meaning the opposite doesn't change that.
To better depict the big picture as I see it: You suggest that a system should shutdown as soon as possible on first sign of failure, which can affect the system. You provide the hospital in a hurricane example. But you don't praise the hospitals, which shutdown on failure, you praise the hospital, which continues to operate in face of an unexpected and uncontrollable disaster in total contradiction with your suggestion to shutdown ASAP. You refer to airplane's ability to not shutdown ASAP and continue operation on unexpected failure as if it corresponds to your suggestion to shutdown ASAP. This makes no sense, you contradict yourself.
You are misrepresenting Walter's position. His whole point was that once a single component has detected a consistency problem within itself, it can no longer be trusted to continue operating and therefore must be shutdown. That, in turn, leads to the conclusion that your system design must include multiple, redundant, independent modules that perform that one function. *That* is the real answer to system reliability. Pretending that a failed component can somehow fix itself is a fantasy. The only way you can be sure you are not making the problem worse is by having multiple redundant units that can perform each other's function. Then when one of the units is known to be malfunctioning, you turn it off and fallback to one of the other, known-to-be-good, components. T -- Error: Keyboard not attached. Press F1 to continue. -- Yoon Ha Lee, CONLANG
Oct 31 2014
parent reply "Kagamin" <spam here.lot> writes:
On Friday, 31 October 2014 at 20:33:54 UTC, H. S. Teoh via 
Digitalmars-d wrote:
 You are misrepresenting Walter's position. His whole point was 
 that once
 a single component has detected a consistency problem within 
 itself, it
 can no longer be trusted to continue operating and therefore 
 must be
 shutdown. That, in turn, leads to the conclusion that your 
 system design
 must include multiple, redundant, independent modules that 
 perform that
 one function. *That* is the real answer to system reliability.
In server software such component is a transaction/request. They are independent.
 Pretending that a failed component can somehow fix itself is a 
 fantasy.
Traditionally a failed transaction is indeed rolled back. It's more a business logic requirement because a partially completed operation would confuse the user.
Oct 31 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Oct 31, 2014 at 09:11:53PM +0000, Kagamin via Digitalmars-d wrote:
 On Friday, 31 October 2014 at 20:33:54 UTC, H. S. Teoh via Digitalmars-d
 wrote:
You are misrepresenting Walter's position. His whole point was that
once a single component has detected a consistency problem within
itself, it can no longer be trusted to continue operating and
therefore must be shutdown. That, in turn, leads to the conclusion
that your system design must include multiple, redundant, independent
modules that perform that one function. *That* is the real answer to
system reliability.
In server software such component is a transaction/request. They are independent.
You're using a different definition of "component". An inconsistency in a transaction is a problem with the input, not a problem with the program logic itself. If something is wrong with the input, the program can detect it and recover by aborting the transaction (rollback the wrong data). But if something is wrong with the program logic itself (e.g., it committed the transaction instead of rolling back when it detected a problem) there is no way to recover within the program itself.
Pretending that a failed component can somehow fix itself is a
fantasy.
Traditionally a failed transaction is indeed rolled back. It's more a business logic requirement because a partially completed operation would confuse the user.
Again, you're using a different definition of "component". A failed transaction is a problem with the data -- this is recoverable to some extent (that's why we have the ACID requirement of databases, for example). For this purpose, you vet the data before trusting that it is correct. If the data verification fails, you reject the request. This is why you should never use assert to verify data -- assert is for checking the program's own consistency, not for checking the validity of data that came from outside. A failed component, OTOH, is a problem with program logic. You cannot recover from that within the program itself, since its own logic has been compromised. You *can* rollback the wrong changes made to data by that malfunctioning program, of course, but the rollback must be done by a decoupled entity outside of that program. Otherwise you might end up causing even more problems (for example, due to the compromised / malfunctioning logic, the program commits the data instead of reverting it, thus turning an intermittent problem into a permanent one). T -- By understanding a machine-oriented language, the programmer will tend to use a much more efficient method; it is much closer to reality. -- D. Knuth
Oct 31 2014
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/31/2014 2:31 PM, H. S. Teoh via Digitalmars-d wrote:
 On Fri, Oct 31, 2014 at 09:11:53PM +0000, Kagamin via Digitalmars-d wrote:
 On Friday, 31 October 2014 at 20:33:54 UTC, H. S. Teoh via Digitalmars-d
 wrote:
 You are misrepresenting Walter's position. His whole point was that
 once a single component has detected a consistency problem within
 itself, it can no longer be trusted to continue operating and
 therefore must be shutdown. That, in turn, leads to the conclusion
 that your system design must include multiple, redundant, independent
 modules that perform that one function. *That* is the real answer to
 system reliability.
In server software such component is a transaction/request. They are independent.
You're using a different definition of "component". An inconsistency in a transaction is a problem with the input, not a problem with the program logic itself. If something is wrong with the input, the program can detect it and recover by aborting the transaction (rollback the wrong data). But if something is wrong with the program logic itself (e.g., it committed the transaction instead of rolling back when it detected a problem) there is no way to recover within the program itself.
 Pretending that a failed component can somehow fix itself is a
 fantasy.
Traditionally a failed transaction is indeed rolled back. It's more a business logic requirement because a partially completed operation would confuse the user.
Again, you're using a different definition of "component". A failed transaction is a problem with the data -- this is recoverable to some extent (that's why we have the ACID requirement of databases, for example). For this purpose, you vet the data before trusting that it is correct. If the data verification fails, you reject the request. This is why you should never use assert to verify data -- assert is for checking the program's own consistency, not for checking the validity of data that came from outside. A failed component, OTOH, is a problem with program logic. You cannot recover from that within the program itself, since its own logic has been compromised. You *can* rollback the wrong changes made to data by that malfunctioning program, of course, but the rollback must be done by a decoupled entity outside of that program. Otherwise you might end up causing even more problems (for example, due to the compromised / malfunctioning logic, the program commits the data instead of reverting it, thus turning an intermittent problem into a permanent one).
This is a good summation of the situation.
Oct 31 2014
prev sibling next sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 31 October 2014 at 21:33:22 UTC, H. S. Teoh via 
Digitalmars-d wrote:
 You're using a different definition of "component". An 
 inconsistency in
 a transaction is a problem with the input, not a problem with 
 the
 program logic itself. If something is wrong with the input, the 
 program
 can detect it and recover by aborting the transaction (rollback 
 the
 wrong data).
Transactions roll back when there is contention for resources and/or when you have any kind of integrity issue. That's why you have retries… so no, it is not only something wrong with the input. Something is temporarily wrong with the situation overall.
Oct 31 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/31/2014 5:38 PM, "Ola Fosheim Grøstad" 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 Transactions roll back when there is contention for resources and/or when you
 have any kind of integrity issue. That's why you have retries… so no, it is
not
 only something wrong with the input. Something is temporarily wrong with the
 situation overall.
Those are environmental errors, not programming bugs, and asserting for those conditions is the wrong approach.
Oct 31 2014
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Saturday, 1 November 2014 at 03:39:02 UTC, Walter Bright wrote:
 Those are environmental errors, not programming bugs, and 
 asserting for those conditions is the wrong approach.
The point is this: what happens in the transaction engine matters, what happens outside of it does not matter much. Asserts do not belong in release code at all...
Oct 31 2014
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Friday, 31 October 2014 at 21:33:22 UTC, H. S. Teoh via 
Digitalmars-d wrote:
 Again, you're using a different definition of "component".
I see no justified reasoning why process can be considered such "component" ad anything else cannot. In practice it is completely dependent on system design as a whole and calling process a silver bullet only creates problems when it is in fact not.
Nov 01 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Nov 01, 2014 at 09:38:23AM +0000, Dicebot via Digitalmars-d wrote:
 On Friday, 31 October 2014 at 21:33:22 UTC, H. S. Teoh via Digitalmars-d
 wrote:
Again, you're using a different definition of "component".
I see no justified reasoning why process can be considered such "component" ad anything else cannot. In practice it is completely dependent on system design as a whole and calling process a silver bullet only creates problems when it is in fact not.
I never said "component" == "process". All I said was that at the OS level, at least with current OSes, processes are the smallest unit that is decoupled from each other. If you go below that level of granularity, you have the possibility of shared memory being corrupted by one thread (or fibre, or whatever smaller than a process) affecting the other threads. So that means they are not fully decoupled, and the failure of one thread makes all other threads no longer trustworthy. Obviously, you can go up to larger units than just processes when designing your system, as long as you can be sure they are decoupled from each other. T -- "No, John. I want formats that are actually useful, rather than over-featured megaliths that address all questions by piling on ridiculous internal links in forms which are hideously over-complex." -- Simon St. Laurent on xml-dev
Nov 01 2014
next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Saturday, 1 November 2014 at 15:02:53 UTC, H. S. Teoh via 
Digitalmars-d wrote:
 I never said "component" == "process". All I said was that at 
 the OS
 level, at least with current OSes, processes are the smallest 
 unit
 that is decoupled from each other. If you go below that level of
 granularity, you have the possibility of shared memory being 
 corrupted
 by one thread (or fibre, or whatever smaller than a process) 
 affecting
 the other threads. So that means they are not fully decoupled, 
 and the
 failure of one thread makes all other threads no longer 
 trustworthy.
This is a question of probability and impact. If my Python program fails unexpectedly, then it could in theory be a bug in a c-library, but it probably is not. So it is better to trap it and continue. If D provides bound checks, is a solid language, has a solid compiler, has a solid runtime, and solid libraries… then the same logic applies! If my C program traps on divison by zero, then it probably is an unlucky incident and not a memory corruption issue. So it is probably safe to continue. If my program cannot find a file, it MIGHT be a kernel issue, but it probably isn't. So it safe to continue. If my critical state is recorded by a wall built on transactions or full blown event logging, then it is safe to continue even if my front might suffer from memory corruption. You need to consider: 1. probability (what is the most likely cause of this signal?) 2. impact (do you have insurance?) 3. alternatives (are you in the middle of an air fight?)
Nov 01 2014
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Saturday, 1 November 2014 at 15:02:53 UTC, H. S. Teoh via 
Digitalmars-d wrote:
 I never said "component" == "process". All I said was that at 
 the OS
 level, at least with current OSes, processes are the smallest 
 unit
 that is decoupled from each other.
Which is exactly the statement I call wrong. With current OSes processes aren't decoupled units at all - it is all about feature set you stick to. Same with any other units.
 If you go below that level of
 granularity, you have the possibility of shared memory being 
 corrupted
 by one thread (or fibre, or whatever smaller than a process) 
 affecting
 the other threads.
You already have that possibility at process level via shared process memory and kernel mode code. And you still don't have that possibility at thread/fiber level if you don't use mutable shared memory (or any global state in general). It is all about system design. Pretty much only reliably decoupled units I can imagine are processes running in different restricted virtual machines (or, better, different physical machines). Everything else gives just certain level of expectations. Walter has experience with certain types of systems where process is indeed most appropriate unit of granularity and calls that a silver bullet by explicitly designing language in a way that makes any other approach inherently complicated and effort-consuming. But there is more than that in software world.
Nov 02 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/2/2014 3:48 AM, Dicebot wrote:
 On Saturday, 1 November 2014 at 15:02:53 UTC, H. S. Teoh via Digitalmars-d
wrote:
 Which is exactly the statement I call wrong. With current OSes processes aren't
 decoupled units at all - it is all about feature set you stick to. Same with
any
 other units.
They have hardware protection against sharing memory between processes. It's a reasonable level of protection.
 If you go below that level of
 granularity, you have the possibility of shared memory being corrupted
 by one thread (or fibre, or whatever smaller than a process) affecting
 the other threads.
You already have that possibility at process level via shared process memory
1. very few processes use shared memory 2. those that do should regard it as input/environmental, and not trust it
 and kernel mode code.
Kernel mode code is the responsibility of the OS system, not the app.
 And you still don't have that possibility at thread/fiber
 level if you don't use mutable shared memory (or any global state in general).
A buffer overflow will render all that protection useless.
 It is all about system design.
It's about the probability of coupling and the level of that your system can stand. Process level protection is adequate for most things.
 Pretty much only reliably decoupled units I can imagine are processes running
in
 different restricted virtual machines (or, better, different physical
machines).
 Everything else gives just certain level of expectations.
Everything is coupled at some level. Again, it's about the level of reliability needed.
 Walter has experience with certain types of systems where process is indeed
most
 appropriate unit of granularity and calls that a silver bullet by explicitly
 designing language
I design the language to do what it can. A language cannot compensate for coupling and bugs in the operating system, nor can a language compensate for two machines being plugged into the same power circuit.
 in a way that makes any other approach inherently complicated
 and effort-consuming.
Using enforce is neither complicated nor effort consuming. The idea that asserts can be recovered from is fundamentally unsound, and makes D unusable for robust critical software. Asserts are for checking for programming bugs. A bug can be tripped because of a buffer overflow, memory corruption, a malicious code injection attack, etc. NO CODE CAN BE RELIABLY EXECUTED PAST THIS POINT. Running arbitrary cleanup code at this point is literally undefined behavior. This is not a failure of language design - no language can offer any guarantees about this. If you want code cleanup to happen, use enforce(). If you are using enforce() to detect programming bugs, well, that's your choice. enforce() isn't any more complicated or effort-consuming than using assert().
Nov 02 2014
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Sunday, 2 November 2014 at 17:53:45 UTC, Walter Bright wrote:
 On 11/2/2014 3:48 AM, Dicebot wrote:
 On Saturday, 1 November 2014 at 15:02:53 UTC, H. S. Teoh via 
 Digitalmars-d wrote:
 Which is exactly the statement I call wrong. With current OSes 
 processes aren't
 decoupled units at all - it is all about feature set you stick 
 to. Same with any
 other units.
They have hardware protection against sharing memory between processes. It's a reasonable level of protection.
reasonable default - yes reasoable level of protection in general - no
 1. very few processes use shared memory
 2. those that do should regard it as input/environmental, and 
 not trust it
This is no different from: 1. very few threads use shared 2. those that do should regard is as input/environmental
 and kernel mode code.
Kernel mode code is the responsibility of the OS system, not the app.
In some (many?) large scale server systems OS is the app or at least heavily integrated. Thinking about app as a single independent user-space process is a bit.. outdated.
 And you still don't have that possibility at thread/fiber
 level if you don't use mutable shared memory (or any global 
 state in general).
A buffer overflow will render all that protection useless.
Nice we have safe and default thread-local memory!
 It is all about system design.
It's about the probability of coupling and the level of that your system can stand. Process level protection is adequate for most things.
Again, I am fine with advocating it as a resonable default. What frustrates me is intentionally making any other design harder than it should be by explicitly allowing normal cleanup to be skipped. This behaviour is easy to achieve by installing custom assert handler (it could be generic Error handler too) but impossible to bail out when it is the default one. Because of abovementioned avoiding more corruption from cleanup does not sound to me as strong enough benefit to force that on everyone. Ironically in system with decent fault protection and safety redundance it won't even matter (everything it can possibly corrupt is duplicated and proof-checked anyway)
 Walter has experience with certain types of systems where 
 process is indeed most
 appropriate unit of granularity and calls that a silver bullet 
 by explicitly
 designing language
I design the language to do what it can. A language cannot compensate for coupling and bugs in the operating system, nor can a language compensate for two machines being plugged into the same power circuit.
I don't expect you to do magic. My blame is about making decisions that support designs you have great expertise with but hamper something different (but still very real) - decisions that are usually uncharacteristic in D (which I believe is non-opinionated language) and don't really belong to system programming language.
 in a way that makes any other approach inherently complicated
 and effort-consuming.
Using enforce is neither complicated nor effort consuming.
 If you want code cleanup to happen, use enforce(). If you are 
 using enforce() to detect programming bugs, well, that's your 
 choice. enforce() isn't any more complicated or 
 effort-consuming than using assert().
I don't have other choice and I don't like it. It is effort consuming because it requires manually maintained exception hierarchy and style rules to keep errors different from exceptions - something that language otherwise provides to you out of the box. And there is always that 3d party library that is hard-coded to throw Error. It is not something that I realistically expect to change in D and there are specific plans for working with it (thanks for helping with it btw!). Just mentioning it as one of few D design decisions I find rather broken conceptually.
 The idea that asserts can be recovered from is fundamentally 
 unsound, and makes D unusable for robust critical software.
Not "recovered" but "terminate user-defined portion of the system".
 Asserts are for checking for programming bugs. A bug can be 
 tripped because of a buffer overflow, memory corruption, a 
 malicious code injection attack, etc.

 NO CODE CAN BE RELIABLY EXECUTED PAST THIS POINT.
As I have already mentioned it almost never can be truly reliable. You simply call one higher reliability chance good enough and other lower one - disastrous. I don't agree this is the language call to make, even if decision is reasonable and fitting 90% cases. This is really no different than GC usage in Phobos before nogc push. If language decision may result in fundamental code base fragmentation (even for relatively small portion of users), it is likely to be overly opinionated decision.
 Running arbitrary cleanup code at this point is literally 
 undefined behavior. This is not a failure of language design - 
 no language can offer any guarantees about this.
Some small chance of undefined behaviour vs 100% chance of resource leaks? Former can be more practical in many cases. And if it isn't for specific application one can always install custom assert handler that kills program right away. Don't see a deal breaker here.
Nov 02 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/2/2014 3:44 PM, Dicebot wrote:
 They have hardware protection against sharing memory between processes. It's a
 reasonable level of protection.
reasonable default - yes reasoable level of protection in general - no
No language can help when that is the requirement.
 1. very few processes use shared memory
 2. those that do should regard it as input/environmental, and not trust it
This is no different from: 1. very few threads use shared 2. those that do should regard is as input/environmental
It is absolutely different because of scale; having 1K of shared memory is very different from having 100Mb shared between processes including the stack and program code.
 and kernel mode code.
Kernel mode code is the responsibility of the OS system, not the app.
In some (many?) large scale server systems OS is the app or at least heavily integrated. Thinking about app as a single independent user-space process is a bit.. outdated.
Haha, I've used such a system (MSDOS) for many years. Switching to process protection was a huge advance. Sad that we're "modernizing" by reverting to such an awful programming environment.
 And you still don't have that possibility at thread/fiber
 level if you don't use mutable shared memory (or any global state in general).
A buffer overflow will render all that protection useless.
Nice we have safe and default thread-local memory!
Assert is to catch program bugs that should never happen, not correctly functioning programs. Nor can D possibly guarantee that called C functions are safe.
 It is all about system design.
It's about the probability of coupling and the level of that your system can stand. Process level protection is adequate for most things.
Again, I am fine with advocating it as a resonable default. What frustrates me is intentionally making any other design harder than it should be by explicitly allowing normal cleanup to be skipped. This behaviour is easy to achieve by installing custom assert handler (it could be generic Error handler too) but impossible to bail out when it is the default one.
Running normal cleanup code when the program is in an undefined, possibly corrupted, state can impede proper shutdown.
 Because of abovementioned avoiding more corruption from cleanup does not sound
 to me as strong enough benefit to force that on everyone.
I have considerable experience with what programs can do when continuing to run after a bug. This was on real mode DOS, which infamously does not seg fault on errors. It's AWFUL. I've had quite enough of having to reboot the operating system after every failure, and even then that often wasn't enough because it might scramble the disk driver code so it won't even boot. I got into the habit of layering in asserts to stop the program when it went bad. "Do not pass go, do not collect $200" is the only strategy that has a hope of working under such systems.
 I don't expect you to do magic. My blame is about making decisions that support
 designs you have great expertise with but hamper something different (but still
 very real) - decisions that are usually uncharacteristic in D (which I believe
 is non-opinionated language) and don't really belong to system programming
 language.
It is my duty to explain how to use the features of the language correctly, including how and why they work the way they do. The how, why, and best practices are not part of a language specification.
 I don't have other choice and I don't like it. It is effort consuming because
it
 requires manually maintained exception hierarchy and style rules to keep errors
 different from exceptions - something that language otherwise provides to you
 out of the box. And there is always that 3d party library that is hard-coded to
 throw Error.

 It is not something that I realistically expect to change in D and there are
 specific plans for working with it (thanks for helping with it btw!). Just
 mentioning it as one of few D design decisions I find rather broken
conceptually.
I hope to eventually change your mind about it being broken.
 NO CODE CAN BE RELIABLY EXECUTED PAST THIS POINT.
As I have already mentioned it almost never can be truly reliable.
That's correct, but not a justification for making it less reliable.
 You simply
 call one higher reliability chance good enough and other lower one -
disastrous.
 I don't agree this is the language call to make, even if decision is reasonable
 and fitting 90% cases.
If D changes assert() to do unwinding, then D will become unusable for building reliable systems until I add in yet another form of assert() that does not.
 This is really no different than GC usage in Phobos before  nogc push. If
 language decision may result in fundamental code base fragmentation (even for
 relatively small portion of users), it is likely to be overly opinionated
decision.
The reason I initiated this thread is to point out the correct way to use assert() and to get that into the culture of best practices for D. This is because if I don't, then in the vacuum people will tend to fill that vacuum with misunderstandings and misuse. It is an extremely important topic.
 Running arbitrary cleanup code at this point is literally undefined behavior.
 This is not a failure of language design - no language can offer any
 guarantees about this.
Some small chance of undefined behaviour vs 100% chance of resource leaks?
If the operating system can't handle resource recovery for a process terminating, it is an unusable operating system.
Nov 02 2014
next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 3 November 2014 at 03:29:05 UTC, Walter Bright wrote:
 I have considerable experience with what programs can do when 
 continuing to run after a bug. This was on real mode DOS, which 
 infamously does not seg fault on errors.

 It's AWFUL. I've had quite enough of having to reboot the 
 operating system after every failure, and even then that often 
 wasn't enough because it might scramble the disk driver code so 
 it won't even boot.
Yep. Fun stuff. http://research.microsoft.com/en-us/people/mickens/thenightwatch.pdf
 I got into the habit of layering in asserts to stop the program 
 when it went bad. "Do not pass go, do not collect $200" is the 
 only strategy that has a hope of working under such systems.
The tough thing is that in D, contracts serve as this early warning system, but the errors this system catches are also sometimes benign and sometimes the programmer knows that they're benign, but if the person writing the test did so with an assert then the decision of how to respond to the error has been made for him.
 It is my duty to explain how to use the features of the 
 language correctly, including how and why they work the way 
 they do. The how, why, and best practices are not part of a 
 language specification.
I don't entirely agree. The standard library serves as the how and how and best practices for a language, and while a programmer can go against this, it's often like swimming upstream. For better or worse, we need to establish how parameters are validated and such in Phobos, and this will serve as the template for nearly all code written in D. The core design goal of Druntime is to make the default behavior as safe as possible, but to allow to discerning user to override this behavior in certain key places. I kind of see the entire D language like this--it has the power of C/C++ but the ease of use of a much higher level language. We should strive for all aspects of the language and standard library to have the same basic behavior: a safe, efficient default but the ability to customize in key areas to meet individual needs. The trick is determining what these key areas are and how much latitude the user should have.
 If D changes assert() to do unwinding, then D will become 
 unusable for building reliable systems until I add in yet 
 another form of assert() that does not.
To be fair, assert currently does unwinding. It always has. The proposal is that it should not.
 The reason I initiated this thread is to point out the correct 
 way to use assert() and to get that into the culture of best 
 practices for D. This is because if I don't, then in the vacuum 
 people will tend to fill that vacuum with misunderstandings and 
 misuse.

 It is an extremely important topic.
I still feel like there's something really important here that we're all grasping at but it hasn't quite come to the fore yet. Along the lines of the idea that a safe program may be able to recover from a logic error. It seems like a uniquely D thing insofar as systems languages are concerned.
 If the operating system can't handle resource recovery for a 
 process terminating, it is an unusable operating system.
There are all kinds of resources, and not all of them are local to the system. Everything will eventually recover though, it just won't happen immediately as is the case with resource cleanup within a process.
Nov 02 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/2/2014 8:54 PM, Sean Kelly wrote:
 On Monday, 3 November 2014 at 03:29:05 UTC, Walter Bright wrote:
 I got into the habit of layering in asserts to stop the program when it went
 bad. "Do not pass go, do not collect $200" is the only strategy that has a
 hope of working under such systems.
The tough thing is that in D, contracts serve as this early warning system, but the errors this system catches are also sometimes benign
If it is benign is not known until it is debugged by a human.
 and sometimes the
 programmer knows that they're benign, but if the person writing the test did so
 with an assert then the decision of how to respond to the error has been made
 for him.
The person who wrote "assert" decided that it was a non-recoverable programming bug. I deliberately wrote "bug", and not "error".
 It is my duty to explain how to use the features of the language correctly,
 including how and why they work the way they do. The how, why, and best
 practices are not part of a language specification.
I don't entirely agree. The standard library serves as the how and how and best practices for a language, and while a programmer can go against this, it's often like swimming upstream. For better or worse, we need to establish how parameters are validated and such in Phobos, and this will serve as the template for nearly all code written in D.
Definitely Phobos should exhibit best practices. Whether bad function argument values are input/environmental errors or bugs is decidable only on a case-by-case basis. There is no overarching rule. Input/environmental errors must not use assert to detect them.
 To be fair, assert currently does unwinding.  It always has.  The proposal is
 that it should not.
Not entirely - a function with only asserts in it is considered "nothrow" and callers may not have exception handlers for them.
 The reason I initiated this thread is to point out the correct way to use
 assert() and to get that into the culture of best practices for D. This is
 because if I don't, then in the vacuum people will tend to fill that vacuum
 with misunderstandings and misuse.

 It is an extremely important topic.
I still feel like there's something really important here that we're all grasping at but it hasn't quite come to the fore yet. Along the lines of the idea that a safe program may be able to recover from a logic error. It seems like a uniquely D thing insofar as systems languages are concerned.
It's a false hope. D cannot offer any guarantees of recovery from programming bugs. Asserts, by definition, can never happen. So when they do, something is broken. Broken programs are not recoverable because one cannot know why they broke until they are debugged. As I mentioned to Dicebot, safe only applies to the function's logic. D programs can call C functions. C functions are not safe. There can be compiler bugs. There can be other threads corrupting memory. There can be hardware failures, operating system bugs, etc., that resulting in tripping the assert. If a programmer "knows" a bug is benign and wants to recover from it, D has a mechanism for it - enforce(). I do not understand the desire to bash assert() into behaving like enforce(). Just use enforce() in the first place. The idea was brought up that one may be using a library that uses assert() to detect input/environmental errors. I do not understand using a library in a system that must be made robust, having the source code to the library, and not being allowed to change that source code to fix bugs in it. A robust application cannot be made using such a library - assert() misuse will not be the only problem with it.
 If the operating system can't handle resource recovery for a process
 terminating, it is an unusable operating system.
There are all kinds of resources, and not all of them are local to the system. Everything will eventually recover though, it just won't happen immediately as is the case with resource cleanup within a process.
I'd say that is a poor design for an operating system. Be that as it may, if you want to recover from assert()s, use enforce() instead. There are other consequences from trying to make assert() recoverable: 1. functions with assert()s cannot be "nothrow" 2. assert()s cannot provide hints to the optimizer Those are high prices to pay for a systems performance language.
Nov 02 2014
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 3 November 2014 at 03:29:05 UTC, Walter Bright wrote:
 On 11/2/2014 3:44 PM, Dicebot wrote:
 They have hardware protection against sharing memory between 
 processes. It's a
 reasonable level of protection.
reasonable default - yes reasoable level of protection in general - no
No language can help when that is the requirement.
Yes, because it is property of system architecture as a whole which is exactly what I am speaking about.
 It is absolutely different because of scale; having 1K of 
 shared memory is very different from having 100Mb shared 
 between processes including the stack and program code.
It is possible to have minimal amount shared mutable memory inside one process. There is nothing inherently blocking one to do so, same as there is nothing inherently preventing one to screw the inter-process shared memory. Being different only because of scale -> not really different.
 Kernel mode code is the responsibility of the OS system, not 
 the app.
In some (many?) large scale server systems OS is the app or at least heavily integrated. Thinking about app as a single independent user-space process is a bit.. outdated.
Haha, I've used such a system (MSDOS) for many years. Switching to process protection was a huge advance. Sad that we're "modernizing" by reverting to such an awful programming environment.
What is huge advance for user land applciation is a problem for server code. Have you ever heard "OS is the problem, not solution" slogan that is slowly becoming more popular in high load networking world?
 It is all about system design.
It's about the probability of coupling and the level of that your system can stand. Process level protection is adequate for most things.
Again, I am fine with advocating it as a resonable default. What frustrates me is intentionally making any other design harder than it should be by explicitly allowing normal cleanup to be skipped. This behaviour is easy to achieve by installing custom assert handler (it could be generic Error handler too) but impossible to bail out when it is the default one.
Running normal cleanup code when the program is in an undefined, possibly corrupted, state can impede proper shutdown.
Preventing cleanup can be done with roughly one line of code from user code. Enabling it back is effectively impossible. With this decision you don't trade safer default for more dangerous default - you trade configurable default for unavoidable. To preserve same safe defaults you could define all thrown Errors to result in plain HLT / abort call with possibility to define user handler that actually throws. That would have addressed all concernc nicely while still not making life of those who want cleanup harder.
 Because of abovementioned avoiding more corruption from 
 cleanup does not sound
 to me as strong enough benefit to force that on everyone.
I have considerable experience with what programs can do when continuing to run after a bug. This was on real mode DOS, which infamously does not seg fault on errors. It's AWFUL. I've had quite enough of having to reboot the operating system after every failure, and even then that often wasn't enough because it might scramble the disk driver code so it won't even boot.
I don't argue necessity to terminate the program. I argue strict relation "program == process" which is impractical and inflexible.
 It is my duty to explain how to use the features of the 
 language correctly, including how and why they work the way 
 they do. The how, why, and best practices are not part of a 
 language specification.
You can't just explain things to make them magically appropriate for user domain. I fully understand how you propose to design applications. Unfortunately, it is completely unacceptable in some cases and quite inconvenient in others. Right now your proposal is effectively "design applications like me or reimplement language / library routines yourself".
 NO CODE CAN BE RELIABLY EXECUTED PAST THIS POINT.
As I have already mentioned it almost never can be truly reliable.
That's correct, but not a justification for making it less reliable.
It is justification for making it more configurable.
 If D changes assert() to do unwinding, then D will become 
 unusable for building reliable systems until I add in yet 
 another form of assert() that does not.
My personal perfect design would be like this: - Exceptions work as they do now - Errors work the same way as exceptions but don't get caught by catch(Exception) - assert does not throw Error but simply aborts the program (configurable with druntime callback) - define "die" which is effectively "assert(false)" - tests don't use assert That would provide default behaviour similar to one we currently have (with all the good things) but leave much more configurable choices for system designer.
 Some small chance of undefined behaviour vs 100% chance of 
 resource leaks?
If the operating system can't handle resource recovery for a process terminating, it is an unusable operating system.
There are many unusable operating systems out there then :) And don't forget about remote network resources - while leak will eventually timeout there it will still have a negative impact.
Nov 09 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/9/2014 1:12 PM, Dicebot wrote:
 On Monday, 3 November 2014 at 03:29:05 UTC, Walter Bright wrote:
 It is absolutely different because of scale; having 1K of shared memory is
 very different from having 100Mb shared between processes including the stack
 and program code.
It is possible to have minimal amount shared mutable memory inside one process.
D's type system tries to minimize it, but the generated code knows nothing at all about the difference between local and shared memory, and has no protection against crossing the boundary. Interprocess protection is done via the hardware.
 There is nothing inherently blocking one to do so, same as there is nothing
 inherently preventing one to screw the inter-process shared memory. Being
 different only because of scale -> not really different.
Sharing 1K of interprocess memory is one millionth of the vulnerability surface of a 1G multithreaded program.
 What is huge advance for user land applciation is a problem for server code.
 Have you ever heard "OS is the problem, not solution" slogan that is slowly
 becoming more popular in high load networking world?
No, but my focus is what D can provide, not what the OS can provide.
 Preventing cleanup can be done with roughly one line of code from user code.
 Enabling it back is effectively impossible. With this decision you don't trade
 safer default for more dangerous default - you trade configurable default for
 unavoidable.

 To preserve same safe defaults you could define all thrown Errors to result in
 plain HLT / abort call with possibility to define user handler that actually
 throws. That would have addressed all concernc nicely while still not making
 life of those who want cleanup harder.
There is already a cleanup solution - use enforce().
 That's correct, but not a justification for making it less reliable.
It is justification for making it more configurable.
In general, some things shouldn't be configurable. For example, one cannot mix 3rd party libraries when each one is trying to customize global behavior.
 My personal perfect design would be like this:

 - Exceptions work as they do now
 - Errors work the same way as exceptions but don't get caught by
catch(Exception)
 - assert does not throw Error but simply aborts the program (configurable with
 druntime callback)
 - define "die" which is effectively "assert(false)"
 - tests don't use assert
Having assert() not throw Error would be a reasonable design choice.
Nov 09 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Sunday, 9 November 2014 at 21:44:53 UTC, Walter Bright wrote:
 Having assert() not throw Error would be a reasonable design 
 choice.
What if you could turn assert() in libraries into enforce() using a compiler switch? Servers should be able to record failure and free network resources/locks even on fatal failure.
Nov 12 2014
next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On the other hand I guess HLT will signal SIGSEGV which can be 
caught using a signal handler, but then D should provide the 
OS-specific infrastructure for obtaining the necessary 
information before exiting.
Nov 12 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/12/2014 11:40 AM, "Ola Fosheim Grøstad" 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 On Sunday, 9 November 2014 at 21:44:53 UTC, Walter Bright wrote:
 Having assert() not throw Error would be a reasonable design choice.
What if you could turn assert() in libraries into enforce() using a compiler switch?
Forgive me for being snarky, but there are text editing utilities where one can: s/assert/enforce/ because if one can use a compiler switch, then one has the source which can be edited. In any case, compiler switches should not change behavior like that. assert() and enforce() are completely different.
Nov 12 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 12 November 2014 at 20:40:45 UTC, Walter Bright 
wrote:
 Forgive me for being snarky, but there are text editing 
 utilities where one can:
Snarky is ok. :)
 In any case, compiler switches should not change behavior like 
 that. assert() and enforce() are completely different.
Well, but I don't understand how assert() can unwind the stack if everyone should assume that the stack might be trashed and therefore invalid? In order to be consistent it with your line of reasoning it should simply HLT, then a SIGSEGV handler should set up a preallocated stack, obtain the information and send it off to a logging service using pure system calls before terminating (or send it to the parent process).
Nov 12 2014
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 12 November 2014 at 20:52:28 UTC, Ola Fosheim 
Grøstad wrote:
 In order to be consistent it with your line of reasoning it 
 should simply HLT, then a SIGSEGV handler should set up a 
 preallocated stack, obtain the information and send it off to a 
 logging service using pure system calls before terminating (or 
 send it to the parent process).
Btw, in C you should get SIGABRT on assert()
Nov 12 2014
prev sibling parent "Kagamin" <spam here.lot> writes:
On Saturday, 1 November 2014 at 16:42:31 UTC, Walter Bright wrote:
 My ideas are what are implemented on airplanes.
For components, not for a system. Nobody said a word against component design, it's systems that people want to be able to design, yet you prohibit it.
 I didn't originate these ideas, they come from the aviation 
 industry.
You're original in claiming it is the only working solution, but aviation industry proves error resilient systems are possible and successful, even though you claim their design is unsound and unusable. Yet you praise them, acknowledging their success, which makes your claims ever so ironical.
 Recall that I was employed as an engineer working on flight 
 critical systems design for the 757.
This is how problem decomposition works: you don't need to understand the whole system to work on a component. On Sunday, 2 November 2014 at 17:53:45 UTC, Walter Bright wrote:
 Kernel mode code is the responsibility of the OS system, not 
 the app.
Suddenly safety becomes not the top priority. If it can't always be the priority, there should be a choice of priorities, but you deny that choice. It's a matter of compliance with reality. Whatever way you design the language, can you change reality that way? I don't see why possibility of choice prevents anything.
Nov 11 2014
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Friday, 31 October 2014 at 21:33:22 UTC, H. S. Teoh via 
Digitalmars-d wrote:
 You're using a different definition of "component".
System granularity is decided by the designer. You either allow people design their systems or force your design on them, if you do both, you contradict yourself.
 An inconsistency in a transaction is a problem with the input, 
 not a problem with the program logic itself.
Distinction between failures doesn't matter. Reliable system manages any failures, especially unexpected and unforeseen ones, without diagnostic.
 If something is wrong with the input, the program
 can detect it and recover by aborting the transaction (rollback 
 the
 wrong data). But if something is wrong with the program logic 
 itself
 (e.g., it committed the transaction instead of rolling back 
 when it
 detected a problem) there is no way to recover within the 
 program
 itself.
Not the case for an airplane: it recovers from any failure within itself. Another indication that an airplane example contradicts to Walter's proposal. See my post about the big picture.
 A failed component, OTOH, is a problem with program logic. You 
 cannot
 recover from that within the program itself, since its own 
 logic has
 been compromised. You *can* rollback the wrong changes made to 
 data by
 that malfunctioning program, of course, but the rollback must 
 be done by
 a decoupled entity outside of that program. Otherwise you might 
 end up
 causing even more problems (for example, due to the compromised 
 /
 malfunctioning logic, the program commits the data instead of 
 reverting
 it, thus turning an intermittent problem into a permanent one).
No misunderstanding, I think that Walter's idea is good, just not always practical, and that real critical systems don't work the way he describes, they make more complicated tradeoffs.
Nov 01 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/1/2014 3:35 AM, Kagamin wrote:
 No misunderstanding, I think that Walter's idea is good, just not always
 practical, and that real critical systems don't work the way he describes, they
 make more complicated tradeoffs.
My ideas are what are implemented on airplanes. I didn't originate these ideas, they come from the aviation industry. Recall that I was employed as an engineer working on flight critical systems design for the 757.
Nov 01 2014
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 28/09/2014 23:00, Walter Bright wrote:
 I can't get behind the notion of "reasonably certain". I certainly would
 not use such techniques in any code that needs to be robust,
On 29/09/2014 04:04, Walter Bright wrote:
 I know I'm hardcore and uncompromising on this issue, but that's where I
 came from (the aviation industry).
Walter, you do understand that not all software has to be robust - in the critical systems sense - to be quality software? And that in fact, the majority of software is not critical systems software?... I was under the impression that D was meant to be a general purpose language, not a language just for critical systems. Yet, on language design issues, you keep making a series or arguments and points that apply *only* to critical systems software. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Oct 01 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/1/2014 6:17 AM, Bruno Medeiros wrote:
 Walter, you do understand that not all software has to be robust -
Yes, I understand that.
 in the critical systems sense - to be quality software? And that in fact, the
majority
 of software is not critical systems software?...

 I was under the impression that D was meant to be a general purpose language,
 not a language just for critical systems. Yet, on language design issues, you
 keep making a series or arguments and points that apply *only* to critical
 systems software.
If someone writes non-robust software, D allows them to do that. However, I won't leave unchallenged attempts to pass such stuff off as robust. Nor will I accept such practices in Phobos, because, as this thread clearly shows, there are a lot of misunderstandings about what robust software is. Phobos needs to CLEARLY default towards solid, robust practice. It's really too bad that I've never seen any engineering courses on reliability. http://www.drdobbs.com/architecture-and-design/safe-systems-from-unreliable-parts/228701716
Oct 04 2014
next sibling parent "Piotrek" <p nonexistent.pl> writes:
On Saturday, 4 October 2014 at 08:39:47 UTC, Walter Bright wrote:

 It's really too bad that I've never seen any engineering 
 courses on reliability.

 http://www.drdobbs.com/architecture-and-design/safe-systems-from-unreliable-parts/228701716
Thanks Walter. I was going to ask you about papers :) Maybe we need a to mention the key things in D guideline or in the future book: "Effective D". Piotrek
Oct 04 2014
prev sibling next sibling parent reply "Joseph Rushton Wakeling" <joseph.wakeling webdrake.net> writes:
On Saturday, 4 October 2014 at 08:39:47 UTC, Walter Bright wrote:
 If someone writes non-robust software, D allows them to do 
 that. However, I won't leave unchallenged attempts to pass such 
 stuff off as robust.

 Nor will I accept such practices in Phobos, because, as this 
 thread clearly shows, there are a lot of misunderstandings 
 about what robust software is. Phobos needs to CLEARLY default 
 towards solid, robust practice.
Would it help to clarify my intentions in this discussion if I said that, on this note, I entirely agree -- and nothing I have said in this discussion is intended to be an argument about how Phobos should be designed?
Oct 04 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/4/2014 4:25 AM, Joseph Rushton Wakeling wrote:
 Would it help to clarify my intentions in this discussion if I said that, on
 this note, I entirely agree -- and nothing I have said in this discussion is
 intended to be an argument about how Phobos should be designed?
Yes. Thank you!
Oct 04 2014
prev sibling next sibling parent reply Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 04/10/14 10:39, Walter Bright via Digitalmars-d wrote:
 If someone writes non-robust software, D allows them to do that. However, I
 won't leave unchallenged attempts to pass such stuff off as robust.

 Nor will I accept such practices in Phobos, because, as this thread clearly
 shows, there are a lot of misunderstandings about what robust software is.
 Phobos needs to CLEARLY default towards solid, robust practice.
A practical question that occurs to me here. Suppose that I implement, in D, a framework creating Erlang-style processes (i.e. properly isolated, lightweight processes within a defined runtime environment, with an appropriate error-handling framework that allows those processes to be brought down and restarted without bringing down the entire application). Is there any reasonable scope for accessing Phobos directly from programs written to operate within that runtime, or is it going to be necessary to wrap all of Phobos in order to ensure that it's accessed in a safe way (e.g. to ensure that the conditions required of in contracts are enforced before the call gets to phobos, etc.)?
Oct 04 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/4/2014 6:36 AM, Joseph Rushton Wakeling via Digitalmars-d wrote:
 Suppose that I implement, in D, a framework creating Erlang-style processes
 (i.e. properly isolated, lightweight processes within a defined runtime
 environment, with an appropriate error-handling framework that allows those
 processes to be brought down and restarted without bringing down the entire
 application).

 Is there any reasonable scope for accessing Phobos directly from programs
 written to operate within that runtime, or is it going to be necessary to wrap
 all of Phobos in order to ensure that it's accessed in a safe way (e.g. to
 ensure that the conditions required of in contracts are enforced before the
call
 gets to phobos, etc.)?
A start to this would be to ensure that the erlang-style processes only call pure functions. Then I'd add pervasive use of immutable data structures. This should help a lot.
Oct 04 2014
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 04/10/2014 09:39, Walter Bright wrote:
 On 10/1/2014 6:17 AM, Bruno Medeiros wrote:
 Walter, you do understand that not all software has to be robust -
Yes, I understand that.
 in the critical systems sense - to be quality software? And that in
 fact, the majority
 of software is not critical systems software?...

 I was under the impression that D was meant to be a general purpose
 language,
 not a language just for critical systems. Yet, on language design
 issues, you
 keep making a series or arguments and points that apply *only* to
 critical
 systems software.
If someone writes non-robust software, D allows them to do that. However, I won't leave unchallenged attempts to pass such stuff off as robust. Nor will I accept such practices in Phobos, because, as this thread clearly shows, there are a lot of misunderstandings about what robust software is. Phobos needs to CLEARLY default towards solid, robust practice. It's really too bad that I've never seen any engineering courses on reliability. http://www.drdobbs.com/architecture-and-design/safe-systems-from-unreliable-parts/228701716
Well, I set myself a trap to get that response... Of course, I too want my software to be robust! I doubt that anyone would disagree that Phobos should be designed to be as robust as possible. But "robust" is too general of a term to be precise here, so this belies my original point. I did say robust-in-the-critical-systems-sense... What I was questioning was whether D and Phobos should be designed in a way that took critical systems software as its main use, and relegate the other kinds of software to secondary importance. (Note: I don't think such dichotomy and compromise *has* to exist in order to design a great D and Phobos. But in this discussion I feel the choices and vision were heading in a way that would likely harm the development of general purpose software in favor of critical systems.) -- Bruno Medeiros https://twitter.com/brunodomedeiros
Oct 08 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/8/2014 8:06 AM, Bruno Medeiros wrote:
 (Note: I don't think such dichotomy and compromise *has* to exist in order to
 design a great D and Phobos. But in this discussion I feel the choices and
 vision were heading in a way that would likely harm the development of general
 purpose software in favor of critical systems.)
I definitely believe that Phobos should aid the implementation of critical systems over sloppy ones. But I don't believe that harms sloppy ones.
Oct 09 2014
prev sibling next sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 28/09/2014 00:15, Walter Bright wrote:
 This issue comes up over and over, in various guises. I feel like
 Yosemite Sam here:

 https://www.youtube.com/watch?v=hBhlQgvHmQ0

 In that vein, Exceptions are for either being able to recover from
 input/environmental errors, or report them to the user of the application.

 When I say "They are NOT for debugging programs", I mean they are NOT
 for debugging programs.
This is incorrect. Yes, the primary purpose of Exceptions is not for debugging, but to report exceptional state to the user (or some other component of the system). But they also have a purpose for debugging, particularly the stack traces of exceptions. Take what you said: "Failure to respond properly to an input/environmental error is a bug. But the input/environmental error is not a bug. If it was, then the program should assert on the error, not throw. " So, some component/function Foo detects an environmental error, and throws and Exception, accordingly. Foo is not responsible for handling these errors, but some other component is. Component/function Bar is the one that should handle such an error (for example, it should display a dialog to the user, and continue the application). But due to a bug, it doesn't do so, and the Exception goes all the way through main(). The programmer notices this happening, and clearly recognizes it's a bug (but doesn't know where the bug is, doesn't know that it's Bar that should be handling it). Now, what is best, to just have the Exception message (something like "File not found") displayed to the programmer - or even an end-user that could report a bug -, or to have the stack trace of the Exception so that the programmer can more easily look at which function should be handling it? -- Bruno Medeiros https://twitter.com/brunodomedeiros
Oct 01 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/1/2014 6:44 AM, Bruno Medeiros wrote:
 This is incorrect.

 Yes, the primary purpose of Exceptions is not for debugging, but to report
 exceptional state to the user (or some other component of the system).

 But they also have a purpose for debugging, particularly the stack traces of
 exceptions. Take what you said:

 "Failure to respond properly to an input/environmental error is a bug.
 But the input/environmental error is not a bug. If it was, then the
 program should assert on the error, not throw. "

 So, some component/function Foo detects an environmental error, and throws and
 Exception, accordingly. Foo is not responsible for handling these errors, but
 some other component is.

 Component/function Bar is the one that should handle such an error (for
example,
 it should display a dialog to the user, and continue the application). But due
 to a bug, it doesn't do so, and the Exception goes all the way through main().
This is like saying "if statements are for debugging".
 The programmer notices this happening, and clearly recognizes it's a bug (but
 doesn't know where the bug is, doesn't know that it's Bar that should be
 handling it). Now, what is best, to just have the Exception message (something
 like "File not found") displayed to the programmer - or even an end-user that
 could report a bug -, or to have the stack trace of the Exception so that the
 programmer can more easily look at which function should be handling it?
Would you agree that every time DMD reports a syntax error in user code, it should also include a stack trace of the DMD source code to where in DMD it reported the error?
Oct 04 2014
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Sat, 04 Oct 2014 01:43:40 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 Would you agree that every time DMD reports a syntax error in user code, it 
 should also include a stack trace of the DMD source code to where in DMD it 
 reported the error?
Of course not. DMD is a compiler and it is part of its normal operation to have sophisticated code to report syntax errors. I distinguish between "expected" exceptions and "unexpected" exceptions. Expected exceptions are those that occur due to validating user input or as part of handling documented error codes at the interface layer with external APIs. Unexpected exceptions are those we don't handle explicitly because we don't expect them. Maybe because we think that code wont throw, since the input had been validated or a bug in an external library or laziness. A FileNotFoundException can be the expected result of an "open file" dialog with the user typing the name of a non-existent file or the unexpected result of loading a static asset with an index that is off-by-one (e.g. icon<idx>.bmp). In the case of DMD, syntax errors are *expected* as part of validating user input. And that's why it prints a single line that can be parsed by IDEs to jump the the source. Now to make things interesting, we also see *unexpected* exceptions in DMD: internal compiler errors. While expected exceptions are commonly handled with a simple message, unexpected exceptions are handled depending on application. In non-interactive applications like DMD they can all be AssertErrors that terminate the program. In interactive software like a video editor, someone might just have spent hours on editing and the user should be allowed to reason about the severity of the fault and decide to ignore the exception or quit the program. There are other solutions like logging and emailing the error to someone. The example above, about the off-by-one when reading an icon is a typical case of an exception that you would want to investigate with an exception stack trace, but keep the program running. -- Marco
Oct 05 2014
prev sibling next sibling parent "rst256" <ussr.24 yandex.ru> writes:
https://en.wikipedia.org/wiki/Code_smell
Do you read this?

Yes phobos.stdio very smell code. Disagree?
107 КБ of source code olny for call
few functions from stdio. Are you sure that
this code are full correctly?
"silly rabbit, y should be positive" - may
be becose his used class like this?

the funny thing is that this code can easily
be written machine based on a few rules and header.

Say that do you think about this code model
File("output"){
...
some file operation
...
}else{
...
optional if defined handle error
...
}
...

this not class instance, is macrodef.
And "output" is not a file, this
external resurce. His may binded to
commandline arg (-output), static file
  and gui(just set asign property in
design form editor). May be get from
network.
That is cool, about this i can say:
"Take it to bank". Gui on macros
like this may create very quick.

File declare as subject not class
(eg in json format) you anyway build it
from exists function.
example:
File = {
export: [ export list ]
blocks: {
main:{},
else: {}
},

}
Firshtein?
see https://en.wikipedia.org/wiki/Subject-oriented_programming
how this make on jscript https://github.com/jonjamz/amethyst
Oct 22 2014
prev sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 9/27/14, 8:15 PM, Walter Bright wrote:
 This issue comes up over and over, in various guises. I feel like
 Yosemite Sam here:

      https://www.youtube.com/watch?v=hBhlQgvHmQ0

 In that vein, Exceptions are for either being able to recover from
 input/environmental errors, or report them to the user of the application.

 When I say "They are NOT for debugging programs", I mean they are NOT
 for debugging programs.

 assert()s and contracts are for debugging programs.
Here's another +1 for exceptions. I want to add a a slash command to Slack (https://slack.zendesk.com/hc/en-us/articles/201259356-Slash-Commands). So, for example, when I say: /bot random phrase This hits a web server that processes that request and returns a random phrase. Now, imagine I have an assert in my application. When the web server hits the assertion it shuts down and the user doesn't get a response. What I'd like to do is to trap that assertion, tell the user that there's a problem, and send me an email telling me to debug it and fix it. That way the user can continue using the bot and I meanwhile I can fix the bug. In the real world where you don't want unhappy users, asserts don't work. Walter: how can you do that with an assertion triggering?
Oct 24 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Oct 24, 2014 at 03:29:43PM -0300, Ary Borenszweig via Digitalmars-d
wrote:
 On 9/27/14, 8:15 PM, Walter Bright wrote:
This issue comes up over and over, in various guises. I feel like
Yosemite Sam here:

     https://www.youtube.com/watch?v=hBhlQgvHmQ0

In that vein, Exceptions are for either being able to recover from
input/environmental errors, or report them to the user of the application.

When I say "They are NOT for debugging programs", I mean they are NOT
for debugging programs.

assert()s and contracts are for debugging programs.
Here's another +1 for exceptions. I want to add a a slash command to Slack (https://slack.zendesk.com/hc/en-us/articles/201259356-Slash-Commands). So, for example, when I say: /bot random phrase This hits a web server that processes that request and returns a random phrase. Now, imagine I have an assert in my application. When the web server hits the assertion it shuts down and the user doesn't get a response. What I'd like to do is to trap that assertion, tell the user that there's a problem, and send me an email telling me to debug it and fix it. That way the user can continue using the bot and I meanwhile I can fix the bug. In the real world where you don't want unhappy users, asserts don't work. Walter: how can you do that with an assertion triggering?
Sure they do. Your application should be running in a separate process from the webserver itself. The webserver receives a request and forwards it to the application process. The application process processes the request and sends the response back to the webserver, which forwards it back on the client socket. Meanwhile, the webserver also monitors the application process; if it crashes before producing a response, it steps in and sends a HTTP 500 response to the client instead. It can also email you about the bug, possibly with the stack trace of the crashed application process, etc.. (And before you complain about inefficiency, there *are* ways of eliminating copying overhead when forwarding requests/responses between the client and the application.) But if the webserver itself triggers an assertion, then it should NOT attempt to send anything back to the client, because the assertion may be indicating some kind of memory corruption or security exploit attempt. You don't know if you might be accidentally sending sensitive personal data (e.g. password for another user) back to the wrong client, because your data structures got scrambled and the wrong data is associated with the wrong client now. Basically, if you want a component to recover from a serious problem like a failed assertion, the recovery code should be in a *separate* component. Otherwise, if the recovery code is within the failing component, you have no way to know if the recovery code itself has been compromised, and trusting that it will do the right thing is very dangerous (and is what often leads to nasty security exploits). The watcher must be separate from the watched, otherwise how can you trust the watcher? T -- Why ask rhetorical questions? -- JC
Oct 24 2014
parent reply "Kagamin" <spam here.lot> writes:
On Friday, 24 October 2014 at 18:47:59 UTC, H. S. Teoh via 
Digitalmars-d wrote:
 Basically, if you want a component to recover from a serious 
 problem
 like a failed assertion, the recovery code should be in a 
 *separate*
 component. Otherwise, if the recovery code is within the failing
 component, you have no way to know if the recovery code itself 
 has been
 compromised, and trusting that it will do the right thing is 
 very
 dangerous (and is what often leads to nasty security exploits). 
 The
 watcher must be separate from the watched, otherwise how can 
 you trust
 the watcher?
You make process isolation sound like a silver bullet, but failure can happen on any scale from a temporary variable to global network. You can't use process isolation to contain a failure of a larger than process scale, and it's an overkill for a failure of a temporary variable scale.
Oct 31 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Oct 31, 2014 at 08:23:04PM +0000, Kagamin via Digitalmars-d wrote:
 On Friday, 24 October 2014 at 18:47:59 UTC, H. S. Teoh via Digitalmars-d
 wrote:
Basically, if you want a component to recover from a serious problem
like a failed assertion, the recovery code should be in a *separate*
component. Otherwise, if the recovery code is within the failing
component, you have no way to know if the recovery code itself has
been compromised, and trusting that it will do the right thing is
very dangerous (and is what often leads to nasty security exploits).
The watcher must be separate from the watched, otherwise how can you
trust the watcher?
You make process isolation sound like a silver bullet, but failure can happen on any scale from a temporary variable to global network. You can't use process isolation to contain a failure of a larger than process scale, and it's an overkill for a failure of a temporary variable scale.
You're missing the point. The point is that a reliable system made of unreliable parts, can only be reliable if you have multiple *redundant* copies of each component that are *decoupled* from each other. The usual unit of isolation at the lowest level is that of a single process, because threads within a process has full access to memory shared by all threads. Therefore, they are not decoupled from each other, and therefore, you cannot put any confidence in the correct functioning of other threads once a single thread has become inconsistent. The only failsafe solution is to have multiple redundant processes, so when one process becomes inconsistent, you fallback to another process, *decoupled* process that is known to be good. This does not mean that process isolation is a "silver bullet" -- I never said any such thing. The same reasoning applies to larger components in the system as well. If you have a server that performs function X, and the server begins to malfunction, you cannot expect the server to fix itself -- because you don't know if a hacker hasn't rooted the server and is running exploit code instead of your application. The only 100% safe way to recover, is to have another redundant server (or more) that also performs function X, shutdown the malfunctioning server for investigation and repair, and in the meantime switch over to the redundant server to continue operations. You don't shutdown the *entire* network unless all redundant components have failed. The reason you cannot go below the process level as a unit of redundancy is because of coupling. The above design of failing over to a redundant module only works if the modules are completely decoupled from each other. Otherwise, you have end up with the situation where you have two redundant modules M1 and M2, but both of them share a common helper module M3. Then if M1 detects a problem, you cannot be 100% sure it's not caused by a problem with M3, so in this case if you just switch to M2, it will also fail in the same way. Similarly, you cannot guarantee that while malfunctioning, M1 may have somehow damaged M3, and thereby also making M2 unreliable. The only way to be 100% sure that failover will actually fix the problem, is to make sure that M1 and M2 are completely isolated from each other (e.g., by having two redundant copies of M3 that are isolated from each other). Since a single process is the unit of isolation in the OS, you can't go below this granularity: as I've already said, if one thread is malfunctioning, it may have trashed the data shared by all other threads in the same process, and therefore none of the other threads can be trusted to continue operating correctly. The only way to be 100% sure that failover will actually fix the problem, is to switch over to another process that you *know* is not coupled to the old, malfunctioning process. Attempting to have a process "fix itself" after detecting an inconsistency is unreliable -- you're leaving it up to chance whether or not the attempted recovery will actually work, and not make the problem worse. You cannot guarantee the recovery code itself hasn't been compromised by the failure -- because the recovery code exists in the same process and is vulnerable to the same problem that caused the original failure, and vulnerable to memory corruption caused by malfunctioning code prior to the point the problem was detected. Therefore, the recovery code is not trustworthy, and cannot be relied on to continue operating correctly. That kind of "maybe, maybe not" recovery is not what I'd want to put any trust in, especially when it comes to critical applications that can cost lives if things go wrong. T -- English has the lovely word "defenestrate", meaning "to execute by throwing someone out a window", or more recently "to remove Windows from a computer and replace it with something useful". :-) -- John Cowan
Oct 31 2014
parent reply "Kagamin" <spam here.lot> writes:
On Friday, 31 October 2014 at 21:06:49 UTC, H. S. Teoh via 
Digitalmars-d wrote:
 This does not mean that process isolation is a "silver bullet" 
 -- I
 never said any such thing.
But made it sound that way:
 The only failsafe solution is to have multiple redundant
 processes, so when one process becomes inconsistent, you 
 fallback to
 another process, *decoupled* process that is known to be good.
If you think a hacker rooted the server, how do you know other perfectly isolated processes are good? Not to mention you suggested to build a system from *communicating* processes, which doesn't sound like perfect isolation at all.
 You don't shutdown the *entire* network unless all redundant 
 components have failed.
If you have a hacker in your network, the network is compromised and is in an unknown state, why do you want the network to continue operation? You contradict yourself.
Nov 01 2014
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Nov 01, 2014 at 10:52:31AM +0000, Kagamin via Digitalmars-d wrote:
 On Friday, 31 October 2014 at 21:06:49 UTC, H. S. Teoh via Digitalmars-d
 wrote:
This does not mean that process isolation is a "silver bullet" -- I
never said any such thing.
But made it sound that way:
The only failsafe solution is to have multiple redundant processes,
so when one process becomes inconsistent, you fallback to another
process, *decoupled* process that is known to be good.
If you think a hacker rooted the server, how do you know other perfectly isolated processes are good? Not to mention you suggested to build a system from *communicating* processes, which doesn't sound like perfect isolation at all.
You're confusing the issue. Process-level isolation is for detecting per-process faults. If you want to handle server-level faults, you need external monitoring per server, so that when it detects a possible exploit on one server, it shuts down the server and fails over to another server known to be OK. And I said decoupled, not isolated. Decoupled means they can still communicate with each other, but with a known protocol that insulates them from each other's faults. E.g. you don't send binary executable code over the communication lines and the receiving process blindly runs it, but you send data in a predefined format that is verified by the receiving party before acting on it. I'm pretty sure this is obvious.
You don't shutdown the *entire* network unless all redundant
components have failed.
If you have a hacker in your network, the network is compromised and is in an unknown state, why do you want the network to continue operation? You contradict yourself.
The only contradiction here is introduced by you. If one or two servers on your network have been compromised, does that mean the *entire* network is compromised? No it doesn't. It just means those one or two servers have been compromised. So you have monitoring tools setup to detect problems within the network and isolate the compromised servers. If you are no longer sure the entire network is in a good state, e.g. if your monitoring tools can't detect certain large-scale problems, then sure, go ahead and shutdown the entire network. It depends on what granularity you're operating at. A properly-designed reliable system needs to have multiple levels of monitoring and failover. You have process-level decoupling, server-level, network-level, etc.. You can't just rely on a single level of granularity and expect it to solve everything. T -- Leather is waterproof. Ever see a cow with an umbrella?
Nov 01 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/24/2014 11:29 AM, Ary Borenszweig wrote:
 On 9/27/14, 8:15 PM, Walter Bright wrote:
 Now, imagine I have an assert in my application. When the web server hits the
 assertion it shuts down and the user doesn't get a response. What I'd like to
do
 is to trap that assertion, tell the user that there's a problem, and send me an
 email telling me to debug it and fix it. That way the user can continue using
 the bot and I meanwhile I can fix the bug.
Don't need an exception for that. You can insert your own handler with core.assertHandler(myAssertHandler). Or you can catch(Error). But you don't want to try doing anything more than notification with that - the program is in an unknown state.
Oct 24 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Friday, 24 October 2014 at 19:09:23 UTC, Walter Bright wrote:
 You can insert your own handler with 
 core.assertHandler(myAssertHandler). Or you can catch(Error). 
 But you don't want to try doing anything more than notification 
 with that - the program is in an unknown state.
Also be aware that if you throw an Exception from the assertHandler you could be violating nothrow guarantees.
Oct 27 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 10/27/2014 1:54 PM, Sean Kelly wrote:
 On Friday, 24 October 2014 at 19:09:23 UTC, Walter Bright wrote:
 You can insert your own handler with core.assertHandler(myAssertHandler). Or
 you can catch(Error). But you don't want to try doing anything more than
 notification with that - the program is in an unknown state.
Also be aware that if you throw an Exception from the assertHandler you could be violating nothrow guarantees.
Right.
Oct 29 2014