www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Alternatives to exceptions for error handling

reply Roman Kashitsyn <romankashicin gmail.com> writes:
In the video from DConf 2020 about  live functions 
(https://youtu.be/XQHAIglE9CU) Walter mentioned that all  live 
functions are nothrow.  He also thinks that exceptions are 
obsolete.

I've been programming in Google's C++, Go & Rust for quite a few 
years and I couldn't agree more with this assessment.  Error 
handling is a very important part of program engineering, and 
having errors reflected in function return types is very useful 
in practice.  This helps a lot with constructing correct programs.

So my question is: does/will D provide support for alternative 
approaches to error handling?  In practice, this typically 
includes 2 things:

1. A sum type that contains either a result or an error. Like 
Result in Rust or Expected that Andrei proposed for C++.  It's 
easy to implement in D, but having this in Phobos would be nice.

2. Some kind of syntactic sugar for propagating errors upstream 
(? macro in Rust, check/handle proposal in Go 
https://go.googlesource.com/proposal/+/master/design/go2draf
-error-handling.md, an ugly macro in C++, etc.).
Nov 22 2020
next sibling parent Paulo Pinto <pjmlp progtools.org> writes:
On Sunday, 22 November 2020 at 17:37:18 UTC, Roman Kashitsyn 
wrote:

 In the video from DConf 2020 about  live functions 
 (https://youtu.be/XQHAIglE9CU) Walter mentioned that all  live 
 functions are nothrow.  He also thinks that exceptions are 
 obsolete.

 I've been programming in Google's C++, Go & Rust for quite a 
 few years and I couldn't agree more with this assessment.  
 Error handling is a very important part of program engineering, 
 and having errors reflected in function return types is very 
 useful in practice.  This helps a lot with constructing correct 
 programs.

 So my question is: does/will D provide support for alternative 
 approaches to error handling?  In practice, this typically 
 includes 2 things:

 1. A sum type that contains either a result or an error. Like 
 Result in Rust or Expected that Andrei proposed for C++.  It's 
 easy to implement in D, but having this in Phobos would be nice.

 2. Some kind of syntactic sugar for propagating errors upstream 
 (? macro in Rust, check/handle proposal in Go 
 https://go.googlesource.com/proposal/+/master/design/go2draf
-error-handling.md, an ugly macro in C++, etc.).


You won't need ugly macros in C++,

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf
Nov 22 2020
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Sunday, 22 November 2020 at 17:37:18 UTC, Roman Kashitsyn 
wrote:

 So my question is: does/will D provide support for alternative 
 approaches to error handling?  In practice, this typically 
 includes 2 things:

 1. A sum type that contains either a result or an error. Like 
 Result in Rust or Expected that Andrei proposed for C++.  It's 
 easy to implement in D, but having this in Phobos would be nice.


The biggest blocker for this is that D currently has no 

have a DIP in the works to address this:

https://github.com/dlang/DIPs/pull/193


 2. Some kind of syntactic sugar for propagating errors upstream 
 (? macro in Rust, check/handle proposal in Go 
 https://go.googlesource.com/proposal/+/master/design/go2draf
-error-handling.md, an ugly macro in C++, etc.).


So far the only proposal I've heard on this front is the idea of 
bringing Herb Sutter's deterministic exceptions [1] to D. But 
there are no concrete plans at this point, not even a DIP.

My expectation is that, like in Rust, the first thing to arrive 
will be monadic `map`/`flatMap` methods, since those can be 
implemented using only library code.

[1] 
http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p0709r0.pdf
Nov 22 2020
next sibling parent reply IGotD- <nise nise.com> writes:
On Sunday, 22 November 2020 at 18:21:44 UTC, Paul Backus wrote:

 The biggest blocker for this is that D currently has no 

 have a DIP in the works to address this:

 https://github.com/dlang/DIPs/pull/193


 2. Some kind of syntactic sugar for propagating errors 
 upstream (? macro in Rust, check/handle proposal in Go 
 https://go.googlesource.com/proposal/+/master/design/go2draf
-error-handling.md, an ugly macro in C++, etc.).


 So far the only proposal I've heard on this front is the idea 
 of bringing Herb Sutter's deterministic exceptions [1] to D. 
 But there are no concrete plans at this point, not even a DIP.

 My expectation is that, like in Rust, the first thing to arrive 
 will be monadic `map`/`flatMap` methods, since those can be 
 implemented using only library code.

 [1] 
 http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p0709r0.pdf


Doesn't the lean exceptions what Herb Sutter proposes have the 
same problem as normal exception, that the data flow analysis 
gives up. Anything that requires unwinding is not going to fit 
for that purpose.

Rust error handling does not rely on exceptions or unwinding but 
rather just a return type which is a tagged union. Essentially 
Rust error handling is like good old return values. The only 
difference is that compiler inserts code when the result type 
isn't handled or passed to the caller. I don't understand why 
nobody thought about that previously because it is kind of simple.

Any method that removes the dependency to libunwind gets my vote.
Nov 22 2020
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 23 November 2020 at 00:12:28 UTC, IGotD- wrote:

 On Sunday, 22 November 2020 at 18:21:44 UTC, Paul Backus wrote:

 [1] 
 http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p0709r0.pdf


 Doesn't the lean exceptions what Herb Sutter proposes have the 
 same problem as normal exception, that the data flow analysis 
 gives up. Anything that requires unwinding is not going to fit 
 for that purpose.


In Herb Sutter's proposal, throwing an exception is syntax sugar 
for a return statement. There is no stack unwinding.
Nov 22 2020
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Nov 23, 2020 at 12:22:10AM +0000, Paul Backus via Digitalmars-d wrote:
[...]

 In Herb Sutter's proposal, throwing an exception is syntax sugar for a
 return statement. There is no stack unwinding.


This is where I'd lean as well. Retain the same syntax, but change the
underlying implementation.


T

-- 
Let's eat some disquits while we format the biskettes.
Nov 22 2020
prev sibling next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 22 November 2020 at 18:21:44 UTC, Paul Backus wrote:

 The biggest blocker for this is that D currently has no 

 have a DIP in the works to address this:

 https://github.com/dlang/DIPs/pull/193


This is great to see. Thank you for working on this.

Another approach that I'm not sure we've explored fully is to 
outright deprecate silently discarding all function return 
values. Functions which return a value that is only sometimes 
useful seem like a relatively rare oddity. I took a stab at 
implementing this as a DScanner check earlier this year:

https://github.com/dlang-community/D-Scanner/pull/819

Perhaps the most interesting product of the above is seeing what 
things were affected in Phobos. Most occurrences were in unit 
tests, which is understandable, and said breakages are probably 
more tolerable. Some valid patterns were affected though, such as 
returning `this` (as in the builder pattern). Perhaps it would 
make more sense to instead allow annotating functions as that 
it's safe to discard their return value ( discardable or such)?
Nov 22 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 23 November 2020 at 02:14:33 UTC, Vladimir Panteleev 
wrote:

 Some valid patterns were affected though, such as returning 
 `this` (as in the builder pattern). Perhaps it would make more 
 sense to instead allow annotating functions as that it's safe 
 to discard their return value ( discardable or such)?


That might work. I also frequently return values on setters and 
that might get tricky as well.

 property int foo(int v) { return this.v = v; }

that kind of thing.

 discardable would be possible there too.

But then there's also generic forwarders:

struct Wrapped(T) {
     T t;
     auto opDispatch(string name, Args...)() { return 
t[name](args); }
}

you know. And if it wraps one of those builder style things, it 
should probably inherit the discardable.... but that's a big 
complication.

Odds are I'd actually just mark the whole wrapped thing 
 discardable anyway.

(and we can say this same thing about  nodiscard, but I think it 
is less important to forward that since it at least still 
compiles if it is not there.)


There's also some common C functions that could get annoying. 
memcpy, for example, unconditionally returns the destination 
pointer you passed in. Obviously we could just mark the binding 
 discardable too.


I currently lean toward  nodiscard as being easier to use but I 
do expect  discardable would work pretty well too, just with a 
more involved transition period.
Nov 22 2020
parent reply Dominikus Dittes Scherkl <dominikus scherkl.de> writes:
On Monday, 23 November 2020 at 02:27:22 UTC, Adam D. Ruppe wrote:

 On Monday, 23 November 2020 at 02:14:33 UTC, Vladimir Panteleev 
 wrote:

 Some valid patterns were affected though, such as returning 
 `this` (as in the builder pattern). Perhaps it would make more 
 sense to instead allow annotating functions as that it's safe 
 to discard their return value ( discardable or such)?





 I currently lean toward  nodiscard as being easier to use but I 
 do expect  discardable would work pretty well too, just with a 
 more involved transition period.


At least pure functions should never be  discardable.
So discarding the return value of a pure function should always 
be an error.

At my work we use the annotation "noSideeffects" (in C), which is 
pretty much the same as pure I think, except the poor syntax 
(need to repeat the function name in a separate macro line).
Depending on this an error is set if the return value is 
discarded. Works like a charm.
Nov 23 2020
parent Paul Backus <snarwin gmail.com> writes:
On Monday, 23 November 2020 at 12:07:55 UTC, Dominikus Dittes 
Scherkl wrote:

 At least pure functions should never be  discardable.
 So discarding the return value of a pure function should always 
 be an error.


Indeed, the D compiler already issues a warning if you discard 
the result of a pure nothrow function.
Nov 23 2020
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Monday, 23 November 2020 at 02:14:33 UTC, Vladimir Panteleev 
wrote:

 Another approach that I'm not sure we've explored fully is to 
 outright deprecate silently discarding all function return 
 values. Functions which return a value that is only sometimes 
 useful seem like a relatively rare oddity.


The main advantage of  nodiscard compared to this approach is 
that it can interact with the type system. For example, if I 
declare a struct like the following:

 nodiscard Result(T, E)
{
     SumType!(T, E) payload;
     // ...
}

...then every expression that evaluates to a Result (not just 
function calls!) automatically becomes non-discardable.
Nov 23 2020
prev sibling parent Elronnd <elronnd elronnd.net> writes:
On Sunday, 22 November 2020 at 18:21:44 UTC, Paul Backus wrote:

 nodiscard
 I  have a DIP in the works to address this:


A value which must be used at least once sounds very similar, at 
least conceptually, to a value which must be used exactly once; 
which is to say linear types, as implemented in  live.

It probably makes sense for the two should be integrated.
Nov 29 2020
prev sibling next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 22 November 2020 at 17:37:18 UTC, Roman Kashitsyn 
wrote:

 1. A sum type that contains either a result or an error.


I think it would also be good to have implicit construction of 
return value.

Sum!(Error, Result) foo() {
     return Error(0);
}

That's currently an error, you must explicitly mention the return 
type by name or by typeof return.

return typeof(return)(Error(0)); // works, but wordy

Of course you can alias the sum type,

alias EC!T = Sum!(Error, T);

EC!Result foo() { return EC!Result(r); }

but still.

So if the compiler would automatically insert that constructor 
call, when necessary, it'd probably make it a lot easier to use. 
You can still just return normal stuff syntax-wise.



 2. Some kind of syntactic sugar for propagating errors upstream 
 (? macro in Rust, check/handle proposal in Go 
 https://go.googlesource.com/proposal/+/master/design/go2draf
-error-handling.md, an ugly macro in C++, etc.).


Yes, the ? thing in rust is pretty cool.

I was thinking about this earlier today too and perhaps reusing 
the `try` or `catch` keywords could do it as well.

But what I'd like to do is have it work with user-defined types 
too, not just compiler magic. So it expands into something like:

// you write
auto r = foo()?; // or auto r = try foo(); or whatever syntax

// and it becomes:
auto tmp = foo();
if(auto error = tmp.opError())
     return error;

auto r = tmp.opValue();


Where the opError and opValue are just defined names, similar to 
operator overloading, that any type can define.


Combined with the above implicit return construction and other 
existing D features, this would enable a lot of things. For 
example:

---
struct null_is_error {
         void* ptr;
         int opError() { ptr is null ? .errno : 0; }
         auto opValue() { return ptr; }
}

import core.stdc.stdlib;

ErrnoOr!(void*) main() {
         auto p = malloc(5).null_is_error?; // note Rust-style ?
}
---

This expands into:

---
auto tmp = malloc(5).null_is_error; // UFCS ctor call of helper 
btw
if(auto err = tmp.opError())
     return err; // which implicit constructs into return 
typeof(return)(err);
auto p = tmp.opValue();
---

So we're able to paint on new features to old functions, adapt to 
a variety of error schemes (could even use classes like our 
current exceptions!) as needed for any user, not just a 
compiler-recognized magic type, and be able to write it 
reasonably conveniently.
Nov 22 2020
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On Sunday, 22 November 2020 at 17:37:18 UTC, Roman Kashitsyn 
wrote:

 In the video from DConf 2020 about  live functions 
 (https://youtu.be/XQHAIglE9CU) Walter mentioned that all  live 
 functions are nothrow.  He also thinks that exceptions are 
 obsolete.


I haven't watched the video yet, but in general I think we need 
some form of error handling that more or less have the same 
semantics as the existing one. There are other ways to implement 
what looks like exception. Just look at the error handling in 
Swift, Zig and the proposal for C++ [1]. I think without any form 
of language support and syntax sugar, error handling is going to 
be a pain.

[1] 
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf

--
/Jacob Carlborg
Nov 23 2020
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 23 November 2020 at 10:58:29 UTC, Jacob Carlborg wrote:

 On Sunday, 22 November 2020 at 17:37:18 UTC, Roman Kashitsyn 
 wrote:

 In the video from DConf 2020 about  live functions 
 (https://youtu.be/XQHAIglE9CU) Walter mentioned that all  live 
 functions are nothrow.  He also thinks that exceptions are 
 obsolete.


 I haven't watched the video yet, but in general I think we need 
 some form of error handling that more or less have the same 
 semantics as the existing one. There are other ways to 
 implement what looks like exception. Just look at the error 
 handling in Swift, Zig and the proposal for C++ [1]. I think 
 without any form of language support and syntax sugar, error 
 handling is going to be a pain.


To get better C++ interop the best solution is to do whatever C++ 
does and focus on the ability to catch C++ exceptions in D code 
and vice versa.
Nov 23 2020
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, November 23, 2020 3:58:29 AM MST Jacob Carlborg via Digitalmars-d 
wrote:

 On Sunday, 22 November 2020 at 17:37:18 UTC, Roman Kashitsyn

 wrote:

 In the video from DConf 2020 about  live functions
 (https://youtu.be/XQHAIglE9CU) Walter mentioned that all  live
 functions are nothrow.  He also thinks that exceptions are
 obsolete.


 I haven't watched the video yet, but in general I think we need
 some form of error handling that more or less have the same
 semantics as the existing one. There are other ways to implement
 what looks like exception. Just look at the error handling in
 Swift, Zig and the proposal for C++ [1]. I think without any form
 of language support and syntax sugar, error handling is going to
 be a pain.

 [1]
 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf


Yeah, there are plenty of times where using a solution other than exceptions
makes more sense, but in general, exceptions are by far the cleanest and
simplest way to deal with error conditions, and I would _really_ hate to see
D move away from them as the default error handling mechanism. Personally, I
really don't care much about how they're implemented underneath the hood,
but the ability to throw an error condition up to code that handles it
higher up, allowing all of the code in between to largely ignore the error
condition is huge, and a lot of code can get pretty disgusting without that.

- Jonathan M Davis
Nov 24 2020
parent reply Roman Kashitsyn <romankashicin gmail.com> writes:
On Tuesday, 24 November 2020 at 20:30:33 UTC, Jonathan M Davis 
wrote:

 On Monday, November 23, 2020 3:58:29 AM MST Jacob Carlborg via 
 Digitalmars-d wrote:

 On Sunday, 22 November 2020 at 17:37:18 UTC,


 the ability to throw an error condition up to code that handles 
 it higher up, allowing all of the code in between to largely 
 ignore the error condition is huge, and a lot of code can get 
 pretty disgusting without that.


It's mostly the question of values: writing nice looking code 
quickly vs writing code that handles all the edge cases 
thoroughly.  It's much more fun to write the first kind of code 
and it's perfectly justified in many cases.   Though as a user 
(of an app or a library) you would probably prefer the second 
kind of code.

Thoughtful error handling never looks good, and exceptions don't 
help with that at all: try/catch syntax is very heavyweight and 
encourages you to propagate errors instead of dealing with them 
where they arise.

Exceptions also make type signatures less precise: it's hard to 
be sure I handled all the errors I can recover from if I don't 
know which errors can actually occur.  There is no way to tell 
without inspecting the full transitive closure of the code I call 
into.

So it would be nice to have an alternative to exceptions even if 
it's not going to be the default mechanism.

Herb Sutter's proposal looks OK but it doesn't solve most of the 
problems I have with exceptions:
  1. The syntax for propagating errors is OK, but the syntax for 
*handling* them is still heavyweight.
  2. All the error types are mashed into std::error, so I still 
don't know which errors are possible at each call.
Nov 29 2020
next sibling parent Ola Fosheim Grostad <ola.fosheim.grostad gmail.com> writes:
On Sunday, 29 November 2020 at 18:17:02 UTC, Roman Kashitsyn 
wrote:

 Thoughtful error handling never looks good, and exceptions 
 don't help with that at all: try/catch syntax is very 
 heavyweight and encourages you to propagate errors instead of 
 dealing with them where they arise.


I have no proble with the syntax, but single inheritance is not 
so great.

One should be able to throw and catch a set of properties in some 
kind of pattern matching way. Maybe also inject handlers that are 
triggered by certain patterns at certain boundaries.


 Exceptions also make type signatures less precise: it's hard to 
 be sure I handled all the errors I can recover from if I don't 
 know which errors can actually occur.  There is no way to tell 
 without inspecting the full transitive closure of the code I 
 call into.


A good IDE should show this, so a tooling issue, not a language 
issue.
Nov 29 2020
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2020-11-29 19:17, Roman Kashitsyn wrote:


 It's mostly the question of values: writing nice looking code quickly vs 
 writing code that handles all the edge cases thoroughly.  It's much more 
 fun to write the first kind of code and it's perfectly justified in many 
 cases.   Though as a user (of an app or a library) you would probably 
 prefer the second kind of code.
 
 Thoughtful error handling never looks good, and exceptions don't help 
 with that at all: try/catch syntax is very heavyweight and encourages 
 you to propagate errors instead of dealing with them where they arise.


In many cases you don't know what to do with an error so you need to 
propagate it up the stack. For example, opening a file. If that fails, 
the function has no idea if that's a critical error or not. It might be 
a config file that is missing and the application cannot start without 
it. Or it might the that a user selected a file which it doesn't have 
permission to open.


 Exceptions also make type signatures less precise: it's hard to be sure 
 I handled all the errors I can recover from if I don't know which errors 
 can actually occur.  There is no way to tell without inspecting the full 
 transitive closure of the code I call into.
 
 So it would be nice to have an alternative to exceptions even if it's 
 not going to be the default mechanism.
 
 Herb Sutter's proposal looks OK but it doesn't solve most of the 
 problems I have with exceptions:
   1. The syntax for propagating errors is OK, but the syntax for 
 *handling* them is still heavyweight.
   2. All the error types are mashed into std::error, so I still don't 
 know which errors are possible at each call.


I have some ideas that I think solve most of these problems. With a 
syntax that (hopefully) won't alienate users. Perhaps I should write 
those down.

-- 
/Jacob Carlborg
Nov 29 2020
parent reply Roman Kashitsyn <romankashicin gmail.com> writes:
On Sunday, 29 November 2020 at 19:52:27 UTC, Jacob Carlborg wrote:

 In many cases you don't know what to do with an error so you 
 need to propagate it up the stack. For example, opening a file. 
 If that fails, the function has no idea if that's a critical 
 error or not. It might be a config file that is missing and the 
 application cannot start without it. Or it might the that a 
 user selected a file which it doesn't have permission to open.


Sure, it's not always possible to handle errors at the level they 
arise, those need to be propagated.
On the other hand, most of the time the caller has even less clue 
on how to deal with the error: the further you go from the point 
where error happened, the less likely it's going to be handled in 
a meaningful way.

Sometimes a crash with a core dump might be preferable to stack 
unwinding as the core dump might contain more useful information 
than a stack trace.

My main point is that it's really nice to see what can go wrong 
just from the function type signature, this transforms the way 
people (well, at least some of them) write software.
Checked exceptions in Java was a failed attempt to achieve this, 
they brought nothing but pain and boilerplate.
I believe sum types + pattern matching + error propagation sugar 
is a much nicer solution for errors that people might care to 
handle. Of course, Errors don't fall into this category.
Nov 29 2020
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 29 November 2020 at 23:20:13 UTC, Roman Kashitsyn 
wrote:

 Sometimes a crash with a core dump might be preferable to stack 
 unwinding as the core dump might contain more useful 
 information than a stack trace.


That's exactly what you get with an uncaught exception (remember 
the druntime catches exceptions by default, turn that off to get 
this behavior). One of my favorite parts of them and i'm not sure 
how a sumtype thing would compare. I guess you could set a 
conditional breakpoint.

I'm pro-exception but I also do think D should go ahead and move 
forward with making the return value type thing work better. If 
we do it right, the supporting features can be useful in other 
contexts too.
Nov 29 2020
prev sibling next sibling parent IGotD- <nise nise.com> writes:
On Sunday, 29 November 2020 at 23:20:13 UTC, Roman Kashitsyn 
wrote:

 I believe sum types + pattern matching + error propagation 
 sugar is a much nicer solution for errors that people might 
 care to handle. Of course, Errors don't fall into this category.


I think it can be too limiting, especially for large software 
projects. Exceptions or whatever is working underneath is usually 
better. Problem is that all sort can go wrong in a function. 
Often there is a memory allocation, which can go wrong. However, 
catching a memory allocation error almost never happens as 99% of 
the time people just assume it will work. This is where 
exceptions come handy as uncaught exceptions will abort the 
program and the programmers can start to debug the problem. One 
thing I sometimes see among badly run projects is that they catch 
all exceptions, like std::exception or (...) in C++ but the 
action will be wrong as they catch anything that they don't 
expect. So in the case of the memory allocation error, it will be 
handled the same as a bunch of other errors which is likely to be 
the wrong action.

Also for script like languages which D really can be. Exceptions 
are great as you don't care about error handling at all when you 
just mock up some script like program. If you get an exception 
the program aborts then you find the problem and fix it, then 
move on.

Uncaught exceptions often help find error that we don't expect 
and that the program aborts is also correct since we didn't know 
the error would happen and therefore didn't handle it.
Nov 29 2020
prev sibling next sibling parent reply Gregor =?UTF-8?B?TcO8Y2ts?= <gregormueckl gmx.de> writes:
On Sunday, 29 November 2020 at 23:20:13 UTC, Roman Kashitsyn 
wrote:

 On Sunday, 29 November 2020 at 19:52:27 UTC, Jacob Carlborg 
 wrote:

 [..]


 Sure, it's not always possible to handle errors at the level 
 they arise, those need to be propagated.
 On the other hand, most of the time the caller has even less 
 clue on how to deal with the error: the further you go from the 
 point where error happened, the less likely it's going to be 
 handled in a meaningful way.


What kind of error conditions are you talking about that you 
consider handleable locally? Do you have concrete examples? I am 
asking because this is way outside the experiences I have made 
regarding error handling and I would like to understand your 
perspective.

Every serious application that I have ever worked on had to deal 
with errors in the context of larger operations. There were 
essentially no locally handleable errors. So every error has to 
go up a few layers until there is even enough context available 
for recovery. This happens for example when an change operation 
triggers a sanity check deep inside a complex data model. The 
failing check has no notion of whether the larger operation needs 
to be rolled back or whether this is expected by the controller 
and there is a fallback strategy.
Nov 30 2020
next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 11/30/20 1:58 AM, Gregor M=C3=BCckl wrote:

 On Sunday, 29 November 2020 at 23:20:13 UTC, Roman Kashitsyn wrote:

 On Sunday, 29 November 2020 at 19:52:27 UTC, Jacob Carlborg wrote:

 [..]


 Sure, it's not always possible to handle errors at the level they=20
 arise, those need to be propagated.
 On the other hand, most of the time the caller has even less clue on=20
 how to deal with the error: the further you go from the point where=20
 error happened, the less likely it's going to be handled in a=20
 meaningful way.


=20
 What kind of error conditions are you talking about that you consider=20
 handleable locally? Do you have concrete examples? I am asking because =



 this is way outside the experiences I have made regarding error handlin=


g=20

 and I would like to understand your perspective.
=20
 Every serious application that I have ever worked on had to deal with=20
 errors in the context of larger operations. There were essentially no=20
 locally handleable errors. So every error has to go up a few layers=20
 until there is even enough context available for recovery. This happens=


=20

 for example when an change operation triggers a sanity check deep insid=


e=20

 a complex data model. The failing check has no notion of whether the=20
 larger operation needs to be rolled back or whether this is expected by=


=20

 the controller and there is a fallback strategy.
=20


That is exactly my experience as well.

My programs catch Exception type in main() to report a user friendly=20
message. (Error is not caught.) There are a couple of exception to this:

- I catch and ignore errors locally in non-essential operations like=20
printing verbose output or collecting timing statistics. (Nobody cares=20
if they failed and everybody is mad if they failed.)

- I catch formatting errors to augment the error because a higher level=20
may not understand a low level error of "cannot convert 'x' to int".

That's all... Exceptions are the simplest error management. For me,=20
their only problem is the performance impact, which I haven't even=20
measured. :)

The code is the cleanest with exceptions: enforce() and assert() checks=20
guarantee that everything is sane. If not, the operation is aborted.

Ali
Nov 30 2020
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 30 November 2020 at 11:02:29 UTC, Ali Çehreli wrote:

 That's all... Exceptions are the simplest error management. For 
 me, their only problem is the performance impact, which I 
 haven't even measured. :)


They are quite simple. The performance impact is simply because 
of a clumsy generic solution with expensive lookup.

But they are also a bit too simple. Like, if accessing external 
resources you often want to do retries. Annoying to throw all the 
way out.

Consider for instance if you try to fetch a file from an url, 
then it fails. It would have been nice to inject a recovery 
handler that can analyze the failure and provide a new url, sleep 
then retry etc.

e.g. something along the lines of this sketch:

fetch_url(url) {
   retry with (url) {
    …download attempt…
    …throw http_fail, server_busy…
   } catch (…){
    …ok cleanup, nobody wanted a retry…
   }
}


main(){
   on http_fail(url){
     url = replace_with_backup_server(url)
     return true; // retry
   }
   data = fetch_url(url)
   …
}
Nov 30 2020
parent reply Jacob Carlborg <doob me.com> writes:
On Monday, 30 November 2020 at 11:43:01 UTC, Ola Fosheim Grøstad 
wrote:


 But they are also a bit too simple. Like, if accessing external 
 resources you often want to do retries. Annoying to throw all 
 the way out.

 Consider for instance if you try to fetch a file from an url, 
 then it fails. It would have been nice to inject a recovery 
 handler that can analyze the failure and provide a new url, 
 sleep then retry etc.

 e.g. something along the lines of this sketch:

 fetch_url(url) {
   retry with (url) {
    …download attempt…
    …throw http_fail, server_busy…
   } catch (…){
    …ok cleanup, nobody wanted a retry…
   }
 }


 main(){
   on http_fail(url){
     url = replace_with_backup_server(url)
     return true; // retry
   }
   data = fetch_url(url)
   …
 }


In Ruby there's the `retry` keyword for this. Although you cannot 
control the recovery handler from the outside, like in your 
example. In D terms you would put `retry` in a `catch` block and 
it would run the code inside the `try` block again.

--
/Jacob Carlborg
Nov 30 2020
parent reply Andre Pany <andre s-e-a-p.de> writes:
On Monday, 30 November 2020 at 12:59:32 UTC, Jacob Carlborg wrote:

 On Monday, 30 November 2020 at 11:43:01 UTC, Ola Fosheim 
 Grøstad wrote:


 [...]


 In Ruby there's the `retry` keyword for this. Although you 
 cannot control the recovery handler from the outside, like in 
 your example. In D terms you would put `retry` in a `catch` 
 block and it would run the code inside the `try` block again.

 --
 /Jacob Carlborg


In java there is a nice library solution called fail safe: 
https://jodah.net/failsafe/

Something like that is missing in D (code.dlang.org).

Kind regards
Andre
Nov 30 2020
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 30 November 2020 at 16:02:32 UTC, Andre Pany wrote:

 On Monday, 30 November 2020 at 12:59:32 UTC, Jacob Carlborg 
 wrote:

 In Ruby there's the `retry` keyword for this. Although you 
 cannot control the recovery handler from the outside, like in 
 your example. In D terms you would put `retry` in a `catch` 
 block and it would run the code inside the `try` block again.




Interesting, I didn't know that.


 In java there is a nice library solution called fail safe: 
 https://jodah.net/failsafe/

 Something like that is missing in D (code.dlang.org).


Yes, maybe we could find ideas in high level frameworks.

I guess one simple solution with the current setup would be for 
the caller to provide a "progress object" which is used for 
tracking progress and has resolvers. When it totally fails the 
library can throw the progress-object which is configured to 
indicate the error... Kinda like NSError in Objective C, but more 
active.

But I think it is more forwardlooking to inject depencies based 
on call-tree analysis (interprocedural analysis). Then you can 
get static errors for wrong configuration at compile time.

Anyway, interesting topic. No clear right or wrong. The main 
problem is to get people to agree on protocols, which is why a 
language feature has some merits.
Nov 30 2020
prev sibling parent reply sighoya <sighoya gmail.com> writes:
On Monday, 30 November 2020 at 11:02:29 UTC, Ali Çehreli wrote:

That's all... Exceptions are the simplest error management.


+1.

Please leave exceptions as default and optionally introduce other 
ways for specific kind of work.


For me, their only problem is the performance impact, which I 
haven't even measured. :)


But exactly this is the way exceptions are defined, they should 
be exceptions, i.e. they should rarely pop up. With this in mind, 
code execution with exceptions is as fast as with no error 
handling at all, which isn't the case with Rust's error handling.

What you may want instead are signals which implicitly pass a 
pointer of possible continuations down the stack or maybe by just 
overwriting a global continuation variable to quickly jump in. 
They are interesting for cases where you expect an event to 
happen more than not.

Disadvantages are however:

- a small footprint registering landing pads (catch 
regions/continuations)
- don't fit that well with reference counted objects
Jan 05 2021
parent reply Max Haughton <maxhaton gmail.com> writes:
On Tuesday, 5 January 2021 at 17:59:41 UTC, sighoya wrote:

 On Monday, 30 November 2020 at 11:02:29 UTC, Ali Çehreli wrote:

That's all... Exceptions are the simplest error management.


 +1.

 Please leave exceptions as default and optionally introduce 
 other ways for specific kind of work.


For me, their only problem is the performance impact, which I 
haven't even measured. :)


 But exactly this is the way exceptions are defined, they should 
 be exceptions, i.e. they should rarely pop up. With this in 
 mind, code execution with exceptions is as fast as with no 
 error handling at all, which isn't the case with Rust's error 
 handling.

 What you may want instead are signals which implicitly pass a 
 pointer of possible continuations down the stack or maybe by 
 just overwriting a global continuation variable to quickly jump 
 in. They are interesting for cases where you expect an event to 
 happen more than not.

 Disadvantages are however:

 - a small footprint registering landing pads (catch 
 regions/continuations)
 - don't fit that well with reference counted objects


You'd have to seriously measure the overhead - exceptions *are* 
nominally zero-cost but compilers don't like optimising when they 
can be thrown because it massively complicates the control flow 
analysis. That's partly what walter was referring to in the first 
place.

At very least exceptions should not be the default for (say) 
parser errors because they aren't exceptional.
Jan 05 2021
parent sighoya <sighoya gmail.com> writes:
On Tuesday, 5 January 2021 at 18:18:27 UTC, Max Haughton wrote:

 You'd have to seriously measure the overhead - exceptions *are* 
 nominally zero-cost but compilers don't like optimising when 
 they can be thrown because it massively complicates the control 
 flow analysis. That's partly what walter was referring to in 
 the first place.


Okay, this is new to me, could you elaborate a bit more please?


 At very least exceptions should not be the default for (say) 
 parser errors because they aren't exceptional.


It depends, if you aren't interested in the content of the parsed 
file at all in case of a parser error, then exceptions might be 
the right choice. You throw them once, that's it.

If you are otherwise interested to gain anyway information from 
the file then errors in the return type make sense as they are 
now part of the considered information.

A good example are IDE compilers, where a parser error line- and 
column range is used for red marking the code. In addition, other 
places of parser errors have to be red marked after the same run.
Jan 05 2021
prev sibling parent reply Roman Kashitsyn <romankashicin gmail.com> writes:
On Monday, 30 November 2020 at 09:58:36 UTC, Gregor Mückl wrote:


 What kind of error conditions are you talking about that you 
 consider handleable locally? Do you have concrete examples? I 
 am asking because this is way outside the experiences I have 
 made regarding error handling and I would like to understand 
 your perspective.



Sure, let me give a couple of examples.

Imagine you are writing a step of a pipeline that needs to 
support backpressure.
When you push data downstream and the call fails because 
downstream is overloaded, a good recovery would be to start 
buffering data and propagate the failure upstream when the local 
buffer fills up.
If we terminate the whole pipeline each time we have a minor 
congestion, we'll never have anything done.


Another example: I was once implementing a distributed task 
execution service using Zookeeper.  It had scheduler processes 
and worker processes distributed across multiple DCs. Only one 
scheduler must be active at any time, and other scheduler 
instances wait in stand-by mode in case the leader dies or 
becomes partitioned.
First, Zookeeper API is callback-based, and throwing exceptions 
into the event-loop not controlled by your application makes 
little sense.  So we already need a different error handling 
mechanism for such cases.
Let's see then what happens if a node looses network connection 
for a short period of time.
For a worker it's not a problem at all: it should continue 
whenever it has been doing and wait for the network to appear 
again.  Tasks were mainly quite expensive to run, so aborting 
them was a bad idea.
A scheduler cannot operate without the network, so simply 
crashing would be an option. However, this involves an expensive 
recovery procedure, and short network disruptions happened very 
often.  The strategy I implemented was to schedule an action that 
retries whatever we were trying to do as soon as the network 
appears again (if this process is still a leader, otherwise 
crash).
Nov 30 2020
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Nov 30, 2020 at 06:48:05PM +0000, Roman Kashitsyn via Digitalmars-d
wrote:

 On Monday, 30 November 2020 at 09:58:36 UTC, Gregor Mckl wrote:
 

 What kind of error conditions are you talking about that you
 consider handleable locally? Do you have concrete examples? I am
 asking because this is way outside the experiences I have made
 regarding error handling and I would like to understand your
 perspective.


 
 
 Sure, let me give a couple of examples.
 
 Imagine you are writing a step of a pipeline that needs to support
 backpressure.
 When you push data downstream and the call fails because downstream is
 overloaded, a good recovery would be to start buffering data and
 propagate the failure upstream when the local buffer fills up.
 If we terminate the whole pipeline each time we have a minor
 congestion, we'll never have anything done.


Why would it terminate the whole pipeline each time there's a
congestion?  Catch the exception at the right point and run fallback
code, or whatever it is, then continue.  That's the whole reason 'catch'
exists.

Of course, aborting because of Errors is a different story. That's to do
with logic problems in the code, in which case terminating may be the
only way to recover properly.  To prevent terminating the whole pipeline
in that case, I'd think about partitioning the application into a master
thread/process and a number of workers, and the workers may abort upon
error but the master simply reschedules the work to another worker.



 Another example: I was once implementing a distributed task execution
 service using Zookeeper.  It had scheduler processes and worker
 processes distributed across multiple DCs. Only one scheduler must be
 active at any time, and other scheduler instances wait in stand-by
 mode in case the leader
 dies or becomes partitioned.
 First, Zookeeper API is callback-based, and throwing exceptions into
 the event-loop not controlled by your application makes little sense.
 So we already need a different error handling mechanism for such
 cases.


[...]

So wrap the entry point into the user code with a try/catch block, and
the catch block will propagate the error using whatever suitable channel
of error communication is (e.g., write a message to a error-handling
socket, or send a message to a controller then quit, etc.).  If you
like, you can even mark it nothrow to ensure any user code will always
catch the exception instead of letting it propagate into the event loop.

Just because you use exceptions, doesn't mean it has to be used
throughout the entire call stack indiscriminately.


T

-- 
It's bad luck to be superstitious. -- YHL
Nov 30 2020
prev sibling parent sighoya <sighoya gmail.com> writes:
On Sunday, 29 November 2020 at 23:20:13 UTC, Roman Kashitsyn 
wrote:

Checked exceptions in Java was a failed attempt to achieve this, 
they brought nothing but pain and boilerplate.
I believe sum types + pattern matching + error propagation sugar 
is a much nicer solution for errors that people might care to 
handle. Of course, Errors don't fall into this category.


They share anyway the same problem:

Deterministic error handling with finite enumerable error types.

It wasn't only the problem of penetrating error messages when 
dealing with non caught exception handling in Java, but also the 
pressure to up propagate all the newly created downstream 
exceptions requiring extending the throws section of all function 
signatures depending on it.

And Rust didn't improve anything regarding that, but no one would 
ever criticize that because its Rust.


On the other hand, most of the time the caller has even less 
clue on how to deal with the error: the further you go from the 
point where error happened, the less likely it's going to be 
handled in a meaningful way.


Yep, that happens but is generally bad practice. For this kind of 
problem chained exception are useful, because each caller is 
adding his context onto the error object, the last caller is then 
able providing a more meaningful error message based on the 
messages and types of errors before.
There is even no need to throw the nested exception though I like 
a more than few in this regard.
Maybe an option to print only the outermost exception per default?
Jan 05 2021
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Nov 24, 2020 at 01:30:33PM -0700, Jonathan M Davis via Digitalmars-d
wrote:

 On Monday, November 23, 2020 3:58:29 AM MST Jacob Carlborg via Digitalmars-d 
 wrote:


[...]

 [...] I think without any form of language support and syntax sugar,
 error handling is going to be a pain.




[...]

 [...] in general, exceptions are by far the cleanest and simplest way
 to deal with error conditions, and I would _really_ hate to see D move
 away from them as the default error handling mechanism.


I've said before, and I'll say again: I suspect that most (all?) of the
current objections against exceptions comes from the current
*implementation* of it, i.e., libunwind, et al..  There are other ways
to implement the concept of exceptions that can potentially avoid these
issues.

For example, instead of inserting stack frames and stack unwinding code,
just use a dedicated register or CPU flag that indicates an error in the
caller.  The callee detects this and branches to the cleanup code at the
end of the function.  Then try/throw/catch become essentially just
syntactic sugar for C-style error return codes.  The syntactic function
return value essentially behaves like an implicit out parameter that
conveys the return value.  Most complex C code is written more-or-less
along these lines anyway, so this isn't really anything new.

Interfacing with C/C++ code will be trickier, of course, but not
insurmountable. E.g., make it so that extern(C) will conform to the
usual ABI, but extern(D) will use the new function call convention. Then
native D code can benefit from this, while still be able to interface
with C code via an extern(C) layer.



 Personally, I really don't care much about how they're implemented
 underneath the hood, but the ability to throw an error condition up to
 code that handles it higher up, allowing all of the code in between to
 largely ignore the error condition is huge, and a lot of code can get
 pretty disgusting without that.


[...]

+1.


T

-- 
Береги платье снову, а здоровье смолоду.
Nov 24 2020
next sibling parent reply IGotD- <nise nise.com> writes:
On Tuesday, 24 November 2020 at 22:14:03 UTC, H. S. Teoh wrote:

 I've said before, and I'll say again: I suspect that most 
 (all?) of the current objections against exceptions comes from 
 the current *implementation* of it, i.e., libunwind, et al..  
 There are other ways to implement the concept of exceptions 
 that can potentially avoid these issues.


There is one thing that I think must go in D and that is chaining 
exception, catching several exceptions. As it is today this 
requires memory allocation itself and I'm not sure that we can 
support exception chains without memory allocation.

My experience is that looping through exceptions is extremely 
unusual and normally it is a first served approach that is used. 
Double, triple or more faults is often too difficult to deal with.
Nov 24 2020
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Nov 24, 2020 at 10:25:23PM +0000, IGotD- via Digitalmars-d wrote:
[...]

 There is one thing that I think must go in D and that is chaining
 exception, catching several exceptions. As it is today this requires
 memory allocation itself and I'm not sure that we can support
 exception chains without memory allocation.
 
 My experience is that looping through exceptions is extremely unusual
 and normally it is a first served approach that is used. Double,
 triple or more faults is often too difficult to deal with.


Yeah, chained exceptions are one of those things that look good on
paper, but in practice rarely used, yet if you look at druntime, there's
a bunch of very delicate and tricky (and ugly) code that's there just to
handle chained exceptions.  In retrospect, we should've just made it so
that throwing while another exception is in transit should just abort
the program, like C++ does (IIRC).

Now we probably can't replace that without breaking stuff. (Though
honestly, I doubt if anything would actually break besides the test
suite!)


T

-- 
If blunt statements had a point, they wouldn't be blunt...
Nov 24 2020
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 24 November 2020 at 22:14:03 UTC, H. S. Teoh wrote:

 I've said before, and I'll say again: I suspect that most 
 (all?) of the current objections against exceptions comes from 
 the current *implementation* of it, i.e., libunwind, et al..


D used to use a different implementation that I frankly thought 
was superior.

But it moved to this one for compatibility with other 
languages....

Another issue with D's exceptions is the allocation one. dip1008 
seems ok though. Or I tried immutable ones but immutable w/ catch 
is broken af :(
Nov 24 2020
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Nov 24, 2020 at 10:41:16PM +0000, Adam D. Ruppe via Digitalmars-d wrote:

 On Tuesday, 24 November 2020 at 22:14:03 UTC, H. S. Teoh wrote:

 I've said before, and I'll say again: I suspect that most (all?) of
 the current objections against exceptions comes from the current
 *implementation* of it, i.e., libunwind, et al..


 
 D used to use a different implementation that I frankly thought was
 superior.

 But it moved to this one for compatibility with other languages....


[...]

Yeah.  And this is why aping other languages is not always a good idea.
There must be adequate justification in-language for it; "language X
does Y and therefore we must also do Y" is fallacious and leads nowhere
good.


T

-- 
They say that "guns don't kill people, people kill people." Well I think the
gun helps. If you just stood there and yelled BANG, I don't think you'd kill
too many people. -- Eddie Izzard, Dressed to Kill
Nov 24 2020