www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - expectations 0.1.0

reply Paul Backus <snarwin gmail.com> writes:
expectations is an error-handling library that lets you bundle 
exceptions together with return values. It is based on Rust's 
Result<T, E> [1] and C++'s proposed std::expected. [2] If you're 
not familiar with those, Andrei's NDC Oslo talk, "Expect the 
Expected" [3], explains the advantages of this approach to error 
handling in considerable detail.

Features:
- `Expected` values can be treated as either return codes or 
exceptions.
- Functions that return `Expected` values can be composed easily 
using a monadic interface (`andThen`).
- `Expected!void` is valid and (hopefully) works the way you'd 
expect.
- Everything, except for `opEquals` (which depends on 
`Object.opEquals`), works in  safe code.

This is very much a work in progress; all comments and feedback 
are welcome.

Documentation: 
https://pbackus.github.io/expectations/expectations.html
DUB: https://code.dlang.org/packages/expectations
Code: https://github.com/pbackus/expectations

[1] https://doc.rust-lang.org/std/result/enum.Result.html
[2] https://wg21.link/p0323r7
[3] https://www.youtube.com/watch?v=nVzgkepAg5Y
Sep 01
next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Sunday, 2 September 2018 at 06:59:20 UTC, Paul Backus wrote:
 expectations is an error-handling library that lets you bundle 
 exceptions together with return values. It is based on Rust's 
 Result<T, E> [1] and C++'s proposed std::expected. [2] If 
 you're not familiar with those, Andrei's NDC Oslo talk, "Expect 
 the Expected" [3], explains the advantages of this approach to 
 error handling in considerable detail.
Incidentally, I've already proposed `Expected` into Phobos std.experimental.typecons here https://github.com/dlang/phobos/pull/6686 Is it ok if I try to merge your effort into this pull request?
Sep 02
parent Paul Backus <snarwin gmail.com> writes:
On Sunday, 2 September 2018 at 23:38:41 UTC, Per Nordlöw wrote:
 On Sunday, 2 September 2018 at 06:59:20 UTC, Paul Backus wrote:
 expectations is an error-handling library that lets you bundle 
 exceptions together with return values. It is based on Rust's 
 Result<T, E> [1] and C++'s proposed std::expected. [2] If 
 you're not familiar with those, Andrei's NDC Oslo talk, 
 "Expect the Expected" [3], explains the advantages of this 
 approach to error handling in considerable detail.
Incidentally, I've already proposed `Expected` into Phobos std.experimental.typecons here https://github.com/dlang/phobos/pull/6686 Is it ok if I try to merge your effort into this pull request?
I don't think expectations has reached a high enough level of quality yet to be ready for inclusion in Phobos. However, if you'd like to submit it for comments as a WIP, or use it as a reference for your own implementation, that's completely fine, and I'd be happy to help you.
Sep 02
prev sibling next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 2 September 2018 at 06:59:20 UTC, Paul Backus wrote:
 expectations is an error-handling library that lets you bundle 
 exceptions together with return values. It is based on Rust's 
 Result<T, E> [1] and C++'s proposed std::expected. [2] If 
 you're not familiar with those, Andrei's NDC Oslo talk, "Expect 
 the Expected" [3], explains the advantages of this approach to 
 error handling in considerable detail.
Sorry, I didn't watch the talk, but this sounds like something that's been on my mind for a while. There are generally two classic approaches to error handling: - Error codes. Plus: no overhead. Minus: you need to remember to check them. Some languages force you to check them, but it results in very noisy code in some cases (e.g. https://stackoverflow.com/a/3539342/21501). - Exceptions. Plus: simple to use. Minus: unnecessary (and sometimes considerable) overhead when failure is not exceptional. Now, Rust's Result works because it forces you to check the error code, and provides some syntax sugar to pass the result up the stack or abort if an error occurred. D, however, has nothing to force checking the return value of a function (except for pure functions, which is inapplicable for things like I/O). Please correct me if I'm wrong, but from looking at the code, given e.g.: Expected!void copyFile(string from, string to); nothing prevents me from writing: void main() { copyFile("nonexistent", "target"); } The success value is silently discarded, so we end up with a "ON ERROR RESUME NEXT" situation again, like badly written C code. One way we could improve on this in theory is to let functions return a successfulness value, which is converted into a thrown exception IFF the function failed AND the caller didn't check if an error occurred. Draft implementation: struct Success(E : Exception) { private E _exception; private bool checked = false; property E exception() { checked = true; return _exception; } property ok() { return exception is null; } disable this(this); ~this() { if (_exception && !checked) throw _exception; } } Success!E failure(E)(E e) { return Success!E(e); } Success!Exception copyFile(string from, string to) { // dummy if (from == "nonexistent") return failure(new Exception("EEXIST")); else return typeof(return)(); } void main() { import std.exception; copyFile("existent", "target"); assert(!copyFile("nonexistent", "target").ok); assertThrown!Exception({ copyFile("nonexistent", "target"); }()); } This combines some of the advantages of the two approaches above. In the above draft I used a real exception object for the payload, constructing which still has a significant overhead (at least over an error code), though we do get rid of a try/catch if an error is not an exceptional situation. The advantage of using a real exception object is that its stack trace is generated when the exception is instantiated, and not when it's thrown, which means that the error location inside the copyFile implementation is recorded; but the same general idea would work with a numerical error code payload too. Any thoughts?
Sep 02
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 3 September 2018 at 00:52:39 UTC, Vladimir Panteleev 
wrote:
 Please correct me if I'm wrong, but from looking at the code, 
 given e.g.:

 Expected!void copyFile(string from, string to);

 nothing prevents me from writing:

 void main() { copyFile("nonexistent", "target"); }

 The success value is silently discarded, so we end up with a 
 "ON ERROR RESUME NEXT" situation again, like badly written C 
 code.
This is definitely a big weakness of `Expected!void`, and one I hadn't considered when writing the code. With a normal `Expected!T`, the fact that you care about the return value is what forces you to check for the error, but that doesn't apply when the return value is `void`. I'm not sure at this point if it's better to leave `Expected!void` in for the sake of completeness, or remove it so that nobody's tempted to shoot themself in the foot. Definitely something to think about.
 One way we could improve on this in theory is to let functions 
 return a successfulness value, which is converted into a thrown 
 exception IFF the function failed AND the caller didn't check 
 if an error occurred.

 Draft implementation:

 struct Success(E : Exception)
 {
 	private E _exception;
 	private bool checked = false;
 	 property E exception() { checked = true; return _exception; }
 	 property ok() { return exception is null; }
 	 disable this(this);
 	~this() { if (_exception && !checked) throw _exception; }
 }
This is a really clever technique. As you said, hard to say whether it's worth it compared to just throwing an exception normally, but still, really clever.
Sep 02
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 09/02/2018 11:23 PM, Paul Backus wrote:
 
 This is a really clever technique. As you said, hard to say whether it's 
 worth it compared to just throwing an exception normally, but still, 
 really clever.
IMO, it's worth it. First of all, it decreases the asymmetry between `Expected!void` and other `Expected!T`. But more than that, there's one of the core benefits of of expected: What's awesome about expected is that by providing only one function, the caller can decide whether they want a `foo()` that throws, or a `tryFoo()` that lets them manually handle the case where it doesn't work (and is potentially nothrow). Note that the above has *nothing* to do with retrieving a value. Retrieving a value is merely used by the implementation as a trigger to lazily decide whether the caller wants `foo` or `tryFoo`. Going out of scope without making the choice could also be considered another trigger point. In fact, this "out-of-scope without being checked" could even be used as an additional trigger for even the non-void variety. After all: what if an error occurs, but the caller checks *neither* value nor hasValue? There's only one possible downside I see: What if the caller *intentionally* wants to ignore the error condition? Yes, that's generally bad practice, and signifies maybe it shouldn't be an exception in the first place. But consider Scriptlike: It has functions like `tryMkdir` and `tryRmdir` with the deliberate purpose letting people say "Unlike Phobos's mkdir/rmdir, I don't care whether the directory already exists or not, just MAKE SURE it exists (or doesn't) and don't bother me with the details!" I suppose for cases like those, it's perfectly worth leaving it up to expectation's user to design, create and document a "Don't worry about the failure" variant, should they so choose. Probably safer that way, anyway.
Sep 02
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 3 September 2018 at 04:49:40 UTC, Nick Sabalausky 
(Abscissa) wrote:
 Note that the above has *nothing* to do with retrieving a 
 value. Retrieving a value is merely used by the implementation 
 as a trigger to lazily decide whether the caller wants `foo` or 
 `tryFoo`. Going out of scope without making the choice could 
 also be considered another trigger point. In fact, this 
 "out-of-scope without being checked" could even be used as an 
 additional trigger for even the non-void variety. After all: 
 what if an error occurs, but the caller checks *neither* value 
 nor hasValue?
The thing is, triggering on explicit access lets you handle errors lazily, whereas triggering at the end of the scope forces you to handle them eagerly. Vladimir's `Success` type is, essentially, a way for a function to send something back up the stack that its caller is forced to acknowledge. Throwing an exception is *also* a way for a function to send something back up the stack that its caller is forced to acknowledge. The exact details are different, but when it comes to overall control-flow semantics, they are basically equivalent. By contrast, a function that returns an `Expected!T` does *not* force its caller to acknowledge it. If an error occurs, and the caller never checks value or hasValue...nothing happens. That's what being lazy means: if you never open the box, it doesn't matter whether the cat is alive or dead. The problem, when it comes to `Expected!void`, is that there's no good way to express what we *actually* care about--the function's side effect--as a value. If we could write `Expected!(IO!Unit)` like in Haskell, everything would be fine. To me, the only acceptable choices are for `Expected!void` to have the same lazy semantics as `Expected!T`, or for `Expected!void` to be removed altogether. Having one specialization be lazy and one be eager would be a nightmare for anyone trying to use the library. For the reasons Vladimir brought up, I'm leaning toward removal--without something like Rust's `#[must_use]` attribute, it seems like `Expected!void` is likely to do more harm than good.
Sep 02
next sibling parent aliak <something something.com> writes:
On Monday, 3 September 2018 at 06:49:41 UTC, Paul Backus wrote:
 To me, the only acceptable choices are for `Expected!void` to 
 have the same lazy semantics as `Expected!T`, or for 
 `Expected!void` to be removed altogether. Having one 
 specialization be lazy and one be eager would be a nightmare 
 for anyone trying to use the library. For the reasons Vladimir 
 brought up, I'm leaning toward removal--without something like 
 Rust's `#[must_use]` attribute, it seems like `Expected!void` 
 is likely to do more harm than good.
I'm leaning on agreeing with removal of Expected!void as well When we get opPostMove then maybe Expected!void can throw on a move or a copy if the result was a failure. This would also not allow the error to be ignored as it'd throw. Or... can it throw in ~this() if it was not checked?
Sep 03
prev sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 09/03/2018 02:49 AM, Paul Backus wrote:
 On Monday, 3 September 2018 at 04:49:40 UTC, Nick Sabalausky (Abscissa) 
 wrote:
 Note that the above has *nothing* to do with retrieving a value. 
 Retrieving a value is merely used by the implementation as a trigger 
 to lazily decide whether the caller wants `foo` or `tryFoo`. Going out 
 of scope without making the choice could also be considered another 
 trigger point. In fact, this "out-of-scope without being checked" 
 could even be used as an additional trigger for even the non-void 
 variety. After all: what if an error occurs, but the caller checks 
 *neither* value nor hasValue?
The thing is, triggering on explicit access lets you handle errors lazily, whereas triggering at the end of the scope forces you to handle them eagerly. Vladimir's `Success` type is, essentially, a way for a function to send something back up the stack that its caller is forced to acknowledge.
Yes, that's correct.
 Throwing an exception is *also* a way for a function to send something 
 back up the stack that its caller is forced to acknowledge.
Yes, but it's heavier-weight AND prevents the callee from being nothrow.
 but when it comes to overall control-flow 
 semantics, they are basically equivalent.
Control-flow semantics, sure, but as I pointed out in my previous sentence, there's more relevant things involved here than just control flow semantics.
 By contrast, a function that returns an `Expected!T` does *not* force 
 its caller to acknowledge it. If an error occurs, and the caller never 
 checks value or hasValue...nothing happens.
That's called squelching an error, and its EXACTLY the same problem as using non-Expect return values to indicate errors. I'd regard that as very notable hole in any Expected design as it breaks one of the core points of using Expect vs returning plain error codes: The user can still accidentally (or deliberately) squelch an error. To clarify: If the caller never checks value or hasValue, that does NOT mean the caller has carefully and deliberately chosen to disregard the error. It *could* mean that, but it could also mean they simply messed up. Deliberately squeching an error should NEVER be implicit, it should always require something like: catch(...) { /+ Do nothing +/ } or if(!x.hasValue) { /+ Do nothing +/ }
 That's what being lazy 
 means: if you never open the box, it doesn't matter whether the cat is 
 alive or dead.
I don't see the laziness here as being the core point. The laziness is the "how", not the raison d'etre. The laziness is simply a tool being used to achieve the real goals of: - Allowing the caller the decide between foo/tryFoo versions without the API duplication. - Decreasing exception-related overhead and increasing utility of nothrow.
 
 Having one specialization be lazy and one be eager 
 would be a nightmare for anyone trying to use the library.
Vladimir's Success vs Expect!T is NOT an example of "eager vs lazy". In BOTH cases, the callee treats errors lazily. And in BOTH cases, the caller (or whomever the caller passes it off to) is expected to, at some point, make a deliberate, explicit choice between "handle or throw". And as I said before, allowing the caller to accidentally (or implicitly) squelch the error is a fundamental breakage in the whole point behind Except.
Sep 03
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 3 September 2018 at 21:55:57 UTC, Nick Sabalausky 
(Abscissa) wrote:
 By contrast, a function that returns an `Expected!T` does 
 *not* force its caller to acknowledge it. If an error occurs, 
 and the caller never checks value or hasValue...nothing 
 happens.
That's called squelching an error, and its EXACTLY the same problem as using non-Expect return values to indicate errors. I'd regard that as very notable hole in any Expected design as it breaks one of the core points of using Expect vs returning plain error codes: The user can still accidentally (or deliberately) squelch an error.
If you receive an `Expected!T`, you have the following choices available to you: 1. Handle the success case locally, and the failure case non-locally (i.e. use `value` directly). 2. Handle both the success case and the failure case locally (i.e. check `hasValue`). 3. Handle both the success case and the failure case non-locally (i.e., pass the `Expected!T` along untouched). The difference between `Expected`, on the one hand, and both `Success` and plain-old exceptions, on the other, is that `Expected` gives you choice #3, and the other two don't. Why is choice #3 important? Because it doesn't branch. Both success and failure follow the same code path. That makes functions that use `Expected` much easier to compose than ones that throw exceptions. For example, if you throw an exception in the middle of a range pipeline, the entire thing comes crashing down--but an `Expected!T` will pass right through, and let you handle it when it comes out the other end. Now, you will probably object--and rightly so--that there is an implicit assumption being made here, which is that "handle the success case" is equivalent to "use the return value." Clearly, this equivalence does not always hold in the presence of side effects. That's why `Expected!void` is so problematic. Nevertheless, I think it holds in enough cases to make `Expected` useful in practice. In particular, it is guaranteed to hold for strongly-pure functions, and will also hold for functions whose side effects are visible only through the return value (e.g., `readln`).
 I don't see the laziness here as being the core point. The 
 laziness is the "how", not the raison d'etre. The laziness is 
 simply a tool being used to achieve the real goals of:

 - Allowing the caller the decide between foo/tryFoo versions 
 without the API duplication.

 - Decreasing exception-related overhead and increasing utility 
 of nothrow.
The laziness (on the part of the caller, i.e., the code that *receives* the `Expected!T`) is important because it's what makes choice #3 possible. It's an essential part of the design.
Sep 03
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 09/04/2018 12:05 AM, Paul Backus wrote:
 On Monday, 3 September 2018 at 21:55:57 UTC, Nick Sabalausky (Abscissa) 
 wrote:
 By contrast, a function that returns an `Expected!T` does *not* force 
 its caller to acknowledge it. If an error occurs, and the caller 
 never checks value or hasValue...nothing happens.
That's called squelching an error, and its EXACTLY the same problem as using non-Expect return values to indicate errors. I'd regard that as very notable hole in any Expected design as it breaks one of the core points of using Expect vs returning plain error codes: The user can still accidentally (or deliberately) squelch an error.
If you receive an `Expected!T`, you have the following choices available to you: 1. Handle the success case locally, and the failure case non-locally (i.e. use `value` directly). 2. Handle both the success case and the failure case locally (i.e. check `hasValue`). 3. Handle both the success case and the failure case non-locally (i.e., pass the `Expected!T` along untouched). The difference between `Expected`, on the one hand, and both `Success` and plain-old exceptions, on the other, is that `Expected` gives you choice #3, and the other two don't.
I think you may be getting hung up on a certain particular detail of Vladimir's exact "draft" implementation of Success, whereas I'm focusing more on Success's more general point of "Once the object is no longer around, guarantee the error doesn't get implicitly squelched." You're right that, *in the draft implementation as-is*, it can be awkward for the caller to then pass the Success along to some other code (another function call, or something higher up the stack). *Although*, still not impossible. So #3 still isn't eliminated, it's simply made awkward... But reference counting would be enough to fix that. (Or a compiler-supported custom datatype that's automatically pass-by-moving, but that's of course not something D has). And you haven't actually directly addressed the issue I've raised about failing to guarantee errors aren't implicitly squelched.
 Why is choice #3 important? Because it doesn't branch. Both success and 
 failure follow the same code path. That makes functions that use 
 `Expected` much easier to compose than ones that throw exceptions. For 
 example, if you throw an exception in the middle of a range pipeline, 
 the entire thing comes crashing down--but an `Expected!T` will pass 
 right through, and let you handle it when it comes out the other end.
Right. And as described above, I'm advocating an approach that preserves that (even for void) while *also* improving Expect so it can not *merely* improve things "in most cases", but would actually *guarantee* errors are not implicitly squelched in ALL cases where Expect!whatever is used.
 I don't see the laziness here as being the core point. The laziness is 
 the "how", not the raison d'etre. The laziness is simply a tool being 
 used to achieve the real goals of:

 - Allowing the caller the decide between foo/tryFoo versions without 
 the API duplication.

 - Decreasing exception-related overhead and increasing utility of 
 nothrow.
The laziness (on the part of the caller, i.e., the code that *receives* the `Expected!T`) is important because it's what makes choice #3 possible. It's an essential part of the design.
Again, what I'm proposing still preserves that.
Sep 04
parent Paul Backus <snarwin gmail.com> writes:
On Tuesday, 4 September 2018 at 22:08:48 UTC, Nick Sabalausky 
(Abscissa) wrote:
 I think you may be getting hung up on a certain particular 
 detail of Vladimir's exact "draft" implementation of Success, 
 whereas I'm focusing more on Success's more general point of 
 "Once the object is no longer around, guarantee the error 
 doesn't get implicitly squelched."

 You're right that, *in the draft implementation as-is*, it can 
 be awkward for the caller to then pass the Success along to 
 some other code (another function call, or something higher up 
 the stack). *Although*, still not impossible. So #3 still isn't 
 eliminated, it's simply made awkward...

 But reference counting would be enough to fix that. (Or a 
 compiler-supported custom datatype that's automatically 
 pass-by-moving, but that's of course not something D has).
Ok, I think I understand what you're proposing now--basically, something comparable to Rust's `#[must_use]` attribute. Thanks for taking the time to explain. I agree that that would be a nice feature for `Expected` to have. The thing is, D already has a mechanism for signalling failures that can't be ignored: exceptions. So adding that functionality to `Expected`, while convenient, doesn't actually let you accomplish anything you couldn't already. Now, if it were easy to implement, then sure, no problem. But it's not. Reference counting in particular is so problematic that Walter and Andrei have proposed *multiple* new language features (copy constructors, __mutable) to make it work cleanly. As things currently stand, making `Expected` reference-counted would mean at the very least giving up compatibility with `const` and `immutable`, which makes `Expected` a worse fit for strongly-pure functions (currently its *best* use-case). It's a shame that D forces us to make this tradeoff, but given the options in front of me, I would rather have `Expected` shine in the area where it has a comparative advantage, even if that means making it less universally-applicable as an error-handling mechanism.
Sep 05
prev sibling parent reply Thomas Mader <thomas.mader gmail.com> writes:
On Monday, 3 September 2018 at 00:52:39 UTC, Vladimir Panteleev 
wrote:
 There are generally two classic approaches to error handling:
std::expected is not the only thing on this topic going on in C++. There is also the proposal from Herb Sutter [1]. It's not a library solution and changes even the ABI but it's an interesting approach. He also tries to get compatibility into C via an extension. (See 4.6.11 in [1]) [1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf
Sep 02
parent reply aliak <something something.com> writes:
On Monday, 3 September 2018 at 06:00:06 UTC, Thomas Mader wrote:
 On Monday, 3 September 2018 at 00:52:39 UTC, Vladimir Panteleev 
 wrote:
 There are generally two classic approaches to error handling:
std::expected is not the only thing on this topic going on in C++. There is also the proposal from Herb Sutter [1]. It's not a library solution and changes even the ABI but it's an interesting approach. He also tries to get compatibility into C via an extension. (See 4.6.11 in [1]) [1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf
This would be great to have in D. Swift [0] has something similar, and personally after using it for a few years, I can say that I've seen next to no unhandled exception errors in iOS code at least. [0] https://www.mikeash.com/pyblog/friday-qa-2017-08-25-swift-error-handling-implementation.html
Sep 03
parent Thomas Mader <thomas.mader gmail.com> writes:
On Monday, 3 September 2018 at 13:00:05 UTC, aliak wrote:
 This would be great to have in D.
Indeed, if it's really going into C++ D needs to think about how to handle that anyway if it wants to offer C++ ABI interfacing.
 Swift [0] has something similar, and personally after using it 
 for a few years, I can say that I've seen next to no unhandled 
 exception errors in iOS code at least.
Thanks, didn't know that Swift is already using something like this.
Sep 03
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Sunday, 2 September 2018 at 06:59:20 UTC, Paul Backus wrote:
 expectations is an error-handling library that lets you bundle 
 exceptions together with return values. It is based on Rust's 
 Result<T, E> [1] and C++'s proposed std::expected. [2] If 
 you're not familiar with those, Andrei's NDC Oslo talk, "Expect 
 the Expected" [3], explains the advantages of this approach to 
 error handling in considerable detail.
expectations 0.2.0 is now available, with the following updates - `hasValue`, `value`, and `exception` now work for const and immutable `Expected` objects. - `Expected!void` has been removed. - `map` and `andThen` can now be partially applied to functions, "lifting" them into the Expected monad. - The documentation has been improved based on the feedback given in this thread.
Sep 04