www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Motive behind !empty() with front() instead of Optional front()

reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

instead of having front returning an optional/maybe type with 
enforced pattern matching?

Lack of builtin Optional type?

Choosing the Optional path would have avoided the need for 
putting error diagnostics such as

https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

in the standard library of such an alternative solution.
Mar 24
next sibling parent reply Meta <jared771 gmail.com> writes:
On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:
 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.
Most/all of the range stuff was Andrei's idea, and his C++ background heavily informed the design. Returning Optional<T> to represent that a calculation might fail is not common in C++, or at least wasn't back then. In general, more functional-oriented techniques have only recently started diffusing into the C++ culture (or at least it feels that way to me; I don't really follow C++ closely anymore).
Mar 24
parent reply IGotD- <nise nise.com> writes:
On Wednesday, 24 March 2021 at 19:32:22 UTC, Meta wrote:
 Most/all of the range stuff was Andrei's idea, and his C++ 
 background heavily informed the design. Returning Optional<T> 
 to represent that a calculation might fail is not common in 
 C++, or at least wasn't back then. In general, more 
 functional-oriented techniques have only recently started 
 diffusing into the C++ culture (or at least it feels that way 
 to me; I don't really follow C++ closely anymore).
That it is because of historical reasons makes sense. From a programming point of view, it doesn't really matter since you must check your Optional<T> return value anyway and you still get a check regardless. Performance wise it doesn't matter. Another motive would be that it would force to check if the return value is not empty, however you might not want that. For example in a loop and you know the amount of elements in advance then the check is unnecessary.
Mar 24
next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 24 March 2021 at 21:09:52 UTC, IGotD- wrote:
 For example in a loop and you know the amount of elements in 
 advance then the check is unnecessary.
Are you talking about a foreach loop? If so, the element type will never have to be an optional type. Regardless of whether `front` is wrapped in an Optional or not.
Mar 24
parent IGotD- <nise nise.com> writes:
On Wednesday, 24 March 2021 at 21:20:40 UTC, Per Nordlöw wrote:
 Are you talking about a foreach loop? If so, the element type 
 will never have to be an optional type. Regardless of whether 
 `front` is wrapped in an Optional or not.
Could be a foreach loop but also a counted loop, you want to go through a certain amount of elements and you know for sure with check before that the elements exist. I'm not sure if I understand your statement, but wrapping such a loop with Optional<T> would lead to an extra overhead and since it is inside a loop it is not insignificant. I wouldn't trust an optimizer would understand how to remove such check. Now the possibility is to have to versions of front, one that returns the element and another that returns Optional<T> (why do I write C++ syntax?). Not sure if that would lead to ambiguous case problems.
Mar 24
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Wednesday, 24 March 2021 at 21:09:52 UTC, IGotD- wrote:
 Another motive would be that it would force to check if the 
 return value is not empty, however you might not want that. For 
 example in a loop and you know the amount of elements in 
 advance then the check is unnecessary.
If you know the Optional isn't empty you can just call `.unwrap` (or whatever it ends up being called) to get the value without checking.
Mar 24
prev sibling next sibling parent Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:
On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:
 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.
Is optional not having a value a good way to know you are at the end of a list? I know that Nullable can use a flag, but if you return an optional with a value of null, feels like you are defeating the purpose of optional.
Mar 24
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:
 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.
The big question is, what if I want a range of Optional!T ?
Mar 25
next sibling parent Rumbu <rumbu rumbu.ro> writes:
On Thursday, 25 March 2021 at 09:35:40 UTC, deadalnix wrote:
 On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:
 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.
The big question is, what if I want a range of Optional!T ?
You have to check on Optional!(Optional!T).
Mar 25
prev sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 25 March 2021 at 09:35:40 UTC, deadalnix wrote:
 The big question is, what if I want a range of Optional!T ?
That might lead to confusing code, yes. Aliases or subtypes could come of use in such a situation. Better ask the Rust forums...
Mar 25
prev sibling next sibling parent reply Piotr Mitana <piotr.mitana gmail.com> writes:
On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:
 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.
Actually changing front() to return optionals would probably break many things. However, it could be handy to add a method called nullableFront or frontOrNull (and nullableBack or backOrNull for bidirectional ranges) to std.range.
Mar 26
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 26 March 2021 at 08:49:30 UTC, Piotr Mitana wrote:
 Actually changing front() to return optionals would probably 
 break many things. However, it could be handy to add a method 
 called nullableFront or frontOrNull (and nullableBack or 
 backOrNull for bidirectional ranges) to std.range.
Agreed. Alternative namings could be maybeFront or optionalFront.
Mar 26
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 26 March 2021 at 09:10:28 UTC, Per Nordlöw wrote:
 On Friday, 26 March 2021 at 08:49:30 UTC, Piotr Mitana wrote:
 Actually changing front() to return optionals would probably 
 break many things. However, it could be handy to add a method 
 called nullableFront or frontOrNull (and nullableBack or 
 backOrNull for bidirectional ranges) to std.range.
Agreed. Alternative namings could be maybeFront or optionalFront.
A D specific solution could be to add compiler diagnostics that checks whether a call to `x.front` and x.popFront() is guarded by an `if (!x.empty)`. Provided no mutation has happened since the call to !empty and `x` is not accessible via other threads.
Mar 26
parent reply Nick Treleaven <nick geany.org> writes:
On Friday, 26 March 2021 at 13:23:25 UTC, Per Nordlöw wrote:
 A D specific solution could be to add compiler diagnostics that 
 checks whether a call to `x.front` and x.popFront() is guarded 
 by an `if (!x.empty)`. Provided no mutation has happened since 
 the call to !empty and `x` is not accessible via other threads.
// in safe code scope p = &r; if (!r.empty) { p.popFront; r.front.writeln; // oops } I don't think it's practical for the compiler to detect more complicated variants of the above. The beauty of languages with compile-time checked optional types is that the value has to exist in order to unwrap the optional.
Mar 26
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 26 March 2021 at 15:02:27 UTC, Nick Treleaven wrote:
 compile-time checked optional types is that the value has to 
 exist in order to unwrap the optional.
Can you elaborate on what you mean by "compile-time checked optional types"?
Mar 26
parent Nick Treleaven <nick geany.org> writes:
On Friday, 26 March 2021 at 15:51:41 UTC, Per Nordlöw wrote:
 Can you elaborate on what you mean by "compile-time checked 
 optional types"?
Like `if let` in Swift: https://www.hackingwithswift.com/sixty/10/2/unwrapping-optionals This form ensures at compile time that a runtime check has made in code that accesses the value held by the optional. (There's also a `guard let` form to unwrap into a variable in the current scope that requires you to terminate the current scope when the optional is empty).
Mar 26
prev sibling next sibling parent reply vitoroak <carvalhogvm gmail.com> writes:
On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:
 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.
I think one reason is that unlike Rust, D doesn't have a safe way to return Optional!(ref T) so we need front and empty so Ranges can return the items by reference.
Mar 26
next sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 26 March 2021 at 13:38:01 UTC, vitoroak wrote:
 I think one reason is that unlike Rust, D doesn't have a safe 
 way to return Optional!(ref T) so we need front and empty so 
 Ranges can return the items by reference.
Ahh, thanks.
Mar 26
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 26 March 2021 at 13:38:01 UTC, vitoroak wrote:
 I think one reason is that unlike Rust, D doesn't have a safe 
 way to return Optional!(ref T) so we need front and empty so 
 Ranges can return the items by reference.
I believe with DIP 1000 it should be possible to return something like `Optional!(Ref!T)`, where `Ref!T` is a safe wrapper around a `T*`. Of course the ideal would be to make `ref` itself into a type qualifier (`ref(T)`).
Mar 26
parent reply Q. Schroll <qs.il.paperinik gmail.com> writes:
On Friday, 26 March 2021 at 15:12:04 UTC, Paul Backus wrote:
 On Friday, 26 March 2021 at 13:38:01 UTC, vitoroak wrote:
 I think one reason is that unlike Rust, D doesn't have a safe 
 way to return `Optional!(ref T)` so we need front and empty so 
 Ranges can return the items by reference.
I believe with DIP 1000 it should be possible to return something like `Optional!(Ref!T)`, where `Ref!T` is a safe wrapper around a `T*`.
When I experimented with `Ref` improving on [`Final`](https://dlang.org/phobos/std_experimental_typecons.html#.Final) for pointers (`Ref!T` is `Final!(T*)`), I found it near useless. The only thing you cannot do with a `Ref!T` but with a `T*` is reassigning it and pointer arithmetic. But ` safe` already protects you against accidental pointer arithmetic. Accidental assignment also isn't really an issue in D. I'm all in for `final` as a type constructor (meaning head-`const`), but it probably won't lift its weight.
 Of course the ideal would be to make `ref` itself into a type 
 qualifier (`ref(T)`).
If you look at all the exceptions C++ has for its reference type constructor, you'll immediately see why Walter created `ref` as a storage class and not a type constructor. C++ reference types in many cases don't compose; e.g. `vector<int&>` isn't a thing. If you really think about it, you don't need `ref` as a type constructor. Although allowing `ref` local variables wouldn't be harmful, I guess.
Apr 05
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 6 April 2021 at 01:47:47 UTC, Q. Schroll wrote:
 On Friday, 26 March 2021 at 15:12:04 UTC, Paul Backus wrote:
 Of course the ideal would be to make `ref` itself into a type 
 qualifier (`ref(T)`).
If you look at all the exceptions C++ has for its reference type constructor, you'll immediately see why Walter created `ref` as a storage class and not a type constructor. C++ reference types in many cases don't compose; e.g. `vector<int&>` isn't a thing. If you really think about it, you don't need `ref` as a type constructor. Although allowing `ref` local variables wouldn't be harmful, I guess.
Here's what the generic identity function currently looks like in D: auto ref T identity(T)(auto ref T arg) { import core.lifetime: forward; return forward!arg; } Here's what the generic identity function *would* look like if `ref` were a type qualifier: T identity(T)(T arg) { return arg; } I rest my case.
Apr 05
next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 April 2021 at 02:14:17 UTC, Paul Backus wrote:
 Here's what the generic identity function *would* look like if 
 `ref` were a type qualifier:

     T identity(T)(T arg)
     {
         return arg;
     }

 I rest my case.
So what was the motive for not making it a type qualifier in D? And what are the obstacles for making it that?
Apr 06
parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 April 2021 at 20:34:08 UTC, Per Nordlöw wrote:
 So what was the motive for not making it a type qualifier in D?

 And what are the obstacles for making it that?
See https://forum.dlang.org/post/mailman.7903.1554072555.29801.digitalmars-d puremagic.com
Apr 06
prev sibling next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 April 2021 at 02:14:17 UTC, Paul Backus wrote:
     auto ref T identity(T)(auto ref T arg)
     {
         import core.lifetime: forward;
         return forward!arg;
     }
When is the call to forward needed in this case?
Apr 06
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 6 April 2021 at 20:47:31 UTC, Per Nordlöw wrote:
 On Tuesday, 6 April 2021 at 02:14:17 UTC, Paul Backus wrote:
     auto ref T identity(T)(auto ref T arg)
     {
         import core.lifetime: forward;
         return forward!arg;
     }
When is the call to forward needed in this case?
For non-copyable types. It's actually needed in both cases--we would need DIP 1040 (or something similar) to get rid of it.
Apr 06
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 April 2021 at 21:09:13 UTC, Paul Backus wrote:
 For non-copyable types. It's actually needed in both cases--we 
 would need DIP 1040 (or something similar) to get rid of it.
So let's help Walter getting DIP-1040 accepted then. :) What else is forward needed for? The doc says "Forwards function arguments while keeping `out`, `ref`, and `lazy` on the parameters." Why can't the compiler do that for us?
Apr 06
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/6/21 5:14 PM, Per Nordlöw wrote:
 On Tuesday, 6 April 2021 at 21:09:13 UTC, Paul Backus wrote:
 For non-copyable types. It's actually needed in both cases--we would 
 need DIP 1040 (or something similar) to get rid of it.
So let's help Walter getting DIP-1040 accepted then. :) What else is forward needed for? The doc says "Forwards function arguments while keeping `out`, `ref`, and `lazy` on the parameters." Why can't the compiler do that for us?
Because sometimes you want the usual semantics, i.e. create a copy of the argument. These things are difficult to automate. I don't think a simple solution exists.
Apr 07
parent Max Haughton <maxhaton gmail.com> writes:
On Thursday, 8 April 2021 at 02:16:31 UTC, Andrei Alexandrescu 
wrote:
 On 4/6/21 5:14 PM, Per Nordlöw wrote:
 On Tuesday, 6 April 2021 at 21:09:13 UTC, Paul Backus wrote:
 For non-copyable types. It's actually needed in both 
 cases--we would need DIP 1040 (or something similar) to get 
 rid of it.
So let's help Walter getting DIP-1040 accepted then. :) What else is forward needed for? The doc says "Forwards function arguments while keeping `out`, `ref`, and `lazy` on the parameters." Why can't the compiler do that for us?
Because sometimes you want the usual semantics, i.e. create a copy of the argument. These things are difficult to automate. I don't think a simple solution exists.
I definitely agree that the "simple solution" probably doesn't exist, however I am kind of partial to just leaving it to the caller (especially for `out`, I've found in some places). It conflicts inside me however, because I like very plastic interfaces - however it kills resolution questions like you raised (lvalue vs. reference to lvalue) above.
Apr 07
prev sibling next sibling parent reply Q. Schroll <qs.il.paperinik gmail.com> writes:
On Tuesday, 6 April 2021 at 02:14:17 UTC, Paul Backus wrote:
 On Tuesday, 6 April 2021 at 01:47:47 UTC, Q. Schroll wrote:
 On Friday, 26 March 2021 at 15:12:04 UTC, Paul Backus wrote:
 Of course the ideal would be to make `ref` itself into a type 
 qualifier (`ref(T)`).
If you look at all the exceptions C++ has for its reference type constructor, you'll immediately see why Walter created `ref` as a storage class and not a type constructor. C++ reference types in many cases don't compose; e.g. `vector<int&>` isn't a thing. If you really think about it, you don't need `ref` as a type constructor. Although allowing `ref` local variables wouldn't be harmful, I guess.
Here's what the generic identity function currently looks like in D: auto ref T identity(T)(auto ref T arg) { import core.lifetime: forward; return forward!arg; } Here's what the generic identity function *would* look like if `ref` were a type qualifier: T identity(T)(T arg) { return arg; } I rest my case.
In C++, the identity function is ```C++ template<typename T> T&& identitiy(T&& arg) { return std::forward<T>(arg); } ``` if you want it to work like `identity(arg)`. **TL;DR:** C++ doesn't do it the simple way for a reason. If you do ```C++ template<typename T> T identitiy(T arg) { return arg; } ``` you get copies unless you call it `identity<int&>(arg)` and `identity<int&&>(1)` (which defeats the purpose, I guess. Template type deduction alone (C++ has 4 different rule sets about type deduction) is complicated. One reason for stripping `ref` from the argument types unless they explicitly bind to `ref` parameters is that references should act like aliases: ```C++ int i; int& r = i; f(i); f(r); ``` Here, `f(r)` should act the same as `f(i)` irrespective of how `f` is defined. `f` could take `int`, `int&`, `const int&`, or `T`, `const T&`, or `T&&` for some `T` that can implicitly constructed from `int`. For `int&&` or `T&`it wouldn't compile. References as type constructors are weird. And please note, this is what I *immediately* could come up with and I verified. There's stuff with function types and static array bounds that I remember have shenanigans with references, but I'd have to do some search for details. The only thing D does wrong in this regard is not having `forward` and `move` in `object.d`.
Apr 07
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 7 April 2021 at 21:31:38 UTC, Q. Schroll wrote:
 Template type deduction alone (C++ has 4 different rule sets 
 about type deduction) is complicated. One reason for stripping 
 `ref` from the argument types unless they explicitly bind to 
 `ref` parameters is that references should act like aliases:
 ```C++
 int i;
 int& r = i;
 f(i);
 f(r);
 ```
 Here, `f(r)` should act the same as `f(i)` irrespective of how 
 `f` is defined. `f` could take `int`, `int&`, `const int&`, or 
 `T`, `const T&`, or `T&&` for some `T` that can implicitly 
 constructed from `int`. For `int&&` or `T&`it wouldn't compile. 
 References as type constructors are weird.
References as type constructors are weird *in C++* because of specific choices made by the designers of C++ (such as: "references should act like aliases"). There is nothing inherent to the concept of a reference type that implies this kind of weirdness. D does not have to make the same choices as C++, and `ref(T)` in D does not have to suffer from the issues that references suffer from in C++.
Apr 07
parent reply Q. Schroll <qs.il.paperinik gmail.com> writes:
On Wednesday, 7 April 2021 at 21:44:52 UTC, Paul Backus wrote:
 On Wednesday, 7 April 2021 at 21:31:38 UTC, Q. Schroll wrote:
 Template type deduction alone (C++ has 4 different rule sets 
 about type deduction) is complicated. One reason for stripping 
 `ref` from the argument types unless they explicitly bind to 
 `ref` parameters is that references should act like aliases:
 ```C++
 int i;
 int& r = i;
 f(i);
 f(r);
 ```
 Here, `f(r)` should act the same as `f(i)` irrespective of how 
 `f` is defined. `f` could take `int`, `int&`, `const int&`, or 
 `T`, `const T&`, or `T&&` for some `T` that can implicitly 
 constructed from `int`. For `int&&` or `T&`it wouldn't 
 compile. References as type constructors are weird.
References as type constructors are weird *in C++* because of specific choices made by the designers of C++ (such as: "references should act like aliases"). There is nothing inherent to the concept of a reference type that implies this kind of weirdness. D does not have to make the same choices as C++, and `ref(T)` in D does not have to suffer from the issues that references suffer from in C++.
As Andrei said, in C++, apart from the `decltype` exception (which is justified), references behave as aliases. In D, it's the same, apart from `__trais(isRef)` which is also justified. What else do you expect from a reference object? Maybe I'm biased, but I don't see how other solutions than C++'s would be better.
Apr 09
parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 9 April 2021 at 20:40:54 UTC, Q. Schroll wrote:
 As Andrei said, in C++, apart from the `decltype` exception 
 (which is justified), references behave as aliases. In D, it's 
 the same, apart from `__trais(isRef)` which is also justified. 
 What else do you expect from a reference object? Maybe I'm 
 biased, but I don't see how other solutions than C++'s would be 
 better.
Hypothetically, if `ref` were a type qualifier, I'd expect it to behave the same way as D's other type qualifiers. We don't strip `const` or `shared` from types during template instantiation (the special case of `const(T[])` → `const(T)[]` notwithstanding), so my baseline expectation is that we wouldn't strip `ref` either. Maybe there's an argument to be made that this is a bad idea, and that you should always be able to replace an lvalue with a reference without any effect on the program's behavior. But I don't think you can take it as given that that's *obviously* the correct way to do references, just because that's what C++ does.
Apr 09
parent Q. Schroll <qs.il.paperinik gmail.com> writes:
On Friday, 9 April 2021 at 21:17:03 UTC, Paul Backus wrote:
 On Friday, 9 April 2021 at 20:40:54 UTC, Q. Schroll wrote:
 As Andrei said, in C++, apart from the `decltype` exception 
 (which is justified), references behave as aliases. In D, it's 
 the same, apart from `__trais(isRef)` which is also justified. 
 What else do you expect from a reference object? Maybe I'm 
 biased, but I don't see how other solutions than C++'s would 
 be better.
Hypothetically, if `ref` were a type qualifier, I'd expect it to behave the same way as D's other type qualifiers. We don't strip `const` or `shared` from types during template instantiation (the special case of `const(T[])` → `const(T)[]` notwithstanding), so my baseline expectation is that we wouldn't strip `ref` either. Maybe there's an argument to be made that this is a bad idea, and that you should always be able to replace an lvalue with a reference without any effect on the program's behavior. But I don't think you can take it as given that that's *obviously* the correct way to do references, just because that's what C++ does.
The "special case of `const`" is exactly the point: If you have a `const(T[]) ts` object and you do ```D auto ts2 = ts; ``` then, `typeof(ts2)` is `const(T)[]`. The whole point of making a copy is that it (or at least the first layer) becomes independent and mutable. I think references should behave the same: Assigning a `ref(T)` to an `auto` variable should create a copy that (or at least the first layer) is independent of the source. Creating another reference is near-useless. When it comes to parameter passing, binding by copy is expected to be the default and binding by reference a specialty. One reason being that binding by copy is more flexible, e.g. it allows (implicit) conversions. I don't think it's obviously the right way because C++ does it that way, but it's a hint. I came to the conclusion myself; thinking of various alternatives, I could see obvious problems. C++'s references have (obvious?) problems, so it's not like it's perfect. IMO, `ref` as a storage class is the Right Thing. D falls short in some places: * Allowing `ref` for local variables would be valuable at times. There's no actual gain in not allowing them. Some stuff like postfix operators even lower to `ref` local variables. * One cannot directly express `ref` returning delegate or function pointer types on many occasions, e.g. neither `is(DG == ref int delegate())` nor `is(DG == int delegate() ref)` compile. Also, one cannot directly specify `ref` returning delegate or function pointer types in parameter lists. In both cases, an alias has to be defined. If `ref` were a type constructor, these would be non-issues for sure. But we're nowhere near *requiring* it to be one to solve these.
Apr 09
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/7/21 5:31 PM, Q. Schroll wrote:
 On Tuesday, 6 April 2021 at 02:14:17 UTC, Paul Backus wrote:
 On Tuesday, 6 April 2021 at 01:47:47 UTC, Q. Schroll wrote:
 On Friday, 26 March 2021 at 15:12:04 UTC, Paul Backus wrote:
 Of course the ideal would be to make `ref` itself into a type 
 qualifier (`ref(T)`).
If you look at all the exceptions C++ has for its reference type constructor, you'll immediately see why Walter created `ref` as a storage class and not a type constructor. C++ reference types in many cases don't compose; e.g. `vector<int&>` isn't a thing. If you really think about it, you don't need `ref` as a type constructor. Although allowing `ref` local variables wouldn't be harmful, I guess.
Here's what the generic identity function currently looks like in D:     auto ref T identity(T)(auto ref T arg)     {         import core.lifetime: forward;         return forward!arg;     } Here's what the generic identity function *would* look like if `ref` were a type qualifier:     T identity(T)(T arg)     {         return arg;     } I rest my case.
In C++, the identity function is ```C++ template<typename T> T&& identitiy(T&& arg) {    return std::forward<T>(arg); } ``` if you want it to work like `identity(arg)`. **TL;DR:** C++ doesn't do it the simple way for a reason. If you do ```C++ template<typename T> T identitiy(T arg) {    return arg; } ``` you get copies unless you call it `identity<int&>(arg)` and `identity<int&&>(1)` (which defeats the purpose, I guess. Template type deduction alone (C++ has 4 different rule sets about type deduction) is complicated. One reason for stripping `ref` from the argument types unless they explicitly bind to `ref` parameters is that references should act like aliases: ```C++ int i; int& r = i; f(i); f(r); ``` Here, `f(r)` should act the same as `f(i)` irrespective of how `f` is defined. `f` could take `int`, `int&`, `const int&`, or `T`, `const T&`, or `T&&` for some `T` that can implicitly constructed from `int`. For `int&&` or `T&`it wouldn't compile. References as type constructors are weird. And please note, this is what I *immediately* could come up with and I verified. There's stuff with function types and static array bounds that I remember have shenanigans with references, but I'd have to do some search for details. The only thing D does wrong in this regard is not having `forward` and `move` in `object.d`.
Very well put, thank you.
Apr 07
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/5/21 10:14 PM, Paul Backus wrote:
 On Tuesday, 6 April 2021 at 01:47:47 UTC, Q. Schroll wrote:
 On Friday, 26 March 2021 at 15:12:04 UTC, Paul Backus wrote:
 Of course the ideal would be to make `ref` itself into a type 
 qualifier (`ref(T)`).
If you look at all the exceptions C++ has for its reference type constructor, you'll immediately see why Walter created `ref` as a storage class and not a type constructor. C++ reference types in many cases don't compose; e.g. `vector<int&>` isn't a thing. If you really think about it, you don't need `ref` as a type constructor. Although allowing `ref` local variables wouldn't be harmful, I guess.
Here's what the generic identity function currently looks like in D:    auto ref T identity(T)(auto ref T arg)    {        import core.lifetime: forward;        return forward!arg;    } Here's what the generic identity function *would* look like if `ref` were a type qualifier:    T identity(T)(T arg)    {        return arg;    }
I'm not so sure. Complications arise when you pass an lvalue to identity - is it to be considered of type T or ref T? There are pros and cons to each, which is why C++ chose to allow both; its unprecedented decltype(auto) was defined especially to be part of function return type when it is to transport refness out. Same class of problems have caused other anomalies in C++, such as decltype(identifier) and decltype((identifier)) having different types. The case is open and hot.
Apr 07
parent Paul Backus <snarwin gmail.com> writes:
On Thursday, 8 April 2021 at 02:10:55 UTC, Andrei Alexandrescu 
wrote:
 I'm not so sure. Complications arise when you pass an lvalue to 
 identity - is it to be considered of type T or ref T? There are 
 pros and cons to each, which is why C++ chose to allow both; 
 its unprecedented decltype(auto) was defined especially to be 
 part of function return type when it is to transport refness 
 out.
In the hypothetical world where `ref` is a type qualifier, `T` and `ref(T)` are distinct types. So if the lvalue's type were `T` you'd get `identity!T`, and if its type were `ref(T)`, you'd get `identity!(ref(T))`. If you had a non-`ref` lvalue that you wanted to pass by reference to a template function that takes an argument not declared as `ref` (or `auto ref`), you'd add an explicit conversion to `ref(T)` at the call site. None of this requires any special cases or complications. Unfortunately, there's no way to get to this world without a fairly substantial breaking change. Currently, it's possible to use by-value and by-reference overloads to distinguish between rvalues and lvalues: void fun(int n) { writeln("rvalue"); } void fun(ref int n) { writeln("lvalue"); } int a = 123; fun(a); // lvalue fun(456); // rvalue Making `ref` into a type qualifier would cause the rvalue overload to be called in both cases, since it would be an exact match vs. a match by implicit conversion. Realistically, this is probably too disruptive a change for too little benefit, so I expect the best we'll get is a library `Ref!T` type, like C++'s `std::reference_wrapper`. Fortunately D has the tools to make such a type fairly ergonomic to use.
Apr 07
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/24/21 3:23 PM, Per Nordlöw wrote:
 What's the motive behinds D's range design choice of needing
 
      if (!empty)
      {
          // use front or back
      }
 
 instead of having front returning an optional/maybe type with enforced 
 pattern matching?
 
 Lack of builtin Optional type?
 
 Choosing the Optional path would have avoided the need for putting error 
 diagnostics such as
 
 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a0445
0c4e6912a13cb5ac694 
 
 
 in the standard library of such an alternative solution.
I would say separation of concerns. Consider: int someExpensiveCalc(int); auto r = someArr.map!someExpensiveCalc; r.empty is simply a forward to someArr.empty. However, r.front calls someExpensiveCalc (if not empty). So code that looks specifically at emptiness might not want to trigger the expensive call unnecessarily. e.g. canFind. Since empty is a distinct abstraction, it's the simplest building block. You can actually MAKE a "range-like" thing that uses empty to produce what you want. It's problematic the other way around. That being said, there is definitely a case for straight-input-ranges to have a different API (maybe one without empty), since iterating a non-forward range can possibly destroy it, and if you change the API to simply an Optional!T next(), then you don't need to cache the element (currently I think the biggest wart in the range system). But such a thing would have to possibly be supported in the language (ranges are a language feature in that they hook to foreach, I would expect the same for such input ranges). Regarding the question about the github commit, that's an entirely different thing. Notice that it's an assert, which means it can be removed in correctly-written programs. If you return an Optional, emptiness MUST be checked on every access to an element, even when you know (or have proven) it's not empty. I would support a new kind of range that uses `Optional!T next()` as its API. I would not support `Optional!T front(); void popFront()`. -Steve
Mar 26
next sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 26 March 2021 at 15:53:39 UTC, Steven Schveighoffer 
wrote:
 ...
 I would support a new kind of range that uses `Optional!T 
 next()` as its API. I would not support `Optional!T front(); 
 void popFront()`.
Interesting. For reference, note that Rust also has https://doc.rust-lang.org/std/iter/struct.Peekable.html alongside https://doc.rust-lang.org/core/iter/trait.Iterator.html
Mar 26
prev sibling parent reply Nick Treleaven <nick geany.org> writes:
On Friday, 26 March 2021 at 15:53:39 UTC, Steven Schveighoffer 
wrote:
 Notice that it's an assert, which means it can be removed in 
 correctly-written programs. If you return an Optional, 
 emptiness MUST be checked on every access to an element, even 
 when you know (or have proven) it's not empty.
An optional would have an `unwrap` method for this purpose, which also only asserts it is not empty and returns a value.
Mar 26
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/26/21 3:20 PM, Nick Treleaven wrote:
 On Friday, 26 March 2021 at 15:53:39 UTC, Steven Schveighoffer wrote:
 Notice that it's an assert, which means it can be removed in 
 correctly-written programs. If you return an Optional, emptiness MUST 
 be checked on every access to an element, even when you know (or have 
 proven) it's not empty.
An optional would have an `unwrap` method for this purpose, which also only asserts it is not empty and returns a value.
That could be possible if the Optional type lazily fetches the value/empty status, in which case now you are dealing with delegates and/or closures. I don't think this is the right path. -Steve
Mar 26
parent reply Nick Treleaven <nick geany.org> writes:
On Saturday, 27 March 2021 at 01:06:37 UTC, Steven Schveighoffer 
wrote:
 That could be possible if the Optional type lazily fetches the 
 value/empty status, in which case now you are dealing with 
 delegates and/or closures. I don't think this is the right path.
You are of course right that returning an optional must eagerly fetch a value. I was just responding to you saying you have to check an optional is empty before accessing the value - you don't.
Mar 27
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/27/21 5:50 AM, Nick Treleaven wrote:
 On Saturday, 27 March 2021 at 01:06:37 UTC, Steven Schveighoffer wrote:
 That could be possible if the Optional type lazily fetches the 
 value/empty status, in which case now you are dealing with delegates 
 and/or closures. I don't think this is the right path.
You are of course right that returning an optional must eagerly fetch a value. I was just responding to you saying you have to check an optional is empty before accessing the value - you don't.
I meant on every call to `front`. With the current system, you can avoid this by disabling asserts. With the proposed system, in order to properly construct the Optional, emptiness must be checked. The only way to defer this check to the user is to have Optional be lazily constructed. -Steve
Mar 27
prev sibling next sibling parent uranuz <neuranuz gmail.com> writes:
On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:
 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.
Hello! It is a good idea what are you talking about. But I agree that it's better to implement it as some method using UFCS. So this will not break anything. Also I want to say that for me most of the times I check for *non empty* rather than *empty*. So every time I need to write exclamation mark before it. It is not very comfortable. But the problem is that I can't say antonym for the word *empty*. It actually shall not be antonym, but the word that say that this range should have at least one element. My variants are: hasNext, hasFront, hasMore, hasAny or smth like that. The problemme with *empty* is that it has some *negative* meaning. But I read somewhere recommendations to prefer *positive* words for boolean flags except for rare cases where *negative* flags are simplier to work with, because of domain area where it's used. But I believe that range is not such a case. And I explained by my own experience that mostly I test for *hasMore* then for *empty*. Sorry for slightly off-topic post ;-)
Mar 29
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/24/21 3:23 PM, Per Nordlöw wrote:
 What's the motive behinds D's range design choice of needing
 
      if (!empty)
      {
          // use front or back
      }
 
 instead of having front returning an optional/maybe type with enforced 
 pattern matching?
Efficiency. It would be impossible to iterate an array as a range without copying each and every element thereof. We investigated a few other possibilities, such as returning a pointer to the next element or null. But that has problems related to safety and escaping pointers.
Mar 29
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 30 March 2021 at 00:51:44 UTC, Andrei Alexandrescu 
wrote:
 We investigated a few other possibilities, such as returning a 
 pointer to the next element or null. But that has problems 
 related to safety and escaping pointers.
Does -preview=dip1000 help at all with these issues? If so, might be worth revisiting.
Mar 30
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Tuesday, 30 March 2021 at 16:05:09 UTC, Paul Backus wrote:
 On Tuesday, 30 March 2021 at 00:51:44 UTC, Andrei Alexandrescu 
 wrote:
 We investigated a few other possibilities, such as returning a 
 pointer to the next element or null. But that has problems 
 related to safety and escaping pointers.
Does -preview=dip1000 help at all with these issues? If so, might be worth revisiting.
DIP1000 is not sufficient, and unsound by itself.
Mar 30
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/30/21 12:05 PM, Paul Backus wrote:
 On Tuesday, 30 March 2021 at 00:51:44 UTC, Andrei Alexandrescu wrote:
 We investigated a few other possibilities, such as returning a pointer 
 to the next element or null. But that has problems related to safety 
 and escaping pointers.
Does -preview=dip1000 help at all with these issues? If so, might be worth revisiting.
A good point.
Mar 30
prev sibling next sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 30 March 2021 at 00:51:44 UTC, Andrei Alexandrescu 
wrote:
 We investigated a few other possibilities, such as returning a 
 pointer to the next element or null. But that has problems 
 related to safety and escaping pointers.
I'm eager to experiment with compiler diagnostics that, for a growing number of cases, detects calls to X.front/back and X.popFront/popBack when X is `empty`. What criteria would need to be fulfilled on `X` and its range interface members in order for such a diagnostics to kick in?
Mar 30
prev sibling next sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 30 March 2021 at 00:51:44 UTC, Andrei Alexandrescu 
wrote:
 Efficiency. It would be impossible to iterate an array as a 
 range without copying each and every element thereof.
Most likely affects performance in non-release mode, at least. I wonder what the generated assembly looks like in Rust for a similar case...
Mar 30
prev sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Tuesday, 30 March 2021 at 00:51:44 UTC, Andrei Alexandrescu 
wrote:
 Efficiency. It would be impossible to iterate an array as a 
 range without copying each and every element thereof.
On an efficiency-related note, it's worth remembering that the `empty` property can be a compile-time constant (as it is e.g. with pseudo-random number generators), which means that in some cases it should be possible (at least in principle) to elide any `empty` checks at runtime. I imagine there is a way to achieve the same effective outcome with an appropriately designed Option type or trait, but I don't know if that would be as straightforward as with the `empty` property.
 We investigated a few other possibilities, such as returning a 
 pointer to the next element or null. But that has problems 
 related to safety and escaping pointers.
This is probably a factor in Rust's decision to use Option<T>: its iterators over containers do use references (i.e. smart pointers), and this is easier option for them because of the stronger reference safety constraints.
Mar 31
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2021-03-24 20:23, Per Nordlöw wrote:
 What's the motive behinds D's range design choice of needing
 
     if (!empty)
     {
         // use front or back
     }
 
 instead of having front returning an optional/maybe type with enforced 
 pattern matching?
In my opinion, the best way to interact with an Optional type is not to use pattern matching but to use algorithms like `map`, `each` and so on. That means an Optional type needs to implement the range API. Might be a bit confusioning if `front` then returns an Optional. -- /Jacob Carlborg
Mar 31
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 31 March 2021 at 07:27:18 UTC, Jacob Carlborg wrote:
way to interact with an Optional type
 is not to use pattern matching but to use algorithms like 
 `map`, `each` and so on.
Interesting. Can you show some pseudocode of this? Can those algorithms be implemented as non-member functions?
Apr 05
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 5 April 2021 at 20:01:46 UTC, Per Nordlöw wrote:
 On Wednesday, 31 March 2021 at 07:27:18 UTC, Jacob Carlborg 
 wrote:
 way to interact with an Optional type
 is not to use pattern matching but to use algorithms like 
 `map`, `each` and so on.
Interesting. Can you show some pseudocode of this? Can those algorithms be implemented as non-member functions?
The `optional` package on dub [1] has some examples in its documentation. [1] https://code.dlang.org/packages/optional
Apr 05
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 5 April 2021 at 20:11:07 UTC, Paul Backus wrote:
 The `optional` package on dub [1] has some examples in its 
 documentation.
Thanks. What about adding these algorithms to std.sumtype?
Apr 05
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 5 April 2021 at 20:33:44 UTC, Per Nordlöw wrote:
 On Monday, 5 April 2021 at 20:11:07 UTC, Paul Backus wrote:
 The `optional` package on dub [1] has some examples in its 
 documentation.
Thanks. What about adding these algorithms to std.sumtype?
Optional implements the standard range interface, so the algorithms it uses are the existing ones in std.algorithm.
Apr 05
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 5 April 2021 at 20:38:33 UTC, Paul Backus wrote:
 Optional implements the standard range interface, so the 
 algorithms it uses are the existing ones in std.algorithm.
Ah, of course. Are you planning on adding Optional to Phobos aswell?
Apr 05
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 5 April 2021 at 20:46:07 UTC, Per Nordlöw wrote:
 On Monday, 5 April 2021 at 20:38:33 UTC, Paul Backus wrote:
 Optional implements the standard range interface, so the 
 algorithms it uses are the existing ones in std.algorithm.
Ah, of course. Are you planning on adding Optional to Phobos aswell?
No, because it's not my package, it's Ali Akhtarzada's.
Apr 05
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 5 April 2021 at 21:43:30 UTC, Paul Backus wrote:
 No, because it's not my package, it's Ali Akhtarzada's.
Is there a place for it in Phobos?
Apr 05
parent Paul Backus <snarwin gmail.com> writes:
On Monday, 5 April 2021 at 22:21:39 UTC, Per Nordlöw wrote:
 On Monday, 5 April 2021 at 21:43:30 UTC, Paul Backus wrote:
 No, because it's not my package, it's Ali Akhtarzada's.
Is there a place for it in Phobos?
I think there's a place for something like it in Phobos. Whether that's `Optional` itself, an improved version of `std.typecons.Nullable`, or something else entirely is an open question. And of course, Atila has the final say.
Apr 05
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2021-04-05 22:01, Per Nordlöw wrote:

 Interesting. Can you show some pseudocode of this? Can those algorithms 
 be implemented as non-member functions?
Here's an article about idiomatic Scala and optional types [1]. I think it applies to D as well. [1] https://www.originate.com/thinking/idiomatic-scala-your-options-do-not-match -- /Jacob Carlborg
Apr 07