www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Discussion Thread: DIP 1033--Implicit Conversion of Expressions to

reply Mike Parker <aldacron gmail.com> writes:
This is the discussion thread for the first round of Community 
Review of DIP 1033, "Implicit Conversion of Expressions to 
Delegates":

https://github.com/dlang/DIPs/blob/7b61411cb6cf8db05d9b8e1df5d2d9bae53a5f1e/DIPs/DIP1033.md

The review period will end at 11:59 PM ET on May 6, or when I 
make a post declaring it complete. Discussion in this thread may 
continue beyond that point.

Here in the discussion thread, you are free to discuss anything 
and everything related to the DIP. Express your support or 
opposition, debate alternatives, argue the merits, etc.

However, if you have any specific feedback on how to improve the 
proposal itself, then please post it in the feedback thread. The 
feedback thread will be the source for the review summary I write 
at the end of this review round. I will post a link to that 
thread immediately following this post. Just be sure to read and 
understand the Reviewer Guidelines before posting there:

https://github.com/dlang/DIPs/blob/master/docs/guidelines-reviewers.md

And my blog post on the difference between the Discussion and 
Feedback threads:

https://dlang.org/blog/2020/01/26/dip-reviews-discussion-vs-feedback/

Please stay on topic here. I will delete posts that are 
completely off-topic.
Apr 22
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Wednesday, 22 April 2020 at 07:41:12 UTC, Mike Parker wrote:

 Here in the discussion thread, you are free to discuss anything 
 and everything related to the DIP. Express your support or 
 opposition, debate alternatives, argue the merits, etc.

 However, if you have any specific feedback on how to improve 
 the proposal itself, then please post it in the feedback 
 thread. The feedback thread will be the source for the review 
 summary I write at the end of this review round. I will post a 
 link to that thread immediately following this post.
The Feedback Thread is here: https://forum.dlang.org/post/nxahrsukobybkezibcsm forum.dlang.org
Apr 22
prev sibling next sibling parent reply kinke <noone nowhere.com> writes:
I don't think this is a good idea. One of the problems with 
`lazy` IMO is that one needs good IDE support to somehow 
highlight that the argument expression isn't evaluated before the 
call, and might not be evaluated at all:

// in some other module:
void foo(lazy int i) {}

void main()
{
     int i = 1;
     foo(++i);
     assert(i == 1); // what? not incremented above? ah yeah, lazy 
and not evaluated at all by foo
}

The proposal would extend that problem to regular delegate params 
too. So I find the explicit version (`() => ++i`) better to make 
that clear. I don't think the 'overhead' of putting an additional 
`() =>` in front of the expression justifies introducing the 
proposed implicit convertibility.
Apr 22
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/22/20 7:56 AM, kinke wrote:
 I don't think this is a good idea. One of the problems with `lazy` IMO 
 is that one needs good IDE support to somehow highlight that the 
 argument expression isn't evaluated before the call, and might not be 
 evaluated at all:
 
 // in some other module:
 void foo(lazy int i) {}
 
 void main()
 {
      int i = 1;
      foo(++i);
      assert(i == 1); // what? not incremented above? ah yeah, lazy and 
 not evaluated at all by foo
 }
This is more of a problem of lazy parameters that mutate something. Most of the time, that isn't the use case of lazy -- it's to prevent calling some expensive thing when it's not needed. Most definitely this DIP would make that problem worse. What could be something that helps is enforcing that expressions to delegate implicit conversions require no mutation. I'm not sure how that could be enforced, but possibly requiring only pure function calls on const data (including the context). It wouldn't be a breaking change, because currently expressions don't bind to delegates, so we can enforce that rule there, then deprecate mutating expressions on lazy parameters, and finally remove lazy.
 
 The proposal would extend that problem to regular delegate params too. 
 So I find the explicit version (`() => ++i`) better to make that clear. 
 I don't think the 'overhead' of putting an additional `() =>` in front 
 of the expression justifies introducing the proposed implicit 
 convertibility.
I've never had much of an issue with the verbosity of delegates, especially since the () => form came about. What I HAVE had an issue with is not being able to pass a delegate into a lazy parameter. e.g.: void foo(lazy int x); foo(() {some; large; calculation; return result; }); // no good. foo(() {...}()); // works but, is a delegate that calls a delegate, ugh. To that end, would it be possible to add lazy binding to a delegate in the same DIP? The more we make lazy behave like a delegate, the easier it will be to remove lazy. -Steve
Apr 22
parent reply kinke <noone nowhere.com> writes:
On Wednesday, 22 April 2020 at 12:30:52 UTC, Steven Schveighoffer 
wrote:
 This is more of a problem of lazy parameters that mutate 
 something. Most of the time, that isn't the use case of lazy -- 
 it's to prevent calling some expensive thing when it's not 
 needed.
I disagree - `foo(myData.recomputeFromScratch())` has the same problem, it's not obvious from the call site alone that the expensive operation is deferred and might be skipped.
 To that end, would it be possible to add lazy binding to a 
 delegate in the same DIP? The more we make lazy behave like a 
 delegate, the easier it will be to remove lazy.
That might be a migration path towards removing lazy, so that the call sites can switch to delegates already, and the lazy params transformed to explicit delegates in a 2nd step. No idea if such a two-step approach would be worth it though.
Apr 22
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/22/20 10:04 AM, kinke wrote:
 On Wednesday, 22 April 2020 at 12:30:52 UTC, Steven Schveighoffer wrote:
 This is more of a problem of lazy parameters that mutate something. 
 Most of the time, that isn't the use case of lazy -- it's to prevent 
 calling some expensive thing when it's not needed.
I disagree - `foo(myData.recomputeFromScratch())` has the same problem, it's not obvious from the call site alone that the expensive operation is deferred and might be skipped.
But if myData.recomputeFromScratch() has no side effects, what does it matter if not used or called? For example, if you have: auto foo(T)(bool cond, T trueValue, T falseValue) { return cond ? trueValue : falseValue; } And you call like: auto x = foo(cond, val, expensivePureCalc()); A sufficiently intelligent compiler is going to avoid calling expensivePureCalc when cond is false. Is that a problem? Really the only use of lazy I use is in enforce. The lazy message might do GC allocations to generate a string that is thrown away if not used. But the string generation doesn't mutate anything, so it's not going to have an effect on the operation of the program if not called. What I'm saying is if the DIP *enforced* that expression-to-delegate happens only for these cases, then there is no worry about whether the expression is evaluated or not. -Steve
Apr 22
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/22/20 11:15 AM, Steven Schveighoffer wrote:
 A sufficiently intelligent compiler is going to avoid calling 
 expensivePureCalc when cond is false. Is that a problem?
ugh, when cond is *true* obviously ;) -Steve
Apr 22
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 22 April 2020 at 15:15:24 UTC, Steven Schveighoffer 
wrote:
 Really the only use of lazy I use is in enforce.
I'd say that object.d has some pretty good use cases in addition to enforce: inout(V) get(K, V)(inout(V[K]) aa, K key, lazy inout(V) defaultValue): https://github.com/dlang/druntime/blob/v2.091.0/src/object.d#L2576 ref V require(K, V)(ref V[K] aa, K key, lazy V value = V.init): https://github.com/dlang/druntime/blob/v2.091.0/src/object.d#L2605
Apr 22
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/22/20 12:50 PM, Petar Kirov [ZombineDev] wrote:
 On Wednesday, 22 April 2020 at 15:15:24 UTC, Steven Schveighoffer wrote:
 Really the only use of lazy I use is in enforce.
I'd say that object.d has some pretty good use cases in addition to enforce: inout(V) get(K, V)(inout(V[K]) aa, K key, lazy inout(V) defaultValue): https://github.com/dlang/druntime/blob/v2.091.0/src/object.d#L2576 ref V require(K, V)(ref V[K] aa, K key, lazy V value = V.init): https://github.com/dlang/druntime/blob/v2.091.0/src/object.d#L2605
Right, and I still think doing mutations in those expressions would be confusing. -Steve
Apr 22
prev sibling parent reply aliak <something something.com> writes:
On Wednesday, 22 April 2020 at 11:56:44 UTC, kinke wrote:
 I don't think this is a good idea. One of the problems with 
 `lazy` IMO is that one needs good IDE support to somehow 
 highlight that the argument expression isn't evaluated before 
 the call, and might not be evaluated at all:

 // in some other module:
 void foo(lazy int i) {}

 void main()
 {
     int i = 1;
     foo(++i);
     assert(i == 1); // what? not incremented above? ah yeah, 
 lazy and not evaluated at all by foo
 }

 The proposal would extend that problem to regular delegate 
 params too. So I find the explicit version (`() => ++i`) better 
 to make that clear. I don't think the 'overhead' of putting an 
 additional `() =>` in front of the expression justifies 
 introducing the proposed implicit convertibility.
I agree with this. Implicit conversion has always been argued against in D - why is it ok now? And this is not the good kind of implicit conversion where the user explicitly asks for it, but the bad kind where the language decides it should happen and the user doesn't know it can even happen. Thinking about it more now feels like the real rationale for the DIP is "in order to deprecate lazy, we need an alternative" ... ???
Apr 23
parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Thursday, 23 April 2020 at 09:33:35 UTC, aliak wrote:
 On Wednesday, 22 April 2020 at 11:56:44 UTC, kinke wrote:
 I don't think this is a good idea. One of the problems with 
 `lazy` IMO is that one needs good IDE support to somehow 
 highlight that the argument expression isn't evaluated before 
 the call, and might not be evaluated at all:

 // in some other module:
 void foo(lazy int i) {}

 void main()
 {
     int i = 1;
     foo(++i);
     assert(i == 1); // what? not incremented above? ah yeah, 
 lazy and not evaluated at all by foo
 }

 The proposal would extend that problem to regular delegate 
 params too. So I find the explicit version (`() => ++i`) 
 better to make that clear. I don't think the 'overhead' of 
 putting an additional `() =>` in front of the expression 
 justifies introducing the proposed implicit convertibility.
I agree with this. Implicit conversion has always been argued against in D - why is it ok now? And this is not the good kind of implicit conversion where the user explicitly asks for it, but the bad kind where the language decides it should happen and the user doesn't know it can even happen. Thinking about it more now feels like the real rationale for the DIP is "in order to deprecate lazy, we need an alternative" ... ???
I always thought that lazy should also be marked at the call site as it is a very different semantic on the caller and on the callee. As it is now, it is true that the example above is unfortunate and a real source of unpleasant surprizes. void foo(lazy int i) {} void main() { int i = 1; foo(lazy ++i); assert(i == 1); // No surprize here, the required annotation tells it. }
Apr 23
next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Thursday, 23 April 2020 at 11:13:34 UTC, Patrick Schluter 
wrote:
 I always thought that lazy should also be marked at the call 
 site as it is a very different semantic on the caller and on 
 the callee. As it is now, it is true that the example above is 
 unfortunate and a real source of unpleasant surprizes.
+1 Step 1: lazy must be used at the call site Step 2: deprecate lazy Step 3: remove this terrible, unreadable idea that was done hastily years ago with no vote whatsoever
Apr 23
parent reply Meta <jared771 gmail.com> writes:
On Thursday, 23 April 2020 at 13:20:08 UTC, Guillaume Piolat 
wrote:
 On Thursday, 23 April 2020 at 11:13:34 UTC, Patrick Schluter 
 wrote:
 I always thought that lazy should also be marked at the call 
 site as it is a very different semantic on the caller and on 
 the callee. As it is now, it is true that the example above is 
 unfortunate and a real source of unpleasant surprizes.
+1 Step 1: lazy must be used at the call site
That would play havoc with generic code and would significantly increase the complexity of standard phobos functions like map, filter, etc.
Apr 23
parent reply aliak <something something.com> writes:
On Thursday, 23 April 2020 at 14:49:53 UTC, Meta wrote:
 On Thursday, 23 April 2020 at 13:20:08 UTC, Guillaume Piolat 
 wrote:
 On Thursday, 23 April 2020 at 11:13:34 UTC, Patrick Schluter 
 wrote:
 I always thought that lazy should also be marked at the call 
 site as it is a very different semantic on the caller and on 
 the callee. As it is now, it is true that the example above 
 is unfortunate and a real source of unpleasant surprizes.
+1 Step 1: lazy must be used at the call site
That would play havoc with generic code and would significantly increase the complexity of standard phobos functions like map, filter, etc.
Can you elaborate? How so?
Apr 23
next sibling parent aliak <something something.com> writes:
On Thursday, 23 April 2020 at 20:15:19 UTC, aliak wrote:
 On Thursday, 23 April 2020 at 14:49:53 UTC, Meta wrote:
 On Thursday, 23 April 2020 at 13:20:08 UTC, Guillaume Piolat 
 wrote:
 On Thursday, 23 April 2020 at 11:13:34 UTC, Patrick Schluter 
 wrote:
 I always thought that lazy should also be marked at the call 
 site as it is a very different semantic on the caller and on 
 the callee. As it is now, it is true that the example above 
 is unfortunate and a real source of unpleasant surprizes.
+1 Step 1: lazy must be used at the call site
That would play havoc with generic code and would significantly increase the complexity of standard phobos functions like map, filter, etc.
Can you elaborate? How so?
Nevermind. Got it! I wonder if phobos generic code should even account for that 🤔
Apr 23
prev sibling parent Meta <jared771 gmail.com> writes:
On Thursday, 23 April 2020 at 20:15:19 UTC, aliak wrote:
 On Thursday, 23 April 2020 at 14:49:53 UTC, Meta wrote:
 On Thursday, 23 April 2020 at 13:20:08 UTC, Guillaume Piolat 
 wrote:
 On Thursday, 23 April 2020 at 11:13:34 UTC, Patrick Schluter 
 wrote:
 I always thought that lazy should also be marked at the call 
 site as it is a very different semantic on the caller and on 
 the callee. As it is now, it is true that the example above 
 is unfortunate and a real source of unpleasant surprizes.
+1 Step 1: lazy must be used at the call site
That would play havoc with generic code and would significantly increase the complexity of standard phobos functions like map, filter, etc.
Can you elaborate? How so?
This is not how those functions are implemented, but it should illustrate my point: auto function map(alias fun)(Range!T range) { T[] result; foreach (i, item; range) result[i] = fun(item); return result; } int evenicize(lazy int n) { if (n % 2 == 1) return n * 2; } auto odds = [1, 3, 5, 7]; //If the call site needs an explicit "lazy" for arguments to lazy parameters: auto evens = odds.map!evenicize(); //Error: must specify "lazy" for parameter n when calling function "evenicize" So then map has to account add a bunch of extra machinery to detect whether "fun" has any lazy parameters: template hasLazyParameter(alias fun) { //Some implementation } auto function map(alias fun)(Range!T range) { //Note, if you had to write this as "fun(lazy T.init)", typeof would fail here typeof(fun(T.init))[] result; foreach (i, item; range) static if (hasLazyParameter!fun) result[i] = fun(lazy item); else result[i] = fun(item); return result; } Which actually is not that bad for map, as the function you pass to it only takes 1 parameter (although IMO, this is already too onerous). But what about something like std.functional.reverseArgs, which has to be able to handle functions with arbitrary arity?
Apr 23
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/23/20 7:13 AM, Patrick Schluter wrote:
 On Thursday, 23 April 2020 at 09:33:35 UTC, aliak wrote:
 On Wednesday, 22 April 2020 at 11:56:44 UTC, kinke wrote:
 I don't think this is a good idea. One of the problems with `lazy` 
 IMO is that one needs good IDE support to somehow highlight that the 
 argument expression isn't evaluated before the call, and might not be 
 evaluated at all:

 // in some other module:
 void foo(lazy int i) {}

 void main()
 {
     int i = 1;
     foo(++i);
     assert(i == 1); // what? not incremented above? ah yeah, lazy and 
 not evaluated at all by foo
 }

 The proposal would extend that problem to regular delegate params 
 too. So I find the explicit version (`() => ++i`) better to make that 
 clear. I don't think the 'overhead' of putting an additional `() =>` 
 in front of the expression justifies introducing the proposed 
 implicit convertibility.
I agree with this. Implicit conversion has always been argued against in D - why is it ok now? And this is not the good kind of implicit conversion where the user explicitly asks for it, but the bad kind where the language decides it should happen and the user doesn't know it can even happen. Thinking about it more now feels like the real rationale for the DIP is "in order to deprecate lazy, we need an alternative" ... ???
I always thought that lazy should also be marked at the call site as it is a very different semantic on the caller and on the callee. As it is now, it is true that the example above is unfortunate and a real source of unpleasant surprizes. void foo(lazy int i) {}  void main()  {      int i = 1;      foo(lazy ++i);      assert(i == 1); // No surprize here, the required annotation tells it.  }
If lazy binds to delegates then this just becomes foo(() => ++i); And then we can require lazy to require a delegate at the call site, and then get rid of lazy. -Steve
Apr 23
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Wednesday, 22 April 2020 at 07:41:12 UTC, Mike Parker wrote:
 Here in the discussion thread, you are free to discuss anything 
 and everything related to the DIP. Express your support or 
 opposition, debate alternatives, argue the merits, etc.
I don't like it. I'm actively avoiding lazy, and delegates are generally not very usable in nothrow nogc -betterC modes. More easy syntax for delegates sounds not that desirable to me. I'd prefer if lazy was removed.
Apr 22
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 22 April 2020 at 16:37:28 UTC, Guillaume Piolat 
wrote:
 On Wednesday, 22 April 2020 at 07:41:12 UTC, Mike Parker wrote:
 Here in the discussion thread, you are free to discuss 
 anything and everything related to the DIP. Express your 
 support or opposition, debate alternatives, argue the merits, 
 etc.
I don't like it. I'm actively avoiding lazy, and delegates are generally not very usable in nothrow nogc -betterC modes. More easy syntax for delegates sounds not that desirable to me. I'd prefer if lazy was removed.
If this DIP is accepted, `lazy` could be deprecated, as it could be fully replaced by scope delegates, which are nothrow nogc -betterC compatible.
Apr 22
parent reply kinke <noone nowhere.com> writes:
On Wednesday, 22 April 2020 at 16:53:09 UTC, Petar Kirov 
[ZombineDev] wrote:
 If this DIP is accepted, `lazy` could be deprecated, as it 
 could be fully replaced by scope delegates, which are nothrow 
  nogc -betterC compatible.
We don't need the DIP for that. It would just allow not having to touch the call sites (just the lazy *params*), but as said, I think the arguments should be forced to be converted to regular delegates too, for legibility and less confusion.
Apr 22
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 22 April 2020 at 17:48:35 UTC, kinke wrote:
 On Wednesday, 22 April 2020 at 16:53:09 UTC, Petar Kirov 
 [ZombineDev] wrote:
 If this DIP is accepted, `lazy` could be deprecated, as it 
 could be fully replaced by scope delegates, which are nothrow 
  nogc -betterC compatible.
We don't need the DIP for that.
We need it in order to avoid API breaking changes. If we deprecate `lazy`, without this DIP this code will break: const result = cache.require(key, calculateValue(key));
 It would just allow not having to touch the call sites (just 
 the lazy *params*), but as said, I think the arguments should 
 be forced to be converted to regular delegates too, for 
 legibility and less confusion.
Reasonable people may disagree. Just like C programmers like the explicitness of passing a pointer to a local variable, while `T&` is being touted as one of the big improvements of C++, compared to C.
Apr 22
parent kinke <noone nowhere.com> writes:
On Wednesday, 22 April 2020 at 19:08:08 UTC, Petar Kirov 
[ZombineDev] wrote:
 We need it in order to avoid API breaking changes. If we 
 deprecate `lazy`, without this DIP this code will break:

   const result = cache.require(key, calculateValue(key));
I thought these cases could later be simplified by overloading these few functions, taking either a value directly or a factory delegate. But after scanning for lazy params in Phobos, it looks like its use is reasonably restricted to a few places where the shortcut syntax is indeed nice (enforce, assertThrown, logging...). But I still think this DIP would be a step in the wrong direction; instead, I'd prefer `lazy` to keep designating params where arguments are eligible for this shortcut syntax / implicit conversion of expressions to delegates. The problems with lazy ('being an oddity means it is hard to reason about, especially with the proliferation of parameter attributes') might be solved by transitioning to requiring a delegate type for lazy parameters, e.g.: void write(string); alias Factory(T) = scope T delegate(); void log(lazy Factory!bool condition, lazy Factory!string msg) { if (condition()) write(msg()); } void foo(int p) { log(true, "bla"); log(() => ++p, "blub"); // enable optional delegate syntax too } lazy is used very sparingly in druntime/Phobos, and AFAIK, nobody has requested the proposed implicit conversion for other expressions so far. So instead of trying out something completely new and opening the door for abuse and confusion, I think we should rather focus on the few sprinkled usages of lazy params.
Apr 22
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
Followup discussion to Walter's response to my feedback:

 int delegate() = 3;

 doesn't use any globals. Why shouldn't this work?

 int function() = 3;
Because it seems kinda pointless to support function pointers only with constants. While it is not impossible to support function pointers with this, I can't think of a compelling use case.
Walter, you literally are using int delegate() = 3 as a use case in the DIP. How is a delegate that returns a constant more useful (or different) than a function pointer that returns a constant? The statement "the expression can only consist of globals" is just factually incorrect. It can consist of globals, new memory, literals, enums, thread local data, the result of some other function call with those types of parameters. I tend to think that it doesn't make a whole lot of sense to exclude function pointers in this DIP, but even if you disagree, the text fails to convey a valid rationale of why they shouldn't be included. -Steve
Apr 22