www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - String lambdas

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
https://github.com/dlang/phobos/pull/3882

I just closed with some regret a nice piece of engineering. Please 
comment if you think string lambdas have a lot of unexploited potential.

One thing we really need in order to 100% replace string lambdas with 
lambdas is function equivalence. Right now we're in the odd situation 
that SomeTemplate!((a, b) => a < b) has distinct types, one per 
instantiation.


Andrei
Apr 26 2016
next sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Tuesday, 26 April 2016 at 17:58:22 UTC, Andrei Alexandrescu 
wrote:
 One thing we really need in order to 100% replace string 
 lambdas with lambdas is function equivalence.
One significant - but undocumented - feature of unaryFun and binaryFun, is that they will auto-import the following modules if needed: import std.traits, std.typecons, std.typetuple; import std.algorithm, std.conv, std.exception, std.math, std.range, std.string; Does anyone actually use this feature?
Apr 26 2016
prev sibling next sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tuesday, April 26, 2016 13:58:22 Andrei Alexandrescu via Digitalmars-d 
wrote:
 https://github.com/dlang/phobos/pull/3882

 I just closed with some regret a nice piece of engineering. Please
 comment if you think string lambdas have a lot of unexploited potential.
Well, they're nicer than the "new" lambda syntax for really short stuff like "a == b" or "a != b", but beyond that, the newer syntax mostly does what the string lambdas were trying to do. And if your lambda gets very long, it really should be replaced with something like a nested function anyway. Pretty much anything complicated shouldn't be a lambda, or it risks being unmaintainable. So, while that PR was indeed a nice piece of engineering, it seems to be solving a problem that really shouldn't be solved with lambdas anyway.
 One thing we really need in order to 100% replace string lambdas with
 lambdas is function equivalence. Right now we're in the odd situation
 that SomeTemplate!((a, b) => a < b) has distinct types, one per
 instantiation.
Once that's solved, we can consider deprecating string lambdas, but equivalence is indeed the one killer feature that non-string lambdas lack. However, from what I recall of discussions on that, it sounded like it was going to be pretty nasty to implement. :( - Jonathan M Davis
Apr 26 2016
prev sibling next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Tuesday, 26 April 2016 at 17:58:22 UTC, Andrei Alexandrescu 
wrote:
 https://github.com/dlang/phobos/pull/3882

 I just closed with some regret a nice piece of engineering. 
 Please comment if you think string lambdas have a lot of 
 unexploited potential.
I'm of the opinion that string lambdas must go. I started, and I really should finish it at some point, removing string lambdas from the documentation: https://github.com/dlang/phobos/pull/3800 I think that the drawback you mentioned does not outweigh the benefits gained from using actual lambdas.
Apr 26 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/26/2016 03:45 PM, Jack Stouffer wrote:
 I think that the drawback you mentioned does not outweigh the benefits
 gained from using actual lambdas.
Actually it turns out to be a major usability issue. -- Andrei
Apr 27 2016
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/27/16 8:31 AM, Andrei Alexandrescu wrote:
 On 04/26/2016 03:45 PM, Jack Stouffer wrote:
 I think that the drawback you mentioned does not outweigh the benefits
 gained from using actual lambdas.
Actually it turns out to be a major usability issue. -- Andrei
Yes, consider that RedBlackTree!(int, (a, b) => a > b) is going to be a different type every time you use it, even if they are 1 line apart! There are actually 2 things the string lambdas have going for them: 1) common instantiation for every usage, and 2) avoiding parentheses with instantiation (impossible to do with a short lambda syntax). I'd still vote for them to go, but we MUST fix the issue with common instantiation first. There has been some discussion in general of using hashing to decrease the symbol size for templates, and some discussion about allowing the compiler to merge identical binary functions to reduce the size of the binaries. Both of those could play in nicely here. -Steve
Apr 27 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/27/2016 11:44 AM, Steven Schveighoffer wrote:
 On 4/27/16 8:31 AM, Andrei Alexandrescu wrote:
 On 04/26/2016 03:45 PM, Jack Stouffer wrote:
 I think that the drawback you mentioned does not outweigh the benefits
 gained from using actual lambdas.
Actually it turns out to be a major usability issue. -- Andrei
Yes, consider that RedBlackTree!(int, (a, b) => a > b) is going to be a different type every time you use it, even if they are 1 line apart! There are actually 2 things the string lambdas have going for them: 1) common instantiation for every usage, and 2) avoiding parentheses with instantiation (impossible to do with a short lambda syntax). I'd still vote for them to go, but we MUST fix the issue with common instantiation first. There has been some discussion in general of using hashing to decrease the symbol size for templates, and some discussion about allowing the compiler to merge identical binary functions to reduce the size of the binaries. Both of those could play in nicely here.
Yes, you get it exactly right. I think a DIP would be warranted here to clarify how lambda equivalence is computed. Could you please draft one? -- Andrei
Apr 27 2016
next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/27/16 1:04 PM, Andrei Alexandrescu wrote:
 On 04/27/2016 11:44 AM, Steven Schveighoffer wrote:
 On 4/27/16 8:31 AM, Andrei Alexandrescu wrote:
 On 04/26/2016 03:45 PM, Jack Stouffer wrote:
 I think that the drawback you mentioned does not outweigh the benefits
 gained from using actual lambdas.
Actually it turns out to be a major usability issue. -- Andrei
Yes, consider that RedBlackTree!(int, (a, b) => a > b) is going to be a different type every time you use it, even if they are 1 line apart! There are actually 2 things the string lambdas have going for them: 1) common instantiation for every usage, and 2) avoiding parentheses with instantiation (impossible to do with a short lambda syntax). I'd still vote for them to go, but we MUST fix the issue with common instantiation first. There has been some discussion in general of using hashing to decrease the symbol size for templates, and some discussion about allowing the compiler to merge identical binary functions to reduce the size of the binaries. Both of those could play in nicely here.
Yes, you get it exactly right. I think a DIP would be warranted here to clarify how lambda equivalence is computed. Could you please draft one?
I started thinking about this, I'm probably not the best to do this by myself, as I'm not familiar with the compiler code. I'm thinking possibly we should talk about this next week in person? BTW, one of the issues with the DIP system I see is that the table for DIPs is not auto-generated, so there is a stray dip which hasn't been finished and no entry in the table: http://wiki.dlang.org/DIP89 -Steve
Apr 27 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 27 April 2016 at 17:04:47 UTC, Andrei Alexandrescu 
wrote:
 Yes, you get it exactly right. I think a DIP would be warranted 
 here to clarify how lambda equivalence is computed. Could you 
 please draft one? -- Andrei
More generally, it is not clear what is allowed to do for merging functions. In C/C++ it is assumed that different function MUST have different identities. Namely, if foo and bar MUST have a different address. It means that, even if foo and bar have the same body, you can't merge them. Compiler can, however, emit a branch to foo's body into bar or vice versa and it is alright. This is a problem for us if we want to merge templates. I think we should abolish this in D, to unlock more merging.
Apr 27 2016
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 28 April 2016 at 00:14:41 UTC, deadalnix wrote:
 On Wednesday, 27 April 2016 at 17:04:47 UTC, Andrei 
 Alexandrescu wrote:
 Yes, you get it exactly right. I think a DIP would be 
 warranted here to clarify how lambda equivalence is computed. 
 Could you please draft one? -- Andrei
More generally, it is not clear what is allowed to do for merging functions. In C/C++ it is assumed that different function MUST have different identities. Namely, if foo and bar MUST have a different address. It means that, even if foo and bar have the same body, you can't merge them. Compiler can, however, emit a branch to foo's body into bar or vice versa and it is alright. This is a problem for us if we want to merge templates. I think we should abolish this in D, to unlock more merging.
Seconded.
Apr 27 2016
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 28 April 2016 at 00:14:41 UTC, deadalnix wrote:
 More generally, it is not clear what is allowed to do for 
 merging functions. In C/C++ it is assumed that different 
 function MUST have different identities.
I don't think this needs to hold true for anonymous functions though. If they have the same representation twice, it is arguably just the same function referenced twice.
Apr 27 2016
parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 28 April 2016 at 03:15:36 UTC, Adam D. Ruppe wrote:
 On Thursday, 28 April 2016 at 00:14:41 UTC, deadalnix wrote:
 More generally, it is not clear what is allowed to do for 
 merging functions. In C/C++ it is assumed that different 
 function MUST have different identities.
I don't think this needs to hold true for anonymous functions though. If they have the same representation twice, it is arguably just the same function referenced twice.
I'm not enough of a language lawyer to know that. But I can tell you that compiler do not take advantage of this at this stage. Maybe it is because lambda are fairly new, I'm not sure.
Apr 28 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 27 April 2016 at 12:31:18 UTC, Andrei Alexandrescu 
wrote:
 On 04/26/2016 03:45 PM, Jack Stouffer wrote:
 I think that the drawback you mentioned does not outweigh the 
 benefits
 gained from using actual lambdas.
Actually it turns out to be a major usability issue. -- Andrei
That would be better for this to go forward in a sensible manner that you explains what is the major usability issue here.
Apr 27 2016
parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wednesday, April 27, 2016 21:18:35 deadalnix via Digitalmars-d wrote:
 On Wednesday, 27 April 2016 at 12:31:18 UTC, Andrei Alexandrescu

 wrote:
 On 04/26/2016 03:45 PM, Jack Stouffer wrote:
 I think that the drawback you mentioned does not outweigh the
 benefits
 gained from using actual lambdas.
Actually it turns out to be a major usability issue. -- Andrei
That would be better for this to go forward in a sensible manner that you explains what is the major usability issue here.
Well, it's problematic with default predicates, because you can't test that the default is being used (which some algorithms do for efficiency). You're forced to duplicate the function rather than use a default. But the far bigger problem is when dealing with types that take a predicate (which is going to happen with some containers and many ranges). e.g. this compiles RedBlackTree!(int, "a < b") a; RedBlackTree!(int, "a < b") b = a; but this does not: RedBlackTree!(int, (a, b) => a < b) a; RedBlackTree!(int, (a, b) => a < b) b = a; You get this wonderfully nonsensical error: q.d(6): Error: cannot implicitly convert expression (a) of type q.main.RedBlackTree!(int, (a, b) => a < b, false) to q.main.RedBlackTree!(int, (a, b) => a < b, false) So, you can't convert an expression of one type to an expression of exactly the same type? Sure, the error message could be improved, but I doubt that very many folks are going to expect this sort of error unless they've run into the problem previously. It totally fails to principle of least surprise. And it's definitely a usability problem. It makes passing around types with predicates a royal pain when you need to type them explicitly (which happens far more with containers than ranges). It also makes having member variables of them harder. To some extent, you can get around it with typeof(a) - which is often what we have to do with ranges anyway, but that's pretty ridiculous when you typed out the exact type to begin with and code-wise, there's no reason why they shouldn't be identical. There are likely other uses cases where this is a problem, but those are the two that we definitely run into with current Phobos code. And while the problem may make sense to a compiler writer, it's not what much of anyone else is going to expect or want to deal with. - Jonathan M Davis
Apr 28 2016
prev sibling parent reply Mint <mintie wute.com> writes:
On Tuesday, 26 April 2016 at 19:45:18 UTC, Jack Stouffer wrote:
 I'm of the opinion that string lambdas must go. I started, and 
 I really should finish it at some point, removing string 
 lambdas from the documentation: 
 https://github.com/dlang/phobos/pull/3800

 I think that the drawback you mentioned does not outweigh the 
 benefits gained from using actual lambdas.
I'm personally somewhat fond of string lambdas for their usefulness in making some operations very concise, without sacrificing any readability. ie. foo.map!"a.bar".reduce!"a * b"; vs. foo.map!(a => a.bar).reduce!((a, b) => a * b); I'm open to an alternative that is equally short and sweet, but replacing them with proper lambda declarations feels like uncalled for verbosity to me.
Apr 30 2016
parent reply Seb <seb wilzba.ch> writes:
On Saturday, 30 April 2016 at 15:10:08 UTC, Mint wrote:
 On Tuesday, 26 April 2016 at 19:45:18 UTC, Jack Stouffer wrote:
 I'm of the opinion that string lambdas must go. I started, and 
 I really should finish it at some point, removing string 
 lambdas from the documentation: 
 https://github.com/dlang/phobos/pull/3800

 I think that the drawback you mentioned does not outweigh the 
 benefits gained from using actual lambdas.
I'm personally somewhat fond of string lambdas for their usefulness in making some operations very concise, without sacrificing any readability. ie. foo.map!"a.bar".reduce!"a * b"; vs. foo.map!(a => a.bar).reduce!((a, b) => a * b); I'm open to an alternative that is equally short and sweet, but replacing them with proper lambda declarations feels like uncalled for verbosity to me.
I really like them too and was impressed that Andrei is so against them, especially if you have more arguments writing the lambda get's more tedious. The PR was created to support `each` string lambdas with more than two functions, e.g.: arr.each!`a + 2*b + 4*c + 8*d` arr.each!((a, b, c, d) = > a + 2*b + 4*c + 8*d) While I agree that it might be a bit hard to read, they're beautifully concise! Also they are quite popular - even in Phobos code.
 grep -r unaryFun * | wc -l
142
 grep -r binaryFun * | wc -l
250
 grep -r '!".*"' * --exclude datetime.d | wc -l
591 (disclaimer: some false positives) Btw at mir we have the convention to use backticks, so it's easier to distinguish them from normal strings - might be a convention that you could adapt or even enforce?
May 01 2016
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 01 May 2016 15:03:41 +0000
Seb via Digitalmars-d <digitalmars-d puremagic.com> wrote:
 I really like them too and was impressed that Andrei is so
 against them, especially if you have more arguments writing the
 lambda get's more tedious.
String lambdas were an innovative solution to make lambdas shorter than having to provide full-on anonymous functions with brackets and the whole bit. However, they quickly proved to have problems as soon as you tried to do much of anything beyond arithmetic and comparisons, because you could only use functions in them that were in modules imported by std.functional. So, at some point there was a discussion or two or introducing a shorter lambda syntax, and we ended up adding the syntax which is currently used which (as I understand it) is the same as what C# uses, and that works in a much broader context than string lambdas do. If we'd had it initially, we probably would never have had string lambdas. Really, the only three reasons that we still have string lambdas at this point is to avoid breaking code, because you can't compare lambdas for equality (whereas you can compare strings), and because the same template instantiated with the same lambda in multiple places results in different instantiations, so they're incompatible. Now, string lambdas are still great for basic stuff like "a != b" and "a + b" and the like, because even the new lambda syntax is longer than that. But having the string lambdas on top of the other is adding more complication to the language/library, and it's going to continue to cause problems when folks try and use the string lambdas to call functions that std.functional doesn't have access to - and it wouldn't surprise me if the recent import fixes actually make it so that std.functional has access to even fewer functions. So, on some level it _will_ suck to lose string lambdas even once we've fixed the issues with non-string lambdas, but they're redundant and will be potentially confusing to keep around. So, most folks seem to have concluded that they should be removed (some even seem to think that they should be removed even without the current issues with non-string lambdas being fixed, but that's unreasonable IMHO). - Jonathan M Davis
May 01 2016
parent ag0aep6g <anonymous example.com> writes:
On 02.05.2016 02:17, Jonathan M Davis via Digitalmars-d wrote:
 So, at some point there was a discussion or two or introducing a shorter
 lambda syntax, and we ended up adding the syntax which is currently used
 which (as I understand it) is the same as what C# uses,
There is a difference. `x => {foo(x);}` has a different meaning in D than in C#. C#'s meaning is expressed with `(x) {foo(x);}` in D, and D's meaning is `x => () => {foo(x);}` in C#.
May 01 2016
prev sibling next sibling parent QAston <qaston gmail.com> writes:
On Tuesday, 26 April 2016 at 17:58:22 UTC, Andrei Alexandrescu 
wrote:
 One thing we really need in order to 100% replace string 
 lambdas with lambdas is function equivalence. Right now we're 
 in the odd situation that SomeTemplate!((a, b) => a < b) has 
 distinct types, one per instantiation.
Well, SomeTemplate!((a, b) => a < b) uses pass by name, so it's sort of expected to be a distinct type for each different object you pass to it. Do you want to bind different lambdas (matching by some notion of equality) to the same symbol? That'd be an enormous hack. Now, if this was: SomeTemplate(FunctorType) // type parameter, not alias here { this(FunctorType f){ ... } ... } and (a, b)=> a < b returned an object of the same type for equivalent functions you could have fn equality without giving different objects the same symbol. It's also inlineable as pass by name is.
Apr 27 2016
prev sibling next sibling parent deadalnix <deadalnix gmail.com> writes:
On Tuesday, 26 April 2016 at 17:58:22 UTC, Andrei Alexandrescu 
wrote:
 https://github.com/dlang/phobos/pull/3882

 I just closed with some regret a nice piece of engineering. 
 Please comment if you think string lambdas have a lot of 
 unexploited potential.

 One thing we really need in order to 100% replace string 
 lambdas with lambdas is function equivalence. Right now we're 
 in the odd situation that SomeTemplate!((a, b) => a < b) has 
 distinct types, one per instantiation.


 Andrei
This can be worked around by defining a less function and using it here. This, however, runs into some subtleties because functions and functions aren't the same thing in D (Confusing ? You bet it is !).
Apr 27 2016
prev sibling next sibling parent reply Bauss <streetzproductionz hotmail.com> writes:
On Tuesday, 26 April 2016 at 17:58:22 UTC, Andrei Alexandrescu 
wrote:
 https://github.com/dlang/phobos/pull/3882

 I just closed with some regret a nice piece of engineering. 
 Please comment if you think string lambdas have a lot of 
 unexploited potential.

 One thing we really need in order to 100% replace string 
 lambdas with lambdas is function equivalence. Right now we're 
 in the odd situation that SomeTemplate!((a, b) => a < b) has 
 distinct types, one per instantiation.


 Andrei
An alternative to string lambads could be something like a shortened version of lambdas like consider "a > b" could be something like {%0 > %1} and then the body of the expression is evaluated to see whether it's a valid D expression where %1 and %2 would equal first and second arguments in the expression and when used you could simply use do something like Compare(T,expression)(T x, T y) { return expression(x,y); // the compiler has to figure out that this should be evaluated as return x < y with the example above. This will ensure that the expression is a valid D expression and cannot really hacked to do other stuff what it's meant to. Of course this is just an idea on top of my head on how string lambdas could be resolved.
Apr 30 2016
parent Bauss <streetzproductionz hotmail.com> writes:
On Saturday, 30 April 2016 at 16:02:02 UTC, Bauss wrote:
 On Tuesday, 26 April 2016 at 17:58:22 UTC, Andrei Alexandrescu 
 wrote:
 [...]
An alternative to string lambads could be something like a shortened version of lambdas like consider [...]
%1 and %2 should be %0 and %1 and also the < should be > in the return statement of course.
Apr 30 2016
prev sibling parent reply poliklosio <poliklosio happypizza.com> writes:
On Tuesday, 26 April 2016 at 17:58:22 UTC, Andrei Alexandrescu 
wrote:
 https://github.com/dlang/phobos/pull/3882

 I just closed with some regret a nice piece of engineering. 
 Please comment if you think string lambdas have a lot of 
 unexploited potential.

 One thing we really need in order to 100% replace string 
 lambdas with lambdas is function equivalence. Right now we're 
 in the odd situation that SomeTemplate!((a, b) => a < b) has 
 distinct types, one per instantiation.


 Andrei
I will just point out an obvious solution to the problem of having distinct types: if a normal lambda is sufficiently simple, and is used as a template parameter, lower the code to the one which uses an equivalent string lambda. The string needs to somewhat normalized so that (a, b) => a < b and (c,d) => c<d are converted to the same string. Whether the compiler allows explicit string lambdas in user code in another issue.
May 01 2016
parent poliklosio <poliklosio happypizza.com> writes:
 Whether the compiler allows explicit string lambdas in user 
 code in another issue.
s/in another issue/is another issue/
May 01 2016