www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - "hijackable"/"customizable" keyword for solving the "customized

reply "monarch_dodra" <monarchdodra gmail.com> writes:
A little while ago, Steven Schveighoffer started a thread called 
"D UFCS anti-pattern". The conversation started in regards to 
"how can a generic algorithm be replaced by a custom type". In 
particular (in my original use case) "retro"

The issue just came up again in ".learn" for ".find". It comes up 
regularly with ".put".

To put it simply, the problem is "given the function std.foo, how 
can I implement myObject.foo, and make sure that that is called 
when we call foo?"

There were two options:
1. Make std.foo test "hasMember!(T, "foo")". This would hardly 
scale though, as applying it to *every* algorithm is not 
acceptable
2. For *all* calls to be UFCS style, and ignore the "anti-pattern 
issue". Even then, it means you give no guarantees as to *what* 
is called, depending on how the user writes the code. So not 
acceptable

I just had a crazy idea. "hijackable" keyword (yeah... another 
keyword):

Given a function:
"Ret foo(T input, Args... args)  hijackable"

Then, when the compiler sees:
"foo(input, args);"

It will always forward directly to T.foo if T.foo exists, 
bypassing std.foo entirely.

I think it is a clean and generic way to allow ALL algorithms in 
D much greater customize-ability.

I didn't put *much* more thought than that to it yet, so I may 
have missed something obvious? In particular, "hijack" might not 
be the correct word? " customizable" ?

Please provide feedback/destruction ?
May 14 2014
next sibling parent "w0rp" <devw0rp gmail.com> writes:
I'm going to get into an opinion on member functions and 
overrides here, and it's kind of "anti-classes," so you can 
choose to ignore this post if you don't want to hear about that.

I think this is a bad idea. One of the things that really appeals 
to me about algorithms presented through functions is that I can 
predict the behaviour of them. It will do something generically 
on a category of types. I think any need to change the behaviour 
of an algorithm either means that the algorithm was written 
incorrectly, or really you just have a different algorithm and 
you should probably use another name for it. One of the things 
I've really lost faith in over the years is method overrides, 
becuase of how confusing it has made code I have had to read. 
These days I prefer generic functions that give me one or maybe a 
couple of places to look when something goes wrong.
May 14 2014
prev sibling next sibling parent "Meta" <jared771 gmail.com> writes:
On Wednesday, 14 May 2014 at 18:05:44 UTC, monarch_dodra wrote:
 A little while ago, Steven Schveighoffer started a thread 
 called "D UFCS anti-pattern". The conversation started in 
 regards to "how can a generic algorithm be replaced by a custom 
 type". In particular (in my original use case) "retro"

 The issue just came up again in ".learn" for ".find". It comes 
 up regularly with ".put".

 To put it simply, the problem is "given the function std.foo, 
 how can I implement myObject.foo, and make sure that that is 
 called when we call foo?"

 There were two options:
 1. Make std.foo test "hasMember!(T, "foo")". This would hardly 
 scale though, as applying it to *every* algorithm is not 
 acceptable
 2. For *all* calls to be UFCS style, and ignore the 
 "anti-pattern issue". Even then, it means you give no 
 guarantees as to *what* is called, depending on how the user 
 writes the code. So not acceptable

 I just had a crazy idea. "hijackable" keyword (yeah... another 
 keyword):

 Given a function:
 "Ret foo(T input, Args... args)  hijackable"

 Then, when the compiler sees:
 "foo(input, args);"

 It will always forward directly to T.foo if T.foo exists, 
 bypassing std.foo entirely.

 I think it is a clean and generic way to allow ALL algorithms 
 in D much greater customize-ability.

 I didn't put *much* more thought than that to it yet, so I may 
 have missed something obvious? In particular, "hijack" might 
 not be the correct word? " customizable" ?

 Please provide feedback/destruction ?

It's a good idea, but yet another function annotation is pretty much a no-go. How bad an idea is it to *always* defer to a member function if the object/struct in question has such a function defined? I thought that was the case already... I suppose it will cause untold amounts of code breakage. Very unfortunate.
May 14 2014
prev sibling next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Wednesday, 14 May 2014 at 23:57:24 UTC, Meta wrote:
 It's a good idea, but yet another function annotation is pretty 
 much a no-go. How bad an idea is it to *always* defer to a 
 member function if the object/struct in question has such a 
 function defined? I thought that was the case already... I 
 suppose it will cause untold amounts of code breakage. Very 
 unfortunate.

UFCS only apply to the method call style, not the the function call style, so it's not a matter of priority here - `foo(myObject)` will not call the `foo.myObject()` method even if there is no `foo` function(in that case it'll just fail). Having the function call style always defer to a member function is a really bad idea - and not just because of code breakage. It'll make it impossible to call a function on an object that has a method of the same name unless you use an alias or a function variable to copy the function - but then you have to make sure the alias\variable name is also not taken! This will be a disaster for anyone who wants to build or use a D library that uses templates. Well, how common can that case be?
May 14 2014
prev sibling next sibling parent "Meta" <jared771 gmail.com> writes:
On Thursday, 15 May 2014 at 01:49:17 UTC, Idan Arye wrote:
 UFCS only apply to the method call style, not the the function 
 call style, so it's not a matter of priority here - 
 `foo(myObject)` will not call the `foo.myObject()` method even 
 if there is no `foo` function(in that case it'll just fail).

 Having the function call style always defer to a member 
 function is a really bad idea - and not just because of code 
 breakage. It'll make it impossible to call a function on an 
 object that has a method of the same name unless you use an 
 alias or a function variable to copy the function - but then 
 you have to make sure the alias\variable name is also not taken!

 This will be a disaster for anyone who wants to build or use a 
 D library that uses templates. Well, how common can that case 
 be?

That's exactly what the proposed hijackable annotation on a function would do, if I understand it correctly.
May 14 2014
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 15 May 2014 at 02:07:40 UTC, Meta wrote:
 On Thursday, 15 May 2014 at 01:49:17 UTC, Idan Arye wrote:
 UFCS only apply to the method call style, not the the function 
 call style, so it's not a matter of priority here - 
 `foo(myObject)` will not call the `foo.myObject()` method even 
 if there is no `foo` function(in that case it'll just fail).

 Having the function call style always defer to a member 
 function is a really bad idea - and not just because of code 
 breakage. It'll make it impossible to call a function on an 
 object that has a method of the same name unless you use an 
 alias or a function variable to copy the function - but then 
 you have to make sure the alias\variable name is also not 
 taken!

 This will be a disaster for anyone who wants to build or use a 
 D library that uses templates. Well, how common can that case 
 be?

That's exactly what the proposed hijackable annotation on a function would do, if I understand it correctly.

Right. The idea though is not to "customize the behavior" of the algorithm, but rather "provide a more efficient implementation". It shouldn't really break any existing code. In any case, it should not cause any more than the breakage that using UFCS *already* causes... this would just (opt-in by the implementation) make sure you get the same behavior with/without UFCS. FYI, there are already a few functions in phobos that delegate to member functions "if they exist", and do something generic if they don't. "Take" and "put" are some of the more trivial examples. "move" will also delegate to "proxyMove". At one point, there were also talks of doing the same for popFrontN too. The issue though is that it's all very hackish, and it makes the implementation jump through hoops to achieve said result, and it only works if the implementation pre-emptivelly thought about doing so.
May 14 2014
prev sibling next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 5/14/14, 3:05 PM, monarch_dodra wrote:
 A little while ago, Steven Schveighoffer started a thread called "D UFCS
 anti-pattern". The conversation started in regards to "how can a generic
 algorithm be replaced by a custom type". In particular (in my original
 use case) "retro"

 The issue just came up again in ".learn" for ".find". It comes up
 regularly with ".put".

 To put it simply, the problem is "given the function std.foo, how can I
 implement myObject.foo, and make sure that that is called when we call
 foo?"

That's the big problem with UFCS. If you were forced to always use UFCS then this wouldn't be a problem: if the type at question provides a more specific implementation *that matches the arguments' types*, the compiler takes it. Otherwise, it checks for free functions that have that type as a first argument and the rest of the arguments. But once you have an option to invoke it as a free function you loose. The "U" of UFCS means "uniform", so in my opinion when you do: foo(x, y); then the compiler must first check if the type of "x" defines "foo(y)". If so, then it invokes that specialized version. If not, it must check if "foo(x, y)" exists somewhere else. If you do: x.foo(y); the compiler would do exactly the same. This is uniformity: either way I invoke a function, the compiler's logic is the same. But if you can write a "same thing" in two different ways but behave differently, that's looking for trouble. If you don't want the compiler to convert foo(x, y) to x.foo(y), then use a qualified name: bar.baz.foo(x, y);
May 15 2014
parent Ary Borenszweig <ary esperanto.org.ar> writes:
On 5/15/14, 7:54 AM, Ary Borenszweig wrote:
 On 5/14/14, 3:05 PM, monarch_dodra wrote:
 A little while ago, Steven Schveighoffer started a thread called "D UFCS
 anti-pattern". The conversation started in regards to "how can a generic
 algorithm be replaced by a custom type". In particular (in my original
 use case) "retro"

 The issue just came up again in ".learn" for ".find". It comes up
 regularly with ".put".

 To put it simply, the problem is "given the function std.foo, how can I
 implement myObject.foo, and make sure that that is called when we call
 foo?"

That's the big problem with UFCS. If you were forced to always use UFCS then this wouldn't be a problem: if the type at question provides a more specific implementation *that matches the arguments' types*, the compiler takes it. Otherwise, it checks for free functions that have that type as a first argument and the rest of the arguments. But once you have an option to invoke it as a free function you loose. The "U" of UFCS means "uniform", so in my opinion when you do: foo(x, y); then the compiler must first check if the type of "x" defines "foo(y)". If so, then it invokes that specialized version. If not, it must check if "foo(x, y)" exists somewhere else. If you do: x.foo(y); the compiler would do exactly the same. This is uniformity: either way I invoke a function, the compiler's logic is the same. But if you can write a "same thing" in two different ways but behave differently, that's looking for trouble. If you don't want the compiler to convert foo(x, y) to x.foo(y), then use a qualified name: bar.baz.foo(x, y);

Of course, the "problem" with this is that when you read this code: foo(x, y); to understand it, you must first check if there's a "foo" definition somewhere in your imported modules that matches. Otherwise, check if x's type defines it. The same is true for this: x.foo(y); To understand it, you must check if x's type defines a "foo(y)". If so, it's taken. Otherwise, check which of the functions in the imported modules matches "foo(x, y)". This makes it really hard to reason about code: everytime you see "foo(x, y)" you must remember which of the modules had that function "foo", or if x defines "foo". In OOP-like land, "foo(x, y)" always means: check the function "foo(x, y)". And "x.foo(y)" always means: check the member "foo" of x's type. No confusion at all for the reader. But "foo(x, y)" is almost never used (global functions), so you never have to remember where methods are located by heart. You just go to that type's code and find the method there.
May 15 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 15 May 2014 02:05:08 -0400, monarch_dodra <monarchdodra gmail.com>  
wrote:

 "move" will also delegate to "proxyMove".

This is the correct solution IMO. The principle of least surprise should dictate that a.foo should always mean the same thing. All that is required to enforce this is to make the "hook" function have a different name. I think the two mechanisms of overriding default behavior: 1. Use a hook, to define the basic minimum functionality. 2. Implement the same-named method, but you must implement ALL functionality (possibly deferring to the global function if necessary). -Steve
May 15 2014
prev sibling next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Thursday, 15 May 2014 at 02:07:40 UTC, Meta wrote:
 On Thursday, 15 May 2014 at 01:49:17 UTC, Idan Arye wrote:
 UFCS only apply to the method call style, not the the function 
 call style, so it's not a matter of priority here - 
 `foo(myObject)` will not call the `foo.myObject()` method even 
 if there is no `foo` function(in that case it'll just fail).

 Having the function call style always defer to a member 
 function is a really bad idea - and not just because of code 
 breakage. It'll make it impossible to call a function on an 
 object that has a method of the same name unless you use an 
 alias or a function variable to copy the function - but then 
 you have to make sure the alias\variable name is also not 
 taken!

 This will be a disaster for anyone who wants to build or use a 
 D library that uses templates. Well, how common can that case 
 be?

That's exactly what the proposed hijackable annotation on a function would do, if I understand it correctly.

Yes, but there is quite a difference between applying this to specific functions where this behavior is desirable and applying this to everything.
May 15 2014
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 15 May 2014 at 12:16:52 UTC, Steven Schveighoffer 
wrote:
 On Thu, 15 May 2014 02:05:08 -0400, monarch_dodra 
 <monarchdodra gmail.com> wrote:

 "move" will also delegate to "proxyMove".

This is the correct solution IMO. The principle of least surprise should dictate that a.foo should always mean the same thing. All that is required to enforce this is to make the "hook" function have a different name.

The issue(s) I have with the "hook must have a different name" is: 1. In the algorithm implementation, you must explicitly test for the hook, which, if we want to expand on, would turn all our algorithms into: void foo(T)(T t) { static if (is(typeof(t.fooHook()))) return t.fooHook(); else ... } 2. The "overriden" hook is only useful if you import the corresponding algorithm. So for example, in my 3rd pary library, if my type has "findHook", I'd *have* to import std.algorithm.find for it to be useful. Unless I want to make a direct call to "findHook", which would be ugly... An example for "2" are range primitives:
 I think the two mechanisms of overriding default behavior:

 1. Use a hook, to define the basic minimum functionality.
 2. Implement the same-named method, but you must implement ALL 
 functionality (possibly deferring to the global function if 
 necessary).

 -Steve

Also: "moveFront"/"moveBack" are other examples of such "globally hijack-able" functions.
May 15 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 15 May 2014 13:08:56 -0400, monarch_dodra <monarchdodra gmail.com>  
wrote:

 On Thursday, 15 May 2014 at 12:16:52 UTC, Steven Schveighoffer wrote:
 On Thu, 15 May 2014 02:05:08 -0400, monarch_dodra  
 <monarchdodra gmail.com> wrote:

 "move" will also delegate to "proxyMove".

This is the correct solution IMO. The principle of least surprise should dictate that a.foo should always mean the same thing. All that is required to enforce this is to make the "hook" function have a different name.

The issue(s) I have with the "hook must have a different name" is: 1. In the algorithm implementation, you must explicitly test for the hook, which, if we want to expand on, would turn all our algorithms into: void foo(T)(T t) { static if (is(typeof(t.fooHook()))) return t.fooHook(); else ... } 2. The "overriden" hook is only useful if you import the corresponding algorithm. So for example, in my 3rd pary library, if my type has "findHook", I'd *have* to import std.algorithm.find for it to be useful. Unless I want to make a direct call to "findHook", which would be ugly...

Obviously, foo is not a good example for hooking. This problem only exists if the functionality of the module-level function exceeds that of the member. The above does not have any overloads, and basically does one thing. I did consider the fact that you would have to import the module-level function in order for it to be part of the API, but that's how the system works. It would be nice to say "if you import module x, then a.foo has more functionality, if you don't, it's limited to this one aspect," but I don't think that's possible. It's not a perfect solution, but the only one that makes sense to me. -Steve
May 15 2014
prev sibling next sibling parent "Yota" <yotaxp thatGoogleMailThing.com> writes:
On Thursday, 15 May 2014 at 17:08:58 UTC, monarch_dodra wrote:
 On Thursday, 15 May 2014 at 12:16:52 UTC, Steven Schveighoffer 
 wrote:
 On Thu, 15 May 2014 02:05:08 -0400, monarch_dodra 
 <monarchdodra gmail.com> wrote:

 "move" will also delegate to "proxyMove".

This is the correct solution IMO. The principle of least surprise should dictate that a.foo should always mean the same thing. All that is required to enforce this is to make the "hook" function have a different name.

The issue(s) I have with the "hook must have a different name" is: 1. In the algorithm implementation, you must explicitly test for the hook, which, if we want to expand on, would turn all our algorithms into: void foo(T)(T t) { static if (is(typeof(t.fooHook()))) return t.fooHook(); else ... } 2. The "overriden" hook is only useful if you import the corresponding algorithm. So for example, in my 3rd pary library, if my type has "findHook", I'd *have* to import std.algorithm.find for it to be useful. Unless I want to make a direct call to "findHook", which would be ugly...

How about a middle ground? Have the function names be identical, and decorate the member version with proxy or hook, rather than decorating the original definition. I'd find this to be less surprising.
May 16 2014
prev sibling next sibling parent "Yota" <yotaxp thatGoogleMailThing.com> writes:
On Wednesday, 14 May 2014 at 18:05:44 UTC, monarch_dodra wrote:
...
 I just had a crazy idea. "hijackable" keyword (yeah... another 
 keyword):

 Given a function:
 "Ret foo(T input, Args... args)  hijackable"

 Then, when the compiler sees:
 "foo(input, args);"

 It will always forward directly to T.foo if T.foo exists, 
 bypassing std.foo entirely.

What about stuff like this: import some.pkg; some.pkg.foo(x); Does this still call x.foo()? What if 'foo' is an alias for 'bar', or vice versa?
May 16 2014
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 16 May 2014 at 16:50:37 UTC, Yota wrote:
 On Wednesday, 14 May 2014 at 18:05:44 UTC, monarch_dodra wrote:
 ...
 I just had a crazy idea. "hijackable" keyword (yeah... another 
 keyword):

 Given a function:
 "Ret foo(T input, Args... args)  hijackable"

 Then, when the compiler sees:
 "foo(input, args);"

 It will always forward directly to T.foo if T.foo exists, 
 bypassing std.foo entirely.

What about stuff like this: import some.pkg; some.pkg.foo(x); Does this still call x.foo()?

In theory, yes, because "some.pkg" would *be* X.foo, or at the very least, a very thin wrapper. You'd get the very same result with "static if" based implementation, such as "moveFront", where "std.range.moveFront(r)" still just calls "r.moveFront()". Ditto for "take", which just calls opSlice.
 What if 'foo' is an alias for 'bar', or vice versa?

An alias shouldn't change anything I think.
May 16 2014
prev sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Fri, 16 May 2014 16:45:28 +0000
Yota via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On Thursday, 15 May 2014 at 17:08:58 UTC, monarch_dodra wrote:
 On Thursday, 15 May 2014 at 12:16:52 UTC, Steven Schveighoffer
 wrote:
 On Thu, 15 May 2014 02:05:08 -0400, monarch_dodra
 <monarchdodra gmail.com> wrote:

 "move" will also delegate to "proxyMove".

This is the correct solution IMO. The principle of least surprise should dictate that a.foo should always mean the same thing. All that is required to enforce this is to make the "hook" function have a different name.

The issue(s) I have with the "hook must have a different name" is: 1. In the algorithm implementation, you must explicitly test for the hook, which, if we want to expand on, would turn all our algorithms into: void foo(T)(T t) { static if (is(typeof(t.fooHook()))) return t.fooHook(); else ... } 2. The "overriden" hook is only useful if you import the corresponding algorithm. So for example, in my 3rd pary library, if my type has "findHook", I'd *have* to import std.algorithm.find for it to be useful. Unless I want to make a direct call to "findHook", which would be ugly...

How about a middle ground? Have the function names be identical, and decorate the member version with proxy or hook, rather than decorating the original definition. I'd find this to be less surprising.

That sounds like an interesting idea, but it would require adding the concept to the compiler - though I suppose that that's a downside with monarch_dodra's original proposal as well. Neither can be done in the library itself. That's not necessarily the end of the world, but at this point, I'm very hesitant to support adding another attribute to the language without a really, really good reason. This particular problem can be solved by Steven's suggestion of using proxy functions with different names, which works within the language as it's currently defined. So, while that might not be an altogether desirable solution, I'm inclined to believe that it's good enough to make it not worth adding additional attributes to the language to solve the problem. - Jonathan M Davis
May 17 2014