www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Annoying thing about auto ref function template

reply Yuxuan Shui <yshuiv7 gmail.com> writes:
An auto ref function template should behave like a normal 
function template, but it doesn't.

You can fully instantiate a function template by specifying all 
of its template parameters, but you can't do that with auto ref 
templates. The only way to instantiate an auto ref template is to 
call it.

This makes auto ref an outlier. Because you get a function by 
instantiate function template, you can pass the result as 
template alias argument, and you can create alias of the 
resulting function. And you can't do that with an auto ref 
template, which makes them quite annoying.

I wonder why is auto ref designed this way? And can we change 
this?
Mar 20
next sibling parent Yuxuan Shui <yshuiv7 gmail.com> writes:
On Monday, 20 March 2017 at 19:49:03 UTC, Yuxuan Shui wrote:
 And you can't do that with an auto ref template, which makes 
 them quite annoying.
BTW, the error message you get when you try to do this, is not very helpful: 'auto' can only be used as part of 'auto ref' for template function parameters
Mar 20
prev sibling next sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, March 20, 2017 19:49:03 Yuxuan Shui via Digitalmars-d wrote:
 An auto ref function template should behave like a normal
 function template, but it doesn't.

 You can fully instantiate a function template by specifying all
 of its template parameters, but you can't do that with auto ref
 templates. The only way to instantiate an auto ref template is to
 call it.

 This makes auto ref an outlier. Because you get a function by
 instantiate function template, you can pass the result as
 template alias argument, and you can create alias of the
 resulting function. And you can't do that with an auto ref
 template, which makes them quite annoying.

 I wonder why is auto ref designed this way? And can we change
 this?
Well, you can explicitly instantiate it and call it at the same time, it does work. When it doesn't work is when you just try and instantiate it without calling it. And the reason is because the refness is inferred from the function argument. If the argument is an lvalue, then the parameter is inferred as ref, whereas if it's an rvalue, it's inferred as non-ref. The refness is not actually part of the type, because ref is not a type qualifier. As such, AFAIK, the language simply has no way to pass the refness as part of the explicit instantiation. In something like auto foo(T)(auto ref T t) { ... } the T in the template parameters has nothing to do with the ref, and in fact, you could have something like auto foo(T)(auto ref T t1, auto ref T t2) { ... } and end up with t1 being ref and t2 being non-ref (or vice versa or both ref or neithe ref). So, the refness would need to somehow be indicated separately from the template argument, and I have no idea what that would even look like. So, yes, this particular restriction can be annoying, but there is a good reason for the restriction (though the error message _is_ pretty bad), and I have no idea how we would fix the problem. - Jonathan M Davis
Mar 20
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, March 20, 2017 13:20:52 Jonathan M Davis via Digitalmars-d wrote:
 So, yes, this particular restriction can be annoying, but there is a good
 reason for the restriction (though the error message _is_ pretty bad), and
 I have no idea how we would fix the problem.
After thinking about this further, it does occur to me that there's a fairly simple workaround. e.g. auto foo(T)(T t) { return bar(t); } auto bar(T)(auto ref T t) { ... } Then you can just use foo!int. Now, that requires you to pick whether it's ref or not, but if there were a way to explicitly instantiate a function with an auto ref parameter, you'd have to do that anyway. - Jonathan M Davis
Mar 20
parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Monday, 20 March 2017 at 21:08:40 UTC, Jonathan M Davis wrote:
 On Monday, March 20, 2017 13:20:52 Jonathan M Davis via 
 Digitalmars-d wrote:
 So, yes, this particular restriction can be annoying, but 
 there is a good reason for the restriction (though the error 
 message _is_ pretty bad), and I have no idea how we would fix 
 the problem.
After thinking about this further, it does occur to me that there's a fairly simple workaround. e.g. auto foo(T)(T t) { return bar(t); } auto bar(T)(auto ref T t) { ... } Then you can just use foo!int. Now, that requires you to pick whether it's ref or not, but if there were a way to explicitly instantiate a function with an auto ref parameter, you'd have to do that anyway. - Jonathan M Davis
This is a bit tedious because it requires you creating a new function. Maybe we can create a template for that. But still, auto ref requires us to do things differently, which is annoying.
Mar 20
parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Monday, 20 March 2017 at 21:34:14 UTC, Yuxuan Shui wrote:
 On Monday, 20 March 2017 at 21:08:40 UTC, Jonathan M Davis 
 wrote:
 [...]
This is a bit tedious because it requires you creating a new function. Maybe we can create a template for that. But still, auto ref requires us to do things differently, which is annoying.
Easy solution: just support auto ref for non-templates. I think someone has already did work on that?
Mar 20
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, March 20, 2017 21:37:26 Yuxuan Shui via Digitalmars-d wrote:
 On Monday, 20 March 2017 at 21:34:14 UTC, Yuxuan Shui wrote:
 On Monday, 20 March 2017 at 21:08:40 UTC, Jonathan M Davis

 wrote:
 [...]
This is a bit tedious because it requires you creating a new function. Maybe we can create a template for that. But still, auto ref requires us to do things differently, which is annoying.
Easy solution: just support auto ref for non-templates. I think someone has already did work on that?
auto ref for non-templates would not be quite the same thing, and regardless, it wouldn't help any with explictly instantiating a template that had an auto ref parameter. So, it really wouldn't solve the problem at all. It would just make it so that if you didn't want a templated function, you could use auto ref. But for auto ref to work with non-templated functions, it would either result in creating a combinatorial explosion of functions rather than just a single function (basically doing what auto ref does but explicitly instantiating it with every combination of refness instead of just the ones that are actually used), or it would just create a version where every auto ref parameter was ref and silently copy rvalues to the stack to pass by ref when they're passed to the function. I think that there was a PR on the issue at one point, and IIRC, the ultimate decision was to reject it and keep the status quo, because there were too many downsides it (e.g. IIRC, there were problems with overriding functions in derived classes). I think that there was some talk semi-recently about creating a proposal for something similar to C++'s const & but which was more restrictive in order to avoid the downsides with C++'s solution, in which case we might end up with something like rvalue ref which accepted rvalues - probably by copying them to the stack first. But I don't think that anything has been formally proposed yet. If that goes anywhere, it will probably be the closest that you'll ever get to having auto ref with non-templated functions. - Jonathan M Davis
Mar 20
parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Monday, 20 March 2017 at 21:53:47 UTC, Jonathan M Davis wrote:
 On Monday, March 20, 2017 21:37:26 Yuxuan Shui via 
 Digitalmars-d wrote:
 [...]
auto ref for non-templates would not be quite the same thing, and regardless, it wouldn't help any with explictly instantiating a template that had an auto ref parameter. So, it really wouldn't solve the problem at all. It would just make it so that if you didn't want a templated function, you could use auto ref. [...]
Makes sense... OK, attempt 2: how about support implicit partial application for templates?
 [...]
Mar 20
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, March 20, 2017 22:14:37 Yuxuan Shui via Digitalmars-d wrote:
 On Monday, 20 March 2017 at 21:53:47 UTC, Jonathan M Davis wrote:
 On Monday, March 20, 2017 21:37:26 Yuxuan Shui via

 Digitalmars-d wrote:
 [...]
auto ref for non-templates would not be quite the same thing, and regardless, it wouldn't help any with explictly instantiating a template that had an auto ref parameter. So, it really wouldn't solve the problem at all. It would just make it so that if you didn't want a templated function, you could use auto ref. [...]
Makes sense... OK, attempt 2: how about support implicit partial application for templates?
Well, you can do something like template foo(T) { auto foo()(auto ref T t) { return t; } } void main() { alias f = foo!int; auto a = f(42); } and then foo!int, but you're not going to get a function pointer out of it, since for that, you need to instantiate the inner function template. It does give partial instantiation though. - Jonathan M Davis
Mar 20
parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Tuesday, 21 March 2017 at 01:10:38 UTC, Jonathan M Davis wrote:
 On Monday, March 20, 2017 22:14:37 Yuxuan Shui via 
 Digitalmars-d wrote:
 On Monday, 20 March 2017 at 21:53:47 UTC, Jonathan M Davis 
 wrote:
 [...]
Makes sense... OK, attempt 2: how about support implicit partial application for templates?
Well, you can do something like template foo(T) { auto foo()(auto ref T t) { return t; } } void main() { alias f = foo!int; auto a = f(42); } and then foo!int, but you're not going to get a function pointer out of it, since for that, you need to instantiate the inner function template. It does give partial instantiation though. - Jonathan M Davis
This is looks doable. Can we do this for all auto ref functions in phobos?
Mar 21
parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tuesday, March 21, 2017 09:27:55 Yuxuan Shui via Digitalmars-d wrote:
 On Tuesday, 21 March 2017 at 01:10:38 UTC, Jonathan M Davis wrote:
 On Monday, March 20, 2017 22:14:37 Yuxuan Shui via

 Digitalmars-d wrote:
 On Monday, 20 March 2017 at 21:53:47 UTC, Jonathan M Davis

 wrote:
 [...]
Makes sense... OK, attempt 2: how about support implicit partial application for templates?
Well, you can do something like template foo(T) { auto foo()(auto ref T t) { return t; } } void main() { alias f = foo!int; auto a = f(42); } and then foo!int, but you're not going to get a function pointer out of it, since for that, you need to instantiate the inner function template. It does give partial instantiation though.
This is looks doable. Can we do this for all auto ref functions in phobos?
Maybe, but I'm not sure that it's merited. On top of the fact that this would make every auto ref function more verbose, remember that doing this would add an extra template to every single auto ref function, which increases their compilation overhead (as it is, we arguably instantiate _way_ too many templates in many D programs). AFAIK, you're the first person to ever bring this issue up, which implies that this is not something that many folks are trying to do - and it _is_ fairly trivial to create wrappers in the cases where you want to have an explicit instantiation. So, it might make sense to do this, or it might just be unnecessary overhead for 99.99% of cases while only mildly helping the .01% case. Certainly, it seems pretty ugly to start saying that folks should be wrapping all auto ref functions in an extra template. I'm inclined to think that we need a strong use case to merit adding the extra overhead to all of these functions rather than having the few who need something like this create wrapper functions. So, what's your use case here? Why would you be looking to explicitly instantiate templates (auto ref or otherwise) frequently enough that it's a problem to just create a wrapper in the cases where you need to do it? Most code does not explicitly instanitate templates, and there's usually no point in doing so. - Jonathan M Davis
Mar 22