www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Annoying thing about auto ref function template

reply Yuxuan Shui <yshuiv7 gmail.com> writes:
An auto ref function template should behave like a normal 
function template, but it doesn't.

You can fully instantiate a function template by specifying all 
of its template parameters, but you can't do that with auto ref 
templates. The only way to instantiate an auto ref template is to 
call it.

This makes auto ref an outlier. Because you get a function by 
instantiate function template, you can pass the result as 
template alias argument, and you can create alias of the 
resulting function. And you can't do that with an auto ref 
template, which makes them quite annoying.

I wonder why is auto ref designed this way? And can we change 
this?
Mar 20 2017
next sibling parent Yuxuan Shui <yshuiv7 gmail.com> writes:
On Monday, 20 March 2017 at 19:49:03 UTC, Yuxuan Shui wrote:

 And you can't do that with an auto ref template, which makes 
 them quite annoying.


BTW, the error message you get when you try to do this, is not 
very helpful:

'auto' can only be used as part of 'auto ref' for template 
function parameters
Mar 20 2017
prev sibling next sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, March 20, 2017 19:49:03 Yuxuan Shui via Digitalmars-d wrote:

 An auto ref function template should behave like a normal
 function template, but it doesn't.

 You can fully instantiate a function template by specifying all
 of its template parameters, but you can't do that with auto ref
 templates. The only way to instantiate an auto ref template is to
 call it.

 This makes auto ref an outlier. Because you get a function by
 instantiate function template, you can pass the result as
 template alias argument, and you can create alias of the
 resulting function. And you can't do that with an auto ref
 template, which makes them quite annoying.

 I wonder why is auto ref designed this way? And can we change
 this?


Well, you can explicitly instantiate it and call it at the same time, it
does work. When it doesn't work is when you just try and instantiate it
without calling it. And the reason is because the refness is inferred from
the function argument. If the argument is an lvalue, then the parameter is
inferred as ref, whereas if it's an rvalue, it's inferred as non-ref. The
refness is not actually part of the type, because ref is not a type
qualifier. As such, AFAIK, the language simply has no way to pass the
refness as part of the explicit instantiation. In something like

auto foo(T)(auto ref T t)
{
    ...
}

the T in the template parameters has nothing to do with the ref, and in
fact, you could have something like

auto foo(T)(auto ref T t1, auto ref T t2)
{
    ...
}

and end up with t1 being ref and t2 being non-ref (or vice versa or both ref
or neithe ref). So, the refness would need to somehow be indicated
separately from the template argument, and I have no idea what that would
even look like.

So, yes, this particular restriction can be annoying, but there is a good
reason for the restriction (though the error message _is_ pretty bad), and I
have no idea how we would fix the problem.

- Jonathan M Davis
Mar 20 2017
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, March 20, 2017 13:20:52 Jonathan M Davis via Digitalmars-d wrote:

 So, yes, this particular restriction can be annoying, but there is a good
 reason for the restriction (though the error message _is_ pretty bad), and
 I have no idea how we would fix the problem.


After thinking about this further, it does occur to me that there's a fairly
simple workaround. e.g.

auto foo(T)(T t)
{
    return bar(t);
}

auto bar(T)(auto ref T t)
{
    ...
}

Then you can just use foo!int. Now, that requires you to pick whether it's
ref or not, but if there were a way to explicitly instantiate a function
with an auto ref parameter, you'd have to do that anyway.

- Jonathan M Davis
Mar 20 2017
parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Monday, 20 March 2017 at 21:08:40 UTC, Jonathan M Davis wrote:

 On Monday, March 20, 2017 13:20:52 Jonathan M Davis via 
 Digitalmars-d wrote:

 So, yes, this particular restriction can be annoying, but 
 there is a good reason for the restriction (though the error 
 message _is_ pretty bad), and I have no idea how we would fix 
 the problem.


 After thinking about this further, it does occur to me that 
 there's a fairly simple workaround. e.g.

 auto foo(T)(T t)
 {
     return bar(t);
 }

 auto bar(T)(auto ref T t)
 {
     ...
 }

 Then you can just use foo!int. Now, that requires you to pick 
 whether it's ref or not, but if there were a way to explicitly 
 instantiate a function with an auto ref parameter, you'd have 
 to do that anyway.

 - Jonathan M Davis


This is a bit tedious because it requires you creating a new 
function.

Maybe we can create a template for that. But still, auto ref 
requires us to do things differently, which is annoying.
Mar 20 2017
parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Monday, 20 March 2017 at 21:34:14 UTC, Yuxuan Shui wrote:

 On Monday, 20 March 2017 at 21:08:40 UTC, Jonathan M Davis 
 wrote:

 [...]


 This is a bit tedious because it requires you creating a new 
 function.

 Maybe we can create a template for that. But still, auto ref 
 requires us to do things differently, which is annoying.


Easy solution: just support auto ref for non-templates.

I think someone has already did work on that?
Mar 20 2017
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, March 20, 2017 21:37:26 Yuxuan Shui via Digitalmars-d wrote:

 On Monday, 20 March 2017 at 21:34:14 UTC, Yuxuan Shui wrote:

 On Monday, 20 March 2017 at 21:08:40 UTC, Jonathan M Davis

 wrote:

 [...]


 This is a bit tedious because it requires you creating a new
 function.

 Maybe we can create a template for that. But still, auto ref
 requires us to do things differently, which is annoying.


 Easy solution: just support auto ref for non-templates.

 I think someone has already did work on that?


auto ref for non-templates would not be quite the same thing, and
regardless, it wouldn't help any with explictly instantiating a template
that had an auto ref parameter. So, it really wouldn't solve the problem at
all. It would just make it so that if you didn't want a templated function,
you could use auto ref.

But for auto ref to work with non-templated functions, it would either
result in creating a combinatorial explosion of functions rather than just a
single function (basically doing what auto ref does but explicitly
instantiating it with every combination of refness instead of just the ones
that are actually used), or it would just create a version where every auto
ref parameter was ref and silently copy rvalues to the stack to pass by ref
when they're passed to the function. I think that there was a PR on the
issue at one point, and IIRC, the ultimate decision was to reject it and
keep the status quo, because there were too many downsides it (e.g. IIRC,
there were problems with overriding functions in derived classes).

I think that there was some talk semi-recently about creating a proposal for
something similar to C++'s const & but which was more restrictive in order
to avoid the downsides with C++'s solution, in which case we might end up
with something like  rvalue ref which accepted rvalues - probably by copying
them to the stack first. But I don't think that anything has been formally
proposed yet. If that goes anywhere, it will probably be the closest that
you'll ever get to having auto ref with non-templated functions.

- Jonathan M Davis
Mar 20 2017
parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Monday, 20 March 2017 at 21:53:47 UTC, Jonathan M Davis wrote:

 On Monday, March 20, 2017 21:37:26 Yuxuan Shui via 
 Digitalmars-d wrote:

 [...]


 auto ref for non-templates would not be quite the same thing, 
 and regardless, it wouldn't help any with explictly 
 instantiating a template that had an auto ref parameter. So, it 
 really wouldn't solve the problem at all. It would just make it 
 so that if you didn't want a templated function, you could use 
 auto ref.

 [...]


Makes sense...

OK, attempt 2: how about support implicit partial application for 
templates?

 [...]
Mar 20 2017
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, March 20, 2017 22:14:37 Yuxuan Shui via Digitalmars-d wrote:

 On Monday, 20 March 2017 at 21:53:47 UTC, Jonathan M Davis wrote:

 On Monday, March 20, 2017 21:37:26 Yuxuan Shui via

 Digitalmars-d wrote:

 [...]


 auto ref for non-templates would not be quite the same thing,
 and regardless, it wouldn't help any with explictly
 instantiating a template that had an auto ref parameter. So, it
 really wouldn't solve the problem at all. It would just make it
 so that if you didn't want a templated function, you could use
 auto ref.

 [...]


 Makes sense...

 OK, attempt 2: how about support implicit partial application for
 templates?


Well, you can do something like

template foo(T)
{
    auto foo()(auto ref T t)
    {
        return t;
    }
}

void main()
{
    alias f = foo!int;
    auto a = f(42);
}

and then foo!int, but you're not going to get a function pointer out of it,
since for that, you need to instantiate the inner function template. It does
give partial instantiation though.

- Jonathan M Davis
Mar 20 2017
parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Tuesday, 21 March 2017 at 01:10:38 UTC, Jonathan M Davis wrote:

 On Monday, March 20, 2017 22:14:37 Yuxuan Shui via 
 Digitalmars-d wrote:

 On Monday, 20 March 2017 at 21:53:47 UTC, Jonathan M Davis 
 wrote:

 [...]


 Makes sense...

 OK, attempt 2: how about support implicit partial application 
 for templates?


 Well, you can do something like

 template foo(T)
 {
     auto foo()(auto ref T t)
     {
         return t;
     }
 }

 void main()
 {
     alias f = foo!int;
     auto a = f(42);
 }

 and then foo!int, but you're not going to get a function 
 pointer out of it, since for that, you need to instantiate the 
 inner function template. It does give partial instantiation 
 though.

 - Jonathan M Davis


This is looks doable. Can we do this for all auto ref functions 
in phobos?
Mar 21 2017
parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tuesday, March 21, 2017 09:27:55 Yuxuan Shui via Digitalmars-d wrote:

 On Tuesday, 21 March 2017 at 01:10:38 UTC, Jonathan M Davis wrote:

 On Monday, March 20, 2017 22:14:37 Yuxuan Shui via

 Digitalmars-d wrote:

 On Monday, 20 March 2017 at 21:53:47 UTC, Jonathan M Davis

 wrote:

 [...]


 Makes sense...

 OK, attempt 2: how about support implicit partial application
 for templates?


 Well, you can do something like

 template foo(T)
 {
     auto foo()(auto ref T t)
     {
         return t;
     }
 }

 void main()
 {
     alias f = foo!int;
     auto a = f(42);
 }

 and then foo!int, but you're not going to get a function
 pointer out of it, since for that, you need to instantiate the
 inner function template. It does give partial instantiation
 though.


 This is looks doable. Can we do this for all auto ref functions
 in phobos?


Maybe, but I'm not sure that it's merited. On top of the fact that this
would make every auto ref function more verbose, remember that doing this
would add an extra template to every single auto ref function, which
increases their compilation overhead (as it is, we arguably instantiate
_way_ too many templates in many D programs). AFAIK, you're the first person
to ever bring this issue up, which implies that this is not something that
many folks are trying to do - and it _is_ fairly trivial to create wrappers
in the cases where you want to have an explicit instantiation. So, it might
make sense to do this, or it might just be unnecessary overhead for 99.99%
of cases while only mildly helping the .01% case. Certainly, it seems pretty
ugly to start saying that folks should be wrapping all auto ref functions in
an extra template. I'm inclined to think that we need a strong use case to
merit adding the extra overhead to all of these functions rather than having
the few who need something like this create wrapper functions.

So, what's your use case here? Why would you be looking to explicitly
instantiate templates (auto ref or otherwise) frequently enough that it's a
problem to just create a wrapper in the cases where you need to do it? Most
code does not explicitly instanitate templates, and there's usually no point
in doing so.

- Jonathan M Davis
Mar 22 2017