www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - auto ref is on the docket

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Walter and I discussed what auto ref for templates should look like and 
reached the conclusion that an approach based on lowering would be best. 
I added a proposed lowering to 
https://github.com/D-Programming-Language/dmd/pull/4717.

Andrei
Jun 21 2015
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/22/15 12:11 AM, Andrei Alexandrescu wrote:
 Walter and I discussed what auto ref for templates
You mean *non-templates*? auto ref for templates has a very well defined meaning. And reading your post, I'm unclear what exactly happens. Does this generate 2 functions from one, and then call the wrapper for auto-ref? So for instance: ref int fun(auto ref int x); What happens here? auto x = &fun; x(5); -Steve
Jun 21 2015
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/21/2015 10:08 PM, Steven Schveighoffer wrote:
 On 6/22/15 12:11 AM, Andrei Alexandrescu wrote:
 Walter and I discussed what auto ref for templates
You mean *non-templates*? auto ref for templates has a very well defined meaning. And reading your post, I'm unclear what exactly happens. Does this generate 2 functions from one, and then call the wrapper for auto-ref? So for instance: ref int fun(auto ref int x); What happens here? auto x = &fun; x(5); -Steve
The idea is that fun(5) would be lowered to: auto tmp = 5; fun(tmp); But when talking to Andrei I didn't realize that it would be subtly different behavior than 'auto ref' for template functions, which makes me concerned that this is not a good idea. Note that one can always rewrite: ref int fun(ref int x); into: ref int fun()(auto ref int x); if auto ref is desired.
Jun 21 2015
next sibling parent "Temtaime" <temtaime gmail.com> writes:
On Monday, 22 June 2015 at 05:25:57 UTC, Walter Bright wrote:
 On 6/21/2015 10:08 PM, Steven Schveighoffer wrote:
 On 6/22/15 12:11 AM, Andrei Alexandrescu wrote:
 Walter and I discussed what auto ref for templates
You mean *non-templates*? auto ref for templates has a very well defined meaning. And reading your post, I'm unclear what exactly happens. Does this generate 2 functions from one, and then call the wrapper for auto-ref? So for instance: ref int fun(auto ref int x); What happens here? auto x = &fun; x(5); -Steve
The idea is that fun(5) would be lowered to: auto tmp = 5; fun(tmp); But when talking to Andrei I didn't realize that it would be subtly different behavior than 'auto ref' for template functions, which makes me concerned that this is not a good idea. Note that one can always rewrite: ref int fun(ref int x); into: ref int fun()(auto ref int x); if auto ref is desired.
I think too that it's unclear and misleading. What will happen if auto ref function will be templated? How it interacts with xurrent logic ? And why not simply allow "ref in" to pass a temporary object?
Jun 21 2015
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, 22 June 2015 at 05:25:57 UTC, Walter Bright wrote:
 The idea is that fun(5) would be lowered to:

    auto tmp = 5;
    fun(tmp);
That's not what Andrei described (though it's what I would have expected us to do). Rather, Andrei seems to be suggesting that having an auto ref parameter would result in defining both a ref and non-ref overload of the function (with the non-ref one forwarding to the ref one), which would unfortunately result in a combinatorial explosion of function definitions (just like the templated version does, except that all of the overloads would just forward to the one which took all refs, and all of the overloads would always exist regardless of whether they were called). So, I'm inclined to think that what you just suggested makes a lot more sense than what Andrei suggested.
 But when talking to Andrei I didn't realize that it would be 
 subtly different behavior than 'auto ref' for template 
 functions, which makes me concerned that this is not a good 
 idea.
I don't see how there's any way around that given that the behavior of the templated auto ref pretty much depends on the function being templated, though Andrei's suggestion comes a lot closer than having the compiler insert a variable for you. Also, by reusing auto ref for this rather than coming up with a new attribute, we can't declare templated functions with the same behavior as the non-templated ones would get for auto ref (though that's a lot more important IMHO if we implement it via introducing a variable rather than introducing overloads, since then we'd be able to avoid the template bloat in the cases where you're just trying to accept both lvalues and rvalues and don't care about forwarding refness).
 Note that one can always rewrite:

     ref int fun(ref int x);

 into:

     ref int fun()(auto ref int x);

 if auto ref is desired.
Yes, but to be honest, the way that Andrei is suggesting that we implement this is so close to just templatizing the function that the only benefit to it I see over the status quo is that it would work with virtual functions, whereas the templated version obviously won't. I really think that we should consider implementing this via _one_ function where all of the auto ref parameters are lowered to ref, and variables are added where necessary by the compiler rather than implementing it via lowering it to a combinatorial explosion of overloads. And if we _do_ implement this via a single function with the compiler adding variables rather than having the compiler add overloads, I really think that we should consider introducing a new attribute for this so that we can use it with templated functions and avoid all of the template bloat that we get with auto ref right now. There are cases where auto ref is exactly what you want, but there are plenty of cases where folks use it just so that they can have their function accept both lvalues and rvalues, and in that case, all of the extra template instantiations are just bloat. So, if we had anyref or whatever we wanted to call it, we could use it both with templated and non-templated functions, and we would avoid the confusion introduced by having auto ref mean different things for templated and non-templated functions. Yes, that would mean introducing another attribute, which kind of sucks, but its use is specific enough that I think that it would be worth having, and I fully expect that it would be a big help in reducing template bloat in many programs - especially from those folks who keep clamoring for C++'s const& in D. - Jonathan M Davis
Jun 21 2015
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/21/15 10:25 PM, Walter Bright wrote:
 On 6/21/2015 10:08 PM, Steven Schveighoffer wrote:
 On 6/22/15 12:11 AM, Andrei Alexandrescu wrote:
 Walter and I discussed what auto ref for templates
You mean *non-templates*? auto ref for templates has a very well defined meaning. And reading your post, I'm unclear what exactly happens. Does this generate 2 functions from one, and then call the wrapper for auto-ref? So for instance: ref int fun(auto ref int x); What happens here? auto x = &fun; x(5); -Steve
The idea is that fun(5) would be lowered to: auto tmp = 5; fun(tmp);
I don't think that lowering is recommended - it prolongs the lifetime of the temporary through the end of the caller. But that may be actually a good thing.
 But when talking to Andrei I didn't realize that it would be subtly
 different behavior than 'auto ref' for template functions, which makes
 me concerned that this is not a good idea.
That's fine.
 Note that one can always rewrite:

      ref int fun(ref int x);

 into:

      ref int fun()(auto ref int x);

 if auto ref is desired.
I don't understand this. My lowering scheme doesn't allow for that. Andrei
Jun 21 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/21/15 11:31 PM, Andrei Alexandrescu wrote:
 On 6/21/15 10:25 PM, Walter Bright wrote:
 The idea is that fun(5) would be lowered to:

     auto tmp = 5;
     fun(tmp);
I don't think that lowering is recommended - it prolongs the lifetime of the temporary through the end of the caller. But that may be actually a good thing.
On second thought - Walter's lowering, which makes the rvalue last more than strictly necessary, may be the most flexible of all at the cost of more resource consumption (for types that define destructors). -- Andrei
Jun 21 2015
next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, 22 June 2015 at 06:38:57 UTC, Andrei Alexandrescu 
wrote:
 On 6/21/15 11:31 PM, Andrei Alexandrescu wrote:
 On 6/21/15 10:25 PM, Walter Bright wrote:
 The idea is that fun(5) would be lowered to:

     auto tmp = 5;
     fun(tmp);
I don't think that lowering is recommended - it prolongs the lifetime of the temporary through the end of the caller. But that may be actually a good thing.
On second thought - Walter's lowering, which makes the rvalue last more than strictly necessary, may be the most flexible of all at the cost of more resource consumption (for types that define destructors). -- Andrei
I would think in the general case, the variable would need to stick around until the statement was completed in order to avoid problems with it going away prematurely, though maybe the recent changes with return as an attribute fix that problem. But as long as its legal for the function to return an auto ref argument via ref or auto ref, keeping the variable around for the duration of the call - or even just the expression - rather than the statement would risk using the variable after it was destroyed. - Jonathan M Davis
Jun 21 2015
prev sibling next sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 22 June 2015 at 06:38:57 UTC, Andrei Alexandrescu 
wrote:
 On 6/21/15 11:31 PM, Andrei Alexandrescu wrote:
 On 6/21/15 10:25 PM, Walter Bright wrote:
 The idea is that fun(5) would be lowered to:

     auto tmp = 5;
     fun(tmp);
I don't think that lowering is recommended - it prolongs the lifetime of the temporary through the end of the caller. But that may be actually a good thing.
On second thought - Walter's lowering, which makes the rvalue last more than strictly necessary, may be the most flexible of all at the cost of more resource consumption (for types that define destructors). -- Andrei
Why would that be desirable? Resource consumption is the least of the problems. It is simply surprising, inconsistent with the lifetime of other temporaries, and it is unnecessary. Just lower it to: { auto tmp = 5; fun(tmp); }
Jun 22 2015
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 22 June 2015 at 10:04:28 UTC, Marc Schütz wrote:
 On Monday, 22 June 2015 at 06:38:57 UTC, Andrei Alexandrescu 
 wrote:
 On 6/21/15 11:31 PM, Andrei Alexandrescu wrote:
 On 6/21/15 10:25 PM, Walter Bright wrote:
 The idea is that fun(5) would be lowered to:

     auto tmp = 5;
     fun(tmp);
I don't think that lowering is recommended - it prolongs the lifetime of the temporary through the end of the caller. But that may be actually a good thing.
On second thought - Walter's lowering, which makes the rvalue last more than strictly necessary, may be the most flexible of all at the cost of more resource consumption (for types that define destructors). -- Andrei
Why would that be desirable? Resource consumption is the least of the problems. It is simply surprising, inconsistent with the lifetime of other temporaries, and it is unnecessary. Just lower it to: { auto tmp = 5; fun(tmp); }
I see now that Andrei already wrote in the PR "creating a named temporary in a scope immediately enclosing the call". I would like to add that AFAIK other temporaries live until the end of the entire statement they appear in, not just for the duration of the call. In any case, the temporaries involved here should not behave different from normal temporaries; in particular, they should have the same lifetime as with pass-by-value.
Jun 22 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/22/15 3:04 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:
 Just lower it to:

      {
          auto tmp = 5;
          fun(tmp);
      }
You need to lower an expression to an expression, not a statement. (e.g. what if fun returns a result?) I considered this lowering for "int fun(ref int);" fun(42) ==>> (function int(int a) { return fun(a); })(42) This does work, but if fun returns a ref int, I found no way to syntactically express that lambda. This does not parse: (function ref int(int a) { return fun(a); })(42) Is this a bug in the grammar? Andrei
Jun 22 2015
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 22 June 2015 at 15:39:38 UTC, Andrei Alexandrescu 
wrote:
 On 6/22/15 3:04 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:
 Just lower it to:

      {
          auto tmp = 5;
          fun(tmp);
      }
You need to lower an expression to an expression, not a statement. (e.g. what if fun returns a result?) I considered this lowering for "int fun(ref int);" fun(42) ==>> (function int(int a) { return fun(a); })(42) This does work, but if fun returns a ref int, I found no way to syntactically express that lambda. This does not parse: (function ref int(int a) { return fun(a); })(42) Is this a bug in the grammar?
Probably. But for lowering, the resulting AST doesn't really need to be representable in the language's syntax. I'm sure the AST _can_ express a lambda returning by reference. Anway, I really think we should not deviate from the normal lifetime rules for temporaries.
Jun 22 2015
prev sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 22 Jun 2015 08:40, "Andrei Alexandrescu via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:
 On 6/21/15 11:31 PM, Andrei Alexandrescu wrote:
 On 6/21/15 10:25 PM, Walter Bright wrote:
 The idea is that fun(5) would be lowered to:

     auto tmp = 5;
     fun(tmp);
I don't think that lowering is recommended - it prolongs the lifetime of the temporary through the end of the caller. But that may be actually a good thing.
On second thought - Walter's lowering, which makes the rvalue last more
than strictly necessary, may be the most flexible of all at the cost of more resource consumption (for types that define destructors). -- Andrei

I think keeping the lifetime of objects strictly in the call expression is
the behaviour that will give least surprise.

int i = 42;
struct S { ~this() { i++; } }

// auto __autoreftmp = S();
foo(S());  // __autoreftmp
// __dtor(__autoreftmp);
assert(i == 43);

As for optimisations, I think it should be possible to assert that the
reference never escapes, and use more aggressive optimisations based on
that, something that is still not possible with 'scope ref' or 'in ref'
parameters ...

Iain.
Jun 24 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Wednesday, 24 June 2015 at 07:12:53 UTC, Iain Buclaw wrote:
 On 22 Jun 2015 08:40, "Andrei Alexandrescu via Digitalmars-d" < 
 digitalmars-d puremagic.com> wrote:
 On 6/21/15 11:31 PM, Andrei Alexandrescu wrote:
 On 6/21/15 10:25 PM, Walter Bright wrote:
 The idea is that fun(5) would be lowered to:

     auto tmp = 5;
     fun(tmp);
I don't think that lowering is recommended - it prolongs the lifetime of the temporary through the end of the caller. But that may be actually a good thing.
On second thought - Walter's lowering, which makes the rvalue last more
than strictly necessary, may be the most flexible of all at the cost of more resource consumption (for types that define destructors). -- Andrei

 I think keeping the lifetime of objects strictly in the call 
 expression is the behaviour that will give least surprise.
Call expression, or statement containing the call expression? For normal temporaries, it's the latter AFAIK.
 int i = 42;
 struct S { ~this() { i++; } }

 // auto __autoreftmp = S();
 foo(S());  // __autoreftmp
 // __dtor(__autoreftmp);
 assert(i == 43);
It would make a difference for: auto x = foo(S()) + foo(S());
 As for optimisations, I think it should be possible to assert 
 that the reference never escapes, and use more aggressive 
 optimisations based on that, something that is still not 
 possible with 'scope ref' or 'in ref' parameters ...
Not possible because it's not implemented, or are there fundamental reasons against it?
Jun 24 2015
parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 24 June 2015 at 19:48, via Digitalmars-d <digitalmars-d puremagic.com> wrote:
 On Wednesday, 24 June 2015 at 07:12:53 UTC, Iain Buclaw wrote:
 On 22 Jun 2015 08:40, "Andrei Alexandrescu via Digitalmars-d" <
 digitalmars-d puremagic.com> wrote:
 On 6/21/15 11:31 PM, Andrei Alexandrescu wrote:
 On 6/21/15 10:25 PM, Walter Bright wrote:
 The idea is that fun(5) would be lowered to:

     auto tmp = 5;
     fun(tmp);
I don't think that lowering is recommended - it prolongs the lifetime of the temporary through the end of the caller. But that may be actually a good thing.
On second thought - Walter's lowering, which makes the rvalue last more
than strictly necessary, may be the most flexible of all at the cost of more resource consumption (for types that define destructors). -- Andrei

 I think keeping the lifetime of objects strictly in the call expression is
 the behaviour that will give least surprise.
Call expression, or statement containing the call expression? For normal temporaries, it's the latter AFAIK.
I suppose DMD doesn't have an equivalent, but they are called BIND_EXPR in GCC, where you have a new local block with assigned temporaries that expire outside of the given bind.
 int i = 42;
 struct S { ~this() { i++; } }

 // auto __autoreftmp = S();
 foo(S());  // __autoreftmp
 // __dtor(__autoreftmp);
 assert(i == 43);
It would make a difference for: auto x = foo(S()) + foo(S());
It works like this, the last result is binded back to the lvalue. auto x = { tmpA = S(); tmpB = S(); foo(tmpA) + foo(tmpB); }
 As for optimisations, I think it should be possible to assert that the
 reference never escapes, and use more aggressive optimisations based on
 that, something that is still not possible with 'scope ref' or 'in ref'
 parameters ...
Not possible because it's not implemented, or are there fundamental reasons against it?
Ignoring the 'ref' part of my comment, so as we are not confusing DIP36 (which I've only just learned about). The last time I checked, the use of 'scope' and 'in' on parameters are ignored by escape analysis, so you can quite happily escape a reference as you see fit and the language will allow you to do that. So, I cannot optimised if the language allows you to break that optimisation without warning or erroring the user. If I did, then ultimately you'll end up seeing your programs becoming silently corrupted.
Jun 24 2015
prev sibling parent reply "Daniel N" <ufo orbiting.us> writes:
On Monday, 22 June 2015 at 05:25:57 UTC, Walter Bright wrote:
 The idea is that fun(5) would be lowered to:

    auto tmp = 5;
    fun(tmp);

 But when talking to Andrei I didn't realize that it would be 
 subtly different behavior than 'auto ref' for template 
 functions, which makes me concerned that this is not a good 
 idea.
Considering we already have working 'Sealed references', why not simply allow rvalues for plain ref? Bonus: No risk of mixup with template auto ref after refactoring etc. Save auto for other potential future use.
Jun 22 2015
next sibling parent reply "Namespace" <rswhite4 gmail.com> writes:
On Monday, 22 June 2015 at 20:42:58 UTC, Daniel N wrote:
 On Monday, 22 June 2015 at 05:25:57 UTC, Walter Bright wrote:
 The idea is that fun(5) would be lowered to:

    auto tmp = 5;
    fun(tmp);

 But when talking to Andrei I didn't realize that it would be 
 subtly different behavior than 'auto ref' for template 
 functions, which makes me concerned that this is not a good 
 idea.
Considering we already have working 'Sealed references', why not simply allow rvalues for plain ref? Bonus: No risk of mixup with template auto ref after refactoring etc. Save auto for other potential future use.
That would be horrible. How would you distinguish between lvalues and rvalues? What if you want to store a pointer to an lvalue? If ref accept both you cannot do that.
Jun 22 2015
parent "Daniel N" <ufo orbiting.us> writes:
On Monday, 22 June 2015 at 20:50:23 UTC, Namespace wrote:
 That would be horrible. How would you distinguish between 
 lvalues and rvalues? What if you want to store a pointer to an 
 lvalue? If ref accept both you cannot do that.
storing requires 'return ref'
Jun 22 2015
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, 22 June 2015 at 20:42:58 UTC, Daniel N wrote:
 Considering we already have working 'Sealed references', why 
 not simply allow rvalues for plain ref?
Please, please, no. That would be horrible. ref indicates an intention to mutate the argument, which makes no sense with rvalues. And if ref is used simply so that you can accept both rvalues and lvalues without copying lvalues, then there's zero indication in the function signature when the point of the ref is to actually set what's being passed in. It works in C++ with const&, because you know that the function won't mutate the argument, but if you allow it with ref in general, then you stand no chance of being able to look at a function signature and deduce whether the function intends to mutate an argument or not. And even if Andrei wasn't so set against const ref that we could go that route in D as well, it wouldn't solve the problem either, because const in D is just too restrictive to use in many cases. We need a way to indicate that a parameter accepts both lvalues and rvalues without intending to mutate the argument but without actually guaranteeing it via const. And that does _not_ work with naked ref. - Jonathan M Davis
Jun 22 2015
parent reply "Daniel N" <ufo orbiting.us> writes:
On Monday, 22 June 2015 at 20:56:12 UTC, Jonathan M Davis wrote:
 won't mutate the argument, but if you allow it with ref in 
 general, then you stand no chance of being able to look at a 
 function signature and deduce whether the function intends to 
 mutate an argument or not.

 - Jonathan M Davis
It's no worse than auto ref. It's only by convention that auto ref functions doesn't mutate, at least I follow that convention, but when looking at someone else's function, all bets are off.
Jun 22 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, 22 June 2015 at 21:30:45 UTC, Daniel N wrote:
 On Monday, 22 June 2015 at 20:56:12 UTC, Jonathan M Davis wrote:
 won't mutate the argument, but if you allow it with ref in 
 general, then you stand no chance of being able to look at a 
 function signature and deduce whether the function intends to 
 mutate an argument or not.

 - Jonathan M Davis
It's no worse than auto ref. It's only by convention that auto ref functions doesn't mutate, at least I follow that convention, but when looking at someone else's function, all bets are off.
No, you can't guarantee that an auto ref parameter won't be mutated, but auto ref clearly indicates that you intend to accept both lvalues and rvalues, meaning that it would be stupid to be writing the function with the idea that you would be passing in an argument which would be mutated. ref on the other hand clearly indicates the intention to mutate the argument. - Jonathan M Davis
Jun 22 2015
parent reply "Daniel N" <ufo orbiting.us> writes:
On Tuesday, 23 June 2015 at 01:07:17 UTC, Jonathan M Davis wrote:
 No, you can't guarantee that an auto ref parameter won't be 
 mutated, but auto ref clearly indicates that you intend to 
 accept both lvalues and rvalues, meaning that it would be 
 stupid to be writing the function with the idea that you would 
 be passing in an argument which would be mutated. ref on the 
 other hand clearly indicates the intention to mutate the 
 argument.

 - Jonathan M Davis
That is why, as a good API designer, you'd express that fact by adding const ontop of it, i.e. const ref. Only for the few rare cases where you need to account for lazy-caching or similar mutating constructs, you'd be grateful to have the nice escape hatch, plain ref. Daniel N
Jun 22 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, 23 June 2015 at 06:49:50 UTC, Daniel N wrote:
 On Tuesday, 23 June 2015 at 01:07:17 UTC, Jonathan M Davis 
 wrote:
 No, you can't guarantee that an auto ref parameter won't be 
 mutated, but auto ref clearly indicates that you intend to 
 accept both lvalues and rvalues, meaning that it would be 
 stupid to be writing the function with the idea that you would 
 be passing in an argument which would be mutated. ref on the 
 other hand clearly indicates the intention to mutate the 
 argument.

 - Jonathan M Davis
That is why, as a good API designer, you'd express that fact by adding const ontop of it, i.e. const ref.
Which does not generally work in D. It can in some cases, but const is so restrictive in D that you simply cannot use it just because you don't intend to mutate a variable. Too many types won't work with const, and many types _can't_ work with const. const has its uses, but you have to be careful with it, and in the general case, that means that you can't put it on function parameters just to indicate that the function argument is not going to mutated. - Jonathan M Davis
Jun 23 2015
parent reply "Daniel N" <ufo orbiting.us> writes:
On Tuesday, 23 June 2015 at 11:03:10 UTC, Jonathan M Davis wrote:
 Which does not generally work in D. It can in some cases, but 
 const is so restrictive in D that you simply cannot use it just 
 because you don't intend to mutate a variable. Too many types 
 won't work with const, and many types _can't_ work with const. 
 const has its uses, but you have to be careful with it, and in 
 the general case, that means that you can't put it on function 
 parameters just to indicate that the function argument is not 
 going to mutated.

 - Jonathan M Davis
Thus the solution cannot require const. auto is worse because, if you later decide you need to add a template parameter then the meaning is changed and you get a _hidden_ performance issue. Thus requiring neither auto nor const is the least evil of the above. You could always use comments, or even dummy aliases to make the code self-documenting: alias view(T...) = T; void example(ref view!int i) {}
Jun 23 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, 23 June 2015 at 13:52:19 UTC, Daniel N wrote:
 On Tuesday, 23 June 2015 at 11:03:10 UTC, Jonathan M Davis 
 wrote:
 Which does not generally work in D. It can in some cases, but 
 const is so restrictive in D that you simply cannot use it 
 just because you don't intend to mutate a variable. Too many 
 types won't work with const, and many types _can't_ work with 
 const. const has its uses, but you have to be careful with it, 
 and in the general case, that means that you can't put it on 
 function parameters just to indicate that the function 
 argument is not going to mutated.

 - Jonathan M Davis
Thus the solution cannot require const. auto is worse because, if you later decide you need to add a template parameter then the meaning is changed and you get a _hidden_ performance issue.
And how does it introduce a hidden performance issue. You'd end up with some template bloat, but that's it, and that's just going to increase the memory footprint of your program slightly, which isn't generally going to be a performance issue. It's just going to make your program slightly larger. - Jonathan M Davis
Jun 23 2015
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 23 June 2015 at 16:27:34 UTC, Jonathan M Davis wrote:
 On Tuesday, 23 June 2015 at 13:52:19 UTC, Daniel N wrote:
 On Tuesday, 23 June 2015 at 11:03:10 UTC, Jonathan M Davis 
 wrote:
 [...]
Thus the solution cannot require const. auto is worse because, if you later decide you need to add a template parameter then the meaning is changed and you get a _hidden_ performance issue.
And how does it introduce a hidden performance issue. You'd end up with some template bloat, but that's it, and that's just going to increase the memory footprint of your program slightly, which isn't generally going to be a performance issue. It's just going to make your program slightly larger. - Jonathan M Davis
instruction cache misses can really hurt
Jun 23 2015
next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, 23 June 2015 at 17:05:41 UTC, John Colvin wrote:
 On Tuesday, 23 June 2015 at 16:27:34 UTC, Jonathan M Davis 
 wrote:
 On Tuesday, 23 June 2015 at 13:52:19 UTC, Daniel N wrote:
 On Tuesday, 23 June 2015 at 11:03:10 UTC, Jonathan M Davis 
 wrote:
 [...]
Thus the solution cannot require const. auto is worse because, if you later decide you need to add a template parameter then the meaning is changed and you get a _hidden_ performance issue.
And how does it introduce a hidden performance issue. You'd end up with some template bloat, but that's it, and that's just going to increase the memory footprint of your program slightly, which isn't generally going to be a performance issue. It's just going to make your program slightly larger. - Jonathan M Davis
instruction cache misses can really hurt
Well, if you're caring about stuff on that level, then you really need to understand what kind of code the compiler is generating, and most anything you do to the program risks changing its performance. So, I don't see much reason to worry about how a slight increase in program size from templatizing a function using auto ref could introduce cache misses. The fact alone that you've templatized the function could change its code gen enough to do that given that the compiler is then guaranteed to have the code available and will be doing attribute inference and whatnot. And most code probably won't see a performance difference anyway. Performance-intensive stuff may care, but you need to be very careful when tweaking that sort of code anyway, simply because all kinds of small tweaks can have unexpected impacts on performance. - Jonathan M Davis
Jun 23 2015
prev sibling parent "weaselcat" <weaselcat gmail.com> writes:
On Tuesday, 23 June 2015 at 17:05:41 UTC, John Colvin wrote:
 On Tuesday, 23 June 2015 at 16:27:34 UTC, Jonathan M Davis 
 wrote:
 On Tuesday, 23 June 2015 at 13:52:19 UTC, Daniel N wrote:
 [...]
And how does it introduce a hidden performance issue. You'd end up with some template bloat, but that's it, and that's just going to increase the memory footprint of your program slightly, which isn't generally going to be a performance issue. It's just going to make your program slightly larger. - Jonathan M Davis
instruction cache misses can really hurt
any time spent optimizing for this would be better spent making GDC/LDC 'good enough' to replace dmd.
Jun 23 2015
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/21/15 10:08 PM, Steven Schveighoffer wrote:
 On 6/22/15 12:11 AM, Andrei Alexandrescu wrote:
 Walter and I discussed what auto ref for templates
You mean *non-templates*? auto ref for templates has a very well defined meaning.
Yes, non-tempaltes.
 And reading your post, I'm unclear what exactly happens. Does this
 generate 2 functions from one, and then call the wrapper for auto-ref?

 So for instance:

 ref int fun(auto ref int x);

 What happens here?

 auto x = &fun;
 x(5);
What you'd expect after lowering: error. Andrei
Jun 21 2015
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, 22 June 2015 at 04:11:41 UTC, Andrei Alexandrescu 
wrote:
 Walter and I discussed what auto ref for templates should look 
 like and reached the conclusion that an approach based on 
 lowering would be best. I added a proposed lowering to 
 https://github.com/D-Programming-Language/dmd/pull/4717.
So, you're basically suggesting that have an auto ref parameter on a non-templated function basically does the combinatorial explosion of function definitions that you get with templated functions except that all of the combinations are instantiated (the one difference then being that all of the overloads except the one with all refs would just be wrappers calling the one with all refs)? That doesn't sound particularly desirable me. It's better than writing all of those wrappers by hand, but that's still likely to be a fair bit of code bloat if you're using auto ref parameters much - and if the folks coming from C++ who have been begging for this get it, they _will_ start using it all over the place. I would have thought that we'd define this such that the compiler introduced a variable restricted to the scope of the statement for each rvalue passed as an auto ref parameter rather than introducing function overloads. No, that's not quite as nice a lowering, but it would avoid a lot of extraneous function definitions. - Jonathan M Davis
Jun 21 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/21/15 11:11 PM, Jonathan M Davis wrote:
 No, that's not quite as nice a lowering, but it would avoid a lot of
 extraneous function definitions.
I think a lowering is a must. Matters are complicated as they are, I don't want to add new rules. Combinatorial issues can be addressed by either generating the functions on demand, or by lowering at the client side. Andrei
Jun 21 2015
prev sibling next sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 22 June 2015 at 04:11:41 UTC, Andrei Alexandrescu 
wrote:
 Walter and I discussed what auto ref for templates should look 
 like and reached the conclusion that an approach based on 
 lowering would be best. I added a proposed lowering to 
 https://github.com/D-Programming-Language/dmd/pull/4717.
I have to concur with Manu's statement in the PR. IMO `auto ref` (i.e. the status quo) is a hack introduced because of the lack of usable `scope`. We shouldn't spread it to other parts of the language, at least not for rvalue references. `scope ref` is the perfect fit for those. (There may still be a use for `auto ref` with the meaning "pass this as efficiently as possible, I don't care whether it's a reference or not", but I'm not convinced of that either.)
Jun 22 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, 22 June 2015 at 13:23:47 UTC, Marc Schütz wrote:
 On Monday, 22 June 2015 at 04:11:41 UTC, Andrei Alexandrescu 
 wrote:
 Walter and I discussed what auto ref for templates should look 
 like and reached the conclusion that an approach based on 
 lowering would be best. I added a proposed lowering to 
 https://github.com/D-Programming-Language/dmd/pull/4717.
I have to concur with Manu's statement in the PR. IMO `auto ref` (i.e. the status quo) is a hack introduced because of the lack of usable `scope`. We shouldn't spread it to other parts of the language, at least not for rvalue references. `scope ref` is the perfect fit for those. (There may still be a use for `auto ref` with the meaning "pass this as efficiently as possible, I don't care whether it's a reference or not", but I'm not convinced of that either.)
I'm afraid that I don't understand this. What on earth does scope have to do with this issue? We need a way to indicate that a parameter should accept both lvalues and rvalues. auto ref was supposed to do that originally, but Walter misunderstood what Andrei was proposing and implemented what we have now, which is great for forwarding, but it doesn't solve the general case, because it only works with templates. That means that if we want that functionality we then need to either implement auto ref for non-templated functions as originally intended, or we need to come up with a new attribute which does that and which could be used with both templated and non-templated functions (thus not requiring the extra template bloat of the current auto ref implementation with templates if all you want is to accept both lvalues and rvalues, not have the refness of the argument forwarded). What on earth does _any_ of that have to do with scope? Theoretically, scope indicates that a variable isn't supposed to escape the function - whatever that really means (it's never actually been defined outside of delegates, and even there, I'm not sure that it's defined all that well). What does that have to do with accepting lvalues or rvalues? - Jonathan M Davis
Jun 22 2015
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 22 June 2015 at 23:49, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 22 June 2015 at 13:23:47 UTC, Marc Schütz wrote:
 On Monday, 22 June 2015 at 04:11:41 UTC, Andrei Alexandrescu wrote:
 Walter and I discussed what auto ref for templates should look like and
 reached the conclusion that an approach based on lowering would be best. I
 added a proposed lowering to
 https://github.com/D-Programming-Language/dmd/pull/4717.
I have to concur with Manu's statement in the PR. IMO `auto ref` (i.e. the status quo) is a hack introduced because of the lack of usable `scope`. We shouldn't spread it to other parts of the language, at least not for rvalue references. `scope ref` is the perfect fit for those. (There may still be a use for `auto ref` with the meaning "pass this as efficiently as possible, I don't care whether it's a reference or not", but I'm not convinced of that either.)
I'm afraid that I don't understand this. What on earth does scope have to do with this issue? We need a way to indicate that a parameter should accept both lvalues and rvalues.
This is what I don't understand; why does this have anything to do with rvalues/lvalues specifically? An rvalue is just a temporary. It can safely be passed to anything that we can be sure won't keep a reference to it; that's why escape analysis seems to be the best approach. This problem extends beyond rvalues, there are other forms of temporaries which exhibit the exact same problem, ie, an rvalue written to the stack explicitly on the line prior to the function call: f(vec3(1,2,3)); // onoes, comple error! vec3 v = vec3(1,2,3); f(v); // no worries bro! This is accepted today, but the safety of the operation is identical to the rejected line above. This is the de facto means to 'work around' the problem, and it's just a pointless nuisance. Both problems should be dealt with in the same way.
 auto ref was supposed to do that originally, but
 Walter misunderstood what Andrei was proposing and implemented what we have
 now, which is great for forwarding, but it doesn't solve the general case,
 because it only works with templates. That means that if we want that
 functionality we then need to either implement auto ref for non-templated
 functions as originally intended, or we need to come up with a new attribute
 which does that and which could be used with both templated and
 non-templated functions (thus not requiring the extra template bloat of the
 current auto ref implementation with templates if all you want is to accept
 both lvalues and rvalues, not have the refness of the argument forwarded).
It's been proposed repeatedly for years. 'scope' as a means of expressing borrowing or escape analysis would solve the problem elegantly, in addition to other uses. There was also an idea Walter had at dconf 13, where he could do runtime checking similar to range checking for safety validation, thereby allowing unqualified ref args to receive temporaries safely.
 What on earth does _any_ of that have to do with scope? Theoretically, scope
 indicates that a variable isn't supposed to escape the function - whatever
 that really means (it's never actually been defined outside of delegates,
 and even there, I'm not sure that it's defined all that well). What does
 that have to do with accepting lvalues or rvalues?
If a parameter was scope, it can safely receive an rvalue.
Jun 22 2015
parent reply "Namespace" <rswhite4 gmail.com> writes:
Rather than raising the matter of scope again and again, we 
should be thankful that a solution for this nasty problem is 
accepted and could be merged. How scope and escape analysis could 
do a better job is unclear and if you want to solve the problem 
this way you will wait a very long time. So please let us 
concentrate and discuss how we could solve this problem with auto 
ref. :)

To repeat my statement from the PR:
I would also prefer to avoid the generation of further functions, 
because I want to avoid the code bloat. I like the way of the 
current implementation, because it is following the way C++ does 
and will be understandable for every person which comes from C++.
Jun 22 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 22 June 2015 at 16:10:10 UTC, Namespace wrote:
 Rather than raising the matter of scope again and again, we 
 should be thankful that a solution for this nasty problem is 
 accepted and could be merged.
I disagree strongly with this. A bad solution is worse than no solution.
 How scope and escape analysis could do a better job is unclear 
 and if you want to solve the problem this way you will wait a 
 very long time.
Not at all. `auto ref` without a working `scope` implementation is unsafe. If that is acceptable for now, then it should be just as acceptable to already use `scope ref` as a syntax for the same semantics, even if the required escape-proofing of `scope` is not yet implemented. If/when we will finally get a working `scope`, using those functions will then be actually verified by the compiler. In the meantime, the `scope` keyword will at least express the intention to the caller and will serve as a reminder to the callee's author. In contrast, with `auto ref` accepting rvalues, we will have the same keywords `auto ref` mean two very different things for templates and non-templates, we still won't have a way to write rvalue-ref-accepting template functions, and when we will finally get `scope`, `auto ref` will be a pointless alternative syntax for `scope ref` that will have to be kept around forever.
 So please let us concentrate and discuss how we could solve 
 this problem with auto ref. :)
Use `scope ref` instead of `auto ref` now, and just ignore that it's unsafe for the moment. Your PR will become even shorter with it, because you no longer need `STCrvref`. Conceptually, just do "#define STCrvref (STCref | STCscope)".
 To repeat my statement from the PR:
 I would also prefer to avoid the generation of further 
 functions, because I want to avoid the code bloat.
That's what my suggestion does, too.
 I like the way of the current implementation, because it is 
 following the way C++ does and will be understandable for every 
 person which comes from C++.
With the difference that C++ requires const-ness.
Jun 22 2015
next sibling parent reply "Namespace" <rswhite4 gmail.com> writes:
We had this discussion already with DIP 36. A PR still exists 
AFAIK, but because it was rejected I don't think that's going to 
happen. :) So auto ref is obviously the choice. If you insist on 
scope/in ref, please make your own thread and ask why it was 
rejected or read the corresponding thread.
Jun 22 2015
parent reply "Temtaime" <temtaime gmail.com> writes:
I see no reasons why « ref in » is bad. Maybe someone explain ?
It's also natural for those who came from C++.
In C++ there's no problem with const&, so why they will be in D?
Jun 22 2015
parent reply "Namespace" <rswhite4 gmail.com> writes:
On Monday, 22 June 2015 at 18:03:43 UTC, Temtaime wrote:
 I see no reasons why « ref in » is bad. Maybe someone explain ?
 It's also natural for those who came from C++.
 In C++ there's no problem with const&, so why they will be in D?
Because const is transitive in D and therefore more restrictive.
Jun 22 2015
parent reply "kinke" <noone nowhere.com> writes:
On Monday, 22 June 2015 at 18:34:37 UTC, Namespace wrote:
 On Monday, 22 June 2015 at 18:03:43 UTC, Temtaime wrote:
 I see no reasons why « ref in » is bad. Maybe someone explain ?
 It's also natural for those who came from C++.
 In C++ there's no problem with const&, so why they will be in 
 D?
Because const is transitive in D and therefore more restrictive.
That's not a reason. It's just an additional restriction imposed by D and only prevents you from using the proposed `in ref T`/`const auto ref T` *shudder* equivalents if const-transitiveness would be violated. In my day-to-day C++ work, I almost never have to do that. C++ "simply" ;) relies on the callee not escaping rvalue references bound to `const T&`. D's `(const/immutable) ref` only allows lvalue arguments. It simply says: 'Hey stupid, don't give me an rvalue reference, it might escape!' So to circumvent this common problem, you'll currently have to either (a) create an `auto ref` template or (b) declare a new variable, potentially having to introduce a new scope. Is it ugly? (a) Leads to code bloat, but provides for in-place parameter construction (and no indirections) for rvalue arguments of suited types. (b) Definitely ugly. Is it safe now? (a) Ideally, the rvalue argument was in-place constructed in the function parameters stack. Noone but you, the dev, prevents the callee from escaping a reference to the parameter though. (b) Well, the code is more verbose and emphasizes the original function argument expression. But it's obviously still all your responsibility to take care of proper lifetime for escaping references. Just declaring the variable right before the function call isn't safer than passing the rvalue reference directly. So none of this really solves the underlying problem. One solution consists in classifying function parameter references as non-escaping ones (proposed `scope ref`/non-template `auto ref` accepting rvalues too, to be verified by compiler) and escaping ones (`ref`), the bad ones to watch out for. I'd like having to syntactically annotate an lvalue argument passed by non-scope ref, analog to C#, i.e., `foo(ref argument)`, to more easily catch the eye.
Jun 22 2015
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/23/2015 12:18 AM, kinke wrote:
 In C++ there's no problem with const&, so why they will be in D?
Because const is transitive in D and therefore more restrictive.
That's not a reason.
Yes it is.
 It's just an additional restriction imposed by D and only prevents
 you from using the proposed `in ref T`/`const auto ref T` *shudder*
 equivalents if const-transitiveness would be violated. In my
 day-to-day C++ work, I almost never have to do that.
Note "almost".
Jun 22 2015
parent reply "kinke" <noone nowhere.com> writes:
On Monday, 22 June 2015 at 23:04:14 UTC, Timon Gehr wrote:
 On 06/23/2015 12:18 AM, kinke wrote:
 In C++ there's no problem with const&, so why they will be 
 in D?
Because const is transitive in D and therefore more restrictive.
That's not a reason.
Yes it is.
 It's just an additional restriction imposed by D and only 
 prevents
 you from using the proposed `in ref T`/`const auto ref T` 
 *shudder*
 equivalents if const-transitiveness would be violated. In my
 day-to-day C++ work, I almost never have to do that.
Note "almost".
Ah, I remember you from another rvalue discussion some years ago. ;) Care to share an example where transitive const is a problem apart from obvious lazily caching stuff? I don't think I've encountered any lately.
Jun 22 2015
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/23/2015 01:27 AM, kinke wrote:
 On Monday, 22 June 2015 at 23:04:14 UTC, Timon Gehr wrote:
 On 06/23/2015 12:18 AM, kinke wrote:
 In C++ there's no problem with const&, so why they will be in D?
Because const is transitive in D and therefore more restrictive.
That's not a reason.
Yes it is.
 It's just an additional restriction imposed by D and only prevents
 you from using the proposed `in ref T`/`const auto ref T` *shudder*
 equivalents if const-transitiveness would be violated. In my
 day-to-day C++ work, I almost never have to do that.
Note "almost".
... Care to share an example where transitive const is a problem apart from obvious lazily caching stuff?
One obvious problem ought to be enough. Some of the remaining cases are obvious as well. Others are not. One way to avoid non-obvious issues is to keep language features orthogonal.
 I don't think I've encountered any lately.
The problems with transitive const to be encountered in day-to-day C++ work are few.
Jun 22 2015
parent "kink" <noone nowhere.com> writes:
On Monday, 22 June 2015 at 23:43:07 UTC, Timon Gehr wrote:
 The problems with transitive const to be encountered in 
 day-to-day C++ work are few.
Trolling as always. You know what I meant. Haven't had to resort to mutable fields and/or const_casts in a long time. Mostly a matter of good design imho. As different behavior for templated `auto ref` and non-template `auto ref` seems to be an issue for more people here, why don't just rename the non-templated one `scope ref`, so that it's usable for templates too and also enables the `in ref` shortcut for const? I find `const auto ref` extremely clumsy.
Jun 23 2015
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/22/15 9:54 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:
 `auto ref` without a working `scope` implementation is unsafe.
I'm unclear on this. With the existence of the return attribute, what's unsafe about auto ref? -- Andrei
Jun 22 2015
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 22 June 2015 at 18:10:11 UTC, Andrei Alexandrescu 
wrote:
 On 6/22/15 9:54 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:
 `auto ref` without a working `scope` implementation is unsafe.
I'm unclear on this. With the existence of the return attribute, what's unsafe about auto ref? -- Andrei
DIP25 _is_ a working `scope` implementation, for that matter, albeit incomplete. (Strictly speaking, it's not the `return` attribute that makes it safe, but the fact that under this proposal `ref` implies what `scope` was originally supposed to mean.) You're right that if DIP25 is all we'll ever get (i.e. the `scope` keyword will never have any meaning), then `auto ref` might not be so bad a choice, although it still leaves the fact that it does something different for template and non-template functions. But it would create other inconsistencies with other kinds of references, e.g. void bar(ref MyStruct s); void foo(MyClass c); void baz() { MyStruct s; Scoped!MyClass c; bar(s); // safe foo(c); // not safe, reference may escape } For consistency, I would much prefer if `ref` and scope-ness were kept independent, because they are orthogonal concepts. Why should one kind of reference imply `scope`, but not others?
Jun 23 2015
prev sibling parent reply "kinke" <noone nowhere.com> writes:
On Monday, 22 June 2015 at 16:54:50 UTC, Marc Schütz wrote:
 On Monday, 22 June 2015 at 16:10:10 UTC, Namespace wrote:
 Rather than raising the matter of scope again and again, we 
 should be thankful that a solution for this nasty problem is 
 accepted and could be merged.
I disagree strongly with this. A bad solution is worse than no solution.
 How scope and escape analysis could do a better job is unclear 
 and if you want to solve the problem this way you will wait a 
 very long time.
Not at all. `auto ref` without a working `scope` implementation is unsafe. If that is acceptable for now, then it should be just as acceptable to already use `scope ref` as a syntax for the same semantics, even if the required escape-proofing of `scope` is not yet implemented. If/when we will finally get a working `scope`, using those functions will then be actually verified by the compiler. In the meantime, the `scope` keyword will at least express the intention to the caller and will serve as a reminder to the callee's author. In contrast, with `auto ref` accepting rvalues, we will have the same keywords `auto ref` mean two very different things for templates and non-templates, we still won't have a way to write rvalue-ref-accepting template functions, and when we will finally get `scope`, `auto ref` will be a pointless alternative syntax for `scope ref` that will have to be kept around forever.
 So please let us concentrate and discuss how we could solve 
 this problem with auto ref. :)
Use `scope ref` instead of `auto ref` now, and just ignore that it's unsafe for the moment. Your PR will become even shorter with it, because you no longer need `STCrvref`. Conceptually, just do "#define STCrvref (STCref | STCscope)".
I definitely see your point and very much agree!
 I like the way of the current implementation, because it is 
 following the way C++ does and will be understandable for 
 every person which comes from C++.
With the difference that C++ requires const-ness.
Yep, so `in ref T` translating to `scope const ref T` would be D's convenient and safe counterpart to C++' `const T&` parameters, for their main use case: passing pure-input arguments of types which are or may be costly to copy (post-blit ctor, dtor, or simply big). I explicitly mention 'may be' here because in templates one often doesn't know (containers...). As I have already pointed out in another thread, I'd go one step further and propose an extremely convenient `in T` for this very common use case: * The argument is passed by value (`const T`) if the compiler assumes moving/copying is more efficient than passing a reference (with its indirection on the callee side) for the particular target environment (hardware, ABI), e.g., for plain-old-datatypes T fitting into 1-2 registers, and Object references obviously. * Otherwise, the argument is passed by-ref (`in ref T`). As `in T` doesn't mention any ref at all, it's clear that the hidden reference cannot escape. Let me give you an example. The Win64 ABI only allows passing POD types <= 64 bit by value (exception: vector/SSE types). All other types need to be passed by ref, which means that D's otherwise cool in-place construction of arguments on the callee's function parameters stack isn't possible. The current `auto ref` implementation for templates is in vain on Win64 for these byref-types, as the ABI forces both versions to take the argument byref anyway. I'm pretty sure I could live without an `auto ref` for templates once I can use `in T` for all my pure-input parameters. If the function may change the argument, I'd go with `scope ref T`, accepting rvalues too. If the function may even let the argument escape, it'll have to be `ref T` or `const/immutable ref T`. And if I just want to mutate the parameter inside my function, but not touch the argument, it'd be `T`. I can't think of any other use cases for function parameters atm.
Jun 22 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 22 June 2015 at 19:05:28 UTC, kinke wrote:
 On Monday, 22 June 2015 at 16:54:50 UTC, Marc Schütz wrote:
 With the difference that C++ requires const-ness.
Yep, so `in ref T` translating to `scope const ref T` would be D's convenient and safe counterpart to C++' `const T&` parameters, for their main use case: passing pure-input arguments of types which are or may be costly to copy (post-blit ctor, dtor, or simply big). I explicitly mention 'may be' here because in templates one often doesn't know (containers...).
To clarify: What I meant by my comment was that const-ness should not be a precondition for allowing rvalue refs. Mutable rvalue refs are fine.
 As I have already pointed out in another thread, I'd go one 
 step further and propose an extremely convenient `in T` for 
 this very common use case:

 * The argument is passed by value (`const T`) if the compiler 
 assumes moving/copying is more efficient than passing a 
 reference (with its indirection on the callee side) for the 
 particular target environment (hardware, ABI), e.g., for 
 plain-old-datatypes T fitting into 1-2 registers, and Object 
 references obviously.
 * Otherwise, the argument is passed by-ref (`in ref T`). As `in 
 T` doesn't mention any ref at all, it's clear that the hidden 
 reference cannot escape.
Theoretically, `immutable ref` by itself would already allow these semantics (without the `scope` that `in` implies). Because (disregarding identity/addresses) for an immutable object there is no observable difference between pass-by-value and pass-by-reference. The same is not always true for `const ref`, whenever aliasing is possible: void foo(const ref int a, ref int b) { int x = a; b++; assert(x == a); // can fail if both refer to the same variable // similar with global variables } To guarantee this from the caller's POV, the callee must be pure and the parameters must be known not to alias each other.
Jun 23 2015
parent reply "kink" <noone nowhere.com> writes:
On Tuesday, 23 June 2015 at 09:57:26 UTC, Marc Schütz wrote:
 On Monday, 22 June 2015 at 19:05:28 UTC, kinke wrote:
 [...]
 To clarify: What I meant by my comment was that const-ness 
 should not be a precondition for allowing rvalue refs. Mutable 
 rvalue refs are fine.
I know and see it the same way.
 As I have already pointed out in another thread, I'd go one 
 step further and propose an extremely convenient `in T` for 
 this very common use case:

 * The argument is passed by value (`const T`) if the compiler 
 assumes moving/copying is more efficient than passing a 
 reference (with its indirection on the callee side) for the 
 particular target environment (hardware, ABI), e.g., for 
 plain-old-datatypes T fitting into 1-2 registers, and Object 
 references obviously.
 * Otherwise, the argument is passed by-ref (`in ref T`). As 
 `in T` doesn't mention any ref at all, it's clear that the 
 hidden reference cannot escape.
Theoretically, `immutable ref` by itself would already allow these semantics (without the `scope` that `in` implies). Because (disregarding identity/addresses) for an immutable object there is no observable difference between pass-by-value and pass-by-reference. The same is not always true for `const ref`, whenever aliasing is possible: void foo(const ref int a, ref int b) { int x = a; b++; assert(x == a); // can fail if both refer to the same variable // similar with global variables } To guarantee this from the caller's POV, the callee must be pure and the parameters must be known not to alias each other.
This is obviously true. Rvalues aren't affected as they cannot alias another parameter by definition. Lvalues are if passed by ref and the same instance is accessible by mutable ref from another parameter or global. But as shown by your example, that danger is always there. The proposed `in` semantics make it less obvious, that's true, but I still think it'd be worth it, as these aliasing bugs are a pain to track down, but in my experience extremely rare.
Jun 23 2015
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 23 June 2015 at 12:45:25 UTC, kink wrote:
 On Tuesday, 23 June 2015 at 09:57:26 UTC, Marc Schütz wrote:
 To guarantee this from the caller's POV, the callee must be 
 pure and the parameters must be known not to alias each other.
This is obviously true. Rvalues aren't affected as they cannot alias another parameter by definition.
You're right, I didn't think of that! They are not _transitively_ unique, but there can indeed be no other references to the rvalues themselves. So for rvalues, const is enough. Nice :-)
 Lvalues are if passed by ref and the same instance is 
 accessible by mutable ref from another parameter or global. But 
 as shown by your example, that danger is always there. The 
 proposed `in` semantics make it less obvious, that's true, but 
 I still think it'd be worth it, as these aliasing bugs are a 
 pain to track down, but in my experience extremely rare.
Jun 23 2015
prev sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 22 June 2015 at 13:49:31 UTC, Jonathan M Davis wrote:
 On Monday, 22 June 2015 at 13:23:47 UTC, Marc Schütz wrote:
 On Monday, 22 June 2015 at 04:11:41 UTC, Andrei Alexandrescu 
 wrote:
 Walter and I discussed what auto ref for templates should 
 look like and reached the conclusion that an approach based 
 on lowering would be best. I added a proposed lowering to 
 https://github.com/D-Programming-Language/dmd/pull/4717.
I have to concur with Manu's statement in the PR. IMO `auto ref` (i.e. the status quo) is a hack introduced because of the lack of usable `scope`. We shouldn't spread it to other parts of the language, at least not for rvalue references. `scope ref` is the perfect fit for those. (There may still be a use for `auto ref` with the meaning "pass this as efficiently as possible, I don't care whether it's a reference or not", but I'm not convinced of that either.)
I'm afraid that I don't understand this. What on earth does scope have to do with this issue? We need a way to indicate that a parameter should accept both lvalues and rvalues.
`scope` is a precondition for doing this safely. As Manu pointed out, rvalues are merely a special case of a more general problem, but with them, the risk of misuse is much higher than with local variables. That was probably the reason why `ref` originally was designed not to accept rvalues. Given a working `scope` feature, that reason is no longer relevant, therefore `scope ref` can safely accept both rvalues and lvalues.
Jun 22 2015
prev sibling next sibling parent "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Monday, 22 June 2015 at 04:11:41 UTC, Andrei Alexandrescu 
wrote:
 Walter and I discussed what auto ref for templates should look 
 like and reached the conclusion that an approach based on 
 lowering would be best. I added a proposed lowering to 
 https://github.com/D-Programming-Language/dmd/pull/4717.
`For example, consider the declaration:` ref int fun(auto ref int x); `That should lower to:` ref int fun(int' x); /* or */ ref int fun('int x);
Jun 22 2015
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/22/2015 06:11 AM, Andrei Alexandrescu wrote:
 Walter and I discussed what auto ref for templates should look like and
 reached the conclusion that an approach based on lowering would be best.
 I added a proposed lowering to
 https://github.com/D-Programming-Language/dmd/pull/4717.

 Andrei
" WalterBright auto ref for templates and non-templates do different things by necessity. ..." The proposed lowering also works for template functions. "... This is not an issue." Yes it is. Walter is right.
Jun 22 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/22/15 3:06 PM, Timon Gehr wrote:
 On 06/22/2015 06:11 AM, Andrei Alexandrescu wrote:
 Walter and I discussed what auto ref for templates should look like and
 reached the conclusion that an approach based on lowering would be best.
 I added a proposed lowering to
 https://github.com/D-Programming-Language/dmd/pull/4717.

 Andrei
" WalterBright auto ref for templates and non-templates do different things by necessity. ..." The proposed lowering also works for template functions. "... This is not an issue." Yes it is. Walter is right.
For templates, auto ref generates two copies of the function. Inside, which was generated can be interrogated by using is(param == ref). For non-templates, only one copy of the function is generated. Using is(param == ref) inside should be an error. The two mechanisms bear many similarities and some differences, too. Andrei
Jun 22 2015
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/23/2015 12:40 AM, Andrei Alexandrescu wrote:
 On 6/22/15 3:06 PM, Timon Gehr wrote:
 On 06/22/2015 06:11 AM, Andrei Alexandrescu wrote:
 Walter and I discussed what auto ref for templates should look like and
 reached the conclusion that an approach based on lowering would be best.
 I added a proposed lowering to
 https://github.com/D-Programming-Language/dmd/pull/4717.

 Andrei
" WalterBright auto ref for templates and non-templates do different things by necessity. ..." The proposed lowering also works for template functions. "... This is not an issue." Yes it is. Walter is right.
For templates, auto ref generates two copies of the function. Inside, which was generated can be interrogated by using is(param == ref). For non-templates, only one copy of the function is generated. Using is(param == ref) inside should be an error. The two mechanisms bear many similarities and some differences, too. ...
There is no reason to prevent templates from using the mechanism that generates only one copy. The two mechanisms shouldn't share the same syntax, because then there is no way to tell them apart for template functions.
Jun 22 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/22/15 4:09 PM, Timon Gehr wrote:
 There is no reason to prevent templates from using the mechanism that
 generates only one copy. The two mechanisms shouldn't share the same
 syntax, because then there is no way to tell them apart for template
 functions.
I understand. For my money I'd be okay with what's being proposed instead of complicating the language yet again for the perfect solution. -- Andrei
Jun 22 2015
next sibling parent "Namespace" <rswhite4 gmail.com> writes:
On Tuesday, 23 June 2015 at 01:01:26 UTC, Andrei Alexandrescu 
wrote:
 On 6/22/15 4:09 PM, Timon Gehr wrote:
 There is no reason to prevent templates from using the 
 mechanism that
 generates only one copy. The two mechanisms shouldn't share 
 the same
 syntax, because then there is no way to tell them apart for 
 template
 functions.
I understand. For my money I'd be okay with what's being proposed instead of complicating the language yet again for the perfect solution. -- Andrei
There is no perfect solution. :) Some are for in/scope ref, some are against it. Some are for auto ref, some are against it and even a few are for const ref but most are against it. So the perfect solution does not exist. And since scope/in ref was already rejected (DIP 36) I think auto ref is the only choice to introduce this feature without introducing a new attribute. And since you reverted the introducing of virtual because it has not enough value, I'm sure that introducing a new attribute just to accept both, lvalues and rvalues, wouldn't have enough value as well. I think the way it is implemented right now is good enough, but what matters is what you and Walter think.
Jun 23 2015
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/23/2015 03:01 AM, Andrei Alexandrescu wrote:
 On 6/22/15 4:09 PM, Timon Gehr wrote:
 There is no reason to prevent templates from using the mechanism that
 generates only one copy. The two mechanisms shouldn't share the same
 syntax, because then there is no way to tell them apart for template
 functions.
I understand. For my money I'd be okay with what's being proposed instead of complicating the language yet again for
We're introducing a new feature either way. I don't agree that the proposed hack complicates the language any less.
 the perfect solution.
We might as well do it right. Why promote unnecessary template bloat?
Jun 23 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, 24 June 2015 at 00:23:42 UTC, Timon Gehr wrote:
 On 06/23/2015 03:01 AM, Andrei Alexandrescu wrote:
 the perfect solution.
We might as well do it right. Why promote unnecessary template bloat?
I'm fine with it either way, but given all of the template bloat that auto ref creates and how common templates are in D (and will only become more so as more stuff is range-based and uses design by introspection), the cost of auto ref generating extra template instantiations will only increase. So, I think that we'd be better off simply adding a new attribute like anyref or whatever we want to call it. But I don't know how much it will ultimately matter. - Jonathan M Davis
Jun 24 2015
parent reply "Temtaime" <temtaime gmail.com> writes:
Simply give a possibility to "ref in" allowing use rvalues.
Don't add any new strange attributes.
Jun 24 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, 24 June 2015 at 09:26:49 UTC, Temtaime wrote:
 Simply give a possibility to "ref in" allowing use rvalues.
That has already been rejected. - Jonathan M Davis
Jun 24 2015
next sibling parent reply "Namespace" <rswhite4 gmail.com> writes:
On Wednesday, 24 June 2015 at 09:54:01 UTC, Jonathan M Davis 
wrote:
 On Wednesday, 24 June 2015 at 09:26:49 UTC, Temtaime wrote:
 Simply give a possibility to "ref in" allowing use rvalues.
That has already been rejected. - Jonathan M Davis
How many times did I say that already? :)
Jun 24 2015
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, 24 June 2015 at 10:02:12 UTC, Namespace wrote:
 On Wednesday, 24 June 2015 at 09:54:01 UTC, Jonathan M Davis 
 wrote:
 On Wednesday, 24 June 2015 at 09:26:49 UTC, Temtaime wrote:
 Simply give a possibility to "ref in" allowing use rvalues.
That has already been rejected. - Jonathan M Davis
How many times did I say that already? :)
Indeed. - Jonathan M Davis
Jun 24 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Wednesday, 24 June 2015 at 09:54:01 UTC, Jonathan M Davis 
wrote:
 On Wednesday, 24 June 2015 at 09:26:49 UTC, Temtaime wrote:
 Simply give a possibility to "ref in" allowing use rvalues.
That has already been rejected.
Then we need to reconsider it. If that was several years ago, much has changed since then. Can you point me to that decision? Who made it, and on what basis? I can't find anything in the threads linked in http://wiki.dlang.org/DIP36
Jun 24 2015
parent reply "Namespace" <rswhite4 gmail.com> writes:
On Wednesday, 24 June 2015 at 17:45:15 UTC, Marc Schütz wrote:
 On Wednesday, 24 June 2015 at 09:54:01 UTC, Jonathan M Davis 
 wrote:
 On Wednesday, 24 June 2015 at 09:26:49 UTC, Temtaime wrote:
 Simply give a possibility to "ref in" allowing use rvalues.
That has already been rejected.
Then we need to reconsider it. If that was several years ago, much has changed since then. Can you point me to that decision? Who made it, and on what basis? I can't find anything in the threads linked in http://wiki.dlang.org/DIP36
Read the thread: http://forum.dlang.org/thread/ylebrhjnrrcajnvtthtt forum.dlang.org?page=1
Jun 24 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Wednesday, 24 June 2015 at 17:47:51 UTC, Namespace wrote:
 On Wednesday, 24 June 2015 at 17:45:15 UTC, Marc Schütz wrote:
 On Wednesday, 24 June 2015 at 09:54:01 UTC, Jonathan M Davis 
 wrote:
 On Wednesday, 24 June 2015 at 09:26:49 UTC, Temtaime wrote:
 Simply give a possibility to "ref in" allowing use rvalues.
That has already been rejected.
Then we need to reconsider it. If that was several years ago, much has changed since then. Can you point me to that decision? Who made it, and on what basis? I can't find anything in the threads linked in http://wiki.dlang.org/DIP36
Read the thread: http://forum.dlang.org/thread/ylebrhjnrrcajnvtthtt forum.dlang.org?page=1
Thanks. There was no link to this thread on the wiki page. But I can't really find a definite rejection in it, in the sense of someone with authority saying "This DIP is rejected". The closest I can find is Andrei making some vague statements about what is proposed being a new feature (I don't know though whether this is an argument against the DIP), and it being unsafe because scope is not implemented (which was true at the time, but we now have DIP25 as a mechanism, albeit incomplete). If you can find anything definite, can you please add it to the wiki page? I.e. officially rejected by NN on YYYY-MM-DD (including link), summary of the arguments for rejection, etc.
Jun 25 2015
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, 25 June 2015 at 08:10:06 UTC, Marc Schütz wrote:
 On Wednesday, 24 June 2015 at 17:47:51 UTC, Namespace wrote:
 On Wednesday, 24 June 2015 at 17:45:15 UTC, Marc Schütz wrote:
 On Wednesday, 24 June 2015 at 09:54:01 UTC, Jonathan M Davis 
 wrote:
 On Wednesday, 24 June 2015 at 09:26:49 UTC, Temtaime wrote:
 Simply give a possibility to "ref in" allowing use rvalues.
That has already been rejected.
Then we need to reconsider it. If that was several years ago, much has changed since then. Can you point me to that decision? Who made it, and on what basis? I can't find anything in the threads linked in http://wiki.dlang.org/DIP36
Read the thread: http://forum.dlang.org/thread/ylebrhjnrrcajnvtthtt forum.dlang.org?page=1
Thanks. There was no link to this thread on the wiki page. But I can't really find a definite rejection in it, in the sense of someone with authority saying "This DIP is rejected". The closest I can find is Andrei making some vague statements about what is proposed being a new feature (I don't know though whether this is an argument against the DIP), and it being unsafe because scope is not implemented (which was true at the time, but we now have DIP25 as a mechanism, albeit incomplete). If you can find anything definite, can you please add it to the wiki page? I.e. officially rejected by NN on YYYY-MM-DD (including link), summary of the arguments for rejection, etc.
If it helps, at dconf, two years ago, when Manu tried to repeatedly convince Walter and Andrei to have scope ref do something similar to what we're talking about auto ref doing, Walter and Andrei repeatedly shot it down. They had no interest in having scope ref as any kind of special attribute. So, as of the beginning of May 2013 (which was less than a month after what the DIP lists as its last modification date), it was being refused by Walter and Andrei in person. And if you look at its revision history, Dicebot (who is one of the two authors of the DIP) has marked it as rejected. - Jonathan M Davis
Jun 25 2015
prev sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Mon, 22 Jun 2015 19:09:45 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:

 There is no reason to prevent templates from using the mechanism that  
 generates only one copy. The two mechanisms shouldn't share the same  
 syntax, because then there is no way to tell them apart for template  
 functions.
You would be losing the optimization of passing primitive types by value, wouldn't you? Bit
Jun 23 2015
parent reply "kinke" <noone nowhere.com> writes:
On Tuesday, 23 June 2015 at 18:11:21 UTC, bitwise wrote:
 On Mon, 22 Jun 2015 19:09:45 -0400, Timon Gehr 
 <timon.gehr gmx.ch> wrote:

 There is no reason to prevent templates from using the 
 mechanism that generates only one copy. The two mechanisms 
 shouldn't share the same syntax, because then there is no way 
 to tell them apart for template functions.
You would be losing the optimization of passing primitive types by value, wouldn't you?
Not if you keep the current `auto ref` template syntax AND introduce another syntax `scope ref` for non-escapable references also accepting rvalues.
Jun 23 2015
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Tue, 23 Jun 2015 14:33:49 -0400, kinke <noone nowhere.com> wrote:

 On Tuesday, 23 June 2015 at 18:11:21 UTC, bitwise wrote:
 On Mon, 22 Jun 2015 19:09:45 -0400, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 There is no reason to prevent templates from using the mechanism that  
 generates only one copy. The two mechanisms shouldn't share the same  
 syntax, because then there is no way to tell them apart for template  
 functions.
You would be losing the optimization of passing primitive types by value, wouldn't you?
Not if you keep the current `auto ref` template syntax AND introduce another syntax `scope ref` for non-escapable references also accepting rvalues.
I don't think 'scope ref' is on the table at this point(although I do believe it should be). And I still think Timon's statement is untrue. There is a reason, which is that the new auto ref syntax forces reference parameters for all types(even primitives), where the old template approach does not. Bit
Jun 23 2015
next sibling parent "kinke" <noone nowhere.com> writes:
On Tuesday, 23 June 2015 at 19:13:28 UTC, bitwise wrote:
 And I still think Timon's statement is untrue. There is a 
 reason, which is that the new auto ref syntax forces reference 
 parameters for all types(even primitives), where the old 
 template approach does not.
Timon suggested using another syntax/name for the new refs, as one couldn't use them in templates [and the different `auto ref` behavior is just prone to confuse beginners, my 2 cents]. One reason for preferring `scope ref` over `auto ref` in templates would be situations where code bloat is more important than runtime performance, another reason the fact that `in ref` is more convenient than `const auto ref`. ;)
Jun 23 2015
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, 23 June 2015 at 19:13:28 UTC, bitwise wrote:
 And I still think Timon's statement is untrue. There is a 
 reason, which is that the new auto ref syntax forces reference 
 parameters for all types(even primitives), where the old 
 template approach does not.
The "old template approach" isn't going anywhere. When used with templated functions, auto ref will continue to function the same way, and if you want that behavior, you templatize your function. AFAIK, there are really only two ways to tackle having a function accept both lvalues and rvalues for the same parameter: 1. Duplicate the entire function and have one overload take an lvalue and the other take an rvalue. This is what the the templated auto ref does (though which overloads get generated depends on what arguments are passed to the function). 2. Assign rvalues to an lvalue and pass that in to the function by ref. This is what's going to happen with non-templated functions. It's just a question of whether that's going to be done by introducing a temporary variable or by having the compiler add overloads which take rvalues and then call the primary function which takes its arguments by ref. And at this point, I think that there are really only three options that we might take in making it so that we can have non-templated functions accept both lvalues and rvalues with the same function declaration: 1. Leave things as they are and not support it except with templated functions (not desirable, but that may be the result of all of this). 2. Implement auto ref for non-templated functions as currently proposed, leaving auto ref on templated functions exactly as it is and providing no way to use the non-templated behavior with templated functions (though _maybe_ we could add compiler optimizations later to use the non-templated version in cases where the compiler can detect that using the non-templated version with a templated function would have the same semantics as the templated version). 3. Add a new attribute which does what's being proposed for auto ref for non-templated functions, in which case, we can use the non-templated behavior with templates as well and thus avoid template bloat when all you want is for your templated function to accept both lvalues and rvalues. auto ref, of course, then stays exactly as it is now. At the moment, it seems that #2 is the most likely, and that's probably fine, but I do wonder if we'd be better off with #3, especially when you consider how much D code tends to be templated and how much code bloat auto ref is likely to generate with templated functions. - Jonathan M Davis
Jun 24 2015
parent reply "kink" <noone nowhere.com> writes:
On Wednesday, 24 June 2015 at 11:19:04 UTC, Jonathan M Davis 
wrote:
 [...]

 3. Add a new attribute which does what's being proposed for 
 auto ref for non-templated functions, in which case, we can use 
 the non-templated behavior with templates as well and thus 
 avoid template bloat when all you want is for your templated 
 function to accept both lvalues and rvalues. auto ref, of 
 course, then stays exactly as it is now.

 At the moment, it seems that #2 is the most likely, and that's 
 probably fine, but I do wonder if we'd be better off with #3, 
 especially when you consider how much D code tends to be 
 templated and how much code bloat auto ref is likely to 
 generate with templated functions.

 - Jonathan M Davis
If that wasn't clear before, I'm all for #3 too. Just call it `scope ref` and simplify the PR a lil' bit as suggested by Marc in an earlier post [http://forum.dlang.org/post/ricvtchihgzyisbkzcgl forum.dlang.org].
Jun 24 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, 24 June 2015 at 16:39:22 UTC, kink wrote:
 On Wednesday, 24 June 2015 at 11:19:04 UTC, Jonathan M Davis 
 wrote:
 [...]

 3. Add a new attribute which does what's being proposed for 
 auto ref for non-templated functions, in which case, we can 
 use the non-templated behavior with templates as well and thus 
 avoid template bloat when all you want is for your templated 
 function to accept both lvalues and rvalues. auto ref, of 
 course, then stays exactly as it is now.

 At the moment, it seems that #2 is the most likely, and that's 
 probably fine, but I do wonder if we'd be better off with #3, 
 especially when you consider how much D code tends to be 
 templated and how much code bloat auto ref is likely to 
 generate with templated functions.

 - Jonathan M Davis
If that wasn't clear before, I'm all for #3 too. Just call it `scope ref` and simplify the PR a lil' bit as suggested by Marc in an earlier post [http://forum.dlang.org/post/ricvtchihgzyisbkzcgl forum.dlang.org].
But this has _nothing_ to do with scope, and scope ref was already rejected. The whole point of this is support having a function accept both rvalues and lvalues, not to do anything with scope. And given that what scope does has never even been properly defined - all that the spec says about scope parameters is "references in the parameter cannot be escaped (e.g. assigned to a global variable)" - and that the only place that scope does anything at all is with delegates, trying to expand it with "scope ref" as if that were simply an extension of scope makes no sense. Before we can even consider what something like scope ref might mean, we'd have to properly define what scope means. And all we have for it is the basic idea of what it's supposed to do - none of the details - and trying to define scope ref before defining what scope means in general could totally hamstring us when properly defining scope later. - Jonathan M Davis
Jun 24 2015
next sibling parent reply "kink" <noone nowhere.com> writes:
On Wednesday, 24 June 2015 at 23:30:53 UTC, Jonathan M Davis 
wrote:
 But this has _nothing_ to do with scope, and scope ref was 
 already rejected. The whole point of this is support having a 
 function accept both rvalues and lvalues, not to do anything 
 with scope.

 And given that what scope does has never even been properly 
 defined - all that the spec says about scope parameters is 
 "references in the parameter cannot be escaped (e.g. assigned 
 to a global variable)".
Yeah right. And all we need to safely pass in rvalue references is exactly the constraint that they won't escape. And scope alone doesn't imply pass-by-ref (`in` params aren't currently passed by ref).
 [...] Before we can even consider what something like scope ref 
 might mean, we'd have to properly define what scope means. And 
 all we have for it is the basic idea of what it's supposed to 
 do - none of the details - and trying to define scope ref 
 before defining what scope means in general could totally 
 hamstring us when properly defining scope later.
Is there a roadmap for "later"? It seems like these things always just get postponed, further postponed and never really done. What about the original argument 2 years ago, when rejecting DIP36, where Andrei explained that the idea was to make `ref` itself not escapable and delegating escaping refs to pointers? I don't see how that could be implemented without a breaking change, so I guess that may be something for D3. But we have been needing a solution for rvalue refs for years, and, as you know, are waiting since then, as `auto ref` for templates alone is simply not enough. To quote Manu from the DIP36 thread:
 It is, without doubt, the single biggest complaint I've heard
 by virtually every programmer I've introduced D to.
+1 kazillion
Jun 25 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, 25 June 2015 at 08:04:09 UTC, kink wrote:
 On Wednesday, 24 June 2015 at 23:30:53 UTC, Jonathan M Davis 
 wrote:
 But this has _nothing_ to do with scope, and scope ref was 
 already rejected. The whole point of this is support having a 
 function accept both rvalues and lvalues, not to do anything 
 with scope.

 And given that what scope does has never even been properly 
 defined - all that the spec says about scope parameters is 
 "references in the parameter cannot be escaped (e.g. assigned 
 to a global variable)".
Yeah right. And all we need to safely pass in rvalue references is exactly the constraint that they won't escape. And scope alone doesn't imply pass-by-ref (`in` params aren't currently passed by ref).
Whether a reference escapes is an orthogonal issue. The return attribute is for dealing with that. The function should be able to return by ref or not and still accept both rvalues and lvalues for its parameters. As long as the temporary variable created for holding an rvalue exists for the duration of the statement that the function call is in, it doesn't matter. It should be perfectly fine to have the function return one of its ref arguments by ref (and thus avoid a copy), even if it was originally an rvalue, because it have been assigned to an lvalue that would be around long enough. The only issue arises when the caller then returns the return value by ref, in which case that's going to be illegal without the return attribute. But in general, the fact that a ref to an argument could be returned by ref is orthogonal to being able to pass in both lvalues and rvalues, because the rvalues all have to become lvalues to be passed in anyway, in which case, you're in exactly the same place with regards to safety that you are with normal ref.
 [...] Before we can even consider what something like scope 
 ref might mean, we'd have to properly define what scope means. 
 And all we have for it is the basic idea of what it's supposed 
 to do - none of the details - and trying to define scope ref 
 before defining what scope means in general could totally 
 hamstring us when properly defining scope later.
Is there a roadmap for "later"? It seems like these things always just get postponed, further postponed and never really done.
Perhaps so, but just because scope hasn't properly defined yet isn't a reason to use it for something else entirely right now - especially when whether the lvalue that gets passed into the function is returned by ref is an orthogonal issue to having a type of ref which accepts an rvalue and silently converts it to an lvalue so that it can be passed by ref. - Jonathan M Davis
Jun 25 2015
parent reply "kinke" <noone nowhere.com> writes:
On Thursday, 25 June 2015 at 10:10:42 UTC, Jonathan M Davis wrote:
 Whether a reference escapes is an orthogonal issue. The return 
 attribute is for dealing with that. The function should be able 
 to return by ref or not and still accept both rvalues and 
 lvalues for its parameters. As long as the temporary variable 
 created for holding an rvalue exists for the duration of the 
 statement that the function call is in, it doesn't matter.
When I was talking about escaping references, I didn't mean returned references; I meant storing a pointer to a ref parameter somewhere outside (global/instance variable). This snippet is a rough sketch for an approach based on `scope ref`, to prevent escaping rvalues: <CODE> struct S { int a; ref S chain() return { return this; } // returns a `scope ref` for rvalues, `ref` for lvalues } // pointers derived from `scope ref` params can safely be passed down to scope pointer params only void manipulateWithoutEscaping(scope ref S s) { manipulate(&s, S.sizeof); } // doesn't escape any part of its `s` reference, hence `scope ref` // analog to S.chain(), returns a `scope ref` for rvalues, `ref` for lvalues ref S forward(return scope ref S s) { return s; } // low-level manipulation functions not escaping any parts of their pointees void manipulate(scope void* ptr, size_t size) { ptr[0..size] = 0; } void manipulate(scope void[] data) { manipulate(data.ptr, data.length); } struct State { S* s; // external ownership; should most likely actually be shared ownership void[] data; // ditto void initByEscaping(ref S s) { this.s = &s; // obvious data = (cast(void*)&s.a)[0 .. int.sizeof]; // less obvious } void manipulate() { if (s) manipulate(data); } } void foo() { // forward rvalue ref and invoke chain() on it twice (each call forwarding the rvalue ref) // rvalue is destructed at the end of the statement forward(S(1)).chain().chain(); // forward rvalue ref and let it be manipulated before destruction manipulateWithoutEscaping(forward(S(2))); // promote forwarded rvalue ref to lvalue S s3 = forward(S(3)); // let the lvalue escape and manipulate it indirectly auto state = new State(); state.initByEscaping(forward(s3)); // requires lvalue `ref`, forward() returns that for lvalue arg state.manipulate(); // GC-managed `state` outlives `s3` => dangling pointers `state.s` and `state.data.ptr`! } </CODE> The first observation is that a `scope ref` system would probably need to introduce scope pointers/slices too, to allow taking the address of `scope ref` params. There'd probably be quite a few `scope` usages, cluttering code and introducing additional complexity. I do see a point in trying to let `ref` alone and the compiler work it out. After all, that scope system would only allow us to prevent ourselves from escaping rvalues (if we cared to type 'scope'). With escape analysis, the compiler could disallow attempts to bind rvalues to escapable references, by inferring scope-ness for ref and pointer params automatically. In the future, escaping params could be disallowed in safe functions, and otherwise require an explicit `ref` keyword when supplying the lvalue argument, hinting at the potential for dangling pointers. In the meantime, what about making all `ref` params accept rvalues via lowering (at least for system)? As a first step until escape analysis is sufficiently implemented and the compiler will help us out with compile errors for escaping rvalues..
Jun 26 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Saturday, 27 June 2015 at 00:13:08 UTC, kinke wrote:
 On Thursday, 25 June 2015 at 10:10:42 UTC, Jonathan M Davis 
 wrote:
 Whether a reference escapes is an orthogonal issue. The return 
 attribute is for dealing with that. The function should be 
 able to return by ref or not and still accept both rvalues and 
 lvalues for its parameters. As long as the temporary variable 
 created for holding an rvalue exists for the duration of the 
 statement that the function call is in, it doesn't matter.
When I was talking about escaping references, I didn't mean returned references; I meant storing a pointer to a ref parameter somewhere outside (global/instance variable).
That's a completely orthogonal issue to ref or auto ref. That's an system operation (since taking the address of a local variable is system), and it's up to you to not screw it up. scope might cover that if it were fully ironed out, since that does involve escaping, but it also might not, since it's an system operation and thus up to you not to screw it up. Regardless, it has nothing to do with whether the function accepts lvalues and rvalues with the same parameter. You have a safety issue regardless if you're talking about taking a pointer to an argument (or to some portion of the argument), since there's no guarantee that the lifetime of the pointer is shorter than that of the argument. That depends entirely on what the argument was or what you do with the pointer, and very few variables have a global lifetime, so whether the argument passed to the ref parameter referred to a local variable is pretty much irrelevant. You have to know what you're doing with you take the address of a variable and make sure that you do it right. Safety with regards to ref is an entirely different issue, because that's safe code. Also, whether the function might return its ref parameter by ref is orthogonal to whether it accepted an rvalue by ref or not, since whether you want to do that or not doesn't necessarily have anything to with what's being returned from the function (rather, it has to do with what you want the function to accept), and you have a safety issue even without having any kind of ref which accepts rvalues, because all it requires is multiple layers of functions which return by ref, and you're screwed anyway. e.g. ref int id(ref int i) { return i; } ref int foo(ref int i) { ++i; return id(i); } ref int bar() { int i; return foo(i); } We introduced the return attribute (currently only with -dip25) to fix this problem. So, with that, we eliminate the safety problem with ref itself, and we can safely have auto ref accept rvalues by having it assign them to a temporary variable first. The rules with returning by ref would then be the same with auto ref as ref and require the return attribute to return a parameter. And all of this is within safe code. Any issues with taking pointers to arguments is firmly in system territory and is thus completely orthogonal. Even if scope were to be implemented in a way that prevented it, it would still have nothing to do with whether the function accepted both rvalues and lvalues with the same parameter.
 In the meantime, what about making all `ref` params accept 
 rvalues
Please, please, no. ref indicates that your intention is to mutate the argument. Having it accept rvalues completely destroys that and makes it that much harder to understand what a function is supposed to do. By having a separate attribute such as auto ref which also accepts rvalues, we cleanly separate out the code which just wants to accept both rvalues and lvalues for efficiency purposes and the code that actually wants to take its argument by ref so that it can mutate it. - Jonathan M Davis
Jun 26 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 27 June 2015 at 01:18:19 UTC, Jonathan M Davis wrote:
 On Saturday, 27 June 2015 at 00:13:08 UTC, kinke wrote:
 When I was talking about escaping references, I didn't mean 
 returned references; I meant storing a pointer to a ref 
 parameter somewhere outside (global/instance variable).
That's a completely orthogonal issue to ref or auto ref. That's an system operation (since taking the address of a local variable is system), and it's up to you to not screw it up. scope might cover that if it were fully ironed out, since that does involve escaping, but it also might not, since it's an system operation and thus up to you not to screw it up.
The point is that with scope, it _doesn't_ need to be system anymore. As an example where this can be useful, see: https://github.com/D-Programming-Language/phobos/blob/41d2027c22d6f8d22edd2df688c3cfd08fb20228/std/digest/hmac.d#L252 We currently can't make this function safe, although it is in fact memory safe.
 Regardless, it has nothing to do with whether the function 
 accepts lvalues and rvalues with the same parameter. You have a 
 safety issue regardless if you're talking about taking a 
 pointer to an argument (or to some portion of the argument), 
 since there's no guarantee that the lifetime of the pointer is 
 shorter than that of the argument.
No, not with scope. It seems you're using the status quo to argue against what could be. But that's not valid, because the problems you mention will no longer exist with scope.
 We introduced the return attribute (currently only with -dip25) 
 to fix this problem. So, with that, we eliminate the safety 
 problem with ref itself, and we can safely have auto ref accept 
 rvalues by having it assign them to a temporary variable first.
Yes this is now possible, but why in all world do you want the keyword that enables it to be `auto ref`?
 The rules with returning by ref would then be the same with 
 auto ref as ref and require the return attribute to return a 
 parameter. And all of this is within  safe code.

 Any issues with taking pointers to arguments is firmly in 
  system territory and is thus completely orthogonal. Even if 
 scope were to be implemented in a way that prevented it, it 
 would still have nothing to do with whether the function 
 accepted both rvalues and lvalues with the same parameter.

 In the meantime, what about making all `ref` params accept 
 rvalues
Please, please, no. ref indicates that your intention is to mutate the argument. Having it accept rvalues completely destroys that and makes it that much harder to understand what a function is supposed to do.
I can accept that argument, although I see cases where it's still justified. For example, if the author of the function decided it wants to mutate the argument, but you (the caller) are not interested in the new value. But still, explicitly using a temporary named `dummy` is preferable because it even documents that you're going to ignore it.
 By having a separate attribute such as auto ref which also 
 accepts rvalues, we cleanly separate out the code which just 
 wants to accept both rvalues and lvalues for efficiency 
 purposes and the code that actually wants to take its argument 
 by ref so that it can mutate it.
Okay, then you mainly want to use special syntax to opt-in to rvalue references. I simply think that `auto ref` is completely inadequate for this purpose, because its existing meaning with templates doesn't have anything to do with rvalue refs. `scope ref` on the other hand is more fitting IMO. It's not that `scope` necessarily implies rvalue refs, but rvalue refs _do_ imply scope! If we go with DIP25 where `ref` by itself already implies that, `scope ref` would then be redundant, and therefore the syntax would be available for the special purpose of accepting rvalue refs. This has multiple advantages: - No new keywords or attributes. - Rvalue refs are still opt-in, you don't get them accidentally just by writing `ref`. - It reinforces the guarantee that the argument will not escape. - It's usable in templates, too, with the same meaning as in normal functions.
Jun 27 2015
next sibling parent "kinke" <noone nowhere.com> writes:
On Saturday, 27 June 2015 at 11:10:49 UTC, Marc Schütz wrote:
 On Saturday, 27 June 2015 at 01:18:19 UTC, Jonathan M Davis 
 wrote:
 That's a completely orthogonal issue to ref or auto ref. 
 That's an  system operation (since taking the address of a 
 local variable is  system), and it's up to you to not screw it 
 up. scope might cover that if it were fully ironed out, since 
 that does involve escaping, but it also might not, since it's 
 an  system operation and thus up to you not to screw it up.
The point is that with scope, it _doesn't_ need to be system anymore.
 Regardless, it has nothing to do with whether the function 
 accepts lvalues and rvalues with the same parameter. You have 
 a safety issue regardless if you're talking about taking a 
 pointer to an argument (or to some portion of the argument), 
 since there's no guarantee that the lifetime of the pointer is 
 shorter than that of the argument.
No, not with scope. [...] the problems you mention will no longer exist with scope.
Exactly, that's the whole point. The relation with lvalue/rvalue reference parameters is that the compiler should be able to statically detect and refuse escaping rvalue bugs. Escaping lvalues with potential for dangling pointers should be system only, but preventing those bugs is a whole different beast. Still, to make (a) taking the address of ref params, locals & globals and (b) using pointer params safe, the compiler needs to know whether the parameter escapes or not. I tend to let the compiler figure that out in the future rather than having to use `scoped` a lot for ref and pointer params.
 We introduced the return attribute (currently only with 
 -dip25) to fix this problem. So, with that, we eliminate the 
 safety problem with ref itself, and we can safely have auto 
 ref accept rvalues by having it assign them to a temporary 
 variable first.
Yes this is now possible, but why in all world do you want the keyword that enables it to be `auto ref`?
Exactly. For a function returning a ref parameter, it doesn't matter whether it's an rvalue or lvalue. The important thing is to propagate that distinction (automatically by the compiler), to disallow binding a returned rvalue reference to an escaping ref parameter (i.e., no `escape(forward(S(1)))`). Where the distinction between lvalues and rvalues matters for a function returning a ref parameter, as for any other function with ref params, is whether the parameter escapes (i.e., by storing a pointer somewhere). If it does, it should be `return ref`, otherwise `return scope ref` (either explicitly or inferred automatically by the compiler).
 Any issues with taking pointers to arguments is firmly in 
  system territory and is thus completely orthogonal. Even if 
 scope were to be implemented in a way that prevented it, it 
 would still have nothing to do with whether the function 
 accepted both rvalues and lvalues with the same parameter.
Of course it would if one made actually use of a scope system (again, either explicit or as a result of compiler escape analysis) to (a) rule out all escaping rvalue bugs and (b) possibly warn about or highlight escaping lvalue params in system code. The main aim of the scope system would be to enhance the flexibility of safe code though.
 In the meantime, what about making all `ref` params accept 
 rvalues
Please, please, no. ref indicates that your intention is to mutate the argument. Having it accept rvalues completely destroys that and makes it that much harder to understand what a function is supposed to do.
If that was the only reason for not accepting rvalues as `ref` params, then rvalues should have been allowed for `const ref` right from the beginning, as there's surely absolutely no intention to mutate a `const ref` argument.
 I can accept that argument, although I see cases where it's 
 still justified. For example, if the author of the function 
 decided it wants to mutate the argument, but you (the caller) 
 are not interested in the new value. But still, explicitly 
 using a temporary named `dummy` is preferable because it even 
 documents that you're going to ignore it.
I find letting the callee decide whether the caller needs to provide an lvalue (`ref`) or not (non-templated `auto ref`), based solely on making sure the mutations are directly visible by the caller after the call, too much. I think it should be left to the caller to decide what mutations it's interested in. But now I see why some would oppose calling that `scope ref`, as that really has nothing to do with scope or safety, that'd just be an arbitrary way of classifying ref params in 'essential output' (mutable lvalue `ref`s, mutations need to be visible by caller) and some other category (lvalue/rvalue `auto ref`s, not just for efficiency purposes). That really is an orthogonal and incompatible semantic compared to what `scope ref` is all about, and doesn't interest me at all. I find flexible safe code and detecting escaping bugs a kazillion times more important than making sure a caller sees all mutations the callee deems essential after the call.
Jun 27 2015
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Saturday, 27 June 2015 at 11:10:49 UTC, Marc Schütz wrote:
 On Saturday, 27 June 2015 at 01:18:19 UTC, Jonathan M Davis 
 wrote:
 No, not with scope. It seems you're using the status quo to 
 argue against what could be. But that's not valid, because the 
 problems you mention will no longer exist with scope.
If we want to define what scope does and implement it, that's a separate issue from having a function accept both rvalues and lvalues with the same parameter. It would be trivial to have a function where you want to guarantee that no pointer to a ref parameter escapes but you still don't want to accept rvalues, because the whole point is to mutate the argument. Wanting to have a function accept lvalues and rvalues is thus fundamentally different from caring about issues with escaping. Yes, there is some overlap in terms of safety issues when it comes to returning by ref, but that's it, and the return attribute solves that problem, but whether you escape pointers to a parameter is fundamentally separate from whether a function should accept rvalues, because escaping is not the only issue, and ref itself has all of the same issues with escaping whether it accepts rvalues or not. So, it really doesn't matter what scope might protect against in terms of escaping and what it might not if we were to fully define it (e.g. maybe it would protect against taking the address of a variable, and maybe not; that could be simply considered system, or it could be protected against). We have to actually define it to know what it would or wouldn't do. All we have is the basic idea of what it's supposed to have been for. The _only_ thing that it applies to at this point is delegates. There's little point in arguing about what it could do until we're talking about actually defining it. But regardless, the issue of accepting rvalues by ref is orthogonal, because that is not solely an issue of escaping. It's also an issue of _why_ the argument is passed by ref in the first place.
 We introduced the return attribute (currently only with 
 -dip25) to fix this problem. So, with that, we eliminate the 
 safety problem with ref itself, and we can safely have auto 
 ref accept rvalues by having it assign them to a temporary 
 variable first.
Yes this is now possible, but why in all world do you want the keyword that enables it to be `auto ref`?
That's what auto ref was originally introduced to do. The reason that it works only with templates is because Walter misunderstood what Andrei was proposing. So, it makes sense to use it for what it was originally intended for and implement it as proposed with non-templated functions. Now, that has the downside that we can't use it with templated functions in a fashion that avoids template bloat, in which case, using a new attribute would make sense, but this _is_ what auto ref was originally for. And if we're going to use a new attribute, scope ref makes no sense precisely because the escaping issue is only part of the problem with accepting lvalues and rvalues with the same parameter, since you could easily have a function which is not supposed to escape the argument in any way, and yet it needs to be mutating the argument via ref for the caller, and it should not be accepting rvalues. So, in that case, if scope were fully ironed out, it might make sense to use scope ref when the intention was that the argument be mutated and not accept rvalues and that in not escape the function. It could also be that scope ref prevents returning by ref, since that's escaping the function, whereas when you pass an rvalue to the non-templated auto ref, you may very well want to return it by ref. So, if we were to use scope ref for this, we'd be mixing two largely orthogonal issues - escaping arguments and having a function accept both lvalues and rvalues - and it would cause us problems if and when we actually, fully defined scope. We need to either use auto ref for this as originally intended or use a new attribute which does so so that we can use it with both templated and non-templated functions. I think that it's quite clear that scope ref does _not_ make sense in this case, and it's already been rejected by Walter and Andrei anyway. - Jonathan M Davis
Jun 27 2015
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 28 June 2015 at 08:50, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 [..]
That's what auto ref was originally introduced to do. The reason that it works only with templates is because Walter misunderstood what Andrei was proposing. So, it makes sense to use it for what it was originally intended for and implement it as proposed with non-templated functions. Now, that has the downside that we can't use it with templated functions in a fashion that avoids template bloat,
Stop repeating that. It's not got anything to do with code bloat, you can't use it because template auto ref doesn't pass rvalues by ref, it passes them by value, which means it's not a ref in any way, and it's not what the user wants. I can easily get the same behaviour by not-typing 'auto ref', which I happily do in all cases that I want something to be passed by value. But we're not talking about by-val, we're talking about ref. It's simple; I want to pass something by value, or I want it to pass by ref... there is no middle ground, it's an explicit binary statement about how the parameter shall be passed. I don't care about lvalues or rvalues. There is no case where I want the compiler to not do the thing I told it to. And if there were a hypothetical case - where ref-ness were dependent on some conditions, I'm confident I could easily express that with existing mechanisms in D. I don't need a weird compiler defined semantic to try and do something smart that turns out never to be what I want.
 in which case, using a new attribute would make
 sense, but this _is_ what auto ref was originally for. And if we're going to
 use a new attribute, scope ref makes no sense precisely because the escaping
 issue is only part of the problem with accepting lvalues and rvalues with
 the same parameter, since you could easily have a function which is not
 supposed to escape the argument in any way, and yet it needs to be mutating
 the argument via ref for the caller, and it should not be accepting rvalues.
 So, in that case, if scope were fully ironed out, it might make sense to use
 scope ref when the intention was that the argument be mutated and not accept
 rvalues and that in not escape the function. It could also be that scope ref
 prevents returning by ref, since that's escaping the function, whereas when
 you pass an rvalue to the non-templated auto ref, you may very well want to
 return it by ref. So, if we were to use scope ref for this, we'd be mixing
 two largely orthogonal issues - escaping arguments and having a function
 accept both lvalues and rvalues - and it would cause us problems if and when
 we actually, fully defined scope.

 We need to either use auto ref for this as originally intended or use a new
 attribute which does so so that we can use it with both templated and
 non-templated functions. I think that it's quite clear that scope ref does
 _not_ make sense in this case, and it's already been rejected by Walter and
 Andrei anyway.
scope ref is only related to the safety of passing temporaries by reference to functions that may retain a pointer, it has nothing to do with lvalues or rvalues; stack locals are equally unsafe right now as rvalues would be. const is the thing you're looking for that says "this isn't to be changed", such that it makes sense to pass an rvalue. scope was proposed as a means to address the "rvalues -> ref is unsafe, the callee might sequester a pointer somehow" argument. If we're not interested in scope (ie, escape analysis), then we're not interested in ref safety in general (proposed scope applies to a much wider set of cases than just rvalues). That's fine, I don't really care, but then you can no longer make the argument above that safety affects this debate, at which point you arrive at C++ where rvalues can pass to const ref, as should be the case here. End this auto-ref nonsense, const ref is enough.
Jun 27 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Sunday, 28 June 2015 at 05:46:11 UTC, Manu wrote:
 On 28 June 2015 at 08:50, Jonathan M Davis via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 [..]
That's what auto ref was originally introduced to do. The reason that it works only with templates is because Walter misunderstood what Andrei was proposing. So, it makes sense to use it for what it was originally intended for and implement it as proposed with non-templated functions. Now, that has the downside that we can't use it with templated functions in a fashion that avoids template bloat,
Stop repeating that. It's not got anything to do with code bloat, you can't use it because template auto ref doesn't pass rvalues by ref, it passes them by value, which means it's not a ref in any way, and it's not what the user wants. I can easily get the same behaviour by not-typing 'auto ref', which I happily do in all cases that I want something to be passed by value. But we're not talking about by-val, we're talking about ref. It's simple; I want to pass something by value, or I want it to pass by ref... there is no middle ground, it's an explicit binary statement about how the parameter shall be passed. I don't care about lvalues or rvalues. There is no case where I want the compiler to not do the thing I told it to. And if there were a hypothetical case - where ref-ness were dependent on some conditions, I'm confident I could easily express that with existing mechanisms in D. I don't need a weird compiler defined semantic to try and do something smart that turns out never to be what I want.
It makes no sense to pass rvalues by ref. The ref has to refer to a memory location so that a pointer can be passed underneath the hood, and rvalues don't qualify for that. A temporary variable _must_ be created in order to pass an rvalue by ref, even if the compiler is able to move the rvalue into the temporary variable rather than copy it. So, I don't understand why you would be looking to pass rvalues by ref. I can see why you would want to pass both rvalues and lvalues to the same function efficiently without caring about whether they're rvalues or lvalues, but that doesn't necessarily mean ref, and in the case of rvalues, it definitely doesn't mean ref. The most efficient way to pass rvalues is by value, because the value can be moved or even simply constructed directly into the right place on the stack to begin with. That's why C++11 added move constructors and why D has postlblit constructors. That's also why it's no longer the advice in C++-land for folks to simply always use const& as much as possible (not that it shouldn't be used, but whether it should be used or not becomes much more complicated). D has moving built-in without the need for move constructors, so our situation is simpler, but regardless, ref really isn't the answer when it comes to rvalues. You _want_ to be passing rvalues by value. It's the lvalues that you don't want to pass by value if you're worried about efficiency, because they're the ones that are going to have to be copied. But still, if what you want is to have rvalues put into a place on the stack where they can be passed by ref and then have them passed by ref, that's what you'll get with auto ref for non-templated functions. It'll pass lvalues by ref, and it'll create temporary variables for the rvalues so that they can be passed by ref. So, as far as I can tell, auto ref for non-templated functions does exactly what you're looking for. It's just that it only works for non-templated functions, and with the templated ones, rvalues are passed efficiently - but not by ref. - Jonathan M Davis
Jun 28 2015
next sibling parent "Temtaime" <temtaime gmail.com> writes:
All values must be passed by ref because otherwise it will be 
leading for code bloat : we want eliminate by having only one 
function that accepts both rvalues and lvalues.
Jun 28 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 28 June 2015 at 07:36:41 UTC, Jonathan M Davis wrote:
 It makes no sense to pass rvalues by ref. The ref has to refer 
 to a memory location so that a pointer can be passed underneath 
 the hood, and rvalues don't qualify for that.
I strongly disagree with that. This is an implementation detail. `ref` describes a specific semantic, namely that no copying (observable by a postblit being called) takes place, and that it refers to the same "thing" as the original (observable by mutations being visible on the original object, if relevant). How a value is stored is completely up to the compiler. It can be in a register, a memory location, or even - if the compiler knows the value in advance - as an immediate of a machine instruction. The same is true for passing values: the compiler is free to implement it as it wants, as long as it keeps the required semantics.
Jun 28 2015
parent reply "rsw0x" <anonymous anonymous.com> writes:
On Sunday, 28 June 2015 at 11:06:01 UTC, Marc Schütz wrote:
 On Sunday, 28 June 2015 at 07:36:41 UTC, Jonathan M Davis wrote:
 It makes no sense to pass rvalues by ref. The ref has to refer 
 to a memory location so that a pointer can be passed 
 underneath the hood, and rvalues don't qualify for that.
I strongly disagree with that. This is an implementation detail. `ref` describes a specific semantic, namely that no copying (observable by a postblit being called) takes place,
no, it means pass by reference. this sounds like some convoluted C++ doublespeak definition.
Jun 28 2015
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 28 June 2015 at 13:08:48 UTC, rsw0x wrote:
 On Sunday, 28 June 2015 at 11:06:01 UTC, Marc Schütz wrote:
 On Sunday, 28 June 2015 at 07:36:41 UTC, Jonathan M Davis 
 wrote:
 It makes no sense to pass rvalues by ref. The ref has to 
 refer to a memory location so that a pointer can be passed 
 underneath the hood, and rvalues don't qualify for that.
I strongly disagree with that. This is an implementation detail. `ref` describes a specific semantic, namely that no copying (observable by a postblit being called) takes place,
no, it means pass by reference. this sounds like some convoluted C++ doublespeak definition.
Well, try experimenting with LDC with -O3 -output-s on some test programs. You will be surprised ;-) There will often be no "reference" in the resulting output, especially with inlining. The definition I used is not necessarily the best one, but it needs to be based on observable behaviour, not implementation specifics, otherwise compilers would have no freedom in optimizing.
Jun 28 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
Thank you Jonathan, I think I (like kinke in his post above) 
finally understand your stance. So I guess from your POV the 
following would be an ideal situation (disregarding the need to 
avoid breaking changes):

1) `auto ref` for non-templates is required to make a function 
accept rvalue and lvalue refs alike.
2) `auto ref` for templates is changed to mean the same thing as 
1).
3) `scope ref` prevents escaping of the reference.

Neither of `auto ref` and `scope ref` would imply the other, 
because you want to treat them as orthogonal. Do I understand you 
right?

The main problems with this is that they aren't in fact 
orthogonal: `auto ref` without `scope ref` almost only makes 
sense if you want to deliberately break something. Furthermore, 
`auto ref` is already taken for something else, and I'm not sure 
Walter would be too happy if we wanted to change its meaning for 
templates.
Jun 28 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Sunday, 28 June 2015 at 10:50:10 UTC, Marc Schütz wrote:
 Thank you Jonathan, I think I (like kinke in his post above) 
 finally understand your stance. So I guess from your POV the 
 following would be an ideal situation (disregarding the need to 
 avoid breaking changes):

 1) `auto ref` for non-templates is required to make a function 
 accept rvalue and lvalue refs alike.
 2) `auto ref` for templates is changed to mean the same thing 
 as 1).
 3) `scope ref` prevents escaping of the reference.

 Neither of `auto ref` and `scope ref` would imply the other, 
 because you want to treat them as orthogonal. Do I understand 
 you right?

 The main problems with this is that they aren't in fact 
 orthogonal: `auto ref` without `scope ref` almost only makes 
 sense if you want to deliberately break something. Furthermore, 
 `auto ref` is already taken for something else, and I'm not 
 sure Walter would be too happy if we wanted to change its 
 meaning for templates.
We definitely don't want to lose the current auto ref that we have with templates. It's critical for forwarding refness, even if we got that ability by accident. IIRC, we don't quite manage perfect forwarding with what we have (though off the top of my head, I don't recall what we're missing), but without auto ref, we definitely don't get there. So, auto ref, as it stands has proven to be surprisingly useful completely aside from efficiency concerns. Also, the fact that it takes rvalues by value can actually be more efficient than taking them by ref, depending on the code, since it can just move the value in that case, whereas it's tied to a temporary variable in the case of what would be the non-templated auto ref, and that could require copying the value at some point when having it be passed by value could have avoided the copy - though obviously, all of that template bloat comes at a cost, so it's definitely a tradeoff. But we don't want to lose the current auto ref regardless. So, if we want to have an attribute which accepts both lvalues and rvalues for non-templated functions and be able to use that same attribute for templated functions, a new attribute is required (even if originally, auto ref was intended to work for both templated and non-templated functions). Otherwise, if we're fine with the template bloat, we can just use the proposed auto ref with non-templated functions and not worry about adding a new attribute. So, ideally, at this point, we'd probably add a new attribute, simply because it's the most flexible, but that does come at the cost of adding a new attribute, so it may not be worth it. Regardless, all of that is orthogonal to the escaping issue, because the escaping issue exists for all ref variables (not just those which refer to temporary variables that rvalues were assigned to). So, whether a ref parameter accepts both lvalues and rvalues is separate from whether they escape. And if scope really prevented the ref from escaping, then you couldn't even return it, and we need to be able to return by ref, so scope ref really doesn't make sense in the context of preventing ref parameters from existing longer than their arguments. It's related, but it would be too restrictive if it truly prevented all escaping. And as far as escaping references to ref parameters by returning them by ref goes, the recently added return attribute already takes care of that. So, arguably, it overlaps with what scope conceivably could or should do, but it does what's required for solving the escape problem for ref, and a more general scope solution is not required for that. It also allows us to distinguish between ensuring that a ref parameter does not exist longer than is safe (i.e. that it's only returned by ref if it's safe to do so) and preventing a ref parameter from escaping the function at all. So, if we were to ever add scope ref, then presumably that would have to do with other types of escaping than simply returning by ref. And whether you wanted to prevent a ref parameter from escaping - either by being returned by ref or by other means - isn't necessarily related to whether it refers to a temporary variable that was an rvalue, much as it's clearly more dangerous to escape anything which refers to a ref parameters if it refers to a temporary variable that won't last beyond the statement that the function was called in. - Jonathan M Davis
Jun 29 2015
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 29 June 2015 at 17:19:48 UTC, Jonathan M Davis wrote:
 On Sunday, 28 June 2015 at 10:50:10 UTC, Marc Schütz wrote:
 The main problems with this is that they aren't in fact 
 orthogonal: `auto ref` without `scope ref` almost only makes 
 sense if you want to deliberately break something. 
 Furthermore, `auto ref` is already taken for something else, 
 and I'm not sure Walter would be too happy if we wanted to 
 change its meaning for templates.
We definitely don't want to lose the current auto ref that we have with templates. It's critical for forwarding refness, even if we got that ability by accident. - snip -
For the record, to avoid further misunderstandings: When I talked about "auto ref" I meant the proposed rvalue refs, not the existing syntax for templates. I don't have such a strong opinion about the latter, except that it is - and should stay - distinct from rvalue refs.
 Regardless, all of that is orthogonal to the escaping issue, 
 because the escaping issue exists for all ref variables (not 
 just those which refer to temporary variables that rvalues were 
 assigned to). So, whether a ref parameter accepts both lvalues 
 and rvalues is separate from whether they escape. And if scope 
 really prevented the ref from escaping, then you couldn't even 
 return it, and we need to be able to return by ref, so scope 
 ref really doesn't make sense in the context of preventing ref 
 parameters from existing longer than their arguments. It's 
 related, but it would be too restrictive if it truly prevented 
 all escaping.
It's not as strict as that. Returning an rvalue ref is not per se unsafe, because the temporary that it refers to lives until the end of the current statement, which can be longer than the current call. It can be safely passed to another function as an rvalue ref, e.g. in a UFCS chain. This is what the `return` attribute enables: it's a means of telling the callee to losen the restrictions on a scoped reference a bit, because the caller is aware of it and can handle it in a safe way.
 And as far as escaping references to ref parameters by 
 returning them by ref goes, the recently added return attribute 
 already takes care of that. So, arguably, it overlaps with what 
 scope conceivably could or should do, but it does what's 
 required for solving the escape problem for ref, and a more 
 general scope solution is not required for that. It also allows 
 us to distinguish between ensuring that a ref parameter does 
 not exist longer than is safe (i.e. that it's only returned by 
 ref if it's safe to do so) and preventing a ref parameter from 
 escaping the function at all. So, if we were to ever add scope 
 ref, then presumably that would have to do with other types of 
 escaping than simply returning by ref. And whether you wanted 
 to prevent a ref parameter from escaping - either by being 
 returned by ref or by other means - isn't necessarily related 
 to whether it refers to a temporary variable that was an 
 rvalue, much as it's clearly more dangerous to escape anything 
 which refers to a ref parameters if it refers to a temporary 
 variable that won't last beyond the statement that the function 
 was called in.
I already addressed this point in the last section of http://forum.dlang.org/post/udnrkqmfqeunsfgwscnk forum.dlang.org
Jun 29 2015
prev sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Monday, 29 June 2015 at 17:19:48 UTC, Jonathan M Davis wrote:
 On Sunday, 28 June 2015 at 10:50:10 UTC, Marc Schütz wrote:
 Thank you Jonathan, I think I (like kinke in his post above) 
 finally understand your stance. So I guess from your POV the 
 following would be an ideal situation (disregarding the need 
 to avoid breaking changes):

 1) `auto ref` for non-templates is required to make a function 
 accept rvalue and lvalue refs alike.
 2) `auto ref` for templates is changed to mean the same thing 
 as 1).
 3) `scope ref` prevents escaping of the reference.

 Neither of `auto ref` and `scope ref` would imply the other, 
 because you want to treat them as orthogonal. Do I understand 
 you right?

 The main problems with this is that they aren't in fact 
 orthogonal: `auto ref` without `scope ref` almost only makes 
 sense if you want to deliberately break something. 
 Furthermore, `auto ref` is already taken for something else, 
 and I'm not sure Walter would be too happy if we wanted to 
 change its meaning for templates.
We definitely don't want to lose the current auto ref that we have with templates. It's critical for forwarding refness, even if we got that ability by accident. IIRC, we don't quite manage perfect forwarding with what we have (though off the top of my head, I don't recall what we're missing), but without auto ref, we definitely don't get there. So, auto ref, as it stands has proven to be surprisingly useful completely aside from efficiency concerns.
I thought perfect forwarding had everything to do with efficiency concerns. At least, that's the motivation for in in C++ AFAICT/AFAIR. If not, (since C++ const& can bind to rvalues), passing by value would be perfectly reasonable. It seems to me that there's a lot of confusion in this thread. I think I understand what you're saying but I'm not sure everyone else does. So correct me if I'm wrong: your stance (which is how I understand how things should be done in D) is that if a person wants to bind an rvalue to a ref parameter for performance, they shouldn't; they should pass it by value since it'll either get constructed in place or moved, not copied. Essentially, what this guy said about C++ (not the original I read, that's now a 404): http://m.blog.csdn.net/blog/CPP_CHEN/17528669 The reason rvalue references even exist in C++11/14 is because rvalues can bind to const&. I remember hearing/reading Andrei say that D doesn't need them precisely because in D, they can't. Pass by value and you get the C++ behaviour when rvalue references are used: a move. Atila
Jun 29 2015
next sibling parent "kink" <noone nowhere.com> writes:
On Monday, 29 June 2015 at 19:10:07 UTC, Atila Neves wrote:
 It seems to me that there's a lot of confusion in this thread. 
 I think I understand what you're saying but I'm not sure 
 everyone else does. So correct me if I'm wrong: your stance 
 (which is how I understand how things should be done in D) is 
 that if a person wants to bind an rvalue to a ref parameter for 
 performance, they shouldn't; they should pass it by value since 
 it'll either get constructed in place or moved, not copied.

 Essentially, what this guy said about C++ (not the original I 
 read, that's now a 404):

 http://m.blog.csdn.net/blog/CPP_CHEN/17528669

 The reason rvalue references even exist in C++11/14 is because 
 rvalues can bind to const&. I remember hearing/reading Andrei 
 say that D doesn't need them precisely because in D, they 
 can't. Pass by value and you get the C++ behaviour when rvalue 
 references are used: a move.
In some cases even better, in-place construction. That's perfect - for rvalues, as you said. But for lvalues, a copy must be made - afaik, you don't have the option to explicitly move an lvalue into a byval-param in D as you can in C++. And even moving can be expensive if the type is big. So for efficiency, template auto-ref is a nice option, but requires a template and leads to code-bloat if both lvalues and rvalues are used as arguments. For these reasons, we want to have the additional option to pass both lvalues and rvalues by ref. Jonathan wants a non-templated function to specify explicitly whether it wants to accept an rvalue by ref, apparently to discriminate between 'output parameters' (mutable `ref`, lvalues only) and another category of `auto ref` parameters which also accept rvalues via lowering (I don't know how one would describe that category - to me it seems completely arbitrary and adds complexity, not to mention different interpretations by different developers). I'm solely interested in the safety of binding rvalues to refs, meaning that the compiler should prevent these rvalue refs from being escaped (I'm not talking about returning the ref via `return ref`, which is safe anyway when propagating the rvalue/lvalue origin, but storing derived pointers in external state). That's where `scope ref` (unescapable reference) comes in. But I prefer the compiler to infer scope-ness automatically in the future and would thus like to allow binding rvalues to `ref` in general - for not-so-performance-critical functions taking a value type by reference (for some value types, copying just doesn't make sense - streams, loggers etc.).
Jun 30 2015
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, 29 June 2015 at 19:10:07 UTC, Atila Neves wrote:
 On Monday, 29 June 2015 at 17:19:48 UTC, Jonathan M Davis wrote:
 On Sunday, 28 June 2015 at 10:50:10 UTC, Marc Schütz wrote:
 Thank you Jonathan, I think I (like kinke in his post above) 
 finally understand your stance. So I guess from your POV the 
 following would be an ideal situation (disregarding the need 
 to avoid breaking changes):

 1) `auto ref` for non-templates is required to make a 
 function accept rvalue and lvalue refs alike.
 2) `auto ref` for templates is changed to mean the same thing 
 as 1).
 3) `scope ref` prevents escaping of the reference.

 Neither of `auto ref` and `scope ref` would imply the other, 
 because you want to treat them as orthogonal. Do I understand 
 you right?

 The main problems with this is that they aren't in fact 
 orthogonal: `auto ref` without `scope ref` almost only makes 
 sense if you want to deliberately break something. 
 Furthermore, `auto ref` is already taken for something else, 
 and I'm not sure Walter would be too happy if we wanted to 
 change its meaning for templates.
We definitely don't want to lose the current auto ref that we have with templates. It's critical for forwarding refness, even if we got that ability by accident. IIRC, we don't quite manage perfect forwarding with what we have (though off the top of my head, I don't recall what we're missing), but without auto ref, we definitely don't get there. So, auto ref, as it stands has proven to be surprisingly useful completely aside from efficiency concerns.
I thought perfect forwarding had everything to do with efficiency concerns. At least, that's the motivation for in in C++ AFAICT/AFAIR. If not, (since C++ const& can bind to rvalues), passing by value would be perfectly reasonable.
I don't remember. It's been a while since I looked at it, so I'd have to look it up again to know quite what it is or exactly why it's needed. I do remember that without auto ref though, we're definitely not there.
 It seems to me that there's a lot of confusion in this thread. 
 I think I understand what you're saying but I'm not sure 
 everyone else does. So correct me if I'm wrong: your stance 
 (which is how I understand how things should be done in D) is 
 that if a person wants to bind an rvalue to a ref parameter for 
 performance, they shouldn't; they should pass it by value since 
 it'll either get constructed in place or moved, not copied.

 Essentially, what this guy said about C++ (not the original I 
 read, that's now a 404):

 http://m.blog.csdn.net/blog/CPP_CHEN/17528669

 The reason rvalue references even exist in C++11/14 is because 
 rvalues can bind to const&. I remember hearing/reading Andrei 
 say that D doesn't need them precisely because in D, they 
 can't. Pass by value and you get the C++ behaviour when rvalue 
 references are used: a move.
Yeah. In general, it's more efficient to pass rvalues by value, because the compiler can avoid copying them and simply move them (and at least some of the time, it doesn't even need to move them, though I don't know enough about how stuff is implemented at that level to know whether it can always avoid even having to move rvalue arguments; it can certainly avoid copying them though). If you pass in an rvalue by reference (via const& in C++ or by the proposed non-templated auto ref in D), then you actually increase the odds that the argument will have to be copied. And if no copy is needed anywhere, then I believe that passing an rvalue by ref and passing it by value end up being more or less equivalent. So, the end result is that it's not actually beneficial to pass an rvalue by reference rather than by value. The problem that you run into is that if you want a parameter to accept both lvalues and rvalues, then the more efficient thing for rvalues is to pass by value, whereas the more efficient thing to do for lvalues is to pass by reference, and const& in C++ forces you to pass by ref, whereas passing by value... forces passing by value. The same would go with the non-templated auto ref. However, the templated auto ref avoids that by inferring the refness of the argument and thus automatically accepts rvalues by value and lvalues by reference. So, you end up passing arguments efficiently regardless of whether they're lvalues or rvalues; it's just that it costs you a lot of template bloat in the process. But a lot of programmers are used to simply using const& everywhere in C++, because prior to C++11, rvalues couldn't generally be moved, because there were no move constructors, and C++ doesn't disallow stuff like having a class/struct on the stack having a pointer to one of its members (which D does disallow - or at least assumes you won't do), and so the best way to avoid extra copies was to ensure that you passed by const& as much as possible. But with C++11, the situation becomes much more complicated, and figuring out what the most efficient thing to do is not as straightforward. And AFAIK, C++ has nothing like our templated auto ref that allows you to efficiently accept both lvalues and rvalues (but maybe that's where perfect forwarding comes in; I don't know; I really need to read up on it), leaving you to figure out whether having a function take its arguments by value or const& is more efficient based on what your function does. Ultimately, it really doesn't seem like there's a simple rule of them that is going to give you efficiency in general. You need to understand the various pros and cons of how arguments are passed if you want to eke out extra efficiency. Though in general, if you simply avoid having particularly large structs on the stack, I would think that just passing arguments by value as the default would be the best way to go and worrying about how to pass more efficiently when profiling shows that it's necessary. But having auto ref for non-templated functions will give us another tool in the toolbox for adjusting how function arguments are passed in code that cares. Of course, if you have large structs for whatever reason, that makes the situation more complicated, but the best way to handle that will likely depend on your code. The only real question I see is whether it's worth using a new attribute (as opposed to auto ref) so that we can use it with templated functions as well, allowing folks to pass rvalues by reference with templated functions if they need to avoid the template bloat or it actually turns out that that's more efficient in that case. As I understand it though, it's pretty much always more efficient to pass rvalues by value. And if that's indeed the case, the only real gain that I see by using a new attribute is that it allows us to avoid template bloat when it needs to be avoided. - Jonathan M Davis
Jun 30 2015
parent "Daniel N" <ufo orbiting.us> writes:
This is semi-off-topic, but I was playing around... and line 12 
broke my personal record for the most insane line ever written, 
maybe someone else finds it amusing also:

[asm.dlang.org]
http://goo.gl/ChdbwD
Jun 30 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Wednesday, 24 June 2015 at 23:30:53 UTC, Jonathan M Davis 
wrote:
 On Wednesday, 24 June 2015 at 16:39:22 UTC, kink wrote:
 On Wednesday, 24 June 2015 at 11:19:04 UTC, Jonathan M Davis 
 wrote:
 [...]

 3. Add a new attribute which does what's being proposed for 
 auto ref for non-templated functions, in which case, we can 
 use the non-templated behavior with templates as well and 
 thus avoid template bloat when all you want is for your 
 templated function to accept both lvalues and rvalues. auto 
 ref, of course, then stays exactly as it is now.

 At the moment, it seems that #2 is the most likely, and 
 that's probably fine, but I do wonder if we'd be better off 
 with #3, especially when you consider how much D code tends 
 to be templated and how much code bloat auto ref is likely to 
 generate with templated functions.

 - Jonathan M Davis
If that wasn't clear before, I'm all for #3 too. Just call it `scope ref` and simplify the PR a lil' bit as suggested by Marc in an earlier post [http://forum.dlang.org/post/ricvtchihgzyisbkzcgl forum.dlang.org].
But this has _nothing_ to do with scope, and scope ref was already rejected. The whole point of this is support having a function accept both rvalues and lvalues, not to do anything with scope.
It everything to do with scope, and I already explained it several times. As for the rejection, see my answer to Namespace: http://forum.dlang.org/post/yxruhtwjllozgvpbhgik forum.dlang.org
 And given that what scope does has never even been properly 
 defined - all that the spec says about scope parameters is 
 "references in the parameter cannot be escaped (e.g. assigned 
 to a global variable)"
Even this current vague definition is already enough of a guarantee to allow rvalue references with it. Of course I'd be the last to object against a more detailed specification, obviously.
 - and that the only place that scope does anything at all is 
 with delegates,
That's simply the status quo of the implementation and therefore can't be used as an argument.
 trying to expand it with "scope ref" as if that were simply an 
 extension of scope makes no sense. Before we can even consider 
 what something like scope ref might mean, we'd have to properly 
 define what scope means. And all we have for it is the basic 
 idea of what it's supposed to do - none of the details - and 
 trying to define scope ref before defining what scope means in 
 general could totally hamstring us when properly defining scope 
 later.
I can assure you that it will not limit us. The very concept of borrowing/scope already requires some very specific restrictions with regards to what a function is allowed to do with a scope parameter. These same restrictions guarantee that it's allowed to pass an rvalue to it. Every working scope proposal will always guarantee that, or it wouldn't be usable. If you still fear that it will impede us later, then at least this current proposal needs to be deferred until we have a scope proposal and have decided on it.
Jun 25 2015
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/25/2015 10:28 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
<schuetzm gmx.net>" wrote:
 trying to expand it with "scope ref" as if that were simply an
 extension of scope makes no sense. Before we can even consider what
 something like scope ref might mean, we'd have to properly define what
 scope means. And all we have for it is the basic idea of what it's
 supposed to do - none of the details - and trying to define scope ref
 before defining what scope means in general could totally hamstring us
 when properly defining scope later.
I can assure you that it will not limit us. The very concept of borrowing/scope already requires some very specific restrictions with regards to what a function is allowed to do with a scope parameter. These same restrictions guarantee that it's allowed to pass an rvalue to it. Every working scope proposal will always guarantee that, or it wouldn't be usable. If you still fear that it will impede us later, then at least this current proposal needs to be deferred until we have a scope proposal and have decided on it.
I think the arguments against allowing rvalues with scope ref are the same as those against allowing rvalues with ref, modulo accidental escaping. (It has been argued that 'ref' signifies an intent to modify, which I wouldn't necessarily agree with.)
Jun 25 2015
prev sibling next sibling parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Monday, 22 June 2015 at 04:11:41 UTC, Andrei Alexandrescu 
wrote:
 Walter and I discussed what auto ref for templates should look 
 like and reached the conclusion that an approach based on 
 lowering would be best. I added a proposed lowering to 
 https://github.com/D-Programming-Language/dmd/pull/4717.

 Andrei
It seems to me that this may be an important target to seek: auto foo(auto ref T bar) {} Should be callable in the same manner auto foo()(auto ref T bar) {} is callable. That is to say it would be unfortunate if turning it into a template would cause different behaviors. If nothing else, deviations should be well documented and explained.
Jun 22 2015
prev sibling parent "ixid" <nuaccount gmail.com> writes:
On Monday, 22 June 2015 at 04:11:41 UTC, Andrei Alexandrescu 
wrote:
 Walter and I discussed what auto ref for templates should look 
 like and reached the conclusion that an approach based on 
 lowering would be best. I added a proposed lowering to 
 https://github.com/D-Programming-Language/dmd/pull/4717.

 Andrei
A question from a place of ignorance- from the thread people say normal function and template function auto ref have different rules. This seems like a really bad idea. Is this necessary and why should this be acceptable? We should surely have normal functions be a simple subset of templates.
Jul 02 2015