www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Const ref and rvalues again...

reply "Malte Skarupke" <malteskarupke web.de> writes:
Hello,

I realize that this has been discussed before, but so far there 
is no solution and this really needs to be a high priority:

We need a way for a function to declare that it doesn't want it's 
argument to be copied, but it also doesn't care whether the 
argument is an rvalue or an lvalue.

The C++ way of doing this would be to declare the argument as a 
const &. Apparently it is not desired that we do the same thing 
for const ref.

Currently, if you want that behavior, you have to write 2^n 
permutations of your function, with n being the number of 
arguments that the function takes.

Here's my attempt at passing a struct to a function that takes 
three arguments without the struct being copied:

int copyCounter = 0;
struct CopyCounter
{
     this(this) { ++copyCounter; }
}
void takeThree(ref in CopyCounter a, ref in CopyCounter b, ref in 
CopyCounter c)
{
     writeln("took three");
}
void takeThree(in CopyCounter a, ref in CopyCounter b, ref in 
CopyCounter c)
{
     takeThree(a, b, c);
}
void takeThree(ref in CopyCounter a, in CopyCounter b, ref in 
CopyCounter c)
{
     takeThree(a, b, c);
}
void takeThree(ref in CopyCounter a, ref in CopyCounter b, in 
CopyCounter c)
{
     takeThree(a, b, c);
}
void takeThree(in CopyCounter a, in CopyCounter b, ref in 
CopyCounter c)
{
     takeThree(a, b, c);
}
void takeThree(in CopyCounter a, ref in CopyCounter b, in 
CopyCounter c)
{
     takeThree(a, b, c);
}
void takeThree(ref in CopyCounter a, in CopyCounter b, in 
CopyCounter c)
{
     takeThree(a, b, c);
}
void takeThree(in CopyCounter a, in CopyCounter b, in CopyCounter 
c)
{
     takeThree(a, b, c);
}
static CopyCounter createCopyCounter()
{
     return CopyCounter();
}
void main()
{
     CopyCounter first;
     CopyCounter second;
     CopyCounter third;
     takeThree(first, second, third);
     takeThree(createCopyCounter(), second, createCopCounter());
     assert(copyCounter == 0); // yay, works
}


My propsed solution is this:
- Make functions that take "ref in" arguments also accept rvalues.
- The user can still provide an overload that accepts an rvalue, 
using the "in" keyword, and that one will be preferred over the 
"ref in" version.


What do you think?

Malte
Oct 17 2012
next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Thursday, 18 October 2012 at 03:07:56 UTC, Malte Skarupke 
wrote:

 What do you think?

 Malte



I've asked for this before.
http://d.puremagic.com/issues/show_bug.cgi?id=8121


Doesn't seem to be a priority. :(
Oct 17 2012
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, October 18, 2012 05:07:52 Malte Skarupke wrote:

 My propsed solution is this:
 - Make functions that take "ref in" arguments also accept rvalues.


There was some discussion of this, though that seems to me to be basically the 
same as making const ref take rvalues only worse. Certainly, it seems 
inherently broken to me to have a function try and take its arguement by ref 
and then be given an rvalue.


 - The user can still provide an overload that accepts an rvalue,
 using the "in" keyword, and that one will be preferred over the
 "ref in" version.
 
 
 What do you think?


in as alias for const scope. scope is generally not applicable to structs (and 
when it is, it's not applied properly - scope currently only works properly 
for delegates), and const just puts us back at const ref again. You appear to 
be suggesting that in do something different here, and I don't think that that 
works.

The most recent, in-depth discussion on this issue can be found here: 
http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com

It wasn't entirely conclusive, but there _was_ some discussion on making ref 
take rvalues. I believe that more research was necessary, and I'm not sure 
where any of it stands now. One of the more promising ideas (IMHO) was to 
simply change it so that auto ref means what it was supposed to mean 
originally - that the programmer wants the argument to be passed efficiently
but 
doesn't care whether it's an lvalue or rvalue - but some people screamed about 
that, because they've found the current version of auto ref to be useful (in 
spite of the fact that it doesn't do what it was supposed to do).

Really, I think that this comes down to us needing to figure out how to make it 
so that ref can take lvalues under at least some set of circumstances (which 
seems like a bad idea to me), or we need to come up with a parameter attribute 
which indicates that we don't want the argument copied and don't care if it's 
an lvalue or rvalue (like const& is usually used for in C++ but making it so 
that it's _exactly_ that rather than just being primarily used for that). And 
if we can't use auto ref for that, then we need to come up with a new 
attribute of some kind to do that. However, IIRC Kenji tried to change auto 
ref to do that but ran into issues of some kind. I don't know where it stands 
now.

- Jonathan M Davis
Oct 17 2012
parent reply "jerro" <a a.com> writes:
On Thursday, 18 October 2012 at 03:31:41 UTC, Jonathan M Davis 
wrote:

 On Thursday, October 18, 2012 05:07:52 Malte Skarupke wrote:

 My propsed solution is this:
 - Make functions that take "ref in" arguments also accept 
 rvalues.


 There was some discussion of this, though that seems to me to 
 be basically the
 same as making const ref take rvalues only worse.


What would be the problem with const ref taking rvalues?
Oct 17 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, October 18, 2012 06:24:08 jerro wrote:

 What would be the problem with const ref taking rvalues?


Read the thread that I already linked to:

http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com

- Jonathan M Davis
Oct 17 2012
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 18 October 2012 at 04:30:17 UTC, Jonathan M Davis 
wrote:

 On Thursday, October 18, 2012 06:24:08 jerro wrote:

 What would be the problem with const ref taking rvalues?


 Read the thread that I already linked to:

 http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com

 - Jonathan M Davis


I read the thread, and not a single one of the "problematic 
cases" are actually valid C++.

Yes: the faulty MSVC has taught people to do retarded things, or 
be afraid of things that were illegal to begin with (in 
particular, pass an rvalue to a ref, WHICH IS ILLEGAL IN C++), 
such as "increment(5)".

There is actually nothing wrong with creating a temporary when 
something is bound to a const ref, provided the compiler follows 
the rules:

*Only LValues with an EXACT type match may be passed to a 
reference.
*In regards to *const* references, RValues may be copied in a 
temporary, and that temporary bound the the ref.

I'm not saying we particularly *need* this in D (C++ has a "by 
ref" paradigm that makes it more important, but D *rarelly* ever 
passes by const ref).

But if the compiler respects the above two rules (which it 
should), then RValue to const ref is both perfectly doable and 
safe (as safe as refs get anyways).
Oct 17 2012
next sibling parent "foobar" <foo bar.com> writes:
On Thursday, 18 October 2012 at 06:11:26 UTC, monarch_dodra wrote:

 On Thursday, 18 October 2012 at 04:30:17 UTC, Jonathan M Davis 
 wrote:

 On Thursday, October 18, 2012 06:24:08 jerro wrote:

 What would be the problem with const ref taking rvalues?


 Read the thread that I already linked to:

 http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com

 - Jonathan M Davis


 I read the thread, and not a single one of the "problematic 
 cases" are actually valid C++.

 Yes: the faulty MSVC has taught people to do retarded things, 
 or be afraid of things that were illegal to begin with (in 
 particular, pass an rvalue to a ref, WHICH IS ILLEGAL IN C++), 
 such as "increment(5)".

 There is actually nothing wrong with creating a temporary when 
 something is bound to a const ref, provided the compiler 
 follows the rules:

 *Only LValues with an EXACT type match may be passed to a 
 reference.
 *In regards to *const* references, RValues may be copied in a 
 temporary, and that temporary bound the the ref.

 I'm not saying we particularly *need* this in D (C++ has a "by 
 ref" paradigm that makes it more important, but D *rarelly* 
 ever passes by const ref).

 But if the compiler respects the above two rules (which it 
 should), then RValue to const ref is both perfectly doable and 
 safe (as safe as refs get anyways).


By allowing the the C++ semantics the function looses semantic 
information - whether the actual parameter was lvalue or rvalue. 
This semantic info can be used bot for compiler optimizations and 
move semantics. This is the reason C++11 added && references.

General question (might not be relevant to current design of D):
How about leaving the decision to the compiler and let the 
programmer only specify usage intent?
E.g.: (I'm speaking semantics here, not syntax)
void foo(const Type t); // 1. I only read the value
void foo (mutate Type t); // 2. I want to also mutate the actual 
parameter
void foo (move Type t); // 3. I want to move the actual parameter

In case 1 above, the compiler is free to pass lvalues by const& 
and rvalues by value or perhaps optimize above certain size to 
const& too.
In case 2, the compiler passes a ref to lvalue, rvalues are not 
accepted at CT.
If I want move semantics, I can use option 3 which accepts 
rvalues by ref. btw, what's the correct semantics for lvalues 
here?

What do you think?
Oct 18 2012
prev sibling parent reply "Malte Skarupke" <malteskarupke web.de> writes:
On Thursday, 18 October 2012 at 06:11:26 UTC, monarch_dodra wrote:

 On Thursday, 18 October 2012 at 04:30:17 UTC, Jonathan M Davis 
 wrote:

 On Thursday, October 18, 2012 06:24:08 jerro wrote:

 What would be the problem with const ref taking rvalues?


 Read the thread that I already linked to:

 http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com

 - Jonathan M Davis


 I read the thread, and not a single one of the "problematic 
 cases" are actually valid C++.

 Yes: the faulty MSVC has taught people to do retarded things, 
 or be afraid of things that were illegal to begin with (in 
 particular, pass an rvalue to a ref, WHICH IS ILLEGAL IN C++), 
 such as "increment(5)".

 There is actually nothing wrong with creating a temporary when 
 something is bound to a const ref, provided the compiler 
 follows the rules:

 *Only LValues with an EXACT type match may be passed to a 
 reference.
 *In regards to *const* references, RValues may be copied in a 
 temporary, and that temporary bound the the ref.

 I'm not saying we particularly *need* this in D (C++ has a "by 
 ref" paradigm that makes it more important, but D *rarelly* 
 ever passes by const ref).

 But if the compiler respects the above two rules (which it 
 should), then RValue to const ref is both perfectly doable and 
 safe (as safe as refs get anyways).


The problem with binding rvalues to const ref is that you could 
take and store the address of it. That's why I'd recommend using 
"in ref" instead.

 Jonathan: I had already read the linked discussion. There are 
many valid points in there, but also many invalid ones (as 
monarch_dodra has pointed out). But I think all problems in that 
thread should be solved by using "in ref" instead of "const ref" 
because then you'd be sure that the passed-in temporary can not 
escape the current function.

 foobar: I like the idea, but it's probably going to break down 
in many cases. If you have a non-trivial copy constructor you 
want the ability to have complete control over when it gets 
copied and when it doesn't. I just don't trust compilers enough 
to think that they'd always make the same choice that I'd make.
And also about losing semantic information: That's why I proposed 
the second point: Give the user the option to provide a function 
which should be preferred for rvalues. That way you don't lose 
semantic information.
Oct 18 2012
next sibling parent "jerro" <a a.com> writes:
 The problem with binding rvalues to const ref is that you could 
 take and store the address of it. That's why I'd recommend 
 using "in ref" instead.


You can also take and store the address of a local variable that 
was passed as a const ref parameter, and accessing it after the 
caller exits will result in undefined behavior too. On the other 
hand,  addresses of both local variables and rvalues will be 
valid at least until the called function returns. Local variables 
and rvalues are equivalent in regard to this problem.
Oct 18 2012
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/19/2012 01:39 AM, Malte Skarupke wrote:

 On Thursday, 18 October 2012 at 06:11:26 UTC, monarch_dodra wrote:

 On Thursday, 18 October 2012 at 04:30:17 UTC, Jonathan M Davis wrote:

 On Thursday, October 18, 2012 06:24:08 jerro wrote:

 What would be the problem with const ref taking rvalues?


 Read the thread that I already linked to:

 http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com

 - Jonathan M Davis


 I read the thread, and not a single one of the "problematic cases" are
 actually valid C++.

 Yes: the faulty MSVC has taught people to do retarded things, or be
 afraid of things that were illegal to begin with (in particular, pass
 an rvalue to a ref, WHICH IS ILLEGAL IN C++), such as "increment(5)".

 There is actually nothing wrong with creating a temporary when
 something is bound to a const ref, provided the compiler follows the
 rules:

 *Only LValues with an EXACT type match may be passed to a reference.
 *In regards to *const* references, RValues may be copied in a
 temporary, and that temporary bound the the ref.

 I'm not saying we particularly *need* this in D (C++ has a "by ref"
 paradigm that makes it more important, but D *rarelly* ever passes by
 const ref).

 But if the compiler respects the above two rules (which it should),
 then RValue to const ref is both perfectly doable and safe (as safe as
 refs get anyways).


 The problem with binding rvalues to const ref is that you could take and
 store the address of it. That's why I'd recommend using "in ref" instead.

  Jonathan: I had already read the linked discussion. There are many
 valid points in there, but also many invalid ones (as monarch_dodra has
 pointed out). But I think all problems in that thread should be solved
 by using "in ref" instead of "const ref" because then you'd be sure that
 the passed-in temporary can not escape the current function.

  foobar: I like the idea, but it's probably going to break down in many
 cases. If you have a non-trivial copy constructor you want the ability
 to have complete control over when it gets copied and when it doesn't. I
 just don't trust compilers enough to think that they'd always make the
 same choice that I'd make.
 And also about losing semantic information: That's why I proposed the
 second point: Give the user the option to provide a function which
 should be preferred for rvalues. That way you don't lose semantic
 information.


Const is different in D and in C++. Relating const and rvalues is 
arbitrary and does not make a lot of sense.

Regarding 'in ref'/'scope ref': What should 'scope' apply to in

void foo(scope ref int* x);
Oct 18 2012
parent reply "Malte Skarupke" <malteskarupke web.de> writes:
On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:

 Const is different in D and in C++. Relating const and rvalues 
 is arbitrary and does not make a lot of sense.

 Regarding 'in ref'/'scope ref': What should 'scope' apply to in

 void foo(scope ref int* x);


Not sure what you mean with "relating." I'm not making any claims 
about there being a relationship between rvalues and constness.

This is about finding a way that you can define a function which 
safely accepts lvalues and rvalues without having to make a copy. 
If we specify the argument as "ref in", then we can safely pass 
for example the number 5 to it. And this would never break 
existing code, so that something like swap(5, 4) would never be 
possible code.

For the example that you gave you'd be unable to store the 
address of x. So doing

int** storage;
void foo(scope ref int * x)
{
     storage = &x;
}

would be illegal.

 jerro: the same thing: I'm not trying to fix the problem that 
you mention. I'm trying to define a function which can safely 
accept rvalues and lvalues without having to make a copy.
Oct 18 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/19/2012 03:26 AM, Malte Skarupke wrote:

 On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:

 Const is different in D and in C++. Relating const and rvalues is
 arbitrary and does not make a lot of sense.

 Regarding 'in ref'/'scope ref': What should 'scope' apply to in

 void foo(scope ref int* x);


 Not sure what you mean with "relating." I'm not making any claims about
 there being a relationship between rvalues and constness.


You do it again right away:


 This is about finding a way that you can define a function which safely
 accepts lvalues and rvalues without having to make a copy. If we specify
 the argument as "ref in", then we can safely pass for example the number


                   ^~~~~~~~
                     const

 5 to it.


   ^
rvalue


 And this would never break existing code, so that something
 like swap(5, 4) would never be possible code.


That does not break existing code except code that checks validity of
code, but every language change does that.


 For the example that you gave you'd be unable to store the address of x.
 So doing

 int** storage;
 void foo(scope ref int * x)
 {
      storage = &x;
 }

 would be illegal.


Then how to specify that the value of x cannot be escaped?
I'm in favour of doing it the other way round and disallow escaping of
ref parameters without an unsafe cast.


  jerro: the same thing: I'm not trying to fix the problem that you
 mention. I'm trying to define a function which can safely accept rvalues
 and lvalues without having to make a copy.
Oct 18 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-10-19 04:48, Timon Gehr wrote:


 Then how to specify that the value of x cannot be escaped?
 I'm in favour of doing it the other way round and disallow escaping of
 ref parameters without an unsafe cast.


"scope" is supposed to be used to prevent this.

-- 
/Jacob Carlborg
Oct 19 2012
next sibling parent "foobar" <foo bar.com> writes:
On Friday, 19 October 2012 at 07:53:30 UTC, Jacob Carlborg wrote:

 On 2012-10-19 04:48, Timon Gehr wrote:


 Then how to specify that the value of x cannot be escaped?
 I'm in favour of doing it the other way round and disallow 
 escaping of
 ref parameters without an unsafe cast.


 "scope" is supposed to be used to prevent this.


I like Timon's idea. "scope" is a bad fit in that the default 
should be the safe option. Unfortunately this does have the 
potential to brake lots of code, perhaps even if we limit it to 
 safe code.
An argument could be made that it's worth the breakage for  safe 
code to insure better safety.
Oct 19 2012
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/19/2012 09:53 AM, Jacob Carlborg wrote:

 On 2012-10-19 04:48, Timon Gehr wrote:


 Then how to specify that the value of x cannot be escaped?
 I'm in favour of doing it the other way round and disallow escaping of
 ref parameters without an unsafe cast.


 "scope" is supposed to be used to prevent this.


Sure, but how?

void goo(scope int* x){
     global0 = x; // should clearly be disallowed
}

void foo(scope ref int*** x){
     global1 = &x; // ?
     global2 = x;  // ?
     global3 = *x; // ?
     globall4 = **x; // ?
}

Maybe we need this:

void foo(scope ref int*** x);   // ?
void foo(ref int(***)scope  x); // no escaping of x, *x, **x
void foo(ref int*(**)scope x);  // may escape **x
void foo(ref int**(*)scope x);  // may escape *x, **x

What about &x?
Oct 19 2012
parent reply "Malte Skarupke" <malteskarupke web.de> writes:
On Friday, 19 October 2012 at 13:00:52 UTC, Timon Gehr wrote:

 On 10/19/2012 09:53 AM, Jacob Carlborg wrote:

 On 2012-10-19 04:48, Timon Gehr wrote:


 Then how to specify that the value of x cannot be escaped?
 I'm in favour of doing it the other way round and disallow 
 escaping of
 ref parameters without an unsafe cast.


 "scope" is supposed to be used to prevent this.


 Sure, but how?

 void goo(scope int* x){
     global0 = x; // should clearly be disallowed
 }

 void foo(scope ref int*** x){
     global1 = &x; // ?
     global2 = x;  // ?
     global3 = *x; // ?
     globall4 = **x; // ?
 }

 Maybe we need this:

 void foo(scope ref int*** x);   // ?
 void foo(ref int(***)scope  x); // no escaping of x, *x, **x
 void foo(ref int*(**)scope x);  // may escape **x
 void foo(ref int**(*)scope x);  // may escape *x, **x

 What about &x?


No scope should mean that you can not escape the address of 
something. So

void goo(scope int* x)
{
     global0 = x;
}

Should be allowed. Scope in this case applies to the pointer. Not 
to the thing it's pointing to. The scope keyword is not 
transitive. That wouldn't make sense. It is perfectly legal to 
have a scoped pointer to something allocated on the heap.


Whereas

void goo(scope ref int x)
{
     global0 = &x;
}

Should NOT be allowed. It looks like it's the same code, but in 
this case x is an integer. In the last example x was a pointer to 
an integer.

So it comes down to this:

void goo(scope int* x)
{
     global0 = x; // copying is allowed
     //global1 = &x; // referencing is not allowed
}
Oct 20 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 10/20/2012 08:25 PM, Malte Skarupke wrote:

 ...

 So it comes down to this:

 void goo(scope int* x)
 {
      global0 = x; // copying is allowed
      //global1 = &x; // referencing is not allowed
 }


That is useless.
Oct 20 2012
prev sibling next sibling parent "jerro" <a a.com> writes:
  jerro: the same thing: I'm not trying to fix the problem that 
 you mention. I'm trying to define a function which can safely 
 accept rvalues and lvalues without having to make a copy.


My point was that saving an address of a const ref parameter is 
already unsafe if you call the function with a local variable as 
the parameter. If this behavior seems problematic to you when it 
concerns rvalues, it should seem equally problematic when it 
comes to local variables. It doesn't make sense to make passing 
rvalues as const ref parameters illegal because of this problem, 
when passing local variables causes the same problem and is 
legal. It would only make sense to introduce "in ref" whith an 
intent to solve this problem, if local variables would also have 
to be passed as "in ref" (or some kind of scope ref, not 
necessarily const). But that would break pretty much all code 
that uses ref.

The only case I can think of when passing a local variable as 
const ref is safe, but passing an rvalue wouldn't be, is when the 
called function returns the address of the const parameter (or 
assigns it to some other ref parameter).
Oct 18 2012
prev sibling parent reply "martin" <kinke libero.it> writes:
On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:

 Const is different in D and in C++. Relating const and rvalues 
 is arbitrary and does not make a lot of sense.


It's actually pretty much the same concept in both languages 
except for the transitiveness in D. An implicit rvalue => _const_ 
ref conversion is something completely different than an rvalue 
=> mutable ref conversion, and this distinction is sadly largely 
lost in the linked thread, which is why I tried to re-initiate 
the discussion with another recent thread. Arguments passed by 
mutable references are expected to be changed by the called 
function, so passing rvalues does indeed not make much sense in 
that case (side effects may get lost, quite possibly 
unintentionally).

On Friday, 19 October 2012 at 01:26:35 UTC, Malte Skarupke wrote:

 This is about finding a way that you can define a function 
 which safely accepts lvalues and rvalues without having to make 
 a copy.


EXACTLY! Something telling the compiler: pass the argument by 
const reference instead of a costly const copy, if possible. So 
for an lvalue, pass directly its address; for an rvalue, allocate 
the temporary on the stack and then pass its address:

<code>
T foo(in ref T bla) { return bla; }

T lvalue;
foo(lvalue); // fine

foo(foo(lvalue)); // error: rvalue!
// required compiler magic:
immutable tmp = foo(lvalue); foo(tmp);
/* possibly in dedicated scope for
    immediate destruction of tmp    */
</code>

Currently having to allocate all rvalues manually is frankly both 
ugly and a pain in the ass, as would be the other alternative: 
overloading foo().

On Friday, 19 October 2012 at 03:49:56 UTC, jerro wrote:

 My point was that saving an address of a const ref parameter is 
 already unsafe if you call the function with a local variable 
 as the parameter. If this behavior seems problematic to you 
 when it concerns rvalues, it should seem equally problematic 
 when it comes to local variables. It doesn't make sense to make 
 passing rvalues as const ref parameters illegal because of this 
 problem, when passing local variables causes the same problem 
 and is legal. [...]
 The only case I can think of when passing a local variable as 
 const ref is safe, but passing an rvalue wouldn't be, is when 
 the called function returns the address of the const parameter 
 (or assigns it to some other ref parameter).


Exactly my thoughts regarding this escaping aspect, which I don't 
see as big issue. The compiler magic illustrated in the previous 
example without dedicated scope for temporaries would make 
passing rvalues exactly as safe/unsafe as passing local variables.
Oct 21 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/22/2012 12:18 AM, martin wrote:

 On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:

 Const is different in D and in C++. Relating const and rvalues is
 arbitrary and does not make a lot of sense.


 It's actually pretty much the same concept in both languages except for
 the transitiveness in D.


Case closed.
Oct 22 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/22/2012 11:38 AM, Timon Gehr wrote:

 On 10/22/2012 12:18 AM, martin wrote:

 On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:

 Const is different in D and in C++. Relating const and rvalues is
 arbitrary and does not make a lot of sense.


 It's actually pretty much the same concept in both languages except for
 the transitiveness in D.


 Case closed.


(But it might be added that C++11 adds a new language feature just in 
order to be able to distinguish const and rvalue references in the
callee, so even with C++'s semantics, this does not seem like a good
idea.)
Oct 22 2012
parent reply "martin" <kinke libero.it> writes:
On Monday, 22 October 2012 at 09:41:38 UTC, Timon Gehr wrote:

 (But it might be added that C++11 adds a new language feature
 just in order to be able to distinguish const and rvalue
 references in the callee, so even with C++'s semantics, this
 does not seem like a good idea.)


Yeah, and that T&& syntax is mainly used for _mutable_ rvalue 
references in move constructors and assignment operators to move 
(hijack) mutable data from the rvalue to another T instance 
instead of copying that data, knowing that the rvalue's data is 
not going to be accessed anymore anyway. So this is a different 
case and does currently not apply to D afaik since there is no 
way to forward rvalues:

void foo(ref T lvalue) { }
void foo(T rvalue)     { }

In the latter overload for rvalues, you aren't given the original 
rvalue, but a copy of it!

Distinguishing between _const_ rvalue and lvalue references 
though makes no sense imho, and that is my whole point.
Oct 22 2012
parent "martin" <kinke libero.it> writes:
On Monday, 22 October 2012 at 11:59:27 UTC, martin wrote:

 In the latter overload for rvalues, you aren't given the 
 original rvalue, but a copy of it!


I need to correct that after a quick test: the rvalue is passed 
directly (moved) instead of copying it (well, at least the copy 
constructor this(this) is not invoked, even in a debug build); 
that makes perfect sense, is efficient and eliminates the need 
for C++ rvalue references (T&&).
It doesn't affect the need for an implicit rvalue => const ref 
propagation though.

What I'd like to see is the following passing scheme for function 
arguments (read-only parameters are denoted by (*)):

               lvalue:       rvalue:
            T: copy          move
(*)     in T: copy          move
        out T: pass pointer  n/a
        ref T: pass pointer  n/a
(*) in ref T: pass pointer  store on the caller's stack and pass 
its address

So only the rvalue passing rule for the "in ref T" case would 
need to be implemented. This would allow to use "foo(in ref T 
bar)" for lvalues (eliding a copy expected to be costly) as well 
as rvalues instead of having to add an overload "foo(in T bar)" 
for rvalues. For rvalues, this would actually implicate a 
performance hit due to pointer indirection, so the compiler could 
attempt to add an automatic "foo(in T bar)" overload if not 
existent. For rvalues, the latter overload (i.e., "in T" 
parameters) should be preferred over "in ref T" parameters - 
exactly as the thread starter Malte proposes:


 - Make functions that take "ref in" arguments also accept 
 rvalues.
 - The user can still provide an overload that accepts an 
 rvalue, using the "in" keyword, and that one will be preferred 
 over the "ref in" version.
Oct 22 2012
prev sibling next sibling parent reply "Rob T" <rob ucora.com> writes:
On Thursday, 18 October 2012 at 03:07:56 UTC, Malte Skarupke 
wrote:

 Hello,

 I realize that this has been discussed before, but so far there 
 is no solution and this really needs to be a high priority:

 We need a way for a function to declare that it doesn't want 
 it's argument to be copied, but it also doesn't care whether 
 the argument is an rvalue or an lvalue.


I'm encountering this problem right now, and found this thread 
while looking for a solution. In my case I'm forced into doubling 
up some code, which is not what I want to be doing, so I agree 
this item should be fixed. It would be nice to have something 
like the proposed "auto ref" solution, like we have with return 
values.

--rt
Nov 04 2012
parent reply "martin" <kinke libero.it> writes:
I find it sad that while this topic seems to be of high priority 
for quite a lot of language users, it is seemingly neglected by 
the head of language development (Walter, Andrei etc.).
Nov 04 2012
next sibling parent "Rob T" <rob ucora.com> writes:
On Monday, 5 November 2012 at 00:58:19 UTC, martin wrote:

 I find it sad that while this topic seems to be of high 
 priority for quite a lot of language users, it is seemingly 
 neglected by the head of language development (Walter, Andrei 
 etc.).


I don't know why this would be ignored, it's rather useful and 
solves a real-world programming problem. I hope this is just one 
of those detials that was missed and is only being ignored due to 
attention from much bigger problems.

Do you know if there is already a feature request or bug report 
on this item?
Nov 04 2012
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/4/12 7:58 PM, martin wrote:

 I find it sad that while this topic seems to be of high priority for
 quite a lot of language users, it is seemingly neglected by the head of
 language development (Walter, Andrei etc.).


I was hoping "auto ref" solves this problem. I think it's currently only 
implemented for templates.

Andrei
Nov 04 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 04, 2012 20:43:36 Andrei Alexandrescu wrote:

 On 11/4/12 7:58 PM, martin wrote:

 I find it sad that while this topic seems to be of high priority for
 quite a lot of language users, it is seemingly neglected by the head of
 language development (Walter, Andrei etc.).


 
 I was hoping "auto ref" solves this problem. I think it's currently only
 implemented for templates.


And when we argued for altering it so that it operated like const ref in C++ 
(which allows const ref in D to continue to function like it does now), some 
folks complained, because they've found the current semantics of auto ref to 
be useful (something to do with propagating the exact, original type, I 
think).

Now, since auto ref currently only works with templates, maybe we can keep its 
current semantics with templated functions but alter them for non-templated 
functions so that it works like const ref does in C++. The downside is that 
the semantics for auto ref between templated functions and non-templated 
functions are slightly different, but they're close enough that I'm not sure 
that it matters.

- Jonathan M Davis
Nov 04 2012
next sibling parent "Malte Skarupke" <malteskarupke web.de> writes:
On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis 
wrote:

 On Sunday, November 04, 2012 20:43:36 Andrei Alexandrescu wrote:

 On 11/4/12 7:58 PM, martin wrote:

 I find it sad that while this topic seems to be of high 
 priority for
 quite a lot of language users, it is seemingly neglected by 
 the head of
 language development (Walter, Andrei etc.).


 
 I was hoping "auto ref" solves this problem. I think it's 
 currently only
 implemented for templates.


 And when we argued for altering it so that it operated like 
 const ref in C++
 (which allows const ref in D to continue to function like it 
 does now), some
 folks complained, because they've found the current semantics 
 of auto ref to
 be useful (something to do with propagating the exact, original 
 type, I
 think).

 Now, since auto ref currently only works with templates, maybe 
 we can keep its
 current semantics with templated functions but alter them for 
 non-templated
 functions so that it works like const ref does in C++. The 
 downside is that
 the semantics for auto ref between templated functions and 
 non-templated
 functions are slightly different, but they're close enough that 
 I'm not sure
 that it matters.

 - Jonathan M Davis


Yes, please. Auto ref for non-template functions would solve the 
problem exactly. I also like it because then the intent of the 
programmer is clear.
Nov 04 2012
prev sibling parent reply "Rob T" <rob ucora.com> writes:
On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis
wrote:

 And when we argued for altering it so that it operated like 
 const ref in C++
 (which allows const ref in D to continue to function like it 
 does now), some
 folks complained, because they've found the current semantics 
 of auto ref to
 be useful (something to do with propagating the exact, original 
 type, I
 think).


I would expect that auto ref for a template and for a non
template should work in exactly the same way, so why would there
be a difference? If there must be a difference, there should be
different semantics for specifying the difference, otherwise the
inconsistent behaviours among identical semantics will only serve
to confuse people.

--rt
Nov 04 2012
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 05, 2012 06:05:07 Rob T wrote:

 On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis
 
 wrote:

 And when we argued for altering it so that it operated like
 const ref in C++
 (which allows const ref in D to continue to function like it
 does now), some
 folks complained, because they've found the current semantics
 of auto ref to
 be useful (something to do with propagating the exact, original
 type, I
 think).


 
 I would expect that auto ref for a template and for a non
 template should work in exactly the same way, so why would there
 be a difference? If there must be a difference, there should be
 different semantics for specifying the difference, otherwise the
 inconsistent behaviours among identical semantics will only serve
 to confuse people.


auto ref's current semantics can't possibly work with non-templated functions, 
and it's clear that there are good reasons for keeping auto ref as it is now 
for templates. So, either we make it work with non-templated functions with 
different (albeit similar) semantics, or we need a new attribute for doing what 
we want to do with non-templated functions.

- Jonathan M Davis
Nov 05 2012
parent "martin" <kinke libero.it> writes:
On Monday, 5 November 2012 at 08:01:57 UTC, Jonathan M Davis 
wrote:

 auto ref's current semantics can't possibly work with 
 non-templated functions,
 and it's clear that there are good reasons for keeping auto ref 
 as it is now
 for templates. So, either we make it work with non-templated 
 functions with
 different (albeit similar) semantics, or we need a new 
 attribute for doing what
 we want to do with non-templated functions.


I agree that the current auto ref semantics are only useful for 
templates since "auto ref T" is replaced by either a reference 
(ref T for lvalues) or a value (T for rvalues), two very 
different things.
I don't know how auto ref for templates is currently implemented, 
i.e., if, given a function

void foo(T)(auto ref T x, auto ref T y) {}

the compiler would instantiate the template max 2^2 times for 
each used type T to cover all lvalue/rvalue combinations for x 
and y. In that case, treating a non-templated function

void foo(auto ref MyStruct x, auto ref MyStruct y) {}

as implicit template (without template parameters) would be a 
viable option imho, i.e.,

void foo()(auto ref MyStruct x, auto ref MyStruct y) {}

If that kind of auto-templating is not what we want, I'd 
definitely opt for allowing rvalues as "in ref" parameters to 
keep things simple.
Nov 05 2012
prev sibling parent Manu <turkeyman gmail.com> writes:
Yeah I really don't understand how 'auto ref' entered this conversation? It
completely misses the point.

The issue as I see it is this:
  void func(ref in Vector m);
  func(v1*v2 + Vector(10, 20, 30));

And countless situations like it, all leading to innumerable temp locals
with irrelevant names:
  Vector v1v2PlusSomeStuff = v1*v2 + Vector(10, 20, 30);
  func(v1v2PlusSomeStuff);

Or even:
  Vector initialiser1 = Vector(10, 20, 30); // <- words can scarcely
describe how annoying this is
  func(initialiser1);

Allowing rvalues to be passed by ref is exactly as unsafe as those
examples, and that workaround is employed every time anyway, so the
restriction is self-defeating.


On 5 November 2012 10:01, Jonathan M Davis <jmdavisProg gmx.com> wrote:


 On Monday, November 05, 2012 06:05:07 Rob T wrote:

 On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis

 wrote:

 And when we argued for altering it so that it operated like
 const ref in C++
 (which allows const ref in D to continue to function like it
 does now), some
 folks complained, because they've found the current semantics
 of auto ref to
 be useful (something to do with propagating the exact, original
 type, I
 think).


 I would expect that auto ref for a template and for a non
 template should work in exactly the same way, so why would there
 be a difference? If there must be a difference, there should be
 different semantics for specifying the difference, otherwise the
 inconsistent behaviours among identical semantics will only serve
 to confuse people.


 auto ref's current semantics can't possibly work with non-templated
 functions,
 and it's clear that there are good reasons for keeping auto ref as it is
 now
 for templates. So, either we make it work with non-templated functions with
 different (albeit similar) semantics, or we need a new attribute for doing
 what
 we want to do with non-templated functions.

 - Jonathan M Davis
Nov 06 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, November 06, 2012 20:40:38 Manu wrote:

 Yeah I really don't understand how 'auto ref' entered this conversation? It
 completely misses the point.


The _entire_ reason that auto ref was introduced in the first place was to 
solve the const ref problem. That's what it's for. It wouldn't exist otherwise.
So,
it's completely relevant to any discussion of const ref.

However, Walter misunderstood what Andrei meant by the feature, so it only got
implemented for templates, and given how Walter implemented it, it _can't_ be
implemented with non-templated functions. But it could be implemented with
slightly
different semantics such that it solves the const ref problem as orignally
intended.
It's just that it would then have slightly different semantics between templated
and non-templated functions.

- Jonathan M Davis
Nov 06 2012
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
But it only really makes sense in the context of templates...?
Why should something called 'auto ref' provide fabrication of temporaries
for the purpose of passing rvalues to functions that receive ref args?

How does auto ref (under some different definition/implementation) address
the rvalue problem?


On 6 November 2012 23:08, Jonathan M Davis <jmdavisProg gmx.com> wrote:


 On Tuesday, November 06, 2012 20:40:38 Manu wrote:

 Yeah I really don't understand how 'auto ref' entered this conversation?


 It

 completely misses the point.


 The _entire_ reason that auto ref was introduced in the first place was to
 solve the const ref problem. That's what it's for. It wouldn't exist
 otherwise. So,
 it's completely relevant to any discussion of const ref.

 However, Walter misunderstood what Andrei meant by the feature, so it only
 got
 implemented for templates, and given how Walter implemented it, it _can't_
 be
 implemented with non-templated functions. But it could be implemented with
 slightly
 different semantics such that it solves the const ref problem as orignally
 intended.
 It's just that it would then have slightly different semantics between
 templated
 and non-templated functions.

 - Jonathan M Davis
Nov 06 2012
parent reply "martin" <kinke libero.it> writes:
On Tuesday, 6 November 2012 at 22:32:57 UTC, Manu wrote:

 But it only really makes sense in the context of templates...?
 Why should something called 'auto ref' provide fabrication of 
 temporaries
 for the purpose of passing rvalues to functions that receive 
 ref args?

 How does auto ref (under some different 
 definition/implementation) address
 the rvalue problem?


The thing is that currently there are 2 workarounds. You 
described the first one - allocating temporaries manually to 
obtain referenceable lvalues:

void func(ref in Vector m);
Vector v1v2PlusSomeStuff = v1*v2 + Vector(10, 20, 30);
func(v1v2PlusSomeStuff);

The other, also painfully annoying workaround is overloading func:

void func(in ref Vector m);
void func(in Vector m);
func(v1*v2 + Vector(10, 20, 30));

'auto ref' implements the second workaround (via a template) and 
therefore requires a single, but templated func() implementation:

void func(T)(in auto ref T m);

This template, as I understand it, gets expanded to:

void func(T)(in ref T m); // for lvalues
void func(T)(in T m);     // for rvalues

So for non-templated functions, I suggest 2 options:

1) The previously described auto-templates (identical 'auto ref' 
semantics), where a function with 'auto ref' parameters is 
treated as implicit template. This may lead to code-bloating (for 
larger functions) and/or higher performance for rvalue arguments 
(rvalues passed to value arguments are moved, not copied; we 
therefore gain nothing by passing a reference, but incur a slight 
performance hit due to pointer indirection instead of accessing 
directly the rvalue on the stack). OR
2) Simple under-the-hood temporary lvalue declaration by the 
compiler (for rvalues passed to 'const ref' parameters) - that 
would be a handy implementation of the first workaround.

I hope you get my point. :)
Nov 06 2012
next sibling parent reply Manu <turkeyman gmail.com> writes:
If the compiler started generating 2 copies of all my ref functions, I'd be
rather unimpressed... bloat is already a problem in D. Perhaps this may be
a handy feature, but I wouldn't call this a 'solution' to this issue.
Also, what if the function is external (likely)... auto ref can't work if
the function is external, an implicit temporary is required in that case.


On 7 November 2012 01:37, martin <kinke libero.it> wrote:


 On Tuesday, 6 November 2012 at 22:32:57 UTC, Manu wrote:


 But it only really makes sense in the context of templates...?
 Why should something called 'auto ref' provide fabrication of temporaries
 for the purpose of passing rvalues to functions that receive ref args?

 How does auto ref (under some different definition/implementation) address
 the rvalue problem?


 The thing is that currently there are 2 workarounds. You described the
 first one - allocating temporaries manually to obtain referenceable lvalues:


 void func(ref in Vector m);
 Vector v1v2PlusSomeStuff = v1*v2 + Vector(10, 20, 30);
 func(v1v2PlusSomeStuff);

 The other, also painfully annoying workaround is overloading func:

 void func(in ref Vector m);
 void func(in Vector m);

 func(v1*v2 + Vector(10, 20, 30));

 'auto ref' implements the second workaround (via a template) and therefore
 requires a single, but templated func() implementation:

 void func(T)(in auto ref T m);

 This template, as I understand it, gets expanded to:

 void func(T)(in ref T m); // for lvalues
 void func(T)(in T m);     // for rvalues

 So for non-templated functions, I suggest 2 options:

 1) The previously described auto-templates (identical 'auto ref'
 semantics), where a function with 'auto ref' parameters is treated as
 implicit template. This may lead to code-bloating (for larger functions)
 and/or higher performance for rvalue arguments (rvalues passed to value
 arguments are moved, not copied; we therefore gain nothing by passing a
 reference, but incur a slight performance hit due to pointer indirection
 instead of accessing directly the rvalue on the stack). OR
 2) Simple under-the-hood temporary lvalue declaration by the compiler (for
 rvalues passed to 'const ref' parameters) - that would be a handy
 implementation of the first workaround.

 I hope you get my point. :)
Nov 06 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/7/2012 3:54 AM, Manu пишет:

 If the compiler started generating 2 copies of all my ref functions, I'd
 be rather unimpressed... bloat is already a problem in D. Perhaps this
 may be a handy feature, but I wouldn't call this a 'solution' to this issue.
 Also, what if the function is external (likely)... auto ref can't work
 if the function is external, an implicit temporary is required in that case.


What's wrong with going this route:

void blah(auto ref X stuff){
...lots of code...
}

is magically expanded to:

void blah(ref X stuff){
...that code..
}

and

void blah(X stuff){
	.blah(stuff); //now here stuff is L-value so use the ref version
}

Yeah, it looks _almost_ like a template now. But unlike with a template 
we can assume it's 2 overloads _always_. External  fucntion issue is 
then solved by treating it as exactly these 2 overloads (one trampoline, 
one real). Basically it becomes one-line declaration of 2 functions.

Given that temporaries are moved anyway the speed should be fine and 
there is as much bloat as you'd do by hand.

Also hopefully inliner can be counted on to do its thing in this simple 
case.



-- 
Dmitry Olshansky
Nov 08 2012
next sibling parent reply "martin" <kinke libero.it> writes:
On Thursday, 8 November 2012 at 18:28:44 UTC, Dmitry Olshansky 
wrote:

 What's wrong with going this route:

 void blah(auto ref X stuff){
 ...lots of code...
 }

 is magically expanded to:

 void blah(ref X stuff){
 ...that code..
 }

 and

 void blah(X stuff){
 	.blah(stuff); //now here stuff is L-value so use the ref 
 version
 }

 Yeah, it looks _almost_ like a template now. But unlike with a 
 template we can assume it's 2 overloads _always_. External  
 fucntion issue is then solved by treating it as exactly these 2 
 overloads (one trampoline, one real). Basically it becomes 
 one-line declaration of 2 functions.

 Given that temporaries are moved anyway the speed should be 
 fine and there is as much bloat as you'd do by hand.

 Also hopefully inliner can be counted on to do its thing in 
 this simple case.


That second overload for rvalues would be a shortcut to save the 
lvalue declarations at each call site - and it really doesn't 
matter if the compiler magically added the lvalue declarations 
before each call or if it magically added the rvalue overload 
(assuming all calls are inlined). But it would create a problem 
if there already was an explicit 'void blah(X)' overload in 
addition to 'void blah(auto ref X)' (not making much sense 
obviously, but this would be something the compiler needed to 
handle somehow).
What this 'auto ref' approach (both as currently implemented for 
templates and proposed here for non-templated functions) lacks is 
the vital distinction between const and mutable parameters.

For the much more common const ref parameters, I repeatedly tried 
to explain why I'm absolutely convinced that we don't need 
another keyword and that 'in/const ref' is sufficient, safe, 
logical and intuitive (coupled with the overload rule that 
pass-by-value (moving) is preferred for rvalues). Please prove me 
wrong.

For the less common mutable ref parameters, I also repeatedly 
tried to explain why I find it dangerous/unsafe to allow rvalues 
to be bound to mutable ref parameters. But if there are enough 
people wanting that, I'd have no problem with an 'auto ref' 
approach for it (only for mutable parameters!). That may actually 
be a good compromise, what do you guys think? :)

'auto ref T' for templates expands to 'ref T' (lvalues) and 'T' 
(rvalues), duplicating the whole function and providing best 
performance - no pointer/reference indirection for rvalues in 
contrast to 'auto ref T' (proposed above) for non-templates, 
otherwise the concept would be exactly the same. But it's only 
for mutable parameters.
Such a templated option may also be worth for const parameters 
though (expanding to 'const ref T' and 'const T'), so maybe 
something like the (ambiguous) 'in/const auto ref T' wouldn't 
actually be that bad (assuming there are only a few use cases, 
and only for templates! It'd still be 'in ref T' for 
non-templates).
Nov 08 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/8/2012 11:30 PM, martin пишет:

 On Thursday, 8 November 2012 at 18:28:44 UTC, Dmitry Olshansky wrote:

 What's wrong with going this route:

 void blah(auto ref X stuff){
 ...lots of code...
 }

 is magically expanded to:

 void blah(ref X stuff){
 ...that code..
 }

 and

 void blah(X stuff){
     .blah(stuff); //now here stuff is L-value so use the ref version
 }

 Yeah, it looks _almost_ like a template now. But unlike with a
 template we can assume it's 2 overloads _always_. External fucntion
 issue is then solved by treating it as exactly these 2 overloads (one
 trampoline, one real). Basically it becomes one-line declaration of 2
 functions.

 Given that temporaries are moved anyway the speed should be fine and
 there is as much bloat as you'd do by hand.

 Also hopefully inliner can be counted on to do its thing in this
 simple case.


 That second overload for rvalues would be a shortcut to save the lvalue
 declarations at each call site - and it really doesn't matter if the
 compiler magically added the lvalue declarations before each call or if
 it magically added the rvalue overload (assuming all calls are inlined).


The scope. It's all about getting the correct scope, destructor call and 
you know, the works. Preferably it can inject it inside temporary scope.

Anticipating bugs in the implementation of this feature let me warn that 
re-writing this:
... code here ...
auto  r = foo(SomeResource(x, y, ..)); //foo is auto ref
... code here ...

Should not change semantics e.g. imagine the resource is a lock, we'd 
better unlock it sooner. That is call destructor right after foo 
returns. So we need {} around the call. But this doesn't work as it 
traps 'r':

{
auto someRef = SomeResource(x, y, ..);
auto r  = foo(someRef);
}

So it's rather something like this:

typeof(foo(...)) r = void;
{
someRef = SomeResource(x, y, ..);
r = foo(someRef); // should in fact construct in place not assign
}

I suspect this is hackable to be more clean inside of the compiler but 
not in terms of a re-write.


 But it would create a problem if there already was an explicit 'void
 blah(X)' overload in addition to 'void blah(auto ref X)' (not making
 much sense obviously, but this would be something the compiler needed to
 handle somehow).


Aye. But even then there is an ambiguity if there is one version of 
function with ref T / T and one with auto ref T.


 What this 'auto ref' approach (both as currently implemented for
 templates and proposed here for non-templated functions) lacks is the
 vital distinction between const and mutable parameters.

 For the much more common const ref parameters, I repeatedly tried to
 explain why I'm absolutely convinced that we don't need another keyword
 and that 'in/const ref' is sufficient, safe, logical and intuitive
 (coupled with the overload rule that pass-by-value (moving) is preferred
 for rvalues). Please prove me wrong.


I'd rather restrict it to 'auto ref' thingie. Though 'in auto ref' 
sounds outright silly.
Simply put const ref implies that callee can save a pointer to it 
somewhere (it's l-value). The same risk is with 'auto ref' but at least 
there an explicitly written 'disclaimer' by the author of accepting 
temporary stuff.

In the ideal world name 'auto ref' would be shorter, logical and more to 
the point but we have what we have.


 For the less common mutable ref parameters, I also repeatedly tried to
 explain why I find it dangerous/unsafe to allow rvalues to be bound to
 mutable ref parameters. But if there are enough people wanting that, I'd
 have no problem with an 'auto ref' approach for it (only for mutable
 parameters!). That may actually be a good compromise, what do you guys
 think? :)


I think that function plucked with auto ref is a enough indication that 
author is fine with passing to it mutable r-values and not seeing 
changes outside and related blah-blah. In most (all?) of cases it means 
that parameter is too big to be passed by copy so rather it takes it by ref.
Also certain stuff can't be properly bitwise const because of C-calls 
and what not. Logical const is the correct term but in the D world it's 
simply mutable.


 'auto ref T' for templates expands to 'ref T' (lvalues) and 'T'
 (rvalues), duplicating the whole function and providing best performance
 - no pointer/reference indirection for rvalues in contrast to 'auto ref
 T' (proposed above) for non-templates, otherwise the concept would be
 exactly the same. But it's only for mutable parameters.


I'd say that even for templates the speed argument is mostly defeated by 
the bloat argument. But that's probably only me.


 Such a templated option may also be worth for const parameters though
 (expanding to 'const ref T' and 'const T'), so maybe something like the
 (ambiguous) 'in/const auto ref T' wouldn't actually be that bad
 (assuming there are only a few use cases, and only for templates! It'd
 still be 'in ref T' for non-templates).




-- 
Dmitry Olshansky
Nov 08 2012
parent "martin" <kinke libero.it> writes:
On Thursday, 8 November 2012 at 20:15:51 UTC, Dmitry Olshansky 
wrote:

 The scope. It's all about getting the correct scope, destructor 
 call and you know, the works. Preferably it can inject it 
 inside temporary scope.

 typeof(foo(...)) r = void;
 {
 someRef = SomeResource(x, y, ..);
 r = foo(someRef); // should in fact construct in place not 
 assign
 }

 I suspect this is hackable to be more clean inside of the 
 compiler but not in terms of a re-write.


Right, I forgot the scope for a moment. I'd illustrate the rvalue 
=> (const) ref binding to a novice language user as follows:

T   const_foo(  in ref int x);
T mutable_foo(auto ref int x);

int bar() { return 5; }

T result;

result = const_foo(bar());
/* expanded to:
{
     immutable int tmp = bar(); // avoidable for literals
     result = const_foo(tmp);
} // destruction of tmp
*/

result = mutable_foo(bar());
/* expanded to:
{
     int tmp = bar();
     result = mutable_foo(tmp);
} // destruction of tmp
*/


 I'd rather restrict it to 'auto ref' thingie. Though 'in auto 
 ref' sounds outright silly.
 Simply put const ref implies that callee can save a pointer to 
 it somewhere (it's l-value). The same risk is with 'auto ref' 
 but at least there an explicitly written 'disclaimer' by the 
 author of accepting temporary stuff.


'in ref' as opposed to 'const ref' should disallow this escaping 
issue we've already tackled in this thread, but I'm not sure if 
it is already/correctly implemented. Anyway, this issue also 
arises with (short-lived) local lvalues at the caller site:

foreach (i; 0 .. 10)
{
     int scopedLvalue = i + 2;
     foo(scopedLvalue); // passed by ref
} // scopedLvalue is gone


 In the ideal world name 'auto ref' would be shorter, logical 
 and more to the point but we have what we have.


+1, but I don't have a better proposal anyway. ;)


 I think that function plucked with auto ref is a enough 
 indication that author is fine with passing to it mutable 
 r-values and not seeing changes outside and related blah-blah.


Agreed.


 Also certain stuff can't be properly bitwise const because of 
 C-calls and what not. Logical const is the correct term but in 
 the D world it's simply mutable.


As you know, I'd definitely allow rvalues to be bound to const 
ref parameters as alternative (that would also be useful for a 
lot of existing code). People who generally don't use const 
(Timon Gehr? :)) are free to only use 'auto ref', I'm most likely 
only going to use 'in ref', and there will certainly be people 
using both. Sounds like a really good compromise to me.


 I'd say that even for templates the speed argument is mostly 
 defeated by the bloat argument. But that's probably only me.


I haven't performed any benchmarks, but I tend to agree with you, 
especially since multiple 'auto ref' parameters lead to 
exponential bloating. I could definitely do without a special 
role for templates, which would further simplify things 
considerably. If performance is really that critical, an explicit 
pass-by-value (move) overload for rvalues ought to be enough 
flexibility imo.
Nov 08 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
That's cute, but it really feels like a hack.
All of a sudden the debugger doesn't work properly anymore, you need to
step-in twice to enter the function, and it's particularly inefficient in
debug builds (a point of great concern for my industry!).

Please just with the compiler creating a temporary in the caller space.
Restrict is to const ref, or better, in ref (scope seems particularly
important here).


On 8 November 2012 20:28, Dmitry Olshansky <dmitry.olsh gmail.com> wrote:


 11/7/2012 3:54 AM, Manu =D0=BF=D0=B8=D1=88=D0=B5=D1=82:

  If the compiler started generating 2 copies of all my ref functions, I'd

 be rather unimpressed... bloat is already a problem in D. Perhaps this
 may be a handy feature, but I wouldn't call this a 'solution' to this
 issue.
 Also, what if the function is external (likely)... auto ref can't work
 if the function is external, an implicit temporary is required in that
 case.


 What's wrong with going this route:

 void blah(auto ref X stuff){
 ...lots of code...
 }

 is magically expanded to:

 void blah(ref X stuff){
 ...that code..
 }

 and

 void blah(X stuff){
         .blah(stuff); //now here stuff is L-value so use the ref version
 }

 Yeah, it looks _almost_ like a template now. But unlike with a template w=


e

 can assume it's 2 overloads _always_. External  fucntion issue is then
 solved by treating it as exactly these 2 overloads (one trampoline, one
 real). Basically it becomes one-line declaration of 2 functions.

 Given that temporaries are moved anyway the speed should be fine and ther=


e

 is as much bloat as you'd do by hand.

 Also hopefully inliner can be counted on to do its thing in this simple
 case.



 --
 Dmitry Olshansky
Nov 08 2012
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, November 08, 2012 21:49:58 Manu wrote:

 That's cute, but it really feels like a hack.
 All of a sudden the debugger doesn't work properly anymore, you need to
 step-in twice to enter the function, and it's particularly inefficient in
 debug builds (a point of great concern for my industry!).
 
 Please just with the compiler creating a temporary in the caller space.
 Restrict is to const ref, or better, in ref (scope seems particularly
 important here).


I honestly wish that in didn't exist in the language. The fact that it it's an 
alias two different attributes is confusing, and people keep using it without 
realizing what they're getting into. If scope worked correctly, you'd only 
want it in specific circumstances, not in general. And since it doesn't work 
correctly aside from delegates, once it _does_ work correctly, it'll break 
code all over the place, because people keep using in, because they like how 
it corresponds with out or whatever.

- Jonathan M Davis
Nov 08 2012
parent "martin" <kinke libero.it> writes:
On Thursday, 8 November 2012 at 22:44:26 UTC, Jonathan M Davis 
wrote:

 I honestly wish that in didn't exist in the language. The fact 
 that it it's an alias two different attributes is confusing, and
 people keep using it without realizing what they're getting 
 into.
 If scope worked correctly, you'd only want it in specific
 circumstances, not in general. And since it doesn't work 
 correctly
 aside from delegates, once it _does_ work correctly, it'll break
 code all over the place, because people keep using in, because 
 they like how it corresponds with out or whatever.


I agree that it may likely be a cause for future issues. I 
wouldn't remove it though, rather relax it to an alias for const 
only (yes, because I like how it corresponds with out (input only 
vs. output only) and especially because it is very short - this 
diff of 3 characters really make a difference in function 
signatures :D). That'd fortunately still be possible without 
breaking existing code.

So please generalize my countless mentionings of 'in ref' to 
'const ref'. ;)
Nov 08 2012
prev sibling parent Manu <turkeyman gmail.com> writes:
On 9 November 2012 00:44, Jonathan M Davis <jmdavisProg gmx.com> wrote:


 On Thursday, November 08, 2012 21:49:58 Manu wrote:

 That's cute, but it really feels like a hack.
 All of a sudden the debugger doesn't work properly anymore, you need to
 step-in twice to enter the function, and it's particularly inefficient in
 debug builds (a point of great concern for my industry!).

 Please just with the compiler creating a temporary in the caller space.
 Restrict is to const ref, or better, in ref (scope seems particularly
 important here).


 I honestly wish that in didn't exist in the language. The fact that it
 it's an
 alias two different attributes is confusing, and people keep using it
 without
 realizing what they're getting into.



I understand it's an alias for 'const scope', but what's this about scope
not working correctly? What's wrong with it?
It seems like precisely what you want in this case... you don't want a
temporary escaping the function you pass it to.
What's the problem?

If scope worked correctly, you'd only

 want it in specific circumstances, not in general. And since it doesn't
 work
 correctly aside from delegates, once it _does_ work correctly, it'll break
 code all over the place, because people keep using in, because they like
 how
 it corresponds with out or whatever.


I like how it promises that the thing I pass can't escape the callee, not
that it's the natural english compliment to 'out'.
Nov 09 2012
prev sibling next sibling parent reply "Rob T" <rob ucora.com> writes:
On Tuesday, 6 November 2012 at 23:37:25 UTC, martin wrote:

 void func(T)(in auto ref T m);

 This template, as I understand it, gets expanded to:

 void func(T)(in ref T m); // for lvalues
 void func(T)(in T m);     // for rvalues

 So for non-templated functions, I suggest 2 options:

 1) The previously described auto-templates (identical 'auto 
 ref' semantics), where a function with 'auto ref' parameters is 
 treated as implicit template. This may lead to code-bloating 
 (for larger functions) and/or higher performance for rvalue 
 arguments (rvalues passed to value arguments are moved, not 
 copied; we therefore gain nothing by passing a reference, but 
 incur a slight performance hit due to pointer indirection 
 instead of accessing directly the rvalue on the stack). OR
 2) Simple under-the-hood temporary lvalue declaration by the 
 compiler (for rvalues passed to 'const ref' parameters) - that 
 would be a handy implementation of the first workaround.

 I hope you get my point. :)


What about the case where we want to pass a source argument 
either by reference or as a copy depending on the l/r value 
situation?

eg
void f( ref a );
void f( a );

--rt
Nov 06 2012
parent reply "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 02:06:09 UTC, Rob T wrote:

 What about the case where we want to pass a source argument 
 either by reference or as a copy depending on the l/r value 
 situation?

 eg
 void f( ref a );
 void f( a );

 --rt


I don't get what you mean - that's why the 2 overloads are for 
(you forgot the const/in keyword - beware! :))

void f( in ref T a );
void f( in T a );

rvalue binds to the latter overload (argument not copied, but 
moved directly).
lvalue binds to the first overload (reference).
Works with v2.060.
Nov 06 2012
parent reply "Rob T" <rob ucora.com> writes:
On Wednesday, 7 November 2012 at 02:34:25 UTC, martin wrote:

 On Wednesday, 7 November 2012 at 02:06:09 UTC, Rob T wrote:

 What about the case where we want to pass a source argument 
 either by reference or as a copy depending on the l/r value 
 situation?

 eg
 void f( ref a );
 void f( a );

 --rt


 I don't get what you mean - that's why the 2 overloads are for 
 (you forgot the const/in keyword - beware! :))

 void f( in ref T a );
 void f( in T a );

 rvalue binds to the latter overload (argument not copied, but 
 moved directly).
 lvalue binds to the first overload (reference).
 Works with v2.060.


Sorry, my explanation was very poor. I'll try again.

I'm trying to describe the case where you want to modify the ref 
argument so that the source is also changed. For example, let's 
say you want to move a resource instead of coping it. Moving 
means the resource is transfered from one object to another, and 
the original is reset without destoying the resource. That way 
only one object can have the same resource. So for moving a 
resource, you cannot use "in ref" or "const ref", you need to 
just use "ref".

void f( ref T a ){
   // take resource away from a
   this.resource = a.resource;
   a.resetResource();
   // this now owns the resource
}

ref T works fine, but if you wish to use f( ref T a ) on a temp 
value returned from another function call, you'll need to 
overload f() to pass by value, which means creating a duplicate.

void f( ref T a ){
   // take resource away from a
   this.resource = a.resource;
   a.resetResource();
   // this now owns the resource
}

Example:

T g(){
    T temp;
    // create resource which is stored inside T
    temp.createResource();
    // move temp contents to return value
    return move(temp);
}

f( g() ); // won't compile with ref

It's annoying to have to create overloaded duplicates for a 
situation like moving, esp if the duplicate is a lot of code. In 
C++ the move problem was solved using && move semantics, and this 
is what I'm trying to emulate in D but I feel like I'm fighting 
with the language.

It is possible that D has a specific way of solving this problem 
that I'm not aware of yet, so correct me if I'm asking for 
something that is simply not needed.

I tried f( move(g()) ) but that fails to work. My best guess is 
that D does a hidden move of the temp instead of a copy to value. 
I can't say for sure because the documentation is not clear and 
is missing important details like this. I also cannot rely on 
clever compiler optimizations that may or may not be implemented 
as a guarantee.

--rt
Nov 06 2012
parent reply "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 06:13:25 UTC, Rob T wrote:

 ref T works fine, but if you wish to use f( ref T a ) on a temp 
 value returned from another function call, you'll need to 
 overload f() to pass by value, which means creating a duplicate.


Duplicating the function, yes, but not duplicating the rvalue 
since it is moved in D. Isn't this analog to the C++ solution?

C++:
void f(T& a) { // for lvalues
     this->resource = a.resource;
     a.resetResource();
}
void f(T&& a) { // for rvalues (moved)
     this->resource = a.resource;
     a.resetResource();
}

D:
void f(ref T a) { // for lvalues
     this.resource = a.resource;
     a.resetResource();
}
void f(T a) { // rvalue argument is not copied, but moved
     this.resource = a.resource;
     a.resetResource();
}

T g() {
     T temp;
     temp.createResource();
     return temp; // D already moves temp!
                  // 'Named Return Value Optimization'
}


 I tried f( move(g()) ) but that fails to work. My best guess is 
 that D does a hidden move of the temp instead of a copy to 
 value. I can't say for sure because the documentation is not 
 clear and is missing important details like this. I also cannot 
 rely on clever compiler optimizations that may or may not be 
 implemented as a guarantee.


You could implement a copy constructor 'this(this)' in your 
struct T and see when it is invoked to check when an instance is 
actually copied. I'd expect that invoking 'f(g());' with above 
implementation doesn't copy anything.
Nov 07 2012
next sibling parent reply "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 14:07:31 UTC, martin wrote:

 C++:
 void f(T& a) { // for lvalues
     this->resource = a.resource;
     a.resetResource();
 }
 void f(T&& a) { // for rvalues (moved)
     this->resource = a.resource;
     a.resetResource();
 }

 D:
 void f(ref T a) { // for lvalues
     this.resource = a.resource;
     a.resetResource();
 }
 void f(T a) { // rvalue argument is not copied, but moved
     this.resource = a.resource;
     a.resetResource();
 }


You could probably get away with a single-line overload, both in 
C++ and D:

C++:
void f(T& a) { // for lvalues
     // convert a to mutable rvalue reference and
     // invoke the main overload f(T&&)
     f(std::move(a));
}

D:
void f(T a) { // rvalue argument is not copied, but moved
     // the original argument is now named a (an lvalue)
     // invoke the main overload f(ref T)
     f(a);
}
Nov 07 2012
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/7/2012 9:04 PM, martin пишет:

 On Wednesday, 7 November 2012 at 14:07:31 UTC, martin wrote:

 C++:
 void f(T& a) { // for lvalues
     this->resource = a.resource;
     a.resetResource();
 }
 void f(T&& a) { // for rvalues (moved)
     this->resource = a.resource;
     a.resetResource();
 }

 D:
 void f(ref T a) { // for lvalues
     this.resource = a.resource;
     a.resetResource();
 }
 void f(T a) { // rvalue argument is not copied, but moved
     this.resource = a.resource;
     a.resetResource();
 }


 You could probably get away with a single-line overload, both in C++ and D:

 C++:
 void f(T& a) { // for lvalues
      // convert a to mutable rvalue reference and
      // invoke the main overload f(T&&)
      f(std::move(a));
 }

 D:
 void f(T a) { // rvalue argument is not copied, but moved
      // the original argument is now named a (an lvalue)
      // invoke the main overload f(ref T)
      f(a);
 }


Yup, and I'd like auto ref to actually do this r-value trampoline for me.

-- 
Dmitry Olshansky
Nov 08 2012
prev sibling parent "Rob T" <rob ucora.com> writes:
On Wednesday, 7 November 2012 at 14:07:31 UTC, martin wrote:


 T g() {
     T temp;
     temp.createResource();
     return temp; // D already moves temp!
                  // 'Named Return Value Optimization'
 }


OK, it seems to be working as you describe.

The main question I have is if I can rely on this behaviour as a 
feature provided by the languge specification vs an opportunistic 
compiler optimization that may or may not happen depending on the 
implementation?

The general problem I'm having with with D, is I need a gaurantee 
that certain behaviors will always be performed when moves/copies 
are done, but it's just not clear if this is the case or not. 
There seems to be a lack of a concise language specification, or 
is there one defined somewhere?


 You could implement a copy constructor 'this(this)' in your 
 struct T and see when it is invoked to check when an instance 
 is actually copied. I'd expect that invoking 'f(g());' with 
 above implementation doesn't copy anything.


I am doing that, but I cannpt seem to hook into the part where a 
move does an "init" on the struct or class. I sort-of can see it 
though, if I set a member value to a known state before the move, 
then display it to console or log after the move during 
destruction. If it was init'ed then I'll see the default value, 
and this seems to be happening as described.

Anyway, what I was hoping for with "auto ref" was for the 
compiler to selectively decide if the ref part should be used or 
not depending on the situation, rather than me manually writing 
two functions to do the exact same thing. What's discussed in 
here about auto ref is something else, although I agree it is 
definitely needed.

--rt
Nov 07 2012
prev sibling parent reply "martin" <kinke libero.it> writes:
On Tuesday, 6 November 2012 at 23:37:25 UTC, martin wrote:

 1) The previously described auto-templates (identical 'auto 
 ref' semantics), where a function with 'auto ref' parameters is 
 treated as implicit template. This may lead to code-bloating 
 (for larger functions) and/or higher performance for rvalue 
 arguments (rvalues passed to value arguments are moved, not 
 copied; we therefore gain nothing by passing a reference, but 
 incur a slight performance hit due to pointer indirection 
 instead of accessing directly the rvalue on the stack). OR
 2) Simple under-the-hood temporary lvalue declaration by the 
 compiler (for rvalues passed to 'const ref' parameters) - that 
 would be a handy implementation of the first workaround.


Please discard option 1, I'm afraid it creates too much confusion 
and was not well thought through. The objective was to expand 
'auto ref T' to either 'in ref T' for lvalues or 'in T' for 
rvalues (i.e., only for const parameters!), but then its caption 
would be horribly misleading (and something like 'in auto ref' 
would be way too ambiguous).
Nov 06 2012
parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Wednesday, 7 November 2012 at 02:45:15 UTC, martin wrote:

 Please discard option 1, I'm afraid it creates too much 
 confusion and was not well thought through. The objective was 
 to expand 'auto ref T' to either 'in ref T' for lvalues or 'in 
 T' for rvalues (I.e., only for const parameters!), but then its 
 caption would be horribly misleading (and something like 'in 
 auto ref' would be way too ambiguous).



  Maybe a very simple change/addition; Like perhaps  ref? 
(Attribute ref, Alternate ref, auto ref.. All sorta fit for it's 
meaning).

  So...

  void func1(ref int x);  //D lvalue-only ref
  void func2( ref int x); //works like c++'s ref

  Seems fairly easy to tell apart, and still leaves const-ness as 
an option.
Nov 06 2012
parent reply "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 02:58:57 UTC, Era Scarecrow 
wrote:

 Maybe a very simple change/addition; Like perhaps  ref? 
 (Attribute ref, Alternate ref, auto ref.. All sorta fit for 
 it's meaning).

 So...

 void func1(ref int x);  //D lvalue-only ref
 void func2( ref int x); //works like c++'s ref

 Seems fairly easy to tell apart, and still leaves const-ness as 
 an option.


Afaik C++ doesn't allow rvalues to be passed to _mutable_ 
references (T&), only to const references, making perfect sense 
imo. I really do not see the point for an additional syntax for 
C++-like const references (const T&) which would also take 
rvalues.
Please give me an example where you want to pass an rvalue to a 
_mutable_ reference parameter. I would simply continue to 
disallow that, since that would mean that changes to the 
referenced rvalue would not be visible for the caller (a 
temporary/literal is changed - how could someone possibly want 
that?).
Nov 06 2012
next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Wednesday, 7 November 2012 at 03:13:22 UTC, martin wrote:

 void func1(ref int x);  //D lvalue-only ref
 void func2( ref int x); //works like c++'s ref

 Seems fairly easy to tell apart, and still leaves const-ness 
 as an option.


 Afaik C++ doesn't allow rvalues to be passed to _mutable_ 
 references (T&), only to const references, making perfect sense 
 imo. I really do not see the point for an additional syntax for 
 C++-like const references (const T&) which would also take 
 rvalues.



 Please give me an example where you want to pass an rvalue to a 
 _mutable_ reference parameter. I would simply continue to 
 disallow that, since that would mean that changes to the 
 referenced rvalue would not be visible for the caller (a 
 temporary/literal is changed - how could someone possibly want 
 that?).


  Still that zlib entry is coming to mind. But more importantly is 
that you still need code duplication to get both accessible.

  //many possible combinations thereof
  int func(const ref x);

  int func(int x) {
    return func(cast(const int) x);
  }


  But regarding zlib.. It's function is something like:
  int compress(char *output, int *size, char *input, int 
inputSize);

  So... if we convert that to something similar we get..

  enum ZlibEnum {}
  ZlibEnum compress(void[] output, void[] input, &ref Zlib state) 
nothrow pure;

  The idea in this case is 'state' would continue to hold the 
input/output pointers and all information needed, so you could 
continue to use it and if it has any 'unflushed' data (output 
size too small?) then it could retain that. however if you knew 
you didn't need it you could ignore it.

  ZlibEnum could be for output C calling code, so success/failure 
needs to be known right away. It makes sense for it to return a 
Zlib type as well, but depends on what has higher priority and 
why.

  string input = "something long";
  ubyte[1000] output;
  ubyte[5] output2;
  Zlib state;

  compress(output, input, null); //we know the buffer is large 
enough
  compress(output, input, state);
Nov 06 2012
prev sibling next sibling parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Wednesday, 7 November 2012 at 03:13:22 UTC, martin wrote:

 void func1(ref int x);  //D lvalue-only ref
 void func2( ref int x); //works like c++'s ref

 Seems fairly easy to tell apart, and still leaves const-ness 
 as an option.


 Afaik C++ doesn't allow rvalues to be passed to _mutable_ 
 references (T&), only to const references, making perfect sense 
 imo. I really do not see the point for an additional syntax for 
 C++-like const references (const T&) which would also take 
 rvalues.



 Please give me an example where you want to pass an rvalue to a 
 _mutable_ reference parameter. I would simply continue to 
 disallow that, since that would mean that changes to the 
 referenced rvalue would not be visible for the caller (a 
 temporary/literal is changed - how could someone possibly want 
 that?).


   Still that zlib entry is coming to mind. But more importantly is
that you still need code duplication to get both accessible.

   //many possible combinations thereof
   int func(const ref x);

   int func(int x) {
     return func(cast(const int) x);
   }


   But regarding zlib.. It's function is something like:
   int compress(char *output, int *size, char *input, int
inputSize);

   So... if we convert that to something similar we get..

   enum ZlibEnum {}
   ZlibEnum compress(void[] output, void[] input, &ref Zlib state)
nothrow pure;

   The idea in this case is 'state' would continue to hold the
input/output pointers and all information needed, so you could
continue to use it and if it has any 'unflushed' data (output
size too small?) then it could retain that. however if you knew
you didn't need it you could ignore it.

   ZlibEnum could be for output C calling code, so success/failure
needs to be known right away. It makes sense for it to return a
Zlib type as well, but depends on what has higher priority and
why.

   string input = "something long";
   ubyte[1000] output;
   ubyte[5] output2;
   Zlib state;

   compress(output, input, null); //we know the buffer is large
enough
   compress(output2, input, state);
Nov 06 2012
parent reply "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 03:35:19 UTC, Era Scarecrow 
wrote:

 Still that zlib entry is coming to mind. But more importantly
 is that you still need code duplication to get both accessible.

 //many possible combinations thereof
 int func(const ref x);

 int func(int x) {
   return func(cast(const int) x);
 }


int func(in ref int x);
int func(int x) { return func(x); }

The latter overload is for rvalues (no need to cast to const) and 
wouldn't be required if the first one took rvalues directly 
(having a _const_ ref parameter). That's not an example for 
mutable references though, it's basically a shortcut so as to not 
having to declare all rvalues for x manually.


 enum ZlibEnum {}
 ZlibEnum compress(void[] output, void[] input,
   &ref Zlib state) nothrow pure;

 string input = "something long";
 ubyte[1000] output;
 ubyte[5] output2;
 Zlib state;

 compress(output, input, null);
 compress(output2, input, state);


Where's the rvalue? You're talking about an optional mutable 
reference here, which is only doable via a pointer (references 
cannot be null, not in C++ and not in D):

ZlibEnum compress(void[] output, void[] input, Zlib* state = 
null);
...
compress(output, input);
compress(output2, input, &state);

That would be correct. But you're completely missing the point 
here. Let's assume you wanted to require a state to be passed:

ZlibEnum compress(void[] output, void[] input, ref Zlib state);

You could now use:

Zlib state; // lvalue
compress(output, input, state);

but not:

compress(output, input, Zlib()); // rvalue

And that is most likely a good thing, since your state would be 
lost after the compress() call. If you don't want it, you don't 
pass it, and as I said, the only way to do that is to pass a 
nullable pointer. So this is off-topic I'm afraid.
Nov 06 2012
parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Wednesday, 7 November 2012 at 04:05:32 UTC, martin wrote:

 int func(in ref int x);
 int func(int x) { return func(x); }

 The latter overload is for rvalues (no need to cast to const) 
 and wouldn't be required if the first one took rvalues directly 
 (having a _const_ ref parameter). That's not an example for 
 mutable references though, it's basically a shortcut so as to 
 not having to declare all rvalues for x manually.


  Maybe... But when working with non built-in types can you 
guarantee that behavior?

//same as 'const ref' basically, unless 'in ref' is accepted
struct S
//int func(in ref S x); //?
int func(const ref S x);
int func(S x) { return func(x); } //potentially infinitely self 
calling


 compress(output, input, null);
 compress(output2, input, state);


 Where's the rvalue? You're talking about an optional mutable 
 reference here, which is only doable via a pointer (references 
 cannot be null, not in C++ and not in D):


  Perhaps null is wrong in this case, but had it been Zlib.init or 
Zlib(), then it would still be applicable. Since it's a reference 
I would think it could accept a null pointer and realize to 
create a temporary which has it's default values.


 That would be correct. But you're completely missing the point 
 here. Let's assume you wanted to require a state to be passed:

 ZlibEnum compress(void[] output, void[] input, ref Zlib state);

 You could now use:

 Zlib state; // lvalue
 compress(output, input, state);

 but not:

 compress(output, input, Zlib()); // rvalue

 And that is most likely a good thing, since your state would be 
 lost after the compress() call. If you don't want it, you don't 
 pass it, and as I said, the only way to do that is to pass a 
 nullable pointer. So this is off-topic I'm afraid.


  True, but rather than having to create a temporary you don't 
plan on using just to satisfy the signature, and you know you 
don't NEED it afterwards, why do you have to go through the extra 
steps? How many wrappers are made in general just to work around 
minor issues like this?

  Quite often we ignore return types if they aren't interesting, 
but we can't ignore a rvalue we only need for one call (or just 
to satisfy the signature)? Say we have a divide function (who 
knows what for), and this is more efficient than normal. So...

  //perhaps ' ref int remainder = 0'?
  int divide(int number, int divisor, ref int remainder);

  Now in this case what if we don't care about the remainder? 
Create a temporary (or a wrapper function)?

  //satisfy remainder
  int divide(int number, int divisor) {
    int tmp; return divide(number, divisor, tmp);
  }

  int modulus(int number, int divisor) {
    int tmp; divide(number, divisor, tmp);
    return tmp;
  }

  But if we want the result of the division AND the remainder 
should we have to make a separate modulus function when the 
original divide can clearly give you the answer? This cuts closer 
to the instruction set as an example.
Nov 06 2012
parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 04:36:59 UTC, Era Scarecrow 
wrote:

 Maybe... But when working with non built-in types can you 
 guarantee that behavior?


Sure, try it out.

What you are referring to here has nothing to do with rvalues. It 
is about side results which are only sometimes needed. It is 
clear that you have to use lvalue arguments for those in order to 
access them after the function call. Using bogus rvalue arguments 
in case you're not interested in them wouldn't really help - 
you'd save the variable declaration, but the function signature 
would still be ugly, and you can't always provide hypothetical 
default values to hide them (depending on parameter order). In my 
experience, these cases are very rare anyway (unless dealing with 
porting old C-style code which you seem to be doing) and function 
overloading is the most elegant way to handle it - take a look at 
the large number of overloads in modern language libraries such 
as .NET.

This is how I would implement it:

// magical, super-fast function ;)
int divide(in int number, in int divisor, int* remainder = null);
auto bla = divide(13, 4); // => 3

// remainder needed as well, hiding the pointer as reference:
int divide(in int number, in int divisor, out int remainder)
{
     return divide(number, divisor, &remainder);
}
int remainder;
auto bla = divide(13, 4, remainder); // => 3, remainder = 1

// only remainder needed:
int modulus(in int number, in int divisor)
{
     int tmp;
     divide(number, divisor, tmp);
     return tmp;
}
auto bla = modulus(13, 4); // => 1

But this is all really off-topic here, let's stop spamming this 
thread.
Nov 06 2012
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/07/2012 04:13 AM, martin wrote:

 On Wednesday, 7 November 2012 at 02:58:57 UTC, Era Scarecrow wrote:

 Maybe a very simple change/addition; Like perhaps  ref? (Attribute
 ref, Alternate ref, auto ref.. All sorta fit for it's meaning).

 So...

 void func1(ref int x);  //D lvalue-only ref
 void func2( ref int x); //works like c++'s ref

 Seems fairly easy to tell apart, and still leaves const-ness as an
 option.


 Afaik C++ doesn't allow rvalues to be passed to _mutable_ references
 (T&), only to const references, making perfect sense imo. I really do
 not see the point for an additional syntax for C++-like const references
 (const T&) which would also take rvalues.


You are still missing that const in C++ is different from const in D.
Also, if the point is to have higher speed, why shouldn't the function 
be allowed to use an rvalue as scratch space without a _deep copy_ ?

const in C++ does not mean anything. It is just loosely enforced 
interface documentation. const in D actually restricts what the callee 
can do with the argument, in a transitive fashion.


 Please give me an example where you want to pass an rvalue to a
 _mutable_ reference parameter.



When my struct does not support any operations that are const, and 
creating a mutable copy is not possible due to indirections?


 I would simply continue to disallow that,
 since that would mean that changes to the referenced rvalue would not be
 visible for the caller (a temporary/literal is changed - how could
 someone possibly want that?).


The change may well be visible...

class C{
     int x;
}
struct W{
     C c;
}

W createW(C c){ return W(c); }
void foo(ref W w){ w.c.x = 2; }

void main(){
     C c = new C;
     assert(c.x==0);
     foo(createW(c));
     assert(c.x==2);
}
Nov 07 2012
next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Wednesday, 7 November 2012 at 10:33:03 UTC, Timon Gehr wrote:

 You are still missing that const in C++ is different from const 
 in D. Also, if the point is to have higher speed, why shouldn't 
 the function be allowed to use an rvalue as scratch space 
 without a _deep copy_ ?

 const in C++ does not mean anything. It is just loosely 
 enforced interface documentation. const in D actually restricts 
 what the callee can do with the argument, in a transitive 
 fashion.


  Also depending on how you think of it, the const ref in C++ 
logically it didn't make sense to pass a mutable rvalue; Say you 
pass the number 3, you can't modify it logically (ie you can't 
redefine PI afterall).


 When my struct does not support any operations that are const, 
 and creating a mutable copy is not possible due to indirections?


 I would simply continue to disallow that, since that would 
 mean that changes to the referenced rvalue would not be 
 visible for the caller (a temporary/literal is changed - how 
 could someone possibly want that?).


 The change may well be visible...

 class C{
     int x;
 }
 struct W{
     C c;
 }

 W createW(C c){ return W(c); }
 void foo(ref W w){ w.c.x = 2; }

 void main(){
     C c = new C;
     assert(c.x==0);
     foo(createW(c));
     assert(c.x==2);
 }


  So basically being named (or not) doesn't change the data, what 
it is or how it's used, only how accessible it is.

  There's some forms of techniques that don't seem to have any 
value until you have a chance to think about it and use them; 
Like scopes, overloading, referencing, Exceptions you can easily 
do without them all (We have C afterall), but it isn't as much 
fun or clean. Some features and uses thereof make more sense for 
the back/private implementation rather than the public interface.

  The reason I gave my 'divide' example is because a while back I 
wrote a BitInt (in C like 10 years ago, and a bit buggy); The 
divide function actually calculated the remainder as a side 
effect, and using wrappers or writing them as separate portions 
would actually be harder and slower than joined as it was.
Nov 07 2012
prev sibling parent reply "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 10:33:03 UTC, Timon Gehr wrote:

 You are still missing that const in C++ is different from const 
 in D.
 const in C++ does not mean anything. It is just loosely 
 enforced interface documentation. const in D actually restricts 
 what the callee can do with the argument, in a transitive 
 fashion.


I still don't get the big difference (except for transitiveness 
for pointers). Given a const reference, I'm only able to invoke 
methods decorated with the const keyword (or inout in D) keyword, 
both in C++ and D. And I can only pass it as argument by ref to 
functions which do not alter it (also taking a const reference, 
that is).


 Also, if the point is to have higher speed, why shouldn't the 
 function be allowed to use an rvalue as scratch space without a 
 _deep copy_ ?


I'm sorry but I don't get what you mean here. Could you please 
elaborate on this?


 When my struct does not support any operations that are const, 
 and creating a mutable copy is not possible due to indirections?


If your struct doesn't support any const operations, it most 
likely has good reasons not to.


 The change may well be visible...


True in this case, but only if you know exactly what foo() does 
when you call it inside the main() function. And that is probably 
an indicator for bad encapsulation - most of the time, you 
shouldn't have the knowledge how foo() is exactly implemented 
when using it from the outside.
It is clear that there are some examples where you want to pass 
an rvalue argument to a mutable ref parameter if you know exactly 
what the function does. But imo these cases are very rare and I 
don't really regard it as big issue if you need to add a line 
'auto tmp = myRvalue;' before the function call to transform it 
to a referenceable lvalue, in these few cases.
Much more commonly, you need a parameter just as read-only input. 
Consider a real-word-example of a 4x4 matrix consisting of 16 
doubles (128 bytes). Most of the time, you'd only need a 
read-only input instance when working with it (combining 
matrices, transforming vectors etc.). Given its size (it's not 
really huge, I acknowledge that ;)), you probably want to avoid 
copying it around and therefore pass it by ref, but want that to 
also work for rvalues (produced by matrix combinations like 
'viewMatrix * modelMatrix', for example).

struct Matrix
{
     double[16] data;

     // this op= other
     ref Matrix opOpAssign(string op)(in ref Matrix other);

     // Matrix result = this op other
     Matrix opBinary(string op)(in ref Matrix other) const;

     // double4 result = this * vector
     // the vector (32 bytes) may be passed by value for AVX
     double4 opBinary(string op)(in ref double4 vector) const
         if (op == "*");
};
Nov 07 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/07/2012 04:01 PM, martin wrote:

 On Wednesday, 7 November 2012 at 10:33:03 UTC, Timon Gehr wrote:

 You are still missing that const in C++ is different from const in D.
 const in C++ does not mean anything. It is just loosely enforced
 interface documentation. const in D actually restricts what the callee
 can do with the argument, in a transitive fashion.


 I still don't get the big difference (except for transitiveness for
 pointers).


That is the main thing, but C++ also has a 'mutable' keyword.


 Given a const reference, I'm only able to invoke methods
 decorated with the const keyword (or inout in D) keyword, both in C++
 and D. And I can only pass it as argument by ref to functions which do
 not alter it (also taking a const reference, that is).


This is not the case in C++ for the D definition of 'not alter'.


 Also, if the point is to have higher speed, why shouldn't the function
 be allowed to use an rvalue as scratch space without a _deep copy_ ?


 I'm sorry but I don't get what you mean here. Could you please elaborate
 on this?


Well, if the changes are not visible, the function can freely change the 
memory in order to perform its computations.


 When my struct does not support any operations that are const, and
 creating a mutable copy is not possible due to indirections?


 If your struct doesn't support any const operations, it most likely has
 good reasons not to.


Either that, or someone hasn't bothered to annotate.


 The change may well be visible...


 True in this case, but only if you know exactly what foo() does when you
 call it inside the main() function. And that is probably an indicator
 for bad encapsulation


The method is called 'foo', that means it is a toy example to show that 
the change may be visible. The change does not necessarily have to be 
visible to the caller directly.


 - most of the time, you shouldn't have the
 knowledge how foo() is exactly implemented when using it from the outside.


You got it exactly reverse. const harms encapsulation because it exposes 
some details about implementations. It is often not applicable in 
sufficiently dynamic code.



 It is clear that there are some examples where you want to pass an
 rvalue argument to a mutable ref parameter if you know exactly what the
 function does. But imo these cases are very rare and I don't really
 regard it as big issue if you need to add a line 'auto tmp = myRvalue;'
 before the function call to transform it to a referenceable lvalue, in
 these few cases.


You do not see issues with changing the interface based on 
implementation details?


 Much more commonly, you need a parameter just as read-only input.
 Consider a real-word-example of a 4x4 matrix consisting of 16 doubles
 (128 bytes). Most of the time, you'd only need a read-only input
 instance when working with it (combining matrices, transforming vectors
 etc.). Given its size (it's not really huge, I acknowledge that ;)), you
 probably want to avoid copying it around and therefore pass it by ref,
 but want that to also work for rvalues (produced by matrix combinations
 like 'viewMatrix * modelMatrix', for example).

 struct Matrix
 {
      double[16] data;

      // this op= other
      ref Matrix opOpAssign(string op)(in ref Matrix other);

      // Matrix result = this op other
      Matrix opBinary(string op)(in ref Matrix other) const;

      // double4 result = this * vector
      // the vector (32 bytes) may be passed by value for AVX
      double4 opBinary(string op)(in ref double4 vector) const
          if (op == "*");
 };


I know your use case, and note that I do not have any issues with 
rvalues always being allowed to bind to ref parameters, which would 
solve your problem in all cases. (others do not like that though, which 
is why we have the current situation and your C++ - inspired design is 
discussed every second week or so.)
Nov 07 2012
parent reply "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 16:57:47 UTC, Timon Gehr wrote:

 That is the main thing, but C++ also has a 'mutable' keyword.


Right, I forgot this inconspicuous little keyword. It really is a 
huge hole in C++'s const system.


 If your struct doesn't support any const operations, it most 
 likely has good reasons not to.


 Either that, or someone hasn't bothered to annotate.


Hehe, yeah, the latter being more likely.


 You got it exactly reverse. const harms encapsulation because 
 it exposes some details about implementations. It is often not 
 applicable in sufficiently dynamic code.


I don't see it that way. I find it very useful to know that an 
argument won't be modified. Example: suppose we have a large 
array (static array or wrapped in a struct) and need it as input 
for some independent operations. By knowing it won't be touched 
(passing by const ref) it is immediately clear that the 
operations can be run in parallel. Otherwise, one would need to 
go through the operations' implementation to determine if 
parallelization is possible.


 You do not see issues with changing the interface based on 
 implementation details?


I find it okay to require all possible changes to an argument 
passed by reference to be required to be directly visible by the 
caller, just to prevent accidentally missing side effects. 
Suppose a function takes a const reference parameter and is 
called at a site using an rvalue. After some time, that function 
is updated so that its parameter is or may be changed, changing 
the const ref parameter to a ref parameter. That may lead to big 
issues for the call sites. A compiler error would make sure the 
affected call sites are inspected and updated instead of 
potentially introducing regressions.
Nov 07 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/07/2012 06:52 PM, martin wrote:

 On Wednesday, 7 November 2012 at 16:57:47 UTC, Timon Gehr wrote:

 That is the main thing, but C++ also has a 'mutable' keyword.


 Right, I forgot this inconspicuous little keyword. It really is a huge
 hole in C++'s const system.


 If your struct doesn't support any const operations, it most likely
 has good reasons not to.


 Either that, or someone hasn't bothered to annotate.


 Hehe, yeah, the latter being more likely.


 You got it exactly reverse. const harms encapsulation because it
 exposes some details about implementations. It is often not applicable
 in sufficiently dynamic code.


 I don't see it that way.


You do not give a justification.


 I find it very useful to know that an argument
 won't be modified.


That is unrelated and does not necessarily follow, but that it does in 
some cases is the main reason why const is useful even though it weakens 
encapsulation.


 Example: suppose we have a large array (static array
 or wrapped in a struct) and need it as input for some independent
 operations. By knowing it won't be touched (passing by const ref) it is
 immediately clear that the operations can be run in parallel. Otherwise,
 one would need to go through the operations' implementation to determine
 if parallelization is possible.


You have to do that anyway if it the operation is not at least 'const 
pure' and you need to guarantee that no other thread is in fact 
modifying the data. But even then I do not see how that helps 
encapsulation. Naive parallelization is just one of the examples that 
shows that encapsulation can be harmful in some specific cases.



 You do not see issues with changing the interface based on
 implementation details?


 I find it okay to require all possible changes to an argument passed by
 reference to be required to be directly visible by the caller, just to
 prevent accidentally missing side effects. Suppose a function takes a
 const reference parameter and is called at a site using an rvalue. After
 some time, that function is updated so that its parameter is or may be
 changed, changing the const ref parameter to a ref parameter. That may
 lead to big issues for the call sites. A compiler error would make sure
 the affected call sites are inspected and updated instead of potentially
 introducing regressions.


Often there is nothing to be seen for the caller. (lazy initialization, 
caching, internal state update part of a larger computation, ...)
Nov 07 2012
parent reply "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 18:07:27 UTC, Timon Gehr wrote:

 You do not give a justification.


 I find it very useful to know that an argument
 won't be modified.


 That is unrelated and does not necessarily follow, but that it 
 does in some cases is the main reason why const is useful even 
 though it weakens encapsulation.


Let's look at encapsulation this way: suppose my function is a 
colleague and his job is to summarize a book I lend him (my 
reference parameter). I don't care how he does it as long as the 
summary (function result) is what I expect. But I do care about 
my book if I plan on using it afterwards (so in case the argument 
is an lvalue used later on) - will I have to check it for 
defects, coffee spillings etc., or can I be sure it's exactly as 
it was before (const)? This is how I see it.
The thing about mutable rvalue references would be unknown or 
unthought-of side effects: what if the book is the same, but my 
colleague, a known psychopath, killed the author (a property of 
the book reference, a reference itself)? I don't use the book 
anymore, okay (it was an rvalue after all), but it may still be 
worth knowing that the author has gone. Due to the transitiveness 
of const, the (const) author would still be alive if the book 
reference had been const.


 You have to do that anyway if it the operation is not at least 
 'const pure' and you need to guarantee that no other thread is 
 in fact modifying the data.


I think we can agree that knowing the parameters aren't touched 
does help a lot in this case.
Nov 07 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/07/2012 07:48 PM, martin wrote:

 On Wednesday, 7 November 2012 at 18:07:27 UTC, Timon Gehr wrote:

 You do not give a justification.


 I find it very useful to know that an argument
 won't be modified.


 That is unrelated and does not necessarily follow, but that it does in
 some cases is the main reason why const is useful even though it
 weakens encapsulation.


 Let's look at encapsulation this way: suppose my function is a colleague
 and his job is to summarize a book I lend him (my reference parameter).
 I don't care how he does it as long as the summary (function result) is
 what I expect. But I do care about my book if I plan on using it
 afterwards (so in case the argument is an lvalue used later on) - will I
 have to check it for defects, coffee spillings etc., or can I be sure
 it's exactly as it was before (const)?


You can pass him an object that does not support operations you want to 
preclude. He does not have to _know_, that your book is not changed when 
he reads it. This is an implementation detail. In fact, you could make 
the book save away his reading schedule without him noticing.


 This is how I see it.
 The thing about mutable rvalue references would be unknown or
 unthought-of side effects: what if the book is the same, but my
 colleague, a known psychopath, killed the author (a property of the book
 reference, a reference itself)? I don't use the book anymore, okay (it
 was an rvalue after all), but it may still be worth knowing that the
 author has gone. Due to the transitiveness of const, the (const) author
 would still be alive if the book reference had been const.


I'd assume that the book you pass him would not support changing the author.


 You have to do that anyway if it the operation is not at least 'const
 pure' and you need to guarantee that no other thread is in fact
 modifying the data.


 I think we can agree that knowing the parameters aren't touched does
 help a lot in this case.


Maybe, but the fact that we know it harms encapsulation.
Nov 07 2012
parent reply "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 21:39:52 UTC, Timon Gehr wrote:

 You can pass him an object that does not support operations you 
 want to preclude. He does not have to _know_, that your book is 
 not changed when he reads it. This is an implementation detail. 
 In fact, you could make the book save away his reading schedule 
 without him noticing.


I don't see where you want to go with this. Do you suggest 
creating tailored objects (book variants) for each function 
you're gonna pass it to just to satisfy perfect theoretical 
encapsulation? So foo() shouldn't be able to change the author => 
change from inout author reference to const reference? bar() 
should only be allowed to read the book title, not the actual 
book contents => hide that string? ;) For the sake of simplicity, 
by using const we have the ability to at least control if the 
object can be modified or not. So although my colleague doesn't 
have to _know_ that he can't modify my book in any way (or know 
that the book is modifiable in the first place), using const is a 
primitive but practical way for me to prevent him from doing so.

In the context of this rvalue => (const) ref discussion, const is 
useful due to a number of reasons.

1) All possible side effects of the function invokation are 
required to be directly visible by the caller. Some people may 
find that annoying, but I find it useful, and there's a 
single-line workaround (lvalue declaration) for the (in my 
opinion very rare) cases where a potential side-effect is either 
known not to occur or simply uninteresting (requiring exact 
knowledge about the function implementation, always, i.e., during 
the whole life-time of that code!).

2) Say we pass a literal string (rvalue) to a const ref 
parameter. The location of the string in memory can then be 
freely chosen by the compiler, possibly in a static data segment 
of the binary (literal optimization - only one location for 
multiple occurrences). If the parameter was a mutable ref, the 
compiler should probably allocate a copy on the stack before 
calling the function, otherwise the literal may not be the same 
when accessed later on, potentially causing funny bugs.

3) Implicit type conversion isn't a problem. Say we pass an int 
rvalue to a mutable double ref parameter. The parameter will then 
be a reference to another rvalue (the int cast to a double) and 
altering it (the hidden double rvalue) may not really be what the 
coder intended. Afaik D doesn't support implicit casting for 
user-defined types, so that may not be a problem (for now at 
least).
Nov 07 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/08/2012 02:45 AM, martin wrote:

 On Wednesday, 7 November 2012 at 21:39:52 UTC, Timon Gehr wrote:

 You can pass him an object that does not support operations you want
 to preclude. He does not have to _know_, that your book is not changed
 when he reads it. This is an implementation detail. In fact, you could
 make the book save away his reading schedule without him noticing.


 I don't see where you want to go with this. Do you suggest creating
 tailored objects (book variants) for each function you're gonna pass it
 to just to satisfy perfect theoretical encapsulation?


No. The point is that the language should _support_ what you call 
'perfect theoretical encapsulation'.


 So foo() shouldn't
 be able to change the author => change from inout author reference to
 const reference? bar() should only be allowed to read the book title,
 not the actual book contents => hide that string? ;) For the sake of
 simplicity, by using const we have the ability to at least control if
 the object can be modified or not.


It is not _just_ the object. Anyway, this is what I stated in my last post.


 So although my colleague doesn't have
 to _know_ that he can't modify my book in any way (or know that the book
 is modifiable in the first place), using const is a primitive but
 practical way for me to prevent him from doing so.


It also weakens encapsulation, which was the point.



 In the context of this rvalue => (const) ref discussion, const is useful
 due to a number of reasons.

 1) All possible side effects of the function invokation are required to
 be directly visible by the caller. Some people may find that annoying,
 but I find it useful, and there's a single-line workaround (lvalue
 declaration) for the (in my opinion very rare) cases where a potential
 side-effect is either known not to occur or simply uninteresting
 (requiring exact knowledge about the function implementation, always,
 i.e., during the whole life-time of that code!).


Wrong. Not everything is a perfect value type. (and anyway, the code 
that actually will observe the change may be a few frames up the call 
stack.)


 2) Say we pass a literal string (rvalue) to a const ref parameter. The
 location of the string in memory can then be freely chosen by the
 compiler, possibly in a static data segment of the binary (literal
 optimization - only one location for multiple occurrences). If the
 parameter was a mutable ref, the compiler should probably allocate a
 copy on the stack before calling the function, otherwise the literal may
 not be the same when accessed later on, potentially causing funny bugs.


Ambiguous to me and all the interpretations are either wrong or irrelevant.


 3) Implicit type conversion isn't a problem. Say we pass an int rvalue
 to a mutable double ref parameter. The parameter will then be a
 reference to another rvalue (the int cast to a double) and altering it
 (the hidden double rvalue) may not really be what the coder intended.
 Afaik D doesn't support implicit casting for user-defined types, so that
 may not be a problem (for now at least).


Maybe you should stop trying to show that 'const' is sufficient for 
resolving those issues. The point is that it is not _necessary_. It is 
too strong.
Nov 08 2012
parent "martin" <kinke libero.it> writes:
On Thursday, 8 November 2012 at 22:34:03 UTC, Timon Gehr wrote:

 Ambiguous to me and all the interpretations are either wrong or 
 irrelevant.


My point is that it may affect performance. If there was no 
const, the compiler would need to allocate a dedicated copy of a 
literal whenever passing it to a mutable ref parameter unless the 
optimizer worked so well it can prove it's not going to be 
modified (which I'm sure you'd expect though :D).


 Maybe you should stop trying to show that 'const' is sufficient 
 for resolving those issues. The point is that it is not 
 _necessary_. It is too strong.


In that case it actually is - who cares if the read-only double 
rvalue the function is passed is the result of an implicit cast 
(and there's a reason for it being implicit) of the original 
argument (int rvalue)?

Anyway, I think we have moved on in this thread, so maybe you 
could contribute to trying to settle this rvalue => (const) ref 
issue once and for all by commenting my latest proposal.
Nov 08 2012
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, November 07, 2012 00:32:48 Manu wrote:

 But it only really makes sense in the context of templates...?
 Why should something called 'auto ref' provide fabrication of temporaries
 for the purpose of passing rvalues to functions that receive ref args?
 
 How does auto ref (under some different definition/implementation) address
 the rvalue problem?


The idea behind auto ref was to create a function which would take its 
argument in the most efficient way possible, whatever that was (be it ref or by 
value), and that the compiler would figure that out. That way, the function 
wouldn't care whether it was given an rvalue or lvalue or a value type or 
reference type or whatever. What actually got implemented was simply 
duplicating functions with auto ref so that there's a ref version of lvalues 
and non-ref versions for rvalues, which really doesn't do the job and only 
works with templates.

What we need is an attribute which indicates that you don't care whether the 
argument is an lvalue or rvalue. You just don't want it to be copied, and you 
don't want the function to mutate it - which is what const& in C++ is normally 
used for. Whether auto ref is the attribute used for that is more or less 
irrelevant except for the fact that that's the problem that it was trying to 
solve in the first place.

Making auto ref do that for non-templated functions would solve the problem 
for non-templated functions but would mean that auto ref on parameters in 
templated functions and non-templated functions would have different (albeit 
similar) semantics, which would be a bit of problem and may or may not be 
acceptable. And I'd have to think it through more to say whether or not auto 
ref as it is on templated functions really solves the problem well enough for 
templated functions or not. If it does, then it may be acceptable to just have 
auto ref on non-templated functions with slightly different semantics. If it 
doesn't, then we either need to change what auto ref does with templated 
functions to match what we want for non-templated ones (which could be a 
problem, since some people have found the current semantics of auto ref useful 
for other stuff like propagating attributes), or we'd need to use a new 
attribute rather than auto ref.

- Jonathan M Davis
Nov 06 2012
parent reply "martin" <kinke libero.it> writes:
Thanks Jonathan for the detailed info. So 'auto ref' is 
implemented the way I thought, by simply duplicating functions. 
That is actually the only way (I can think of) to solve your 
problem 'pass this argument in the most efficient way possible 
(do not copy)'.

But that is not the only problem. More commonly, we want to avoid 
copying lvalues for read-only parameters and therefore pass by 
reference. At the same time, we want to use the same function for 
rvalues - we simply can't stand having to declare temporaries 
manually or having to overload functions! Since the parameter is 
read-only, we couldn't care less if the argument is actually an 
lvalue or an rvalue. The only thing is that a dedicated function 
overload for rvalues would be slightly more efficient (for 
rvalues only) since the rvalue would be accessed directly on the 
stack instead of having to access it indirectly via its reference.
This problem here would be 'pass this argument as read-only in 
the most efficient way possible for lvalues (do not copy) but 
also allow rvalues at a slight cost (compared to a dedicated 
function overload)'.

So we do not really need 'auto ref' for non-templated functions 
or a new, even more confusing keyword which you, Jonathan, seem 
to insist on - 'const ref' (or, more elegantly 'in ref') is all 
we need. We simply want the compiler to automatically declare 
rvalues passed to 'const ref' parameters as local lvalues - 
problem solved for the majority of cases. For the remaining few 
which care about the involved cost for rvalue arguments, allow 
them to provide a function overload for rvalues (either manually 
or via 'auto ref' templates) - problem solved completely in my 
point of view. And that is actually exactly what the thread 
starter Malte proposes (and I already pointed that out in an 
earlier post).

Again hoping to be clear enough. :)
Nov 06 2012
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, November 07, 2012 01:32:03 martin wrote:

 So we do not really need 'auto ref' for non-templated functions
 or a new, even more confusing keyword which you, Jonathan, seem
 to insist on - 'const ref' (or, more elegantly 'in ref') is all
 we need.


No, because that would be doing the same thing as C++, which Walter and Andrei 
have already rejected. They specifically do _not_ want there to be any 
ambiguity between whether a const ref variable is an lvalue or rvalue. If they 
were willing to make const ref work the same as C++'s const&, then we would 
never have had this problem in the first place. We specifically need something 
other than const ref. The const ref can continue to work as it does now, but 
we'll have a way to get semantics similar to C++'s const& when we want them.

- Jonathan M Davis
Nov 06 2012
parent reply "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 00:47:04 UTC, Jonathan M Davis 
wrote:

 No, because that would be doing the same thing as C++, which 
 Walter and Andrei
 have already rejected. They specifically do _not_ want there to 
 be any
 ambiguity between whether a const ref variable is an lvalue or 
 rvalue. If they
 were willing to make const ref work the same as C++'s const&, 
 then we would
 never have had this problem in the first place. We specifically 
 need something
 other than const ref. The const ref can continue to work as it 
 does now, but
 we'll have a way to get semantics similar to C++'s const& when 
 we want them.


Wow, now this finally sheds some light on this topic. Do you have 
any links regarding their reasoning?
I simply and utterly fail to comprehend the need for a compiler 
to distinguish between a const reference to a named variable and 
a const reference to an unnamed temporary/literal (which the 
compiler is free to allocate wherever it makes most sense). 
Especially since the compiler could simply allocate and name the 
original rvalue as local variable on the caller's stack before 
the function call and hence transform it automatically to an 
lvalue, i.e., do that, what currently most of us have to 
painfully do by hand.
Nov 06 2012
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, November 07, 2012 02:15:26 martin wrote:

 Wow, now this finally sheds some light on this topic. Do you have
 any links regarding their reasoning?


The most recent discussion where Walter and Andrei were part of the discussion 
was here:

http://forum.dlang.org/post/4F84D6DD.5090405 digitalmars.com

A key component of the problem is that conversions cause problems. Also, IIRC 
allowing const ref to take rvalues causes some issues with overloading.

Making some changes to ref with regards to escaping apparently could change
the situation a bit, and there's even some discussion in there of allowing ref
to take rvalues if certain other things in the language are changed, but it's
all up in the air at this point. Certainly, it's not a simple matter of just
making const ref work with rvalues like most of the people coming from C++ want
and expect.

- Jonathan M Davis
Nov 06 2012
parent reply "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 01:33:49 UTC, Jonathan M Davis 
wrote:

 The most recent discussion where Walter and Andrei were part of 
 the discussion
 was here:

 http://forum.dlang.org/post/4F84D6DD.5090405 digitalmars.com


That thread is quite misleading and, I'm sad to say, not very 
useful (rather damaging to this discussion) in my opinion - 
especially because the distinction between rvalue => 'const ref' 
and rvalue => ref is largely neglected, and that distinction is 
of extremely high importance, I can't stress that enough. 
Walter's 3 C++ examples (2 of them invalid anyway afaik) don't 
relate to _const_ references. The implicit type conversion 
problem in that thread isn't a problem for _const_ references, 
just to point out one tiny aspect.
rvalue => ref/out propagation makes no sense imho, as does 
treating literals as lvalues (proposed by Walter iirc). The 
current 'auto ref' semantics also fail to cover the special role 
of _const_ references for rvalues (also illustrated by 
Scarecrow's post).


 Certainly, it's not a simple matter of just making const
 ref work with rvalues like most of the people coming from
 C++ want and expect.


Well I absolutely do _not_ share this point of view. It just 
seems so logical to me. I'm still waiting for a plausible 
argument to prove me wrong. All the required info is in this 
thread, e.g., we covered the escaping issue you mentioned.
Nov 06 2012
parent Manu <turkeyman gmail.com> writes:
On 7 November 2012 04:22, martin <kinke libero.it> wrote:


 On Wednesday, 7 November 2012 at 01:33:49 UTC, Jonathan M Davis wrote:


 The most recent discussion where Walter and Andrei were part of the
 discussion
 was here:

 http://forum.dlang.org/post/**4F84D6DD.5090405 digitalmars.**com<http://forum.dlang.org/post/4F84D6DD.5090405 digitalmars.com>


 That thread is quite misleading and, I'm sad to say, not very useful
 (rather damaging to this discussion) in my opinion - especially because the
 distinction between rvalue => 'const ref' and rvalue => ref is largely
 neglected, and that distinction is of extremely high importance, I can't
 stress that enough. Walter's 3 C++ examples (2 of them invalid anyway
 afaik) don't relate to _const_ references. The implicit type conversion
 problem in that thread isn't a problem for _const_ references, just to
 point out one tiny aspect.
 rvalue => ref/out propagation makes no sense imho, as does treating
 literals as lvalues (proposed by Walter iirc). The current 'auto ref'
 semantics also fail to cover the special role of _const_ references for
 rvalues (also illustrated by Scarecrow's post).


  Certainly, it's not a simple matter of just making const

 ref work with rvalues like most of the people coming from
 C++ want and expect.


 Well I absolutely do _not_ share this point of view. It just seems so
 logical to me. I'm still waiting for a plausible argument to prove me
 wrong. All the required info is in this thread, e.g., we covered the
 escaping issue you mentioned.


+1
I couldn't possibly agree more with your entire post.
Nov 07 2012
prev sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 01:15:27 UTC, martin wrote:

 I simply and utterly fail to comprehend the need for a compiler 
 to distinguish between a const reference to a named variable 
 and a const reference to an unnamed temporary/literal (which 
 the compiler is free to allocate wherever it makes most sense).


Let me be more precise: I don't see why a called function would 
need to know whether a passed const reference is named or unnamed 
at the caller's site. The developer's reasoning for a const 
reference is simply to avoid costly copying of the argument to be 
passed (only applicable for lvalues) which is guaranteed not to 
be modified by the callee. Decorating such a parameter with 
'const ref' seems as logical as it gets (don't pass me a copy of 
the real thing, pass me a reference instead - I'm not going to 
modify it).
The only reason I can think of to distinguish between lvalues and 
rvalues at the callee is the slight performance hit due to 
pointer indirection for rvalues. And that would be addressable by 
preferring by-value-passing of rvalues when selecting between 
function overloads.
Nov 06 2012
prev sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Wednesday, 7 November 2012 at 00:32:04 UTC, martin wrote:

 So we do not really need 'auto ref' for non-templated functions 
 or a new, even more confusing keyword which you, Jonathan, seem 
 to insist on - 'const ref' (or, more elegantly 'in ref') is all 
 we need. We simply want the compiler to automatically declare 
 rvalues passed to 'const ref' parameters as local lvalues - 
 problem solved for the majority of cases. For the remaining few 
 which care about the involved cost for rvalue arguments, allow 
 them to provide a function overload for rvalues (either 
 manually or via 'auto ref' templates) - problem solved 
 completely in my point of view. And that is actually exactly 
 what the thread starter Malte proposes (and I already pointed 
 that out in an earlier post).


  'in ref' may work, but what it implies doesn't quite make sense. 
'auto ref' already being used for templates should be able to 
carry over to functions, however auto in the same section along 
with type declaration may seem a little confusing. Let's glance 
at a few of how they would be called.

//either
void func(int x);
void func(const int x);

//lvalue
void func(ref int x);
void func(const ref int x);

//either? auto ref/in ref
void func(auto ref int x);
void func(const auto ref int x);
void func(in ref int x);

  Assuming auto can't be used to do type referencing the 'const 
auto ref int' seems rather long. alternatively perhaps inout? 
(accepts any type, partially skewed?)

void func(inout ref int x);
void func(const inout ref int x);

  At this point for const it's just as large as the const auto 
ref.. So..

void func(inout ref int x); //mutable
void func(in ref int x);    //const version

void func(out ref int x);   //optional write-only variable

  'out ref int x' could then be legal..  Somehow reminds me of 
zlib's compress utility function that returns how many bytes the 
output buffer used but sometimes you didn't care. So.. if we 
defined and used...


void func(out ref int x) {
   x = 100;
}

int x;
func(x);
assert(x == 100);

func(0);    //0 and null (and legal) basically ignored.
func(null);
Nov 06 2012
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, October 18, 2012 05:07:52 Malte Skarupke wrote:

 What do you think?


Okay. Here are more links to Andrei discussing the problem:

http://forum.dlang.org/post/4F83DBE5.20800 erdani.com
http://www.mail-archive.com/digitalmars-d puremagic.com/msg44070.html
http://www.mail-archive.com/digitalmars-d puremagic.com/msg43769.html
http://forum.dlang.org/post/hg62rq$2c2n$1 digitalmars.com

He is completely convinced that letting rvalues bind to const& in C++ was a 
huge mistake, and it seems to come down mainly to this:

"The problem with binding rvalues to const ref is that once that is in place 
you have no way to distinguish an rvalue from a const ref on the callee site."

And apparenly it required adding the concept of rvalue references to C++, 
which complicated things considerably.

It's too bad that he hasn't replied to some of the more detailed questions on 
the matter in this thread, but if you want rvalues to bind to const ref in D, 
you're going to have to convince him.

At this point, I expect that the most likely solution is that auto ref will 
continue to work like it does in templates but for non-templated functions it 
will become like const ref is in C++ and accept rvalues without copying 
lvalues. Andrei suggests that in at least one of those posts, and AFAIK, it 
would work just fine. The only real downside that I'm aware of is that then the 
semantics of auto ref are slightly different for non-templated functions, but 
you're doing basically the same thing with auto ref in both cases - avoiding 
copying lvalues and only copying rvalues when you have to - so it really 
shouldn't be a problem. The main problem is that someone needs to go and 
implement it.

- Jonathan M Davis
Nov 07 2012
parent "martin" <kinke libero.it> writes:
On Thursday, 8 November 2012 at 03:07:00 UTC, Jonathan M Davis 
wrote:

 Okay. Here are more links to Andrei discussing the problem:

 http://forum.dlang.org/post/4F83DBE5.20800 erdani.com
 http://www.mail-archive.com/digitalmars-d puremagic.com/msg44070.html
 http://www.mail-archive.com/digitalmars-d puremagic.com/msg43769.html
 http://forum.dlang.org/post/hg62rq$2c2n$1 digitalmars.com


Thank you so much for these links, Jonathan.

So fortunately the special role of _const_ ref parameters has 
been acknowledged.

 From the 2nd  link:

 The problem with binding rvalues to const ref is that
 once that is in place you have no way to distinguish an
 rvalue from a const ref on the callee site. If you do want
 to distinguish, you must rely on complicated conversion
 priorities. For example, consider:

 void foo(ref const Widget);
 void foo(Widget);

 You'd sometimes want to do that because you want to exploit
 an rvalue by e.g. moving its state instead of copying it.
 However, if rvalues become convertible to ref const, then
 they are motivated to go either way. A rule could be put in
 place that gives priority to the second declaration. However,
 things quickly get complicated in the presence of other
 applicable rules, multiple parameters etc. Essentially it
 was impossible for C++ to go this way and that's how rvalue
 references were born.

 For D I want to avoid all that aggravation and have a simple
 rule: rvalues don't bind to references to const. If you don't
 care, use auto ref. This is a simple rule that works
 promisingly well in various forwarding scenarios.


This is exactly what we propose (to be able to avoid 
pointer/reference indirection for rvalues in some absolutely 
performance-critical cases). Unlike Andrei though, I don't find 
the required overloading rules complicated at all, quite the 
contrary in fact.

 From the 3rd link:

 Binding rvalues to const references was probably the single
 most hurtful design decisions for C++. I don't have time to
 explain now, but in short I think all of the problems that
 were addressed by rvalue references, and most of the
 aggravation within the design of rvalue references, are owed
 by that one particular conversion.


Here's where I totally disagree with Andrei. C++ rvalue 
references (T&&) aren't used to distinguish between lvalues and 
rvalues when expecting a _const_ reference (I still have to see a 
use case for 'const T&&'). They are used for _mutable_ references 
and primarily to enforce efficient move semantics in C++, i.e., 
to move _mutable_ rvalue arguments (instead of copying them) and 
to enforce 'Named Return Value Optimization' when returning 
lvalues (by using std::move; goal again is to avoid a redundant 
copy). D fortunately seems to implement move semantics 
out-of-the-box (at least now in v2.060), in both cases, see Rob 
T's posts and my replies in this thread.
Besides implementing move semantics, C++ with its rvalue refs 
also implicitly provides a way to distinguish between _mutable_ 
lvalue and rvalue references and so allows optimized 
implementations - that is something we'd also need in D, but 
that's what we've just covered with regard to the 2nd link.

So I still don't see a valid reason to preclude binding rvalues 
to const ref parameters.
Nov 08 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, November 09, 2012 10:33:45 Manu wrote:

 On 9 November 2012 00:44, Jonathan M Davis <jmdavisProg gmx.com> wrote:

 On Thursday, November 08, 2012 21:49:58 Manu wrote:

 That's cute, but it really feels like a hack.
 All of a sudden the debugger doesn't work properly anymore, you need to
 step-in twice to enter the function, and it's particularly inefficient
 in
 debug builds (a point of great concern for my industry!).
 
 Please just with the compiler creating a temporary in the caller space.
 Restrict is to const ref, or better, in ref (scope seems particularly
 important here).


 
 I honestly wish that in didn't exist in the language. The fact that it
 it's an
 alias two different attributes is confusing, and people keep using it
 without
 realizing what they're getting into.


 
 I understand it's an alias for 'const scope', but what's this about scope
 not working correctly? What's wrong with it?
 It seems like precisely what you want in this case... you don't want a
 temporary escaping the function you pass it to.
 What's the problem?


It only works with delegates. In all other cases, it's ignored.


 If scope worked correctly, you'd only
 

 want it in specific circumstances, not in general. And since it doesn't
 work
 correctly aside from delegates, once it _does_ work correctly, it'll break
 code all over the place, because people keep using in, because they like
 how
 it corresponds with out or whatever.


 
 I like how it promises that the thing I pass can't escape the callee, not
 that it's the natural english compliment to 'out'.


There are cases where it's useful, but there are a ton of cases where it would 
be really annoying if it worked properly. For instance, in a _lot_ of cases, 
it would cause problems for a function taking an array, because it couldn't 
return a slice of the array. Any struct holding any reference types would be 
in the same boat, as would any class or AA.

In most cases, it really doesn't matter whether references escape, because the 
stuff is on the GC heap, and it'll collect it when it's appropriate. It would 
be a much bigger deal if you had to worry about ownership.

For delegates, it's great because it allows you to avoid allocating a closure, 
so if you don't need the delegate to escape, it makes perfect sense to make it 
scope. For everything else, whether it makes sense depends on what you're 
doing, and sometimes it would be useful, but most of the time, it really 
wouldn't be.

But since in currently only works for delegates, it's absolutely pointless to 
use it instead of const for anything else, and if/when scope is ever fixed to 
work with reference types in general, then all kinds of code that uses in will 
break, because it'll be escaping references, and the programmer didn't notice 
it. So, you might as well just not use it at this point except for delegates.

Regardless, you don't need in to use scope. You can just use scope directly. 
So, having in buys you _nothing_, and it unfortunately encourages people to 
use it (which IMHO is bad due to the issues with scope), because they like how 
it's the opposite of out. So, if we didn't have in, we'd be able to do exactly 
what we can do now, but people wouldn't be using scope all over the place via 
in just because they liked the idea of in being the opposite of out.

But originally, in comes from D1 (where it had a meaning similar to const 
IIRC, but scope had nothing to do with it), so it's been around for a long 
time, and there's pretty much no way that it would be removed from the 
language. I just think that it was mistake to have it given its current 
semantics. It's also a bit pointless IMHO to have an attribute which is an 
alias for other attributes. It's a needless duplication of functionality and 
just increases confusion.

- Jonathan M Davis
Nov 09 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On 9 November 2012 10:46, Jonathan M Davis <jmdavisProg gmx.com> wrote:


 On Friday, November 09, 2012 10:33:45 Manu wrote:

 On 9 November 2012 00:44, Jonathan M Davis <jmdavisProg gmx.com> wrote:

 On Thursday, November 08, 2012 21:49:58 Manu wrote:

 That's cute, but it really feels like a hack.
 All of a sudden the debugger doesn't work properly anymore, you need






 to

 step-in twice to enter the function, and it's particularly






 inefficient

 in
 debug builds (a point of great concern for my industry!).

 Please just with the compiler creating a temporary in the caller






 space.

 Restrict is to const ref, or better, in ref (scope seems particularly
 important here).


 I honestly wish that in didn't exist in the language. The fact that it
 it's an
 alias two different attributes is confusing, and people keep using it
 without
 realizing what they're getting into.


 I understand it's an alias for 'const scope', but what's this about scope
 not working correctly? What's wrong with it?
 It seems like precisely what you want in this case... you don't want a
 temporary escaping the function you pass it to.
 What's the problem?


 It only works with delegates. In all other cases, it's ignored.


 If scope worked correctly, you'd only


 want it in specific circumstances, not in general. And since it doesn't
 work
 correctly aside from delegates, once it _does_ work correctly, it'll




 break

 code all over the place, because people keep using in, because they




 like

 how
 it corresponds with out or whatever.


 I like how it promises that the thing I pass can't escape the callee, not
 that it's the natural english compliment to 'out'.


 There are cases where it's useful, but there are a ton of cases where it
 would
 be really annoying if it worked properly. For instance, in a _lot_ of
 cases,
 it would cause problems for a function taking an array, because it couldn't
 return a slice of the array.



Does that actually make sense? Surely a function that receives a scope
argument can return that argument, since it's only passing it back to the
same function that already owns it... it knows it can trust that function,
since it was received from that function.

Any struct holding any reference types would be

 in the same boat, as would any class or AA.


I don't follow the problem with reference args. surely they can be
evaluated just fine? Just that nothing can escape the function...

In most cases, it really doesn't matter whether references escape, because

 the
 stuff is on the GC heap, and it'll collect it when it's appropriate. It
 would
 be a much bigger deal if you had to worry about ownership.


It's true, and stack locals are perhaps the most important thing that scope
could possibly protect... which makes it particularly applicable here.

For delegates, it's great because it allows you to avoid allocating a

 closure,
 so if you don't need the delegate to escape, it makes perfect sense to
 make it
 scope. For everything else, whether it makes sense depends on what you're
 doing, and sometimes it would be useful, but most of the time, it really
 wouldn't be.


Of course, and whether you apply the scope attribute to a function argument
equally depends on what you're doing :)
I still can't see any inherent problems from what you're saying. It's still
just an access restriction to enhance safety.

But since in currently only works for delegates, it's absolutely pointless

 to
 use it instead of const for anything else, and if/when scope is ever fixed
 to
 work with reference types in general, then all kinds of code that uses in
 will
 break, because it'll be escaping references, and the programmer didn't
 notice
 it. So, you might as well just not use it at this point except for
 delegates.


Very good point. I had no idea it didn't work. It'd be nice to see this
fixed.

Regardless, you don't need in to use scope. You can just use scope directly.

 So, having in buys you _nothing_, and it unfortunately encourages people to
 use it (which IMHO is bad due to the issues with scope), because they like
 how
 it's the opposite of out.



The issue here is that 'scope' is an incomplete feature. I think it's
theoretically sound.
Every conversation/thing I've read on the topic certainly gave me the
impression it worked...

So, if we didn't have in, we'd be able to do exactly

 what we can do now, but people wouldn't be using scope all over the place
 via
 in just because they liked the idea of in being the opposite of out.

 But originally, in comes from D1 (where it had a meaning similar to const
 IIRC, but scope had nothing to do with it), so it's been around for a long
 time, and there's pretty much no way that it would be removed from the
 language.



Oh really? That's not at all what I had imagined. I presumed 'in' was
invented when it was realised that 'const scope' was a super-common pattern.
I still don't see any less value in it. Just that it should work properly.

I just think that it was mistake to have it given its current

 semantics. It's also a bit pointless IMHO to have an attribute which is an
 alias for other attributes. It's a needless duplication of functionality
 and
 just increases confusion.


Side topic: what I find myself constantly wanting is alias for attributes!
Especially when dealing with GDC/LDC which have their own non-standard
attributes. I'm often wanting to create a meaning alias for the set of
attributes applicable to my code.
And you can't currently use GDC attributes without static if and physically
duplicating all the attributed code >_<
Nov 09 2012
prev sibling next sibling parent reply "martin" <kinke libero.it> writes:
Let me summarize my (final, I guess) proposal. I think it makes 
sense to compare it to C++ in order to anticipate and hopefully 
invalidate (mainly Andrei's) objections.

      parameter type     |   lvalue    |    rvalue
                         | C++     D   | C++     D
------------------------|-------------|------------
T                       | copy   copy | copy   move
T& / ref T              | ref    ref  | n/a    n/a
out T (D only)          |        ref  |        n/a
T&& (C++ only)          | n/a         | move
auto ref T (D only) (*) |        ref  |        ref
------------------------|-------------|------------
const T                 | copy   copy | copy   move
const T& / const ref T  | ref    ref  | ref    ref (*)
const T&& (C++ only)    | n/a         | move

(*): proposed additions

For lvalues in both C++ and D, there are 2 options: either copy 
the argument (pass-by-value) or pass it by ref. There's no real 
difference between both languages except for D's additional 'out' 
keyword and, with the proposed 'auto ref' syntax, an (imo 
negligible) ambiguity between 'ref T' and 'auto ref T' in D.
Rvalues are a different topic though. There are 3 possibilites in 
general: copy, move and pass by ref. Copying rvalue arguments 
does not make sense - the argument won't be used by the caller 
after the invokation, so a copy is redundant and hurts 
performance. D corrects this design flaw of C++ (which had to 
introduce rvalue refs to add move semantics on top of the default 
copy semantics) and therefore only supports moving instead. C++ 
additionally supports pass-by-ref of rvalues to const refs, but 
not to mutable refs. I propose to allow pass-by-ref to both const 
(identical syntax as C++, it's perfectly safe and logical) and 
mutable refs (new syntax with 'auto ref' to emphasize that the 
parameter may be an rvalue reference, with related consequences 
such as potentially missing side effects).

Regarding the required overloading priorities for the proposed 
additions to work properly, I propose:
1) lvalues: prefer pass-by-ref
    so: ref/out T -> auto ref T (*) -> const ref T -> (const) T
    - const lvalues:   const ref T -> (const) T
    - mutable lvalues: ref/out T -> auto ref T (*) -> const ref T 
-> (const) T
2) rvalues: prefer pass-by-value (moving: argument allocated 
directly on
    callee's stack (parameter) vs. pointer/reference indirection 
implied by
    pass-by-ref)
    so: (const) T -> auto ref T (*) -> const ref T (*)

Finally, regarding templates, I'm in favor of dropping the 
current 'auto ref' semantics and propose to simply adopt the 
proposed semantics for consistency and simplicity and to avoid 
excessive code bloating. That shouldn't break existing code I 
hope (unless parameters have been denoted with 'const auto ref 
T', which would need to be changed to 'const ref T').

Now please go ahead and shoot. :)
Nov 09 2012
parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Fri, 09 Nov 2012 16:26:21 -0000, martin <kinke libero.it> wrote:

 Let me summarize my (final, I guess) proposal. I think it makes sense to  
 compare it to C++ in order to anticipate and hopefully invalidate  
 (mainly Andrei's) objections.

       parameter type     |   lvalue    |    rvalue
                          | C++     D   | C++     D
 ------------------------|-------------|------------
 T                       | copy   copy | copy   move
 T& / ref T              | ref    ref  | n/a    n/a
 out T (D only)          |        ref  |        n/a
 T&& (C++ only)          | n/a         | move
 auto ref T (D only) (*) |        ref  |        ref
 ------------------------|-------------|------------
 const T                 | copy   copy | copy   move
 const T& / const ref T  | ref    ref  | ref    ref (*)
 const T&& (C++ only)    | n/a         | move

 (*): proposed additions

 For lvalues in both C++ and D, there are 2 options: either copy the  
 argument (pass-by-value) or pass it by ref. There's no real difference  
 between both languages except for D's additional 'out' keyword and, with  
 the proposed 'auto ref' syntax, an (imo negligible) ambiguity between  
 'ref T' and 'auto ref T' in D.
 Rvalues are a different topic though. There are 3 possibilites in  
 general: copy, move and pass by ref. Copying rvalue arguments does not  
 make sense - the argument won't be used by the caller after the  
 invokation, so a copy is redundant and hurts performance. D corrects  
 this design flaw of C++ (which had to introduce rvalue refs to add move  
 semantics on top of the default copy semantics) and therefore only  
 supports moving instead. C++ additionally supports pass-by-ref of  
 rvalues to const refs, but not to mutable refs. I propose to allow  
 pass-by-ref to both const (identical syntax as C++, it's perfectly safe  
 and logical) and mutable refs (new syntax with 'auto ref' to emphasize  
 that the parameter may be an rvalue reference, with related consequences  
 such as potentially missing side effects).

 Regarding the required overloading priorities for the proposed additions  
 to work properly, I propose:
 1) lvalues: prefer pass-by-ref
     so: ref/out T -> auto ref T (*) -> const ref T -> (const) T
     - const lvalues:   const ref T -> (const) T
     - mutable lvalues: ref/out T -> auto ref T (*) -> const ref T ->  
 (const) T
 2) rvalues: prefer pass-by-value (moving: argument allocated directly on
     callee's stack (parameter) vs. pointer/reference indirection implied  
 by
     pass-by-ref)
     so: (const) T -> auto ref T (*) -> const ref T (*)

 Finally, regarding templates, I'm in favor of dropping the current 'auto  
 ref' semantics and propose to simply adopt the proposed semantics for  
 consistency and simplicity and to avoid excessive code bloating. That  
 shouldn't break existing code I hope (unless parameters have been  
 denoted with 'const auto ref T', which would need to be changed to  
 'const ref T').

 Now please go ahead and shoot. :)


Nice detailed proposal.  I would suggest starting a new thread with it, to  
catch anyone who dozed off in this thread :p

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 09 2012
parent "martin" <kinke libero.it> writes:
On Friday, 9 November 2012 at 16:53:01 UTC, Regan Heath wrote:

 Nice detailed proposal.  I would suggest starting a new thread 
 with it, to catch anyone who dozed off in this thread :p


Thanks, and you're probably right. Here's the link:

http://forum.dlang.org/thread/zteryxwxyngvyqvukqkm forum.dlang.org

I've elaborated on it a little bit to avoid redundant posts.
Nov 09 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, November 09, 2012 15:55:12 Manu wrote:

 Does that actually make sense? Surely a function that receives a scope
 argument can return that argument, since it's only passing it back to the
 same function that already owns it... it knows it can trust that function,
 since it was received from that function.


It can't. That would mean that the reference escaped. That would be 
particularly deadly for delegates. Think about what happens if the scoped 
delegate is put into a struct which is returned.

struct Result
{
 delegate........ del;
 ...........
}

Result foo(scope delegate....... bar)
{
 ..........
 return Result(bar);
}

auto baz()
{
 Result r;
 {
 int n = 5;
 r = foo((){writeln(n);});
 }
 r.del();
}

baz has no idea where the delegate in r came from. It has no idea that it 
wasn't allocated as a closure. So, it's not going to allocate one, which means 
that the delegate refers to a part of the stack which won't exist anymore when 
the delegate actually gets called. If scope wasn't used, that wouldn't have 
been a problem, because a closure would have been allocated as soon as the 
delegate had been passed to foo, but because scope was used, it knows that the 
delegate won't escape, so it doesn't allocate the closure (since it's not 
necessary). But that only works because scope prevents escaping - including by 
the return value. So, the above code _must_ be invalid.


 Any struct holding any reference types would be
 in the same boat, as would any class or AA.


 
 I don't follow the problem with reference args. surely they can be
 evaluated just fine? Just that nothing can escape the function...


It's the fact that they can't escape the function which is the problem. If 
scope is working correctly, than any and all reference types which are passed 
to the function via a scope parameter (be it directly or within another 
object) cannot escape the function in any way shape or form. So, you can't 
assign any of them to any static or module-level variables (not generally a 
big deal) and you can't use any of them in the return value (often a big 
deal). Sometimes, that's exactly what you want, but in the general case, you 
don't want to prevent anything you pass into a function from being returned 
from it, so scope quickly becames overly restrictive.

- Jonathan M Davis
Nov 09 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On 9 November 2012 21:39, Jonathan M Davis <jmdavisProg gmx.com> wrote:


 On Friday, November 09, 2012 15:55:12 Manu wrote:

 Does that actually make sense? Surely a function that receives a scope
 argument can return that argument, since it's only passing it back to the
 same function that already owns it... it knows it can trust that


 function,

 since it was received from that function.


 It can't. That would mean that the reference escaped. That would be
 particularly deadly for delegates. Think about what happens if the scoped
 delegate is put into a struct which is returned.

 struct Result
 {
  delegate........ del;
  ...........
 }

 Result foo(scope delegate....... bar)
 {
  ..........
  return Result(bar);
 }

 auto baz()
 {
  Result r;
  {
  int n = 5;
  r = foo((){writeln(n);});
  }
  r.del();
 }

 baz has no idea where the delegate in r came from. It has no idea that it
 wasn't allocated as a closure. So, it's not going to allocate one, which
 means
 that the delegate refers to a part of the stack which won't exist anymore
 when
 the delegate actually gets called. If scope wasn't used, that wouldn't have
 been a problem, because a closure would have been allocated as soon as the
 delegate had been passed to foo, but because scope was used, it knows that
 the
 delegate won't escape, so it doesn't allocate the closure (since it's not
 necessary). But that only works because scope prevents escaping -
 including by
 the return value. So, the above code _must_ be invalid.


Okay, makes sense.


 Any struct holding any reference types would be

 in the same boat, as would any class or AA.


 I don't follow the problem with reference args. surely they can be
 evaluated just fine? Just that nothing can escape the function...


 It's the fact that they can't escape the function which is the problem. If
 scope is working correctly, than any and all reference types which are
 passed
 to the function via a scope parameter (be it directly or within another
 object) cannot escape the function in any way shape or form. So, you can't
 assign any of them to any static or module-level variables (not generally a
 big deal) and you can't use any of them in the return value (often a big
 deal). Sometimes, that's exactly what you want, but in the general case,
 you
 don't want to prevent anything you pass into a function from being returned
 from it, so scope quickly becames overly restrictive.


I'm still not buying this. Here's a common struct I will pass by ref
(perhaps the most common struct in my industry):

struct Vector { float, x,y,z,w; }
struct Matrix { Vector xRow, yRow, zRow, wRow; }

Vector mul( scope const ref Matrix m, scope const Vector v)
{
  Vector v;
  // perform a matrix multiply against the vector...
  // this work uses every single field of the inputs given, but the result
it produces has to references to the sources.
  // everything is operated on and copied to the output struct, which is
returned.
  return result;
}

Why should this be a problem?
The majority of my work-horse structs apply to this pattern. This is what I
imagine 'scope' to be for...
The main advantage I expect is that I can have confidence that passing
rvalues (temporaries) is safe, and that external code won't take references
to memory that I may not own/control. Is that not the point?

Surely the problem that scope should be protecting against is a pointer to
any part of the argument escaping. *Copies* of values contained in the
argument/s are fine.
Nov 10 2012
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, November 10, 2012 13:21:42 Manu wrote:

 I'm still not buying this. Here's a common struct I will pass by ref
 (perhaps the most common struct in my industry):
=20
 struct Vector { float, x,y,z,w; }
 struct Matrix { Vector xRow, yRow, zRow, wRow; }
=20
 Vector mul( scope const ref Matrix m, scope const Vector v)
 {
   Vector v;
   // perform a matrix multiply against the vector...
   // this work uses every single field of the inputs given, but the r=


esult

 it produces has to references to the sources.
   // everything is operated on and copied to the output struct, which=


 is

 returned.
   return result;
 }
=20
 Why should this be a problem?
 The majority of my work-horse structs apply to this pattern. This is =


what I

 imagine 'scope' to be for...
 The main advantage I expect is that I can have confidence that passin=


g

 rvalues (temporaries) is safe, and that external code won't take refe=


rences

 to memory that I may not own/control. Is that not the point?
=20
 Surely the problem that scope should be protecting against is a point=


er to

 any part of the argument escaping. *Copies* of values contained in th=


e

 argument/s are fine.


Hmmmm. scope on value types is pointless, because there are no referenc=
es to=20
escape, but if you pass by ref, then it does become possible for a poin=
ter to=20
the argument to escape, but I don't know that that's actually actually =
covered=20
by scope. The description for scope in docs is that "ref=C2=ADer=C2=ADe=
nces in the pa=C2=AD
ra=C2=ADme=C2=ADter can=C2=ADnot be es=C2=ADcaped (e.g. as=C2=ADsigned =
to a global vari=C2=ADable)." And=20
taking the address of a local variable (which is the only way that any =
sort of=20
reference to the data could escape) is never  safe anyway. If you passe=
d in a=20
pointer, and scope were fully working, then you'd be protected against =
the=20
pointer escaping, but passing by ref isn't really the same thing. I'd h=
ave=20
thought that taking the address of a variable passed by ref would fall =
into=20
pretty much the same camp as taking the address of any other local vari=
able,=20
which is completely unsafe to escape to the point that I'm not sure tha=
t=20
there's any point in protecting against it. It's just completely stupid=
 to do=20
anyway and is definitely  system. Outside of taking the address of a re=
f=20
parameter, taking the address of a local variable and escpaing it is _a=
lways_=20
going to result in garbage, and ref parameters aren't really references=
 in the=20
normal sense, so I don't know.

You bring up a good point, but I don't know if it's applicable. Certain=
ly,=20
without the ref there (like is the case with the Vector that you're pas=
sing=20
in), scope would never do anything, because it doesn't even theoretical=
ly have=20
anything to do. It's purely a value type that's not even being passed b=
y ref.

In general though, putting scope on struct parameters would cause a lot=
 of=20
problems, because of arrays that they might hold and whatnot. Slices wo=
uldn't=20
be able to escape (and so copies of the struct wouldn't be able escape =
without=20
deep copying, let alone the array itself). So, while scope may be very =
useful=20
in some such cases (assuming that it worked), it's not necessarily some=
thing=20
that you'd want as a matter of course. Part of it probably depends on y=
our=20
programming style though. If you have a lot of functions that take argu=
ments=20
and don't return anything that was in them ever, then scope is less of =
a big=20
deal, but that's the sort of thing that happens a _lot_ in my experienc=
e, so=20
scope would very quickly become extremely annoying.

And actually, to make matters worse, I'm not sure that scope on delegat=
es is=20
working correctly. I thought that it was, but this code compiles:

import std.stdio;

void delegate() global;

void foo(scope void delegate() del)
{
    global =3D del;
}

void main()
{
    {
        char[5] bar =3D "hello";
        foo((){writeln(bar);});
    }
    char[7] baz =3D "goodbye";

    global();
}

It also prints out "hello", and if a closure had not been allocated, I =
would=20
have at least half-expected it to print out "goodb", because I'd have t=
hought=20
that baz would have been taking up the same memory that bar had been. S=
o, it=20
looks like scope may be completely and utterly broken at this point. I =
don't=20
know.

- Jonathan M Davis
Nov 10 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/10/2012 11:07 PM, Jonathan M Davis пишет:

 And actually, to make matters worse, I'm not sure that scope on delegates is
 working correctly. I thought that it was, but this code compiles:

 import std.stdio;

 void delegate() global;

 void foo(scope void delegate() del)
 {
      global = del;
 }

 void main()
 {
      {
          char[5] bar = "hello";
          foo((){writeln(bar);});
      }
      char[7] baz = "goodbye";

      global();
 }

 It also prints out "hello", and if a closure had not been allocated,


I would

 have at least half-expected it to print out "goodb", because I'd have thought
 that baz would have been taking up the same memory that bar had been.


Nope. It's just that the stack is intact and contains: hello and goodbye 
one after another. Without optimizations { } scope doesn't mean reuse 
stack space.

Now if play with stack a bit, for me the next one prints:
­-²↑

import std.stdio;

void delegate() global;

void foo(scope void delegate() del)
{
     global = del;
}


void f()
{
     {
         char[5] bar = "hello";
         foo((){writeln(bar);});
     }
}

void main()
{
     char[7] baz = "goodbye";
     f();

     global();
}

-- 
Dmitry Olshansky
Nov 11 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 11, 2012 13:36:05 Dmitry Olshansky wrote:

 Nope. It's just that the stack is intact and contains: hello and good=


bye

 one after another. Without optimizations { } scope doesn't mean reuse=



 stack space.
=20
 Now if play with stack a bit, for me the next one prints:
 =C2=AD-=C2=B2=E2=86=91
=20
 import std.stdio;
=20
 void delegate() global;
=20
 void foo(scope void delegate() del)
 {
      global =3D del;
 }
=20
=20
 void f()
 {
      {
          char[5] bar =3D "hello";
          foo((){writeln(bar);});
      }
 }
=20
 void main()
 {
      char[7] baz =3D "goodbye";
      f();
=20
      global();
 }


It still prints "hello", even with full optimations turned on. So, it m=
ust be=20
allocating a closure in spite of scope. So, it looks to me like scope i=
s just=20
completely ignored and does absolutely nothing at this point, unless I'=
m just=20
completely missing something here.

- Jonathan M Davis
Nov 11 2012
next sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Sunday, 11 November 2012 at 10:09:17 UTC, Jonathan M Davis 
wrote:

 It still prints "hello", even with full optimations turned on. 
 So, it must be
 allocating a closure in spite of scope. So, it looks to me like 
 scope is just
 completely ignored and does absolutely nothing at this point, 
 unless I'm just
 completely missing something here.


Try this:

---
import std.stdio;

void delegate() global;

void foo(scope void delegate() del)
{
      global = del;
}


void f()
{
      {
          char[5] bar = "hello";
          foo((){writeln(bar);});
      }
}

void smashStack() {
     uint[1000] dummy = 0xbadcab1e;
     asm { nop; }
}

void main()
{
      char[7] baz = "goodbye";
      f();
      smashStack();
      global();
}
---

David
Nov 11 2012
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 11, 2012 12:10:59 David Nadlinger wrote:

 On Sunday, 11 November 2012 at 10:09:17 UTC, Jonathan M Davis
 
 wrote:

 It still prints "hello", even with full optimations turned on.
 So, it must be
 allocating a closure in spite of scope. So, it looks to me like
 scope is just
 completely ignored and does absolutely nothing at this point,
 unless I'm just
 completely missing something here.


 
 Try this:
 
 ---
 import std.stdio;
 
 void delegate() global;
 
 void foo(scope void delegate() del)
 {
       global = del;
 }
 
 
 void f()
 {
       {
           char[5] bar = "hello";
           foo((){writeln(bar);});
       }
 }
 
 void smashStack() {
      uint[1000] dummy = 0xbadcab1e;
      asm { nop; }
 }
 
 void main()
 {
       char[7] baz = "goodbye";
       f();
       smashStack();
       global();
 }
 ---


That did it. And if scope is removed, it works again. I clearly don't get the 
low level stuff though, since it's just plain bizarre to me that the previous 
example didn't do it.

- Jonathan M Davis
Nov 11 2012
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/11/2012 2:08 PM, Jonathan M Davis пишет:

 On Sunday, November 11, 2012 13:36:05 Dmitry Olshansky wrote:

 Nope. It's just that the stack is intact and contains: hello and goodbye
 one after another. Without optimizations { } scope doesn't mean reuse
 stack space.

 Now if play with stack a bit, for me the next one prints:
 ­-²↑

 import std.stdio;

 void delegate() global;

 void foo(scope void delegate() del)
 {
       global = del;
 }


 void f()
 {
       {
           char[5] bar = "hello";
           foo((){writeln(bar);});
       }
 }

 void main()
 {
       char[7] baz = "goodbye";
       f();

       global();
 }


 It still prints "hello", even with full optimations turned on.


I tried with and without optimizations. I get garbage as expected.

So, it must be

 allocating a closure in spite of scope. So, it looks to me like scope is just
 completely ignored and does absolutely nothing at this point, unless I'm just
 completely missing something here.


Something must be screwed up. I dunno what, I use near-latest DMD from 
github and Win32 binaries.
For good measure try making stack variables larger if you are on 64bit. 
Drop in a  couple of calls to writeln before and after calling 'f' it 
should scramble the stack.

-- 
Dmitry Olshansky
Nov 11 2012
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 11, 2012 15:17:03 Dmitry Olshansky wrote:

 Something must be screwed up. I dunno what, I use near-latest DMD from
 github and Win32 binaries.


I'm on 64-bit Linux, so that may change things a bit.

- Jonathan M Davis
Nov 11 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On 10 November 2012 21:07, Jonathan M Davis <jmdavisProg gmx.com> wrote:


 On Saturday, November 10, 2012 13:21:42 Manu wrote:

 I'm still not buying this. Here's a common struct I will pass by ref
 (perhaps the most common struct in my industry):

 struct Vector { float, x,y,z,w; }
 struct Matrix { Vector xRow, yRow, zRow, wRow; }

 Vector mul( scope const ref Matrix m, scope const Vector v)
 {
   Vector v;
   // perform a matrix multiply against the vector...
   // this work uses every single field of the inputs given, but the


 result

 it produces has to references to the sources.
   // everything is operated on and copied to the output struct, which i=




s

 returned.
   return result;
 }

 Why should this be a problem?
 The majority of my work-horse structs apply to this pattern. This is


 what I

 imagine 'scope' to be for...
 The main advantage I expect is that I can have confidence that passing
 rvalues (temporaries) is safe, and that external code won't take


 references

 to memory that I may not own/control. Is that not the point?

 Surely the problem that scope should be protecting against is a pointer


 to

 any part of the argument escaping. *Copies* of values contained in the
 argument/s are fine.


 Hmmmm. scope on value types is pointless, because there are no references
 to
 escape, but if you pass by ref, then it does become possible for a pointe=


r

 to
 the argument to escape,



Precisely, to me, this seems like the ENTIRE POINT of 'in'?  (I originally
presumed 'in' implied ref, but I was wrong, 'in ref' is supported however)


but I don't know that that's actually actually covered

 by scope. The description for scope in docs is that "ref=C2=ADer=C2=ADenc=


es in the

 pa=C2=AD
 ra=C2=ADme=C2=ADter can=C2=ADnot be es=C2=ADcaped (e.g. as=C2=ADsigned to=


 a global vari=C2=ADable)." And

 taking the address of a local variable (which is the only way that any
 sort of
 reference to the data could escape) is never  safe anyway.



When did ' safe'ty enter into it? Are you saying that a ref variable is
somehow a local variable? It's a local pointer to a foreign variable... and
a function can usually operate on that data however it likes.
Scope would promise that nothing other than the function I give it to will
get its grubby little hands on it.
Let's say that function wanted to call through to some other function and
pass the variable along (by ref). Obviously, the second function would also
have to have it's inputs marked scope, to promise that it never escapes
from there.
I imagine scope similarly to const, once it goes scope, the whole callstack
must maintain the scope property, otherwise the outermost function can't
trust it anymore.

This makes perfect sense for any function that is likely to receive
immediate or local variables by reference (which is extremely common).
It also seems absolutely relevant to the rvalues -> ref thing.

If you passed in a

 pointer, and scope were fully working, then you'd be protected against th=


e

 pointer escaping, but passing by ref isn't really the same thing. I'd hav=


e

 thought that taking the address of a variable passed by ref would fall in=


to

 pretty much the same camp as taking the address of any other local
 variable,
 which is completely unsafe to escape to the point that I'm not sure that
 there's any point in protecting against it. It's just completely stupid t=


o

 do
 anyway and is definitely  system. Outside of taking the address of a ref
 parameter, taking the address of a local variable and escpaing it is
 _always_
 going to result in garbage, and ref parameters aren't really references i=


n

 the
 normal sense, so I don't know.


What do you mean 'aren't really references in the normal sense'?


You bring up a good point, but I don't know if it's applicable. Certainly,

 without the ref there (like is the case with the Vector that you're passi=


ng

 in), scope would never do anything, because it doesn't even theoretically
 have
 anything to do. It's purely a value type that's not even being passed by
 ref.


Correct, scope on a purely value type passed by-value means absolutely
nothing.
scope on a pointer parameter means something; I would expect the pointer
its self, nor a pointer INTO anything under the pointer could escape.
scope on a by-value parameter that contains pointers (like
delegates/slices) has meaning, I presume scope would be transitive like
const, apply the pointer rule above.
ref is sugar for a pointer, and the above applies verbatim. If you take the
address of the ref argument, you have the pointer, and it mustn't escape,
likewise, no pointers or pointer to anything beneath it.

If my imagination of this concept completely wrong?
This sounds useful to me, I can't imagine another scenario where the
keyword a) makes sense, and b) is useful...

In general though, putting scope on struct parameters would cause a lot of

 problems, because of arrays that they might hold and whatnot. Slices
 wouldn't
 be able to escape (and so copies of the struct wouldn't be able escape
 without
 deep copying, let alone the array itself).



That would be the point though. If you don't want that, then you don't want
scope.

That said, I would suggest that it may be considered, for practicality,
that immutable members WERE allowed to escape from scope controlled
structs...
I think this makes sense, and is certainly practical. Strings are the most
likely thing to fall under the scenario you illustrated above. Deep-copying
strings (or any similarly immutable data) would be a bit silly.

So, while scope may be very useful

 in some such cases (assuming that it worked), it's not necessarily
 something
 that you'd want as a matter of course.



Of course not.

Part of it probably depends on your

 programming style though. If you have a lot of functions that take
 arguments
 and don't return anything that was in them ever, then scope is less of a
 big
 deal, but that's the sort of thing that happens a _lot_ in my experience,
 so
 scope would very quickly become extremely annoying.

 And actually, to make matters worse, I'm not sure that scope on delegates
 is
 working correctly. I thought that it was, but this code compiles:

 import std.stdio;

 void delegate() global;

 void foo(scope void delegate() del)
 {
     global =3D del;
 }

 void main()
 {
     {
         char[5] bar =3D "hello";
         foo((){writeln(bar);});
     }
     char[7] baz =3D "goodbye";

     global();
 }

 It also prints out "hello", and if a closure had not been allocated, I
 would
 have at least half-expected it to print out "goodb", because I'd have
 thought
 that baz would have been taking up the same memory that bar had been. So,
 it
 looks like scope may be completely and utterly broken at this point. I
 don't
 know.

 - Jonathan M Davis
Nov 11 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 11, 2012 13:30:12 Manu wrote:

 What do you mean 'aren't really references in the normal sense'?


ref is not really part of a variable's type. It's just a storage class and is 
really only applicable in very specific circumstances. A function parameter is 
a local variable, but ref makes it so that it just so happens to affect a 
variable outside the function as well. The semantics of the function itself or 
how the parameter is used aren't any different either way. So, if you were to 
remove ref from the parameter, the function itself would be unaffected 
semantically-speaking (it would affect the code generation some though). It's 
the caller whose semantics change.

This is in direct contrast with a pointer or reference type where the fact 
that it refers to something else outside the function is fully part of the 
type.

Taking the address of a local variable is already something that is incredibly 
unsafe and doesn't work. So, anyone doing it is being an idiot anyway. And if 
it's _always_ stupid, then protecting it with scope isn't really necessary. 
pure mostly stops anything like that from happening in the general case, but 
doesn't prevent people from doing stupid stuff like

int* foo() pure
{
    int i;
    return bar(&i);
}

int* bar(int* p) pure
{
    return p;
}

Regardless, my point was that the refness of the parameter isn't really part 
of the type of the parameter, whereas all other issues with reference escaping 
that scope would affect _are_ part of the type. So, you're talking about scope 
protecting against something quite different from what it would protect from in 
all other circumstances. Protecting against escaping a ref parameter and a 
reference type escaping are two different (albeit not completely unrelated) 
things.

Honestly, before you brought up the possibility of scope protecting against 
pointers to ref parameters escaping, it had never occurred to me. I've never 
seen anyone bring it up before. And as you're talking about protecting against 
escaping a pointer to a local variable, which is an incredibly stupid thing to 
do anyway, I'm not sure that the protection is really needed.

But I don't know what Walter's or Andrei's intentions were with regards to 
whether scope would extend to the fact that the parameter is ref. I don't 
think that it's necessarily the case that it wouldn't, but you're then 
protecting against escaping a pointer to a local variable (albeit one which 
affects a variable outside of the function) rather than against escaping a 
reference type. And from everything I've seen, it's protecting against 
escaping reference types which was its purpose.

Unfortunately, TDPL doesn't seem to discuss scope parameters at all (I thought 
that it did, but I can't find it anywhere now if it does), meaning that the 
online docs are the only official documentation on scope, and all other 
information on it comes from newsgroup discussions on the matter. It's quite 
possible that many of us have misunderstood aspects of what scope parameters 
were intended to do.

So, assuming that if/when scope actually starts affecting more than delegates 
as quite a few us think that it's supposed to do, then what you're looking for 
may very well be on the table, much as I wouldn't have expected it to be. I 
don't know.

Feel free to create an enhancement request for it. Even it wasn't part of the 
original intention of scope parameters, Walter may think that it's worth 
making it so that it is.


 That would be the point though. If you don't want that, then you don't want
 scope.


The problem is that in most cases, I really don't think that that is what the 
average programmer wants, and there _are_ people who use in as a matter 
course. It would be one thing if you had to explicitly use scope, then I 
suspect that it wouldn't be used as a matter of course by much of anyone. For 
the most part, it would then just be used when the programmer knew that that's 
what they wanted. But with in, so many people seem to really like the concept 
of it being the opposite of out that they use it as a matter of course without 
understanding the consequences.

- Jonathan M Davis
Nov 11 2012
prev sibling parent reply luka8088 <luka8088 owave.net> writes:
What about making this a default behavior and introducing a new keyword 
if the function wants to modify the argument but it is not ref (pass by 
value) ? The reason I think that this should be a default behavior 
because not many functions actually modify their arguments and so it 
leaves a lot of space for optimization.

For example:

void f (int x, val int y, ref int z) {
   x = 1; // x is not copied
          // compiler throws an error, x is not passed by value
          // and therefor could not / should not be changed
   y = 2; // ok, y is copied
   z = 3; // ok, z is a reference
}

On 18.10.2012 5:07, Malte Skarupke wrote:

 Hello,

 I realize that this has been discussed before, but so far there is no
 solution and this really needs to be a high priority:

 We need a way for a function to declare that it doesn't want it's
 argument to be copied, but it also doesn't care whether the argument is
 an rvalue or an lvalue.

 The C++ way of doing this would be to declare the argument as a const &.
 Apparently it is not desired that we do the same thing for const ref.

 Currently, if you want that behavior, you have to write 2^n permutations
 of your function, with n being the number of arguments that the function
 takes.

 Here's my attempt at passing a struct to a function that takes three
 arguments without the struct being copied:

 int copyCounter = 0;
 struct CopyCounter
 {
 this(this) { ++copyCounter; }
 }
 void takeThree(ref in CopyCounter a, ref in CopyCounter b, ref in
 CopyCounter c)
 {
 writeln("took three");
 }
 void takeThree(in CopyCounter a, ref in CopyCounter b, ref in
 CopyCounter c)
 {
 takeThree(a, b, c);
 }
 void takeThree(ref in CopyCounter a, in CopyCounter b, ref in
 CopyCounter c)
 {
 takeThree(a, b, c);
 }
 void takeThree(ref in CopyCounter a, ref in CopyCounter b, in
 CopyCounter c)
 {
 takeThree(a, b, c);
 }
 void takeThree(in CopyCounter a, in CopyCounter b, ref in CopyCounter c)
 {
 takeThree(a, b, c);
 }
 void takeThree(in CopyCounter a, ref in CopyCounter b, in CopyCounter c)
 {
 takeThree(a, b, c);
 }
 void takeThree(ref in CopyCounter a, in CopyCounter b, in CopyCounter c)
 {
 takeThree(a, b, c);
 }
 void takeThree(in CopyCounter a, in CopyCounter b, in CopyCounter c)
 {
 takeThree(a, b, c);
 }
 static CopyCounter createCopyCounter()
 {
 return CopyCounter();
 }
 void main()
 {
 CopyCounter first;
 CopyCounter second;
 CopyCounter third;
 takeThree(first, second, third);
 takeThree(createCopyCounter(), second, createCopCounter());
 assert(copyCounter == 0); // yay, works
 }


 My propsed solution is this:
 - Make functions that take "ref in" arguments also accept rvalues.
 - The user can still provide an overload that accepts an rvalue, using
 the "in" keyword, and that one will be preferred over the "ref in" version.


 What do you think?

 Malte
Nov 12 2012
parent reply "martin" <kinke libero.it> writes:
On Monday, 12 November 2012 at 23:38:43 UTC, luka8088 wrote:

 What about making this a default behavior and introducing a new 
 keyword if the function wants to modify the argument but it is 
 not ref (pass by value) ? The reason I think that this should 
 be a default behavior because not many functions actually 
 modify their arguments and so it leaves a lot of space for 
 optimization.

 For example:

 void f (int x, val int y, ref int z) {
   x = 1; // x is not copied
          // compiler throws an error, x is not passed by value
          // and therefor could not / should not be changed
   y = 2; // ok, y is copied
   z = 3; // ok, z is a reference
 }


Your proposal isn't really related to this thread's topic, but I 
understand what you mean (although your code comments distract me 
a bit):

void f(const int x, int y, ref int z); =>
void f(int x, val/mutable int y, ref int z);

I use const/in ;) parameters a lot in my code too to prevent 
accidental modifications, so my function signatures may be more 
compact by treating normal pass-by-value parameters as const if 
not denoted with a special keyword. I guess it wouldn't be very 
important for optimization though because I'd expect the 
optimizer to detect unchanged parameters. Anyway, your proposal 
would completely break existing code.
Nov 12 2012
parent reply luka8088 <luka8088 owave.net> writes:
On 13.11.2012 2:16, martin wrote:

 On Monday, 12 November 2012 at 23:38:43 UTC, luka8088 wrote:

 What about making this a default behavior and introducing a new
 keyword if the function wants to modify the argument but it is not ref
 (pass by value) ? The reason I think that this should be a default
 behavior because not many functions actually modify their arguments
 and so it leaves a lot of space for optimization.

 For example:

 void f (int x, val int y, ref int z) {
 x = 1; // x is not copied
 // compiler throws an error, x is not passed by value
 // and therefor could not / should not be changed
 y = 2; // ok, y is copied
 z = 3; // ok, z is a reference
 }


 Your proposal isn't really related to this thread's topic, but I


Um, "Const ref and rvalues again", I suggest it to be the default 
behavior, how is this not related ?


 understand what you mean (although your code comments distract me a bit):

 void f(const int x, int y, ref int z); =>
 void f(int x, val/mutable int y, ref int z);


Yes, you understood correctly:
void f (const ref int x, int y, ref int z); =>
void f (int x, val int y, ref int z);

The point here is to make "We need a way for a function to declare that 
it doesn't want it's argument to be copied, but it also doesn't care 
whether the argument is an rvalue or an lvalue. " a default behavior.


 I use const/in ;) parameters a lot in my code too to prevent accidental
 modifications, so my function signatures may be more compact by treating
 normal pass-by-value parameters as const if not denoted with a special
 keyword. I guess it wouldn't be very important for optimization though
 because I'd expect the optimizer to detect unchanged parameters. Anyway,
 your proposal would completely break existing code.


Would it ? How many functions actually change their non ref/out 
arguments ? Can you point out any existing public code that would be 
broken ?
Nov 13 2012
next sibling parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Tuesday, 13 November 2012 at 08:34:19 UTC, luka8088 wrote:

 Would it ? How many functions actually change their non ref/out 
 arguments ? Can you point out any existing public code that 
 would be broken ?


  It would be possible that if the language became 
const-preference that a simple regex tool could be made that 
would do the conversions, thereby any code broken in this way 
could be un-broken just as easily; But that assumes you aren't 
using mixins or magic as part of your signatures.

  Somehow this reminds me a little of when I worked at a company 
where we were trying out asp as a web server; The whole VB script 
was by default 'by ref' so you littered all your functions with 
'byVal' in order for your behavior to act as you expected.


  Anyways, my take on this is consistency would be a lot more 
difficult and annoying unless you had different rules for the 
signature vs all other references... I doubt you would say 'this 
is mutable here but immutable here' type of thing. So.... 
assuming 'mutable' is used, then the following would be 
comparable...

//D as of now
int func(int x, int y, const ref int z) {
   int something; //mutable far more likely
   int something2;
   const int lessOften;
}

//then would become...
//if x & y aren't ever changed then mutable may be unneeded.
mutable int func(mutable int x, mutable int y, ref int z) {
   mutable int something;
   mutable int something2;
   int lessOften;      //const (once set)
}

//or for inconsistancy..
//mutable or const as a return? (Or either?)
//and which would/should you use to reverse it?
int func(mutable int x, mutable int y, ref int z) {
   int something;       //mutable
   int something2;
   const int lessOften; //const
}

  Seems in a function body you are far more likely to have mutable 
items, while in the signature you're more likely to have const 
items; But mixing them or changing how you do it would likely 
break code very easily if it isn't signature only, but it doesn't 
seem like a good idea...

  Now in the above the function may not specify 'x' is const it 
doesn't guarantees it ever changes it (but it's a local copy so 
does it matter?), but specifically specifying it may be more 
clutter than actually useful.

  All in all it seems like it would have far more confusion (and 
break code) than help; although having it prefer const versions 
of functions/methods to non-const ones should probably have a 
higher priority (Although you then couldn't have a non-const one 
unless it was as part of the struct/class constness and not the 
variables, (and ref preferred over non-ref)).
Nov 13 2012
parent reply luka8088 <luka8088 owave.net> writes:
On 13.11.2012 11:00, Era Scarecrow wrote:

 On Tuesday, 13 November 2012 at 08:34:19 UTC, luka8088 wrote:

 Would it ? How many functions actually change their non ref/out
 arguments ? Can you point out any existing public code that would be
 broken ?


 It would be possible that if the language became const-preference that a
 simple regex tool could be made that would do the conversions, thereby
 any code broken in this way could be un-broken just as easily; But that
 assumes you aren't using mixins or magic as part of your signatures.

 Somehow this reminds me a little of when I worked at a company where we
 were trying out asp as a web server; The whole VB script was by default
 'by ref' so you littered all your functions with 'byVal' in order for
 your behavior to act as you expected.


 Anyways, my take on this is consistency would be a lot more difficult
 and annoying unless you had different rules for the signature vs all
 other references... I doubt you would say 'this is mutable here but
 immutable here' type of thing. So.... assuming 'mutable' is used, then
 the following would be comparable...

 //D as of now
 int func(int x, int y, const ref int z) {
 int something; //mutable far more likely
 int something2;
 const int lessOften;
 }

 //then would become...
 //if x & y aren't ever changed then mutable may be unneeded.
 mutable int func(mutable int x, mutable int y, ref int z) {
 mutable int something;
 mutable int something2;
 int lessOften; //const (once set)
 }

 //or for inconsistancy..
 //mutable or const as a return? (Or either?)
 //and which would/should you use to reverse it?
 int func(mutable int x, mutable int y, ref int z) {
 int something; //mutable
 int something2;
 const int lessOften; //const
 }

 Seems in a function body you are far more likely to have mutable items,
 while in the signature you're more likely to have const items; But
 mixing them or changing how you do it would likely break code very
 easily if it isn't signature only, but it doesn't seem like a good idea...

 Now in the above the function may not specify 'x' is const it doesn't
 guarantees it ever changes it (but it's a local copy so does it
 matter?), but specifically specifying it may be more clutter than
 actually useful.

 All in all it seems like it would have far more confusion (and break
 code) than help; although having it prefer const versions of
 functions/methods to non-const ones should probably have a higher
 priority (Although you then couldn't have a non-const one unless it was
 as part of the struct/class constness and not the variables, (and ref
 preferred over non-ref)).


Can you point out any existing public code that would be broken ?
Nov 13 2012
parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Tuesday, 13 November 2012 at 10:09:27 UTC, luka8088 wrote:

 Can you point out any existing public code that would be broken?


  Off hand, no.. I'm not that familiar with a lot of the code or 
the in depth details of phoboes; However suddenly reversing what 
is mutable and what is const is bound to break a lot of things 
even unintentionally.

  Hmmm.. Does remind me of a bit of my code sometimes... Something 
went like...

  class S {
    void func(S s) {
      if (!s)
        s = new S(); //create one if not passed in.

      //process using S and s (be it new or passed in)
    }
  }

  That would certainly cause a problem.. In my own code example it 
may empty the pointer/reference if certain requirements were met, 
and then use if the class reference was set or null for later 
logic (even if the object referenced to wasn't changed...). 
Course being passed something like a string... (seems the most 
likely place you'd find it), in which case using .dup on the 
input string right back into the variable would be completely 
understandable. ie:

string someStringTransformationWithCOW(string x) {
   //if COW'd then
   x = x.dup;

   //...
   return x;
}

  Course having the input as char[] rather than string makes more 
sense for on the fly changes before returning it...
Nov 13 2012
prev sibling parent reply "martin" <kinke libero.it> writes:
On Tuesday, 13 November 2012 at 08:34:19 UTC, luka8088 wrote:

 Your proposal isn't really related to this thread's topic


 Um, "Const ref and rvalues again", I suggest it to be the 
 default behavior, how is this not related ?


The topic here is binding rvalues to (const) ref parameters. You, 
on the other hand, are suggesting to flip the constness of 
by-value parameters (int => val/mutable int, const int => int), 
which affects both rvalues and lvalues (no difference between 
them) and only by-value parameters.


 Yes, you understood correctly:
 void f (const ref int x, int y, ref int z); =>
 void f (int x, val int y, ref int z);

 The point here is to make "We need a way for a function to 
 declare that it doesn't want it's argument to be copied, but it 
 also doesn't care whether the argument is an rvalue or an 
 lvalue. " a default behavior.


So now tell me why argument x wouldn't be copied. It's passed by 
value, so of course it is copied (lvalues)/moved (rvalues) just 
as it is now. The only difference is that the parameter won't be 
modified by f().

I guess what you furthermore implicate is that you'd expect the 
compiler to automatically pass appropriate arguments to such 
parameters by reference to avoid copying (for large structs or 
structs with non-trivial copy constructors). Such a (handy!) 
optimization is sadly not possible due to aliasing issues, e.g.:

int foo(ref int dst, const int src)
{
     dst = 2*src;
     return src;
}
// "optimized" foo():
int bar(ref int dst, const ref int src)
{
     dst = 2*src;
     return src;
}

int i = 1;
assert(foo(i, i) == 1 && i == 2); // okay
i = 1;
assert(bar(i, i) == 2 && i == 2); // wtf?!
// the const src parameter is actually modified since the
// original argument i is also used as mutable dst parameter!


 Would it ? How many functions actually change their non ref/out 
 arguments ? Can you point out any existing public code that 
 would be broken ?


I don't want to look for examples in Phobos etc. as it should be 
trivial to imagine cases such as:

void bla(float x)
{
     // restrict x to safe range [0,1]
     x = max(0, min(1, x));
}
Nov 13 2012
parent luka8088 <luka8088 owave.net> writes:
On 13.11.2012 15:07, martin wrote:

 On Tuesday, 13 November 2012 at 08:34:19 UTC, luka8088 wrote:

 Your proposal isn't really related to this thread's topic


 Um, "Const ref and rvalues again", I suggest it to be the default
 behavior, how is this not related ?


 The topic here is binding rvalues to (const) ref parameters. You, on the
 other hand, are suggesting to flip the constness of by-value parameters
 (int => val/mutable int, const int => int), which affects both rvalues
 and lvalues (no difference between them) and only by-value parameters.


 Yes, you understood correctly:
 void f (const ref int x, int y, ref int z); =>
 void f (int x, val int y, ref int z);

 The point here is to make "We need a way for a function to declare
 that it doesn't want it's argument to be copied, but it also doesn't
 care whether the argument is an rvalue or an lvalue. " a default
 behavior.


 So now tell me why argument x wouldn't be copied. It's passed by value,
 so of course it is copied (lvalues)/moved (rvalues) just as it is now.
 The only difference is that the parameter won't be modified by f().

 I guess what you furthermore implicate is that you'd expect the compiler
 to automatically pass appropriate arguments to such parameters by
 reference to avoid copying (for large structs or structs with
 non-trivial copy constructors). Such a (handy!) optimization is sadly
 not possible due to aliasing issues, e.g.:

 int foo(ref int dst, const int src)
 {
 dst = 2*src;
 return src;
 }
 // "optimized" foo():
 int bar(ref int dst, const ref int src)
 {
 dst = 2*src;
 return src;
 }

 int i = 1;
 assert(foo(i, i) == 1 && i == 2); // okay
 i = 1;
 assert(bar(i, i) == 2 && i == 2); // wtf?!
 // the const src parameter is actually modified since the
 // original argument i is also used as mutable dst parameter!


 Would it ? How many functions actually change their non ref/out
 arguments ? Can you point out any existing public code that would be
 broken ?


 I don't want to look for examples in Phobos etc. as it should be trivial
 to imagine cases such as:

 void bla(float x)
 {
 // restrict x to safe range [0,1]
 x = max(0, min(1, x));
 }


I see, you are correct, if it is not copied then it can be changed 
before function finished through some other references hence it must be 
copied.
Nov 14 2012