www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Const ref and rvalues again...

reply "Malte Skarupke" <malteskarupke web.de> writes:
Hello,

I realize that this has been discussed before, but so far there 
is no solution and this really needs to be a high priority:

We need a way for a function to declare that it doesn't want it's 
argument to be copied, but it also doesn't care whether the 
argument is an rvalue or an lvalue.

The C++ way of doing this would be to declare the argument as a 
const &. Apparently it is not desired that we do the same thing 
for const ref.

Currently, if you want that behavior, you have to write 2^n 
permutations of your function, with n being the number of 
arguments that the function takes.

Here's my attempt at passing a struct to a function that takes 
three arguments without the struct being copied:

int copyCounter = 0;
struct CopyCounter
{
     this(this) { ++copyCounter; }
}
void takeThree(ref in CopyCounter a, ref in CopyCounter b, ref in 
CopyCounter c)
{
     writeln("took three");
}
void takeThree(in CopyCounter a, ref in CopyCounter b, ref in 
CopyCounter c)
{
     takeThree(a, b, c);
}
void takeThree(ref in CopyCounter a, in CopyCounter b, ref in 
CopyCounter c)
{
     takeThree(a, b, c);
}
void takeThree(ref in CopyCounter a, ref in CopyCounter b, in 
CopyCounter c)
{
     takeThree(a, b, c);
}
void takeThree(in CopyCounter a, in CopyCounter b, ref in 
CopyCounter c)
{
     takeThree(a, b, c);
}
void takeThree(in CopyCounter a, ref in CopyCounter b, in 
CopyCounter c)
{
     takeThree(a, b, c);
}
void takeThree(ref in CopyCounter a, in CopyCounter b, in 
CopyCounter c)
{
     takeThree(a, b, c);
}
void takeThree(in CopyCounter a, in CopyCounter b, in CopyCounter 
c)
{
     takeThree(a, b, c);
}
static CopyCounter createCopyCounter()
{
     return CopyCounter();
}
void main()
{
     CopyCounter first;
     CopyCounter second;
     CopyCounter third;
     takeThree(first, second, third);
     takeThree(createCopyCounter(), second, createCopCounter());
     assert(copyCounter == 0); // yay, works
}


My propsed solution is this:
- Make functions that take "ref in" arguments also accept rvalues.
- The user can still provide an overload that accepts an rvalue, 
using the "in" keyword, and that one will be preferred over the 
"ref in" version.


What do you think?

Malte
Oct 17 2012
next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Thursday, 18 October 2012 at 03:07:56 UTC, Malte Skarupke 
wrote:
 What do you think?

 Malte

I've asked for this before. http://d.puremagic.com/issues/show_bug.cgi?id=8121 Doesn't seem to be a priority. :(
Oct 17 2012
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, October 18, 2012 05:07:52 Malte Skarupke wrote:
 My propsed solution is this:
 - Make functions that take "ref in" arguments also accept rvalues.

There was some discussion of this, though that seems to me to be basically the same as making const ref take rvalues only worse. Certainly, it seems inherently broken to me to have a function try and take its arguement by ref and then be given an rvalue.
 - The user can still provide an overload that accepts an rvalue,
 using the "in" keyword, and that one will be preferred over the
 "ref in" version.
 
 
 What do you think?

in as alias for const scope. scope is generally not applicable to structs (and when it is, it's not applied properly - scope currently only works properly for delegates), and const just puts us back at const ref again. You appear to be suggesting that in do something different here, and I don't think that that works. The most recent, in-depth discussion on this issue can be found here: http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com It wasn't entirely conclusive, but there _was_ some discussion on making ref take rvalues. I believe that more research was necessary, and I'm not sure where any of it stands now. One of the more promising ideas (IMHO) was to simply change it so that auto ref means what it was supposed to mean originally - that the programmer wants the argument to be passed efficiently but doesn't care whether it's an lvalue or rvalue - but some people screamed about that, because they've found the current version of auto ref to be useful (in spite of the fact that it doesn't do what it was supposed to do). Really, I think that this comes down to us needing to figure out how to make it so that ref can take lvalues under at least some set of circumstances (which seems like a bad idea to me), or we need to come up with a parameter attribute which indicates that we don't want the argument copied and don't care if it's an lvalue or rvalue (like const& is usually used for in C++ but making it so that it's _exactly_ that rather than just being primarily used for that). And if we can't use auto ref for that, then we need to come up with a new attribute of some kind to do that. However, IIRC Kenji tried to change auto ref to do that but ran into issues of some kind. I don't know where it stands now. - Jonathan M Davis
Oct 17 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/19/2012 01:39 AM, Malte Skarupke wrote:
 On Thursday, 18 October 2012 at 06:11:26 UTC, monarch_dodra wrote:
 On Thursday, 18 October 2012 at 04:30:17 UTC, Jonathan M Davis wrote:
 On Thursday, October 18, 2012 06:24:08 jerro wrote:
 What would be the problem with const ref taking rvalues?

Read the thread that I already linked to: http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com - Jonathan M Davis

I read the thread, and not a single one of the "problematic cases" are actually valid C++. Yes: the faulty MSVC has taught people to do retarded things, or be afraid of things that were illegal to begin with (in particular, pass an rvalue to a ref, WHICH IS ILLEGAL IN C++), such as "increment(5)". There is actually nothing wrong with creating a temporary when something is bound to a const ref, provided the compiler follows the rules: *Only LValues with an EXACT type match may be passed to a reference. *In regards to *const* references, RValues may be copied in a temporary, and that temporary bound the the ref. I'm not saying we particularly *need* this in D (C++ has a "by ref" paradigm that makes it more important, but D *rarelly* ever passes by const ref). But if the compiler respects the above two rules (which it should), then RValue to const ref is both perfectly doable and safe (as safe as refs get anyways).

The problem with binding rvalues to const ref is that you could take and store the address of it. That's why I'd recommend using "in ref" instead. Jonathan: I had already read the linked discussion. There are many valid points in there, but also many invalid ones (as monarch_dodra has pointed out). But I think all problems in that thread should be solved by using "in ref" instead of "const ref" because then you'd be sure that the passed-in temporary can not escape the current function. foobar: I like the idea, but it's probably going to break down in many cases. If you have a non-trivial copy constructor you want the ability to have complete control over when it gets copied and when it doesn't. I just don't trust compilers enough to think that they'd always make the same choice that I'd make. And also about losing semantic information: That's why I proposed the second point: Give the user the option to provide a function which should be preferred for rvalues. That way you don't lose semantic information.

Const is different in D and in C++. Relating const and rvalues is arbitrary and does not make a lot of sense. Regarding 'in ref'/'scope ref': What should 'scope' apply to in void foo(scope ref int* x);
Oct 18 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/19/2012 03:26 AM, Malte Skarupke wrote:
 On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:
 Const is different in D and in C++. Relating const and rvalues is
 arbitrary and does not make a lot of sense.

 Regarding 'in ref'/'scope ref': What should 'scope' apply to in

 void foo(scope ref int* x);

Not sure what you mean with "relating." I'm not making any claims about there being a relationship between rvalues and constness.

You do it again right away:
 This is about finding a way that you can define a function which safely
 accepts lvalues and rvalues without having to make a copy. If we specify
 the argument as "ref in", then we can safely pass for example the number

const
 5 to it.

rvalue
 And this would never break existing code, so that something
 like swap(5, 4) would never be possible code.

That does not break existing code except code that checks validity of code, but every language change does that.
 For the example that you gave you'd be unable to store the address of x.
 So doing

 int** storage;
 void foo(scope ref int * x)
 {
      storage = &x;
 }

 would be illegal.

Then how to specify that the value of x cannot be escaped? I'm in favour of doing it the other way round and disallow escaping of ref parameters without an unsafe cast.
  jerro: the same thing: I'm not trying to fix the problem that you
 mention. I'm trying to define a function which can safely accept rvalues
 and lvalues without having to make a copy.

Oct 18 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-10-19 04:48, Timon Gehr wrote:

 Then how to specify that the value of x cannot be escaped?
 I'm in favour of doing it the other way round and disallow escaping of
 ref parameters without an unsafe cast.

"scope" is supposed to be used to prevent this. -- /Jacob Carlborg
Oct 19 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/19/2012 09:53 AM, Jacob Carlborg wrote:
 On 2012-10-19 04:48, Timon Gehr wrote:

 Then how to specify that the value of x cannot be escaped?
 I'm in favour of doing it the other way round and disallow escaping of
 ref parameters without an unsafe cast.

"scope" is supposed to be used to prevent this.

Sure, but how? void goo(scope int* x){ global0 = x; // should clearly be disallowed } void foo(scope ref int*** x){ global1 = &x; // ? global2 = x; // ? global3 = *x; // ? globall4 = **x; // ? } Maybe we need this: void foo(scope ref int*** x); // ? void foo(ref int(***)scope x); // no escaping of x, *x, **x void foo(ref int*(**)scope x); // may escape **x void foo(ref int**(*)scope x); // may escape *x, **x What about &x?
Oct 19 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 10/20/2012 08:25 PM, Malte Skarupke wrote:
 ...

 So it comes down to this:

 void goo(scope int* x)
 {
      global0 = x; // copying is allowed
      //global1 = &x; // referencing is not allowed
 }

That is useless.
Oct 20 2012
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/22/2012 12:18 AM, martin wrote:
 On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:
 Const is different in D and in C++. Relating const and rvalues is
 arbitrary and does not make a lot of sense.

It's actually pretty much the same concept in both languages except for the transitiveness in D.

Case closed.
Oct 22 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 10/22/2012 11:38 AM, Timon Gehr wrote:
 On 10/22/2012 12:18 AM, martin wrote:
 On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:
 Const is different in D and in C++. Relating const and rvalues is
 arbitrary and does not make a lot of sense.

It's actually pretty much the same concept in both languages except for the transitiveness in D.

Case closed.

(But it might be added that C++11 adds a new language feature just in order to be able to distinguish const and rvalue references in the callee, so even with C++'s semantics, this does not seem like a good idea.)
Oct 22 2012
prev sibling next sibling parent "jerro" <a a.com> writes:
On Thursday, 18 October 2012 at 03:31:41 UTC, Jonathan M Davis 
wrote:
 On Thursday, October 18, 2012 05:07:52 Malte Skarupke wrote:
 My propsed solution is this:
 - Make functions that take "ref in" arguments also accept 
 rvalues.

There was some discussion of this, though that seems to me to be basically the same as making const ref take rvalues only worse.

What would be the problem with const ref taking rvalues?
Oct 17 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, October 18, 2012 06:24:08 jerro wrote:
 What would be the problem with const ref taking rvalues?

Read the thread that I already linked to: http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com - Jonathan M Davis
Oct 17 2012
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 18 October 2012 at 04:30:17 UTC, Jonathan M Davis 
wrote:
 On Thursday, October 18, 2012 06:24:08 jerro wrote:
 What would be the problem with const ref taking rvalues?

Read the thread that I already linked to: http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com - Jonathan M Davis

I read the thread, and not a single one of the "problematic cases" are actually valid C++. Yes: the faulty MSVC has taught people to do retarded things, or be afraid of things that were illegal to begin with (in particular, pass an rvalue to a ref, WHICH IS ILLEGAL IN C++), such as "increment(5)". There is actually nothing wrong with creating a temporary when something is bound to a const ref, provided the compiler follows the rules: *Only LValues with an EXACT type match may be passed to a reference. *In regards to *const* references, RValues may be copied in a temporary, and that temporary bound the the ref. I'm not saying we particularly *need* this in D (C++ has a "by ref" paradigm that makes it more important, but D *rarelly* ever passes by const ref). But if the compiler respects the above two rules (which it should), then RValue to const ref is both perfectly doable and safe (as safe as refs get anyways).
Oct 17 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Thursday, 18 October 2012 at 06:11:26 UTC, monarch_dodra wrote:
 On Thursday, 18 October 2012 at 04:30:17 UTC, Jonathan M Davis 
 wrote:
 On Thursday, October 18, 2012 06:24:08 jerro wrote:
 What would be the problem with const ref taking rvalues?

Read the thread that I already linked to: http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com - Jonathan M Davis

I read the thread, and not a single one of the "problematic cases" are actually valid C++. Yes: the faulty MSVC has taught people to do retarded things, or be afraid of things that were illegal to begin with (in particular, pass an rvalue to a ref, WHICH IS ILLEGAL IN C++), such as "increment(5)". There is actually nothing wrong with creating a temporary when something is bound to a const ref, provided the compiler follows the rules: *Only LValues with an EXACT type match may be passed to a reference. *In regards to *const* references, RValues may be copied in a temporary, and that temporary bound the the ref. I'm not saying we particularly *need* this in D (C++ has a "by ref" paradigm that makes it more important, but D *rarelly* ever passes by const ref). But if the compiler respects the above two rules (which it should), then RValue to const ref is both perfectly doable and safe (as safe as refs get anyways).

By allowing the the C++ semantics the function looses semantic information - whether the actual parameter was lvalue or rvalue. This semantic info can be used bot for compiler optimizations and move semantics. This is the reason C++11 added && references. General question (might not be relevant to current design of D): How about leaving the decision to the compiler and let the programmer only specify usage intent? E.g.: (I'm speaking semantics here, not syntax) void foo(const Type t); // 1. I only read the value void foo (mutate Type t); // 2. I want to also mutate the actual parameter void foo (move Type t); // 3. I want to move the actual parameter In case 1 above, the compiler is free to pass lvalues by const& and rvalues by value or perhaps optimize above certain size to const& too. In case 2, the compiler passes a ref to lvalue, rvalues are not accepted at CT. If I want move semantics, I can use option 3 which accepts rvalues by ref. btw, what's the correct semantics for lvalues here? What do you think?
Oct 18 2012
prev sibling next sibling parent "Malte Skarupke" <malteskarupke web.de> writes:
On Thursday, 18 October 2012 at 06:11:26 UTC, monarch_dodra wrote:
 On Thursday, 18 October 2012 at 04:30:17 UTC, Jonathan M Davis 
 wrote:
 On Thursday, October 18, 2012 06:24:08 jerro wrote:
 What would be the problem with const ref taking rvalues?

Read the thread that I already linked to: http://forum.dlang.org/thread/4F84D6DD.5090405 digitalmars.com - Jonathan M Davis

I read the thread, and not a single one of the "problematic cases" are actually valid C++. Yes: the faulty MSVC has taught people to do retarded things, or be afraid of things that were illegal to begin with (in particular, pass an rvalue to a ref, WHICH IS ILLEGAL IN C++), such as "increment(5)". There is actually nothing wrong with creating a temporary when something is bound to a const ref, provided the compiler follows the rules: *Only LValues with an EXACT type match may be passed to a reference. *In regards to *const* references, RValues may be copied in a temporary, and that temporary bound the the ref. I'm not saying we particularly *need* this in D (C++ has a "by ref" paradigm that makes it more important, but D *rarelly* ever passes by const ref). But if the compiler respects the above two rules (which it should), then RValue to const ref is both perfectly doable and safe (as safe as refs get anyways).

The problem with binding rvalues to const ref is that you could take and store the address of it. That's why I'd recommend using "in ref" instead. Jonathan: I had already read the linked discussion. There are many valid points in there, but also many invalid ones (as monarch_dodra has pointed out). But I think all problems in that thread should be solved by using "in ref" instead of "const ref" because then you'd be sure that the passed-in temporary can not escape the current function. foobar: I like the idea, but it's probably going to break down in many cases. If you have a non-trivial copy constructor you want the ability to have complete control over when it gets copied and when it doesn't. I just don't trust compilers enough to think that they'd always make the same choice that I'd make. And also about losing semantic information: That's why I proposed the second point: Give the user the option to provide a function which should be preferred for rvalues. That way you don't lose semantic information.
Oct 18 2012
prev sibling next sibling parent "jerro" <a a.com> writes:
 The problem with binding rvalues to const ref is that you could 
 take and store the address of it. That's why I'd recommend 
 using "in ref" instead.

You can also take and store the address of a local variable that was passed as a const ref parameter, and accessing it after the caller exits will result in undefined behavior too. On the other hand, addresses of both local variables and rvalues will be valid at least until the called function returns. Local variables and rvalues are equivalent in regard to this problem.
Oct 18 2012
prev sibling next sibling parent "Malte Skarupke" <malteskarupke web.de> writes:
On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:
 Const is different in D and in C++. Relating const and rvalues 
 is arbitrary and does not make a lot of sense.

 Regarding 'in ref'/'scope ref': What should 'scope' apply to in

 void foo(scope ref int* x);

Not sure what you mean with "relating." I'm not making any claims about there being a relationship between rvalues and constness. This is about finding a way that you can define a function which safely accepts lvalues and rvalues without having to make a copy. If we specify the argument as "ref in", then we can safely pass for example the number 5 to it. And this would never break existing code, so that something like swap(5, 4) would never be possible code. For the example that you gave you'd be unable to store the address of x. So doing int** storage; void foo(scope ref int * x) { storage = &x; } would be illegal. jerro: the same thing: I'm not trying to fix the problem that you mention. I'm trying to define a function which can safely accept rvalues and lvalues without having to make a copy.
Oct 18 2012
prev sibling next sibling parent "jerro" <a a.com> writes:
  jerro: the same thing: I'm not trying to fix the problem that 
 you mention. I'm trying to define a function which can safely 
 accept rvalues and lvalues without having to make a copy.

My point was that saving an address of a const ref parameter is already unsafe if you call the function with a local variable as the parameter. If this behavior seems problematic to you when it concerns rvalues, it should seem equally problematic when it comes to local variables. It doesn't make sense to make passing rvalues as const ref parameters illegal because of this problem, when passing local variables causes the same problem and is legal. It would only make sense to introduce "in ref" whith an intent to solve this problem, if local variables would also have to be passed as "in ref" (or some kind of scope ref, not necessarily const). But that would break pretty much all code that uses ref. The only case I can think of when passing a local variable as const ref is safe, but passing an rvalue wouldn't be, is when the called function returns the address of the const parameter (or assigns it to some other ref parameter).
Oct 18 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Friday, 19 October 2012 at 07:53:30 UTC, Jacob Carlborg wrote:
 On 2012-10-19 04:48, Timon Gehr wrote:

 Then how to specify that the value of x cannot be escaped?
 I'm in favour of doing it the other way round and disallow 
 escaping of
 ref parameters without an unsafe cast.

"scope" is supposed to be used to prevent this.

I like Timon's idea. "scope" is a bad fit in that the default should be the safe option. Unfortunately this does have the potential to brake lots of code, perhaps even if we limit it to safe code. An argument could be made that it's worth the breakage for safe code to insure better safety.
Oct 19 2012
prev sibling next sibling parent "Malte Skarupke" <malteskarupke web.de> writes:
On Friday, 19 October 2012 at 13:00:52 UTC, Timon Gehr wrote:
 On 10/19/2012 09:53 AM, Jacob Carlborg wrote:
 On 2012-10-19 04:48, Timon Gehr wrote:

 Then how to specify that the value of x cannot be escaped?
 I'm in favour of doing it the other way round and disallow 
 escaping of
 ref parameters without an unsafe cast.

"scope" is supposed to be used to prevent this.

Sure, but how? void goo(scope int* x){ global0 = x; // should clearly be disallowed } void foo(scope ref int*** x){ global1 = &x; // ? global2 = x; // ? global3 = *x; // ? globall4 = **x; // ? } Maybe we need this: void foo(scope ref int*** x); // ? void foo(ref int(***)scope x); // no escaping of x, *x, **x void foo(ref int*(**)scope x); // may escape **x void foo(ref int**(*)scope x); // may escape *x, **x What about &x?

No scope should mean that you can not escape the address of something. So void goo(scope int* x) { global0 = x; } Should be allowed. Scope in this case applies to the pointer. Not to the thing it's pointing to. The scope keyword is not transitive. That wouldn't make sense. It is perfectly legal to have a scoped pointer to something allocated on the heap. Whereas void goo(scope ref int x) { global0 = &x; } Should NOT be allowed. It looks like it's the same code, but in this case x is an integer. In the last example x was a pointer to an integer. So it comes down to this: void goo(scope int* x) { global0 = x; // copying is allowed //global1 = &x; // referencing is not allowed }
Oct 20 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Friday, 19 October 2012 at 00:03:49 UTC, Timon Gehr wrote:
 Const is different in D and in C++. Relating const and rvalues 
 is arbitrary and does not make a lot of sense.

It's actually pretty much the same concept in both languages except for the transitiveness in D. An implicit rvalue => _const_ ref conversion is something completely different than an rvalue => mutable ref conversion, and this distinction is sadly largely lost in the linked thread, which is why I tried to re-initiate the discussion with another recent thread. Arguments passed by mutable references are expected to be changed by the called function, so passing rvalues does indeed not make much sense in that case (side effects may get lost, quite possibly unintentionally). On Friday, 19 October 2012 at 01:26:35 UTC, Malte Skarupke wrote:
 This is about finding a way that you can define a function 
 which safely accepts lvalues and rvalues without having to make 
 a copy.

EXACTLY! Something telling the compiler: pass the argument by const reference instead of a costly const copy, if possible. So for an lvalue, pass directly its address; for an rvalue, allocate the temporary on the stack and then pass its address: <code> T foo(in ref T bla) { return bla; } T lvalue; foo(lvalue); // fine foo(foo(lvalue)); // error: rvalue! // required compiler magic: immutable tmp = foo(lvalue); foo(tmp); /* possibly in dedicated scope for immediate destruction of tmp */ </code> Currently having to allocate all rvalues manually is frankly both ugly and a pain in the ass, as would be the other alternative: overloading foo(). On Friday, 19 October 2012 at 03:49:56 UTC, jerro wrote:
 My point was that saving an address of a const ref parameter is 
 already unsafe if you call the function with a local variable 
 as the parameter. If this behavior seems problematic to you 
 when it concerns rvalues, it should seem equally problematic 
 when it comes to local variables. It doesn't make sense to make 
 passing rvalues as const ref parameters illegal because of this 
 problem, when passing local variables causes the same problem 
 and is legal. [...]
 The only case I can think of when passing a local variable as 
 const ref is safe, but passing an rvalue wouldn't be, is when 
 the called function returns the address of the const parameter 
 (or assigns it to some other ref parameter).

Exactly my thoughts regarding this escaping aspect, which I don't see as big issue. The compiler magic illustrated in the previous example without dedicated scope for temporaries would make passing rvalues exactly as safe/unsafe as passing local variables.
Oct 21 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Monday, 22 October 2012 at 09:41:38 UTC, Timon Gehr wrote:
 (But it might be added that C++11 adds a new language feature
 just in order to be able to distinguish const and rvalue
 references in the callee, so even with C++'s semantics, this
 does not seem like a good idea.)

Yeah, and that T&& syntax is mainly used for _mutable_ rvalue references in move constructors and assignment operators to move (hijack) mutable data from the rvalue to another T instance instead of copying that data, knowing that the rvalue's data is not going to be accessed anymore anyway. So this is a different case and does currently not apply to D afaik since there is no way to forward rvalues: void foo(ref T lvalue) { } void foo(T rvalue) { } In the latter overload for rvalues, you aren't given the original rvalue, but a copy of it! Distinguishing between _const_ rvalue and lvalue references though makes no sense imho, and that is my whole point.
Oct 22 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Monday, 22 October 2012 at 11:59:27 UTC, martin wrote:
 In the latter overload for rvalues, you aren't given the 
 original rvalue, but a copy of it!

I need to correct that after a quick test: the rvalue is passed directly (moved) instead of copying it (well, at least the copy constructor this(this) is not invoked, even in a debug build); that makes perfect sense, is efficient and eliminates the need for C++ rvalue references (T&&). It doesn't affect the need for an implicit rvalue => const ref propagation though. What I'd like to see is the following passing scheme for function arguments (read-only parameters are denoted by (*)): lvalue: rvalue: T: copy move (*) in T: copy move out T: pass pointer n/a ref T: pass pointer n/a (*) in ref T: pass pointer store on the caller's stack and pass its address So only the rvalue passing rule for the "in ref T" case would need to be implemented. This would allow to use "foo(in ref T bar)" for lvalues (eliding a copy expected to be costly) as well as rvalues instead of having to add an overload "foo(in T bar)" for rvalues. For rvalues, this would actually implicate a performance hit due to pointer indirection, so the compiler could attempt to add an automatic "foo(in T bar)" overload if not existent. For rvalues, the latter overload (i.e., "in T" parameters) should be preferred over "in ref T" parameters - exactly as the thread starter Malte proposes:
 - Make functions that take "ref in" arguments also accept 
 rvalues.
 - The user can still provide an overload that accepts an 
 rvalue, using the "in" keyword, and that one will be preferred 
 over the "ref in" version.

Oct 22 2012
prev sibling next sibling parent reply "Rob T" <rob ucora.com> writes:
On Thursday, 18 October 2012 at 03:07:56 UTC, Malte Skarupke 
wrote:
 Hello,

 I realize that this has been discussed before, but so far there 
 is no solution and this really needs to be a high priority:

 We need a way for a function to declare that it doesn't want 
 it's argument to be copied, but it also doesn't care whether 
 the argument is an rvalue or an lvalue.

I'm encountering this problem right now, and found this thread while looking for a solution. In my case I'm forced into doubling up some code, which is not what I want to be doing, so I agree this item should be fixed. It would be nice to have something like the proposed "auto ref" solution, like we have with return values. --rt
Nov 04 2012
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/4/12 7:58 PM, martin wrote:
 I find it sad that while this topic seems to be of high priority for
 quite a lot of language users, it is seemingly neglected by the head of
 language development (Walter, Andrei etc.).

I was hoping "auto ref" solves this problem. I think it's currently only implemented for templates. Andrei
Nov 04 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
I find it sad that while this topic seems to be of high priority 
for quite a lot of language users, it is seemingly neglected by 
the head of language development (Walter, Andrei etc.).
Nov 04 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Monday, 5 November 2012 at 00:58:19 UTC, martin wrote:
 I find it sad that while this topic seems to be of high 
 priority for quite a lot of language users, it is seemingly 
 neglected by the head of language development (Walter, Andrei 
 etc.).

I don't know why this would be ignored, it's rather useful and solves a real-world programming problem. I hope this is just one of those detials that was missed and is only being ignored due to attention from much bigger problems. Do you know if there is already a feature request or bug report on this item?
Nov 04 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 04, 2012 20:43:36 Andrei Alexandrescu wrote:
 On 11/4/12 7:58 PM, martin wrote:
 I find it sad that while this topic seems to be of high priority for
 quite a lot of language users, it is seemingly neglected by the head of
 language development (Walter, Andrei etc.).

I was hoping "auto ref" solves this problem. I think it's currently only implemented for templates.

And when we argued for altering it so that it operated like const ref in C++ (which allows const ref in D to continue to function like it does now), some folks complained, because they've found the current semantics of auto ref to be useful (something to do with propagating the exact, original type, I think). Now, since auto ref currently only works with templates, maybe we can keep its current semantics with templated functions but alter them for non-templated functions so that it works like const ref does in C++. The downside is that the semantics for auto ref between templated functions and non-templated functions are slightly different, but they're close enough that I'm not sure that it matters. - Jonathan M Davis
Nov 04 2012
prev sibling next sibling parent "Malte Skarupke" <malteskarupke web.de> writes:
On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis 
wrote:
 On Sunday, November 04, 2012 20:43:36 Andrei Alexandrescu wrote:
 On 11/4/12 7:58 PM, martin wrote:
 I find it sad that while this topic seems to be of high 
 priority for
 quite a lot of language users, it is seemingly neglected by 
 the head of
 language development (Walter, Andrei etc.).

I was hoping "auto ref" solves this problem. I think it's currently only implemented for templates.

And when we argued for altering it so that it operated like const ref in C++ (which allows const ref in D to continue to function like it does now), some folks complained, because they've found the current semantics of auto ref to be useful (something to do with propagating the exact, original type, I think). Now, since auto ref currently only works with templates, maybe we can keep its current semantics with templated functions but alter them for non-templated functions so that it works like const ref does in C++. The downside is that the semantics for auto ref between templated functions and non-templated functions are slightly different, but they're close enough that I'm not sure that it matters. - Jonathan M Davis

Yes, please. Auto ref for non-template functions would solve the problem exactly. I also like it because then the intent of the programmer is clear.
Nov 04 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis
wrote:
 And when we argued for altering it so that it operated like 
 const ref in C++
 (which allows const ref in D to continue to function like it 
 does now), some
 folks complained, because they've found the current semantics 
 of auto ref to
 be useful (something to do with propagating the exact, original 
 type, I
 think).

I would expect that auto ref for a template and for a non template should work in exactly the same way, so why would there be a difference? If there must be a difference, there should be different semantics for specifying the difference, otherwise the inconsistent behaviours among identical semantics will only serve to confuse people. --rt
Nov 04 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 05, 2012 06:05:07 Rob T wrote:
 On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis
 
 wrote:
 And when we argued for altering it so that it operated like
 const ref in C++
 (which allows const ref in D to continue to function like it
 does now), some
 folks complained, because they've found the current semantics
 of auto ref to
 be useful (something to do with propagating the exact, original
 type, I
 think).

I would expect that auto ref for a template and for a non template should work in exactly the same way, so why would there be a difference? If there must be a difference, there should be different semantics for specifying the difference, otherwise the inconsistent behaviours among identical semantics will only serve to confuse people.

auto ref's current semantics can't possibly work with non-templated functions, and it's clear that there are good reasons for keeping auto ref as it is now for templates. So, either we make it work with non-templated functions with different (albeit similar) semantics, or we need a new attribute for doing what we want to do with non-templated functions. - Jonathan M Davis
Nov 05 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Monday, 5 November 2012 at 08:01:57 UTC, Jonathan M Davis 
wrote:
 auto ref's current semantics can't possibly work with 
 non-templated functions,
 and it's clear that there are good reasons for keeping auto ref 
 as it is now
 for templates. So, either we make it work with non-templated 
 functions with
 different (albeit similar) semantics, or we need a new 
 attribute for doing what
 we want to do with non-templated functions.

I agree that the current auto ref semantics are only useful for templates since "auto ref T" is replaced by either a reference (ref T for lvalues) or a value (T for rvalues), two very different things. I don't know how auto ref for templates is currently implemented, i.e., if, given a function void foo(T)(auto ref T x, auto ref T y) {} the compiler would instantiate the template max 2^2 times for each used type T to cover all lvalue/rvalue combinations for x and y. In that case, treating a non-templated function void foo(auto ref MyStruct x, auto ref MyStruct y) {} as implicit template (without template parameters) would be a viable option imho, i.e., void foo()(auto ref MyStruct x, auto ref MyStruct y) {} If that kind of auto-templating is not what we want, I'd definitely opt for allowing rvalues as "in ref" parameters to keep things simple.
Nov 05 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--bcaec5014c7131984504cdd7f04e
Content-Type: text/plain; charset=UTF-8

Yeah I really don't understand how 'auto ref' entered this conversation? It
completely misses the point.

The issue as I see it is this:
  void func(ref in Vector m);
  func(v1*v2 + Vector(10, 20, 30));

And countless situations like it, all leading to innumerable temp locals
with irrelevant names:
  Vector v1v2PlusSomeStuff = v1*v2 + Vector(10, 20, 30);
  func(v1v2PlusSomeStuff);

Or even:
  Vector initialiser1 = Vector(10, 20, 30); // <- words can scarcely
describe how annoying this is
  func(initialiser1);

Allowing rvalues to be passed by ref is exactly as unsafe as those
examples, and that workaround is employed every time anyway, so the
restriction is self-defeating.


On 5 November 2012 10:01, Jonathan M Davis <jmdavisProg gmx.com> wrote:

 On Monday, November 05, 2012 06:05:07 Rob T wrote:
 On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis

 wrote:
 And when we argued for altering it so that it operated like
 const ref in C++
 (which allows const ref in D to continue to function like it
 does now), some
 folks complained, because they've found the current semantics
 of auto ref to
 be useful (something to do with propagating the exact, original
 type, I
 think).

I would expect that auto ref for a template and for a non template should work in exactly the same way, so why would there be a difference? If there must be a difference, there should be different semantics for specifying the difference, otherwise the inconsistent behaviours among identical semantics will only serve to confuse people.

auto ref's current semantics can't possibly work with non-templated functions, and it's clear that there are good reasons for keeping auto ref as it is now for templates. So, either we make it work with non-templated functions with different (albeit similar) semantics, or we need a new attribute for doing what we want to do with non-templated functions. - Jonathan M Davis

--bcaec5014c7131984504cdd7f04e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div>Yeah I really don&#39;t understand how &#39;auto ref&#39; entered this= conversation? It completely misses the point.<br></div><div><br></div><div=
The issue as I see it is this:</div><div>=C2=A0 void func(ref in Vector m)=

</div><div>=C2=A0 func(v1*v2 + Vector(10, 20, 30));<br><div><br></div></div=
<div>And countless situations like it, all leading to innumerable temp loc=

*v2 + Vector(10, 20, 30);<br> </div><div class=3D"gmail_extra">=C2=A0 func(v1v2PlusSomeStuff);</div><div = class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">Or even:</div><d= iv class=3D"gmail_extra"><div>=C2=A0 Vector initialiser1 =3D Vector(10, 20,= 30);=C2=A0// &lt;- words can scarcely describe how annoying this is<br> </div><div>=C2=A0 func(initialiser1);<br></div></div><div class=3D"gmail_ex= tra"><div><br></div><div>Allowing rvalues to be passed by ref is exactly as= unsafe as those examples, and that workaround is employed every time anywa= y, so the restriction is self-defeating.</div> <div><br></div><div><br></div><div class=3D"gmail_quote">On 5 November 2012= 10:01, Jonathan M Davis <span dir=3D"ltr">&lt;<a href=3D"mailto:jmdavisPro= g gmx.com" target=3D"_blank">jmdavisProg gmx.com</a>&gt;</span> wrote:<br><= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;pa= dding-left:1ex"> <div class=3D""><div class=3D"h5">On Monday, November 05, 2012 06:05:07 Rob= T wrote:<br> &gt; On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis<br> &gt;<br> &gt; wrote:<br> &gt; &gt; And when we argued for altering it so that it operated like<br> &gt; &gt; const ref in C++<br> &gt; &gt; (which allows const ref in D to continue to function like it<br> &gt; &gt; does now), some<br> &gt; &gt; folks complained, because they&#39;ve found the current semantics= <br> &gt; &gt; of auto ref to<br> &gt; &gt; be useful (something to do with propagating the exact, original<b= r> &gt; &gt; type, I<br> &gt; &gt; think).<br> &gt;<br> &gt; I would expect that auto ref for a template and for a non<br> &gt; template should work in exactly the same way, so why would there<br> &gt; be a difference? If there must be a difference, there should be<br> &gt; different semantics for specifying the difference, otherwise the<br> &gt; inconsistent behaviours among identical semantics will only serve<br> &gt; to confuse people.<br> <br> </div></div>auto ref&#39;s current semantics can&#39;t possibly work with n= on-templated functions,<br> and it&#39;s clear that there are good reasons for keeping auto ref as it i= s now<br> for templates. So, either we make it work with non-templated functions with= <br> different (albeit similar) semantics, or we need a new attribute for doing = what<br> we want to do with non-templated functions.<br> <span class=3D""><font color=3D"#888888"><br> - Jonathan M Davis<br> </font></span></blockquote></div><br></div> --bcaec5014c7131984504cdd7f04e--
Nov 06 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, November 06, 2012 20:40:38 Manu wrote:
 Yeah I really don't understand how 'auto ref' entered this conversation? It
 completely misses the point.

The _entire_ reason that auto ref was introduced in the first place was to solve the const ref problem. That's what it's for. It wouldn't exist otherwise. So, it's completely relevant to any discussion of const ref. However, Walter misunderstood what Andrei meant by the feature, so it only got implemented for templates, and given how Walter implemented it, it _can't_ be implemented with non-templated functions. But it could be implemented with slightly different semantics such that it solves the const ref problem as orignally intended. It's just that it would then have slightly different semantics between templated and non-templated functions. - Jonathan M Davis
Nov 06 2012
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
--f46d043890b97d1ec004cddb2ed3
Content-Type: text/plain; charset=UTF-8

But it only really makes sense in the context of templates...?
Why should something called 'auto ref' provide fabrication of temporaries
for the purpose of passing rvalues to functions that receive ref args?

How does auto ref (under some different definition/implementation) address
the rvalue problem?


On 6 November 2012 23:08, Jonathan M Davis <jmdavisProg gmx.com> wrote:

 On Tuesday, November 06, 2012 20:40:38 Manu wrote:
 Yeah I really don't understand how 'auto ref' entered this conversation?

 completely misses the point.

The _entire_ reason that auto ref was introduced in the first place was to solve the const ref problem. That's what it's for. It wouldn't exist otherwise. So, it's completely relevant to any discussion of const ref. However, Walter misunderstood what Andrei meant by the feature, so it only got implemented for templates, and given how Walter implemented it, it _can't_ be implemented with non-templated functions. But it could be implemented with slightly different semantics such that it solves the const ref problem as orignally intended. It's just that it would then have slightly different semantics between templated and non-templated functions. - Jonathan M Davis

--f46d043890b97d1ec004cddb2ed3 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable But it only really makes sense in the context of templates...?<div>Why shou= ld something called &#39;auto ref&#39; provide fabrication of temporaries f= or the purpose of passing rvalues to functions that receive ref args?</div> <div><br></div><div>How does auto ref (under some different definition/impl= ementation) address the rvalue problem?</div><div class=3D"gmail_extra"><br=
<br><div class=3D"gmail_quote">On 6 November 2012 23:08, Jonathan M Davis =

nk">jmdavisProg gmx.com</a>&gt;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im">On Tuesday, November 06, 2= 012 20:40:38 Manu wrote:<br> &gt; Yeah I really don&#39;t understand how &#39;auto ref&#39; entered this= conversation? It<br> &gt; completely misses the point.<br> <br> </div>The _entire_ reason that auto ref was introduced in the first place w= as to<br> solve the const ref problem. That&#39;s what it&#39;s for. It wouldn&#39;t = exist otherwise. So,<br> it&#39;s completely relevant to any discussion of const ref.<br> <br> However, Walter misunderstood what Andrei meant by the feature, so it only = got<br> implemented for templates, and given how Walter implemented it, it _can&#39= ;t_ be<br> implemented with non-templated functions. But it could be implemented with = slightly<br> different semantics such that it solves the const ref problem as orignally = intended.<br> It&#39;s just that it would then have slightly different semantics between = templated<br> and non-templated functions.<br> <span class=3D"HOEnZb"><font color=3D"#888888"><br> - Jonathan M Davis<br> </font></span></blockquote></div><br></div> --f46d043890b97d1ec004cddb2ed3--
Nov 06 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/07/2012 04:13 AM, martin wrote:
 On Wednesday, 7 November 2012 at 02:58:57 UTC, Era Scarecrow wrote:
 Maybe a very simple change/addition; Like perhaps  ref? (Attribute
 ref, Alternate ref, auto ref.. All sorta fit for it's meaning).

 So...

 void func1(ref int x);  //D lvalue-only ref
 void func2( ref int x); //works like c++'s ref

 Seems fairly easy to tell apart, and still leaves const-ness as an
 option.

Afaik C++ doesn't allow rvalues to be passed to _mutable_ references (T&), only to const references, making perfect sense imo. I really do not see the point for an additional syntax for C++-like const references (const T&) which would also take rvalues.

You are still missing that const in C++ is different from const in D. Also, if the point is to have higher speed, why shouldn't the function be allowed to use an rvalue as scratch space without a _deep copy_ ? const in C++ does not mean anything. It is just loosely enforced interface documentation. const in D actually restricts what the callee can do with the argument, in a transitive fashion.
 Please give me an example where you want to pass an rvalue to a
 _mutable_ reference parameter.

When my struct does not support any operations that are const, and creating a mutable copy is not possible due to indirections?
 I would simply continue to disallow that,
 since that would mean that changes to the referenced rvalue would not be
 visible for the caller (a temporary/literal is changed - how could
 someone possibly want that?).

The change may well be visible... class C{ int x; } struct W{ C c; } W createW(C c){ return W(c); } void foo(ref W w){ w.c.x = 2; } void main(){ C c = new C; assert(c.x==0); foo(createW(c)); assert(c.x==2); }
Nov 07 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/07/2012 04:01 PM, martin wrote:
 On Wednesday, 7 November 2012 at 10:33:03 UTC, Timon Gehr wrote:
 You are still missing that const in C++ is different from const in D.
 const in C++ does not mean anything. It is just loosely enforced
 interface documentation. const in D actually restricts what the callee
 can do with the argument, in a transitive fashion.

I still don't get the big difference (except for transitiveness for pointers).

That is the main thing, but C++ also has a 'mutable' keyword.
 Given a const reference, I'm only able to invoke methods
 decorated with the const keyword (or inout in D) keyword, both in C++
 and D. And I can only pass it as argument by ref to functions which do
 not alter it (also taking a const reference, that is).

This is not the case in C++ for the D definition of 'not alter'.
 Also, if the point is to have higher speed, why shouldn't the function
 be allowed to use an rvalue as scratch space without a _deep copy_ ?

I'm sorry but I don't get what you mean here. Could you please elaborate on this?

Well, if the changes are not visible, the function can freely change the memory in order to perform its computations.
 When my struct does not support any operations that are const, and
 creating a mutable copy is not possible due to indirections?

If your struct doesn't support any const operations, it most likely has good reasons not to.

Either that, or someone hasn't bothered to annotate.
 The change may well be visible...

True in this case, but only if you know exactly what foo() does when you call it inside the main() function. And that is probably an indicator for bad encapsulation

The method is called 'foo', that means it is a toy example to show that the change may be visible. The change does not necessarily have to be visible to the caller directly.
 - most of the time, you shouldn't have the
 knowledge how foo() is exactly implemented when using it from the outside.

You got it exactly reverse. const harms encapsulation because it exposes some details about implementations. It is often not applicable in sufficiently dynamic code.
 It is clear that there are some examples where you want to pass an
 rvalue argument to a mutable ref parameter if you know exactly what the
 function does. But imo these cases are very rare and I don't really
 regard it as big issue if you need to add a line 'auto tmp = myRvalue;'
 before the function call to transform it to a referenceable lvalue, in
 these few cases.

You do not see issues with changing the interface based on implementation details?
 Much more commonly, you need a parameter just as read-only input.
 Consider a real-word-example of a 4x4 matrix consisting of 16 doubles
 (128 bytes). Most of the time, you'd only need a read-only input
 instance when working with it (combining matrices, transforming vectors
 etc.). Given its size (it's not really huge, I acknowledge that ;)), you
 probably want to avoid copying it around and therefore pass it by ref,
 but want that to also work for rvalues (produced by matrix combinations
 like 'viewMatrix * modelMatrix', for example).

 struct Matrix
 {
      double[16] data;

      // this op= other
      ref Matrix opOpAssign(string op)(in ref Matrix other);

      // Matrix result = this op other
      Matrix opBinary(string op)(in ref Matrix other) const;

      // double4 result = this * vector
      // the vector (32 bytes) may be passed by value for AVX
      double4 opBinary(string op)(in ref double4 vector) const
          if (op == "*");
 };

I know your use case, and note that I do not have any issues with rvalues always being allowed to bind to ref parameters, which would solve your problem in all cases. (others do not like that though, which is why we have the current situation and your C++ - inspired design is discussed every second week or so.)
Nov 07 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/07/2012 06:52 PM, martin wrote:
 On Wednesday, 7 November 2012 at 16:57:47 UTC, Timon Gehr wrote:
 That is the main thing, but C++ also has a 'mutable' keyword.

Right, I forgot this inconspicuous little keyword. It really is a huge hole in C++'s const system.
 If your struct doesn't support any const operations, it most likely
 has good reasons not to.

Either that, or someone hasn't bothered to annotate.

Hehe, yeah, the latter being more likely.
 You got it exactly reverse. const harms encapsulation because it
 exposes some details about implementations. It is often not applicable
 in sufficiently dynamic code.

I don't see it that way.

You do not give a justification.
 I find it very useful to know that an argument
 won't be modified.

That is unrelated and does not necessarily follow, but that it does in some cases is the main reason why const is useful even though it weakens encapsulation.
 Example: suppose we have a large array (static array
 or wrapped in a struct) and need it as input for some independent
 operations. By knowing it won't be touched (passing by const ref) it is
 immediately clear that the operations can be run in parallel. Otherwise,
 one would need to go through the operations' implementation to determine
 if parallelization is possible.

You have to do that anyway if it the operation is not at least 'const pure' and you need to guarantee that no other thread is in fact modifying the data. But even then I do not see how that helps encapsulation. Naive parallelization is just one of the examples that shows that encapsulation can be harmful in some specific cases.
 You do not see issues with changing the interface based on
 implementation details?

I find it okay to require all possible changes to an argument passed by reference to be required to be directly visible by the caller, just to prevent accidentally missing side effects. Suppose a function takes a const reference parameter and is called at a site using an rvalue. After some time, that function is updated so that its parameter is or may be changed, changing the const ref parameter to a ref parameter. That may lead to big issues for the call sites. A compiler error would make sure the affected call sites are inspected and updated instead of potentially introducing regressions.

Often there is nothing to be seen for the caller. (lazy initialization, caching, internal state update part of a larger computation, ...)
Nov 07 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/07/2012 07:48 PM, martin wrote:
 On Wednesday, 7 November 2012 at 18:07:27 UTC, Timon Gehr wrote:
 You do not give a justification.

 I find it very useful to know that an argument
 won't be modified.

That is unrelated and does not necessarily follow, but that it does in some cases is the main reason why const is useful even though it weakens encapsulation.

Let's look at encapsulation this way: suppose my function is a colleague and his job is to summarize a book I lend him (my reference parameter). I don't care how he does it as long as the summary (function result) is what I expect. But I do care about my book if I plan on using it afterwards (so in case the argument is an lvalue used later on) - will I have to check it for defects, coffee spillings etc., or can I be sure it's exactly as it was before (const)?

You can pass him an object that does not support operations you want to preclude. He does not have to _know_, that your book is not changed when he reads it. This is an implementation detail. In fact, you could make the book save away his reading schedule without him noticing.
 This is how I see it.
 The thing about mutable rvalue references would be unknown or
 unthought-of side effects: what if the book is the same, but my
 colleague, a known psychopath, killed the author (a property of the book
 reference, a reference itself)? I don't use the book anymore, okay (it
 was an rvalue after all), but it may still be worth knowing that the
 author has gone. Due to the transitiveness of const, the (const) author
 would still be alive if the book reference had been const.

I'd assume that the book you pass him would not support changing the author.
 You have to do that anyway if it the operation is not at least 'const
 pure' and you need to guarantee that no other thread is in fact
 modifying the data.

I think we can agree that knowing the parameters aren't touched does help a lot in this case.

Maybe, but the fact that we know it harms encapsulation.
Nov 07 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 11/08/2012 02:45 AM, martin wrote:
 On Wednesday, 7 November 2012 at 21:39:52 UTC, Timon Gehr wrote:
 You can pass him an object that does not support operations you want
 to preclude. He does not have to _know_, that your book is not changed
 when he reads it. This is an implementation detail. In fact, you could
 make the book save away his reading schedule without him noticing.

I don't see where you want to go with this. Do you suggest creating tailored objects (book variants) for each function you're gonna pass it to just to satisfy perfect theoretical encapsulation?

No. The point is that the language should _support_ what you call 'perfect theoretical encapsulation'.
 So foo() shouldn't
 be able to change the author => change from inout author reference to
 const reference? bar() should only be allowed to read the book title,
 not the actual book contents => hide that string? ;) For the sake of
 simplicity, by using const we have the ability to at least control if
 the object can be modified or not.

It is not _just_ the object. Anyway, this is what I stated in my last post.
 So although my colleague doesn't have
 to _know_ that he can't modify my book in any way (or know that the book
 is modifiable in the first place), using const is a primitive but
 practical way for me to prevent him from doing so.

It also weakens encapsulation, which was the point.
 In the context of this rvalue => (const) ref discussion, const is useful
 due to a number of reasons.

 1) All possible side effects of the function invokation are required to
 be directly visible by the caller. Some people may find that annoying,
 but I find it useful, and there's a single-line workaround (lvalue
 declaration) for the (in my opinion very rare) cases where a potential
 side-effect is either known not to occur or simply uninteresting
 (requiring exact knowledge about the function implementation, always,
 i.e., during the whole life-time of that code!).

Wrong. Not everything is a perfect value type. (and anyway, the code that actually will observe the change may be a few frames up the call stack.)
 2) Say we pass a literal string (rvalue) to a const ref parameter. The
 location of the string in memory can then be freely chosen by the
 compiler, possibly in a static data segment of the binary (literal
 optimization - only one location for multiple occurrences). If the
 parameter was a mutable ref, the compiler should probably allocate a
 copy on the stack before calling the function, otherwise the literal may
 not be the same when accessed later on, potentially causing funny bugs.

Ambiguous to me and all the interpretations are either wrong or irrelevant.
 3) Implicit type conversion isn't a problem. Say we pass an int rvalue
 to a mutable double ref parameter. The parameter will then be a
 reference to another rvalue (the int cast to a double) and altering it
 (the hidden double rvalue) may not really be what the coder intended.
 Afaik D doesn't support implicit casting for user-defined types, so that
 may not be a problem (for now at least).

Maybe you should stop trying to show that 'const' is sufficient for resolving those issues. The point is that it is not _necessary_. It is too strong.
Nov 08 2012
prev sibling next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/7/2012 3:54 AM, Manu пишет:
 If the compiler started generating 2 copies of all my ref functions, I'd
 be rather unimpressed... bloat is already a problem in D. Perhaps this
 may be a handy feature, but I wouldn't call this a 'solution' to this issue.
 Also, what if the function is external (likely)... auto ref can't work
 if the function is external, an implicit temporary is required in that case.

What's wrong with going this route: void blah(auto ref X stuff){ ...lots of code... } is magically expanded to: void blah(ref X stuff){ ...that code.. } and void blah(X stuff){ .blah(stuff); //now here stuff is L-value so use the ref version } Yeah, it looks _almost_ like a template now. But unlike with a template we can assume it's 2 overloads _always_. External fucntion issue is then solved by treating it as exactly these 2 overloads (one trampoline, one real). Basically it becomes one-line declaration of 2 functions. Given that temporaries are moved anyway the speed should be fine and there is as much bloat as you'd do by hand. Also hopefully inliner can be counted on to do its thing in this simple case. -- Dmitry Olshansky
Nov 08 2012
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/8/2012 11:30 PM, martin пишет:
 On Thursday, 8 November 2012 at 18:28:44 UTC, Dmitry Olshansky wrote:
 What's wrong with going this route:

 void blah(auto ref X stuff){
 ...lots of code...
 }

 is magically expanded to:

 void blah(ref X stuff){
 ...that code..
 }

 and

 void blah(X stuff){
     .blah(stuff); //now here stuff is L-value so use the ref version
 }

 Yeah, it looks _almost_ like a template now. But unlike with a
 template we can assume it's 2 overloads _always_. External fucntion
 issue is then solved by treating it as exactly these 2 overloads (one
 trampoline, one real). Basically it becomes one-line declaration of 2
 functions.

 Given that temporaries are moved anyway the speed should be fine and
 there is as much bloat as you'd do by hand.

 Also hopefully inliner can be counted on to do its thing in this
 simple case.

That second overload for rvalues would be a shortcut to save the lvalue declarations at each call site - and it really doesn't matter if the compiler magically added the lvalue declarations before each call or if it magically added the rvalue overload (assuming all calls are inlined).

The scope. It's all about getting the correct scope, destructor call and you know, the works. Preferably it can inject it inside temporary scope. Anticipating bugs in the implementation of this feature let me warn that re-writing this: ... code here ... auto r = foo(SomeResource(x, y, ..)); //foo is auto ref ... code here ... Should not change semantics e.g. imagine the resource is a lock, we'd better unlock it sooner. That is call destructor right after foo returns. So we need {} around the call. But this doesn't work as it traps 'r': { auto someRef = SomeResource(x, y, ..); auto r = foo(someRef); } So it's rather something like this: typeof(foo(...)) r = void; { someRef = SomeResource(x, y, ..); r = foo(someRef); // should in fact construct in place not assign } I suspect this is hackable to be more clean inside of the compiler but not in terms of a re-write.
 But it would create a problem if there already was an explicit 'void
 blah(X)' overload in addition to 'void blah(auto ref X)' (not making
 much sense obviously, but this would be something the compiler needed to
 handle somehow).

Aye. But even then there is an ambiguity if there is one version of function with ref T / T and one with auto ref T.
 What this 'auto ref' approach (both as currently implemented for
 templates and proposed here for non-templated functions) lacks is the
 vital distinction between const and mutable parameters.

 For the much more common const ref parameters, I repeatedly tried to
 explain why I'm absolutely convinced that we don't need another keyword
 and that 'in/const ref' is sufficient, safe, logical and intuitive
 (coupled with the overload rule that pass-by-value (moving) is preferred
 for rvalues). Please prove me wrong.

I'd rather restrict it to 'auto ref' thingie. Though 'in auto ref' sounds outright silly. Simply put const ref implies that callee can save a pointer to it somewhere (it's l-value). The same risk is with 'auto ref' but at least there an explicitly written 'disclaimer' by the author of accepting temporary stuff. In the ideal world name 'auto ref' would be shorter, logical and more to the point but we have what we have.
 For the less common mutable ref parameters, I also repeatedly tried to
 explain why I find it dangerous/unsafe to allow rvalues to be bound to
 mutable ref parameters. But if there are enough people wanting that, I'd
 have no problem with an 'auto ref' approach for it (only for mutable
 parameters!). That may actually be a good compromise, what do you guys
 think? :)

I think that function plucked with auto ref is a enough indication that author is fine with passing to it mutable r-values and not seeing changes outside and related blah-blah. In most (all?) of cases it means that parameter is too big to be passed by copy so rather it takes it by ref. Also certain stuff can't be properly bitwise const because of C-calls and what not. Logical const is the correct term but in the D world it's simply mutable.
 'auto ref T' for templates expands to 'ref T' (lvalues) and 'T'
 (rvalues), duplicating the whole function and providing best performance
 - no pointer/reference indirection for rvalues in contrast to 'auto ref
 T' (proposed above) for non-templates, otherwise the concept would be
 exactly the same. But it's only for mutable parameters.

I'd say that even for templates the speed argument is mostly defeated by the bloat argument. But that's probably only me.
 Such a templated option may also be worth for const parameters though
 (expanding to 'const ref T' and 'const T'), so maybe something like the
 (ambiguous) 'in/const auto ref T' wouldn't actually be that bad
 (assuming there are only a few use cases, and only for templates! It'd
 still be 'in ref T' for non-templates).

-- Dmitry Olshansky
Nov 08 2012
prev sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/7/2012 9:04 PM, martin пишет:
 On Wednesday, 7 November 2012 at 14:07:31 UTC, martin wrote:
 C++:
 void f(T& a) { // for lvalues
     this->resource = a.resource;
     a.resetResource();
 }
 void f(T&& a) { // for rvalues (moved)
     this->resource = a.resource;
     a.resetResource();
 }

 D:
 void f(ref T a) { // for lvalues
     this.resource = a.resource;
     a.resetResource();
 }
 void f(T a) { // rvalue argument is not copied, but moved
     this.resource = a.resource;
     a.resetResource();
 }

You could probably get away with a single-line overload, both in C++ and D: C++: void f(T& a) { // for lvalues // convert a to mutable rvalue reference and // invoke the main overload f(T&&) f(std::move(a)); } D: void f(T a) { // rvalue argument is not copied, but moved // the original argument is now named a (an lvalue) // invoke the main overload f(ref T) f(a); }

Yup, and I'd like auto ref to actually do this r-value trampoline for me. -- Dmitry Olshansky
Nov 08 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Tuesday, 6 November 2012 at 22:32:57 UTC, Manu wrote:
 But it only really makes sense in the context of templates...?
 Why should something called 'auto ref' provide fabrication of 
 temporaries
 for the purpose of passing rvalues to functions that receive 
 ref args?

 How does auto ref (under some different 
 definition/implementation) address
 the rvalue problem?

The thing is that currently there are 2 workarounds. You described the first one - allocating temporaries manually to obtain referenceable lvalues: void func(ref in Vector m); Vector v1v2PlusSomeStuff = v1*v2 + Vector(10, 20, 30); func(v1v2PlusSomeStuff); The other, also painfully annoying workaround is overloading func: void func(in ref Vector m); void func(in Vector m); func(v1*v2 + Vector(10, 20, 30)); 'auto ref' implements the second workaround (via a template) and therefore requires a single, but templated func() implementation: void func(T)(in auto ref T m); This template, as I understand it, gets expanded to: void func(T)(in ref T m); // for lvalues void func(T)(in T m); // for rvalues So for non-templated functions, I suggest 2 options: 1) The previously described auto-templates (identical 'auto ref' semantics), where a function with 'auto ref' parameters is treated as implicit template. This may lead to code-bloating (for larger functions) and/or higher performance for rvalue arguments (rvalues passed to value arguments are moved, not copied; we therefore gain nothing by passing a reference, but incur a slight performance hit due to pointer indirection instead of accessing directly the rvalue on the stack). OR 2) Simple under-the-hood temporary lvalue declaration by the compiler (for rvalues passed to 'const ref' parameters) - that would be a handy implementation of the first workaround. I hope you get my point. :)
Nov 06 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, November 07, 2012 00:32:48 Manu wrote:
 But it only really makes sense in the context of templates...?
 Why should something called 'auto ref' provide fabrication of temporaries
 for the purpose of passing rvalues to functions that receive ref args?
 
 How does auto ref (under some different definition/implementation) address
 the rvalue problem?

The idea behind auto ref was to create a function which would take its argument in the most efficient way possible, whatever that was (be it ref or by value), and that the compiler would figure that out. That way, the function wouldn't care whether it was given an rvalue or lvalue or a value type or reference type or whatever. What actually got implemented was simply duplicating functions with auto ref so that there's a ref version of lvalues and non-ref versions for rvalues, which really doesn't do the job and only works with templates. What we need is an attribute which indicates that you don't care whether the argument is an lvalue or rvalue. You just don't want it to be copied, and you don't want the function to mutate it - which is what const& in C++ is normally used for. Whether auto ref is the attribute used for that is more or less irrelevant except for the fact that that's the problem that it was trying to solve in the first place. Making auto ref do that for non-templated functions would solve the problem for non-templated functions but would mean that auto ref on parameters in templated functions and non-templated functions would have different (albeit similar) semantics, which would be a bit of problem and may or may not be acceptable. And I'd have to think it through more to say whether or not auto ref as it is on templated functions really solves the problem well enough for templated functions or not. If it does, then it may be acceptable to just have auto ref on non-templated functions with slightly different semantics. If it doesn't, then we either need to change what auto ref does with templated functions to match what we want for non-templated ones (which could be a problem, since some people have found the current semantics of auto ref useful for other stuff like propagating attributes), or we'd need to use a new attribute rather than auto ref. - Jonathan M Davis
Nov 06 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--20cf307f3a9e33c48b04cddc543f
Content-Type: text/plain; charset=UTF-8

If the compiler started generating 2 copies of all my ref functions, I'd be
rather unimpressed... bloat is already a problem in D. Perhaps this may be
a handy feature, but I wouldn't call this a 'solution' to this issue.
Also, what if the function is external (likely)... auto ref can't work if
the function is external, an implicit temporary is required in that case.


On 7 November 2012 01:37, martin <kinke libero.it> wrote:

 On Tuesday, 6 November 2012 at 22:32:57 UTC, Manu wrote:

 But it only really makes sense in the context of templates...?
 Why should something called 'auto ref' provide fabrication of temporaries
 for the purpose of passing rvalues to functions that receive ref args?

 How does auto ref (under some different definition/implementation) address
 the rvalue problem?

The thing is that currently there are 2 workarounds. You described the first one - allocating temporaries manually to obtain referenceable lvalues: void func(ref in Vector m); Vector v1v2PlusSomeStuff = v1*v2 + Vector(10, 20, 30); func(v1v2PlusSomeStuff); The other, also painfully annoying workaround is overloading func: void func(in ref Vector m); void func(in Vector m); func(v1*v2 + Vector(10, 20, 30)); 'auto ref' implements the second workaround (via a template) and therefore requires a single, but templated func() implementation: void func(T)(in auto ref T m); This template, as I understand it, gets expanded to: void func(T)(in ref T m); // for lvalues void func(T)(in T m); // for rvalues So for non-templated functions, I suggest 2 options: 1) The previously described auto-templates (identical 'auto ref' semantics), where a function with 'auto ref' parameters is treated as implicit template. This may lead to code-bloating (for larger functions) and/or higher performance for rvalue arguments (rvalues passed to value arguments are moved, not copied; we therefore gain nothing by passing a reference, but incur a slight performance hit due to pointer indirection instead of accessing directly the rvalue on the stack). OR 2) Simple under-the-hood temporary lvalue declaration by the compiler (for rvalues passed to 'const ref' parameters) - that would be a handy implementation of the first workaround. I hope you get my point. :)

--20cf307f3a9e33c48b04cddc543f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable If the compiler started generating 2 copies of all my ref functions, I&#39;= d be rather unimpressed... bloat is already a problem in D. Perhaps this ma= y be a handy feature, but I wouldn&#39;t call this a &#39;solution&#39; to = this issue.<div> Also, what if the function is external (likely)... auto ref can&#39;t work = if the function is external, an implicit temporary is required in that case= .</div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On 7 N= ovember 2012 01:37, martin <span dir=3D"ltr">&lt;<a href=3D"mailto:kinke li= bero.it" target=3D"_blank">kinke libero.it</a>&gt;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im">On Tuesday, 6 November 201= 2 at 22:32:57 UTC, Manu wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> But it only really makes sense in the context of templates...?<br> Why should something called &#39;auto ref&#39; provide fabrication of tempo= raries<br> for the purpose of passing rvalues to functions that receive ref args?<br> <br> How does auto ref (under some different definition/implementation) address<= br> the rvalue problem?<br> </blockquote> <br></div> The thing is that currently there are 2 workarounds. You described the firs= t one - allocating temporaries manually to obtain referenceable lvalues:<di= v class=3D"im"><br> <br> void func(ref in Vector m);<br></div><div class=3D"im"> Vector v1v2PlusSomeStuff =3D v1*v2 + Vector(10, 20, 30);<br> func(v1v2PlusSomeStuff);<br> <br></div> The other, also painfully annoying workaround is overloading func:<br> <br> void func(in ref Vector m);<br> void func(in Vector m);<div class=3D"im"><br> func(v1*v2 + Vector(10, 20, 30));<br> <br></div> &#39;auto ref&#39; implements the second workaround (via a template) and th= erefore requires a single, but templated func() implementation:<br> <br> void func(T)(in auto ref T m);<br> <br> This template, as I understand it, gets expanded to:<br> <br> void func(T)(in ref T m); // for lvalues<br> void func(T)(in T m); =C2=A0 =C2=A0 // for rvalues<br> <br> So for non-templated functions, I suggest 2 options:<br> <br> 1) The previously described auto-templates (identical &#39;auto ref&#39; se= mantics), where a function with &#39;auto ref&#39; parameters is treated as= implicit template. This may lead to code-bloating (for larger functions) a= nd/or higher performance for rvalue arguments (rvalues passed to value argu= ments are moved, not copied; we therefore gain nothing by passing a referen= ce, but incur a slight performance hit due to pointer indirection instead o= f accessing directly the rvalue on the stack). OR<br> 2) Simple under-the-hood temporary lvalue declaration by the compiler (for = rvalues passed to &#39;const ref&#39; parameters) - that would be a handy i= mplementation of the first workaround.<br> <br> I hope you get my point. :)<br> </blockquote></div><br></div> --20cf307f3a9e33c48b04cddc543f--
Nov 06 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
Thanks Jonathan for the detailed info. So 'auto ref' is 
implemented the way I thought, by simply duplicating functions. 
That is actually the only way (I can think of) to solve your 
problem 'pass this argument in the most efficient way possible 
(do not copy)'.

But that is not the only problem. More commonly, we want to avoid 
copying lvalues for read-only parameters and therefore pass by 
reference. At the same time, we want to use the same function for 
rvalues - we simply can't stand having to declare temporaries 
manually or having to overload functions! Since the parameter is 
read-only, we couldn't care less if the argument is actually an 
lvalue or an rvalue. The only thing is that a dedicated function 
overload for rvalues would be slightly more efficient (for 
rvalues only) since the rvalue would be accessed directly on the 
stack instead of having to access it indirectly via its reference.
This problem here would be 'pass this argument as read-only in 
the most efficient way possible for lvalues (do not copy) but 
also allow rvalues at a slight cost (compared to a dedicated 
function overload)'.

So we do not really need 'auto ref' for non-templated functions 
or a new, even more confusing keyword which you, Jonathan, seem 
to insist on - 'const ref' (or, more elegantly 'in ref') is all 
we need. We simply want the compiler to automatically declare 
rvalues passed to 'const ref' parameters as local lvalues - 
problem solved for the majority of cases. For the remaining few 
which care about the involved cost for rvalue arguments, allow 
them to provide a function overload for rvalues (either manually 
or via 'auto ref' templates) - problem solved completely in my 
point of view. And that is actually exactly what the thread 
starter Malte proposes (and I already pointed that out in an 
earlier post).

Again hoping to be clear enough. :)
Nov 06 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, November 07, 2012 01:32:03 martin wrote:
 So we do not really need 'auto ref' for non-templated functions
 or a new, even more confusing keyword which you, Jonathan, seem
 to insist on - 'const ref' (or, more elegantly 'in ref') is all
 we need.

No, because that would be doing the same thing as C++, which Walter and Andrei have already rejected. They specifically do _not_ want there to be any ambiguity between whether a const ref variable is an lvalue or rvalue. If they were willing to make const ref work the same as C++'s const&, then we would never have had this problem in the first place. We specifically need something other than const ref. The const ref can continue to work as it does now, but we'll have a way to get semantics similar to C++'s const& when we want them. - Jonathan M Davis
Nov 06 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 00:47:04 UTC, Jonathan M Davis 
wrote:
 No, because that would be doing the same thing as C++, which 
 Walter and Andrei
 have already rejected. They specifically do _not_ want there to 
 be any
 ambiguity between whether a const ref variable is an lvalue or 
 rvalue. If they
 were willing to make const ref work the same as C++'s const&, 
 then we would
 never have had this problem in the first place. We specifically 
 need something
 other than const ref. The const ref can continue to work as it 
 does now, but
 we'll have a way to get semantics similar to C++'s const& when 
 we want them.

Wow, now this finally sheds some light on this topic. Do you have any links regarding their reasoning? I simply and utterly fail to comprehend the need for a compiler to distinguish between a const reference to a named variable and a const reference to an unnamed temporary/literal (which the compiler is free to allocate wherever it makes most sense). Especially since the compiler could simply allocate and name the original rvalue as local variable on the caller's stack before the function call and hence transform it automatically to an lvalue, i.e., do that, what currently most of us have to painfully do by hand.
Nov 06 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, November 07, 2012 02:15:26 martin wrote:
 Wow, now this finally sheds some light on this topic. Do you have
 any links regarding their reasoning?

The most recent discussion where Walter and Andrei were part of the discussion was here: http://forum.dlang.org/post/4F84D6DD.5090405 digitalmars.com A key component of the problem is that conversions cause problems. Also, IIRC allowing const ref to take rvalues causes some issues with overloading. Making some changes to ref with regards to escaping apparently could change the situation a bit, and there's even some discussion in there of allowing ref to take rvalues if certain other things in the language are changed, but it's all up in the air at this point. Certainly, it's not a simple matter of just making const ref work with rvalues like most of the people coming from C++ want and expect. - Jonathan M Davis
Nov 06 2012
prev sibling next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Wednesday, 7 November 2012 at 00:32:04 UTC, martin wrote:
 So we do not really need 'auto ref' for non-templated functions 
 or a new, even more confusing keyword which you, Jonathan, seem 
 to insist on - 'const ref' (or, more elegantly 'in ref') is all 
 we need. We simply want the compiler to automatically declare 
 rvalues passed to 'const ref' parameters as local lvalues - 
 problem solved for the majority of cases. For the remaining few 
 which care about the involved cost for rvalue arguments, allow 
 them to provide a function overload for rvalues (either 
 manually or via 'auto ref' templates) - problem solved 
 completely in my point of view. And that is actually exactly 
 what the thread starter Malte proposes (and I already pointed 
 that out in an earlier post).

'in ref' may work, but what it implies doesn't quite make sense. 'auto ref' already being used for templates should be able to carry over to functions, however auto in the same section along with type declaration may seem a little confusing. Let's glance at a few of how they would be called. //either void func(int x); void func(const int x); //lvalue void func(ref int x); void func(const ref int x); //either? auto ref/in ref void func(auto ref int x); void func(const auto ref int x); void func(in ref int x); Assuming auto can't be used to do type referencing the 'const auto ref int' seems rather long. alternatively perhaps inout? (accepts any type, partially skewed?) void func(inout ref int x); void func(const inout ref int x); At this point for const it's just as large as the const auto ref.. So.. void func(inout ref int x); //mutable void func(in ref int x); //const version void func(out ref int x); //optional write-only variable 'out ref int x' could then be legal.. Somehow reminds me of zlib's compress utility function that returns how many bytes the output buffer used but sometimes you didn't care. So.. if we defined and used... void func(out ref int x) { x = 100; } int x; func(x); assert(x == 100); func(0); //0 and null (and legal) basically ignored. func(null);
Nov 06 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 01:15:27 UTC, martin wrote:
 I simply and utterly fail to comprehend the need for a compiler 
 to distinguish between a const reference to a named variable 
 and a const reference to an unnamed temporary/literal (which 
 the compiler is free to allocate wherever it makes most sense).

Let me be more precise: I don't see why a called function would need to know whether a passed const reference is named or unnamed at the caller's site. The developer's reasoning for a const reference is simply to avoid costly copying of the argument to be passed (only applicable for lvalues) which is guaranteed not to be modified by the callee. Decorating such a parameter with 'const ref' seems as logical as it gets (don't pass me a copy of the real thing, pass me a reference instead - I'm not going to modify it). The only reason I can think of to distinguish between lvalues and rvalues at the callee is the slight performance hit due to pointer indirection for rvalues. And that would be addressable by preferring by-value-passing of rvalues when selecting between function overloads.
Nov 06 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Tuesday, 6 November 2012 at 23:37:25 UTC, martin wrote:
 void func(T)(in auto ref T m);

 This template, as I understand it, gets expanded to:

 void func(T)(in ref T m); // for lvalues
 void func(T)(in T m);     // for rvalues

 So for non-templated functions, I suggest 2 options:

 1) The previously described auto-templates (identical 'auto 
 ref' semantics), where a function with 'auto ref' parameters is 
 treated as implicit template. This may lead to code-bloating 
 (for larger functions) and/or higher performance for rvalue 
 arguments (rvalues passed to value arguments are moved, not 
 copied; we therefore gain nothing by passing a reference, but 
 incur a slight performance hit due to pointer indirection 
 instead of accessing directly the rvalue on the stack). OR
 2) Simple under-the-hood temporary lvalue declaration by the 
 compiler (for rvalues passed to 'const ref' parameters) - that 
 would be a handy implementation of the first workaround.

 I hope you get my point. :)

What about the case where we want to pass a source argument either by reference or as a copy depending on the l/r value situation? eg void f( ref a ); void f( a ); --rt
Nov 06 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 01:33:49 UTC, Jonathan M Davis 
wrote:
 The most recent discussion where Walter and Andrei were part of 
 the discussion
 was here:

 http://forum.dlang.org/post/4F84D6DD.5090405 digitalmars.com

That thread is quite misleading and, I'm sad to say, not very useful (rather damaging to this discussion) in my opinion - especially because the distinction between rvalue => 'const ref' and rvalue => ref is largely neglected, and that distinction is of extremely high importance, I can't stress that enough. Walter's 3 C++ examples (2 of them invalid anyway afaik) don't relate to _const_ references. The implicit type conversion problem in that thread isn't a problem for _const_ references, just to point out one tiny aspect. rvalue => ref/out propagation makes no sense imho, as does treating literals as lvalues (proposed by Walter iirc). The current 'auto ref' semantics also fail to cover the special role of _const_ references for rvalues (also illustrated by Scarecrow's post).
 Certainly, it's not a simple matter of just making const
 ref work with rvalues like most of the people coming from
 C++ want and expect.

Well I absolutely do _not_ share this point of view. It just seems so logical to me. I'm still waiting for a plausible argument to prove me wrong. All the required info is in this thread, e.g., we covered the escaping issue you mentioned.
Nov 06 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 02:06:09 UTC, Rob T wrote:
 What about the case where we want to pass a source argument 
 either by reference or as a copy depending on the l/r value 
 situation?

 eg
 void f( ref a );
 void f( a );

 --rt

I don't get what you mean - that's why the 2 overloads are for (you forgot the const/in keyword - beware! :)) void f( in ref T a ); void f( in T a ); rvalue binds to the latter overload (argument not copied, but moved directly). lvalue binds to the first overload (reference). Works with v2.060.
Nov 06 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Tuesday, 6 November 2012 at 23:37:25 UTC, martin wrote:
 1) The previously described auto-templates (identical 'auto 
 ref' semantics), where a function with 'auto ref' parameters is 
 treated as implicit template. This may lead to code-bloating 
 (for larger functions) and/or higher performance for rvalue 
 arguments (rvalues passed to value arguments are moved, not 
 copied; we therefore gain nothing by passing a reference, but 
 incur a slight performance hit due to pointer indirection 
 instead of accessing directly the rvalue on the stack). OR
 2) Simple under-the-hood temporary lvalue declaration by the 
 compiler (for rvalues passed to 'const ref' parameters) - that 
 would be a handy implementation of the first workaround.

Please discard option 1, I'm afraid it creates too much confusion and was not well thought through. The objective was to expand 'auto ref T' to either 'in ref T' for lvalues or 'in T' for rvalues (i.e., only for const parameters!), but then its caption would be horribly misleading (and something like 'in auto ref' would be way too ambiguous).
Nov 06 2012
prev sibling next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Wednesday, 7 November 2012 at 02:45:15 UTC, martin wrote:
 Please discard option 1, I'm afraid it creates too much 
 confusion and was not well thought through. The objective was 
 to expand 'auto ref T' to either 'in ref T' for lvalues or 'in 
 T' for rvalues (I.e., only for const parameters!), but then its 
 caption would be horribly misleading (and something like 'in 
 auto ref' would be way too ambiguous).

Maybe a very simple change/addition; Like perhaps ref? (Attribute ref, Alternate ref, auto ref.. All sorta fit for it's meaning). So... void func1(ref int x); //D lvalue-only ref void func2( ref int x); //works like c++'s ref Seems fairly easy to tell apart, and still leaves const-ness as an option.
Nov 06 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 02:58:57 UTC, Era Scarecrow 
wrote:
 Maybe a very simple change/addition; Like perhaps  ref? 
 (Attribute ref, Alternate ref, auto ref.. All sorta fit for 
 it's meaning).

 So...

 void func1(ref int x);  //D lvalue-only ref
 void func2( ref int x); //works like c++'s ref

 Seems fairly easy to tell apart, and still leaves const-ness as 
 an option.

Afaik C++ doesn't allow rvalues to be passed to _mutable_ references (T&), only to const references, making perfect sense imo. I really do not see the point for an additional syntax for C++-like const references (const T&) which would also take rvalues. Please give me an example where you want to pass an rvalue to a _mutable_ reference parameter. I would simply continue to disallow that, since that would mean that changes to the referenced rvalue would not be visible for the caller (a temporary/literal is changed - how could someone possibly want that?).
Nov 06 2012
prev sibling next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Wednesday, 7 November 2012 at 03:13:22 UTC, martin wrote:
 void func1(ref int x);  //D lvalue-only ref
 void func2( ref int x); //works like c++'s ref

 Seems fairly easy to tell apart, and still leaves const-ness 
 as an option.

Afaik C++ doesn't allow rvalues to be passed to _mutable_ references (T&), only to const references, making perfect sense imo. I really do not see the point for an additional syntax for C++-like const references (const T&) which would also take rvalues.

 Please give me an example where you want to pass an rvalue to a 
 _mutable_ reference parameter. I would simply continue to 
 disallow that, since that would mean that changes to the 
 referenced rvalue would not be visible for the caller (a 
 temporary/literal is changed - how could someone possibly want 
 that?).

Still that zlib entry is coming to mind. But more importantly is that you still need code duplication to get both accessible. //many possible combinations thereof int func(const ref x); int func(int x) { return func(cast(const int) x); } But regarding zlib.. It's function is something like: int compress(char *output, int *size, char *input, int inputSize); So... if we convert that to something similar we get.. enum ZlibEnum {} ZlibEnum compress(void[] output, void[] input, &ref Zlib state) nothrow pure; The idea in this case is 'state' would continue to hold the input/output pointers and all information needed, so you could continue to use it and if it has any 'unflushed' data (output size too small?) then it could retain that. however if you knew you didn't need it you could ignore it. ZlibEnum could be for output C calling code, so success/failure needs to be known right away. It makes sense for it to return a Zlib type as well, but depends on what has higher priority and why. string input = "something long"; ubyte[1000] output; ubyte[5] output2; Zlib state; compress(output, input, null); //we know the buffer is large enough compress(output, input, state);
Nov 06 2012
prev sibling next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Wednesday, 7 November 2012 at 03:13:22 UTC, martin wrote:
 void func1(ref int x);  //D lvalue-only ref
 void func2( ref int x); //works like c++'s ref

 Seems fairly easy to tell apart, and still leaves const-ness 
 as an option.

Afaik C++ doesn't allow rvalues to be passed to _mutable_ references (T&), only to const references, making perfect sense imo. I really do not see the point for an additional syntax for C++-like const references (const T&) which would also take rvalues.

 Please give me an example where you want to pass an rvalue to a 
 _mutable_ reference parameter. I would simply continue to 
 disallow that, since that would mean that changes to the 
 referenced rvalue would not be visible for the caller (a 
 temporary/literal is changed - how could someone possibly want 
 that?).

Still that zlib entry is coming to mind. But more importantly is that you still need code duplication to get both accessible. //many possible combinations thereof int func(const ref x); int func(int x) { return func(cast(const int) x); } But regarding zlib.. It's function is something like: int compress(char *output, int *size, char *input, int inputSize); So... if we convert that to something similar we get.. enum ZlibEnum {} ZlibEnum compress(void[] output, void[] input, &ref Zlib state) nothrow pure; The idea in this case is 'state' would continue to hold the input/output pointers and all information needed, so you could continue to use it and if it has any 'unflushed' data (output size too small?) then it could retain that. however if you knew you didn't need it you could ignore it. ZlibEnum could be for output C calling code, so success/failure needs to be known right away. It makes sense for it to return a Zlib type as well, but depends on what has higher priority and why. string input = "something long"; ubyte[1000] output; ubyte[5] output2; Zlib state; compress(output, input, null); //we know the buffer is large enough compress(output2, input, state);
Nov 06 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 03:35:19 UTC, Era Scarecrow 
wrote:
 Still that zlib entry is coming to mind. But more importantly
 is that you still need code duplication to get both accessible.

 //many possible combinations thereof
 int func(const ref x);

 int func(int x) {
   return func(cast(const int) x);
 }

int func(in ref int x); int func(int x) { return func(x); } The latter overload is for rvalues (no need to cast to const) and wouldn't be required if the first one took rvalues directly (having a _const_ ref parameter). That's not an example for mutable references though, it's basically a shortcut so as to not having to declare all rvalues for x manually.
 enum ZlibEnum {}
 ZlibEnum compress(void[] output, void[] input,
   &ref Zlib state) nothrow pure;

 string input = "something long";
 ubyte[1000] output;
 ubyte[5] output2;
 Zlib state;

 compress(output, input, null);
 compress(output2, input, state);

Where's the rvalue? You're talking about an optional mutable reference here, which is only doable via a pointer (references cannot be null, not in C++ and not in D): ZlibEnum compress(void[] output, void[] input, Zlib* state = null); ... compress(output, input); compress(output2, input, &state); That would be correct. But you're completely missing the point here. Let's assume you wanted to require a state to be passed: ZlibEnum compress(void[] output, void[] input, ref Zlib state); You could now use: Zlib state; // lvalue compress(output, input, state); but not: compress(output, input, Zlib()); // rvalue And that is most likely a good thing, since your state would be lost after the compress() call. If you don't want it, you don't pass it, and as I said, the only way to do that is to pass a nullable pointer. So this is off-topic I'm afraid.
Nov 06 2012
prev sibling next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Wednesday, 7 November 2012 at 04:05:32 UTC, martin wrote:
 int func(in ref int x);
 int func(int x) { return func(x); }

 The latter overload is for rvalues (no need to cast to const) 
 and wouldn't be required if the first one took rvalues directly 
 (having a _const_ ref parameter). That's not an example for 
 mutable references though, it's basically a shortcut so as to 
 not having to declare all rvalues for x manually.

Maybe... But when working with non built-in types can you guarantee that behavior? //same as 'const ref' basically, unless 'in ref' is accepted struct S //int func(in ref S x); //? int func(const ref S x); int func(S x) { return func(x); } //potentially infinitely self calling
 compress(output, input, null);
 compress(output2, input, state);

Where's the rvalue? You're talking about an optional mutable reference here, which is only doable via a pointer (references cannot be null, not in C++ and not in D):

Perhaps null is wrong in this case, but had it been Zlib.init or Zlib(), then it would still be applicable. Since it's a reference I would think it could accept a null pointer and realize to create a temporary which has it's default values.
 That would be correct. But you're completely missing the point 
 here. Let's assume you wanted to require a state to be passed:

 ZlibEnum compress(void[] output, void[] input, ref Zlib state);

 You could now use:

 Zlib state; // lvalue
 compress(output, input, state);

 but not:

 compress(output, input, Zlib()); // rvalue

 And that is most likely a good thing, since your state would be 
 lost after the compress() call. If you don't want it, you don't 
 pass it, and as I said, the only way to do that is to pass a 
 nullable pointer. So this is off-topic I'm afraid.

True, but rather than having to create a temporary you don't plan on using just to satisfy the signature, and you know you don't NEED it afterwards, why do you have to go through the extra steps? How many wrappers are made in general just to work around minor issues like this? Quite often we ignore return types if they aren't interesting, but we can't ignore a rvalue we only need for one call (or just to satisfy the signature)? Say we have a divide function (who knows what for), and this is more efficient than normal. So... //perhaps ' ref int remainder = 0'? int divide(int number, int divisor, ref int remainder); Now in this case what if we don't care about the remainder? Create a temporary (or a wrapper function)? //satisfy remainder int divide(int number, int divisor) { int tmp; return divide(number, divisor, tmp); } int modulus(int number, int divisor) { int tmp; divide(number, divisor, tmp); return tmp; } But if we want the result of the division AND the remainder should we have to make a separate modulus function when the original divide can clearly give you the answer? This cuts closer to the instruction set as an example.
Nov 06 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 04:36:59 UTC, Era Scarecrow 
wrote:
 Maybe... But when working with non built-in types can you 
 guarantee that behavior?

Sure, try it out. What you are referring to here has nothing to do with rvalues. It is about side results which are only sometimes needed. It is clear that you have to use lvalue arguments for those in order to access them after the function call. Using bogus rvalue arguments in case you're not interested in them wouldn't really help - you'd save the variable declaration, but the function signature would still be ugly, and you can't always provide hypothetical default values to hide them (depending on parameter order). In my experience, these cases are very rare anyway (unless dealing with porting old C-style code which you seem to be doing) and function overloading is the most elegant way to handle it - take a look at the large number of overloads in modern language libraries such as .NET. This is how I would implement it: // magical, super-fast function ;) int divide(in int number, in int divisor, int* remainder = null); auto bla = divide(13, 4); // => 3 // remainder needed as well, hiding the pointer as reference: int divide(in int number, in int divisor, out int remainder) { return divide(number, divisor, &remainder); } int remainder; auto bla = divide(13, 4, remainder); // => 3, remainder = 1 // only remainder needed: int modulus(in int number, in int divisor) { int tmp; divide(number, divisor, tmp); return tmp; } auto bla = modulus(13, 4); // => 1 But this is all really off-topic here, let's stop spamming this thread.
Nov 06 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Wednesday, 7 November 2012 at 02:34:25 UTC, martin wrote:
 On Wednesday, 7 November 2012 at 02:06:09 UTC, Rob T wrote:
 What about the case where we want to pass a source argument 
 either by reference or as a copy depending on the l/r value 
 situation?

 eg
 void f( ref a );
 void f( a );

 --rt

I don't get what you mean - that's why the 2 overloads are for (you forgot the const/in keyword - beware! :)) void f( in ref T a ); void f( in T a ); rvalue binds to the latter overload (argument not copied, but moved directly). lvalue binds to the first overload (reference). Works with v2.060.

Sorry, my explanation was very poor. I'll try again. I'm trying to describe the case where you want to modify the ref argument so that the source is also changed. For example, let's say you want to move a resource instead of coping it. Moving means the resource is transfered from one object to another, and the original is reset without destoying the resource. That way only one object can have the same resource. So for moving a resource, you cannot use "in ref" or "const ref", you need to just use "ref". void f( ref T a ){ // take resource away from a this.resource = a.resource; a.resetResource(); // this now owns the resource } ref T works fine, but if you wish to use f( ref T a ) on a temp value returned from another function call, you'll need to overload f() to pass by value, which means creating a duplicate. void f( ref T a ){ // take resource away from a this.resource = a.resource; a.resetResource(); // this now owns the resource } Example: T g(){ T temp; // create resource which is stored inside T temp.createResource(); // move temp contents to return value return move(temp); } f( g() ); // won't compile with ref It's annoying to have to create overloaded duplicates for a situation like moving, esp if the duplicate is a lot of code. In C++ the move problem was solved using && move semantics, and this is what I'm trying to emulate in D but I feel like I'm fighting with the language. It is possible that D has a specific way of solving this problem that I'm not aware of yet, so correct me if I'm asking for something that is simply not needed. I tried f( move(g()) ) but that fails to work. My best guess is that D does a hidden move of the temp instead of a copy to value. I can't say for sure because the documentation is not clear and is missing important details like this. I also cannot rely on clever compiler optimizations that may or may not be implemented as a guarantee. --rt
Nov 06 2012
prev sibling next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Wednesday, 7 November 2012 at 10:33:03 UTC, Timon Gehr wrote:
 You are still missing that const in C++ is different from const 
 in D. Also, if the point is to have higher speed, why shouldn't 
 the function be allowed to use an rvalue as scratch space 
 without a _deep copy_ ?

 const in C++ does not mean anything. It is just loosely 
 enforced interface documentation. const in D actually restricts 
 what the callee can do with the argument, in a transitive 
 fashion.

Also depending on how you think of it, the const ref in C++ logically it didn't make sense to pass a mutable rvalue; Say you pass the number 3, you can't modify it logically (ie you can't redefine PI afterall).
 When my struct does not support any operations that are const, 
 and creating a mutable copy is not possible due to indirections?

 I would simply continue to disallow that, since that would 
 mean that changes to the referenced rvalue would not be 
 visible for the caller (a temporary/literal is changed - how 
 could someone possibly want that?).

The change may well be visible... class C{ int x; } struct W{ C c; } W createW(C c){ return W(c); } void foo(ref W w){ w.c.x = 2; } void main(){ C c = new C; assert(c.x==0); foo(createW(c)); assert(c.x==2); }

So basically being named (or not) doesn't change the data, what it is or how it's used, only how accessible it is. There's some forms of techniques that don't seem to have any value until you have a chance to think about it and use them; Like scopes, overloading, referencing, Exceptions you can easily do without them all (We have C afterall), but it isn't as much fun or clean. Some features and uses thereof make more sense for the back/private implementation rather than the public interface. The reason I gave my 'divide' example is because a while back I wrote a BitInt (in C like 10 years ago, and a bit buggy); The divide function actually calculated the remainder as a side effect, and using wrappers or writing them as separate portions would actually be harder and slower than joined as it was.
Nov 07 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 06:13:25 UTC, Rob T wrote:
 ref T works fine, but if you wish to use f( ref T a ) on a temp 
 value returned from another function call, you'll need to 
 overload f() to pass by value, which means creating a duplicate.

Duplicating the function, yes, but not duplicating the rvalue since it is moved in D. Isn't this analog to the C++ solution? C++: void f(T& a) { // for lvalues this->resource = a.resource; a.resetResource(); } void f(T&& a) { // for rvalues (moved) this->resource = a.resource; a.resetResource(); } D: void f(ref T a) { // for lvalues this.resource = a.resource; a.resetResource(); } void f(T a) { // rvalue argument is not copied, but moved this.resource = a.resource; a.resetResource(); } T g() { T temp; temp.createResource(); return temp; // D already moves temp! // 'Named Return Value Optimization' }
 I tried f( move(g()) ) but that fails to work. My best guess is 
 that D does a hidden move of the temp instead of a copy to 
 value. I can't say for sure because the documentation is not 
 clear and is missing important details like this. I also cannot 
 rely on clever compiler optimizations that may or may not be 
 implemented as a guarantee.

You could implement a copy constructor 'this(this)' in your struct T and see when it is invoked to check when an instance is actually copied. I'd expect that invoking 'f(g());' with above implementation doesn't copy anything.
Nov 07 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 10:33:03 UTC, Timon Gehr wrote:
 You are still missing that const in C++ is different from const 
 in D.
 const in C++ does not mean anything. It is just loosely 
 enforced interface documentation. const in D actually restricts 
 what the callee can do with the argument, in a transitive 
 fashion.

I still don't get the big difference (except for transitiveness for pointers). Given a const reference, I'm only able to invoke methods decorated with the const keyword (or inout in D) keyword, both in C++ and D. And I can only pass it as argument by ref to functions which do not alter it (also taking a const reference, that is).
 Also, if the point is to have higher speed, why shouldn't the 
 function be allowed to use an rvalue as scratch space without a 
 _deep copy_ ?

I'm sorry but I don't get what you mean here. Could you please elaborate on this?
 When my struct does not support any operations that are const, 
 and creating a mutable copy is not possible due to indirections?

If your struct doesn't support any const operations, it most likely has good reasons not to.
 The change may well be visible...

True in this case, but only if you know exactly what foo() does when you call it inside the main() function. And that is probably an indicator for bad encapsulation - most of the time, you shouldn't have the knowledge how foo() is exactly implemented when using it from the outside. It is clear that there are some examples where you want to pass an rvalue argument to a mutable ref parameter if you know exactly what the function does. But imo these cases are very rare and I don't really regard it as big issue if you need to add a line 'auto tmp = myRvalue;' before the function call to transform it to a referenceable lvalue, in these few cases. Much more commonly, you need a parameter just as read-only input. Consider a real-word-example of a 4x4 matrix consisting of 16 doubles (128 bytes). Most of the time, you'd only need a read-only input instance when working with it (combining matrices, transforming vectors etc.). Given its size (it's not really huge, I acknowledge that ;)), you probably want to avoid copying it around and therefore pass it by ref, but want that to also work for rvalues (produced by matrix combinations like 'viewMatrix * modelMatrix', for example). struct Matrix { double[16] data; // this op= other ref Matrix opOpAssign(string op)(in ref Matrix other); // Matrix result = this op other Matrix opBinary(string op)(in ref Matrix other) const; // double4 result = this * vector // the vector (32 bytes) may be passed by value for AVX double4 opBinary(string op)(in ref double4 vector) const if (op == "*"); };
Nov 07 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 14:07:31 UTC, martin wrote:
 C++:
 void f(T& a) { // for lvalues
     this->resource = a.resource;
     a.resetResource();
 }
 void f(T&& a) { // for rvalues (moved)
     this->resource = a.resource;
     a.resetResource();
 }

 D:
 void f(ref T a) { // for lvalues
     this.resource = a.resource;
     a.resetResource();
 }
 void f(T a) { // rvalue argument is not copied, but moved
     this.resource = a.resource;
     a.resetResource();
 }

You could probably get away with a single-line overload, both in C++ and D: C++: void f(T& a) { // for lvalues // convert a to mutable rvalue reference and // invoke the main overload f(T&&) f(std::move(a)); } D: void f(T a) { // rvalue argument is not copied, but moved // the original argument is now named a (an lvalue) // invoke the main overload f(ref T) f(a); }
Nov 07 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 16:57:47 UTC, Timon Gehr wrote:
 That is the main thing, but C++ also has a 'mutable' keyword.

Right, I forgot this inconspicuous little keyword. It really is a huge hole in C++'s const system.
 If your struct doesn't support any const operations, it most 
 likely has good reasons not to.

Either that, or someone hasn't bothered to annotate.

Hehe, yeah, the latter being more likely.
 You got it exactly reverse. const harms encapsulation because 
 it exposes some details about implementations. It is often not 
 applicable in sufficiently dynamic code.

I don't see it that way. I find it very useful to know that an argument won't be modified. Example: suppose we have a large array (static array or wrapped in a struct) and need it as input for some independent operations. By knowing it won't be touched (passing by const ref) it is immediately clear that the operations can be run in parallel. Otherwise, one would need to go through the operations' implementation to determine if parallelization is possible.
 You do not see issues with changing the interface based on 
 implementation details?

I find it okay to require all possible changes to an argument passed by reference to be required to be directly visible by the caller, just to prevent accidentally missing side effects. Suppose a function takes a const reference parameter and is called at a site using an rvalue. After some time, that function is updated so that its parameter is or may be changed, changing the const ref parameter to a ref parameter. That may lead to big issues for the call sites. A compiler error would make sure the affected call sites are inspected and updated instead of potentially introducing regressions.
Nov 07 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 18:07:27 UTC, Timon Gehr wrote:
 You do not give a justification.

 I find it very useful to know that an argument
 won't be modified.

That is unrelated and does not necessarily follow, but that it does in some cases is the main reason why const is useful even though it weakens encapsulation.

Let's look at encapsulation this way: suppose my function is a colleague and his job is to summarize a book I lend him (my reference parameter). I don't care how he does it as long as the summary (function result) is what I expect. But I do care about my book if I plan on using it afterwards (so in case the argument is an lvalue used later on) - will I have to check it for defects, coffee spillings etc., or can I be sure it's exactly as it was before (const)? This is how I see it. The thing about mutable rvalue references would be unknown or unthought-of side effects: what if the book is the same, but my colleague, a known psychopath, killed the author (a property of the book reference, a reference itself)? I don't use the book anymore, okay (it was an rvalue after all), but it may still be worth knowing that the author has gone. Due to the transitiveness of const, the (const) author would still be alive if the book reference had been const.
 You have to do that anyway if it the operation is not at least 
 'const pure' and you need to guarantee that no other thread is 
 in fact modifying the data.

I think we can agree that knowing the parameters aren't touched does help a lot in this case.
Nov 07 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Wednesday, 7 November 2012 at 14:07:31 UTC, martin wrote:

 T g() {
     T temp;
     temp.createResource();
     return temp; // D already moves temp!
                  // 'Named Return Value Optimization'
 }

OK, it seems to be working as you describe. The main question I have is if I can rely on this behaviour as a feature provided by the languge specification vs an opportunistic compiler optimization that may or may not happen depending on the implementation? The general problem I'm having with with D, is I need a gaurantee that certain behaviors will always be performed when moves/copies are done, but it's just not clear if this is the case or not. There seems to be a lack of a concise language specification, or is there one defined somewhere?
 You could implement a copy constructor 'this(this)' in your 
 struct T and see when it is invoked to check when an instance 
 is actually copied. I'd expect that invoking 'f(g());' with 
 above implementation doesn't copy anything.

I am doing that, but I cannpt seem to hook into the part where a move does an "init" on the struct or class. I sort-of can see it though, if I set a member value to a known state before the move, then display it to console or log after the move during destruction. If it was init'ed then I'll see the default value, and this seems to be happening as described. Anyway, what I was hoping for with "auto ref" was for the compiler to selectively decide if the ref part should be used or not depending on the situation, rather than me manually writing two functions to do the exact same thing. What's discussed in here about auto ref is something else, although I agree it is definitely needed. --rt
Nov 07 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--047d7b6773a6d1127904cdf07ece
Content-Type: text/plain; charset=UTF-8

On 7 November 2012 04:22, martin <kinke libero.it> wrote:

 On Wednesday, 7 November 2012 at 01:33:49 UTC, Jonathan M Davis wrote:

 The most recent discussion where Walter and Andrei were part of the
 discussion
 was here:

 http://forum.dlang.org/post/**4F84D6DD.5090405 digitalmars.**com<http://forum.dlang.org/post/4F84D6DD.5090405 digitalmars.com>

That thread is quite misleading and, I'm sad to say, not very useful (rather damaging to this discussion) in my opinion - especially because the distinction between rvalue => 'const ref' and rvalue => ref is largely neglected, and that distinction is of extremely high importance, I can't stress that enough. Walter's 3 C++ examples (2 of them invalid anyway afaik) don't relate to _const_ references. The implicit type conversion problem in that thread isn't a problem for _const_ references, just to point out one tiny aspect. rvalue => ref/out propagation makes no sense imho, as does treating literals as lvalues (proposed by Walter iirc). The current 'auto ref' semantics also fail to cover the special role of _const_ references for rvalues (also illustrated by Scarecrow's post). Certainly, it's not a simple matter of just making const
 ref work with rvalues like most of the people coming from
 C++ want and expect.

Well I absolutely do _not_ share this point of view. It just seems so logical to me. I'm still waiting for a plausible argument to prove me wrong. All the required info is in this thread, e.g., we covered the escaping issue you mentioned.

+1 I couldn't possibly agree more with your entire post. --047d7b6773a6d1127904cdf07ece Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 7 November 2012 04:22, martin <span dir=3D"ltr">&lt;<a href=3D"mailto:ki= nke libero.it" target=3D"_blank">kinke libero.it</a>&gt;</span> wrote:<br><= div class=3D"gmail_extra"><div class=3D"gmail_quote"><blockquote class=3D"g= mail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-l= eft:1ex"> <div class=3D"im">On Wednesday, 7 November 2012 at 01:33:49 UTC, Jonathan M= Davis wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> The most recent discussion where Walter and Andrei were part of the discuss= ion<br> was here:<br> <br> <a href=3D"http://forum.dlang.org/post/4F84D6DD.5090405 digitalmars.com" ta= rget=3D"_blank">http://forum.dlang.org/post/<u></u>4F84D6DD.5090405 digital= mars.<u></u>com</a><br> </blockquote> <br></div> That thread is quite misleading and, I&#39;m sad to say, not very useful (r= ather damaging to this discussion) in my opinion - especially because the d= istinction between rvalue =3D&gt; &#39;const ref&#39; and rvalue =3D&gt; re= f is largely neglected, and that distinction is of extremely high importanc= e, I can&#39;t stress that enough. Walter&#39;s 3 C++ examples (2 of them i= nvalid anyway afaik) don&#39;t relate to _const_ references. The implicit t= ype conversion problem in that thread isn&#39;t a problem for _const_ refer= ences, just to point out one tiny aspect.<br> rvalue =3D&gt; ref/out propagation makes no sense imho, as does treating li= terals as lvalues (proposed by Walter iirc). The current &#39;auto ref&#39;= semantics also fail to cover the special role of _const_ references for rv= alues (also illustrated by Scarecrow&#39;s post).<div class=3D"im"> <br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Certainly, it&#39;s not a simple matter of just making const<br> ref work with rvalues like most of the people coming from<br> C++ want and expect.<br> </blockquote> <br></div> Well I absolutely do _not_ share this point of view. It just seems so logic= al to me. I&#39;m still waiting for a plausible argument to prove me wrong.= All the required info is in this thread, e.g., we covered the escaping iss= ue you mentioned.<br> </blockquote></div><br></div><div class=3D"gmail_extra">+1</div><div class= =3D"gmail_extra">I couldn&#39;t possibly agree more with your entire post.<= /div> --047d7b6773a6d1127904cdf07ece--
Nov 07 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Wednesday, 7 November 2012 at 21:39:52 UTC, Timon Gehr wrote:
 You can pass him an object that does not support operations you 
 want to preclude. He does not have to _know_, that your book is 
 not changed when he reads it. This is an implementation detail. 
 In fact, you could make the book save away his reading schedule 
 without him noticing.

I don't see where you want to go with this. Do you suggest creating tailored objects (book variants) for each function you're gonna pass it to just to satisfy perfect theoretical encapsulation? So foo() shouldn't be able to change the author => change from inout author reference to const reference? bar() should only be allowed to read the book title, not the actual book contents => hide that string? ;) For the sake of simplicity, by using const we have the ability to at least control if the object can be modified or not. So although my colleague doesn't have to _know_ that he can't modify my book in any way (or know that the book is modifiable in the first place), using const is a primitive but practical way for me to prevent him from doing so. In the context of this rvalue => (const) ref discussion, const is useful due to a number of reasons. 1) All possible side effects of the function invokation are required to be directly visible by the caller. Some people may find that annoying, but I find it useful, and there's a single-line workaround (lvalue declaration) for the (in my opinion very rare) cases where a potential side-effect is either known not to occur or simply uninteresting (requiring exact knowledge about the function implementation, always, i.e., during the whole life-time of that code!). 2) Say we pass a literal string (rvalue) to a const ref parameter. The location of the string in memory can then be freely chosen by the compiler, possibly in a static data segment of the binary (literal optimization - only one location for multiple occurrences). If the parameter was a mutable ref, the compiler should probably allocate a copy on the stack before calling the function, otherwise the literal may not be the same when accessed later on, potentially causing funny bugs. 3) Implicit type conversion isn't a problem. Say we pass an int rvalue to a mutable double ref parameter. The parameter will then be a reference to another rvalue (the int cast to a double) and altering it (the hidden double rvalue) may not really be what the coder intended. Afaik D doesn't support implicit casting for user-defined types, so that may not be a problem (for now at least).
Nov 07 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, October 18, 2012 05:07:52 Malte Skarupke wrote:
 What do you think?

Okay. Here are more links to Andrei discussing the problem: http://forum.dlang.org/post/4F83DBE5.20800 erdani.com http://www.mail-archive.com/digitalmars-d puremagic.com/msg44070.html http://www.mail-archive.com/digitalmars-d puremagic.com/msg43769.html http://forum.dlang.org/post/hg62rq$2c2n$1 digitalmars.com He is completely convinced that letting rvalues bind to const& in C++ was a huge mistake, and it seems to come down mainly to this: "The problem with binding rvalues to const ref is that once that is in place you have no way to distinguish an rvalue from a const ref on the callee site." And apparenly it required adding the concept of rvalue references to C++, which complicated things considerably. It's too bad that he hasn't replied to some of the more detailed questions on the matter in this thread, but if you want rvalues to bind to const ref in D, you're going to have to convince him. At this point, I expect that the most likely solution is that auto ref will continue to work like it does in templates but for non-templated functions it will become like const ref is in C++ and accept rvalues without copying lvalues. Andrei suggests that in at least one of those posts, and AFAIK, it would work just fine. The only real downside that I'm aware of is that then the semantics of auto ref are slightly different for non-templated functions, but you're doing basically the same thing with auto ref in both cases - avoiding copying lvalues and only copying rvalues when you have to - so it really shouldn't be a problem. The main problem is that someone needs to go and implement it. - Jonathan M Davis
Nov 07 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Thursday, 8 November 2012 at 03:07:00 UTC, Jonathan M Davis 
wrote:
 Okay. Here are more links to Andrei discussing the problem:

 http://forum.dlang.org/post/4F83DBE5.20800 erdani.com
 http://www.mail-archive.com/digitalmars-d puremagic.com/msg44070.html
 http://www.mail-archive.com/digitalmars-d puremagic.com/msg43769.html
 http://forum.dlang.org/post/hg62rq$2c2n$1 digitalmars.com

Thank you so much for these links, Jonathan. So fortunately the special role of _const_ ref parameters has been acknowledged. From the 2nd link:
 The problem with binding rvalues to const ref is that
 once that is in place you have no way to distinguish an
 rvalue from a const ref on the callee site. If you do want
 to distinguish, you must rely on complicated conversion
 priorities. For example, consider:

 void foo(ref const Widget);
 void foo(Widget);

 You'd sometimes want to do that because you want to exploit
 an rvalue by e.g. moving its state instead of copying it.
 However, if rvalues become convertible to ref const, then
 they are motivated to go either way. A rule could be put in
 place that gives priority to the second declaration. However,
 things quickly get complicated in the presence of other
 applicable rules, multiple parameters etc. Essentially it
 was impossible for C++ to go this way and that's how rvalue
 references were born.

 For D I want to avoid all that aggravation and have a simple
 rule: rvalues don't bind to references to const. If you don't
 care, use auto ref. This is a simple rule that works
 promisingly well in various forwarding scenarios.

This is exactly what we propose (to be able to avoid pointer/reference indirection for rvalues in some absolutely performance-critical cases). Unlike Andrei though, I don't find the required overloading rules complicated at all, quite the contrary in fact. From the 3rd link:
 Binding rvalues to const references was probably the single
 most hurtful design decisions for C++. I don't have time to
 explain now, but in short I think all of the problems that
 were addressed by rvalue references, and most of the
 aggravation within the design of rvalue references, are owed
 by that one particular conversion.

Here's where I totally disagree with Andrei. C++ rvalue references (T&&) aren't used to distinguish between lvalues and rvalues when expecting a _const_ reference (I still have to see a use case for 'const T&&'). They are used for _mutable_ references and primarily to enforce efficient move semantics in C++, i.e., to move _mutable_ rvalue arguments (instead of copying them) and to enforce 'Named Return Value Optimization' when returning lvalues (by using std::move; goal again is to avoid a redundant copy). D fortunately seems to implement move semantics out-of-the-box (at least now in v2.060), in both cases, see Rob T's posts and my replies in this thread. Besides implementing move semantics, C++ with its rvalue refs also implicitly provides a way to distinguish between _mutable_ lvalue and rvalue references and so allows optimized implementations - that is something we'd also need in D, but that's what we've just covered with regard to the 2nd link. So I still don't see a valid reason to preclude binding rvalues to const ref parameters.
Nov 08 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Thursday, 8 November 2012 at 18:28:44 UTC, Dmitry Olshansky 
wrote:
 What's wrong with going this route:

 void blah(auto ref X stuff){
 ...lots of code...
 }

 is magically expanded to:

 void blah(ref X stuff){
 ...that code..
 }

 and

 void blah(X stuff){
 	.blah(stuff); //now here stuff is L-value so use the ref 
 version
 }

 Yeah, it looks _almost_ like a template now. But unlike with a 
 template we can assume it's 2 overloads _always_. External  
 fucntion issue is then solved by treating it as exactly these 2 
 overloads (one trampoline, one real). Basically it becomes 
 one-line declaration of 2 functions.

 Given that temporaries are moved anyway the speed should be 
 fine and there is as much bloat as you'd do by hand.

 Also hopefully inliner can be counted on to do its thing in 
 this simple case.

That second overload for rvalues would be a shortcut to save the lvalue declarations at each call site - and it really doesn't matter if the compiler magically added the lvalue declarations before each call or if it magically added the rvalue overload (assuming all calls are inlined). But it would create a problem if there already was an explicit 'void blah(X)' overload in addition to 'void blah(auto ref X)' (not making much sense obviously, but this would be something the compiler needed to handle somehow). What this 'auto ref' approach (both as currently implemented for templates and proposed here for non-templated functions) lacks is the vital distinction between const and mutable parameters. For the much more common const ref parameters, I repeatedly tried to explain why I'm absolutely convinced that we don't need another keyword and that 'in/const ref' is sufficient, safe, logical and intuitive (coupled with the overload rule that pass-by-value (moving) is preferred for rvalues). Please prove me wrong. For the less common mutable ref parameters, I also repeatedly tried to explain why I find it dangerous/unsafe to allow rvalues to be bound to mutable ref parameters. But if there are enough people wanting that, I'd have no problem with an 'auto ref' approach for it (only for mutable parameters!). That may actually be a good compromise, what do you guys think? :) 'auto ref T' for templates expands to 'ref T' (lvalues) and 'T' (rvalues), duplicating the whole function and providing best performance - no pointer/reference indirection for rvalues in contrast to 'auto ref T' (proposed above) for non-templates, otherwise the concept would be exactly the same. But it's only for mutable parameters. Such a templated option may also be worth for const parameters though (expanding to 'const ref T' and 'const T'), so maybe something like the (ambiguous) 'in/const auto ref T' wouldn't actually be that bad (assuming there are only a few use cases, and only for templates! It'd still be 'in ref T' for non-templates).
Nov 08 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--047d7b5da809df457404ce01231a
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

That's cute, but it really feels like a hack.
All of a sudden the debugger doesn't work properly anymore, you need to
step-in twice to enter the function, and it's particularly inefficient in
debug builds (a point of great concern for my industry!).

Please just with the compiler creating a temporary in the caller space.
Restrict is to const ref, or better, in ref (scope seems particularly
important here).


On 8 November 2012 20:28, Dmitry Olshansky <dmitry.olsh gmail.com> wrote:

 11/7/2012 3:54 AM, Manu =D0=BF=D0=B8=D1=88=D0=B5=D1=82:

  If the compiler started generating 2 copies of all my ref functions, I'd
 be rather unimpressed... bloat is already a problem in D. Perhaps this
 may be a handy feature, but I wouldn't call this a 'solution' to this
 issue.
 Also, what if the function is external (likely)... auto ref can't work
 if the function is external, an implicit temporary is required in that
 case.

void blah(auto ref X stuff){ ...lots of code... } is magically expanded to: void blah(ref X stuff){ ...that code.. } and void blah(X stuff){ .blah(stuff); //now here stuff is L-value so use the ref version } Yeah, it looks _almost_ like a template now. But unlike with a template w=

 can assume it's 2 overloads _always_. External  fucntion issue is then
 solved by treating it as exactly these 2 overloads (one trampoline, one
 real). Basically it becomes one-line declaration of 2 functions.

 Given that temporaries are moved anyway the speed should be fine and ther=

 is as much bloat as you'd do by hand.

 Also hopefully inliner can be counted on to do its thing in this simple
 case.



 --
 Dmitry Olshansky

--047d7b5da809df457404ce01231a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable That&#39;s cute, but it really feels like a hack.<div>All of a sudden the d= ebugger doesn&#39;t work properly anymore, you need to step-in twice to ent= er the function, and it&#39;s particularly inefficient in debug builds (a p= oint of great concern for my industry!).</div> <div><br></div><div>Please just with the compiler creating a temporary in t= he caller space. Restrict is to const ref, or better, in ref (scope seems p= articularly important here).</div><div class=3D"gmail_extra"><br><br><div c= lass=3D"gmail_quote"> On 8 November 2012 20:28, Dmitry Olshansky <span dir=3D"ltr">&lt;<a href=3D= "mailto:dmitry.olsh gmail.com" target=3D"_blank">dmitry.olsh gmail.com</a>&= gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 = 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> 11/7/2012 3:54 AM, Manu =D0=BF=D0=B8=D1=88=D0=B5=D1=82:<div class=3D"im"><b= r> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> If the compiler started generating 2 copies of all my ref functions, I&#39;= d<br> be rather unimpressed... bloat is already a problem in D. Perhaps this<br> may be a handy feature, but I wouldn&#39;t call this a &#39;solution&#39; t= o this issue.<br> Also, what if the function is external (likely)... auto ref can&#39;t work<= br> if the function is external, an implicit temporary is required in that case= .<br> <br> </blockquote> <br></div> What&#39;s wrong with going this route:<br> <br> void blah(auto ref X stuff){<br> ...lots of code...<br> }<br> <br> is magically expanded to:<br> <br> void blah(ref X stuff){<br> ...that code..<br> }<br> <br> and<br> <br> void blah(X stuff){<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 .blah(stuff); //now here stuff is L-value so us= e the ref version<br> }<br> <br> Yeah, it looks _almost_ like a template now. But unlike with a template we = can assume it&#39;s 2 overloads _always_. External =C2=A0fucntion issue is = then solved by treating it as exactly these 2 overloads (one trampoline, on= e real). Basically it becomes one-line declaration of 2 functions.<br> <br> Given that temporaries are moved anyway the speed should be fine and there = is as much bloat as you&#39;d do by hand.<br> <br> Also hopefully inliner can be counted on to do its thing in this simple cas= e.<span class=3D"HOEnZb"><font color=3D"#888888"><br> <br> <br> <br> -- <br> Dmitry Olshansky<br> </font></span></blockquote></div><br></div> --047d7b5da809df457404ce01231a--
Nov 08 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Thursday, 8 November 2012 at 20:15:51 UTC, Dmitry Olshansky 
wrote:
 The scope. It's all about getting the correct scope, destructor 
 call and you know, the works. Preferably it can inject it 
 inside temporary scope.

 typeof(foo(...)) r = void;
 {
 someRef = SomeResource(x, y, ..);
 r = foo(someRef); // should in fact construct in place not 
 assign
 }

 I suspect this is hackable to be more clean inside of the 
 compiler but not in terms of a re-write.

Right, I forgot the scope for a moment. I'd illustrate the rvalue => (const) ref binding to a novice language user as follows: T const_foo( in ref int x); T mutable_foo(auto ref int x); int bar() { return 5; } T result; result = const_foo(bar()); /* expanded to: { immutable int tmp = bar(); // avoidable for literals result = const_foo(tmp); } // destruction of tmp */ result = mutable_foo(bar()); /* expanded to: { int tmp = bar(); result = mutable_foo(tmp); } // destruction of tmp */
 I'd rather restrict it to 'auto ref' thingie. Though 'in auto 
 ref' sounds outright silly.
 Simply put const ref implies that callee can save a pointer to 
 it somewhere (it's l-value). The same risk is with 'auto ref' 
 but at least there an explicitly written 'disclaimer' by the 
 author of accepting temporary stuff.

'in ref' as opposed to 'const ref' should disallow this escaping issue we've already tackled in this thread, but I'm not sure if it is already/correctly implemented. Anyway, this issue also arises with (short-lived) local lvalues at the caller site: foreach (i; 0 .. 10) { int scopedLvalue = i + 2; foo(scopedLvalue); // passed by ref } // scopedLvalue is gone
 In the ideal world name 'auto ref' would be shorter, logical 
 and more to the point but we have what we have.

+1, but I don't have a better proposal anyway. ;)
 I think that function plucked with auto ref is a enough 
 indication that author is fine with passing to it mutable 
 r-values and not seeing changes outside and related blah-blah.

Agreed.
 Also certain stuff can't be properly bitwise const because of 
 C-calls and what not. Logical const is the correct term but in 
 the D world it's simply mutable.

As you know, I'd definitely allow rvalues to be bound to const ref parameters as alternative (that would also be useful for a lot of existing code). People who generally don't use const (Timon Gehr? :)) are free to only use 'auto ref', I'm most likely only going to use 'in ref', and there will certainly be people using both. Sounds like a really good compromise to me.
 I'd say that even for templates the speed argument is mostly 
 defeated by the bloat argument. But that's probably only me.

I haven't performed any benchmarks, but I tend to agree with you, especially since multiple 'auto ref' parameters lead to exponential bloating. I could definitely do without a special role for templates, which would further simplify things considerably. If performance is really that critical, an explicit pass-by-value (move) overload for rvalues ought to be enough flexibility imo.
Nov 08 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, November 08, 2012 21:49:58 Manu wrote:
 That's cute, but it really feels like a hack.
 All of a sudden the debugger doesn't work properly anymore, you need to
 step-in twice to enter the function, and it's particularly inefficient in
 debug builds (a point of great concern for my industry!).
 
 Please just with the compiler creating a temporary in the caller space.
 Restrict is to const ref, or better, in ref (scope seems particularly
 important here).

I honestly wish that in didn't exist in the language. The fact that it it's an alias two different attributes is confusing, and people keep using it without realizing what they're getting into. If scope worked correctly, you'd only want it in specific circumstances, not in general. And since it doesn't work correctly aside from delegates, once it _does_ work correctly, it'll break code all over the place, because people keep using in, because they like how it corresponds with out or whatever. - Jonathan M Davis
Nov 08 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Thursday, 8 November 2012 at 22:34:03 UTC, Timon Gehr wrote:
 Ambiguous to me and all the interpretations are either wrong or 
 irrelevant.

My point is that it may affect performance. If there was no const, the compiler would need to allocate a dedicated copy of a literal whenever passing it to a mutable ref parameter unless the optimizer worked so well it can prove it's not going to be modified (which I'm sure you'd expect though :D).
 Maybe you should stop trying to show that 'const' is sufficient 
 for resolving those issues. The point is that it is not 
 _necessary_. It is too strong.

In that case it actually is - who cares if the read-only double rvalue the function is passed is the result of an implicit cast (and there's a reason for it being implicit) of the original argument (int rvalue)? Anyway, I think we have moved on in this thread, so maybe you could contribute to trying to settle this rvalue => (const) ref issue once and for all by commenting my latest proposal.
Nov 08 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Thursday, 8 November 2012 at 22:44:26 UTC, Jonathan M Davis 
wrote:
 I honestly wish that in didn't exist in the language. The fact 
 that it it's an alias two different attributes is confusing, and
 people keep using it without realizing what they're getting 
 into.
 If scope worked correctly, you'd only want it in specific
 circumstances, not in general. And since it doesn't work 
 correctly
 aside from delegates, once it _does_ work correctly, it'll break
 code all over the place, because people keep using in, because 
 they like how it corresponds with out or whatever.

I agree that it may likely be a cause for future issues. I wouldn't remove it though, rather relax it to an alias for const only (yes, because I like how it corresponds with out (input only vs. output only) and especially because it is very short - this diff of 3 characters really make a difference in function signatures :D). That'd fortunately still be possible without breaking existing code. So please generalize my countless mentionings of 'in ref' to 'const ref'. ;)
Nov 08 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--20cf303b41215e31bd04ce0bcff0
Content-Type: text/plain; charset=UTF-8

On 9 November 2012 00:44, Jonathan M Davis <jmdavisProg gmx.com> wrote:

 On Thursday, November 08, 2012 21:49:58 Manu wrote:
 That's cute, but it really feels like a hack.
 All of a sudden the debugger doesn't work properly anymore, you need to
 step-in twice to enter the function, and it's particularly inefficient in
 debug builds (a point of great concern for my industry!).

 Please just with the compiler creating a temporary in the caller space.
 Restrict is to const ref, or better, in ref (scope seems particularly
 important here).

I honestly wish that in didn't exist in the language. The fact that it it's an alias two different attributes is confusing, and people keep using it without realizing what they're getting into.

I understand it's an alias for 'const scope', but what's this about scope not working correctly? What's wrong with it? It seems like precisely what you want in this case... you don't want a temporary escaping the function you pass it to. What's the problem? If scope worked correctly, you'd only
 want it in specific circumstances, not in general. And since it doesn't
 work
 correctly aside from delegates, once it _does_ work correctly, it'll break
 code all over the place, because people keep using in, because they like
 how
 it corresponds with out or whatever.

I like how it promises that the thing I pass can't escape the callee, not that it's the natural english compliment to 'out'. --20cf303b41215e31bd04ce0bcff0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <br><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On 9 Nove= mber 2012 00:44, Jonathan M Davis <span dir=3D"ltr">&lt;<a href=3D"mailto:j= mdavisProg gmx.com" target=3D"_blank">jmdavisProg gmx.com</a>&gt;</span> wr= ote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im">On Thursday, November 08, = 2012 21:49:58 Manu wrote:<br> &gt; That&#39;s cute, but it really feels like a hack.<br> &gt; All of a sudden the debugger doesn&#39;t work properly anymore, you ne= ed to<br> &gt; step-in twice to enter the function, and it&#39;s particularly ineffic= ient in<br> &gt; debug builds (a point of great concern for my industry!).<br> &gt;<br> &gt; Please just with the compiler creating a temporary in the caller space= .<br> &gt; Restrict is to const ref, or better, in ref (scope seems particularly<= br> &gt; important here).<br> <br> </div>I honestly wish that in didn&#39;t exist in the language. The fact th= at it it&#39;s an<br> alias two different attributes is confusing, and people keep using it witho= ut<br> realizing what they&#39;re getting into.</blockquote><div><br></div><div>I = understand it&#39;s an alias for &#39;const scope&#39;, but what&#39;s this= about scope not working correctly? What&#39;s wrong with it?</div><div> It seems like precisely what you want in this case... you don&#39;t want a = temporary escaping the function you pass it to.</div><div>What&#39;s the pr= oblem?</div><div><br></div><blockquote class=3D"gmail_quote" style=3D"margi= n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> If scope worked correctly, you&#39;d only<br> want it in specific circumstances, not in general. And since it doesn&#39;t= work<br> correctly aside from delegates, once it _does_ work correctly, it&#39;ll br= eak<br> code all over the place, because people keep using in, because they like ho= w<br> it corresponds with out or whatever.<br></blockquote><div><br></div><div>I = like how it promises that the thing I pass can&#39;t escape the callee, not= that it&#39;s the natural english compliment to &#39;out&#39;.</div></div> </div> --20cf303b41215e31bd04ce0bcff0--
Nov 09 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, November 09, 2012 10:33:45 Manu wrote:
 On 9 November 2012 00:44, Jonathan M Davis <jmdavisProg gmx.com> wrote:
 On Thursday, November 08, 2012 21:49:58 Manu wrote:
 That's cute, but it really feels like a hack.
 All of a sudden the debugger doesn't work properly anymore, you need to
 step-in twice to enter the function, and it's particularly inefficient
 in
 debug builds (a point of great concern for my industry!).
 
 Please just with the compiler creating a temporary in the caller space.
 Restrict is to const ref, or better, in ref (scope seems particularly
 important here).

I honestly wish that in didn't exist in the language. The fact that it it's an alias two different attributes is confusing, and people keep using it without realizing what they're getting into.

I understand it's an alias for 'const scope', but what's this about scope not working correctly? What's wrong with it? It seems like precisely what you want in this case... you don't want a temporary escaping the function you pass it to. What's the problem?

It only works with delegates. In all other cases, it's ignored.
 If scope worked correctly, you'd only
 
 want it in specific circumstances, not in general. And since it doesn't
 work
 correctly aside from delegates, once it _does_ work correctly, it'll break
 code all over the place, because people keep using in, because they like
 how
 it corresponds with out or whatever.

I like how it promises that the thing I pass can't escape the callee, not that it's the natural english compliment to 'out'.

There are cases where it's useful, but there are a ton of cases where it would be really annoying if it worked properly. For instance, in a _lot_ of cases, it would cause problems for a function taking an array, because it couldn't return a slice of the array. Any struct holding any reference types would be in the same boat, as would any class or AA. In most cases, it really doesn't matter whether references escape, because the stuff is on the GC heap, and it'll collect it when it's appropriate. It would be a much bigger deal if you had to worry about ownership. For delegates, it's great because it allows you to avoid allocating a closure, so if you don't need the delegate to escape, it makes perfect sense to make it scope. For everything else, whether it makes sense depends on what you're doing, and sometimes it would be useful, but most of the time, it really wouldn't be. But since in currently only works for delegates, it's absolutely pointless to use it instead of const for anything else, and if/when scope is ever fixed to work with reference types in general, then all kinds of code that uses in will break, because it'll be escaping references, and the programmer didn't notice it. So, you might as well just not use it at this point except for delegates. Regardless, you don't need in to use scope. You can just use scope directly. So, having in buys you _nothing_, and it unfortunately encourages people to use it (which IMHO is bad due to the issues with scope), because they like how it's the opposite of out. So, if we didn't have in, we'd be able to do exactly what we can do now, but people wouldn't be using scope all over the place via in just because they liked the idea of in being the opposite of out. But originally, in comes from D1 (where it had a meaning similar to const IIRC, but scope had nothing to do with it), so it's been around for a long time, and there's pretty much no way that it would be removed from the language. I just think that it was mistake to have it given its current semantics. It's also a bit pointless IMHO to have an attribute which is an alias for other attributes. It's a needless duplication of functionality and just increases confusion. - Jonathan M Davis
Nov 09 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--f46d043890b9efb9d204ce104c65
Content-Type: text/plain; charset=UTF-8

On 9 November 2012 10:46, Jonathan M Davis <jmdavisProg gmx.com> wrote:

 On Friday, November 09, 2012 10:33:45 Manu wrote:
 On 9 November 2012 00:44, Jonathan M Davis <jmdavisProg gmx.com> wrote:
 On Thursday, November 08, 2012 21:49:58 Manu wrote:
 That's cute, but it really feels like a hack.
 All of a sudden the debugger doesn't work properly anymore, you need



 step-in twice to enter the function, and it's particularly



 in
 debug builds (a point of great concern for my industry!).

 Please just with the compiler creating a temporary in the caller



 Restrict is to const ref, or better, in ref (scope seems particularly
 important here).

I honestly wish that in didn't exist in the language. The fact that it it's an alias two different attributes is confusing, and people keep using it without realizing what they're getting into.

I understand it's an alias for 'const scope', but what's this about scope not working correctly? What's wrong with it? It seems like precisely what you want in this case... you don't want a temporary escaping the function you pass it to. What's the problem?

It only works with delegates. In all other cases, it's ignored.
 If scope worked correctly, you'd only

 want it in specific circumstances, not in general. And since it doesn't
 work
 correctly aside from delegates, once it _does_ work correctly, it'll


 code all over the place, because people keep using in, because they


 how
 it corresponds with out or whatever.

I like how it promises that the thing I pass can't escape the callee, not that it's the natural english compliment to 'out'.

There are cases where it's useful, but there are a ton of cases where it would be really annoying if it worked properly. For instance, in a _lot_ of cases, it would cause problems for a function taking an array, because it couldn't return a slice of the array.

Does that actually make sense? Surely a function that receives a scope argument can return that argument, since it's only passing it back to the same function that already owns it... it knows it can trust that function, since it was received from that function. Any struct holding any reference types would be
 in the same boat, as would any class or AA.

I don't follow the problem with reference args. surely they can be evaluated just fine? Just that nothing can escape the function... In most cases, it really doesn't matter whether references escape, because
 the
 stuff is on the GC heap, and it'll collect it when it's appropriate. It
 would
 be a much bigger deal if you had to worry about ownership.

It's true, and stack locals are perhaps the most important thing that scope could possibly protect... which makes it particularly applicable here. For delegates, it's great because it allows you to avoid allocating a
 closure,
 so if you don't need the delegate to escape, it makes perfect sense to
 make it
 scope. For everything else, whether it makes sense depends on what you're
 doing, and sometimes it would be useful, but most of the time, it really
 wouldn't be.

Of course, and whether you apply the scope attribute to a function argument equally depends on what you're doing :) I still can't see any inherent problems from what you're saying. It's still just an access restriction to enhance safety. But since in currently only works for delegates, it's absolutely pointless
 to
 use it instead of const for anything else, and if/when scope is ever fixed
 to
 work with reference types in general, then all kinds of code that uses in
 will
 break, because it'll be escaping references, and the programmer didn't
 notice
 it. So, you might as well just not use it at this point except for
 delegates.

Very good point. I had no idea it didn't work. It'd be nice to see this fixed. Regardless, you don't need in to use scope. You can just use scope directly.
 So, having in buys you _nothing_, and it unfortunately encourages people to
 use it (which IMHO is bad due to the issues with scope), because they like
 how
 it's the opposite of out.

The issue here is that 'scope' is an incomplete feature. I think it's theoretically sound. Every conversation/thing I've read on the topic certainly gave me the impression it worked... So, if we didn't have in, we'd be able to do exactly
 what we can do now, but people wouldn't be using scope all over the place
 via
 in just because they liked the idea of in being the opposite of out.

 But originally, in comes from D1 (where it had a meaning similar to const
 IIRC, but scope had nothing to do with it), so it's been around for a long
 time, and there's pretty much no way that it would be removed from the
 language.

Oh really? That's not at all what I had imagined. I presumed 'in' was invented when it was realised that 'const scope' was a super-common pattern. I still don't see any less value in it. Just that it should work properly. I just think that it was mistake to have it given its current
 semantics. It's also a bit pointless IMHO to have an attribute which is an
 alias for other attributes. It's a needless duplication of functionality
 and
 just increases confusion.

Side topic: what I find myself constantly wanting is alias for attributes! Especially when dealing with GDC/LDC which have their own non-standard attributes. I'm often wanting to create a meaning alias for the set of attributes applicable to my code. And you can't currently use GDC attributes without static if and physically duplicating all the attributed code >_< --f46d043890b9efb9d204ce104c65 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 9 November 2012 10:46, Jonathan M Davis <span dir=3D"ltr">&lt;<a href=3D= "mailto:jmdavisProg gmx.com" target=3D"_blank">jmdavisProg gmx.com</a>&gt;<= /span> wrote:<br><div class=3D"gmail_extra"><div class=3D"gmail_quote"><blo= ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c= cc solid;padding-left:1ex"> <div class=3D"im">On Friday, November 09, 2012 10:33:45 Manu wrote:<br> &gt; On 9 November 2012 00:44, Jonathan M Davis &lt;<a href=3D"mailto:jmdav= isProg gmx.com">jmdavisProg gmx.com</a>&gt; wrote:<br> &gt; &gt; On Thursday, November 08, 2012 21:49:58 Manu wrote:<br> &gt; &gt; &gt; That&#39;s cute, but it really feels like a hack.<br> &gt; &gt; &gt; All of a sudden the debugger doesn&#39;t work properly anymo= re, you need to<br> &gt; &gt; &gt; step-in twice to enter the function, and it&#39;s particular= ly inefficient<br> &gt; &gt; &gt; in<br> &gt; &gt; &gt; debug builds (a point of great concern for my industry!).<br=

&gt; &gt; &gt; Please just with the compiler creating a temporary in the ca= ller space.<br> &gt; &gt; &gt; Restrict is to const ref, or better, in ref (scope seems par= ticularly<br> &gt; &gt; &gt; important here).<br> &gt; &gt;<br> &gt; &gt; I honestly wish that in didn&#39;t exist in the language. The fac= t that it<br> &gt; &gt; it&#39;s an<br> &gt; &gt; alias two different attributes is confusing, and people keep usin= g it<br> &gt; &gt; without<br> &gt; &gt; realizing what they&#39;re getting into.<br> &gt;<br> &gt; I understand it&#39;s an alias for &#39;const scope&#39;, but what&#39= ;s this about scope<br> &gt; not working correctly? What&#39;s wrong with it?<br> &gt; It seems like precisely what you want in this case... you don&#39;t wa= nt a<br> &gt; temporary escaping the function you pass it to.<br> &gt; What&#39;s the problem?<br> <br> </div>It only works with delegates. In all other cases, it&#39;s ignored.<b= r> <div class=3D"im"><br> &gt; If scope worked correctly, you&#39;d only<br> &gt;<br> &gt; &gt; want it in specific circumstances, not in general. And since it d= oesn&#39;t<br> &gt; &gt; work<br> &gt; &gt; correctly aside from delegates, once it _does_ work correctly, it= &#39;ll break<br> &gt; &gt; code all over the place, because people keep using in, because th= ey like<br> &gt; &gt; how<br> &gt; &gt; it corresponds with out or whatever.<br> &gt;<br> &gt; I like how it promises that the thing I pass can&#39;t escape the call= ee, not<br> &gt; that it&#39;s the natural english compliment to &#39;out&#39;.<br> <br> </div>There are cases where it&#39;s useful, but there are a ton of cases w= here it would<br> be really annoying if it worked properly. For instance, in a _lot_ of cases= ,<br> it would cause problems for a function taking an array, because it couldn&#= 39;t<br> return a slice of the array.</blockquote><div><br></div><div>Does that actu= ally make sense? Surely a function that receives a scope argument can retur= n that argument, since it&#39;s only passing it back to the same function t= hat already owns it... it knows it can trust that function, since it was re= ceived from that function.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"> Any struct holding any refer= ence types would be<br> in the same boat, as would any class or AA.<br></blockquote><div><br></div>= <div>I don&#39;t follow the problem with reference args. surely they can be= evaluated just fine? Just that nothing can escape the function...</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex">In most cases, it really does= n&#39;t matter whether references escape, because the<br> stuff is on the GC heap, and it&#39;ll collect it when it&#39;s appropriate= . It would<br> be a much bigger deal if you had to worry about ownership.<br></blockquote>= <div><br></div><div>It&#39;s true, and stack locals are perhaps the most im= portant thing that scope could possibly protect... which makes it particula= rly applicable here.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex">For delegates, it&#39;s great= because it allows you to avoid allocating a closure,<br> so if you don&#39;t need the delegate to escape, it makes perfect sense to = make it<br> scope. For everything else, whether it makes sense depends on what you&#39;= re<br> doing, and sometimes it would be useful, but most of the time, it really<br=

you apply the scope attribute to a function argument equally depends on wh= at you&#39;re doing :)</div><div>I still can&#39;t see any inherent problem= s from what you&#39;re saying. It&#39;s still just an access restriction to= enhance safety.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"> But since in currently only works for delegates, it&#39;s absolutely pointl= ess to<br> use it instead of const for anything else, and if/when scope is ever fixed = to<br> work with reference types in general, then all kinds of code that uses in w= ill<br> break, because it&#39;ll be escaping references, and the programmer didn&#3= 9;t notice<br> it. So, you might as well just not use it at this point except for delegate= s.<br></blockquote><div><br></div><div>Very good point. I had no idea it di= dn&#39;t work. It&#39;d be nice to see this fixed.</div><div><br></div> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Regardless, you don&#39;t need in to use scope. You can just use scope dire= ctly.<br> So, having in buys you _nothing_, and it unfortunately encourages people to= <br> use it (which IMHO is bad due to the issues with scope), because they like = how<br> it&#39;s the opposite of out.</blockquote><div><br></div><div>The issue her= e is that &#39;scope&#39; is an incomplete feature. I think it&#39;s theore= tically sound.</div><div>Every conversation/thing I&#39;ve read on the topi= c certainly gave me the impression it worked...</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"> So, if we didn&#39;t have in= , we&#39;d be able to do exactly<br> what we can do now, but people wouldn&#39;t be using scope all over the pla= ce via<br> in just because they liked the idea of in being the opposite of out.<br> <br> But originally, in comes from D1 (where it had a meaning similar to const<b= r> IIRC, but scope had nothing to do with it), so it&#39;s been around for a l= ong<br> time, and there&#39;s pretty much no way that it would be removed from the<= br> language.</blockquote><div><br></div><div>Oh really? That&#39;s not at all = what I had imagined. I presumed &#39;in&#39; was invented when it was reali= sed that &#39;const scope&#39; was a super-common pattern.</div><div>I stil= l don&#39;t see any less value in it. Just that it should work properly.</d= iv> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"> I just think that it was mis= take to have it given its current<br> semantics. It&#39;s also a bit pointless IMHO to have an attribute which is= an<br> alias for other attributes. It&#39;s a needless duplication of functionalit= y and<br> just increases confusion.<br></blockquote><div><br></div><div>Side topic: w= hat I find myself constantly wanting is alias for attributes!<br>Especially= when dealing with GDC/LDC which have their own non-standard attributes. I&= #39;m often wanting to create a meaning alias for the set of attributes app= licable to my code.</div> <div>And you can&#39;t currently use GDC attributes without static if and p= hysically duplicating all the attributed code &gt;_&lt;</div></div></div> --f46d043890b9efb9d204ce104c65--
Nov 09 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
Let me summarize my (final, I guess) proposal. I think it makes 
sense to compare it to C++ in order to anticipate and hopefully 
invalidate (mainly Andrei's) objections.

      parameter type     |   lvalue    |    rvalue
                         | C++     D   | C++     D
------------------------|-------------|------------
T                       | copy   copy | copy   move
T& / ref T              | ref    ref  | n/a    n/a
out T (D only)          |        ref  |        n/a
T&& (C++ only)          | n/a         | move
auto ref T (D only) (*) |        ref  |        ref
------------------------|-------------|------------
const T                 | copy   copy | copy   move
const T& / const ref T  | ref    ref  | ref    ref (*)
const T&& (C++ only)    | n/a         | move

(*): proposed additions

For lvalues in both C++ and D, there are 2 options: either copy 
the argument (pass-by-value) or pass it by ref. There's no real 
difference between both languages except for D's additional 'out' 
keyword and, with the proposed 'auto ref' syntax, an (imo 
negligible) ambiguity between 'ref T' and 'auto ref T' in D.
Rvalues are a different topic though. There are 3 possibilites in 
general: copy, move and pass by ref. Copying rvalue arguments 
does not make sense - the argument won't be used by the caller 
after the invokation, so a copy is redundant and hurts 
performance. D corrects this design flaw of C++ (which had to 
introduce rvalue refs to add move semantics on top of the default 
copy semantics) and therefore only supports moving instead. C++ 
additionally supports pass-by-ref of rvalues to const refs, but 
not to mutable refs. I propose to allow pass-by-ref to both const 
(identical syntax as C++, it's perfectly safe and logical) and 
mutable refs (new syntax with 'auto ref' to emphasize that the 
parameter may be an rvalue reference, with related consequences 
such as potentially missing side effects).

Regarding the required overloading priorities for the proposed 
additions to work properly, I propose:
1) lvalues: prefer pass-by-ref
    so: ref/out T -> auto ref T (*) -> const ref T -> (const) T
    - const lvalues:   const ref T -> (const) T
    - mutable lvalues: ref/out T -> auto ref T (*) -> const ref T 
-> (const) T
2) rvalues: prefer pass-by-value (moving: argument allocated 
directly on
    callee's stack (parameter) vs. pointer/reference indirection 
implied by
    pass-by-ref)
    so: (const) T -> auto ref T (*) -> const ref T (*)

Finally, regarding templates, I'm in favor of dropping the 
current 'auto ref' semantics and propose to simply adopt the 
proposed semantics for consistency and simplicity and to avoid 
excessive code bloating. That shouldn't break existing code I 
hope (unless parameters have been denoted with 'const auto ref 
T', which would need to be changed to 'const ref T').

Now please go ahead and shoot. :)
Nov 09 2012
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Fri, 09 Nov 2012 16:26:21 -0000, martin <kinke libero.it> wrote:
 Let me summarize my (final, I guess) proposal. I think it makes sense to  
 compare it to C++ in order to anticipate and hopefully invalidate  
 (mainly Andrei's) objections.

       parameter type     |   lvalue    |    rvalue
                          | C++     D   | C++     D
 ------------------------|-------------|------------
 T                       | copy   copy | copy   move
 T& / ref T              | ref    ref  | n/a    n/a
 out T (D only)          |        ref  |        n/a
 T&& (C++ only)          | n/a         | move
 auto ref T (D only) (*) |        ref  |        ref
 ------------------------|-------------|------------
 const T                 | copy   copy | copy   move
 const T& / const ref T  | ref    ref  | ref    ref (*)
 const T&& (C++ only)    | n/a         | move

 (*): proposed additions

 For lvalues in both C++ and D, there are 2 options: either copy the  
 argument (pass-by-value) or pass it by ref. There's no real difference  
 between both languages except for D's additional 'out' keyword and, with  
 the proposed 'auto ref' syntax, an (imo negligible) ambiguity between  
 'ref T' and 'auto ref T' in D.
 Rvalues are a different topic though. There are 3 possibilites in  
 general: copy, move and pass by ref. Copying rvalue arguments does not  
 make sense - the argument won't be used by the caller after the  
 invokation, so a copy is redundant and hurts performance. D corrects  
 this design flaw of C++ (which had to introduce rvalue refs to add move  
 semantics on top of the default copy semantics) and therefore only  
 supports moving instead. C++ additionally supports pass-by-ref of  
 rvalues to const refs, but not to mutable refs. I propose to allow  
 pass-by-ref to both const (identical syntax as C++, it's perfectly safe  
 and logical) and mutable refs (new syntax with 'auto ref' to emphasize  
 that the parameter may be an rvalue reference, with related consequences  
 such as potentially missing side effects).

 Regarding the required overloading priorities for the proposed additions  
 to work properly, I propose:
 1) lvalues: prefer pass-by-ref
     so: ref/out T -> auto ref T (*) -> const ref T -> (const) T
     - const lvalues:   const ref T -> (const) T
     - mutable lvalues: ref/out T -> auto ref T (*) -> const ref T ->  
 (const) T
 2) rvalues: prefer pass-by-value (moving: argument allocated directly on
     callee's stack (parameter) vs. pointer/reference indirection implied  
 by
     pass-by-ref)
     so: (const) T -> auto ref T (*) -> const ref T (*)

 Finally, regarding templates, I'm in favor of dropping the current 'auto  
 ref' semantics and propose to simply adopt the proposed semantics for  
 consistency and simplicity and to avoid excessive code bloating. That  
 shouldn't break existing code I hope (unless parameters have been  
 denoted with 'const auto ref T', which would need to be changed to  
 'const ref T').

 Now please go ahead and shoot. :)

Nice detailed proposal. I would suggest starting a new thread with it, to catch anyone who dozed off in this thread :p R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 09 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Friday, 9 November 2012 at 16:53:01 UTC, Regan Heath wrote:
 Nice detailed proposal.  I would suggest starting a new thread 
 with it, to catch anyone who dozed off in this thread :p

Thanks, and you're probably right. Here's the link: http://forum.dlang.org/thread/zteryxwxyngvyqvukqkm forum.dlang.org I've elaborated on it a little bit to avoid redundant posts.
Nov 09 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, November 09, 2012 15:55:12 Manu wrote:
 Does that actually make sense? Surely a function that receives a scope
 argument can return that argument, since it's only passing it back to the
 same function that already owns it... it knows it can trust that function,
 since it was received from that function.

It can't. That would mean that the reference escaped. That would be particularly deadly for delegates. Think about what happens if the scoped delegate is put into a struct which is returned. struct Result { delegate........ del; ........... } Result foo(scope delegate....... bar) { .......... return Result(bar); } auto baz() { Result r; { int n = 5; r = foo((){writeln(n);}); } r.del(); } baz has no idea where the delegate in r came from. It has no idea that it wasn't allocated as a closure. So, it's not going to allocate one, which means that the delegate refers to a part of the stack which won't exist anymore when the delegate actually gets called. If scope wasn't used, that wouldn't have been a problem, because a closure would have been allocated as soon as the delegate had been passed to foo, but because scope was used, it knows that the delegate won't escape, so it doesn't allocate the closure (since it's not necessary). But that only works because scope prevents escaping - including by the return value. So, the above code _must_ be invalid.
 Any struct holding any reference types would be
 in the same boat, as would any class or AA.

I don't follow the problem with reference args. surely they can be evaluated just fine? Just that nothing can escape the function...

It's the fact that they can't escape the function which is the problem. If scope is working correctly, than any and all reference types which are passed to the function via a scope parameter (be it directly or within another object) cannot escape the function in any way shape or form. So, you can't assign any of them to any static or module-level variables (not generally a big deal) and you can't use any of them in the return value (often a big deal). Sometimes, that's exactly what you want, but in the general case, you don't want to prevent anything you pass into a function from being returned from it, so scope quickly becames overly restrictive. - Jonathan M Davis
Nov 09 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--20cf307f3a9eccf8d704ce224512
Content-Type: text/plain; charset=UTF-8

On 9 November 2012 21:39, Jonathan M Davis <jmdavisProg gmx.com> wrote:

 On Friday, November 09, 2012 15:55:12 Manu wrote:
 Does that actually make sense? Surely a function that receives a scope
 argument can return that argument, since it's only passing it back to the
 same function that already owns it... it knows it can trust that

 since it was received from that function.

It can't. That would mean that the reference escaped. That would be particularly deadly for delegates. Think about what happens if the scoped delegate is put into a struct which is returned. struct Result { delegate........ del; ........... } Result foo(scope delegate....... bar) { .......... return Result(bar); } auto baz() { Result r; { int n = 5; r = foo((){writeln(n);}); } r.del(); } baz has no idea where the delegate in r came from. It has no idea that it wasn't allocated as a closure. So, it's not going to allocate one, which means that the delegate refers to a part of the stack which won't exist anymore when the delegate actually gets called. If scope wasn't used, that wouldn't have been a problem, because a closure would have been allocated as soon as the delegate had been passed to foo, but because scope was used, it knows that the delegate won't escape, so it doesn't allocate the closure (since it's not necessary). But that only works because scope prevents escaping - including by the return value. So, the above code _must_ be invalid.

Okay, makes sense.
 Any struct holding any reference types would be
 in the same boat, as would any class or AA.

I don't follow the problem with reference args. surely they can be evaluated just fine? Just that nothing can escape the function...

It's the fact that they can't escape the function which is the problem. If scope is working correctly, than any and all reference types which are passed to the function via a scope parameter (be it directly or within another object) cannot escape the function in any way shape or form. So, you can't assign any of them to any static or module-level variables (not generally a big deal) and you can't use any of them in the return value (often a big deal). Sometimes, that's exactly what you want, but in the general case, you don't want to prevent anything you pass into a function from being returned from it, so scope quickly becames overly restrictive.

I'm still not buying this. Here's a common struct I will pass by ref (perhaps the most common struct in my industry): struct Vector { float, x,y,z,w; } struct Matrix { Vector xRow, yRow, zRow, wRow; } Vector mul( scope const ref Matrix m, scope const Vector v) { Vector v; // perform a matrix multiply against the vector... // this work uses every single field of the inputs given, but the result it produces has to references to the sources. // everything is operated on and copied to the output struct, which is returned. return result; } Why should this be a problem? The majority of my work-horse structs apply to this pattern. This is what I imagine 'scope' to be for... The main advantage I expect is that I can have confidence that passing rvalues (temporaries) is safe, and that external code won't take references to memory that I may not own/control. Is that not the point? Surely the problem that scope should be protecting against is a pointer to any part of the argument escaping. *Copies* of values contained in the argument/s are fine. --20cf307f3a9eccf8d704ce224512 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 9 November 2012 21:39, Jonathan M Davis <span dir=3D"ltr">&lt;<a href=3D= "mailto:jmdavisProg gmx.com" target=3D"_blank">jmdavisProg gmx.com</a>&gt;<= /span> wrote:<br><div class=3D"gmail_extra"><div class=3D"gmail_quote"><blo= ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c= cc solid;padding-left:1ex"> <div class=3D"im">On Friday, November 09, 2012 15:55:12 Manu wrote:<br> &gt; Does that actually make sense? Surely a function that receives a scope= <br> &gt; argument can return that argument, since it&#39;s only passing it back= to the<br> &gt; same function that already owns it... it knows it can trust that funct= ion,<br> &gt; since it was received from that function.<br> <br> </div>It can&#39;t. That would mean that the reference escaped. That would = be<br> particularly deadly for delegates. Think about what happens if the scoped<b= r> delegate is put into a struct which is returned.<br> <br> struct Result<br> {<br> =C2=A0delegate........ del;<br> =C2=A0...........<br> }<br> <br> Result foo(scope delegate....... bar)<br> {<br> =C2=A0..........<br> =C2=A0return Result(bar);<br> }<br> <br> auto baz()<br> {<br> =C2=A0Result r;<br> =C2=A0{<br> =C2=A0int n =3D 5;<br> =C2=A0r =3D foo((){writeln(n);});<br> =C2=A0}<br> =C2=A0r.del();<br> }<br> <br> baz has no idea where the delegate in r came from. It has no idea that it<b= r> wasn&#39;t allocated as a closure. So, it&#39;s not going to allocate one, = which means<br> that the delegate refers to a part of the stack which won&#39;t exist anymo= re when<br> the delegate actually gets called. If scope wasn&#39;t used, that wouldn&#3= 9;t have<br> been a problem, because a closure would have been allocated as soon as the<= br> delegate had been passed to foo, but because scope was used, it knows that = the<br> delegate won&#39;t escape, so it doesn&#39;t allocate the closure (since it= &#39;s not<br> necessary). But that only works because scope prevents escaping - including= by<br> the return value. So, the above code _must_ be invalid.<br></blockquote><di= v><br></div><div>Okay, makes sense.</div><div><br></div><blockquote class= =3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd= ing-left:1ex"> <div class=3D"im"> &gt;&gt; Any struct holding any reference types would be<br> &gt; &gt; in the same boat, as would any class or AA.<br> &gt;<br> &gt; I don&#39;t follow the problem with reference args. surely they can be= <br> &gt; evaluated just fine? Just that nothing can escape the function...<br> <br> </div>It&#39;s the fact that they can&#39;t escape the function which is th= e problem. If<br> scope is working correctly, than any and all reference types which are pass= ed<br> to the function via a scope parameter (be it directly or within another<br> object) cannot escape the function in any way shape or form. So, you can&#3= 9;t<br> assign any of them to any static or module-level variables (not generally a= <br> big deal) and you can&#39;t use any of them in the return value (often a bi= g<br> deal). Sometimes, that&#39;s exactly what you want, but in the general case= , you<br> don&#39;t want to prevent anything you pass into a function from being retu= rned<br> from it, so scope quickly becames overly restrictive.<br></blockquote><div>= <br></div><div>I&#39;m still not buying this. Here&#39;s a common struct I = will pass by ref (perhaps the most common struct in my industry):</div> <div><br></div><div>struct Vector { float, x,y,z,w; }</div><div>struct Matr= ix { Vector xRow, yRow, zRow, wRow; }</div><div><br></div><div>Vector mul( = scope const ref Matrix m, scope const Vector v)</div><div>{</div><div>=C2= =A0 Vector v;</div> <div>=C2=A0 // perform a matrix multiply against the vector...</div><div>= =C2=A0 // this work uses every single field of the inputs given, but the re= sult it produces has to references to the sources.</div><div>=C2=A0 // ever= ything is operated on and copied to the output struct, which is returned.</= div> <div>=C2=A0 return result;</div><div>}</div><div><br></div><div>Why should = this be a problem?</div><div>The majority of my work-horse structs apply to= this pattern. This is what I imagine &#39;scope&#39; to be for...</div><di= v> The main advantage I expect is that I can have confidence that passing rval= ues (temporaries) is safe, and that external code won&#39;t take references= to memory that I may not own/control. Is that not the point?</div><div> <br></div><div>Surely the problem that scope should be protecting against i= s a pointer to any part of the argument escaping. *Copies* of values contai= ned in the argument/s are fine.</div></div></div> --20cf307f3a9eccf8d704ce224512--
Nov 10 2012
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, November 10, 2012 13:21:42 Manu wrote:
 I'm still not buying this. Here's a common struct I will pass by ref
 (perhaps the most common struct in my industry):
=20
 struct Vector { float, x,y,z,w; }
 struct Matrix { Vector xRow, yRow, zRow, wRow; }
=20
 Vector mul( scope const ref Matrix m, scope const Vector v)
 {
   Vector v;
   // perform a matrix multiply against the vector...
   // this work uses every single field of the inputs given, but the r=

 it produces has to references to the sources.
   // everything is operated on and copied to the output struct, which=

 returned.
   return result;
 }
=20
 Why should this be a problem?
 The majority of my work-horse structs apply to this pattern. This is =

 imagine 'scope' to be for...
 The main advantage I expect is that I can have confidence that passin=

 rvalues (temporaries) is safe, and that external code won't take refe=

 to memory that I may not own/control. Is that not the point?
=20
 Surely the problem that scope should be protecting against is a point=

 any part of the argument escaping. *Copies* of values contained in th=

 argument/s are fine.

Hmmmm. scope on value types is pointless, because there are no referenc= es to=20 escape, but if you pass by ref, then it does become possible for a poin= ter to=20 the argument to escape, but I don't know that that's actually actually = covered=20 by scope. The description for scope in docs is that "ref=C2=ADer=C2=ADe= nces in the pa=C2=AD ra=C2=ADme=C2=ADter can=C2=ADnot be es=C2=ADcaped (e.g. as=C2=ADsigned = to a global vari=C2=ADable)." And=20 taking the address of a local variable (which is the only way that any = sort of=20 reference to the data could escape) is never safe anyway. If you passe= d in a=20 pointer, and scope were fully working, then you'd be protected against = the=20 pointer escaping, but passing by ref isn't really the same thing. I'd h= ave=20 thought that taking the address of a variable passed by ref would fall = into=20 pretty much the same camp as taking the address of any other local vari= able,=20 which is completely unsafe to escape to the point that I'm not sure tha= t=20 there's any point in protecting against it. It's just completely stupid= to do=20 anyway and is definitely system. Outside of taking the address of a re= f=20 parameter, taking the address of a local variable and escpaing it is _a= lways_=20 going to result in garbage, and ref parameters aren't really references= in the=20 normal sense, so I don't know. You bring up a good point, but I don't know if it's applicable. Certain= ly,=20 without the ref there (like is the case with the Vector that you're pas= sing=20 in), scope would never do anything, because it doesn't even theoretical= ly have=20 anything to do. It's purely a value type that's not even being passed b= y ref. In general though, putting scope on struct parameters would cause a lot= of=20 problems, because of arrays that they might hold and whatnot. Slices wo= uldn't=20 be able to escape (and so copies of the struct wouldn't be able escape = without=20 deep copying, let alone the array itself). So, while scope may be very = useful=20 in some such cases (assuming that it worked), it's not necessarily some= thing=20 that you'd want as a matter of course. Part of it probably depends on y= our=20 programming style though. If you have a lot of functions that take argu= ments=20 and don't return anything that was in them ever, then scope is less of = a big=20 deal, but that's the sort of thing that happens a _lot_ in my experienc= e, so=20 scope would very quickly become extremely annoying. And actually, to make matters worse, I'm not sure that scope on delegat= es is=20 working correctly. I thought that it was, but this code compiles: import std.stdio; void delegate() global; void foo(scope void delegate() del) { global =3D del; } void main() { { char[5] bar =3D "hello"; foo((){writeln(bar);}); } char[7] baz =3D "goodbye"; global(); } It also prints out "hello", and if a closure had not been allocated, I = would=20 have at least half-expected it to print out "goodb", because I'd have t= hought=20 that baz would have been taking up the same memory that bar had been. S= o, it=20 looks like scope may be completely and utterly broken at this point. I = don't=20 know. - Jonathan M Davis
Nov 10 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/10/2012 11:07 PM, Jonathan M Davis пишет:
 And actually, to make matters worse, I'm not sure that scope on delegates is
 working correctly. I thought that it was, but this code compiles:

 import std.stdio;

 void delegate() global;

 void foo(scope void delegate() del)
 {
      global = del;
 }

 void main()
 {
      {
          char[5] bar = "hello";
          foo((){writeln(bar);});
      }
      char[7] baz = "goodbye";

      global();
 }

 It also prints out "hello", and if a closure had not been allocated,

 have at least half-expected it to print out "goodb", because I'd have thought
 that baz would have been taking up the same memory that bar had been.

Nope. It's just that the stack is intact and contains: hello and goodbye one after another. Without optimizations { } scope doesn't mean reuse stack space. Now if play with stack a bit, for me the next one prints: ­-²↑ import std.stdio; void delegate() global; void foo(scope void delegate() del) { global = del; } void f() { { char[5] bar = "hello"; foo((){writeln(bar);}); } } void main() { char[7] baz = "goodbye"; f(); global(); } -- Dmitry Olshansky
Nov 11 2012
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/11/2012 2:08 PM, Jonathan M Davis пишет:
 On Sunday, November 11, 2012 13:36:05 Dmitry Olshansky wrote:
 Nope. It's just that the stack is intact and contains: hello and goodbye
 one after another. Without optimizations { } scope doesn't mean reuse
 stack space.

 Now if play with stack a bit, for me the next one prints:
 ­-²↑

 import std.stdio;

 void delegate() global;

 void foo(scope void delegate() del)
 {
       global = del;
 }


 void f()
 {
       {
           char[5] bar = "hello";
           foo((){writeln(bar);});
       }
 }

 void main()
 {
       char[7] baz = "goodbye";
       f();

       global();
 }

It still prints "hello", even with full optimations turned on.

I tried with and without optimizations. I get garbage as expected. So, it must be
 allocating a closure in spite of scope. So, it looks to me like scope is just
 completely ignored and does absolutely nothing at this point, unless I'm just
 completely missing something here.

Something must be screwed up. I dunno what, I use near-latest DMD from github and Win32 binaries. For good measure try making stack variables larger if you are on 64bit. Drop in a couple of calls to writeln before and after calling 'f' it should scramble the stack. -- Dmitry Olshansky
Nov 11 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 11, 2012 13:36:05 Dmitry Olshansky wrote:
 Nope. It's just that the stack is intact and contains: hello and good=

 one after another. Without optimizations { } scope doesn't mean reuse=

 stack space.
=20
 Now if play with stack a bit, for me the next one prints:
 =C2=AD-=C2=B2=E2=86=91
=20
 import std.stdio;
=20
 void delegate() global;
=20
 void foo(scope void delegate() del)
 {
      global =3D del;
 }
=20
=20
 void f()
 {
      {
          char[5] bar =3D "hello";
          foo((){writeln(bar);});
      }
 }
=20
 void main()
 {
      char[7] baz =3D "goodbye";
      f();
=20
      global();
 }

It still prints "hello", even with full optimations turned on. So, it m= ust be=20 allocating a closure in spite of scope. So, it looks to me like scope i= s just=20 completely ignored and does absolutely nothing at this point, unless I'= m just=20 completely missing something here. - Jonathan M Davis
Nov 11 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Sunday, 11 November 2012 at 10:09:17 UTC, Jonathan M Davis 
wrote:
 It still prints "hello", even with full optimations turned on. 
 So, it must be
 allocating a closure in spite of scope. So, it looks to me like 
 scope is just
 completely ignored and does absolutely nothing at this point, 
 unless I'm just
 completely missing something here.

Try this: --- import std.stdio; void delegate() global; void foo(scope void delegate() del) { global = del; } void f() { { char[5] bar = "hello"; foo((){writeln(bar);}); } } void smashStack() { uint[1000] dummy = 0xbadcab1e; asm { nop; } } void main() { char[7] baz = "goodbye"; f(); smashStack(); global(); } --- David
Nov 11 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--bcaec51d25d4103be704ce3682d5
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 10 November 2012 21:07, Jonathan M Davis <jmdavisProg gmx.com> wrote:

 On Saturday, November 10, 2012 13:21:42 Manu wrote:
 I'm still not buying this. Here's a common struct I will pass by ref
 (perhaps the most common struct in my industry):

 struct Vector { float, x,y,z,w; }
 struct Matrix { Vector xRow, yRow, zRow, wRow; }

 Vector mul( scope const ref Matrix m, scope const Vector v)
 {
   Vector v;
   // perform a matrix multiply against the vector...
   // this work uses every single field of the inputs given, but the

 it produces has to references to the sources.
   // everything is operated on and copied to the output struct, which i=


 returned.
   return result;
 }

 Why should this be a problem?
 The majority of my work-horse structs apply to this pattern. This is

 imagine 'scope' to be for...
 The main advantage I expect is that I can have confidence that passing
 rvalues (temporaries) is safe, and that external code won't take

 to memory that I may not own/control. Is that not the point?

 Surely the problem that scope should be protecting against is a pointer

 any part of the argument escaping. *Copies* of values contained in the
 argument/s are fine.

Hmmmm. scope on value types is pointless, because there are no references to escape, but if you pass by ref, then it does become possible for a pointe=

 to
 the argument to escape,

Precisely, to me, this seems like the ENTIRE POINT of 'in'? (I originally presumed 'in' implied ref, but I was wrong, 'in ref' is supported however) but I don't know that that's actually actually covered
 by scope. The description for scope in docs is that "ref=C2=ADer=C2=ADenc=

 pa=C2=AD
 ra=C2=ADme=C2=ADter can=C2=ADnot be es=C2=ADcaped (e.g. as=C2=ADsigned to=

 taking the address of a local variable (which is the only way that any
 sort of
 reference to the data could escape) is never  safe anyway.

When did ' safe'ty enter into it? Are you saying that a ref variable is somehow a local variable? It's a local pointer to a foreign variable... and a function can usually operate on that data however it likes. Scope would promise that nothing other than the function I give it to will get its grubby little hands on it. Let's say that function wanted to call through to some other function and pass the variable along (by ref). Obviously, the second function would also have to have it's inputs marked scope, to promise that it never escapes from there. I imagine scope similarly to const, once it goes scope, the whole callstack must maintain the scope property, otherwise the outermost function can't trust it anymore. This makes perfect sense for any function that is likely to receive immediate or local variables by reference (which is extremely common). It also seems absolutely relevant to the rvalues -> ref thing. If you passed in a
 pointer, and scope were fully working, then you'd be protected against th=

 pointer escaping, but passing by ref isn't really the same thing. I'd hav=

 thought that taking the address of a variable passed by ref would fall in=

 pretty much the same camp as taking the address of any other local
 variable,
 which is completely unsafe to escape to the point that I'm not sure that
 there's any point in protecting against it. It's just completely stupid t=

 do
 anyway and is definitely  system. Outside of taking the address of a ref
 parameter, taking the address of a local variable and escpaing it is
 _always_
 going to result in garbage, and ref parameters aren't really references i=

 the
 normal sense, so I don't know.

What do you mean 'aren't really references in the normal sense'? You bring up a good point, but I don't know if it's applicable. Certainly,
 without the ref there (like is the case with the Vector that you're passi=

 in), scope would never do anything, because it doesn't even theoretically
 have
 anything to do. It's purely a value type that's not even being passed by
 ref.

Correct, scope on a purely value type passed by-value means absolutely nothing. scope on a pointer parameter means something; I would expect the pointer its self, nor a pointer INTO anything under the pointer could escape. scope on a by-value parameter that contains pointers (like delegates/slices) has meaning, I presume scope would be transitive like const, apply the pointer rule above. ref is sugar for a pointer, and the above applies verbatim. If you take the address of the ref argument, you have the pointer, and it mustn't escape, likewise, no pointers or pointer to anything beneath it. If my imagination of this concept completely wrong? This sounds useful to me, I can't imagine another scenario where the keyword a) makes sense, and b) is useful... In general though, putting scope on struct parameters would cause a lot of
 problems, because of arrays that they might hold and whatnot. Slices
 wouldn't
 be able to escape (and so copies of the struct wouldn't be able escape
 without
 deep copying, let alone the array itself).

That would be the point though. If you don't want that, then you don't want scope. That said, I would suggest that it may be considered, for practicality, that immutable members WERE allowed to escape from scope controlled structs... I think this makes sense, and is certainly practical. Strings are the most likely thing to fall under the scenario you illustrated above. Deep-copying strings (or any similarly immutable data) would be a bit silly. So, while scope may be very useful
 in some such cases (assuming that it worked), it's not necessarily
 something
 that you'd want as a matter of course.

Of course not. Part of it probably depends on your
 programming style though. If you have a lot of functions that take
 arguments
 and don't return anything that was in them ever, then scope is less of a
 big
 deal, but that's the sort of thing that happens a _lot_ in my experience,
 so
 scope would very quickly become extremely annoying.

 And actually, to make matters worse, I'm not sure that scope on delegates
 is
 working correctly. I thought that it was, but this code compiles:

 import std.stdio;

 void delegate() global;

 void foo(scope void delegate() del)
 {
     global =3D del;
 }

 void main()
 {
     {
         char[5] bar =3D "hello";
         foo((){writeln(bar);});
     }
     char[7] baz =3D "goodbye";

     global();
 }

 It also prints out "hello", and if a closure had not been allocated, I
 would
 have at least half-expected it to print out "goodb", because I'd have
 thought
 that baz would have been taking up the same memory that bar had been. So,
 it
 looks like scope may be completely and utterly broken at this point. I
 don't
 know.

 - Jonathan M Davis

--bcaec51d25d4103be704ce3682d5 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 10 November 2012 21:07, Jonathan M Davis <span dir=3D"ltr">&lt;<a href= =3D"mailto:jmdavisProg gmx.com" target=3D"_blank">jmdavisProg gmx.com</a>&g= t;</span> wrote:<br><div class=3D"gmail_extra"><div class=3D"gmail_quote"><= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex"> <div class=3D"im">On Saturday, November 10, 2012 13:21:42 Manu wrote:<br> &gt; I&#39;m still not buying this. Here&#39;s a common struct I will pass = by ref<br> &gt; (perhaps the most common struct in my industry):<br> &gt;<br> &gt; struct Vector { float, x,y,z,w; }<br> &gt; struct Matrix { Vector xRow, yRow, zRow, wRow; }<br> &gt;<br> &gt; Vector mul( scope const ref Matrix m, scope const Vector v)<br> &gt; {<br> &gt; =C2=A0 Vector v;<br> &gt; =C2=A0 // perform a matrix multiply against the vector...<br> &gt; =C2=A0 // this work uses every single field of the inputs given, but t= he result<br> &gt; it produces has to references to the sources.<br> &gt; =C2=A0 // everything is operated on and copied to the output struct, w= hich is<br> &gt; returned.<br> &gt; =C2=A0 return result;<br> &gt; }<br> &gt;<br> &gt; Why should this be a problem?<br> &gt; The majority of my work-horse structs apply to this pattern. This is w= hat I<br> &gt; imagine &#39;scope&#39; to be for...<br> &gt; The main advantage I expect is that I can have confidence that passing= <br> &gt; rvalues (temporaries) is safe, and that external code won&#39;t take r= eferences<br> &gt; to memory that I may not own/control. Is that not the point?<br> &gt;<br> &gt; Surely the problem that scope should be protecting against is a pointe= r to<br> &gt; any part of the argument escaping. *Copies* of values contained in the= <br> &gt; argument/s are fine.<br> <br> </div>Hmmmm. scope on value types is pointless, because there are no refere= nces to<br> escape, but if you pass by ref, then it does become possible for a pointer = to<br> the argument to escape,</blockquote><div><br></div><div>Precisely, to me, t= his seems like the ENTIRE POINT of &#39;in&#39;? =C2=A0(I originally presum= ed &#39;in&#39; implied ref, but I was wrong, &#39;in ref&#39; is supported= however)</div> <div><br></div><div><br></div><blockquote class=3D"gmail_quote" style=3D"ma= rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> but I don&#39= ;t know that that&#39;s actually actually covered<br> by scope. The description for scope in docs is that &quot;ref=C2=ADer=C2=AD= ences in the pa=C2=AD<br> ra=C2=ADme=C2=ADter can=C2=ADnot be es=C2=ADcaped (e.g. as=C2=ADsigned to a= global vari=C2=ADable).&quot; And<br> taking the address of a local variable (which is the only way that any sort= of<br> reference to the data could escape) is never safe anyway.</blockquote><div=
<br></div><div>When did &#39; safe&#39;ty enter into it? Are you saying th=

a foreign variable... and a function can usually operate on that data howev= er it likes.</div> <div>Scope would promise that nothing other than the function I give it to = will get its grubby little hands on it.</div><div>Let&#39;s say that functi= on wanted to call through to some other function and pass the variable alon= g (by ref). Obviously, the second function would also have to have it&#39;s= inputs marked scope, to promise that it never escapes from there.</div> <div>I imagine scope similarly to const, once it goes scope, the whole call= stack must maintain the scope property, otherwise the outermost function ca= n&#39;t trust it anymore.</div><div><br></div><div>This makes perfect sense= for any function that is likely to receive immediate or local variables by= reference (which is extremely common).</div> <div>It also seems absolutely relevant to the rvalues -&gt; ref thing.</div=
<div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8e=

pointer, and scope were fully working, then you&#39;d be protected against = the<br> pointer escaping, but passing by ref isn&#39;t really the same thing. I&#39= ;d have<br> thought that taking the address of a variable passed by ref would fall into= <br> pretty much the same camp as taking the address of any other local variable= ,<br> which is completely unsafe to escape to the point that I&#39;m not sure tha= t<br> there&#39;s any point in protecting against it. It&#39;s just completely st= upid to do<br> anyway and is definitely system. Outside of taking the address of a ref<br=

s_<br> going to result in garbage, and ref parameters aren&#39;t really references= in the<br> normal sense, so I don&#39;t know.<br></blockquote><div><br></div><div>What= do you mean &#39;aren&#39;t really references in the normal sense&#39;?</d= iv><div><br></div><div><br></div><blockquote class=3D"gmail_quote" style=3D= "margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> You bring up a good point, but I don&#39;t know if it&#39;s applicable. Cer= tainly,<br> without the ref there (like is the case with the Vector that you&#39;re pas= sing<br> in), scope would never do anything, because it doesn&#39;t even theoretical= ly have<br> anything to do. It&#39;s purely a value type that&#39;s not even being pass= ed by ref.<br></blockquote><div><br></div><div>Correct, scope on a purely v= alue type passed by-value means absolutely nothing.</div><div>scope on a po= inter parameter means something; I would expect the pointer its self, nor a= pointer INTO anything under the pointer could escape.</div> <div>scope on a by-value parameter that contains pointers (like delegates/s= lices) has meaning, I presume scope would be transitive like const, apply t= he pointer rule above.</div><div>ref is sugar for a pointer, and the above = applies=C2=A0verbatim. If you take the address of the ref argument, you hav= e the pointer, and it mustn&#39;t escape, likewise, no pointers or pointer = to anything beneath it.</div> <div><br></div><div>If my imagination of this concept completely wrong?</di= v><div>This sounds useful to me, I can&#39;t imagine another scenario where= the keyword a) makes sense, and b) is useful...</div><div><br></div><block= quote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc= solid;padding-left:1ex"> In general though, putting scope on struct parameters would cause a lot of<= br> problems, because of arrays that they might hold and whatnot. Slices wouldn= &#39;t<br> be able to escape (and so copies of the struct wouldn&#39;t be able escape = without<br> deep copying, let alone the array itself).</blockquote><div><br></div><div>= That would be the point though. If you don&#39;t want that, then you don&#3= 9;t want scope.</div><div><br></div><div>That said, I would suggest that it= may be considered, for practicality, that immutable members WERE allowed t= o escape from scope controlled structs...</div> <div>I think this makes sense, and is certainly practical. Strings are the = most likely thing to fall under the scenario you illustrated above. Deep-co= pying strings (or any similarly immutable data) would be a bit silly.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"> So, while scope may be very = useful<br> in some such cases (assuming that it worked), it&#39;s not necessarily some= thing<br> that you&#39;d want as a matter of course.</blockquote><div><br></div><div>= Of course not.</div><div><br></div><blockquote class=3D"gmail_quote" style= =3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Part of = it probably depends on your<br> programming style though. If you have a lot of functions that take argument= s<br> and don&#39;t return anything that was in them ever, then scope is less of = a big<br> deal, but that&#39;s the sort of thing that happens a _lot_ in my experienc= e, so<br> scope would very quickly become extremely annoying.<br> <br> And actually, to make matters worse, I&#39;m not sure that scope on delegat= es is<br> working correctly. I thought that it was, but this code compiles:<br> <br> import std.stdio;<br> <br> void delegate() global;<br> <br> void foo(scope void delegate() del)<br> {<br> =C2=A0 =C2=A0 global =3D del;<br> }<br> <br> void main()<br> {<br> =C2=A0 =C2=A0 {<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 char[5] bar =3D &quot;hello&quot;;<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 foo((){writeln(bar);});<br> =C2=A0 =C2=A0 }<br> =C2=A0 =C2=A0 char[7] baz =3D &quot;goodbye&quot;;<br> <br> =C2=A0 =C2=A0 global();<br> }<br> <br> It also prints out &quot;hello&quot;, and if a closure had not been allocat= ed, I would<br> have at least half-expected it to print out &quot;goodb&quot;, because I&#3= 9;d have thought<br> that baz would have been taking up the same memory that bar had been. So, i= t<br> looks like scope may be completely and utterly broken at this point. I don&= #39;t<br> know.<br> <span class=3D"HOEnZb"><font color=3D"#888888"><br> - Jonathan M Davis<br> </font></span></blockquote></div><br></div> --bcaec51d25d4103be704ce3682d5--
Nov 11 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 11, 2012 12:10:59 David Nadlinger wrote:
 On Sunday, 11 November 2012 at 10:09:17 UTC, Jonathan M Davis
 
 wrote:
 It still prints "hello", even with full optimations turned on.
 So, it must be
 allocating a closure in spite of scope. So, it looks to me like
 scope is just
 completely ignored and does absolutely nothing at this point,
 unless I'm just
 completely missing something here.

Try this: --- import std.stdio; void delegate() global; void foo(scope void delegate() del) { global = del; } void f() { { char[5] bar = "hello"; foo((){writeln(bar);}); } } void smashStack() { uint[1000] dummy = 0xbadcab1e; asm { nop; } } void main() { char[7] baz = "goodbye"; f(); smashStack(); global(); } ---

That did it. And if scope is removed, it works again. I clearly don't get the low level stuff though, since it's just plain bizarre to me that the previous example didn't do it. - Jonathan M Davis
Nov 11 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 11, 2012 15:17:03 Dmitry Olshansky wrote:
 Something must be screwed up. I dunno what, I use near-latest DMD from
 github and Win32 binaries.

I'm on 64-bit Linux, so that may change things a bit. - Jonathan M Davis
Nov 11 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 11, 2012 13:30:12 Manu wrote:
 What do you mean 'aren't really references in the normal sense'?

ref is not really part of a variable's type. It's just a storage class and is really only applicable in very specific circumstances. A function parameter is a local variable, but ref makes it so that it just so happens to affect a variable outside the function as well. The semantics of the function itself or how the parameter is used aren't any different either way. So, if you were to remove ref from the parameter, the function itself would be unaffected semantically-speaking (it would affect the code generation some though). It's the caller whose semantics change. This is in direct contrast with a pointer or reference type where the fact that it refers to something else outside the function is fully part of the type. Taking the address of a local variable is already something that is incredibly unsafe and doesn't work. So, anyone doing it is being an idiot anyway. And if it's _always_ stupid, then protecting it with scope isn't really necessary. pure mostly stops anything like that from happening in the general case, but doesn't prevent people from doing stupid stuff like int* foo() pure { int i; return bar(&i); } int* bar(int* p) pure { return p; } Regardless, my point was that the refness of the parameter isn't really part of the type of the parameter, whereas all other issues with reference escaping that scope would affect _are_ part of the type. So, you're talking about scope protecting against something quite different from what it would protect from in all other circumstances. Protecting against escaping a ref parameter and a reference type escaping are two different (albeit not completely unrelated) things. Honestly, before you brought up the possibility of scope protecting against pointers to ref parameters escaping, it had never occurred to me. I've never seen anyone bring it up before. And as you're talking about protecting against escaping a pointer to a local variable, which is an incredibly stupid thing to do anyway, I'm not sure that the protection is really needed. But I don't know what Walter's or Andrei's intentions were with regards to whether scope would extend to the fact that the parameter is ref. I don't think that it's necessarily the case that it wouldn't, but you're then protecting against escaping a pointer to a local variable (albeit one which affects a variable outside of the function) rather than against escaping a reference type. And from everything I've seen, it's protecting against escaping reference types which was its purpose. Unfortunately, TDPL doesn't seem to discuss scope parameters at all (I thought that it did, but I can't find it anywhere now if it does), meaning that the online docs are the only official documentation on scope, and all other information on it comes from newsgroup discussions on the matter. It's quite possible that many of us have misunderstood aspects of what scope parameters were intended to do. So, assuming that if/when scope actually starts affecting more than delegates as quite a few us think that it's supposed to do, then what you're looking for may very well be on the table, much as I wouldn't have expected it to be. I don't know. Feel free to create an enhancement request for it. Even it wasn't part of the original intention of scope parameters, Walter may think that it's worth making it so that it is.
 That would be the point though. If you don't want that, then you don't want
 scope.

The problem is that in most cases, I really don't think that that is what the average programmer wants, and there _are_ people who use in as a matter course. It would be one thing if you had to explicitly use scope, then I suspect that it wouldn't be used as a matter of course by much of anyone. For the most part, it would then just be used when the programmer knew that that's what they wanted. But with in, so many people seem to really like the concept of it being the opposite of out that they use it as a matter of course without understanding the consequences. - Jonathan M Davis
Nov 11 2012
prev sibling next sibling parent reply luka8088 <luka8088 owave.net> writes:
What about making this a default behavior and introducing a new keyword 
if the function wants to modify the argument but it is not ref (pass by 
value) ? The reason I think that this should be a default behavior 
because not many functions actually modify their arguments and so it 
leaves a lot of space for optimization.

For example:

void f (int x, val int y, ref int z) {
   x = 1; // x is not copied
          // compiler throws an error, x is not passed by value
          // and therefor could not / should not be changed
   y = 2; // ok, y is copied
   z = 3; // ok, z is a reference
}

On 18.10.2012 5:07, Malte Skarupke wrote:
 Hello,

 I realize that this has been discussed before, but so far there is no
 solution and this really needs to be a high priority:

 We need a way for a function to declare that it doesn't want it's
 argument to be copied, but it also doesn't care whether the argument is
 an rvalue or an lvalue.

 The C++ way of doing this would be to declare the argument as a const &.
 Apparently it is not desired that we do the same thing for const ref.

 Currently, if you want that behavior, you have to write 2^n permutations
 of your function, with n being the number of arguments that the function
 takes.

 Here's my attempt at passing a struct to a function that takes three
 arguments without the struct being copied:

 int copyCounter = 0;
 struct CopyCounter
 {
 this(this) { ++copyCounter; }
 }
 void takeThree(ref in CopyCounter a, ref in CopyCounter b, ref in
 CopyCounter c)
 {
 writeln("took three");
 }
 void takeThree(in CopyCounter a, ref in CopyCounter b, ref in
 CopyCounter c)
 {
 takeThree(a, b, c);
 }
 void takeThree(ref in CopyCounter a, in CopyCounter b, ref in
 CopyCounter c)
 {
 takeThree(a, b, c);
 }
 void takeThree(ref in CopyCounter a, ref in CopyCounter b, in
 CopyCounter c)
 {
 takeThree(a, b, c);
 }
 void takeThree(in CopyCounter a, in CopyCounter b, ref in CopyCounter c)
 {
 takeThree(a, b, c);
 }
 void takeThree(in CopyCounter a, ref in CopyCounter b, in CopyCounter c)
 {
 takeThree(a, b, c);
 }
 void takeThree(ref in CopyCounter a, in CopyCounter b, in CopyCounter c)
 {
 takeThree(a, b, c);
 }
 void takeThree(in CopyCounter a, in CopyCounter b, in CopyCounter c)
 {
 takeThree(a, b, c);
 }
 static CopyCounter createCopyCounter()
 {
 return CopyCounter();
 }
 void main()
 {
 CopyCounter first;
 CopyCounter second;
 CopyCounter third;
 takeThree(first, second, third);
 takeThree(createCopyCounter(), second, createCopCounter());
 assert(copyCounter == 0); // yay, works
 }


 My propsed solution is this:
 - Make functions that take "ref in" arguments also accept rvalues.
 - The user can still provide an overload that accepts an rvalue, using
 the "in" keyword, and that one will be preferred over the "ref in" version.


 What do you think?

 Malte

Nov 12 2012
parent reply luka8088 <luka8088 owave.net> writes:
On 13.11.2012 2:16, martin wrote:
 On Monday, 12 November 2012 at 23:38:43 UTC, luka8088 wrote:
 What about making this a default behavior and introducing a new
 keyword if the function wants to modify the argument but it is not ref
 (pass by value) ? The reason I think that this should be a default
 behavior because not many functions actually modify their arguments
 and so it leaves a lot of space for optimization.

 For example:

 void f (int x, val int y, ref int z) {
 x = 1; // x is not copied
 // compiler throws an error, x is not passed by value
 // and therefor could not / should not be changed
 y = 2; // ok, y is copied
 z = 3; // ok, z is a reference
 }

Your proposal isn't really related to this thread's topic, but I

Um, "Const ref and rvalues again", I suggest it to be the default behavior, how is this not related ?
 understand what you mean (although your code comments distract me a bit):

 void f(const int x, int y, ref int z); =>
 void f(int x, val/mutable int y, ref int z);

Yes, you understood correctly: void f (const ref int x, int y, ref int z); => void f (int x, val int y, ref int z); The point here is to make "We need a way for a function to declare that it doesn't want it's argument to be copied, but it also doesn't care whether the argument is an rvalue or an lvalue. " a default behavior.
 I use const/in ;) parameters a lot in my code too to prevent accidental
 modifications, so my function signatures may be more compact by treating
 normal pass-by-value parameters as const if not denoted with a special
 keyword. I guess it wouldn't be very important for optimization though
 because I'd expect the optimizer to detect unchanged parameters. Anyway,
 your proposal would completely break existing code.

Would it ? How many functions actually change their non ref/out arguments ? Can you point out any existing public code that would be broken ?
Nov 13 2012
next sibling parent luka8088 <luka8088 owave.net> writes:
On 13.11.2012 11:00, Era Scarecrow wrote:
 On Tuesday, 13 November 2012 at 08:34:19 UTC, luka8088 wrote:
 Would it ? How many functions actually change their non ref/out
 arguments ? Can you point out any existing public code that would be
 broken ?

It would be possible that if the language became const-preference that a simple regex tool could be made that would do the conversions, thereby any code broken in this way could be un-broken just as easily; But that assumes you aren't using mixins or magic as part of your signatures. Somehow this reminds me a little of when I worked at a company where we were trying out asp as a web server; The whole VB script was by default 'by ref' so you littered all your functions with 'byVal' in order for your behavior to act as you expected. Anyways, my take on this is consistency would be a lot more difficult and annoying unless you had different rules for the signature vs all other references... I doubt you would say 'this is mutable here but immutable here' type of thing. So.... assuming 'mutable' is used, then the following would be comparable... //D as of now int func(int x, int y, const ref int z) { int something; //mutable far more likely int something2; const int lessOften; } //then would become... //if x & y aren't ever changed then mutable may be unneeded. mutable int func(mutable int x, mutable int y, ref int z) { mutable int something; mutable int something2; int lessOften; //const (once set) } //or for inconsistancy.. //mutable or const as a return? (Or either?) //and which would/should you use to reverse it? int func(mutable int x, mutable int y, ref int z) { int something; //mutable int something2; const int lessOften; //const } Seems in a function body you are far more likely to have mutable items, while in the signature you're more likely to have const items; But mixing them or changing how you do it would likely break code very easily if it isn't signature only, but it doesn't seem like a good idea... Now in the above the function may not specify 'x' is const it doesn't guarantees it ever changes it (but it's a local copy so does it matter?), but specifically specifying it may be more clutter than actually useful. All in all it seems like it would have far more confusion (and break code) than help; although having it prefer const versions of functions/methods to non-const ones should probably have a higher priority (Although you then couldn't have a non-const one unless it was as part of the struct/class constness and not the variables, (and ref preferred over non-ref)).

Can you point out any existing public code that would be broken ?
Nov 13 2012
prev sibling parent luka8088 <luka8088 owave.net> writes:
On 13.11.2012 15:07, martin wrote:
 On Tuesday, 13 November 2012 at 08:34:19 UTC, luka8088 wrote:
 Your proposal isn't really related to this thread's topic

Um, "Const ref and rvalues again", I suggest it to be the default behavior, how is this not related ?

The topic here is binding rvalues to (const) ref parameters. You, on the other hand, are suggesting to flip the constness of by-value parameters (int => val/mutable int, const int => int), which affects both rvalues and lvalues (no difference between them) and only by-value parameters.
 Yes, you understood correctly:
 void f (const ref int x, int y, ref int z); =>
 void f (int x, val int y, ref int z);

 The point here is to make "We need a way for a function to declare
 that it doesn't want it's argument to be copied, but it also doesn't
 care whether the argument is an rvalue or an lvalue. " a default
 behavior.

So now tell me why argument x wouldn't be copied. It's passed by value, so of course it is copied (lvalues)/moved (rvalues) just as it is now. The only difference is that the parameter won't be modified by f(). I guess what you furthermore implicate is that you'd expect the compiler to automatically pass appropriate arguments to such parameters by reference to avoid copying (for large structs or structs with non-trivial copy constructors). Such a (handy!) optimization is sadly not possible due to aliasing issues, e.g.: int foo(ref int dst, const int src) { dst = 2*src; return src; } // "optimized" foo(): int bar(ref int dst, const ref int src) { dst = 2*src; return src; } int i = 1; assert(foo(i, i) == 1 && i == 2); // okay i = 1; assert(bar(i, i) == 2 && i == 2); // wtf?! // the const src parameter is actually modified since the // original argument i is also used as mutable dst parameter!
 Would it ? How many functions actually change their non ref/out
 arguments ? Can you point out any existing public code that would be
 broken ?

I don't want to look for examples in Phobos etc. as it should be trivial to imagine cases such as: void bla(float x) { // restrict x to safe range [0,1] x = max(0, min(1, x)); }

I see, you are correct, if it is not copied then it can be changed before function finished through some other references hence it must be copied.
Nov 14 2012
prev sibling next sibling parent "martin" <kinke libero.it> writes:
On Monday, 12 November 2012 at 23:38:43 UTC, luka8088 wrote:
 What about making this a default behavior and introducing a new 
 keyword if the function wants to modify the argument but it is 
 not ref (pass by value) ? The reason I think that this should 
 be a default behavior because not many functions actually 
 modify their arguments and so it leaves a lot of space for 
 optimization.

 For example:

 void f (int x, val int y, ref int z) {
   x = 1; // x is not copied
          // compiler throws an error, x is not passed by value
          // and therefor could not / should not be changed
   y = 2; // ok, y is copied
   z = 3; // ok, z is a reference
 }

Your proposal isn't really related to this thread's topic, but I understand what you mean (although your code comments distract me a bit): void f(const int x, int y, ref int z); => void f(int x, val/mutable int y, ref int z); I use const/in ;) parameters a lot in my code too to prevent accidental modifications, so my function signatures may be more compact by treating normal pass-by-value parameters as const if not denoted with a special keyword. I guess it wouldn't be very important for optimization though because I'd expect the optimizer to detect unchanged parameters. Anyway, your proposal would completely break existing code.
Nov 12 2012
prev sibling next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Tuesday, 13 November 2012 at 08:34:19 UTC, luka8088 wrote:
 Would it ? How many functions actually change their non ref/out 
 arguments ? Can you point out any existing public code that 
 would be broken ?

It would be possible that if the language became const-preference that a simple regex tool could be made that would do the conversions, thereby any code broken in this way could be un-broken just as easily; But that assumes you aren't using mixins or magic as part of your signatures. Somehow this reminds me a little of when I worked at a company where we were trying out asp as a web server; The whole VB script was by default 'by ref' so you littered all your functions with 'byVal' in order for your behavior to act as you expected. Anyways, my take on this is consistency would be a lot more difficult and annoying unless you had different rules for the signature vs all other references... I doubt you would say 'this is mutable here but immutable here' type of thing. So.... assuming 'mutable' is used, then the following would be comparable... //D as of now int func(int x, int y, const ref int z) { int something; //mutable far more likely int something2; const int lessOften; } //then would become... //if x & y aren't ever changed then mutable may be unneeded. mutable int func(mutable int x, mutable int y, ref int z) { mutable int something; mutable int something2; int lessOften; //const (once set) } //or for inconsistancy.. //mutable or const as a return? (Or either?) //and which would/should you use to reverse it? int func(mutable int x, mutable int y, ref int z) { int something; //mutable int something2; const int lessOften; //const } Seems in a function body you are far more likely to have mutable items, while in the signature you're more likely to have const items; But mixing them or changing how you do it would likely break code very easily if it isn't signature only, but it doesn't seem like a good idea... Now in the above the function may not specify 'x' is const it doesn't guarantees it ever changes it (but it's a local copy so does it matter?), but specifically specifying it may be more clutter than actually useful. All in all it seems like it would have far more confusion (and break code) than help; although having it prefer const versions of functions/methods to non-const ones should probably have a higher priority (Although you then couldn't have a non-const one unless it was as part of the struct/class constness and not the variables, (and ref preferred over non-ref)).
Nov 13 2012
prev sibling next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Tuesday, 13 November 2012 at 10:09:27 UTC, luka8088 wrote:
 Can you point out any existing public code that would be broken?

Off hand, no.. I'm not that familiar with a lot of the code or the in depth details of phoboes; However suddenly reversing what is mutable and what is const is bound to break a lot of things even unintentionally. Hmmm.. Does remind me of a bit of my code sometimes... Something went like... class S { void func(S s) { if (!s) s = new S(); //create one if not passed in. //process using S and s (be it new or passed in) } } That would certainly cause a problem.. In my own code example it may empty the pointer/reference if certain requirements were met, and then use if the class reference was set or null for later logic (even if the object referenced to wasn't changed...). Course being passed something like a string... (seems the most likely place you'd find it), in which case using .dup on the input string right back into the variable would be completely understandable. ie: string someStringTransformationWithCOW(string x) { //if COW'd then x = x.dup; //... return x; } Course having the input as char[] rather than string makes more sense for on the fly changes before returning it...
Nov 13 2012
prev sibling parent "martin" <kinke libero.it> writes:
On Tuesday, 13 November 2012 at 08:34:19 UTC, luka8088 wrote:
 Your proposal isn't really related to this thread's topic

Um, "Const ref and rvalues again", I suggest it to be the default behavior, how is this not related ?

The topic here is binding rvalues to (const) ref parameters. You, on the other hand, are suggesting to flip the constness of by-value parameters (int => val/mutable int, const int => int), which affects both rvalues and lvalues (no difference between them) and only by-value parameters.
 Yes, you understood correctly:
 void f (const ref int x, int y, ref int z); =>
 void f (int x, val int y, ref int z);

 The point here is to make "We need a way for a function to 
 declare that it doesn't want it's argument to be copied, but it 
 also doesn't care whether the argument is an rvalue or an 
 lvalue. " a default behavior.

So now tell me why argument x wouldn't be copied. It's passed by value, so of course it is copied (lvalues)/moved (rvalues) just as it is now. The only difference is that the parameter won't be modified by f(). I guess what you furthermore implicate is that you'd expect the compiler to automatically pass appropriate arguments to such parameters by reference to avoid copying (for large structs or structs with non-trivial copy constructors). Such a (handy!) optimization is sadly not possible due to aliasing issues, e.g.: int foo(ref int dst, const int src) { dst = 2*src; return src; } // "optimized" foo(): int bar(ref int dst, const ref int src) { dst = 2*src; return src; } int i = 1; assert(foo(i, i) == 1 && i == 2); // okay i = 1; assert(bar(i, i) == 2 && i == 2); // wtf?! // the const src parameter is actually modified since the // original argument i is also used as mutable dst parameter!
 Would it ? How many functions actually change their non ref/out 
 arguments ? Can you point out any existing public code that 
 would be broken ?

I don't want to look for examples in Phobos etc. as it should be trivial to imagine cases such as: void bla(float x) { // restrict x to safe range [0,1] x = max(0, min(1, x)); }
Nov 13 2012