www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - endless loop with ref and non-ref parameter

reply "Namespace" <rswhite4 googlemail.com> writes:
In relation to this post:
http://forum.dlang.org/thread/hlyospppnjiziyokfvdd forum.dlang.org?page=2#post-qxqvreqpniftjnwxvqgt:40forum.dlang.org

I dicided to test a bit with "manual" auto ref.
This Code works as expected.

[code]
import std.stdio;

struct A {
public:
	
}

void foo(A a) {
	writeln("without ref");
	foo(a);
}

void foo(ref A a) {
	writeln("with ref");
}

void main() {
	foo(A());
}
[/code]

it prints:
without ref
with ref

but if I change
ref A a
to
ref const A a

it turns into an endless loop.

My question is:
Is this behavior intended?
I know "const" is part of the type, but shouldn't recognize the
compiler this case?
Jan 24 2013
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, January 24, 2013 18:24:32 Namespace wrote:

 In relation to this post:
 http://forum.dlang.org/thread/hlyospppnjiziyokfvdd forum.dlang.org?page=2#po
 st-qxqvreqpniftjnwxvqgt:40forum.dlang.org
 
 I dicided to test a bit with "manual" auto ref.
 This Code works as expected.
 
 [code]
 import std.stdio;
 
 struct A {
 public:
 
 }
 
 void foo(A a) {
 writeln("without ref");
 foo(a);
 }
 
 void foo(ref A a) {
 writeln("with ref");
 }
 
 void main() {
 foo(A());
 }
 [/code]
 
 it prints:
 without ref
 with ref
 
 but if I change
 ref A a
 to
 ref const A a
 
 it turns into an endless loop.
 
 My question is:
 Is this behavior intended?
 I know "const" is part of the type, but shouldn't recognize the
 compiler this case?


It's intended. constness matters more than refness when selecting a function 

overloading ):

-----
Func­tions are over­loaded based on how well the ar­gu­ments to a
func­tion 
can match up with the pa­ra­me­ters. The func­tion with the best match is
se­
lected. The lev­els of match­ing are: 

no match
match with im­plicit con­ver­sions
match with con­ver­sion to const
exact match
-----

So, you need to put const on the parameters for both functions, or you need to 
create other overloads which do. In general, if you're overloading on ref, 
make sure that the constness matches, or you're going to get infinite 
recursion. So, if you're mixing const and ref, then you're probably either 
going to want 4 different overloads (non-ref, ref, const non-ref, const ref) or 
2 overloads which use inout instead of const (inout and inout ref).

- Jonathan M Davis
Jan 24 2013
next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 01/24/2013 11:33 AM, Jonathan M Davis wrote:


 It's intended. constness matters more than refness when selecting a 


function


 overloading ):

 -----
 Functions are overloaded based on how well the arguments to a function
 can match up with the parameters. The function with the best match is se
 lected. The levels of matching are:

 no match
 match with implicit conversions
 match with conversion to const
 exact match
 -----


That doesn't explain why the first case selects the ref function:

void foo(A a) {
     writeln("without ref");
     foo(a);                    // <-- why is this foo(ref A)?
}

void foo(ref A a) {
     writeln("with ref");
}

foo(A) is the exact match there because the type of a is A.

What is also considered in function selection is the rvalue-lvalue 
distinction, which shouldn't affect the outcome here either.

Ali
Jan 24 2013
next sibling parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Thursday, 24 January 2013 at 20:06:34 UTC, Ali Çehreli wrote:

 On 01/24/2013 11:33 AM, Jonathan M Davis wrote:


 It's intended. constness matters more than refness when 
 selecting a function overload. From the docs ( 


 -----
 Functions are overloaded based on how well the arguments to a 
 function can match up with the parameters. The function with 
 the best match is selected. The levels of matching are:

 no match
 match with implicit conversions
 match with conversion to const
 exact match
 -----


 That doesn't explain why the first case selects the ref 
 function:

 void foo(A a) {
     writeln("without ref");
     foo(a);                    // <-- why is this foo(ref A)?
 }


  Because your local variable (in the signature) becomes a lvalue 
and thereby reference-able.


 void foo(ref A a) {
     writeln("with ref");
 }

 foo(A) is the exact match there because the type of a is A.



 What is also considered in function selection is the 
 rvalue-lvalue distinction, which shouldn't affect the outcome 
 here either.


  In case David's explanation was too confusing, then let's look 
at it. If your variable is non-const, it will always select a 
non-const matching one first. If there is no matching non-const, 
it converts it to const, then tries again.

   void foo(A a) {
     writeln("without ref");
     //a is an lvalue but not const
     //Foo(A) matches closer. Infinite loop
     foo(a);
   }

   void foo(const ref A a) {
     writeln("with ref");
   }

   void foo2(A a) {
     writeln("foo2 - without ref");
     //const lvalue, foo(const ref A) is closer
     foo(cast(const A) a)

     //rvalue, postblit call foo(A) (if it can)
     foo(cast(const A) A());
   }


  I know this very issue seems like a big annoyance, to me too. 
I'd say the order of preference should be const ref, const 
non-ref, ref, non-ref. But then you can't have const & non const 
versions unless the signature differs, like ref, more input 
variables or the function itself is const.

  Until/unless that is changed all 4 versions need to be written 
(as David said) to make the proper calls.
Jan 24 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 01/24/2013 12:25 PM, Era Scarecrow wrote:


 What is also considered in function selection is the rvalue-lvalue
 distinction, which shouldn't affect the outcome here either.


 In case David's explanation was too confusing, then let's look at it. If
 your variable is non-const, it will always select a non-const matching
 one first. If there is no matching non-const, it converts it to const,
 then tries again.

 void foo(A a) {
 writeln("without ref");
 //a is an lvalue but not const
 //Foo(A) matches closer. Infinite loop


And that's exactly why this issue is so confusing.

Normally, const vs. immutable parameters are different in the way that 
they accept arguments:

- immutable is "limiting" because it insists that the argument is immutable.

- const is "welcoming" because it accepts mutable, const, and immutable.

However, according to your example and explanation above, in this case 
const is not welcoming but limiting! What the example shows is that, 
because the following function takes 'const ref A', now it "wants" 
'const A' but not 'A'. See, how in this case 'const' is not welcoming?

That is the problem in this whole confusing situation.

If there is another explanation that would bring sanity to the way I see 
things, I would love to hear about it. Does my understanging above 
regarding "limiting" vs. "welcoming" off? Perhaps that's where I go wrong?


 foo(a);
 }

 void foo(const ref A a) {
 writeln("with ref");
 }

 void foo2(A a) {
 writeln("foo2 - without ref");
 //const lvalue, foo(const ref A) is closer
 foo(cast(const A) a)

 //rvalue, postblit call foo(A) (if it can)
 foo(cast(const A) A());
 }


Ali
Jan 24 2013
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, January 24, 2013 17:13:36 Ali Çehreli wrote:

 On 01/24/2013 12:25 PM, Era Scarecrow wrote:

 What is also considered in function selection is the rvalue-lvalue
 distinction, which shouldn't affect the outcome here either.


 
 In case David's explanation was too confusing, then let's look at it. If
 your variable is non-const, it will always select a non-const matching
 one first. If there is no matching non-const, it converts it to const,
 then tries again.
 
 void foo(A a) {
 writeln("without ref");
 //a is an lvalue but not const
 //Foo(A) matches closer. Infinite loop


 
 And that's exactly why this issue is so confusing.
 
 Normally, const vs. immutable parameters are different in the way that
 they accept arguments:
 
 - immutable is "limiting" because it insists that the argument is immutable.
 
 - const is "welcoming" because it accepts mutable, const, and immutable.
 
 However, according to your example and explanation above, in this case
 const is not welcoming but limiting! What the example shows is that,
 because the following function takes 'const ref A', now it "wants"
 'const A' but not 'A'. See, how in this case 'const' is not welcoming?
 
 That is the problem in this whole confusing situation.
 
 If there is another explanation that would bring sanity to the way I see
 things, I would love to hear about it. Does my understanging above
 regarding "limiting" vs. "welcoming" off? Perhaps that's where I go wrong?


I've never heard anyone describe it that way before. const is more generic, 
whereas immutable is more specific. But from the compiler's point of view, 
passing a non-const object to a const function means doing a conversion (from 
mutable to const), whereas calling a function of the same type does not 
require a conversion. I believe that _that_ is the core of why ref takes 
precedence over const.

The overload rules avoid converting types as much as possible. So, if you have




A a;
const A ca;



The same goes with ref. When passing an lvalue, the ref requires no conversion 
or copy, whereas without ref, a copy is required. So,




A a;



The question is what happens when you start mixing const and ref. You have to 
choose which has precedence. It's obvious what to do when you have all of the 
combinations






A a;
const A ca;





The problem is what to do when you only have a subset:




A a;
const A ca;





or




A a;
const A ca;





or etc...

The compiler _has_ to pick either const or ref as having higher precedence. 
ref was almost certainly chosen as having higher precedence because it avoids 
a conversion, but also by picking ref, you end up with fewer unnecessary 
copies, making it the more efficient choice. So, from the standpoint of both 
type conversions and efficiency, it makes more sense for ref to have precedence 
over const than the other way around.

The reality of the matter is that regardless of whether const or ref had 
precedence, you'd still need all 4 overloads or you'd have problems. auto ref 
and inout can help reduce the number of overloads required, but the function 
needs to accept all of the type combinations in order to avoid having to 
convert the type or make unnecessary copies.

- Jonathan M Davis
Jan 24 2013
next sibling parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Friday, 25 January 2013 at 01:57:02 UTC, Jonathan M Davis 
wrote:

 The compiler _has_ to pick either const or ref as having higher 
 precedence. ref was almost certainly chosen as having higher 
 precedence because it avoids a conversion, but also by picking 
 ref, you end up with fewer unnecessary copies, making it the 
 more efficient choice. So, from the standpoint of both type 
 conversions and efficiency, it makes more sense for ref to have 
 precedence over const than the other way around.

 The reality of the matter is that regardless of whether const 
 or ref had precedence, you'd still need all 4 overloads or 
 you'd have problems. auto ref and inout can help reduce the 
 number of overloads required, but the function needs to accept 
 all of the type combinations in order to avoid having to 
 convert the type or make unnecessary copies.


  Personally that doesn't seem like it makes sense. I know it does 
logically, but at the same time it seems silly. If you have 
something as const, why do you need a non-const version?

  Example might be if you were to re-write strlen for C, you have 
a const version but no need for a non-const version as the data 
will never change.  Having multiple ones doing different behavior 
likely can only get confusing. Say we have the following.

   ///public interface
   int strlen(const char[] input);

   //no public/DDoc interface
   //does what the above does but also swaps upper/lower case
   int strlen(char[] input);

  Now when you change something from const to non-const the 
behavior changes drastically. Even if the Behavior didn't change, 
there shouldn't be a need for two versions of it for the same 
name in order for it to work.

  There's cases where I've tried to make use of only incorporating 
a few const/ref differences and yet it fails terribly unless I 
fill in the other two with unexpected behavior. I see a variable 
with 'const' and it's I declare 'this variable cannot be changed 
by me', where as for a function signature it's 'I promise (in 
good Faith) not to change this'. In these aspects the const 
should have equal footing as the non-const when determining which 
to call.

  Here's a older example although I'll have to redo it.

   struct S {
     //move semantics as it's rvalue
     void opAssign(S s);

     //If lvalue, we only want to reference it and
     //make manual updates to this one
     void opAssign(const ref S s);
   }

  In this case the opAssign(S) would always be called unless you 
made the lvalue (or cast it) as const. This personally seems 
wrong. If const ref had the highest priority the issue goes away, 
although you cannot have a non-const ref otherwise with the same 
signature, but why would you need one? It seems to be causing 
more issues than it helps, kinda like multiple inheritance.

     //with above if const ref was highest
     void opAssign(ref S s); //error, opAssign(const ref S) trumps

  If auto ref does get accepted, then it will help alleviate some 
of the problems, but doesn't feel like all of them will be 
handled.
Jan 24 2013
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, January 25, 2013 05:14:45 Era Scarecrow wrote:

 On Friday, 25 January 2013 at 01:57:02 UTC, Jonathan M Davis
 
 wrote:

 The compiler _has_ to pick either const or ref as having higher
 precedence. ref was almost certainly chosen as having higher
 precedence because it avoids a conversion, but also by picking
 ref, you end up with fewer unnecessary copies, making it the
 more efficient choice. So, from the standpoint of both type
 conversions and efficiency, it makes more sense for ref to have
 precedence over const than the other way around.
 
 The reality of the matter is that regardless of whether const
 or ref had precedence, you'd still need all 4 overloads or
 you'd have problems. auto ref and inout can help reduce the
 number of overloads required, but the function needs to accept
 all of the type combinations in order to avoid having to
 convert the type or make unnecessary copies.


 
   Personally that doesn't seem like it makes sense. I know it does
 logically, but at the same time it seems silly. If you have
 something as const, why do you need a non-const version?


With templated code, it can be important. But then again, if there's no point 
in having a non-const overload, you can simply not declare any overloads 
without const. You only run into problems when you mix const and non-const.

The compiler has to be able to deal with various combinations of const and ref 
regardless of what it actually makes sense to declare. The only way that I can 
think of to get rid of that problem is to make it illegal to declare both 
const and non-const overloads at the same time, which seems unnecessarily 
restrictive (especially with regards to generic code), even if it doesn't 
normally make sense to overload on const.

- Jonathan M Davis
Jan 24 2013
parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Friday, 25 January 2013 at 04:26:18 UTC, Jonathan M Davis 
wrote:

 With templated code, it can be important. But then again, if 
 there's no point in having a non-const overload, you can simply 
 not declare any overloads without const. You only run into 
 problems when you mix const and non-const.

 The compiler has to be able to deal with various combinations 
 of const and ref regardless of what it actually makes sense to 
 declare. The only way that I can think of to get rid of that 
 problem is to make it illegal to declare both const and 
 non-const overloads at the same time, which seems unnecessarily 
 restrictive (especially with regards to generic code), even if 
 it doesn't normally make sense to overload on const.


  True, but still it seems overtly annoying. I noticed most of 
this from TDPL. pg 257

[quote]
   The problem is that opAssign as defined expects a ref Widget, 
that is, an lvalue of type Widget. To accept assignment from 
rvalue in addition to lvalues, Widget must define two assignment 
operators:

   import std.algorithm;

   struct Widget {
     private int[] array;
     ref Widget opAssign(ref Widget rhs) {
       array = rhs.array.dup;
       return this;
     }
     ref Widget opAssign(Widget rhs) {
       swap(array, rhs.array);
       return this;
     }
   }

  There's no more .dup in the version takint an rvalue. Why? Well 
the rvalue (with it's array in tow) is practically owned by the 
second opAssign: It was copied prior to entering the function and 
will be destroyed just before the function returns. This means 
there's no more need to duplicate rhs.array because nobody will 
miss it. Swapping rhs.array with this.array is enough. When 
opAssign returns, rhs goes away with this's old array, and this 
stays with rhs's old array--perfect conservation of state.

  We now could remove the first overload of opAssign altogether: 
The one taking rhs by value takes care of everything (lvalues are 
automatically converted to rvalues). But keeping the lvalue 
version allows for a useful optimization: instead of .duping the 
source, opAssign can check whether the current array has space 
for accommodating the new contents, in which case an overwrite is 
enough.
[/quote]

  The whole time I look at the example code, I can see how the ref 
Widget can be const and continue to work perfectly, but the 
non-ref cannot be const (without ugly unsafe casting). But if you 
try to pass const data to opAssign it will either try to copy it 
(if it's POD or postblit it), or fail outright. In my mind I 
shouldn't have to double the functions to have it 'do the right 
thing', even if it's just to forward them. This is not a case 
where 'auto ref' could help, as lvalue/rvalue distinction needs 
to be kept.
Jan 24 2013
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 01/24/2013 05:56 PM, Jonathan M Davis wrote:

 On Thursday, January 24, 2013 17:13:36 Ali Çehreli wrote:



 Normally, const vs. immutable parameters are different in the way that
 they accept arguments:

 - immutable is "limiting" because it insists that the argument is 




immutable.

 - const is "welcoming" because it accepts mutable, const, and immutable.





 I've never heard anyone describe it that way before.


We must find easier ways of explaining these semantics. I appreciate the 
examples that you have provided but we can't expect newcomers to extract 
meaning out of this.

We must be able to come up with a few function design guidelines. My 
choice of words like "welcoming" and "limiting" are attempts at finding 
sanity.


 const is more generic,
 whereas immutable is more specific. But from the compiler's point of 


view,

 passing a non-const object to a const function means doing a 


conversion (from

 mutable to const)


That is news to me. There shouldn't be any conversion from non-const to 
const because that is a conversion that should have no runtime effect. 
As Era Scarecrow put it, it means "this variable cannot be changed by 
me". As I understand it, the compiler should merely enforce that 
guarantee at compile time without performing any conversions.


, whereas calling a function of the same type does not
 require a conversion. I believe that _that_ is the core of why ref takes
 precedence over const.


If const really requires a conversion then we have a language design 
problem here.


 So, if you have


[...]

These examples are great but as I said, they don't help the programmer 
at function signature stage.

immutable is easy: If I am going to need immutable data, then I make the 
parameter immutable.

By-value is also understandable: I don't care about the original. 
Additionally, when a 'ref' overload exists, by-value gives me the chance 
to move because only the rvalue can be bound to by-value. Pretty 
indirect way of thinking but we can deal with it...

const should be easy as well: "I am not going to modify". Just like in 
C++... Unfortunately that is not the case.

Ali
Jan 24 2013
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, January 24, 2013 21:38:42 Ali =C3=87ehreli wrote:

 If const really requires a conversion then we have a language design
 problem here.


No. const T is _not_ the same type as T, therefore assigning a T to a c=
onst T=20
_is_ a type conversion. _Any_ time that a variable of one type is assig=
ned to=20
a variable of another type is a conversion regardless of the types invo=
lved.=20
True, the bits of the object won't have changed when converting from T =
to=20
const T (assuming that no alias this has been declared for the conversi=
on=20
anyway), but it's still a type conversion.


  > So, if you have
=20
 [...]
=20
 These examples are great but as I said, they don't help the programme=


r

 at function signature stage.
=20
 immutable is easy: If I am going to need immutable data, then I make =


the

 parameter immutable.
=20
 By-value is also understandable: I don't care about the original.
 Additionally, when a 'ref' overload exists, by-value gives me the cha=


nce

 to move because only the rvalue can be bound to by-value. Pretty
 indirect way of thinking but we can deal with it...
=20
 const should be easy as well: "I am not going to modify". Just like i=


n

 C++... Unfortunately that is not the case.


I don't see what could be done to make it simpler other than making it =
illegal=20
to overload on constness or refness, which would cause other problems. =
When=20
you're dealing with a variety of overloads, it's always going to get=20=

complicated on some level. Stuff like auto ref and inout help, but it's=
=20
complicated by its very nature. Besides, the simple rule of making sure=
 that=20
either all of your overloads match in terms of constness or that you ha=
ve=20
overloads for both const and non-const parameters of matching refness s=
olves=20
the problem. And if you screw it up, it's obvious _very_ quickly, since=
 it=20
causes infinite recursion.

- Jonathan M Davis
Jan 24 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, January 24, 2013 12:06:33 Ali Çehreli wrote:

 On 01/24/2013 11:33 AM, Jonathan M Davis wrote:

 It's intended. constness matters more than refness when selecting a


 
 function
 


 overloading ):
 
 -----
 Functions are overloaded based on how well the arguments to a function
 can match up with the parameters. The function with the best match is se
 lected. The levels of matching are:
 
 no match
 match with implicit conversions
 match with conversion to const
 exact match
 -----


 
 That doesn't explain why the first case selects the ref function:
 
 void foo(A a) {
 writeln("without ref");
 foo(a); // <-- why is this foo(ref A)?
 }
 
 void foo(ref A a) {
 writeln("with ref");
 }
 
 foo(A) is the exact match there because the type of a is A.
 
 What is also considered in function selection is the rvalue-lvalue
 distinction, which shouldn't affect the outcome here either.


Whether an argument is an lvalue or rvalue is very important to the issue. 
That's _exactly_ why the ref overload gets called in the first case. If the 
argument is an lvalue, the ref overload is selected. If the argument is an 
rvalue, the non-ref overload is selected. const only enters into it _after_ 
that.

- Jonathan M Davis
Jan 24 2013
prev sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Thursday, 24 January 2013 at 20:06:34 UTC, Ali Çehreli wrote:

 On 01/24/2013 11:33 AM, Jonathan M Davis wrote:


 It's intended. constness matters more than refness when


 selecting a function

 overload. From the docs (




 overloading ):

 -----
 Functions are overloaded based on how well the arguments to a


 function

 can match up with the parameters. The function with the best


 match is se

 lected. The levels of matching are:

 no match
 match with implicit conversions
 match with conversion to const
 exact match
 -----


 That doesn't explain why the first case selects the ref 
 function:

 void foo(A a) {
     writeln("without ref");
     foo(a);                    // <-- why is this foo(ref A)?
 }

 void foo(ref A a) {
     writeln("with ref");
 }

 foo(A) is the exact match there because the type of a is A.

 What is also considered in function selection is the 
 rvalue-lvalue distinction, which shouldn't affect the outcome 
 here either.

 Ali


 From http://dlang.org/function.html

"If two or more functions have the same match level, then partial 
ordering is used to try to find the best match. Partial ordering 
finds the most specialized function. If neither function is more 
specialized than the other, then it is an ambiguity error. 
Partial ordering is determined for functions f() and g() by 
taking the parameter types of f(), constructing a list of 
arguments by taking the default values of those types, and 
attempting to match them against g(). If it succeeds, then g() is 
at least as specialized as f()."

It is possible to pass an lvalue to both ref and non-ref 
parameters, however rvalue cannot be passed to ref-parameter. 
Hence ref version for lvalue is more specialized and selected.
Jan 24 2013
prev sibling parent reply "Namespace" <rswhite4 googlemail.com> writes:
 no match
 match with im­plicit con­ver­sions
 match with con­ver­sion to const
 exact match


Explain me, why this code prints:

Error: A() is not an lvalue

[code]
import std.stdio;

struct A { }

void foo(A a, float r) {
	
}

void foo(const A a, float r) {

}

void foo(ref A a, float r) {
	
}

void main()
{
    foo(A(), 12);
}
[/code]

Why is the ref A a functions chosen?
I create already a pull request but a explanation would be nice.
Jan 27 2013
parent "Namespace" <rswhite4 googlemail.com> writes:
pull request -> bug report. I'm a bit confused today.
Jan 27 2013