www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - endless loop with ref and non-ref parameter

reply "Namespace" <rswhite4 googlemail.com> writes:
In relation to this post:
http://forum.dlang.org/thread/hlyospppnjiziyokfvdd forum.dlang.org?page=2#post-qxqvreqpniftjnwxvqgt:40forum.dlang.org

I dicided to test a bit with "manual" auto ref.
This Code works as expected.

[code]
import std.stdio;

struct A {
public:
	
}

void foo(A a) {
	writeln("without ref");
	foo(a);
}

void foo(ref A a) {
	writeln("with ref");
}

void main() {
	foo(A());
}
[/code]

it prints:
without ref
with ref

but if I change
ref A a
to
ref const A a

it turns into an endless loop.

My question is:
Is this behavior intended?
I know "const" is part of the type, but shouldn't recognize the
compiler this case?
Jan 24 2013
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, January 24, 2013 18:24:32 Namespace wrote:
 In relation to this post:
 http://forum.dlang.org/thread/hlyospppnjiziyokfvdd forum.dlang.org?page=2#po
 st-qxqvreqpniftjnwxvqgt:40forum.dlang.org
 
 I dicided to test a bit with "manual" auto ref.
 This Code works as expected.
 
 [code]
 import std.stdio;
 
 struct A {
 public:
 
 }
 
 void foo(A a) {
 writeln("without ref");
 foo(a);
 }
 
 void foo(ref A a) {
 writeln("with ref");
 }
 
 void main() {
 foo(A());
 }
 [/code]
 
 it prints:
 without ref
 with ref
 
 but if I change
 ref A a
 to
 ref const A a
 
 it turns into an endless loop.
 
 My question is:
 Is this behavior intended?
 I know "const" is part of the type, but shouldn't recognize the
 compiler this case?
It's intended. constness matters more than refness when selecting a function overload. From the docs ( http://dlang.org/function.html#<u>function</u>- overloading ): ----- Func­tions are over­loaded based on how well the ar­gu­ments to a func­tion can match up with the pa­ra­me­ters. The func­tion with the best match is se­ lected. The lev­els of match­ing are: no match match with im­plicit con­ver­sions match with con­ver­sion to const exact match ----- So, you need to put const on the parameters for both functions, or you need to create other overloads which do. In general, if you're overloading on ref, make sure that the constness matches, or you're going to get infinite recursion. So, if you're mixing const and ref, then you're probably either going to want 4 different overloads (non-ref, ref, const non-ref, const ref) or 2 overloads which use inout instead of const (inout and inout ref). - Jonathan M Davis
Jan 24 2013
next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 01/24/2013 11:33 AM, Jonathan M Davis wrote:

 It's intended. constness matters more than refness when selecting a 
function
 overload. From the docs ( http://dlang.org/function.html#<u>function</u>-
 overloading ):

 -----
 Functions are overloaded based on how well the arguments to a function
 can match up with the parameters. The function with the best match is se
 lected. The levels of matching are:

 no match
 match with implicit conversions
 match with conversion to const
 exact match
 -----
That doesn't explain why the first case selects the ref function: void foo(A a) { writeln("without ref"); foo(a); // <-- why is this foo(ref A)? } void foo(ref A a) { writeln("with ref"); } foo(A) is the exact match there because the type of a is A. What is also considered in function selection is the rvalue-lvalue distinction, which shouldn't affect the outcome here either. Ali
Jan 24 2013
next sibling parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Thursday, 24 January 2013 at 20:06:34 UTC, Ali Çehreli wrote:
 On 01/24/2013 11:33 AM, Jonathan M Davis wrote:

 It's intended. constness matters more than refness when 
 selecting a function overload. From the docs ( 
 http://dlang.org/function.html#<u>function</u>-overloading ):

 -----
 Functions are overloaded based on how well the arguments to a 
 function can match up with the parameters. The function with 
 the best match is selected. The levels of matching are:

 no match
 match with implicit conversions
 match with conversion to const
 exact match
 -----
That doesn't explain why the first case selects the ref function: void foo(A a) { writeln("without ref"); foo(a); // <-- why is this foo(ref A)? }
Because your local variable (in the signature) becomes a lvalue and thereby reference-able.
 void foo(ref A a) {
     writeln("with ref");
 }

 foo(A) is the exact match there because the type of a is A.
 What is also considered in function selection is the 
 rvalue-lvalue distinction, which shouldn't affect the outcome 
 here either.
In case David's explanation was too confusing, then let's look at it. If your variable is non-const, it will always select a non-const matching one first. If there is no matching non-const, it converts it to const, then tries again. void foo(A a) { writeln("without ref"); //a is an lvalue but not const //Foo(A) matches closer. Infinite loop foo(a); } void foo(const ref A a) { writeln("with ref"); } void foo2(A a) { writeln("foo2 - without ref"); //const lvalue, foo(const ref A) is closer foo(cast(const A) a) //rvalue, postblit call foo(A) (if it can) foo(cast(const A) A()); } I know this very issue seems like a big annoyance, to me too. I'd say the order of preference should be const ref, const non-ref, ref, non-ref. But then you can't have const & non const versions unless the signature differs, like ref, more input variables or the function itself is const. Until/unless that is changed all 4 versions need to be written (as David said) to make the proper calls.
Jan 24 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 01/24/2013 12:25 PM, Era Scarecrow wrote:

 What is also considered in function selection is the rvalue-lvalue
 distinction, which shouldn't affect the outcome here either.
In case David's explanation was too confusing, then let's look at it. If your variable is non-const, it will always select a non-const matching one first. If there is no matching non-const, it converts it to const, then tries again. void foo(A a) { writeln("without ref"); //a is an lvalue but not const //Foo(A) matches closer. Infinite loop
And that's exactly why this issue is so confusing. Normally, const vs. immutable parameters are different in the way that they accept arguments: - immutable is "limiting" because it insists that the argument is immutable. - const is "welcoming" because it accepts mutable, const, and immutable. However, according to your example and explanation above, in this case const is not welcoming but limiting! What the example shows is that, because the following function takes 'const ref A', now it "wants" 'const A' but not 'A'. See, how in this case 'const' is not welcoming? That is the problem in this whole confusing situation. If there is another explanation that would bring sanity to the way I see things, I would love to hear about it. Does my understanging above regarding "limiting" vs. "welcoming" off? Perhaps that's where I go wrong?
 foo(a);
 }

 void foo(const ref A a) {
 writeln("with ref");
 }

 void foo2(A a) {
 writeln("foo2 - without ref");
 //const lvalue, foo(const ref A) is closer
 foo(cast(const A) a)

 //rvalue, postblit call foo(A) (if it can)
 foo(cast(const A) A());
 }
Ali
Jan 24 2013
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, January 24, 2013 17:13:36 Ali Çehreli wrote:
 On 01/24/2013 12:25 PM, Era Scarecrow wrote:
 What is also considered in function selection is the rvalue-lvalue
 distinction, which shouldn't affect the outcome here either.
In case David's explanation was too confusing, then let's look at it. If your variable is non-const, it will always select a non-const matching one first. If there is no matching non-const, it converts it to const, then tries again. void foo(A a) { writeln("without ref"); //a is an lvalue but not const //Foo(A) matches closer. Infinite loop
And that's exactly why this issue is so confusing. Normally, const vs. immutable parameters are different in the way that they accept arguments: - immutable is "limiting" because it insists that the argument is immutable. - const is "welcoming" because it accepts mutable, const, and immutable. However, according to your example and explanation above, in this case const is not welcoming but limiting! What the example shows is that, because the following function takes 'const ref A', now it "wants" 'const A' but not 'A'. See, how in this case 'const' is not welcoming? That is the problem in this whole confusing situation. If there is another explanation that would bring sanity to the way I see things, I would love to hear about it. Does my understanging above regarding "limiting" vs. "welcoming" off? Perhaps that's where I go wrong?
I've never heard anyone describe it that way before. const is more generic, whereas immutable is more specific. But from the compiler's point of view, passing a non-const object to a const function means doing a conversion (from mutable to const), whereas calling a function of the same type does not require a conversion. I believe that _that_ is the core of why ref takes precedence over const. The overload rules avoid converting types as much as possible. So, if you have auto foo(A a) {...} //#1 auto foo(const A a) {...} //#2 A a; const A ca; foo(a); //calls #1 foo(ca); //calls #2 The same goes with ref. When passing an lvalue, the ref requires no conversion or copy, whereas without ref, a copy is required. So, auto foo(A a) {...} //#1 auto foo(ref A a) {...} //#2 A a; foo(a); //calls #2 foo(createA()); //calls #1 The question is what happens when you start mixing const and ref. You have to choose which has precedence. It's obvious what to do when you have all of the combinations auto foo(A a) {...} //#1 auto foo(ref A) {...} #2 auto foo(const A) {...} //#3 auto foo(ref const A) {...} //#4 A a; const A ca; foo(a); //Calls #2 foo(ca); //Calls #4 foo(createA()); //Calls #1 foo(createConstA()); //Calls #3 The problem is what to do when you only have a subset: auto foo(ref A) {...} #2 auto foo(const A) {...} //#3 A a; const A ca; foo(a); //Calls #2 foo(ca); //Calls #3 foo(createA()); //Calls #3 foo(createConstA()); //Calls #3 or auto foo(A a) {...} //#1 auto foo(ref const A) {...} //#4 A a; const A ca; foo(a); //Calls #4 foo(ca); //Calls #4 foo(createA()); //Calls #1 foo(createConstA()); //Calls #1 or etc... The compiler _has_ to pick either const or ref as having higher precedence. ref was almost certainly chosen as having higher precedence because it avoids a conversion, but also by picking ref, you end up with fewer unnecessary copies, making it the more efficient choice. So, from the standpoint of both type conversions and efficiency, it makes more sense for ref to have precedence over const than the other way around. The reality of the matter is that regardless of whether const or ref had precedence, you'd still need all 4 overloads or you'd have problems. auto ref and inout can help reduce the number of overloads required, but the function needs to accept all of the type combinations in order to avoid having to convert the type or make unnecessary copies. - Jonathan M Davis
Jan 24 2013
next sibling parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Friday, 25 January 2013 at 01:57:02 UTC, Jonathan M Davis 
wrote:
 The compiler _has_ to pick either const or ref as having higher 
 precedence. ref was almost certainly chosen as having higher 
 precedence because it avoids a conversion, but also by picking 
 ref, you end up with fewer unnecessary copies, making it the 
 more efficient choice. So, from the standpoint of both type 
 conversions and efficiency, it makes more sense for ref to have 
 precedence over const than the other way around.

 The reality of the matter is that regardless of whether const 
 or ref had precedence, you'd still need all 4 overloads or 
 you'd have problems. auto ref and inout can help reduce the 
 number of overloads required, but the function needs to accept 
 all of the type combinations in order to avoid having to 
 convert the type or make unnecessary copies.
Personally that doesn't seem like it makes sense. I know it does logically, but at the same time it seems silly. If you have something as const, why do you need a non-const version? Example might be if you were to re-write strlen for C, you have a const version but no need for a non-const version as the data will never change. Having multiple ones doing different behavior likely can only get confusing. Say we have the following. ///public interface int strlen(const char[] input); //no public/DDoc interface //does what the above does but also swaps upper/lower case int strlen(char[] input); Now when you change something from const to non-const the behavior changes drastically. Even if the Behavior didn't change, there shouldn't be a need for two versions of it for the same name in order for it to work. There's cases where I've tried to make use of only incorporating a few const/ref differences and yet it fails terribly unless I fill in the other two with unexpected behavior. I see a variable with 'const' and it's I declare 'this variable cannot be changed by me', where as for a function signature it's 'I promise (in good Faith) not to change this'. In these aspects the const should have equal footing as the non-const when determining which to call. Here's a older example although I'll have to redo it. struct S { //move semantics as it's rvalue void opAssign(S s); //If lvalue, we only want to reference it and //make manual updates to this one void opAssign(const ref S s); } In this case the opAssign(S) would always be called unless you made the lvalue (or cast it) as const. This personally seems wrong. If const ref had the highest priority the issue goes away, although you cannot have a non-const ref otherwise with the same signature, but why would you need one? It seems to be causing more issues than it helps, kinda like multiple inheritance. //with above if const ref was highest void opAssign(ref S s); //error, opAssign(const ref S) trumps If auto ref does get accepted, then it will help alleviate some of the problems, but doesn't feel like all of them will be handled.
Jan 24 2013
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, January 25, 2013 05:14:45 Era Scarecrow wrote:
 On Friday, 25 January 2013 at 01:57:02 UTC, Jonathan M Davis
 
 wrote:
 The compiler _has_ to pick either const or ref as having higher
 precedence. ref was almost certainly chosen as having higher
 precedence because it avoids a conversion, but also by picking
 ref, you end up with fewer unnecessary copies, making it the
 more efficient choice. So, from the standpoint of both type
 conversions and efficiency, it makes more sense for ref to have
 precedence over const than the other way around.
 
 The reality of the matter is that regardless of whether const
 or ref had precedence, you'd still need all 4 overloads or
 you'd have problems. auto ref and inout can help reduce the
 number of overloads required, but the function needs to accept
 all of the type combinations in order to avoid having to
 convert the type or make unnecessary copies.
Personally that doesn't seem like it makes sense. I know it does logically, but at the same time it seems silly. If you have something as const, why do you need a non-const version?
With templated code, it can be important. But then again, if there's no point in having a non-const overload, you can simply not declare any overloads without const. You only run into problems when you mix const and non-const. The compiler has to be able to deal with various combinations of const and ref regardless of what it actually makes sense to declare. The only way that I can think of to get rid of that problem is to make it illegal to declare both const and non-const overloads at the same time, which seems unnecessarily restrictive (especially with regards to generic code), even if it doesn't normally make sense to overload on const. - Jonathan M Davis
Jan 24 2013
parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Friday, 25 January 2013 at 04:26:18 UTC, Jonathan M Davis 
wrote:
 With templated code, it can be important. But then again, if 
 there's no point in having a non-const overload, you can simply 
 not declare any overloads without const. You only run into 
 problems when you mix const and non-const.

 The compiler has to be able to deal with various combinations 
 of const and ref regardless of what it actually makes sense to 
 declare. The only way that I can think of to get rid of that 
 problem is to make it illegal to declare both const and 
 non-const overloads at the same time, which seems unnecessarily 
 restrictive (especially with regards to generic code), even if 
 it doesn't normally make sense to overload on const.
True, but still it seems overtly annoying. I noticed most of this from TDPL. pg 257 [quote] The problem is that opAssign as defined expects a ref Widget, that is, an lvalue of type Widget. To accept assignment from rvalue in addition to lvalues, Widget must define two assignment operators: import std.algorithm; struct Widget { private int[] array; ref Widget opAssign(ref Widget rhs) { array = rhs.array.dup; return this; } ref Widget opAssign(Widget rhs) { swap(array, rhs.array); return this; } } There's no more .dup in the version takint an rvalue. Why? Well the rvalue (with it's array in tow) is practically owned by the second opAssign: It was copied prior to entering the function and will be destroyed just before the function returns. This means there's no more need to duplicate rhs.array because nobody will miss it. Swapping rhs.array with this.array is enough. When opAssign returns, rhs goes away with this's old array, and this stays with rhs's old array--perfect conservation of state. We now could remove the first overload of opAssign altogether: The one taking rhs by value takes care of everything (lvalues are automatically converted to rvalues). But keeping the lvalue version allows for a useful optimization: instead of .duping the source, opAssign can check whether the current array has space for accommodating the new contents, in which case an overwrite is enough. [/quote] The whole time I look at the example code, I can see how the ref Widget can be const and continue to work perfectly, but the non-ref cannot be const (without ugly unsafe casting). But if you try to pass const data to opAssign it will either try to copy it (if it's POD or postblit it), or fail outright. In my mind I shouldn't have to double the functions to have it 'do the right thing', even if it's just to forward them. This is not a case where 'auto ref' could help, as lvalue/rvalue distinction needs to be kept.
Jan 24 2013
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 01/24/2013 05:56 PM, Jonathan M Davis wrote:
 On Thursday, January 24, 2013 17:13:36 Ali Çehreli wrote:
 Normally, const vs. immutable parameters are different in the way that
 they accept arguments:

 - immutable is "limiting" because it insists that the argument is 
immutable.
 - const is "welcoming" because it accepts mutable, const, and immutable.
 I've never heard anyone describe it that way before.
We must find easier ways of explaining these semantics. I appreciate the examples that you have provided but we can't expect newcomers to extract meaning out of this. We must be able to come up with a few function design guidelines. My choice of words like "welcoming" and "limiting" are attempts at finding sanity.
 const is more generic,
 whereas immutable is more specific. But from the compiler's point of 
view,
 passing a non-const object to a const function means doing a 
conversion (from
 mutable to const)
That is news to me. There shouldn't be any conversion from non-const to const because that is a conversion that should have no runtime effect. As Era Scarecrow put it, it means "this variable cannot be changed by me". As I understand it, the compiler should merely enforce that guarantee at compile time without performing any conversions.
, whereas calling a function of the same type does not
 require a conversion. I believe that _that_ is the core of why ref takes
 precedence over const.
If const really requires a conversion then we have a language design problem here.
 So, if you have
[...] These examples are great but as I said, they don't help the programmer at function signature stage. immutable is easy: If I am going to need immutable data, then I make the parameter immutable. By-value is also understandable: I don't care about the original. Additionally, when a 'ref' overload exists, by-value gives me the chance to move because only the rvalue can be bound to by-value. Pretty indirect way of thinking but we can deal with it... const should be easy as well: "I am not going to modify". Just like in C++... Unfortunately that is not the case. Ali
Jan 24 2013
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, January 24, 2013 21:38:42 Ali =C3=87ehreli wrote:
 If const really requires a conversion then we have a language design
 problem here.
No. const T is _not_ the same type as T, therefore assigning a T to a c= onst T=20 _is_ a type conversion. _Any_ time that a variable of one type is assig= ned to=20 a variable of another type is a conversion regardless of the types invo= lved.=20 True, the bits of the object won't have changed when converting from T = to=20 const T (assuming that no alias this has been declared for the conversi= on=20 anyway), but it's still a type conversion.
  > So, if you have
=20
 [...]
=20
 These examples are great but as I said, they don't help the programme=
r
 at function signature stage.
=20
 immutable is easy: If I am going to need immutable data, then I make =
the
 parameter immutable.
=20
 By-value is also understandable: I don't care about the original.
 Additionally, when a 'ref' overload exists, by-value gives me the cha=
nce
 to move because only the rvalue can be bound to by-value. Pretty
 indirect way of thinking but we can deal with it...
=20
 const should be easy as well: "I am not going to modify". Just like i=
n
 C++... Unfortunately that is not the case.
I don't see what could be done to make it simpler other than making it = illegal=20 to overload on constness or refness, which would cause other problems. = When=20 you're dealing with a variety of overloads, it's always going to get=20= complicated on some level. Stuff like auto ref and inout help, but it's= =20 complicated by its very nature. Besides, the simple rule of making sure= that=20 either all of your overloads match in terms of constness or that you ha= ve=20 overloads for both const and non-const parameters of matching refness s= olves=20 the problem. And if you screw it up, it's obvious _very_ quickly, since= it=20 causes infinite recursion. - Jonathan M Davis
Jan 24 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, January 24, 2013 12:06:33 Ali Çehreli wrote:
 On 01/24/2013 11:33 AM, Jonathan M Davis wrote:
 It's intended. constness matters more than refness when selecting a
function
 overload. From the docs ( http://dlang.org/function.html#<u>function</u>-
 overloading ):
 
 -----
 Functions are overloaded based on how well the arguments to a function
 can match up with the parameters. The function with the best match is se
 lected. The levels of matching are:
 
 no match
 match with implicit conversions
 match with conversion to const
 exact match
 -----
That doesn't explain why the first case selects the ref function: void foo(A a) { writeln("without ref"); foo(a); // <-- why is this foo(ref A)? } void foo(ref A a) { writeln("with ref"); } foo(A) is the exact match there because the type of a is A. What is also considered in function selection is the rvalue-lvalue distinction, which shouldn't affect the outcome here either.
Whether an argument is an lvalue or rvalue is very important to the issue. That's _exactly_ why the ref overload gets called in the first case. If the argument is an lvalue, the ref overload is selected. If the argument is an rvalue, the non-ref overload is selected. const only enters into it _after_ that. - Jonathan M Davis
Jan 24 2013
prev sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Thursday, 24 January 2013 at 20:06:34 UTC, Ali Çehreli wrote:
 On 01/24/2013 11:33 AM, Jonathan M Davis wrote:

 It's intended. constness matters more than refness when
selecting a function
 overload. From the docs (
http://dlang.org/function.html#<u>function</u>-
 overloading ):

 -----
 Functions are overloaded based on how well the arguments to a
function
 can match up with the parameters. The function with the best
match is se
 lected. The levels of matching are:

 no match
 match with implicit conversions
 match with conversion to const
 exact match
 -----
That doesn't explain why the first case selects the ref function: void foo(A a) { writeln("without ref"); foo(a); // <-- why is this foo(ref A)? } void foo(ref A a) { writeln("with ref"); } foo(A) is the exact match there because the type of a is A. What is also considered in function selection is the rvalue-lvalue distinction, which shouldn't affect the outcome here either. Ali
From http://dlang.org/function.html "If two or more functions have the same match level, then partial ordering is used to try to find the best match. Partial ordering finds the most specialized function. If neither function is more specialized than the other, then it is an ambiguity error. Partial ordering is determined for functions f() and g() by taking the parameter types of f(), constructing a list of arguments by taking the default values of those types, and attempting to match them against g(). If it succeeds, then g() is at least as specialized as f()." It is possible to pass an lvalue to both ref and non-ref parameters, however rvalue cannot be passed to ref-parameter. Hence ref version for lvalue is more specialized and selected.
Jan 24 2013
prev sibling parent reply "Namespace" <rswhite4 googlemail.com> writes:
 no match
 match with im­plicit con­ver­sions
 match with con­ver­sion to const
 exact match
Explain me, why this code prints: Error: A() is not an lvalue [code] import std.stdio; struct A { } void foo(A a, float r) { } void foo(const A a, float r) { } void foo(ref A a, float r) { } void main() { foo(A(), 12); } [/code] Why is the ref A a functions chosen? I create already a pull request but a explanation would be nice.
Jan 27 2013
parent "Namespace" <rswhite4 googlemail.com> writes:
pull request -> bug report. I'm a bit confused today.
Jan 27 2013