www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - return by auto ref horribly broken ?

reply "monarch_dodra" <monarchdodra gmail.com> writes:
I think I'm opening a can of worms here, in regards to inferring 
the escape of references, but a quick investigation showed me 
that return by auto-ref is horribly broken.

Basically, the only thing it does is check if the very last value 
it returns is a ref, but a ref to what? The possibilities of 
returning a ref to a local are HUGE. For example, simple 
returning the index of a tuple, or of a static array, and you're 
in it deep:

//----
import std.typecons;

auto ref foo(T)(auto ref T t)
{
     return t[0];
}

void main()
{
     int* p = &foo(tuple(1, 2));
}
//----

Here, both foo will return a ref to a local. But the compiler 
won't see, and more importantly, it gets blind sided because it 
*can't* see it (AFAIK).

These are trivial examples, but imagine what happens if you start 
mixing in templates, in particular, things like 
std.algorithm.map, that returns its front by auto-ref.
This is combined with an improvement I'm trying to make to 
*aryFun, to infer return type reference of. The two *do*not* mix. 
At all.

This investigation leads me to believe that using auto ref in 
anything but trivial template code.

--------

There have been lots of talks about such problems recently in 
regards to safe code and escaping references, but I just realized 
that auto-ref is just as vulnerable to the problem as is safe 
code.

There were talks about safe code banning the taking and/or 
passing/returning references if the compiler couldn't prove this 
was safe. I'd suggest, staying on the side of safety, we also 
apply this to auto ref, and ban it from returning a ref if it 
can't *prove* the object returned is not a local.

Am I wrong in my analysis? More importantly, how would this mix 
with DIP 25 (http://wiki.dlang.org/DIP25) ?
Feb 18 2013
next sibling parent reply "Zach the Mystic" <reachBUTMINUSTHISzach gOOGLYmail.com> writes:
On Monday, 18 February 2013 at 11:10:38 UTC, monarch_dodra wrote:

 I think I'm opening a can of worms here, in regards to 
 inferring the escape of references, but a quick investigation 
 showed me that return by auto-ref is horribly broken.

 Basically, the only thing it does is check if the very last 
 value it returns is a ref, but a ref to what? The possibilities 
 of returning a ref to a local are HUGE. For example, simple 
 returning the index of a tuple, or of a static array, and 
 you're in it deep:

 //----
 import std.typecons;

 auto ref foo(T)(auto ref T t)
 {
     return t[0];
 }

 void main()
 {
     int* p = &foo(tuple(1, 2));
 }
 //----

 Here, both foo will return a ref to a local. But the compiler 
 won't see, and more importantly, it gets blind sided because it 
 *can't* see it (AFAIK).


If you take the address of a value returning type, you must 
either ban doing it outright or treat the assigned pointer as 
dangerous. To take the address of a value type returned from the 
stack is especially dangerous - I can see banning it outright and 
I don't know what the spec currently says about this. My 
assumption would be that the only legal version of this would be 
the one which returns 'ref'. But tuple(1,2) is an rvalue struct 
type if I'm not mistaken, which means it would be passed as a 
value. The compiler should not allowed a type passed as a value 
(or any part of that value) to be returned as a reference, right? 
So I don't see a way to take the address of this result legally. 
I don't think it should return a reference at all with 
'tuple(1,2)'. That's all I know.
Feb 18 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 18 February 2013 at 21:32:13 UTC, Zach the Mystic 
wrote:

 On Monday, 18 February 2013 at 11:10:38 UTC, monarch_dodra 
 wrote:

 I think I'm opening a can of worms here, in regards to 
 inferring the escape of references, but a quick investigation 
 showed me that return by auto-ref is horribly broken.

 Basically, the only thing it does is check if the very last 
 value it returns is a ref, but a ref to what? The 
 possibilities of returning a ref to a local are HUGE. For 
 example, simple returning the index of a tuple, or of a static 
 array, and you're in it deep:

 //----
 import std.typecons;

 auto ref foo(T)(auto ref T t)
 {
    return t[0];
 }

 void main()
 {
    int* p = &foo(tuple(1, 2));
 }
 //----

 Here, both foo will return a ref to a local. But the compiler 
 won't see, and more importantly, it gets blind sided because 
 it *can't* see it (AFAIK).


 If you take the address of a value returning type, you must 
 either ban doing it outright or treat the assigned pointer as 
 dangerous. To take the address of a value type returned from 
 the stack is especially dangerous - I can see banning it 
 outright and I don't know what the spec currently says about 
 this.


What I wanted to show was that since the code compiled, foo 
returned by ref. At this point, the assigned pointer shouldn't 
even be considered as "dangerous", since we are already in 
undefined behavior.

I could have replaced the code with:
"int a = foo(tuple(1, 2));"

The bug would have been less obvious, but there are chances this 
creates a (very) hard to catch bug.


 My assumption would be that the only legal version of this 
 would be the one which returns 'ref'. But tuple(1,2) is an 
 rvalue struct type if I'm not mistaken, which means it would be 
 passed as a value. The compiler should not allowed a type 
 passed as a value (or any part of that value) to be returned as 
 a reference, right? So I don't see a way to take the address of 
 this result legally. I don't think it should return a reference 
 at all with 'tuple(1,2)'. That's all I know.


Indeed, there is no way to take the address of the returned value 
in this case, since it shouldn't return by ref.

But it does...
Feb 19 2013
parent "Zach the Mystic" <reachBUTMINUSTHISzach gOOGLYmail.com> writes:
On Tuesday, 19 February 2013 at 10:16:47 UTC, monarch_dodra wrote:

 On Monday, 18 February 2013 at 21:32:13 UTC, Zach the Mystic 
 wrote:

 On Monday, 18 February 2013 at 11:10:38 UTC, monarch_dodra 
 wrote:

 I think I'm opening a can of worms here, in regards to 
 inferring the escape of references, but a quick investigation 
 showed me that return by auto-ref is horribly broken.

 Basically, the only thing it does is check if the very last 
 value it returns is a ref, but a ref to what? The 
 possibilities of returning a ref to a local are HUGE. For 
 example, simple returning the index of a tuple, or of a 
 static array, and you're in it deep:

 //----
 import std.typecons;

 auto ref foo(T)(auto ref T t)
 {
   return t[0];
 }

 void main()
 {
   int* p = &foo(tuple(1, 2));
 }
 //----

 Here, both foo will return a ref to a local. But the compiler 
 won't see, and more importantly, it gets blind sided because 
 it *can't* see it (AFAIK).


 If you take the address of a value returning type, you must 
 either ban doing it outright or treat the assigned pointer as 
 dangerous. To take the address of a value type returned from 
 the stack is especially dangerous - I can see banning it 
 outright and I don't know what the spec currently says about 
 this.


 What I wanted to show was that since the code compiled, foo 
 returned by ref. At this point, the assigned pointer shouldn't 
 even be considered as "dangerous", since we are already in 
 undefined behavior.


Well, "extra dangerous" then, undefined, and probably should be 
detected and made illegal.


 I could have replaced the code with:
 "int a = foo(tuple(1, 2));"

 The bug would have been less obvious, but there are chances 
 this creates a (very) hard to catch bug.


Well, my guess would be that this is actually safe, because a is 
assigned by value here and not by reference. (Unless you're 
saying that foo ends up smashing tuple(1,2)'s location, but yeah, 
this is referring to a part of the stack which should be 
considered 'void'.)


 My assumption would be that the only legal version of this 
 would be the one which returns 'ref'. But tuple(1,2) is an 
 rvalue struct type if I'm not mistaken, which means it would 
 be passed as a value. The compiler should not allowed a type 
 passed as a value (or any part of that value) to be returned 
 as a reference, right? So I don't see a way to take the 
 address of this result legally. I don't think it should return 
 a reference at all with 'tuple(1,2)'. That's all I know.


 Indeed, there is no way to take the address of the returned 
 value in this case, since it shouldn't return by ref.

 But it does...


I will start by assuming it's a bug and not a problem with 
language design. There are clear points at which it cannot be 
justified to be considered legal, from my perspective. I'll file 
it as a bug.

http://d.puremagic.com/issues/show_bug.cgi?id=9537

My guess is that 'foo' doesn't realize that 't[0]' is a reference 
derived from a local parameter.
Feb 19 2013
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, February 18, 2013 12:10:37 monarch_dodra wrote:

 Am I wrong in my analysis? More importantly, how would this mix
 with DIP 25 (http://wiki.dlang.org/DIP25) ?


I think that auto ref falls in pretty much exactly the same camp that ref 
does. It's not really any different except that under some circumstances, it 
ends up being ref, and in some it doesn't. And at minimum, as long as it's 
ref, it's going to need to follow exactly the same restrictions that normal 
ref would per DIP 25, and it may be that it needs to follow the same 
restrictions regardless (since if it didn't, then you could end up with a 
function that worked as long as the return type wasn't ref but failed to 
compile if it were, in which case, you might as well have just used auto).

- Jonathan M Davis
Feb 20 2013