digitalmars.D - return by auto ref horribly broken ?
- "monarch_dodra" <monarchdodra gmail.com> Feb 18 2013
- "Zach the Mystic" <reachBUTMINUSTHISzach gOOGLYmail.com> Feb 18 2013
- "monarch_dodra" <monarchdodra gmail.com> Feb 19 2013
- "Zach the Mystic" <reachBUTMINUSTHISzach gOOGLYmail.com> Feb 19 2013
- "Jonathan M Davis" <jmdavisProg gmx.com> Feb 20 2013
I think I'm opening a can of worms here, in regards to inferring
the escape of references, but a quick investigation showed me
that return by auto-ref is horribly broken.
Basically, the only thing it does is check if the very last value
it returns is a ref, but a ref to what? The possibilities of
returning a ref to a local are HUGE. For example, simple
returning the index of a tuple, or of a static array, and you're
in it deep:
//----
import std.typecons;
auto ref foo(T)(auto ref T t)
{
return t[0];
}
void main()
{
int* p = &foo(tuple(1, 2));
}
//----
Here, both foo will return a ref to a local. But the compiler
won't see, and more importantly, it gets blind sided because it
*can't* see it (AFAIK).
These are trivial examples, but imagine what happens if you start
mixing in templates, in particular, things like
std.algorithm.map, that returns its front by auto-ref.
This is combined with an improvement I'm trying to make to
*aryFun, to infer return type reference of. The two *do*not* mix.
At all.
This investigation leads me to believe that using auto ref in
anything but trivial template code.
--------
There have been lots of talks about such problems recently in
regards to safe code and escaping references, but I just realized
that auto-ref is just as vulnerable to the problem as is safe
code.
There were talks about safe code banning the taking and/or
passing/returning references if the compiler couldn't prove this
was safe. I'd suggest, staying on the side of safety, we also
apply this to auto ref, and ban it from returning a ref if it
can't *prove* the object returned is not a local.
Am I wrong in my analysis? More importantly, how would this mix
with DIP 25 (http://wiki.dlang.org/DIP25) ?
Feb 18 2013
"Zach the Mystic" <reachBUTMINUSTHISzach gOOGLYmail.com> On Monday, 18 February 2013 at 11:10:38 UTC, monarch_dodra wrote:I think I'm opening a can of worms here, in regards to inferring the escape of references, but a quick investigation showed me that return by auto-ref is horribly broken. Basically, the only thing it does is check if the very last value it returns is a ref, but a ref to what? The possibilities of returning a ref to a local are HUGE. For example, simple returning the index of a tuple, or of a static array, and you're in it deep: //---- import std.typecons; auto ref foo(T)(auto ref T t) { return t[0]; } void main() { int* p = &foo(tuple(1, 2)); } //---- Here, both foo will return a ref to a local. But the compiler won't see, and more importantly, it gets blind sided because it *can't* see it (AFAIK).
If you take the address of a value returning type, you must either ban doing it outright or treat the assigned pointer as dangerous. To take the address of a value type returned from the stack is especially dangerous - I can see banning it outright and I don't know what the spec currently says about this. My assumption would be that the only legal version of this would be the one which returns 'ref'. But tuple(1,2) is an rvalue struct type if I'm not mistaken, which means it would be passed as a value. The compiler should not allowed a type passed as a value (or any part of that value) to be returned as a reference, right? So I don't see a way to take the address of this result legally. I don't think it should return a reference at all with 'tuple(1,2)'. That's all I know.
Feb 18 2013
On Monday, 18 February 2013 at 21:32:13 UTC, Zach the Mystic wrote:On Monday, 18 February 2013 at 11:10:38 UTC, monarch_dodra wrote:I think I'm opening a can of worms here, in regards to inferring the escape of references, but a quick investigation showed me that return by auto-ref is horribly broken. Basically, the only thing it does is check if the very last value it returns is a ref, but a ref to what? The possibilities of returning a ref to a local are HUGE. For example, simple returning the index of a tuple, or of a static array, and you're in it deep: //---- import std.typecons; auto ref foo(T)(auto ref T t) { return t[0]; } void main() { int* p = &foo(tuple(1, 2)); } //---- Here, both foo will return a ref to a local. But the compiler won't see, and more importantly, it gets blind sided because it *can't* see it (AFAIK).
If you take the address of a value returning type, you must either ban doing it outright or treat the assigned pointer as dangerous. To take the address of a value type returned from the stack is especially dangerous - I can see banning it outright and I don't know what the spec currently says about this.
What I wanted to show was that since the code compiled, foo returned by ref. At this point, the assigned pointer shouldn't even be considered as "dangerous", since we are already in undefined behavior. I could have replaced the code with: "int a = foo(tuple(1, 2));" The bug would have been less obvious, but there are chances this creates a (very) hard to catch bug.My assumption would be that the only legal version of this would be the one which returns 'ref'. But tuple(1,2) is an rvalue struct type if I'm not mistaken, which means it would be passed as a value. The compiler should not allowed a type passed as a value (or any part of that value) to be returned as a reference, right? So I don't see a way to take the address of this result legally. I don't think it should return a reference at all with 'tuple(1,2)'. That's all I know.
Indeed, there is no way to take the address of the returned value in this case, since it shouldn't return by ref. But it does...
Feb 19 2013
On Tuesday, 19 February 2013 at 10:16:47 UTC, monarch_dodra wrote:On Monday, 18 February 2013 at 21:32:13 UTC, Zach the Mystic wrote:On Monday, 18 February 2013 at 11:10:38 UTC, monarch_dodra wrote:I think I'm opening a can of worms here, in regards to inferring the escape of references, but a quick investigation showed me that return by auto-ref is horribly broken. Basically, the only thing it does is check if the very last value it returns is a ref, but a ref to what? The possibilities of returning a ref to a local are HUGE. For example, simple returning the index of a tuple, or of a static array, and you're in it deep: //---- import std.typecons; auto ref foo(T)(auto ref T t) { return t[0]; } void main() { int* p = &foo(tuple(1, 2)); } //---- Here, both foo will return a ref to a local. But the compiler won't see, and more importantly, it gets blind sided because it *can't* see it (AFAIK).
If you take the address of a value returning type, you must either ban doing it outright or treat the assigned pointer as dangerous. To take the address of a value type returned from the stack is especially dangerous - I can see banning it outright and I don't know what the spec currently says about this.
What I wanted to show was that since the code compiled, foo returned by ref. At this point, the assigned pointer shouldn't even be considered as "dangerous", since we are already in undefined behavior.
Well, "extra dangerous" then, undefined, and probably should be detected and made illegal.I could have replaced the code with: "int a = foo(tuple(1, 2));" The bug would have been less obvious, but there are chances this creates a (very) hard to catch bug.
Well, my guess would be that this is actually safe, because a is assigned by value here and not by reference. (Unless you're saying that foo ends up smashing tuple(1,2)'s location, but yeah, this is referring to a part of the stack which should be considered 'void'.)My assumption would be that the only legal version of this would be the one which returns 'ref'. But tuple(1,2) is an rvalue struct type if I'm not mistaken, which means it would be passed as a value. The compiler should not allowed a type passed as a value (or any part of that value) to be returned as a reference, right? So I don't see a way to take the address of this result legally. I don't think it should return a reference at all with 'tuple(1,2)'. That's all I know.
Indeed, there is no way to take the address of the returned value in this case, since it shouldn't return by ref. But it does...
I will start by assuming it's a bug and not a problem with language design. There are clear points at which it cannot be justified to be considered legal, from my perspective. I'll file it as a bug. http://d.puremagic.com/issues/show_bug.cgi?id=9537 My guess is that 'foo' doesn't realize that 't[0]' is a reference derived from a local parameter.
Feb 19 2013
On Monday, February 18, 2013 12:10:37 monarch_dodra wrote:Am I wrong in my analysis? More importantly, how would this mix with DIP 25 (http://wiki.dlang.org/DIP25) ?
I think that auto ref falls in pretty much exactly the same camp that ref does. It's not really any different except that under some circumstances, it ends up being ref, and in some it doesn't. And at minimum, as long as it's ref, it's going to need to follow exactly the same restrictions that normal ref would per DIP 25, and it may be that it needs to follow the same restrictions regardless (since if it didn't, then you could end up with a function that worked as long as the return type wasn't ref but failed to compile if it were, in which case, you might as well have just used auto). - Jonathan M Davis
Feb 20 2013