www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - `return ref`, DIP25, and struct/class lifetimes

reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
Hello all,

Consider a struct that wraps a pointer to some other piece of 
data as follows:

     struct MyWrapper(T)
     {
         private T* data;

         public this(ref T input)
         {
             this.data = &input;
         }

         ... other methods that use `this.data` ...
     }

Is there any way to guarantee at compile time that the input data 
will outlive the wrapper struct?

I'd assumed (dangerous thing to do...) that DIP25 would allow 
this to be guaranteed by `return ref`, but compiling/running the 
following program, with or without the --dip25 flag, would appear 
to suggest otherwise:

////////////////////////////////////////////////////////////

struct MyWrapper(T)
{
     private T* data;

     public this(return ref T input)
     {
         this.data = &input;
     }

     public T get() return
     {
         return *(this.data);
     }

     invariant()
     {
         assert(this.data !is null);
     }
}

auto badWrapper()
{
     double x = 5.0;
     return MyWrapper!double(x);
}

void main()
{
     import std.stdio;
     auto badWrap = badWrapper();
     writeln(badWrap.get());
}

////////////////////////////////////////////////////////////

Is there any current way to achieve what I'm looking for here, or 
is this all on a hiding to nothing? :-(

N.B. for motivation behind this request, see:
https://github.com/WebDrake/dxorshift/pull/1
May 16 2016
parent reply Dicebot <public dicebot.lv> writes:
tl; dr: DIP25 is so heavily under-implemented in its current 
shape it can be considered 100% broken and experimenting will 
uncover even more glaring holes.

To be more precise, judging by experimental observation, 
currently dip25 only works when there is explicitly a `ref` 
return value in function or method. Any escaping of reference 
into a pointer confuses it completely:

ref int wrap ( return ref int input )
{
     return input;
}

ref int badWrapper()
{
     int x = 5;
     return wrap(x); // Error: escaping reference to local 
variable x
}

void main()
{
     auto badWrap = badWrapper();
}

vs

struct S
{
     int* ptr;
}

S wrap ( return ref int input )
{
     return S(&input);
}

S badWrapper()
{
     int x = 5;
     return wrap(x); // OK!
}

void main()
{
     auto badWrap = badWrapper();
}

vs

struct S
{
     int* ptr;
}

ref S wrap ( return ref int input )
{
     static S tmp; // spot another hole :)
     tmp = S(&input);
     return tmp;
}

ref S badWrapper()
{
     int x = 5;
     return wrap(x); // Error: escaping reference to local 
variable x
}

void main()
{
     auto badWrap = badWrapper();
}

You can probably spot the pattern here - compiler matches `return 
ref` in parameter declaration to `ref` in return value and 
enforces identical lifetime for those. No `ref` in return value - 
no enforcing, but it will also happily accept nonsense `return 
ref` annotations.

Theoretically it was supposed to be protected by ` safe` 
attribute as one can't take address of local variable in safe 
code. But there isn't any way to write such wrapper struct 
without using pointers AFAIK.

In your actual example putting `return` on `get` method 
annotation is additionally very misleading because it only 
implies ensuring result does not outlive struct instance itself - 
but it gets passed around by value anyway.
May 16 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 16 May 2016 at 15:33:09 UTC, Dicebot wrote:

 tl; dr: DIP25 is so heavily under-implemented in its current 
 shape it can be considered 100% broken and experimenting will 
 uncover even more glaring holes.


Well, it's always fun to find the holes in things ... :-)


 To be more precise, judging by experimental observation, 
 currently dip25 only works when there is explicitly a `ref` 
 return value in function or method. Any escaping of reference 
 into a pointer confuses it completely:


To be fair, this is all in line with the DIP25 spec that I 
re-read after running into these issues with my wrapper struct.  
AFAICS pretty much the only case where it really relates to 
structs is when a struct method is returning a reference to an 
internal variable.

It's just frustrating there _isn't_ any thought for the kind of 
wrapper I have in mind, because as you say,


 But there isn't any way to write such wrapper struct without 
 using pointers AFAIK.


As for your point:


 In your actual example putting `return` on `get` method 
 annotation is additionally very misleading because it only 
 implies ensuring result does not outlive struct instance itself 
 - but it gets passed around by value anyway.


I thought much the same, but thought I'd try it on the off chance 
it would make a difference to detection of the problem.

Worst part of all this is that even an invariant to 
assert(this.data !is null) won't protect against issues: the 
pointer doesn't get reset to 0 after the data it points to goes 
out of scope, it just now points to potentially garbage data.

In fact, it's only with compiler optimizations enabled that the 
example I posted even generates the wrong result in its 
`writeln()` call :-P

Basically, it sounds to me like there _is_ no way to guarantee 
the safety/validity of wrapping data via pointer in this way ... 
? :-(
May 16 2016
parent Dicebot <public dicebot.lv> writes:
There is also another counter-obvious bit regarding current 
implementation - it only tracks lifetime of actual references. 
Check this example:

ref int wrap ( return ref int input )
{
     return input;
}

int badWrapper()
{
     int z;
     {
         int x = 42;
         z = wrap(x);
     }
     return z;
}


it looks obvious that this compiles OK with dip25 check because 
once value assignment happens, there is no more reference to 
track. However it is very common to expect different semantics if 
return type contains reference types - probably because it would 
be very useful and because Rust has changed expectations of what 
lifetime control can do :) And yet it will still work exactly the 
same with no warnings from compiler, creating false sense of 
correctness.
May 16 2016