www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Problem with const correctness

reply Gor Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
--f46d0420a695d0a77b04d043e3ec
Content-Type: text/plain; charset=UTF-8

Consider this example:

struct Array(Type_)
{
public:
this(Type_ array_[]...)
{
_array = array_;
}
 this(this)
{
_array = _array.dup;
}
 ref Array!Type_ opAssign(in Array!Type_ array_)
{
_array = array_._array.dup;
return this;
}
 private:
Type_[] _array;
}
 unittest
{
Array!int one = Array!int(1, 2, 3, 4, 5);
immutable Array!int two = one; //Error: conversion error from Array!(int)
to immutable(Array!(int))
}
I enforce value-type semantics by duplicating the arrays on copy, so it
should behave like a value type with no indirections (implicitly convert to
immutable).
What do I need to do for this to work?

-- 
Bye,
Gor Gyolchanyan.

--f46d0420a695d0a77b04d043e3ec
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div><span class=3D"" style=3D"white-space:pre">Consider this example:</spa=
n></div><div><span class=3D"" style=3D"white-space:pre"><br></span></div><d=
iv><span class=3D"" style=3D"white-space:pre">	</span>struct Array(Type_)<b=
r></div><div>
<span class=3D"" style=3D"white-space:pre">	</span>{</div><div><span class=
=3D"" style=3D"white-space:pre">	</span>public:</div><div><span class=3D"" =
style=3D"white-space:pre">		</span>this(Type_ array_[]...)</div><div><span =
class=3D"" style=3D"white-space:pre">		</span>{</div>
<div><span class=3D"" style=3D"white-space:pre">			</span>_array =3D array_=
;</div><div><span class=3D"" style=3D"white-space:pre">		</span>}</div><div=
<span class=3D"" style=3D"white-space:pre">	</span></div><div><span class=

<div><span class=3D"" style=3D"white-space:pre"> </span>{</div><div><span = class=3D"" style=3D"white-space:pre"> </span>_array =3D _array.dup;</div>= <div><span class=3D"" style=3D"white-space:pre"> </span>}</div><div><span = class=3D"" style=3D"white-space:pre"> </span></div> <div><span class=3D"" style=3D"white-space:pre"> </span>ref Array!Type_ op= Assign(in Array!Type_ array_)</div><div><span class=3D"" style=3D"white-spa= ce:pre"> </span>{</div><div><span class=3D"" style=3D"white-space:pre"> = </span>_array =3D array_._array.dup;</div> <div><span class=3D"" style=3D"white-space:pre"> </span>return this;</div=
<div><span class=3D"" style=3D"white-space:pre">		</span>}</div><div><span=

style=3D"white-space:pre"> </span>private:</div> <div><span class=3D"" style=3D"white-space:pre"> </span>Type_[] _array;</d= iv><div><span class=3D"" style=3D"white-space:pre"> </span>}</div><div><spa= n class=3D"" style=3D"white-space:pre"> </span></div><div><span class=3D"" = style=3D"white-space:pre"> </span>unittest</div> <div><span class=3D"" style=3D"white-space:pre"> </span>{</div><div><span c= lass=3D"" style=3D"white-space:pre"> </span>Array!int one =3D Array!int(1,= 2, 3, 4, 5);</div><div><span class=3D"" style=3D"white-space:pre"> </span=
immutable Array!int two =3D one<span style=3D"white-space:pre">; //Error: =

<div><span class=3D"" style=3D"white-space:pre"> </span>}</div><div><span c= lass=3D"" style=3D"white-space:pre"> </span></div><div>I enforce value-type= semantics by duplicating the arrays on copy, so it should behave like a va= lue type with no indirections (implicitly convert to immutable).<br> </div><div>What do I need to do for this to work?</div><div><br></div>-- <b= r>Bye,<br>Gor Gyolchanyan.<br> --f46d0420a695d0a77b04d043e3ec--
Dec 07 2012
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Gor Gyolchanyan:

 I enforce value-type semantics by duplicating the arrays on 
 copy, so it
 should behave like a value type with no indirections 
 (implicitly convert to
 immutable).
 What do I need to do for this to work?

This is a first try, not tested much, and I don't know how many copies it performs: import std.traits; struct Array(T) { this(T items[]...) { this._array = items; } this(this) { static if (isMutable!T) this._array = this._array.dup; else this._array = this._array.idup; } ref Array opAssign(in Array other) { static if (isMutable!T) this._array = other._array.dup; else this._array = other._array.idup; return this; } property Array!(immutable(T)) idup() { return typeof(return)(this._array.idup); } private T[] _array; } void main() { auto one = Array!int(1, 2, 3, 4, 5); immutable two = one.idup; } Bye, bearophile
Dec 07 2012
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
  On 12/07/2012 08:42 AM, bearophile wrote:
 H. S. Teoh:

 Workarounds:
 - Use items.dup. Problem: if you're passing an actual array to the ctor,
 it's unnecessary and inefficient.

 - Use an array literal: auto x = Array!int([1,2,3]);, which I believe
 should allocate the array on the heap, and so you're safe to just copy
 the slice. This defeats the purpose of the "items..." syntax, though.

I think the right solution is to fix the design+compiler, because it's safer. (But how do you get the original GC-less behaviour if you need max performance and you know what you are doing?)

I've run into the same issue before and have decided that I should just forget about T[]...-style parameters and just require an actual array: this(T items[]) That is both safe and efficient. If it is really cumbersome, a single-item overload can be provided as well: this(T item) { this([ item ]); // untested but should work :p } Ali
Dec 07 2012
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
 struct Array(T) {
      this(T items[]...) {
          this._array = items;
      }

For a D design bug, I think those items don't get copied. So be careful and test the code well. Bye, bearophile
Dec 07 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 07, 2012 at 04:18:00PM +0100, bearophile wrote:
struct Array(T) {
     this(T items[]...) {
         this._array = items;
     }

For a D design bug, I think those items don't get copied. So be careful and test the code well.

Yeah I ran into this issue before. Calling the above ctor by passing in implicit array (e.g., like "auto x = Array!int(1,2,3);") passes a *slice* of the function arguments _on the runtime stack_. Then the assignment statement above simply copies the slice (*not* the data) to this._array, which is most probably *not* what you want. Namely, once the function that calls the ctor goes out of scope, your struct will be holding a slice of invalid memory. (In fact, it doesn't even have to go out of scope; if later code in the function uses up more runtime stack space, it will overwrite the original array and thus invalidate the slice.) Workarounds: - Use items.dup. Problem: if you're passing an actual array to the ctor, it's unnecessary and inefficient. - Use an array literal: auto x = Array!int([1,2,3]);, which I believe should allocate the array on the heap, and so you're safe to just copy the slice. This defeats the purpose of the "items..." syntax, though. T -- Chance favours the prepared mind. -- Louis Pasteur
Dec 07 2012
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
H. S. Teoh:

 Workarounds:
 - Use items.dup. Problem: if you're passing an actual array to 
 the ctor,
   it's unnecessary and inefficient.

 - Use an array literal: auto x = Array!int([1,2,3]);, which I 
 believe
   should allocate the array on the heap, and so you're safe to 
 just copy
   the slice. This defeats the purpose of the "items..." syntax, 
 though.

I think the right solution is to fix the design+compiler, because it's safer. (But how do you get the original GC-less behaviour if you need max performance and you know what you are doing?) Bye, bearophile
Dec 07 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 07, 2012 at 05:42:18PM +0100, bearophile wrote:
 H. S. Teoh:
 
Workarounds:
- Use items.dup. Problem: if you're passing an actual array to the
  ctor, it's unnecessary and inefficient.

- Use an array literal: auto x = Array!int([1,2,3]);, which I believe
  should allocate the array on the heap, and so you're safe to just
  copy the slice. This defeats the purpose of the "items..." syntax,
  though.

I think the right solution is to fix the design+compiler, because it's safer.

Agreed.
 (But how do you get the original GC-less behaviour if you need max
 performance and you know what you are doing?)

I think there's already an issue open where the compiler should warn you of escaping reference to stack arguments. So the variadic arguments syntax will work for the GC-less case, but when you need a GC you have to explicitly make a heap array, e.g. using an array literal. The problem comes because there is an ambiguity between runtime stack slices and heap slices, which are conflated under a single syntax. When there is no escaping reference, it doesn't matter which two they are, but when there is, there's a problem. T -- Microsoft is to operating systems & security ... what McDonalds is to gourmet cooking.
Dec 07 2012
prev sibling next sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 12/07/2012 06:18 AM, Gor Gyolchanyan wrote:

   this(this)
 {
 _array = _array.dup;
 }
   ref Array!Type_ opAssign(in Array!Type_ array_)
 {
 _array = array_._array.dup;
 return this;
 }

There has been discussions on the D.learn forum recently. Apparently, the compiler-generated opAssign calls the postblit automatically anyway. (Maybe also for user-defined opAssign as well. I am not sure.) Unless there is a compiler bug about that, you may not need the opAssign definition after all. Ali
Dec 07 2012
prev sibling next sibling parent "Dan" <dbdavidson yahoo.com> writes:
On Friday, 7 December 2012 at 14:18:50 UTC, Gor Gyolchanyan wrote:
 Consider this example:
 I enforce value-type semantics by duplicating the arrays on 
 copy, so it
 should behave like a value type with no indirections 
 (implicitly convert to
 immutable).
 What do I need to do for this to work?

My approach is to have a general dup function. I call it gdup, for global dup so the name does not conflict with the existing dup. It dup's fields recursively. Feel free to have a look and any suggestions appreciated. Would greatly appreciate if seasoned D developers like (Ali, bearophile, ...) would review - as I use these mixins to simplify development with structs. https://github.com/patefacio/d-help/tree/master/d-help/opmix docs at https://github.com/patefacio/d-help/blob/master/doc/canonical.pdf Since the gdup makes all new deep copies (as dup should) the cast to immutable is safe. For reference structs with aliasing you have to worry about assignment of immutable to mutable and the reverse. Due to transitive immutability, both are disallowed without something special. dup, idup and gdup in general serve that purpose. As Ali points out, I also don't think you need the opAssign at all in this case - since default opAssign calls postblit (which is nice). To get value semantics just implement postblit. If you add fields you should not need to worry about it. As others pointed out - you want deep copy on the items in the array for your custom ctor. Since gdup is recursive and does not require the composed structs to have a postblit - it works even on arrays of T where T has aliasing but no postblit. Thanks, Dan import std.stdio; import std.traits; import opmix.mix; struct Array(Type_) { public: mixin(PostBlit); this(Type_[] array...) { _array = array.gdup; } private: Type_[] _array; } unittest { { Array!int one = Array!int(1, 2, 3, 4, 5); immutable Array!int two = cast(immutable)one.gdup; assert(two._array.ptr != one._array.ptr); assert(two._array == one._array); } { struct S { // look ma' no postblit char[] c; } Array!S one = Array!S(S(['d']), S(['o','g']), S(['b','o','y'])); auto immutable two = cast(immutable)one.gdup; // Ensure the copy was deep (i.e. no sharing) assert(cast(DeepUnqual!(typeof(two._array.ptr)))two._array.ptr != one._array.ptr); assert(typesDeepEqual(two, one)); } }
Dec 08 2012
prev sibling next sibling parent "Thiez" <thiezz gmail.com> writes:
On Saturday, 8 December 2012 at 21:47:32 UTC, Dan wrote:
 My approach is to have a general dup function. I call it gdup, 
 for global dup so the name does not conflict with the existing 
 dup. It dup's fields recursively. Feel free to have a look and 
 any suggestions appreciated. Would greatly appreciate if 
 seasoned D developers like (Ali,  bearophile, ...) would review 
 - as I use these mixins to simplify development with structs.

What would happen to the recursive dup if the structure contains a cycle (e.g. A has a reference to B, which has a reference to C, which has a reference to the original A)?
Dec 10 2012
prev sibling next sibling parent "Dan" <dbdavidson yahoo.com> writes:
On Monday, 10 December 2012 at 11:39:24 UTC, Thiez wrote:
 On Saturday, 8 December 2012 at 21:47:32 UTC, Dan wrote:
 My approach is to have a general dup function. I call it gdup, 
 for global dup so the name does not conflict with the existing 
 dup. It dup's fields recursively. Feel free to have a look and 
 any suggestions appreciated. Would greatly appreciate if 
 seasoned D developers like (Ali,  bearophile, ...) would 
 review - as I use these mixins to simplify development with 
 structs.

What would happen to the recursive dup if the structure contains a cycle (e.g. A has a reference to B, which has a reference to C, which has a reference to the original A)?

By reference I assume you mean pointer. That would be an infinite loop. If you have a compile time cycle you would likely need your own custom dups anyway, as you are doing low level and heap allocating already. But for the simpler cases without cycles, if dup encounters a pointer it creates a new instance on the heap (if compilation is possible) and dup's into it.
Dec 10 2012
prev sibling next sibling parent "Thiez" <thiezz gmail.com> writes:
On Monday, 10 December 2012 at 12:45:16 UTC, Dan wrote:
 That would be an infinite loop. If you have a compile time 
 cycle you would likely need your own custom dups anyway, as you 
 are doing low level and heap allocating already. But for the 
 simpler cases without cycles, if dup encounters a pointer it 
 creates a new instance on the heap (if compilation is possible) 
 and dup's into it.

Wouldn't it be better to do a cycle detection here? I imagine such a thing could be done quite easily by adding every pointer in the original struct to an associative array along with its (new) copy. Then, whenever you encounter a new pointer, you can check if it is already in the AA, and if so use the copy you made before. Of course this has some overhead compared to your suggestion, but it seems to me it would be safe in all cases, which makes more sense with a 'general dup'.
Dec 10 2012
prev sibling parent "Dan" <dbdavidson yahoo.com> writes:
On Monday, 10 December 2012 at 13:37:46 UTC, Thiez wrote:
 On Monday, 10 December 2012 at 12:45:16 UTC, Dan wrote:
 That would be an infinite loop. If you have a compile time 
 cycle you would likely need your own custom dups anyway, as 
 you are doing low level and heap allocating already. But for 
 the simpler cases without cycles, if dup encounters a pointer 
 it creates a new instance on the heap (if compilation is 
 possible) and dup's into it.

Wouldn't it be better to do a cycle detection here? I imagine such a thing could be done quite easily by adding every pointer in the original struct to an associative array along with its (new) copy. Then, whenever you encounter a new pointer, you can check if it is already in the AA, and if so use the copy you made before. Of course this has some overhead compared to your suggestion, but it seems to me it would be safe in all cases, which makes more sense with a 'general dup'.

I see a few possibilities: (0) Do nothing and caveat coder (1) Compile time check for possibility of cycle and if exists do not compile. This way the scenario you mention, which may or may not really have instance cycles, would not even compile. This may be overly aggressive. (2) Compile time check for possibility of cycle and runtime checks ensuring there are none. (3) Disallow dup on structs with embedded pointers (excepting array and associative array). Similar to (0) but now basic structs with pointers to other basic structs and no chance of cycles would not be dupable. I'm fine with any of these because I figure if you are allocating your own objects you probably want to write your own postblit and dup. But, you are correct, cycle detection at runtime would be better - assuming there was no runtime performance hit for the case when cycles are not possible which is known at compile time. I'm not so sure it is as easy as a simple AA, though. ------- struct S { struct Guts { } Guts guts; } S s; ------- In this case both &s and &s.guts have the same address. So you might want to have an AA per type? But where would those exist? On the stack? If so how would you pass it through to all the recursive calls? I'm not saying it is not doable - I just think it may be a pretty big effort. Thanks, Dan
Dec 10 2012