www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Move semantics for D

reply Benjamin Thaut <code benjamin-thaut.de> writes:
Move semantics in C++0x are quite nice for optimization purposes. 
Thinking about it, it should be fairly easy to implement move semantics 
in D as structs don't have identity. Therefor a move constructor would 
not be required. You can already move value types for example within an 
array just by plain moving the data of the value around. With a little 
new keyword 'mov' or 'move' it would also be possible to move value 
types into and out of functions, something like this:

mov Range findNext(mov Range r)
{
   //do stuff here
}

With something like this it would not be neccessary to copy the range 
twice during the call of this function, the compiler could just plain 
copy the data and reinitialize the origin in case of the argument.
In case of the return value to only copying would be neccessary as the 
data goes out of scope anyway.

The only question would be if this causes any problems with out contracts.

The pre C++0x move trick, reserving a bit in the value representation 
for marking that the next copy is a move, is unfortunately not possible 
D because D does not have a copy constructor.

I for example have a range that iterates over a octree and thus needs to 
internally track which nodes it already visited and which ones are still 
left. This is done with a stack container. That needs to be copied 
everytime the range is copied, which causes quite some overhead.

Kind Regards
Benjamin Thaut
Jul 12 2012
next sibling parent reply "monarch_dodra" <monarch_dodra gmail.com> writes:
On Friday, 13 July 2012 at 06:50:02 UTC, Benjamin Thaut wrote:
 Move semantics in C++0x are quite nice for optimization 
 purposes. Thinking about it, it should be fairly easy to 
 implement move semantics in D as structs don't have identity. 
 Therefor a move constructor would not be required. You can 
 already move value types for example within an array just by 
 plain moving the data of the value around. With a little new 
 keyword 'mov' or 'move' it would also be possible to move value 
 types into and out of functions, something like this:

 mov Range findNext(mov Range r)
 {
   //do stuff here
 }

 With something like this it would not be neccessary to copy the 
 range twice during the call of this function, the compiler 
 could just plain copy the data and reinitialize the origin in 
 case of the argument.
 In case of the return value to only copying would be neccessary 
 as the data goes out of scope anyway.

 The only question would be if this causes any problems with out 
 contracts.

 The pre C++0x move trick, reserving a bit in the value 
 representation for marking that the next copy is a move, is 
 unfortunately not possible D because D does not have a copy 
 constructor.

 I for example have a range that iterates over a octree and thus 
 needs to internally track which nodes it already visited and 
 which ones are still left. This is done with a stack container. 
 That needs to be copied everytime the range is copied, which 
 causes quite some overhead.

 Kind Regards
 Benjamin Thaut

I'm pretty sure D already has it: Values types are "moved" in and out of function, implicitly, when possible, without any special semantics. For "return by value", the value is simply "blit copied" (memcopy), when returning a local variable. Neither the source constructor is called, nor the target postblit constructor. Just binary bit copy. For passing in by value, the compiler will do the same trick "if and when" it can detect you are never going to use said value again. If you are, you can always force a move using an explicit std.algorithm.move: "myRange = findNext(move(myRange));" You get pretty the same effect as in C++11, except: a) the compiler will eagerly move stuff for you b) you don't even have to define fun(T&&) (Whew) Finally, if you want to remove the responsibility of the move from the caller to the callee, I *guess* you can always just pass by ref and do stuff there: Range findNext(ref Range r) { auto r2; move(r, r2); //move r into r2 //do stuff here return r2; //move return } Note that while it might seem useless to move r into r2 (since you already have r by reference), you still have to move into a local temporary so that the compiler can "move" r2 out of findNext. The above function will make 0 calls to this(this) and 0 calls to ~this. There will be about two copies of this.init. ...of course, at this point, one could wonder why: a) you don't just take a ref, and return void? b) Use the safer just as efficient "myRange = findNext(move(myRange));"
Jul 13 2012
next sibling parent Benjamin Thaut <code benjamin-thaut.de> writes:
Am 13.07.2012 10:06, schrieb monarch_dodra:
 On Friday, 13 July 2012 at 06:50:02 UTC, Benjamin Thaut wrote:
 Move semantics in C++0x are quite nice for optimization purposes.
 Thinking about it, it should be fairly easy to implement move
 semantics in D as structs don't have identity. Therefor a move
 constructor would not be required. You can already move value types
 for example within an array just by plain moving the data of the value
 around. With a little new keyword 'mov' or 'move' it would also be
 possible to move value types into and out of functions, something like
 this:

 mov Range findNext(mov Range r)
 {
   //do stuff here
 }

 With something like this it would not be neccessary to copy the range
 twice during the call of this function, the compiler could just plain
 copy the data and reinitialize the origin in case of the argument.
 In case of the return value to only copying would be neccessary as the
 data goes out of scope anyway.

 The only question would be if this causes any problems with out
 contracts.

 The pre C++0x move trick, reserving a bit in the value representation
 for marking that the next copy is a move, is unfortunately not
 possible D because D does not have a copy constructor.

 I for example have a range that iterates over a octree and thus needs
 to internally track which nodes it already visited and which ones are
 still left. This is done with a stack container. That needs to be
 copied everytime the range is copied, which causes quite some overhead.

 Kind Regards
 Benjamin Thaut

I'm pretty sure D already has it: Values types are "moved" in and out of function, implicitly, when possible, without any special semantics. For "return by value", the value is simply "blit copied" (memcopy), when returning a local variable. Neither the source constructor is called, nor the target postblit constructor. Just binary bit copy. For passing in by value, the compiler will do the same trick "if and when" it can detect you are never going to use said value again. If you are, you can always force a move using an explicit std.algorithm.move: "myRange = findNext(move(myRange));" You get pretty the same effect as in C++11, except: a) the compiler will eagerly move stuff for you b) you don't even have to define fun(T&&) (Whew) Finally, if you want to remove the responsibility of the move from the caller to the callee, I *guess* you can always just pass by ref and do stuff there: Range findNext(ref Range r) { auto r2; move(r, r2); //move r into r2 //do stuff here return r2; //move return } Note that while it might seem useless to move r into r2 (since you already have r by reference), you still have to move into a local temporary so that the compiler can "move" r2 out of findNext. The above function will make 0 calls to this(this) and 0 calls to ~this. There will be about two copies of this.init. ...of course, at this point, one could wonder why: a) you don't just take a ref, and return void? b) Use the safer just as efficient "myRange = findNext(move(myRange));"

I didn't know know about the compiler already moving structs as return values. But not having to do the move for a argument manuall would be nice, espeically as ref does not work with non-lvalues Kind Regards Benjamin Thaut
Jul 13 2012
prev sibling parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
Am 13.07.2012 10:19, schrieb Mehrdad:
 On Friday, 13 July 2012 at 08:06:08 UTC, monarch_dodra wrote:
 Move semantics in C++0x are quite nice for optimization purposes.

That's not why they were introduced. They were introduced for making things like unique_ptr possible.

Well thats your point of view. I'm coming from the gaming industry, and everyone I have talked to so far, is happy because they can use it to optimize better. Kind Regards Benjamin Thaut
Jul 13 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/13/12 5:11 AM, Jonathan M Davis wrote:
 On Friday, July 13, 2012 10:45:03 Benjamin Thaut wrote:
 Am 13.07.2012 10:19, schrieb Mehrdad:
 On Friday, 13 July 2012 at 08:06:08 UTC, monarch_dodra wrote:
 Move semantics in C++0x are quite nice for optimization purposes.

That's not why they were introduced. They were introduced for making things like unique_ptr possible.

Well thats your point of view. I'm coming from the gaming industry, and everyone I have talked to so far, is happy because they can use it to optimize better.

I expect that the reality of the matter is that it was introduced for both. - Jonathan M Davis

Three. These two plus perfect forwarding. Which is imperfect :o). Andrei
Jul 13 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/13/12 9:05 PM, Mehrdad wrote:
 On Friday, 13 July 2012 at 11:04:00 UTC, Andrei Alexandrescu wrote:
 Three. These two plus perfect forwarding.

Yup, that too.
 Which is imperfect :o).

Why/how?

https://www.facebook.com/video/video.php?v=10151094464083109 Andrei
Jul 13 2012
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
  On 07/13/2012 09:45 PM, Mehrdad wrote:> On Saturday, 14 July 2012 at 
03:16:20 UTC, Andrei Alexandrescu wrote:
 https://www.facebook.com/video/video.php?v=10151094464083109

 Andrei


 Still though, I feel calling it "imperfect" is kinda strong. It doesn't
 have any _problems_ per se, just shortcomings.

If I remember correctly, Scott calls it imperfect as well. :) (Have you watched both parts of the presentation? I think it was closer to the end.) The hoops that one needs to go through to make the compiler agree with what is needed, was simply scary for me. For example, the way enable_if is used is so indirect: Enabling or disabling a template implementation by using a strange return type is simply WAT! :) In that sense, the feature is not perfect. Ali
Jul 14 2012
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Friday, 13 July 2012 at 08:06:08 UTC, monarch_dodra wrote:
 Move semantics in C++0x are quite nice for optimization 
 purposes.

That's not why they were introduced. They were introduced for making things like unique_ptr possible.
Jul 13 2012
prev sibling next sibling parent reply travert phare.normalesup.org (Christophe Travert) writes:
Benjamin Thaut , dans le message (digitalmars.D:172207), a écrit :
 Move semantics in C++0x are quite nice for optimization purposes. 
 Thinking about it, it should be fairly easy to implement move semantics 
 in D as structs don't have identity. Therefor a move constructor would 
 not be required. You can already move value types for example within an 
 array just by plain moving the data of the value around. With a little 
 new keyword 'mov' or 'move' it would also be possible to move value 
 types into and out of functions, something like this:
 
 mov Range findNext(mov Range r)
 {
    //do stuff here
 }
 
 With something like this it would not be neccessary to copy the range 
 twice during the call of this function, the compiler could just plain 
 copy the data and reinitialize the origin in case of the argument.
 In case of the return value to only copying would be neccessary as the 
 data goes out of scope anyway.

If Range is a Rvalue, it will be moved, not copied. It it's a Lvalue, your operation is dangerous, and does not bring you much more than using ref (it may be faster to copy the range than to take the reference, but that's an optimiser issue). auto ref seems to be the solution.
 I for example have a range that iterates over a octree and thus needs to 
 internally track which nodes it already visited and which ones are still 
 left. This is done with a stack container. That needs to be copied 
 everytime the range is copied, which causes quite some overhead.

I would share the tracking data between several instance of the range, making bitwise copy suitable. Tracking data would be duplicated only on call to save or opSlice(). You'd hit the issue of foreach not calling save when it should, but opSlice would solve this, and you could still overload opApply if you want to be sure. -- Christophe
Jul 13 2012
next sibling parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
Am 13.07.2012 10:59, schrieb Christophe Travert:
 Benjamin Thaut , dans le message (digitalmars.D:172207), a écrit :
 Move semantics in C++0x are quite nice for optimization purposes.
 Thinking about it, it should be fairly easy to implement move semantics
 in D as structs don't have identity. Therefor a move constructor would
 not be required. You can already move value types for example within an
 array just by plain moving the data of the value around. With a little
 new keyword 'mov' or 'move' it would also be possible to move value
 types into and out of functions, something like this:

 mov Range findNext(mov Range r)
 {
     //do stuff here
 }

 With something like this it would not be neccessary to copy the range
 twice during the call of this function, the compiler could just plain
 copy the data and reinitialize the origin in case of the argument.
 In case of the return value to only copying would be neccessary as the
 data goes out of scope anyway.

If Range is a Rvalue, it will be moved, not copied. It it's a Lvalue, your operation is dangerous, and does not bring you much more than using ref (it may be faster to copy the range than to take the reference, but that's an optimiser issue). auto ref seems to be the solution.
 I for example have a range that iterates over a octree and thus needs to
 internally track which nodes it already visited and which ones are still
 left. This is done with a stack container. That needs to be copied
 everytime the range is copied, which causes quite some overhead.

I would share the tracking data between several instance of the range, making bitwise copy suitable. Tracking data would be duplicated only on call to save or opSlice(). You'd hit the issue of foreach not calling save when it should, but opSlice would solve this, and you could still overload opApply if you want to be sure.

I couldn't find anything in the documentation about foreach calling save or opSlice(). So I assume foreach calls opSlice if aviable? foreach(el; range) { ... } translates to: for(auto r = range[]; !r.empty(); r.popFront() { auto el = r.front(); ... } Kind Regards Benjamin Thaut
Jul 13 2012
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, July 13, 2012 11:23:41 monarch_dodra wrote:
 I think that in the case of your example, if "range" fulfills the
 requirements for (at least) an input range, then "save" *should*
 be called instead of "opSlice". I'm *think* this is what the
 compiler does, but I'm not 100% sure.

The compiler never calls save. It will call opSlice on non-ranges, but it'll never call save on anything. It doesn't even know that save exists. It's unnecessary for ranges which aren't reference types, and you already have to worry about calling save on ranges that could be reference types any time that you pass them to a function that you don't want to consume it, so it's not really a big deal to have to call save with foreach for such ranges. Most range-based functions don't even use foreach much anyway. - Jonathan M Davis
Jul 13 2012
prev sibling parent "monarch_dodra" <monarch_dodra gmail.com> writes:
On Friday, 13 July 2012 at 09:47:26 UTC, Jonathan M Davis wrote:
 On Friday, July 13, 2012 11:23:41 monarch_dodra wrote:
 I think that in the case of your example, if "range" fulfills 
 the
 requirements for (at least) an input range, then "save" 
 *should*
 be called instead of "opSlice". I'm *think* this is what the
 compiler does, but I'm not 100% sure.

The compiler never calls save. It will call opSlice on non-ranges, but it'll never call save on anything. It doesn't even know that save exists. It's unnecessary for ranges which aren't reference types, and you already have to worry about calling save on ranges that could be reference types any time that you pass them to a function that you don't want to consume it, so it's not really a big deal to have to call save with foreach for such ranges. Most range-based functions don't even use foreach much anyway. - Jonathan M Davis

What exactly are the semantics of save? The reference in std.range isn't very clear. It would appear it is only useful it its *existence* that promotes a range from input to forward. However, how is it different from a simple assignment? Also, if you are supposed to call save before a foreach "consumes" your range, then why does foreach even bother making a copy of the range before iterating on it? Isn't this behavior promoting dangerous usage for ranges where save is simply "{return this;}", but bites you in the ass the day you use a range with a specific save? I'm confused...
Jul 13 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, July 13, 2012 11:09:04 Benjamin Thaut wrote:
 Am 13.07.2012 10:59, schrieb Christophe Travert:
 Benjamin Thaut , dans le message (digitalmars.D:172207), a =C3=A9cr=


 Move semantics in C++0x are quite nice for optimization purposes.
 Thinking about it, it should be fairly easy to implement move sema=



 in D as structs don't have identity. Therefor a move constructor w=



 not be required. You can already move value types for example with=



 array just by plain moving the data of the value around. With a li=



 new keyword 'mov' or 'move' it would also be possible to move valu=



 types into and out of functions, something like this:
=20
 mov Range findNext(mov Range r)
 {
=20
     //do stuff here
=20
 }
=20
 With something like this it would not be neccessary to copy the ra=



 twice during the call of this function, the compiler could just pl=



 copy the data and reinitialize the origin in case of the argument.=



 In case of the return value to only copying would be neccessary as=



 data goes out of scope anyway.

If Range is a Rvalue, it will be moved, not copied. It it's a Lvalue, your operation is dangerous, and does not bring y=


 much more than using ref (it may be faster to copy the range than t=


 take the reference, but that's an optimiser issue).
=20
 auto ref seems to be the solution.
=20
 I for example have a range that iterates over a octree and thus ne=



 internally track which nodes it already visited and which ones are=



 left. This is done with a stack container. That needs to be copied=



 everytime the range is copied, which causes quite some overhead.

I would share the tracking data between several instance of the ran=


 making bitwise copy suitable. Tracking data would be duplicated onl=


 call to save or opSlice(). You'd hit the issue of foreach not calli=


 save when it should, but opSlice would solve this, and you could st=


 overload opApply if you want to be sure.

I couldn't find anything in the documentation about foreach calling s=

 or opSlice(). So I assume foreach calls opSlice if aviable?

If you have foreach(e; obj) { ... } and obj is a range, then it becomes for(auto __range =3D obj; !__range.empty; __range.popFront()) { auto e =3D __range.front; ... } If obj is not a range, but it does declare opSlice, then it's sliced, s= o you=20 get something like for(auto __range =3D obj[]; !__range.empty; __range.popFront()) { auto e =3D __range.front; ... } There's also opApply to consider though, and it probably gets precedenc= e over=20 opSlice if a type defines both. I don't know exactly how all of that's = laid out=20 right now though. However, there's a recent thread discussing it ("opAp= ply not=20 called for foeach(container)"), so you can read that, and I think that = they=20 give better details on precedence. I haven't read it in great detail th= ough,=20 so I don't know the details other than the fact that it sounds like the= =20 precedence rules for foreach probably neet do be ironed out better. - Jonathan M Davis
Jul 13 2012
prev sibling parent "monarch_dodra" <monarch_dodra gmail.com> writes:
On Friday, 13 July 2012 at 09:09:03 UTC, Benjamin Thaut wrote:
 Am 13.07.2012 10:59, schrieb Christophe Travert:
 Benjamin Thaut , dans le message (digitalmars.D:172207), a 
 écrit :
 Move semantics in C++0x are quite nice for optimization 
 purposes.
 Thinking about it, it should be fairly easy to implement move 
 semantics
 in D as structs don't have identity. Therefor a move 
 constructor would
 not be required. You can already move value types for example 
 within an
 array just by plain moving the data of the value around. With 
 a little
 new keyword 'mov' or 'move' it would also be possible to move 
 value
 types into and out of functions, something like this:

 mov Range findNext(mov Range r)
 {
    //do stuff here
 }

 With something like this it would not be neccessary to copy 
 the range
 twice during the call of this function, the compiler could 
 just plain
 copy the data and reinitialize the origin in case of the 
 argument.
 In case of the return value to only copying would be 
 neccessary as the
 data goes out of scope anyway.

If Range is a Rvalue, it will be moved, not copied. It it's a Lvalue, your operation is dangerous, and does not bring you much more than using ref (it may be faster to copy the range than to take the reference, but that's an optimiser issue). auto ref seems to be the solution.
 I for example have a range that iterates over a octree and 
 thus needs to
 internally track which nodes it already visited and which 
 ones are still
 left. This is done with a stack container. That needs to be 
 copied
 everytime the range is copied, which causes quite some 
 overhead.

I would share the tracking data between several instance of the range, making bitwise copy suitable. Tracking data would be duplicated only on call to save or opSlice(). You'd hit the issue of foreach not calling save when it should, but opSlice would solve this, and you could still overload opApply if you want to be sure.

I couldn't find anything in the documentation about foreach calling save or opSlice(). So I assume foreach calls opSlice if aviable? foreach(el; range) { ... } translates to: for(auto r = range[]; !r.empty(); r.popFront() { auto el = r.front(); ... } Kind Regards Benjamin Thaut

Depends if you are asking about "what the compiler does", or "what the compiler should do" or "what the documentation says". There is a discussion about it here: http://forum.dlang.org/thread/rxpbtrawpjzvdfuuwmwp forum.dlang.org I think that in the case of your example, if "range" fulfills the requirements for (at least) an input range, then "save" *should* be called instead of "opSlice". I'm *think* this is what the compiler does, but I'm not 100% sure.
Jul 13 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, July 13, 2012 10:45:03 Benjamin Thaut wrote:
 Am 13.07.2012 10:19, schrieb Mehrdad:
 On Friday, 13 July 2012 at 08:06:08 UTC, monarch_dodra wrote:
 Move semantics in C++0x are quite nice for optimization purposes.

That's not why they were introduced. They were introduced for making things like unique_ptr possible.

Well thats your point of view. I'm coming from the gaming industry, and everyone I have talked to so far, is happy because they can use it to optimize better.

I expect that the reality of the matter is that it was introduced for both. - Jonathan M Davis
Jul 13 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, July 13, 2012 08:50:03 Benjamin Thaut wrote:
 Move semantics in C++0x are quite nice for optimization purposes.
 Thinking about it, it should be fairly easy to implement move semantics
 in D as structs don't have identity. Therefor a move constructor would
 not be required. You can already move value types for example within an
 array just by plain moving the data of the value around.

http://stackoverflow.com/questions/4200190/does-d-have-something-akin-to-c0xs- move-semantics http://stackoverflow.com/questions/6884996/questions-about-postblit-and-move- semantics - Jonathan M Davis
Jul 13 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, July 13, 2012 12:24:45 monarch_dodra wrote:
 On Friday, 13 July 2012 at 09:47:26 UTC, Jonathan M Davis wrote:
 On Friday, July 13, 2012 11:23:41 monarch_dodra wrote:
 I think that in the case of your example, if "range" fulfills
 the
 requirements for (at least) an input range, then "save"
 *should*
 be called instead of "opSlice". I'm *think* this is what the
 compiler does, but I'm not 100% sure.

The compiler never calls save. It will call opSlice on non-ranges, but it'll never call save on anything. It doesn't even know that save exists. It's unnecessary for ranges which aren't reference types, and you already have to worry about calling save on ranges that could be reference types any time that you pass them to a function that you don't want to consume it, so it's not really a big deal to have to call save with foreach for such ranges. Most range-based functions don't even use foreach much anyway. - Jonathan M Davis

What exactly are the semantics of save? The reference in std.range isn't very clear. It would appear it is only useful it its *existence* that promotes a range from input to forward. However, how is it different from a simple assignment? Also, if you are supposed to call save before a foreach "consumes" your range, then why does foreach even bother making a copy of the range before iterating on it? Isn't this behavior promoting dangerous usage for ranges where save is simply "{return this;}", but bites you in the ass the day you use a range with a specific save? I'm confused...

save exists so that classes can be forward ranges. Arrays and most structs are copied when you assign them to another variable or pass them to a function, but classes aren't. So, save was introduced to make it so that there is a way to explicitly copy a range. It becomes useful even for structs and arrays in that it documents that you're copying it, but it's outright necessary for classes. Unfortunately, since arrays and structs are by far the most common range types, save doesn't get used anywhere near as much as it should be. Basically, you use save if you want to guarantee that the range is copied, and you don't use save if you don't care. You can also look at this: http://stackoverflow.com/questions/11190204/how-do-you-use-ranges-in- d/11194827#11194827 - Jonathan M Davis
Jul 13 2012
prev sibling next sibling parent "monarch_dodra" <monarch_dodra gmail.com> writes:
On Friday, 13 July 2012 at 10:33:13 UTC, Jonathan M Davis wrote:
 On Friday, July 13, 2012 12:24:45 monarch_dodra wrote:
 On Friday, 13 July 2012 at 09:47:26 UTC, Jonathan M Davis 
 wrote:
 On Friday, July 13, 2012 11:23:41 monarch_dodra wrote:
 I think that in the case of your example, if "range" 
 fulfills
 the
 requirements for (at least) an input range, then "save"
 *should*
 be called instead of "opSlice". I'm *think* this is what the
 compiler does, but I'm not 100% sure.

The compiler never calls save. It will call opSlice on non-ranges, but it'll never call save on anything. It doesn't even know that save exists. It's unnecessary for ranges which aren't reference types, and you already have to worry about calling save on ranges that could be reference types any time that you pass them to a function that you don't want to consume it, so it's not really a big deal to have to call save with foreach for such ranges. Most range-based functions don't even use foreach much anyway. - Jonathan M Davis

What exactly are the semantics of save? The reference in std.range isn't very clear. It would appear it is only useful it its *existence* that promotes a range from input to forward. However, how is it different from a simple assignment? Also, if you are supposed to call save before a foreach "consumes" your range, then why does foreach even bother making a copy of the range before iterating on it? Isn't this behavior promoting dangerous usage for ranges where save is simply "{return this;}", but bites you in the ass the day you use a range with a specific save? I'm confused...

save exists so that classes can be forward ranges. Arrays and most structs are copied when you assign them to another variable or pass them to a function, but classes aren't. So, save was introduced to make it so that there is a way to explicitly copy a range. It becomes useful even for structs and arrays in that it documents that you're copying it, but it's outright necessary for classes. Unfortunately, since arrays and structs are by far the most common range types, save doesn't get used anywhere near as much as it should be. Basically, you use save if you want to guarantee that the range is copied, and you don't use save if you don't care. You can also look at this: http://stackoverflow.com/questions/11190204/how-do-you-use-ranges-in- d/11194827#11194827 - Jonathan M Davis

I guess that "save" makes sense thanks in that context. Thank you. However, foreach is starting to look very dangerous to me: Isn't the fact that it (potentially) calls opSlice, or makes a copy of your input just a asking for potential problems? Or is this a "struct vs class" issue, that I do not yet fully grasp? Shouldn't we instead enforce an _explicit_ opSlice/save? Eg: auto SomeContainer = ... ; auto SomeRange = SomeContainer[ ... ]; foreach(v; SomeContainer[]) ... ; //Fine, iterate on a new range, and consume that foreach(v; SomeRange.save) ... ; //Fine, iterate on a copy of the range, and comue that foreach(v; SomeContainer) ... ; //Fine, I will NOT call opSlice, and _consume_ your container foreach(v; SomeRange) ... ; //Fine, I will NOT copy, and _consume_ your range ... I suppose the "recommendation" is to use the above form, and that is what I will be doing as of now on. But I still feel that the call to opSlice/copy is really a just trap disguised as a safety net...
Jul 13 2012
prev sibling next sibling parent "monarch_dodra" <monarch_dodra gmail.com> writes:
On Friday, 13 July 2012 at 10:33:13 UTC, Jonathan M Davis wrote:
 On Friday, July 13, 2012 12:24:45 monarch_dodra wrote:
 On Friday, 13 July 2012 at 09:47:26 UTC, Jonathan M Davis 
 wrote:
 On Friday, July 13, 2012 11:23:41 monarch_dodra wrote:
 I think that in the case of your example, if "range" 
 fulfills
 the
 requirements for (at least) an input range, then "save"
 *should*
 be called instead of "opSlice". I'm *think* this is what the
 compiler does, but I'm not 100% sure.

The compiler never calls save. It will call opSlice on non-ranges, but it'll never call save on anything. It doesn't even know that save exists. It's unnecessary for ranges which aren't reference types, and you already have to worry about calling save on ranges that could be reference types any time that you pass them to a function that you don't want to consume it, so it's not really a big deal to have to call save with foreach for such ranges. Most range-based functions don't even use foreach much anyway. - Jonathan M Davis

What exactly are the semantics of save? The reference in std.range isn't very clear. It would appear it is only useful it its *existence* that promotes a range from input to forward. However, how is it different from a simple assignment? Also, if you are supposed to call save before a foreach "consumes" your range, then why does foreach even bother making a copy of the range before iterating on it? Isn't this behavior promoting dangerous usage for ranges where save is simply "{return this;}", but bites you in the ass the day you use a range with a specific save? I'm confused...

save exists so that classes can be forward ranges. Arrays and most structs are copied when you assign them to another variable or pass them to a function, but classes aren't. So, save was introduced to make it so that there is a way to explicitly copy a range. It becomes useful even for structs and arrays in that it documents that you're copying it, but it's outright necessary for classes. Unfortunately, since arrays and structs are by far the most common range types, save doesn't get used anywhere near as much as it should be. Basically, you use save if you want to guarantee that the range is copied, and you don't use save if you don't care. - Jonathan M Davis

Thanks a lot for the explanation. It makes a lot of sense. However, foreach is starting to look very dangerous to me: Isn't the fact that it (potentially) calls opSlice, or makes a copy of your input just asking for potential problems? Or is this more of a "struct vs class" issue, that I do not yet fully grasp? Shouldn't D _enforce_ an _explicit_ opSlice/save? Eg: auto SomeContainer = ... ; auto SomeRange = SomeContainer[ ... ]; foreach(v; SomeContainer[]) ... ; //Fine, iterate on a new range, and consume that foreach(v; SomeRange.save) ... ; //Fine, iterate on a copy of the range, and comue that foreach(v; SomeContainer) ... ; //Fine, I will NOT call opSlice, and _consume_ your container foreach(v; SomeRange) ... ; //Fine, I will NOT copy, and _consume_ your range On a side note, it would also make "foreach"s behavior clearer... ... I suppose the "recommendation" is to use the above form, and that is what I will be doing as of now on. But I still feel that the internal call to opSlice/copy is really a just trap disguised as a safety net...
Jul 13 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, July 13, 2012 14:11:44 monarch_dodra wrote:
 But I still feel that the internal call to opSlice/copy is really
 a just trap disguised as a safety net...

Yeah. Kind of. The thing is that there's no way around the fact that structs and arrays are effectively value types as far as ranges go, whereas classes are reference types. If you want them to be copied consistently, you need to use save. If you don't care whether a copy is made or not, then you don't worry about it. But foreach has to be treated specially with ranges anyway, because of strings. foreach(e; str) {} is going to iterate over char, not dchar, whereas strings are ranges of dchar. So, if you iterate over a range generically, you need to do foreach(ElementType!R e; range) {} So, you have to be careful with foreach already, and having it _not_ save but having it copy the range automatically (and therefore implicitly saving for non-reference type forward ranges) is completely consistent with how it works with passing ranges to functions, so in that sense, it really isn't all that bad. If anything, things are _more_ consistent that way. The only way to really solve the save problem would be to disallow forward ranges which were reference types (which you could mostly do by disallowing classes, but I don' think that there's any real way to statically check whether a struct is a reference type or not). Then save wouldn't be needed, and ranges would all function the same as long as no one made structs which were reference types into ranges. But that would also cut off some useful idioms (sometimes, you _want_ a range to be consumed by a function rather than being implicitly copied - which is why I added std.range.RefRange for 2.060). So, I don't know of any way to really fix the problem. You need save to make it possible to copy reference types, and there's no way to reasonably avoid having to deal with both value type and reference type ranges. So, we just deal with it. Unfortunately, far too often, the result of that is that value type ranges work properly and reference type ranges don't, since range-based functions are usually tested with just value types ranges, but improved testing fixes that, and that's not all that hard to do. - Jonathan M Davis
Jul 13 2012
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Friday, 13 July 2012 at 11:04:00 UTC, Andrei Alexandrescu 
wrote:
 Three. These two plus perfect forwarding.

Yup, that too.
 Which is imperfect :o).

Why/how?
Jul 13 2012
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Saturday, 14 July 2012 at 03:16:20 UTC, Andrei Alexandrescu 
wrote:
 https://www.facebook.com/video/video.php?v=10151094464083109

 Andrei

Interesting, I'd never even thought about those cases. That definitely _does_ make it less than perfect... but then again, I never really would have expected them to work anyway. (0/NULL? That's why they have nullptr! or I never would've expected bitfields to work either, etc.) Still though, I feel calling it "imperfect" is kinda strong. It doesn't have any _problems_ per se, just shortcomings.
Jul 13 2012
prev sibling parent "monarch_dodra" <monarch_dodra gmail.com> writes:
On Friday, 13 July 2012 at 22:27:01 UTC, Jonathan M Davis wrote:
 On Friday, July 13, 2012 14:11:44 monarch_dodra wrote:
 But I still feel that the internal call to opSlice/copy is 
 really
 a just trap disguised as a safety net...

Yeah. Kind of. The thing is that there's no way around the fact that structs and arrays are effectively value types as far as ranges go, whereas classes are reference types. If you want them to be copied consistently, you need to use save. If you don't care whether a copy is made or not, then you don't worry about it. But foreach has to be treated specially with ranges anyway, because of strings. foreach(e; str) {} is going to iterate over char, not dchar, whereas strings are ranges of dchar. So, if you iterate over a range generically, you need to do foreach(ElementType!R e; range) {} So, you have to be careful with foreach already, and having it _not_ save but having it copy the range automatically (and therefore implicitly saving for non-reference type forward ranges) is completely consistent with how it works with passing ranges to functions, so in that sense, it really isn't all that bad. If anything, things are _more_ consistent that way. The only way to really solve the save problem would be to disallow forward ranges which were reference types (which you could mostly do by disallowing classes, but I don' think that there's any real way to statically check whether a struct is a reference type or not). Then save wouldn't be needed, and ranges would all function the same as long as no one made structs which were reference types into ranges. But that would also cut off some useful idioms (sometimes, you _want_ a range to be consumed by a function rather than being implicitly copied - which is why I added std.range.RefRange for 2.060). So, I don't know of any way to really fix the problem. You need save to make it possible to copy reference types, and there's no way to reasonably avoid having to deal with both value type and reference type ranges. So, we just deal with it. Unfortunately, far too often, the result of that is that value type ranges work properly and reference type ranges don't, since range-based functions are usually tested with just value types ranges, but improved testing fixes that, and that's not all that hard to do. - Jonathan M Davis

Very good points. I appreciate your answer. I guess I should consider the "copy" the standard byproduct of "passing to foreach" => I should _also_ remember to save before passing a range to a function.
Jul 14 2012