www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Discussion Thread: DIP 1040--Copying, Moving, and

reply Mike Parker <aldacron gmail.com> writes:
This is the discussion thread for the first round of Community 
Review of DIP 1040, "Copying, Moving, and Forwarding":

https://github.com/dlang/DIPs/blob/a9c553b0dbab1c2983a801b5e89b51c5c33d5180/DIPs/DIP1040.md

The review period will end at 11:59 PM ET on March 19, or when I 
make a post declaring it complete. Discussion in this thread may 
continue beyond that point.

Here in the discussion thread, you are free to discuss anything 
and everything related to the DIP. Express your support or 
opposition, debate alternatives, argue the merits, etc.

However, if you have any specific feedback on how to improve the 
proposal itself, then please post it in the feedback thread. The 
feedback thread will be the source for the review summary that I 
will write at the end of this review round. I will post a link to 
that thread immediately following this post. Just be sure to read 
and understand the Reviewer Guidelines before posting there:

https://github.com/dlang/DIPs/blob/master/docs/guidelines-reviewers.md

And my blog post on the difference between the Discussion and 
Feedback threads:

https://dlang.org/blog/2020/01/26/dip-reviews-discussion-vs-feedback/

Please stay on topic here. I will delete posts that are 
completely off-topic.
Mar 05
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:

 However, if you have any specific feedback on how to improve 
 the proposal itself, then please post it in the feedback 
 thread. The feedback thread will be the source for the review 
 summary that I will write at the end of this review round. I 
 will post a link to that thread immediately following this post.
The Feedback Thread is here: https://forum.dlang.org/post/axgfyrxvndxdmffkxvhs forum.dlang.org
Mar 05
prev sibling next sibling parent reply Ben Jones <fake fake.fake> writes:
On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of Community 
 Review of DIP 1040, "Copying, Moving, and Forwarding":

 https://github.com/dlang/DIPs/blob/a9c553b0dbab1c2983a801b5e89b51c5c33d5180/DIPs/DIP1040.md
I had a couple of thoughts on how to make the DIP description a bit more accessible (I don't think they change the mechanics at all). First, I think a possible use of move constructors/move assignment is to support objects with internal pointers like a small sized optimized vector/string. An example of what a struct like that would look like would be nice. Second, it might be good to include a discussion of how structs are passed at the ABI level (I assume that's in the spec, but it would be helpful context to just highlight in the DIP). My understanding is that this proposal doesn't actually change the function ABIs at all (for struct parameters and return values the caller always passes a pointer to the appropriate structs). At the ABI level, for the move constructor + assignment, the `this` pointer and the pointer to the argument point to distinct structs, and the argument struct will not be used after the function call, correct? To be clear, the value attribute is only used for extern(C++) code where we want to force a copy, right?
Mar 05
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 5 March 2021 at 18:35:39 UTC, Ben Jones wrote:
 Second, it might be good to include a discussion of how structs 
 are passed at the ABI level (I assume that's in the spec, but 
 it would be helpful context to just highlight in the DIP).  My 
 understanding is that this proposal doesn't actually change the 
 function ABIs at all (for struct parameters and return values 
 the caller always passes a pointer to the appropriate structs).
Very small structs are often placed in registers when passed by value. This is true both arguments, and return values. For example, when compiled with LDC for AMD64, the function below does not use any stack memory at all - only registers. struct S { int* ptr; size_t length; } S sliceroo(S s) { const bump = int(s.length >= 2); return S(s.ptr + bump, s.length - bump); } (You can check out the disassembly online with the https://godbolt.org/ Compiler Explorer.) This optimization would need to be disabled for any type with a move constructor.
Mar 05
next sibling parent reply Ben Jones <fake fake.fake> writes:
On Friday, 5 March 2021 at 20:29:01 UTC, tsbockman wrote:
 Very small structs are often placed in registers when passed by 
 value.
 This optimization would need to be disabled for any type with a 
 move constructor.
I'm not sure it would need to be disabled. I think the only part that would really apply here is who has to call the constructor.
Mar 05
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 5 March 2021 at 21:04:23 UTC, Ben Jones wrote:
 On Friday, 5 March 2021 at 20:29:01 UTC, tsbockman wrote:
 Very small structs are often placed in registers when passed 
 by value.
 This optimization would need to be disabled for any type with 
 a move constructor.
I'm not sure it would need to be disabled. I think the only part that would really apply here is who has to call the constructor.
If the main use case for move constructors is to update pointers into a struct instance, how can that work for an instance that no longer has an address in memory at all? The move constructor itself takes its `this` parameter by reference. How would you even call the move constructor?
Mar 05
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Mar 05, 2021 at 09:29:26PM +0000, tsbockman via Digitalmars-d wrote:
 On Friday, 5 March 2021 at 21:04:23 UTC, Ben Jones wrote:
 On Friday, 5 March 2021 at 20:29:01 UTC, tsbockman wrote:
 Very small structs are often placed in registers when passed by
 value.
 This optimization would need to be disabled for any type with a
 move constructor.
I'm not sure it would need to be disabled. I think the only part that would really apply here is who has to call the constructor.
If the main use case for move constructors is to update pointers into a struct instance, how can that work for an instance that no longer has an address in memory at all? The move constructor itself takes its `this` parameter by reference. How would you even call the move constructor?
I think this is confusing semantics with implementation details. An optimizing compiler may decide to keep the struct inside registers but that doesn't mean it can't also allocate the struct on the stack, the location of which will serve as the address of the struct. As long as the registers are written back to the stack at the point where a pointer might be dereferenced, that's good enough. And if the pointer is never actually dereferenced, then the whole thing could be elided completely and the struct will exist only in registers. It's entirely possible that a struct that keeps a pointer to itself with a move ctor that updates the pointer may never actually call the move ctor (e.g., it's used only as a local variable with no escaping references); an optimizing compiler can deduce that no address is actually taken of the struct so it can be held completely in registers. Or there may be some calls to the move ctor implied by the original text of the code, but after eliding some dead code it's determined that the updated pointer never gets read, then the optimizer can just elide the whole thing and enregister the struct. None of this has to do with the semantics of the move ctor specified by the language. The language specifies certain semantics for certain constructs, but the optimizer is free to arbitrarily transform the program, as long as the *overall* semantics do not change. (Case in point: programs that always produce the same output are sometimes optimized into an empty main() by LDC, because none of the implied complex semantics actually matter as far as actual output is concerned. However, this is not in the purview of the language spec.) T -- Customer support: the art of getting your clients to pay for your own incompetence.
Mar 05
prev sibling parent reply Max Haughton <maxhaton gmail.com> writes:
On Friday, 5 March 2021 at 20:29:01 UTC, tsbockman wrote:
 On Friday, 5 March 2021 at 18:35:39 UTC, Ben Jones wrote:
 Second, it might be good to include a discussion of how 
 structs are passed at the ABI level (I assume that's in the 
 spec, but it would be helpful context to just highlight in the 
 DIP).  My understanding is that this proposal doesn't actually 
 change the function ABIs at all (for struct parameters and 
 return values the caller always passes a pointer to the 
 appropriate structs).
Very small structs are often placed in registers when passed by value. This is true both arguments, and return values. For example, when compiled with LDC for AMD64, the function below does not use any stack memory at all - only registers. struct S { int* ptr; size_t length; } S sliceroo(S s) { const bump = int(s.length >= 2); return S(s.ptr + bump, s.length - bump); } (You can check out the disassembly online with the https://godbolt.org/ Compiler Explorer.) This optimization would need to be disabled for any type with a move constructor.
This is true, however it's worth saying that typically the reason for having move semantics isn't so much performance (for small objects) but managing the lifetime of the struct properly. In this case for move semantics to be useful S would probably have a destructor, which prevents passing in registers anyway.
Mar 05
parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 5 March 2021 at 22:04:27 UTC, Max Haughton wrote:
 This is true, however it's worth saying that typically the 
 reason for having move semantics isn't so much performance (for 
 small objects) but managing the lifetime of the struct 
 properly. In this case for move semantics to be useful S would 
 probably have a destructor, which prevents passing in registers 
 anyway.
That is true in C++, that is NOT true in D at the moment. If the proposal ends up forcing passage of non POD by reference rather than value at the ABI level (like in C++), then it's a non starter IMO.
Mar 06
parent Max Haughton <maxhaton gmail.com> writes:
On Saturday, 6 March 2021 at 19:16:51 UTC, deadalnix wrote:
 On Friday, 5 March 2021 at 22:04:27 UTC, Max Haughton wrote:
 This is true, however it's worth saying that typically the 
 reason for having move semantics isn't so much performance 
 (for small objects) but managing the lifetime of the struct 
 properly. In this case for move semantics to be useful S would 
 probably have a destructor, which prevents passing in 
 registers anyway.
That is true in C++, that is NOT true in D at the moment. If the proposal ends up forcing passage of non POD by reference rather than value at the ABI level (like in C++), then it's a non starter IMO.
The point about destructors at least is true at least according to ldc and gdc.
Mar 06
prev sibling next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of Community 
 Review of DIP 1040, "Copying, Moving, and Forwarding":
From the DIP:
 A Move Constructor for struct S is declared as:
     this(S s) { ... }
 A Move Assignment Operator for struct S is declared as:
     void opAssign(S s) { ... }
Is the parameter to these methods really pass-by-value? If so, why??? Making a copy during every move seems unnecessary, and very inefficient. Moreover, for types that define a copy constructor, it could reasonably be argued that calling it OR not calling it BOTH violate the principle of least surprise. I expect moves NOT to call the copy constructor, but I expect pass-by-value WILL call it: a contradiction. If the parameter is, in fact, intended to be pass-by-reference, then I must strenuously object to the chosen syntax. Please do not make this the one-and-only place in the entire language where the pass-by-value syntax secretly means pass-by-reference. The fact that `this(ref typeof(this))` is already taken by copy constructors is NOT a good reason to violate the established semantics of parameter lists in D. Just use a template parameter to distinguish them, or something.
Mar 05
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 5 March 2021 at 23:03:57 UTC, tsbockman wrote:
 On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of Community 
 Review of DIP 1040, "Copying, Moving, and Forwarding":
From the DIP:
 A Move Constructor for struct S is declared as:
     this(S s) { ... }
 A Move Assignment Operator for struct S is declared as:
     void opAssign(S s) { ... }
Is the parameter to these methods really pass-by-value? If so, why???
1) When you pass an rvalue to a by-value parameter in D, it's moved, not copied. 2) If you have a by-ref overload and a by-value overload for the same function, the ref one is called for lvalues and the by-value one is called for rvalues. In fact, this is already how you have to write your opAssign if you want to support explicit move-assignment from rvalues of non-copyable types. See for example SumType's opAssign overload [1], and the corresponding unit test [2]. [1] https://github.com/dlang/phobos/blob/51a70ee267026e150b9b5d81a66ad1fe33112623/std/sumtype.d#L629-L640 [2] https://github.com/dlang/phobos/blob/51a70ee267026e150b9b5d81a66ad1fe33112623/std/sumtype.d#L1141-L1167
Mar 05
parent tsbockman <thomas.bockman gmail.com> writes:
On Friday, 5 March 2021 at 23:32:04 UTC, Paul Backus wrote:
 On Friday, 5 March 2021 at 23:03:57 UTC, tsbockman wrote:
 From the DIP:
 A Move Constructor for struct S is declared as:
     this(S s) { ... }
 A Move Assignment Operator for struct S is declared as:
     void opAssign(S s) { ... }
Is the parameter to these methods really pass-by-value? If so, why???
1) When you pass an rvalue to a by-value parameter in D, it's moved, not copied.
This is good to know, but doesn't solve the problem I asked about; it just makes it more complicated. According to the DIP, lvalues are moved in some circumstances also, not just rvalues. So, the move constructor and move assignment operator taking their argument by value thus imply that every custom move operation *either* infinitely recurses (for rvalues), *or* triggers a redundant copy (for lvalues).
 2) If you have a by-ref overload and a by-value overload for 
 the same function, the ref one is called for lvalues and the 
 by-value one is called for rvalues.
You seem focused on overload selection, but I'm asking what would actually happen when the move operator is called: 1) Does it copy the data to a temporary, or not? 2) Is the address of the move operator's argument the address of the original value to be moved, or not? The currently proposed syntax implies that the answers are (1) yes and (2) no, but I think the answers need to be (1) no and (2) yes for sanity, performance, and flexibility reasons. For example, knowing the source address for the move would allow the move operator to actually tell whether something is an interior pointer that needs to be updated, or not.
 In fact, this is already how you have to write your opAssign if 
 you want to support explicit move-assignment from rvalues of 
 non-copyable types. See for example SumType's opAssign overload 
 [1], and the corresponding unit test [2].
It's the move constructor, which we don't have yet, that implies infinite recursion (https://issues.dlang.org/show_bug.cgi?id=20424). Also, this proposal is about adding proper support for custom move operators to the language, because what we have right now isn't good enough. So, "that's how it works now" isn't a compelling reason to keep doing it that way.
Mar 06
prev sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 5 March 2021 at 23:03:57 UTC, tsbockman wrote:
 On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of Community 
 Review of DIP 1040, "Copying, Moving, and Forwarding":
From the DIP:
 A Move Constructor for struct S is declared as:
     this(S s) { ... }
 A Move Assignment Operator for struct S is declared as:
     void opAssign(S s) { ... }
Is the parameter to these methods really pass-by-value? ... If the parameter is, in fact, intended to be pass-by-reference, then I must strenuously object to the chosen syntax.
Over in the feedback thread, Atila Neves also concluded that the syntax is misleading here: On Wednesday, 10 March 2021 at 21:27:25 UTC, Atila Neves wrote:
 I eventually understood what this meant, but this confused me 
 when I read it the first time. I'd reword it to mention that 
 the syntax looks like a by-value parameter but ends up being 
 passed by reference. It also confused me that the 2nd function 
 had `ref` in there.
Am I the only one who thinks that it would be better to have syntax that accurately reflects the semantics, instead of just documenting "this syntax is a lie"?
Mar 10
next sibling parent reply Max Haughton <maxhaton gmail.com> writes:
On Wednesday, 10 March 2021 at 22:51:58 UTC, tsbockman wrote:
 On Friday, 5 March 2021 at 23:03:57 UTC, tsbockman wrote:
 On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of 
 Community Review of DIP 1040, "Copying, Moving, and 
 Forwarding":
From the DIP:
 A Move Constructor for struct S is declared as:
     this(S s) { ... }
 A Move Assignment Operator for struct S is declared as:
     void opAssign(S s) { ... }
Is the parameter to these methods really pass-by-value? ... If the parameter is, in fact, intended to be pass-by-reference, then I must strenuously object to the chosen syntax.
Over in the feedback thread, Atila Neves also concluded that the syntax is misleading here: On Wednesday, 10 March 2021 at 21:27:25 UTC, Atila Neves wrote:
 I eventually understood what this meant, but this confused me 
 when I read it the first time. I'd reword it to mention that 
 the syntax looks like a by-value parameter but ends up being 
 passed by reference. It also confused me that the 2nd function 
 had `ref` in there.
Am I the only one who thinks that it would be better to have syntax that accurately reflects the semantics, instead of just documenting "this syntax is a lie"?
I'm still fairly open minded about this, however aesthetically at least I really like the idea of move semantics being fairly invisible - e.g. it's not passing a struct by move but rather being up to the struct. When you take into account the object lifetime in a move it's not pass by value of reference. Definitely needs to be carefully considered.
Mar 10
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 10 March 2021 at 23:00:35 UTC, Max Haughton wrote:
 On Wednesday, 10 March 2021 at 22:51:58 UTC, tsbockman wrote:
 On Friday, 5 March 2021 at 23:03:57 UTC, tsbockman wrote:
 On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of 
 Community Review of DIP 1040, "Copying, Moving, and 
 Forwarding":
From the DIP:
 A Move Constructor for struct S is declared as:
     this(S s) { ... }
 A Move Assignment Operator for struct S is declared as:
     void opAssign(S s) { ... }
Am I the only one who thinks that it would be better to have syntax that accurately reflects the semantics, instead of just documenting "this syntax is a lie"?
I'm still fairly open minded about this, however aesthetically at least I really like the idea of move semantics being fairly invisible - e.g. it's not passing a struct by move but rather being up to the struct. When you take into account the object lifetime in a move it's not pass by value of reference.
I agree that move semantics should be fairly invisible outside the custom move operators themselves, but I don't understand how that is an argument either for or against using the `ref` keyword like all other by-reference value type parameters must. Either way the custom move operator (if one is needed) must be defined explicitly, and either way calls to it must be implicitly inserted by the compiler at special points that can only be correctly and completely predicted by understanding the rules in the DIP, and not from standard overload resolution rules.
 Definitely needs to be carefully considered.
Thank you for taking the time to do so. Please consider that the more that surprising special cases like this are added to the language, the harder it gets for non-experts to read and comprehend D code, for newcomers to learn the language, and especially for meta-programmers to write clean and correct code. There is a consequence to adding exception to otherwise simple rules like, "Value type parameters that are passed by reference are labeled `ref` or `out`." In generic code (like most of Phobos) exceptions lead to bugs, bugs lead to frustration, frustration leads to `static if` branches, and too many `static if` branches leads to templates getting down-graded to string mixins (probably still with bugs).
Mar 10
parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Thursday, 11 March 2021 at 00:56:19 UTC, tsbockman wrote:
 On Wednesday, 10 March 2021 at 23:00:35 UTC, Max Haughton wrote:
 On Wednesday, 10 March 2021 at 22:51:58 UTC, tsbockman wrote:
 On Friday, 5 March 2021 at 23:03:57 UTC, tsbockman wrote:
 On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of 
 Community Review of DIP 1040, "Copying, Moving, and 
 Forwarding":
Definitely needs to be carefully considered.
Thank you for taking the time to do so. Please consider that the more that surprising special cases like this are added to the language, the harder it gets for non-experts to read and comprehend D code, for newcomers to learn the language, and especially for meta-programmers to write clean and correct code.
This is important
Mar 11
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 10 March 2021 at 22:51:58 UTC, tsbockman wrote:
 Am I the only one who thinks that it would be better to have 
 syntax that accurately reflects the semantics, instead of just 
 documenting "this syntax is a lie"?
No, I think there is a problem with using opAssign here, because "this" will refers to something that is possibly uninitialized, and the old value may not be consumed fully. than an opAssign. argument. Which leave me with the one conclusion: the postblit needs to be recycled into a move constructor.
Mar 10
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 11 March 2021 at 00:31:01 UTC, deadalnix wrote:
 On Wednesday, 10 March 2021 at 22:51:58 UTC, tsbockman wrote:
 Am I the only one who thinks that it would be better to have 
 syntax that accurately reflects the semantics, instead of just 
 documenting "this syntax is a lie"?
No, I think there is a problem with using opAssign here, because "this" will refers to something that is possibly uninitialized, and the old value may not be consumed fully. than an opAssign.
Yeah, studying the DIP I can't figure out what problem the move `opAssign` is supposed to solve that the constructor doesn't: https://forum.dlang.org/post/kzgybicwqwlfyiiefucc forum.dlang.org

 as argument.
I think forbidding the move constructor from explicitly accessing the source at all is overly restrictive for a systems programming language. Full access should still be available, even if it can't be compiler-verified safe.
 Which leave me with the one conclusion: the postblit needs to 
 be recycled into a move constructor.
Since the `this(this)` syntax of the postblit is unique, that would be a valid solution to my concern about misleading syntax.
Mar 10
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/10/2021 5:27 PM, tsbockman wrote:
 On Thursday, 11 March 2021 at 00:31:01 UTC, deadalnix wrote:
 On Wednesday, 10 March 2021 at 22:51:58 UTC, tsbockman wrote:
 Am I the only one who thinks that it would be better to have syntax that 
 accurately reflects the semantics, instead of just documenting "this syntax 
 is a lie"?
No, I think there is a problem with using opAssign here, because "this" will refers to something that is possibly uninitialized, and the old value may not be consumed fully.
opAssign is only for assigning to initialized objects. Constructors are for uninitialized objects.
 Yeah, studying the DIP I can't figure out what problem the move `opAssign` is 
 supposed to solve that the constructor doesn't:
 https://forum.dlang.org/post/kzgybicwqwlfyiiefucc forum.dlang.org
The thing about "destroy after move" is to deal with the case of both the source and the destination referring to the same object. The concern is that destroying the destination's original contents first will destroy them for the source before it gets moved in. It's the same problem "swap" has. It's also necessary semantics for a reference counted object.
Mar 10
next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 11 March 2021 at 03:33:31 UTC, Walter Bright wrote:
 On 3/10/2021 5:27 PM, tsbockman wrote:
 Yeah, studying the DIP I can't figure out what problem the 
 move `opAssign` is supposed to solve that the constructor 
 doesn't:
 https://forum.dlang.org/post/kzgybicwqwlfyiiefucc forum.dlang.org
The thing about "destroy after move" is to deal with the case of both the source and the destination referring to the same object. The concern is that destroying the destination's original contents first will destroy them for the source before it gets moved in.
Wouldn't it make more sense to just skip the move operation entirely when the source and destination are the same? Or are there circumstances in which that cannot be determined, even at runtime? Also, doesn't this mean that every move of an object with a destructor requires a copy? Does the copy constructor get called in such cases? If so, what benefit do moves have over copies? If not, how do you know you're not breaking assumptions that the destructor depends upon for correct functioning by making this the one circumstance in which a value may be copied without calling its copy constructor? Which function sees the "real" address of the object: the move operator, or the destructor?
Mar 10
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/10/2021 8:17 PM, tsbockman wrote:
 On Thursday, 11 March 2021 at 03:33:31 UTC, Walter Bright wrote:
 On 3/10/2021 5:27 PM, tsbockman wrote:
 Yeah, studying the DIP I can't figure out what problem the move `opAssign` is 
 supposed to solve that the constructor doesn't:
 https://forum.dlang.org/post/kzgybicwqwlfyiiefucc forum.dlang.org
The thing about "destroy after move" is to deal with the case of both the source and the destination referring to the same object. The concern is that destroying the destination's original contents first will destroy them for the source before it gets moved in.
Wouldn't it make more sense to just skip the move operation entirely when the source and destination are the same? Or are there circumstances in which that cannot be determined, even at runtime?
The idea is that the move assignment operation takes care of this, and makes it "as if" the move was done, then the destruction.
Mar 11
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 11 March 2021 at 08:36:18 UTC, Walter Bright wrote:
 On 3/10/2021 8:17 PM, tsbockman wrote:
 Wouldn't it make more sense to just skip the move operation 
 entirely when the source and destination are the same? Or are 
 there circumstances in which that cannot be determined, even 
 at runtime?
The idea is that the move assignment operation takes care of this, and makes it "as if" the move was done, then the destruction.
Is it the responsibility of the explicit body of the custom move assignment operator to do the destruction, or the responsibility of the compiler to insert a call to the destructor somewhere? If the compiler inserts the call, is this done inside the move assignment operator, or at each call site? My concern is how to make code like this work correctly: ///////////////////////////////////////////////// module app; ptrdiff_t netAllocCount = 0; void* malloc(size_t size) system nothrow nogc { import core.stdc.stdlib : malloc; void* ptr = malloc(size); netAllocCount += int(ptr !is null); return ptr; } void free(void* ptr) system nothrow nogc { import core.stdc.stdlib : free; netAllocCount -= int(ptr !is null); free(ptr); } struct S { private: struct Data { uint data; int refCount = 1; } Data* ptr; Data internal; public: bool isUnique() const pure safe nothrow nogc { return (ptr is &internal); } ref inout(uint) data() return inout pure safe nothrow nogc { return ptr.data; } // normal construction and assignment this(bool unique) trusted nothrow nogc { construct(unique); } private void construct(bool unique) trusted nothrow nogc { pragma(inline, false); // Don't let inlining make things work by accident. if(unique) { ptr = &internal; internal = Data.init; } else { ptr = cast(Data*) malloc(size_t.sizeof * 2); (*ptr) = Data.init; } } ref typeof(this) opAssign(bool unique) return safe nothrow nogc { destruct(this); construct(unique); return this; } // copy construction disable this(this); this(ref typeof(this) source) pure trusted nothrow nogc { if(source.isUnique) { ptr = &internal; internal = source.internal; } else { ptr = source.ptr; ptr.refCount += 1; } } // move construction and assignment (using the DIP's misleading syntax): this(S source) pure trusted nothrow nogc { if(source.isUnique) { // This works, because source is really by reference. ptr = &internal; internal = source.internal; } else ptr = source.ptr; } void opAssign(S source) trusted nothrow nogc { // What goes here, and what is the full lowering of a move assignment call? } // destructor ~this() safe nothrow nogc { destruct(this); } private void destruct(ref S s) trusted nothrow nogc { pragma(inline, false); // Don't let inlining make things work by accident. if(!s.isUnique) { s.ptr.refCount -= 1; if(s.ptr.refCount <= 0) free(s.ptr); } s.ptr = null; } } void main() safe { import std.stdio : writeln; { S a = true, b = false, c = b; a.data = 1; b.data = 2; a = b; c.data = 3; assert(a.data == 3); } assert(netAllocCount == 0); } ///////////////////////////////////////////////// Syntax aside, I think the way it should be done is this: // Lower the `a = b;` line in main, above, to this: if(&b !is &a) { // Moving a variable into itself is a no-op. destroy(a); /* Just call the move constructor; there is no point to the move assignment operator as its body would be identical: */ a.__ctor(b); } I've tested this algorithm, and it seems to work as intended (although of course it has to be expressed somewhat differently without the DIP). But, that's clearly not what you have in mind. I just can't figure out what you think the alternative is: // Maybe this is your proposed lowering for the `a = b;` line in main, above? { S oldDest = a; // You all say no copy constructor call is necessary here, but... a.opAssign(b); // (Body is the same as that of the move constructor.) // ...the implicit destroy(oldDest); at scope exit crashes if oldDest was only blit. } You can put the lowering logic inside of the move opAssign instead, but it doesn't change the fundamental problem. If no actual general-purpose lowering is possible that accurately reflects the intended semantics of, "After the move is complete, the destructor is called on the original contents of the constructed object," then the DIP's description of the semantics is simply incorrect and should be replaced with something more rigorous.
Mar 11
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 11 March 2021 at 21:45:49 UTC, tsbockman wrote:
 On Thursday, 11 March 2021 at 08:36:18 UTC, Walter Bright wrote:
 On 3/10/2021 8:17 PM, tsbockman wrote:
 Wouldn't it make more sense to just skip the move operation 
 entirely when the source and destination are the same? Or are 
 there circumstances in which that cannot be determined, even 
 at runtime?
The idea is that the move assignment operation takes care of this, and makes it "as if" the move was done, then the destruction.
My concern is how to make code like this work correctly: ... // Maybe this is your proposed lowering for the `a = b;` line in main, above? { S oldDest = a; // You all say no copy constructor call is necessary here, but... a.opAssign(b); // (Body is the same as that of the move constructor.) // ...the implicit destroy(oldDest); at scope exit crashes if oldDest was only blit. } You can put the lowering logic inside of the move opAssign instead, but it doesn't change the fundamental problem. If no actual general-purpose lowering is possible that accurately reflects the intended semantics of, "After the move is complete, the destructor is called on the original contents of the constructed object," then the DIP's description of the semantics is simply incorrect and should be replaced with something more rigorous.
I think I finally figure out how to make some sense out of the DIP's description. However, the lowering cannot be expressed clearly with the DIP's syntax, so I will use an alternative notation: void moveConstruct(ref S source) nothrow nogc { if(source.isUnique) { ptr = &internal; internal = source.internal; } else ptr = source.ptr; } void moveAssign(ref S source) trusted nothrow nogc { S oldDest = void; oldDest.moveConstruct(this); // Move the old value to a temporary. moveConstruct(source); // Implicitly destroy the old value. } Is this correct? If so, the DIP really needs to explain it more clearly - especially if the user is expected to implement some equivalent in the custom move operator himself, rather than the compiler doing it for him.
Mar 11
next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 12 March 2021 at 01:40:22 UTC, tsbockman wrote:
 On Thursday, 11 March 2021 at 21:45:49 UTC, tsbockman wrote:
 You can put the lowering logic inside of the move opAssign 
 instead, but it doesn't change the fundamental problem. If no 
 actual general-purpose lowering is possible that accurately 
 reflects the intended semantics of, "After the move is 
 complete, the destructor is called on the original contents of 
 the constructed object," then the DIP's description of the 
 semantics is simply incorrect and should be replaced with 
 something more rigorous.
I think I finally figure out how to make some sense out of the DIP's description. However, the lowering cannot be expressed clearly with the DIP's syntax, so I will use an alternative notation: void moveConstruct(ref S source) nothrow nogc { if(source.isUnique) { ptr = &internal; internal = source.internal; } else ptr = source.ptr; } void moveAssign(ref S source) trusted nothrow nogc { S oldDest = void; oldDest.moveConstruct(this); // Move the old value to a temporary. moveConstruct(source); // Implicitly destroy the old value. } Is this correct? If so, the DIP really needs to explain it more clearly - especially if the user is expected to implement some equivalent in the custom move operator himself, rather than the compiler doing it for him.
Updated runnable example, for context: https://gist.github.com/run-dlang/c5805ebb9e9b9734e032ca5e81fcfa90 This version seems to work correctly with either lowering enabled.
Mar 11
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 12 March 2021 at 01:43:01 UTC, tsbockman wrote:
 On Friday, 12 March 2021 at 01:40:22 UTC, tsbockman wrote:
 I think I finally figure out how to make some sense out of the 
 DIP's description. However, the lowering cannot be expressed 
 clearly with the DIP's syntax, so I will use an alternative 
 notation:

 void moveConstruct(ref S source) nothrow  nogc {
     if(source.isUnique) {
         ptr = &internal;
         internal = source.internal;
     } else
         ptr = source.ptr;
 }
 void moveAssign(ref S source)  trusted nothrow  nogc {
     S oldDest = void;
     oldDest.moveConstruct(this); // Move the old value to a 
 temporary.
     moveConstruct(source);
     // Implicitly destroy the old value.
 }

 Is this correct?
Nope, still not correct. I had a bug in the surrounding context, and wasn't testing enough things. Here's another attempt that passes a more thorough test: https://gist.github.com/run-dlang/b789714c01905f091a44ee2666276433 The important bit: /* move construction and assignment (these must be called manually and do not use the DIP syntax, since it's not implemented yet): */ void moveConstruct(ref S source) system nothrow nogc { // system since this must not be called by itself on an already-initialized object. if(source.isUnique) { ptr = &internal; internal = source.internal; } else ptr = source.ptr; source.ptr = null; } void moveAssign(ref S source) trusted nothrow nogc { static if(useDIPLowering) { // destroy after (the DIP's proposal): S newVal = void; newVal.moveConstruct(source); S oldVal = void; oldVal.moveConstruct(this); moveConstruct(newVal); // Implicitly destruct(oldVal). } else { // conditionally move and destroy before (my proposal): if(&source !is &this) { destruct(this); moveConstruct(source); } } } Key changes: the move constructor must put the source into a state where the destructor is a no-op, and for the move assignment operation to destroy the old value *after* the move, as required by the DIP, TWO extra moves are required. That is a lot of extra work and confusion just to avoid explicitly checking if the source and destination are the same. This seems especially silly given that the optimizer can probably detect the move-to-self case at compile time in many cases, and eliminate either the test, or the entire move during compilation. Is there some other motivation for destroying after, rather than before, besides the self-move case?
Mar 11
parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Friday, 12 March 2021 at 06:52:09 UTC, tsbockman wrote:
 On Friday, 12 March 2021 at 01:43:01 UTC, tsbockman wrote:
 On Friday, 12 March 2021 at 01:40:22 UTC, tsbockman wrote:
 [...]
Nope, still not correct. I had a bug in the surrounding context, and wasn't testing enough things. Here's another attempt that passes a more thorough test: https://gist.github.com/run-dlang/b789714c01905f091a44ee2666276433 The important bit: /* move construction and assignment (these must be called manually and do not use the DIP syntax, since it's not implemented yet): */ void moveConstruct(ref S source) system nothrow nogc { // system since this must not be called by itself on an already-initialized object. if(source.isUnique) { ptr = &internal; internal = source.internal; } else ptr = source.ptr; source.ptr = null; } void moveAssign(ref S source) trusted nothrow nogc { static if(useDIPLowering) { // destroy after (the DIP's proposal): S newVal = void; newVal.moveConstruct(source); S oldVal = void; oldVal.moveConstruct(this); moveConstruct(newVal); // Implicitly destruct(oldVal). } else { // conditionally move and destroy before (my proposal): if(&source !is &this) { destruct(this); moveConstruct(source); } } } Key changes: the move constructor must put the source into a state where the destructor is a no-op, and for the move assignment operation to destroy the old value *after* the move, as required by the DIP, TWO extra moves are required. That is a lot of extra work and confusion just to avoid explicitly checking if the source and destination are the same. This seems especially silly given that the optimizer can probably detect the move-to-self case at compile time in many cases, and eliminate either the test, or the entire move during compilation. Is there some other motivation for destroying after, rather than before, besides the self-move case?
Not trying to be rude, but it's a bit worrying that even those who know D really well have a hard time understanding the DIP... (I don't count myself as one of them yet) Maybe it should provide even more "end-to-end" code examples to show how all parts work? Maybe we're over-analyzing things, idk, but I guess we want to be sure everything is 100% correct and can be explained (relatively easy) to a certain percentage of the community. The discussion implies there are parts not fully understood and that makes people nervous.
Mar 12
next sibling parent reply Max Haughton <maxhaton gmail.com> writes:
On Friday, 12 March 2021 at 11:35:03 UTC, Imperatorn wrote:
 On Friday, 12 March 2021 at 06:52:09 UTC, tsbockman wrote:
 [...]
Not trying to be rude, but it's a bit worrying that even those who know D really well have a hard time understanding the DIP... (I don't count myself as one of them yet) Maybe it should provide even more "end-to-end" code examples to show how all parts work? Maybe we're over-analyzing things, idk, but I guess we want to be sure everything is 100% correct and can be explained (relatively easy) to a certain percentage of the community. The discussion implies there are parts not fully understood and that makes people nervous.
The DIP will be fleshed out more. However I should stress that the DIP should be viewed as a piece of legalese rather than a sales pitch as per se - i.e. this isn't the place to be talking about pedagogy
Mar 12
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Friday, 12 March 2021 at 11:47:49 UTC, Max Haughton wrote:
 [snip]

 The DIP will be fleshed out more. However I should stress that 
 the DIP should be viewed as a piece of legalese rather than a 
 sales pitch as per se - i.e. this isn't the place to be talking 
 about pedagogy
I think Herb Sutter's C++ proposals do a good job of doing both. For instance, the metaclasses proposal [1] contains high level explanations and simple examples, but also gets into more detail (maybe less of the legalese in that particular one though). I think the legalese is important, but communicating why the idea is important and how it should be used (and useful) in simple cases is not something that should be discounted as just a "sales pitch" or "pedagogy", IMO. [1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0707r0.pdf
Mar 12
prev sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Friday, 12 March 2021 at 11:47:49 UTC, Max Haughton wrote:
 On Friday, 12 March 2021 at 11:35:03 this isn't the place to be 
 talking about pedagogy
So what's the proper place?
Mar 12
prev sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Friday, 12 March 2021 at 11:35:03 UTC, Imperatorn wrote:

 The discussion implies there are parts not fully understood and 
 that makes people nervous.
That's why we're here. The point of the DIP review process is to improve the DIP. Does it need more examples? Does it need more clarity? Did the author miss something important? Discussion threads go in all sorts of directions. Ultimately, the author will decide whether or not changes are warranted based on the discussion here and the comments in the Feedback Thread. Very rare is the DIP that gets through the process without revision.
Mar 12
parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Friday, 12 March 2021 at 12:18:56 UTC, Mike Parker wrote:
 On Friday, 12 March 2021 at 11:35:03 UTC, Imperatorn wrote:

 The discussion implies there are parts not fully understood 
 and that makes people nervous.
That's why we're here. The point of the DIP review process is to improve the DIP. Does it need more examples? Does it need more clarity? Did the author miss something important? Discussion threads go in all sorts of directions. Ultimately, the author will decide whether or not changes are warranted based on the discussion here and the comments in the Feedback Thread. Very rare is the DIP that gets through the process without revision.
👍
Mar 12
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/11/2021 5:40 PM, tsbockman wrote:
 I think I finally figure out how to make some sense out of the DIP's 
 description. However, the lowering cannot be expressed clearly with the DIP's 
 syntax, so I will use an alternative notation:
 
 void moveConstruct(ref S source) nothrow  nogc {
      if(source.isUnique) {
          ptr = &internal;
          internal = source.internal;
      } else
          ptr = source.ptr;
 }
 void moveAssign(ref S source)  trusted nothrow  nogc {
      S oldDest = void;
      oldDest.moveConstruct(this); // Move the old value to a temporary.
      moveConstruct(source);
      // Implicitly destroy the old value.
 }
 
 Is this correct? If so, the DIP really needs to explain it more clearly - 
 especially if the user is expected to implement some equivalent in the custom 
 move operator himself, rather than the compiler doing it for him.
The idea of the move constructor is the user takes over the task of doing whatever it takes to make it work, i.e. handle the destruction. Think of it this way. You start with two objects, A and B. Moving B into the storage occupied by A must result in the destruction of the original object occupying A, and whatever is in B after the move is garbage, and what was in B is now in A. Everything follows from that.
Mar 14
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Monday, 15 March 2021 at 05:45:01 UTC, Walter Bright wrote:
 On 3/11/2021 5:40 PM, tsbockman wrote:
 Is this correct?
The idea of the move constructor is the user takes over the task of doing whatever it takes to make it work, i.e. handle the destruction.
That's not what I was asking about. I understand what a move *constructor* does. Rather, I was trying to figure out, "How can your definition of move *assignment* be represented in code?" You tried to answer this, my actual question, earlier, but your answer was just as vague as the DIP: From the DIP:
 After the move is complete, the destructor is called on the
 original contents of the constructed object.
Since the move overwrites the original contents, it's not immediately obvious how the destructor can be called on those original contents, which are gone, *after* the move. On Thursday, 11 March 2021 at 08:36:18 UTC, Walter Bright wrote:
 The idea is that the move assignment operation takes care of
 this, and makes it "as if" the move was done, then the
 destruction.
If the algorithm is unclear or missing steps, saying, "You don't actually have to do it, you just have to do something equivalent," doesn't help. A clear, complete, and logically valid algorithm needs to be defined *first*, to determine whether another algorithm is equivalent. I was trying to figure out what your original, general-purpose move *assignment* algorithm could be. That's what my code example is for. I found a candidate that seems to work, and to fit your description (`this` and `source` are both passed in by reference): 0) Move-construct the `source` into a temporary `newValue`. 1) Clear `source` in such a way that destroying its value would be a no-op. 2) Move-construct the original value of the destination `this` into a temporary `oldValue`. 3) Move-construct `newValue` into the destination `this`. 4) Destroy `oldValue`. This is complicated, but everything simpler I tried either doesn't match your description, or doesn't pass my tests. Runnable example: https://gist.github.com/run-dlang/b789714c01905f091a44ee2666276433 Is this correct?
Mar 15
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/15/2021 1:09 PM, tsbockman wrote:
 I was trying to figure out what your original, general-purpose move
*assignment* 
 algorithm could be. That's what my code example is for.
The answer is, it depends on how the programmer set up the contents of the object, in particular, how the ownership works. It's up to the programmer to make it work so the user of the type sees the semantics as described. For example, if it's pointing to unique objects, then the destructor for the destination is called first. If it's pointing to ref counted objects, then the destructor is called last. The language doesn't specify that. The programmer does, and the programmer needs to make it work. Hence the "as if" rule.
Mar 16
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Tuesday, 16 March 2021 at 08:22:07 UTC, Walter Bright wrote:
 On 3/15/2021 1:09 PM, tsbockman wrote:
 I was trying to figure out what your original, general-purpose 
 move *assignment* algorithm could be. That's what my code 
 example is for.
The answer is, it depends on how the programmer set up the contents of the object, in particular, how the ownership works. It's up to the programmer to make it work so the user of the type sees the semantics as described.
The semantics as described in the DIP are unsound. You've left out some implied extra step or assumption.
 For example, if it's pointing to unique objects, then the 
 destructor for the destination is called first. If it's 
 pointing to ref counted objects, then the destructor is called 
 last.
When the source and destination refer to the same object, that object still ends up either destroyed, or in an invalid state, regardless of which order a single move and destruction are performed. You stated earlier that making such self-moves work correctly is the specific motivation for the current wording in the DIP, but it clearly doesn't do that in either of your examples. I've tested this with actual runnable code, so please stop responding like I just don't get it, unless you're going to critique the code example itself.
 The language doesn't specify that. The programmer does, and the 
 programmer needs to make it work. Hence the "as if" rule.
An algorithm that works "as if" it prematurely destroys the object in self-moves, must still prematurely destroy the object in self moves. The "as if" rule allows different implementations, not different results. You just have different results in mind from what your wrote in the DIP.
Mar 16
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/16/2021 11:11 AM, tsbockman wrote:
 You just have different results in mind from what your wrote in the DIP.
The desired result is clear - two objects exist before the move assignment, and one afterwards. If both objects are the same instance, then it should be a no-op. The wording could be improved - want to make a stab at it?
Mar 18
parent tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 18 March 2021 at 10:01:12 UTC, Walter Bright wrote:
 On 3/16/2021 11:11 AM, tsbockman wrote:
 You just have different results in mind from what your wrote 
 in the DIP.
The desired result is clear - two objects exist before the move assignment, and one afterwards. If both objects are the same instance, then it should be a no-op. The wording could be improved - want to make a stab at it?
Sure. Old wording:
 A Move Assignment Operator is a struct member assignment 
 operator
 that moves, rather than copies, the argument corresponding to 
 its
 first parameter into the constructed object. After the move is
 complete, the destructor is called on the original contents of
 the constructed object. The argument is invalid after this move,
 and is not destructed.
Proposed new wording:
 A Move Assignment Operator is a struct member assignment 
 operator
 that moves, rather than copies, the source argument into the
 destination argument. The source argument corresponds to the 
 first
 parameter, and the destination argument corresponds to the 
 implicit
 `this` parameter.

 If both arguments refer to the same object, the Move Assignment
 Operator shall have no effect, and that object remains valid.

 Otherwise, the effect shall be to destroy the old contents of 
 the
 destination and to move the contents of the source into the
 destination. The source is invalid after the Move Assignment
 Operator returns, and is not destroyed.

 The Move Assignment Operator implementation may perform the
 destruction and move in any order, however it must ensure that
 the source is not destroyed when the old contents of the
 destination are destroyed, even if the source is indirectly 
 owned
 by the old contents of the destination.
The final paragraph is the tricky part: I thought about the "destroy after" thing some more, and realized that while it is neither necessary nor helpful when the source and destination refer to the exact same object, it *is* helpful when the old destination may *indirectly* own the source. For example, what if there is a singly-linked list where each `next` link is a unique smart pointer? When removing the current `head` of the list, we may wish to move `head.next` into `head`. If `head` is destroyed first, it will wrongly destroy the rest of the list before the move is complete, including `head.next`. However, requiring that the destruction of the old destination be performed after the move of source to destination doesn't fix this problem by itself; an additional step is required: set `source` to `null` as part of the move. So, requiring a specific order is undesirable, because it doesn't fix the problem without also requiring the existence of a `null` state for every movable type. (My earlier code example uses a `null` state.) Also, requiring a specific order is undesirable because it is not the only possible solution to the indirect ownership problem; for example, the move assignment operator could scan the old destination's indirections and just skip destroying any that would invalidate the source. Thus, I propose we formally leave the details up to each move assignment operator's implementer, and limit the language spec to forbidding the problem itself, rather than require semantic equivalence to a specific solution. Types without problematic indirections can destroy before, types with problematic indirections and a `null` state can destroy after, and those rare types that don't fit into either category can do something more creative.
Mar 18
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 March 2021 at 04:17:48 UTC, tsbockman wrote:
 Also, doesn't this mean that every move of an object with a 
 destructor requires a copy?
No, only if said object was initialized first.
 Does the copy constructor get  called in such cases?
Why? That would not allow for the removal of the destruction, in fact now you'd have 2 object to destroy instead of 1.
 If so, what benefit do moves have over copies?
They do not create a new object to destroy.
Mar 11
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 11 March 2021 at 10:09:25 UTC, deadalnix wrote:
 On Thursday, 11 March 2021 at 04:17:48 UTC, tsbockman wrote:
 Also, doesn't this mean that every move of an object with a 
 destructor requires a copy?
No, only if said object was initialized first.
 Does the copy constructor get  called in such cases?
Why? That would not allow for the removal of the destruction, in fact now you'd have 2 object to destroy instead of 1.
 If so, what benefit do moves have over copies?
They do not create a new object to destroy.
I have written an example to clarify what I'm asking about: https://forum.dlang.org/post/igagderdongxdhvfjihj forum.dlang.org
Mar 11
parent tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 11 March 2021 at 21:48:44 UTC, tsbockman wrote:
 I have written an example to clarify what I'm asking about:
     
 https://forum.dlang.org/post/igagderdongxdhvfjihj forum.dlang.org
Here's a version that can be compiled and successfully run with DMD today, without the DIP, in case anyone else wants to play around with it: https://gist.github.com/run-dlang/247bea2c2815e794a1a52012bc70ef9f Set `enum useDIPLowering = true;` at the top to see the problem I'm poking at.
Mar 11
prev sibling next sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Thursday, 11 March 2021 at 03:33:31 UTC, Walter Bright wrote:
 The thing about "destroy after move" is to deal with the case 
 of both the source and the destination referring to the same 
 object. The concern is that destroying the destination's 
 original contents first will destroy them for the source before 
 it gets moved in.
I thought of a much simpler way to demonstrate why the DIP's proposed move-assignment semantics are unsound:
 After the move is complete, the destructor is called on the
 original contents of the constructed object. The argument is
 invalid after this move, and is not destructed.
In the self-move case, "the original contents of the constructed object", "the argument", and the destination are all the same thing. So, the DIP requires this one object to be: 1) valid (because it is the destination) 2) "invalid" (because it is "the argument") 3) "not destructed" (because it is "the argument"), and 4) destructed (because it is "the original contents of the constructed object") This is a severe contradiction that needs to be addressed, not dismissed with the magic words "as if". The simple solution is to formally require that self-moves have no effect.
Mar 16
prev sibling parent reply Q. Schroll <qs.il.paperinik gmail.com> writes:
On Thursday, 11 March 2021 at 03:33:31 UTC, Walter Bright wrote:
 On 3/10/2021 5:27 PM, tsbockman wrote:
 On Thursday, 11 March 2021 at 00:31:01 UTC, deadalnix wrote:
 On Wednesday, 10 March 2021 at 22:51:58 UTC, tsbockman wrote:
 Am I the only one who thinks that it would be better to have 
 syntax that accurately reflects the semantics, instead of 
 just documenting "this syntax is a lie"?
No, I think there is a problem with using opAssign here, because "this" will refers to something that is possibly uninitialized, and the old value may not be consumed fully. than an opAssign.
opAssign is only for assigning to initialized objects. Constructors are for uninitialized objects.
 Yeah, studying the DIP I can't figure out what problem the 
 move `opAssign` is supposed to solve that the constructor 
 doesn't:
 https://forum.dlang.org/post/kzgybicwqwlfyiiefucc forum.dlang.org
The thing about "destroy after move" is to deal with the case of both the source and the destination referring to the same object. The concern is that destroying the destination's original contents first will destroy them for the source before it gets moved in. It's the same problem "swap" has. It's also necessary semantics for a reference counted object.
It might be a stupid question, but why have move assignment in the first place? In C++, there's the copy-and-swap idiom[1]. Maybe it's obvious why it does not apply in D, but if using a swap function makes implementing a copy assignment and move assignment trivial, why not requiring opSwap instead of opAssign for an elaborate move object? Basically, opSwap takes a typeof(this) lvalue (by reference), well, swaps contents with `this`. Usually, this means swapping all member variables (can be auto-generated easily). Then, ref typeof(this) opAssign(typeof(this) rhs) { this.opSwap(rhs); return this; } does the deed. Note that the call to opAssign is to be treated like any old member function call. If the argument is an rvalue (or by the DIP the last use of an lvalue), the move constructor is used to initialize the parameter. Otherwise the copy constructor is used to initialize the parameter. This is not a "let's do it like C++ guides" but rather "let's not repeat the mistakes C++ made". Because in C++, if one don't know copy-and-swap, one's copy/move assignment operator is probably worse than the copy-and-swap one. Note that in C++, too, copy-and-swap only applies to elaborate move objects. [1] https://stackoverflow.com/questions/3279543/what-is-the-copy-and-swap-idiom
Mar 17
parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 17 March 2021 at 17:14:04 UTC, Q. Schroll wrote:
 It might be a stupid question, but why have move assignment in 
 the first place? In C++, there's the copy-and-swap idiom[1]. 
 Maybe it's obvious why it does not apply in D, but if using a 
 swap function makes implementing a copy assignment and move 
 assignment trivial, why not requiring opSwap instead of 
 opAssign for an elaborate move object?
This isn't a stupid question, this is THE question. It is easy to assume things are necessary because other went there and did it, but I find that questioning these assumptions is how the greatest design ideas came up. D's pure attribute is the perfect example of this.
 [1] 
 https://stackoverflow.com/questions/3279543/what-is-the-copy-and-swap-idiom
Doing this has major issue: it require all movable structs to have a null state (as in C++) or make other unsavory tradeofs (see https://forum.dlang.org/post/bkfqchwpnonngjrtybbe forum.dlang.org for a more thorough explanation). Nevertheless, if the struct naturally has a null state, this is indeed a very good way to do it.
Mar 18
parent Q. Schroll <qs.il.paperinik gmail.com> writes:
On Thursday, 18 March 2021 at 19:53:49 UTC, deadalnix wrote:
 On Wednesday, 17 March 2021 at 17:14:04 UTC, Q. Schroll wrote:
 It might be a stupid question, but why have move assignment in 
 the first place? In C++, there's the copy-and-swap idiom[1]. 
 Maybe it's obvious why it does not apply in D, but if using a 
 swap function makes implementing a copy assignment and move 
 assignment trivial, why not requiring opSwap instead of 
 opAssign for an elaborate move object?
This isn't a stupid question, this is THE question. It is easy to assume things are necessary because other went there and did it, but I find that questioning these assumptions is how the greatest design ideas came up.
My suggestion isn't really don't do ever implement opAssign (the copy one and the move one) manually, but rather: Leave it up to the compiler, except you know you really cannot use copy and swap.
 Doing this has major issue: it require all movable structs to 
 have a null state (as in C++) or make other unsavory tradeofs 
 (see 
 https://forum.dlang.org/post/bkfqchwpnonngjrtybbe forum.dlang.org for a more
thorough explanation).

 Nevertheless, if the struct naturally has a null state, this is 
 indeed a very good way to do it.
I was never sure if that was a good or bad decision D made, but doesn't D require every type to have a null value (.init)? Or do you mean something else?
Mar 18
prev sibling next sibling parent reply Arafel <er.krali gmail.com> writes:
On 10/3/21 23:51, tsbockman wrote:
 On Friday, 5 March 2021 at 23:03:57 UTC, tsbockman wrote:
 On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of Community Review 
 of DIP 1040, "Copying, Moving, and Forwarding":
From the DIP:
 A Move Constructor for struct S is declared as:
     this(S s) { ... }
 A Move Assignment Operator for struct S is declared as:
     void opAssign(S s) { ... }
Is the parameter to these methods really pass-by-value? ... If the parameter is, in fact, intended to be pass-by-reference, then I must strenuously object to the chosen syntax.
Over in the feedback thread, Atila Neves also concluded that the syntax is misleading here: On Wednesday, 10 March 2021 at 21:27:25 UTC, Atila Neves wrote:
 I eventually understood what this meant, but this confused me when I 
 read it the first time. I'd reword it to mention that the syntax looks 
 like a by-value parameter but ends up being passed by reference. It 
 also confused me that the 2nd function had `ref` in there.
Am I the only one who thinks that it would be better to have syntax that accurately reflects the semantics, instead of just documenting "this syntax is a lie"?
There is another issue with the proposed semantics, unless I'm missing something. How can I implement both an "identity assignment operator" [1] and a "move assignment operator"? The proposed syntax is co-opting an existing pattern for a different use case. Let's say I have a struct that includes an associative array, where I implement deep copy for the assignment operator. Will in this case my deep copy be reused automatically for movement? That's obviously not what I want, for that I'd just want to copy the reference to the existing AA. If anything, this should be added to the breaking changes and deprecations, or at least mentioned as something to check for. I could have used a `ref` parameter, but this wouldn't work with rvalues, and in any case the syntax is currently allowed, unlike constructors (bug 20424 [2]) which are mentioned in the DIP... although [1]: https://dlang.org/spec/operatoroverloading.html#assignment [2]: https://issues.dlang.org/show_bug.cgi?id=20424
Mar 10
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/10/2021 11:44 PM, Arafel wrote:
 There is another issue with the proposed semantics, unless I'm missing
something.
 
 How can I implement both an "identity assignment operator" [1] and a "move 
 assignment operator"? The proposed syntax is co-opting an existing pattern for
a 
 different use case.
 
 Let's say I have a struct that includes an associative array, where I
implement 
 deep copy for the assignment operator.
 
 Will in this case my deep copy be reused automatically for movement? That's 
 obviously not what I want, for that I'd just want to copy the reference to the 
 existing AA.
 
 If anything, this should be added to the breaking changes and deprecations, or 
 at least mentioned as something to check for.
 
 I could have used a `ref` parameter, but this wouldn't work with rvalues, and
in 
 any case the syntax is currently allowed, unlike constructors (bug 20424 [2]) 
 which are mentioned in the DIP... although even there I tend to agree with 

 
 [1]: https://dlang.org/spec/operatoroverloading.html#assignment
 [2]: https://issues.dlang.org/show_bug.cgi?id=20424
Constructing from an rvalue essentially is move construction.
Mar 11
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/11/2021 12:42 AM, Walter Bright wrote:
 Constructing from an rvalue essentially is move construction.
I forgot to mention that the new semantics only apply to EMO objects, which require both a move constructor and a move assignment operator. The move constructor is new syntax. Therefore, it shouldn't break existing code.
Mar 11
parent reply Arafel <er.krali gmail.com> writes:
On 11/3/21 10:01, Walter Bright wrote:
 On 3/11/2021 12:42 AM, Walter Bright wrote:
 Constructing from an rvalue essentially is move construction.
I forgot to mention that the new semantics only apply to EMO objects, which require both a move constructor and a move assignment operator. The move constructor is new syntax. Therefore, it shouldn't break existing code.
But we also read in the DIP, as it has already been mentioned:
 If a Move Constructor is not defined for a struct that has a Move Assignment
Operator, a default Move Constructor is defined and implemented as a move for
each of its fields, in lexical order.
It's not clear if a struct would be considered an EMO if either the Move Assignment Operator or the Move Constructor are defined by default. If that's the case, any struct with an identity assignment operator would be silently "upgraded" to EMO, thus potentially breaking existing code: the original identity assignment might even throw, which according to the DIP will no longer be allowed.
Mar 11
parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/11/2021 1:15 AM, Arafel wrote:
 On 11/3/21 10:01, Walter Bright wrote:
 On 3/11/2021 12:42 AM, Walter Bright wrote:
 Constructing from an rvalue essentially is move construction.
I forgot to mention that the new semantics only apply to EMO objects, which require both a move constructor and a move assignment operator. The move constructor is new syntax. Therefore, it shouldn't break existing code.
But we also read in the DIP, as it has already been mentioned:
 If a Move Constructor is not defined for a struct that has a Move Assignment 
 Operator, a default Move Constructor is defined and implemented as a move for 
 each of its fields, in lexical order.
It's not clear if a struct would be considered an EMO if either the Move Assignment Operator or the Move Constructor are defined by default. If that's the case, any struct with an identity assignment operator would be silently "upgraded" to EMO, thus potentially breaking existing code: the original identity assignment might even throw, which according to the DIP will no longer be allowed.
Yes, it does appear to have a conflict with `this(S)`, which wasn't part of D when the DIP was originally worked on. That syntax is part of the "-preview=rvaluerefparam" feature. Perhaps that feature can be simply replaced with DIP1040.
Mar 18
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 March 2021 at 08:42:32 UTC, Walter Bright wrote:
 Constructing from an rvalue essentially is move construction.
No, you need to destroy that rvalue. You'll note that C++ move constructor still require the old object to be in a "null" state so that its destruction can effectively be a noop. When a constructor is involved, you create a new object, and need to do something with the existing ones. This is why I was suggesting to recycle the postblit instead as a "postmove" constructor.
Mar 11
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/11/2021 2:19 AM, deadalnix wrote:
 On Thursday, 11 March 2021 at 08:42:32 UTC, Walter Bright wrote:
 Constructing from an rvalue essentially is move construction.
No, you need to destroy that rvalue.
Why? There can be no other uses of the rvalue, so why not simply move it?
Mar 11
parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 March 2021 at 21:41:59 UTC, Walter Bright wrote:
 Why? There can be no other uses of the rvalue, so why not 
 simply move it?
If you have two objects in your move constructor, but only one at the end, then a destruction must take place somewhere.
Mar 11
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/11/2021 4:53 PM, deadalnix wrote:
 On Thursday, 11 March 2021 at 21:41:59 UTC, Walter Bright wrote:
 Why? There can be no other uses of the rvalue, so why not simply move it?
If you have two objects in your move constructor, but only one at the end, then a destruction must take place somewhere.
The whole point of move construction is to move an initialized object to an unconstructed object. No destruction is needed. It's move assignment that needs to destruct something (the original value of the destination).
Mar 12
parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 12 March 2021 at 10:07:41 UTC, Walter Bright wrote:
 On 3/11/2021 4:53 PM, deadalnix wrote:
 On Thursday, 11 March 2021 at 21:41:59 UTC, Walter Bright 
 wrote:
 Why? There can be no other uses of the rvalue, so why not 
 simply move it?
If you have two objects in your move constructor, but only one at the end, then a destruction must take place somewhere.
The whole point of move construction is to move an initialized object to an unconstructed object. No destruction is needed. It's move assignment that needs to destruct something (the original value of the destination).
How do you ensures that there aren't any leftover that need to be destroyed or enforce that there are no leftovers? Enforcing it seems like an impossible task to be, or at least extremely complex. So that means you need to call the destructor anyways and that forces the struct to have a null state.
Mar 12
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/12/2021 4:38 AM, deadalnix wrote:
 On Friday, 12 March 2021 at 10:07:41 UTC, Walter Bright wrote:
 On 3/11/2021 4:53 PM, deadalnix wrote:
 On Thursday, 11 March 2021 at 21:41:59 UTC, Walter Bright wrote:
 Why? There can be no other uses of the rvalue, so why not simply move it?
If you have two objects in your move constructor, but only one at the end, then a destruction must take place somewhere.
The whole point of move construction is to move an initialized object to an unconstructed object. No destruction is needed. It's move assignment that needs to destruct something (the original value of the destination).
How do you ensures that there aren't any leftover that need to be destroyed or enforce that there are no leftovers?
I don't see any way to do that. When you write a custom destructor, the compiler can't verify it, either.
 Enforcing it seems like an impossible task 
 to be, or at least extremely complex. So that means you need to call the 
 destructor anyways and that forces the struct to have a null state.
Both the move constructor and the destructor are in the same struct, and should be developed at the same time. At some point, the program has to rely on the programmer knowing what he's doing when doing storage management.
Mar 15
parent reply deadalnix <deadalnix gmail.com> writes:
On Monday, 15 March 2021 at 07:48:04 UTC, Walter Bright wrote:
 Both the move constructor and the destructor are in the same 
 struct, and should be developed at the same time. At some 
 point, the program has to rely on the programmer knowing what 
 he's doing when doing storage management.
That is a notoriously difficult thing to get right, and why the mechanism of constructor and destructor has been invented in the first place. Consider for instance that destructor design also destructs each field of the struct, in addition of running the user's code. One could argue that the the developer could destruct all fields manually, and in fact, it is certainly possible to do so. But the design chosen doesn't do that because we know ahead of time what would happen: leakage galore. One way to think of it is in term of solidity. If I change something at point A, will something else break unexpectedly in a subtle way at point B? If yes, then the design is not solid. When a new member is added to a struct, then this member will be destroyed automatically too, just like the others. This is correct by design. This is solid. And this is composable as the destruction of each fields simply work and the code that has to be written is limited to ensuring some cross fields invariants remains true. If one uses the exemple of a RC smart pointer for instance, the smart pointer does not need to know how to destroy its payload, the payload knows this. The RC simply keep the RC count up to date and chooses to destroy or not its payload based on this, ie it maintains the invariant between the state of the RC and the state of the payload. If no such invariant needs to be maintained, then the destructor can remain empty and everything works fine. Now what happens with move? Well,t he natural way to transpose the described design to move is as follow: 1/ move all fields one by one 2/ call the move constructor on the result to maintain invariants if need be. To me, 2/ furiously sounds like a postblit, but I'm open to the fact that alternatives. I however know for a fact that the proposed magic will open a bag of worm because it doesn't go with the same set of design principle as the rest of the contruction/destruction business. For instance, if a field were to be added to a struct, then immediately the move assignment becomes invalid, silently. Worse, if the field itself contains something detructible, now there is something seriously wrong, potentially outside of the struct I'm working with. For instance, if that new field is a smart pointer, then the guarantee provided y the smart pointer are broken, silently. Now we might decide, instead, that all field are going to be destroyed at the end of the move assign in the move struct, inc are there are leftovers. But we are now back to the situation where all struct MUST have a null state, or you won't be able to have them as fields of other structs. Or we break the guarantees provided by the ctor/dtor mechanism, but in this case, why have it at all? The whole point of ctor and dtor is to ensure that invariant are kept within the program. Let's not break this invariant.
Mar 16
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/16/2021 3:24 PM, deadalnix wrote:
 Now what happens with move? Well,t he natural way to transpose the described 
 design to move is as follow:
 1/ move all fields one by one
 2/ call the move constructor on the result to maintain invariants if need be.
 
 To me, 2/ furiously sounds like a postblit, but I'm open to the fact that 
 alternatives.
The move constructor is a postblit, but with two arguments and without the implicit initial copy.
 I however know for a fact that the proposed magic will open a bag 
 of worm because it doesn't go with the same set of design principle as the
rest 
 of the contruction/destruction business.
 
 For instance, if a field were to be added to a struct, then immediately the
move 
 assignment becomes invalid, silently. Worse, if the field itself contains 
 something detructible, now there is something seriously wrong, potentially 
 outside of the struct I'm working with. For instance, if that new field is a 
 smart pointer, then the guarantee provided y the smart pointer are broken, 
 silently.
 
 Now we might decide, instead, that all field are going to be destroyed at the 
 end of the move assign in the move struct, inc are there are leftovers. But we 
 are now back to the situation where all struct MUST have a null state, or you 
 won't be able to have them as fields of other structs.
 
 Or we break the guarantees provided by the ctor/dtor mechanism, but in this 
 case, why have it at all? The whole point of ctor and dtor is to ensure that 
 invariant are kept within the program. Let's not break this invariant.
 
If a field with a move constructor is added to a struct S without one, a default move constructor will be created for S that calls the field's move constructor. If one is added to a struct with an explicit move constructor, it is up to the struct programmer to fold it in explicitly, just like he does for explicit constructors and destructors.
Mar 18
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 18.03.21 10:51, Walter Bright wrote:
 
 If one is added to a struct with an explicit move constructor, it is up 
 to the struct programmer to fold it in explicitly, just like he does for 
 explicit constructors and destructors.
This is not the case, you usually don't have to do it explicitly in explicit destructors: --- import std.stdio; struct T{ ~this(){ writeln("T destructor called"); } } struct S{ T t; ~this(){ writeln("S destructor called"); // NOTE: does not explicitly call t.~this() } } void main(){ S s; } --- S destructor called T destructor called --- You may be able to argue that it's not important, but let's not pretend that nothing new is going on here. If you don't explicitly _move_ a field, the destructor will _not_ be called. Destructors do the right thing by default, even if you don't update them after adding a new field. This is not true for move constructors.
Mar 18
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 18 March 2021 at 09:51:27 UTC, Walter Bright wrote:
 If a field with a move constructor is added to a struct S 
 without one, a default move constructor will be created for S 
 that calls the field's move constructor.
This is all good so far.
 If one is added to a struct with an explicit move constructor, 
 it is up to the struct programmer to fold it in explicitly, 
 just like he does for explicit constructors and destructors.
As explained in https://forum.dlang.org/post/bkfqchwpnonngjrtybbe forum.dlang.org , this is where things go wrong. There are a few ways this can be designed, but they all drop some important property that would be detrimental overall as we don't gain much in return. The error that you are making is that it is fundamentally different from ctor and dtor in nature. Yes, in both cases, you expect the dev to do something sensible, but the comparison stop there. And you know it is not enough, because if that was, then why do array bound checks? or have ctor/dtor at all to begin with? Just call the destroy function in all codepath and be done with it! The reality is that the set of assumption broken here is much larger than for ctor/dtor, even going as far as breaking assumptions provided by ctor/dtor, such as guaranteed pairwise construction/destruction.
Mar 18
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 10 March 2021 at 22:51:58 UTC, tsbockman wrote:
 On Friday, 5 March 2021 at 23:03:57 UTC, tsbockman wrote:
 On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 [...]
From the DIP:
     [...]
Is the parameter to these methods really pass-by-value? ... If the parameter is, in fact, intended to be pass-by-reference, then I must strenuously object to the chosen syntax.
Over in the feedback thread, Atila Neves also concluded that the syntax is misleading here: On Wednesday, 10 March 2021 at 21:27:25 UTC, Atila Neves wrote:
 I eventually understood what this meant, but this confused me 
 when I read it the first time. I'd reword it to mention that 
 the syntax looks like a by-value parameter but ends up being 
 passed by reference. It also confused me that the 2nd function 
 had `ref` in there.
I didn't object to the syntax, especially since it's the same syntax used right now for moves. I got confused by the explanation, since it mixes syntax with the underlying mechanics.
Mar 12
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 12 March 2021 at 19:50:57 UTC, Atila Neves wrote:
 On Wednesday, 10 March 2021 at 22:51:58 UTC, tsbockman wrote:
 Over in the feedback thread, Atila Neves also concluded that 
 the syntax is misleading here:

 On Wednesday, 10 March 2021 at 21:27:25 UTC, Atila Neves wrote:
 I eventually understood what this meant, but this confused me 
 when I read it the first time. I'd reword it to mention that 
 the syntax looks like a by-value parameter but ends up being 
 passed by reference. It also confused me that the 2nd 
 function had `ref` in there.
I didn't object to the syntax,
I didn't mean to imply that you objected, just that you confirmed that by-value parameter syntax is being used where the parameter is actually semantically by-reference.
 especially since it's the same syntax used right now for moves.
No, it's not. You are conflating "moves" with "custom move operators definitions", a proposed implementation detail of the lowering for actual moves. This is the syntax for moves and copies, depending on the surrounding context: B b = a; C c = b; return f(c); The closest thing we have to a dedicated move syntax is this: import std.algorithm.mutation : move; move(a, b); As for the proposed move operator syntax - no, we do not currently use that syntax for move operators. None of the various methods of moving things call them: ////////////////////////////// struct S { int x; this(int x) safe { this.x = x; } this(S s) safe { x = s.x * 2; } void opAssign(S s) safe { x = -s.x; } } S g(S s) { return s; } void main() system { import std.stdio : writeln; import std.algorithm.mutation : move; S s = 1, t = s, u; writeln(t); move(t, u); writeln(u); writeln(g(u)); } ////////////////////////////// Output: S(1) S(1) S(1) Even if I manually call S.opAssign, a copy is performed for the parameter, so it's *really* not a move operator. We don't have custom move operators at all right now; that's the point of this DIP, isn't it?
Mar 12
parent tsbockman <thomas.bockman gmail.com> writes:
On Friday, 12 March 2021 at 22:46:05 UTC, tsbockman wrote:
 As for the proposed move operator syntax - no, we do not 
 currently use that syntax for move operators. None of the 
 various methods of moving things call them:
 ...
 Even if I manually call S.opAssign, a copy is performed for the 
 parameter, so it's *really* not a move operator.
I accidentally left out assignment in my example, but that's OK because it requires a different example to demonstrate the copy problem: //////////////////////////////////// struct S { int x; this(int x) safe { this.x = x; } this(ref typeof(this) s) safe { x = 0; } void opAssign(S s) safe { x = -s.x; } } void main() system { import std.stdio : writeln; import std.algorithm.mutation : move; S s = 1, t; t = s; writeln(t); } //////////////////////////////////// Output: S(0) So, the opAssign does get called in that case, but it's not a move because the copy constructor gets called first.
Mar 12
prev sibling next sibling parent 12345swordy <alexanderheistermann gmail.com> writes:
On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of Community 
 Review of DIP 1040, "Copying, Moving, and Forwarding":

 [...]
Does this DIP allow for creation for move constructors for external(c++) classes for c++ version? It would be missed opportunity not include it in the dip. -Alex
Mar 05
prev sibling next sibling parent reply RazvanN <razvan.nitu1305 gmail.com> writes:
On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:

1. The "Returning an EMO by value" section says:

"
S func()
{
     S s;
     return s;
}

This works exactly as it does currently for non-EMO objects.
"

So this code is allowed to compile. However, in the "Returning an 
EMO by move ref" section we have this code:

"
S func(return S s)
{
     S s2;
     return s2;  // error, can't return local by Move Ref
}
"

Isn't this rather restrictive? I can imagine a scenario where you 
would like to return the parameter if some condition is met or 
return a local otherwise:

S func(return S s)
{
    S s2;
    if (_some_condition)
      return s;
    else
      return s2;
}

Moreover, what happens in this case if we have a struct that's 
not an EMO, but defines a move constructor?

2. What are the situations where a move constructor call is 
implicitly inserted by compiler? This is not explicitly stated in 
the DIP and it is rather confusing. For example, when an instance 
is passed by `move ref` is there a move constructor call?

3. The DIP should explicitly state what happens when you pass an 
rvalue instance of an EMO by ref. How does that interact with 
`-preview=rvaluerefparam` ?

4. The perfect forwarding section is superficially described and 
it is hard to assess its correctness and relationship to the move 
constructor.

5. Structs with internal pointers that are moved should be part 
of the motivation of the DIP.

Cheers,
RazvanN
Mar 08
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/8/2021 12:23 AM, RazvanN wrote:
 On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:

 
 1. The "Returning an EMO by value" section says:
 
 "
 S func()
 {
      S s;
      return s;
 }
 
 This works exactly as it does currently for non-EMO objects.
 "
 
 So this code is allowed to compile. However, in the "Returning an EMO by move 
 ref" section we have this code:
 
 "
 S func(return S s)
 {
      S s2;
      return s2;  // error, can't return local by Move Ref
 }
 "
 
 Isn't this rather restrictive?
The trouble is determining when the caller allocates the space for local variable, as it does with NRVO. Hence the restriction.
 I can imagine a scenario where you would like to 
 return the parameter if some condition is met or return a local otherwise:
 
 S func(return S s)
 {
     S s2;
     if (_some_condition)
       return s;
     else
       return s2;
 }
 
 Moreover, what happens in this case if we have a struct that's not an EMO, but 
 defines a move constructor?
That means it does not have a Move Assignment Operator. It doesn't get EMO semantics. The Move Constructor section applies.
 2. What are the situations where a move constructor call is implicitly
inserted 
 by compiler? This is not explicitly stated in the DIP and it is rather 
 confusing.
It's in the Move Constructor section.
 For example, when an instance is passed by `move ref` is there a move 
 constructor call?
No, because it is passed by ref.
 3. The DIP should explicitly state what happens when you pass an rvalue
instance 
 of an EMO by ref. How does that interact with `-preview=rvaluerefparam` ?
With EMOs, there is no need to use the 'ref' annotation. If you do use 'ref', the special EMO semantics do not apply.
 4. The perfect forwarding section is superficially described and it is hard to 
 assess its correctness and relationship to the move constructor.
Please be more specific?
 5. Structs with internal pointers that are moved should be part of the 
 motivation of the DIP.
Yes.
Mar 08
parent reply RazvanN <razvan.nitu1305 gmail.com> writes:
On Monday, 8 March 2021 at 10:38:25 UTC, Walter Bright wrote:
 On 3/8/2021 12:23 AM, RazvanN wrote:
 On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:

 Moreover, what happens in this case if we have a struct that's 
 not an EMO, but defines a move constructor?
That means it does not have a Move Assignment Operator. It doesn't get EMO semantics. The Move Constructor section applies.
Even if the move assignment operator is implicitly generated? The DIP states: "If a Move Assignment Operator is not defined for a struct that has a Move constructor, a default Move Assignment Operator is defined and implemented as a move for each of its fields, in lexical order." Is it possible to have a non-EMO struct that defines solely a move constructor or solely a move assignment operator? It seems like if you define one, you implicitly get the other one.
 2. What are the situations where a move constructor call is 
 implicitly inserted by compiler? This is not explicitly stated 
 in the DIP and it is rather confusing.
It's in the Move Constructor section.
 For example, when an instance is passed by `move ref` is there 
 a move constructor call?
No, because it is passed by ref.
Ok, correct me if I am wrong, but it seems that if you define a move constructor, you implicitly get a move assignment operator and viceversa. This means that your struct becomes an EMO if you define one or the other. Once you have an EMO struct, besides the trivial `S a = b` are there any other situations where the move constructor may be called? It seems that EMOs are always passed by reference. If that is the case, why bother defining a move constructor when it will not get called? If I am mistaken, can you please provide a non-trivial example where the move constructor gets called? Also, will a move constructor ever get called when an argument is passed to a function?
 3. The DIP should explicitly state what happens when you pass 
 an rvalue instance of an EMO by ref. How does that interact 
 with `-preview=rvaluerefparam` ?
With EMOs, there is no need to use the 'ref' annotation. If you do use 'ref', the special EMO semantics do not apply.
So I assume you get an error? Also, what happens with `auto ref` deduction when called with an EMO.
 4. The perfect forwarding section is superficially described 
 and it is hard to assess its correctness and relationship to 
 the move constructor.
Please be more specific?
Sorry for being un-informative. I will explain in more detail. The DIP has this example: ref S fwd(return ref S s) { return s; } void f(S s); ... S s; f(fwd(s)); f(fwd(S()); Assuming S is an EMO, when we have `f(fwd(S()))` what happens here? Is a reference to the rvalue passed to `fwd` or does the move constructor get called? If we simply call `f(S())`, what happens here? Is `S()` passed by move ref or do we have a move constructor call? The DIP talks about move refs, but the examples only use lvalues. Is the move constructor ever called for an rvalue instance of an EMO?
 5. Structs with internal pointers that are moved should be 
 part of the motivation of the DIP.
Yes.
Mar 08
parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/8/2021 5:21 AM, RazvanN wrote:
 On Monday, 8 March 2021 at 10:38:25 UTC, Walter Bright wrote:
 On 3/8/2021 12:23 AM, RazvanN wrote:
 On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:

 Moreover, what happens in this case if we have a struct that's not an EMO, 
 but defines a move constructor?
That means it does not have a Move Assignment Operator. It doesn't get EMO semantics. The Move Constructor section applies.
Even if the move assignment operator is implicitly generated? The DIP states: "If a Move Assignment Operator is not defined for a struct that has a Move constructor, a default Move Assignment Operator is defined and implemented as a move for each of its fields, in lexical order." Is it possible to have a non-EMO struct that defines solely a move constructor or solely a move assignment operator? It seems like if you define one, you implicitly get the other one.
That's a good point. Perhaps it should be an error to define only one.
 2. What are the situations where a move constructor call is implicitly 
 inserted by compiler? This is not explicitly stated in the DIP and it is 
 rather confusing.
It's in the Move Constructor section.
 For example, when an instance is passed by `move ref` is there a move 
 constructor call?
No, because it is passed by ref.
Ok, correct me if I am wrong, but it seems that if you define a move constructor, you implicitly get a move assignment operator and viceversa. This means that your struct becomes an EMO if you define one or the other. Once you have an EMO struct, besides the trivial `S a = b` are there any other situations where the move constructor may be called?
void g(S b) { S a = b; // calls copy constructor (not last use of b) S c = b; // calls move constructor (last use of b) } It seems that EMOs are always passed
 by reference. If that is the case, why bother defining a move constructor when 
 it will not get called? If I am mistaken, can you please provide a non-trivial 
 example where the move constructor gets called?
void f(S s); void g(S a) { f(a); // copy constructor called because not last use of `a` f(a); // move constructor called because last use of `a` }
 3. The DIP should explicitly state what happens when you pass an rvalue 
 instance of an EMO by ref. How does that interact with 
 `-preview=rvaluerefparam` ?
With EMOs, there is no need to use the 'ref' annotation. If you do use 'ref', the special EMO semantics do not apply.
So I assume you get an error?
No, it just works the way it does now.
 Also, what happens with `auto ref` deduction when 
 called with an EMO.
Auto ref parameters are only for template functions. "An auto ref function template parameter becomes a ref parameter if its corresponding argument is an lvalue, otherwise it becomes a value parameter" https://dlang.org/spec/template.html#auto-ref-parameters It will continue to do exactly what it says.
 The DIP has this example:
 
 ref S fwd(return ref S s) { return s; }
 
 void f(S s);
 ...
 S s;
 f(fwd(s));
 f(fwd(S());
 
 Assuming S is an EMO, when we have `f(fwd(S()))` what happens here? Is a 
 reference to the rvalue passed to `fwd` or does the move constructor get
called?
 If we simply call `f(S())`, what happens here? Is `S()` passed by move ref or
do 
 we have a move constructor call?
 
 The DIP talks about move refs, but the examples only use lvalues. Is the move 
 constructor ever called for an rvalue instance of an EMO?
What happens is just what the spec says: "Ownership of the argument to fwd() is retained by the caller, and so the caller will be responsible for its destruction. When the call is made to f(), a copy is made." No move copies or move assignments are done. BUT, if the compiler can look inside the fwd() function, it can see that `s` can be moved directly to `f` in the first case, and `S()` can be moved directly to `f` in the second. Thus, here the move constructor is used as an optimization.
Mar 18
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Monday, 8 March 2021 at 08:23:55 UTC, RazvanN wrote:
 "
 S func(return S s)
 {
     S s2;
     return s2;  // error, can't return local by Move Ref
 }
 "

 Isn't this rather restrictive?
It is important for soundness. When you pass by ref, the owner remains the caller, not the callee.
 I can imagine a scenario where you would like to return the 
 parameter if some condition is met or return a local otherwise:
Yes, absolutely, in which case you either want to return it by ref, or take it by value.
Mar 08
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 05.03.21 13:19, Mike Parker wrote:
 This is the discussion thread for the first round of Community Review of 
 DIP 1040, "Copying, Moving, and Forwarding":
 
 https://github.com/dlang/DIPs/blob/a9c553b0dbab1c2983a801b5e89b51c5c33d
180/DIPs/DIP1040.md 
 
 
 ...
I haven't had time yet to contribute a thorough review, but one thing that stands out to me is that there seems to be no discussion of interaction with `const`/`immutable`, etc. Given that that's a source of unsoundness for postblit, maybe it deserves some explicit consideration?
Mar 11
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/11/2021 10:43 AM, Timon Gehr wrote:
 I haven't had time yet to contribute a thorough review, but one thing that 
 stands out to me is that there seems to be no discussion of interaction with 
 `const`/`immutable`, etc. Given that that's a source of unsoundness for 
 postblit, maybe it deserves some explicit consideration?
All the problems with const/immutable revolved around postblit. That's why this proposal has zero reliance on postblit.
Mar 11
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 March 2021 at 22:39:21 UTC, Walter Bright wrote:
 All the problems with const/immutable revolved around postblit. 
 That's why this proposal has zero reliance on postblit.
These problems seems to arise due to the fact postblit did not distinguish between move and copy. postblit is inappropriate for copy, but as far as I can tell, not only does it work for move, but it's the only path that do not involve creating yet more magic that's going to bite us in the ass at some point.
Mar 11
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/11/2021 4:57 PM, deadalnix wrote:
 These problems seems to arise due to the fact postblit did not distinguish 
 between move and copy.
 
 postblit is inappropriate for copy, but as far as I can tell, not only does it 
 work for move, but it's the only path that do not involve creating yet more 
 magic that's going to bite us in the ass at some point.
postblit bit us in the ass quite a bit. (postblit didn't do moves)
Mar 11
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 12 March 2021 at 02:06:19 UTC, Walter Bright wrote:
 On 3/11/2021 4:57 PM, deadalnix wrote:
 These problems seems to arise due to the fact postblit did not 
 distinguish between move and copy.
 
 postblit is inappropriate for copy, but as far as I can tell, 
 not only does it work for move, but it's the only path that do 
 not involve creating yet more magic that's going to bite us in 
 the ass at some point.
postblit bit us in the ass quite a bit. (postblit didn't do moves)
You are making my point, yet for some reason miss it anyways. Postblit works with *1* object. It fails at being a good copy mechanism because copying involves *2* objects, by definition. Move works with *1* object. opAssign will, similarly to what happened when using postblit for copies, lead to problems because it inherently works with *2* objects. Postblit is a natural fit for moves because both work with *1* object, thus not leaving an object out there is some sort of magic state that we can never figure out what to do with. There is already a fair bit of magic that is proposed to be added to opAssign in this proposal, yet it is now obvious to me that there are already holes in it. For instance, if one of the fields of the object ends up not being moved, how will this fields ends up being destroyed? How is that even fixable considering the whole thing can be n-levels deep? The only way I this being fixed is be reintroducing the notion that structs must have a null state and destroy the object anyways post move, but going there would be a big optimization barrier, while not going there creates a situation where things that should be destroyed ends up not being, which is also pretty bad. On the other hand, the postblit by its very nature does not allow for this whole situation to arise to begin with. If a field is not going to be moved, it will have to see some new value assigned there, causing the destruction of the old value.
Mar 12
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/12/2021 4:29 AM, deadalnix wrote:
 On Friday, 12 March 2021 at 02:06:19 UTC, Walter Bright wrote:
 On 3/11/2021 4:57 PM, deadalnix wrote:
 These problems seems to arise due to the fact postblit did not distinguish 
 between move and copy.

 postblit is inappropriate for copy, but as far as I can tell, not only does 
 it work for move, but it's the only path that do not involve creating yet 
 more magic that's going to bite us in the ass at some point.
postblit bit us in the ass quite a bit. (postblit didn't do moves)
You are making my point, yet for some reason miss it anyways. Postblit works with *1* object. It fails at being a good copy mechanism because copying involves *2* objects, by definition. Move works with *1* object. opAssign will, similarly to what happened when using postblit for copies, lead to problems because it inherently works with *2* objects.
Postblit's problems arose from it not having access to both objects. The opAssign does have access to both, and the qualifiers can be applied to both parameters, so I don't see a barrier to it working.
 Postblit is a natural fit for moves because both work with *1* object, thus
not 
 leaving an object out there is some sort of magic state that we can never
figure 
 out what to do with.
 
 There is already a fair bit of magic that is proposed to be added to opAssign
in 
 this proposal, yet it is now obvious to me that there are already holes in it. 
 For instance, if one of the fields of the object ends up not being moved, how 
 will this fields ends up being destroyed? How is that even fixable considering 
 the whole thing can be n-levels deep? The only way I this being fixed is be 
 reintroducing the notion that structs must have a null state and destroy the 
 object anyways post move, but going there would be a big optimization barrier, 
 while not going there creates a situation where things that should be
destroyed 
 ends up not being, which is also pretty bad.
 
 On the other hand, the postblit by its very nature does not allow for this
whole 
 situation to arise to begin with. If a field is not going to be moved, it will 
 have to see some new value assigned there, causing the destruction of the old 
 value.
An opAssign gives the implementer complete control over the operation of it, including when and how destruction takes place of the original destination's contents.
Mar 16
parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 16 March 2021 at 09:14:58 UTC, Walter Bright wrote:
 Postblit's problems arose from it not having access to both 
 objects. The opAssign does have access to both, and the 
 qualifiers can be applied to both parameters, so I don't see a 
 barrier to it working.
YES! This is why it is unsuitable for copies. During a copy, there are 2 objects. Any solution, that provide to objects when moving, a situation where only one object exists, will just open a can of worm of the same nature as postblit for copies opened. This is self evident. This is so obvious that I don't know how to unpack it any further. Pretend you have one object when you have two => problems. Pretend you have two objects when you have one => problems.
 An opAssign gives the implementer complete control over the 
 operation of it, including when and how destruction takes place 
 of the original destination's contents.
That break all the invariant provided by the ctor/dtor mechanism and because struct are composable (you can use structs as member of structs) then the mess is not bounded to the one struct you are toying with.
Mar 16
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 16.03.21 23:30, deadalnix wrote:
 On Tuesday, 16 March 2021 at 09:14:58 UTC, Walter Bright wrote:
 Postblit's problems arose from it not having access to both objects. 
 The opAssign does have access to both, and the qualifiers can be 
 applied to both parameters, so I don't see a barrier to it working.
YES! This is why it is unsuitable for copies. During a copy, there are 2 objects. Any solution, that provide to objects when moving, a situation where only one object exists, will just open a can of worm of the same nature as postblit for copies opened. This is self evident. This is so obvious that I don't know how to unpack it any further. Pretend you have one object when you have two => problems. Pretend you have two objects when you have one => problems.
 An opAssign gives the implementer complete control over the operation 
 of it, including when and how destruction takes place of the original 
 destination's contents.
That break all the invariant provided by the ctor/dtor mechanism and because struct are composable (you can use structs as member of structs) then the mess is not bounded to the one struct you are toying with.
This reasoning makes sense to me, but perhaps sometimes you don't want to actually destroy the move target. E.g., it's possible that you want to move a dynamic array element-wise instead of by reference to avoid creating dangling references (if you had pointers to the array elements). There's no good reason why that should be possible for static array members but not dynamic array members. This is what Walter means by complete control. So a "better" solution would allow for this while still doing the right thing for new fields by default. For the common case, I think the move opAssign should be auto-generated from the destructor and the move constructor/postblit. So whatever the solution is, having a move constructor/postblit should not require an implementation of the move opAssign, but it should rather auto-generate it using the move constructor/postblit. So I think the reasoning may not fully apply to move opAssign. OTOH, I think you are making a very strong point that a (move) postblit is in the common case better than a move _constructor_. Postblit for copies is bad, but for moves into destructed/uninitialized memory, it is often precisely what we want. I guess the main benefit of a move constructor is that it allows you to fix internal references, while you are out of luck with postblit. Summary: move constructor: move into uninitialized memory, error prone, internal references supported (move) postblit: move into uninitialized memory, composes well, internal references not supported move opAssign: move into existing memory, error prone, allows for full control but can usually be auto-generated from destructor and move constructor/postblit So, obvious question: What design satisfies the following constraints? move constructor v2: move into uninitialized memory, composes well, internal references supported move opAssign v2: move into existing memory, composes well, allows for full control, but can be auto-generated in the common case It's pretty clear what a move constructor v2 would be: Require all fields to be explicitly initialized, even those that have a field initializer. This would allow to a manual postblit-like implementation, i.e., first this.tupleof=move(other.tupleof), then do fixup, as well as repairing internal references if that's required. Issues with this: - It does not really make sense to reinitialize fields of immutable objects that have a field initializer. - You can't truly get postblit behavior as IIRC std.algorithm.move actually writes the init value over its argument, so maybe still supporting postblit is better. (Unless the optimizer is reliably good enough here.) What's a move opAssign v2? Maybe move opAssign but require all fields to be mentioned? One issue with original DIP that occurred to me while writing this stuff down: - How do you manually move an object field-wise in a move constructor or move opAssign? You'd want to move out the fields, but the compiler does not really have a good way to see that. If you use std.algorithm.move, you get additional overhead that a direct call to the move constructor or move opAssign would not suffer. Do we rely on the optimizer here?
Mar 16
parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 17 March 2021 at 01:06:45 UTC, Timon Gehr wrote:
 [...]
Lots of good stuff in there that I didn't quote fully to not spam. This kind of lead me to look at things in a new way: The main problem here is that we have loosely defined requirement, and when this is the case, it is very easy to fool oneself and fullfill most requirement most of the time, but actually provide zero useful guarantee, be it to the dev or the optimizer. So here, we have an existing system: ctor/dtor. The goal of this system is to ensure that an object is available to the program once it has been put in a proper state, and that for each object constructed, there will be a corresponding dtor call that will give an opportunity to undo this state. The guarantee provided is that ctor/dtor go by pair and the compiler ensures this. What has been constructed will be destroyed and vice versa. This causes a problem: what to do when an object is duplicated? Then it is required to construct a new object, from the previous one, and that new object, like the previous one, will be destroyed. Because construction and destruction might be expensive, we want to be able to group a copy and a destruction operation of an object (granted there are no further use between that copy and that destruction) into one: a move operation. Which lead us to a primitive set of requirements: 1/ Construction and destruction of object map 1:1 2/ An object cannot be used prior its construction and after its destruction. 3/ As an optimization, we want to be able to remove copy/destruction pair when the object is not used after the copy. 1/ and 2/ are already provided, but might be inefficient. 3/ is a way to make things more efficient, and in more way than what you'd think. One notorious difference between C++ and D is that in C++, objects must have a fixed address, while in D, they do not. This means that the D compiler is free to move objects around. This has more consequences that one would expect, consider for instance the following sample code: https://godbolt.org/z/9hWzxb It is clear from the disassembly that *2* dereferences are happening, when the C++ code only has one. How come? Because structs in C++ are not movable by default, and the struct preexist the function call, it must be living somewhere in the caller's stack, and is passed by reference at the ABI level. To make it look like it is passed by value, the called will make a copy and then destroy it after the call. In practice, this is worse than it look in this simplified exemple. Because not only this means that a vast number of dereferences are executed across the program, but this is only the 1st order effect. Because numerous things now go through pointers, it forces the optimizer to prove a ton of things using alias analysis that would be self evident without that extra indirection, and in many cases, it cannot. For instance: void foo(unique<int> a, unique<int> b) { a = make_unique<int>(...); // The compiler has to assume that b may have been modified here, because both a and b could be the same object, and therefore b be modified by this new assignment. } This is a major problem of C++ object model. This is not a problem D has at the moment. This is simply the wrong default. This leaves us with one more requirement: 4/ Objects must be usable by value at the ABI level, including object with ctor/dtor. The obvious problem with this are interior pointers, and I'll come back to them later on. It must however be understood that the vast majority of struct usage do not involve interior pointers, and therefore throwing away 4/ for interior pointer supports seems to be self defeating. Another peculiarity of the C++ object model can be seen in this sample: https://godbolt.org/z/KGeffj In the code generated, we can see that the smart pointer is allocated and initialized, then passed to the function by reference, which we expect. But what's interesting comes after the function call: there is a test and a branch. The generated code test the value of the smart pointer, and only free the memory is the pointer isn't null. for readability, I made the fuinction nothrow, but remove it and you'll see that a vast amount of code has to be generate for exception handling too. This is happening because the function could have moved the object away. While the sensible way to handle an object that has been moved is to not use it at all, this was not possible for C++ for backward compatibility reasons. As a result, a moved object is put in a "null" state, where the destruction is a noop. This null state is the source of a ton of extra work by destructor and a ton of generated code for nothing. If the function moves the object, then we want no destruction at all, and if it doesn't we want to destroy without checking against a null state. Obviously, we want the callee to do that as the caller doesn't have the infos, but that turns out to be a complex task when the object is passed by ref to the callee due to the previous problem. This leave us with one more requirement: 5/ Object must not require a null state. It is to be noted that the current DIP proposes to add 3 to D, but at the cost of either 1/ or 5/ being broken, which IMO is self defeating. Let's see why. In the current proposed scheme, the move constructor or move assignment have the object things are moving from available. One of the following MUST happen: a/ The previous object is left as this after the move. This means that 1/ is not ensured by default by the constructor anymore as any leftover won't be destroyed. It is easy to say that dev will be careful but it pretty much bound to fail, because it does the wrong thing by default - a change in the object might break 1/ silently - and it can do so non locally - a change to a member of a member of a member can break 1/ silently. b/ The previous object is destroyed, but in this case, we ought to place in in a null state as we move. This is the approach C++ is taking and it break 5/ . This problem is fundamentally unavoidable, because we have 2 object when semantically we should only have one. We have to either do something with the leftovers (which break 5/) or ignore the leftover (which breaks 1/). There are no other options than do something or do nothing once that path is taken. So, what do we want to do with move constructors anyways? Can't we just move the struct field by field recursively and be done with it? Yes, and I'd argue there is a problem if this isn't enough for 95% of the cases. Which leads to the two use cases I was able to identify: - Non movable struct. It is important that such a struct doesn't move. For instance, when the struct is some sort of header or a larger data segment. Another example is a struct that represent some kind of guard that needs to see its construction/destruction done in order. This can be achieved by disabling the move constructor, whatever the move constructor is defined as. It is fairly easy to realize such use case, the move constructor simply needs to exist at all. - Movable struct that require some form of bookkeeping. For these cases, a postblit would work with one exception: interior pointers. What I refers as interior pointers are struct containing pointer to elements which are within the struct itself. While this idiom exist, it is vanishingly rare and becoming rarer over time. The main reason for this is that memory has become slower, computation faster, and pointer larger, which in turn lead people to use "relative pointers", namely pointer defined as an offset from this. Unless is is expected that the struct may be more than 4GB in size - which is always the case, then it's all good. The extra addition required is well worth the memory saved (and increase hit rate in the cache that result from it). See https://www.youtube.com/watch?v=G3bpj-4tWVU for instance on how the swift runtime started using such techniques. I'll be blunt, once these techniques are known, I've actually never encountered a case of interior pointers that would not be solved by disabling move altogether. I'm not pretending it doesn't exist, but I've never seen it. It simply doesn't make sense to sacrifice any of the above mentioned requirement for it, even it turns out this is really needed, because, well, this is the edge case of the edge case, and while enabling it might be an option, throwing away thing which are good in the general case for it just doesn't make sense. I suspect that even then, making the struct unmovable and then definition custom method to move it manually would do the trick just fine. But just in case, here is what I propose: simply add an intrinsic, such as `void* __pre_move_address()` that can be called in the postblit, returning the address of the premove object. Any object using it would, of course, discard 4/ and not be usable as a value and instead always be passed by reference at the ABI level. This is the least constraining requirement to break, because it impact exclusively performances and never correctness like 1/ or 5/ would. However, considering it is possible to it custom once you disable move, I strongly suspect the bang is not worth the effort.
Mar 18
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/18/2021 12:50 PM, deadalnix wrote:
 So, what do we want to do with move constructors anyways? Can't we just move
the 
 struct field by field recursively and be done with it? Yes, and I'd argue
there 
 is a problem if this isn't enough for 95% of the cases. Which leads to the two 
 use cases I was able to identify:
   - Non movable struct. It is important that such a struct doesn't move. For 
 instance, when the struct is some sort of header or a larger data segment. 
 Another example is a struct that represent some kind of guard that needs to
see 
 its construction/destruction done in order. This can be achieved by disabling 
 the move constructor, whatever the move constructor is defined as. It is
fairly 
 easy to realize such use case, the move constructor simply needs to exist at
all.
Ok.
   - Movable struct that require some form of bookkeeping. For these cases, a 
 postblit would work with one exception: interior pointers.
This bookkeeping was the motivation for #DIP1014: "For example, D structs also may not use the constructor/destructor to register themselves with a global registry that keeps track of all instances in the system, e.g. via a linked list. This also severely limits the ability to store delegates that reference the struct instance from outside the struct." https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1014.md Marking the struct as immovable would resolve this problem, too.
 What I refers as interior pointers are struct containing pointer to elements 
 which are within the struct itself. While this idiom exist, it is vanishingly 
 rare and becoming rarer over time. The main reason for this is that memory has 
 become slower, computation faster, and pointer larger, which in turn lead
people 
 to use "relative pointers", namely pointer defined as an offset from this. 
 Unless is is expected that the struct may be more than 4GB in size - which is 
 always the case, then it's all good. The extra addition required is well worth 
 the memory saved (and increase hit rate in the cache that result from it). See 
 https://www.youtube.com/watch?v=G3bpj-4tWVU for instance on how the swift 
 runtime started using such techniques.
I didn't know this. This is good info.
 I'll be blunt, once these techniques are known, I've actually never
encountered 
 a case of interior pointers that would not be solved by disabling move 
 altogether. I'm not pretending it doesn't exist, but I've never seen it. It 
 simply doesn't make sense to sacrifice any of the above mentioned requirement 
 for it, even it turns out this is really needed, because, well, this is the
edge 
 case of the edge case, and while enabling it might be an option, throwing away 
 thing which are good in the general case for it just doesn't make sense.
 
 I suspect that even then, making the struct unmovable and then definition
custom 
 method to move it manually would do the trick just fine. But just in case,
here 
 is what I propose: simply add an intrinsic, such as `void*
__pre_move_address()` 
 that can be called in the postblit, returning the address of the premove
object. 
 Any object using it would, of course, discard 4/ and not be usable as a value 
 and instead always be passed by reference at the ABI level. This is the least 
 constraining requirement to break, because it impact exclusively performances 
 and never correctness like 1/ or 5/ would. However, considering it is possible 
 to it custom once you disable move, I strongly suspect the bang is not worth
the 
 effort.
This is more or less what DIP1014 proposed.
Mar 20
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
One problem unaddressed is, for moveable structs, what if there *are* interior 
pointers, and they wind up pointing to a defunct object? The current compiler 
never moves structs, so this problem never occurs.

Some off the top of my head possibilities:

1. never move structs in  safe code (or at least disable moving structs that 
contain pointers). Unfortunately, ref counted objects must have a payload 
pointer in them.

2. add a runtime check on field pointer assignments

3. add an invariant() runtime check on field pointer values

4. insist that field pointers be marked with  system (there's another DIP for 
marking variables as  system)

P.S. the original reason for not allowing interior pointers is so a compacting 
garbage collector could be used.
Mar 20
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Mar 20, 2021 at 02:20:06AM -0700, Walter Bright via Digitalmars-d wrote:
 One problem unaddressed is, for moveable structs, what if there *are*
 interior pointers, and they wind up pointing to a defunct object? The
 current compiler never moves structs, so this problem never occurs.
[...] Is there a typo somewhere here? I'm *pretty* sure the current compiler *does* move structs in some cases, and that has caused problems in the past where structs that store pointers to themselves will end up with dangling pointers after, e.g., being returned from a function. T -- May you live all the days of your life. -- Jonathan Swift
Mar 20
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/20/2021 8:12 AM, H. S. Teoh wrote:
 Is there a typo somewhere here?  I'm *pretty* sure the current compiler
 *does* move structs in some cases, and that has caused problems in the
 past where structs that store pointers to themselves will end up with
 dangling pointers after, e.g., being returned from a function.
It doesn't. Even in the case of NRVO, it doesn't actually move them. The main reason it doesn't is because data flow analysis is necessary to see if it can be moved (no pointers to the source object). In order to determine "last use", DFA is used.
Mar 20
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 21 March 2021 at 00:18:43 UTC, Walter Bright wrote:
 On 3/20/2021 8:12 AM, H. S. Teoh wrote:
 Is there a typo somewhere here?  I'm *pretty* sure the current 
 compiler
 *does* move structs in some cases, and that has caused 
 problems in the
 past where structs that store pointers to themselves will end 
 up with
 dangling pointers after, e.g., being returned from a function.
It doesn't. Even in the case of NRVO, it doesn't actually move them. The main reason it doesn't is because data flow analysis is necessary to see if it can be moved (no pointers to the source object). In order to determine "last use", DFA is used.
idk for GDC, but LDC will, because LLVM will.
Mar 20
parent reply Max Haughton <maxhaton gmail.com> writes:
On Sunday, 21 March 2021 at 01:33:56 UTC, deadalnix wrote:
 On Sunday, 21 March 2021 at 00:18:43 UTC, Walter Bright wrote:
 On 3/20/2021 8:12 AM, H. S. Teoh wrote:
 Is there a typo somewhere here?  I'm *pretty* sure the 
 current compiler
 *does* move structs in some cases, and that has caused 
 problems in the
 past where structs that store pointers to themselves will end 
 up with
 dangling pointers after, e.g., being returned from a function.
It doesn't. Even in the case of NRVO, it doesn't actually move them. The main reason it doesn't is because data flow analysis is necessary to see if it can be moved (no pointers to the source object). In order to determine "last use", DFA is used.
idk for GDC, but LDC will, because LLVM will.
Godbolt example?
Mar 21
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 21 March 2021 at 07:51:45 UTC, Max Haughton wrote:
 Godbolt example?
https://godbolt.org/z/eK7dYx You'll note that there are no loads in the generated code.
Mar 21
parent reply Max Haughton <maxhaton gmail.com> writes:
On Sunday, 21 March 2021 at 14:48:49 UTC, deadalnix wrote:
 On Sunday, 21 March 2021 at 07:51:45 UTC, Max Haughton wrote:
 Godbolt example?
https://godbolt.org/z/eK7dYx You'll note that there are no loads in the generated code.
Is that strictly a move or "just" the struct ABI? i.e. If I add a destructor then we do go through [RDI]. https://godbolt.org/z/bn4nrEbnn dmd also generates the same code (minus some peephole optimizations for the stack operations - not trying to nerd snipe Walter!)
Mar 21
parent deadalnix <deadalnix gmail.com> writes:
On Sunday, 21 March 2021 at 21:33:47 UTC, Max Haughton wrote:
 Is that strictly a move or "just" the struct ABI? i.e. If I add 
 a destructor then we do go through [RDI].
It's not "just an ABI thing". It means that the original address of the struct is never passed down. And if it is never passed down, it is not even possible to attempt to not move the struct. As for the destruction thing, I assume this because dmd/ldc decided to go with the same ABI as C++ and non POD are passed by ref, always, due to the way C++ does its business (I already described this in great length in the topic).
Mar 22
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/16/2021 3:30 PM, deadalnix wrote:
 This is self evident. This is so obvious that I don't know how to unpack it
any 
 further.
I'm sorry, I just don't understand your objection.
Mar 18
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 18.03.21 10:07, Walter Bright wrote:
 On 3/16/2021 3:30 PM, deadalnix wrote:
 This is self evident. This is so obvious that I don't know how to 
 unpack it any further.
I'm sorry, I just don't understand your objection.
In condensed form, I think the main complaint is this. Let's start with a struct: struct S{ T field0; this(S r){ field0=move(r.field0); } } Now someone adds a new field, but forgets to update the move constructor: struct S{ T field0, field1; this(S r){ field0=move(r.field0); } } field1 is now leaked: _Its destructor will never run_. And this can happen in safe code. (Ignoring the issue that field1 of the moved object will be the init value.) This design is error-prone. postblit does not have this issue, because fields that are not explicitly referred to are moved correctly by default. Hence the suggestion in my previous post to perhaps require all fields to be initialized in a move constructor and similar thoughts about opAssign. This mitigates the risk, but unfortunately it does not eliminate it. (It is furthermore possible that such an error would be annoying in some cases.) A possible scenario is: 1. There is a struct S, it's never moved around, so the move constructor is dead code. 2. Someone adds a new field, everything works fine, even when they forget to update the move constructor. 3. The compiler is updated to use more clever flow analysis, suddenly struct S is sometimes moved, leading to memory leaks and other bugs. 4. Spurious regression bug report, reputation damage, etc. (PS: Sorry for emails that went to your inbox instead of the newsgroup. Thunderbird changed the interface for no reason.)
Mar 18
parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/18/2021 11:12 AM, Timon Gehr wrote:
 In condensed form, I think the main complaint is this. Let's start with a
struct:
 
 struct S{
      T field0;
      this(S r){
          field0=move(r.field0);
      }
 }
 
 
 Now someone adds a new field, but forgets to update the move constructor:
 
 struct S{
      T field0, field1;
      this(S r){
          field0=move(r.field0);
      }
 }
 
 field1 is now leaked: _Its destructor will never run_. And this can happen in 
  safe code. (Ignoring the issue that field1 of the moved object will be the
init 
 value.)
Thanks, I understand it now.
 This design is error-prone. postblit does not have this issue, because fields 
 that are not explicitly referred to are moved correctly by default.
 
 Hence the suggestion in my previous post to perhaps require all fields to be 
 initialized in a move constructor and similar thoughts about opAssign.
 This mitigates the risk, but unfortunately it does not eliminate it. (It is 
 furthermore possible that such an error would be annoying in some cases.)
Not sure how it doesn't eliminate the risk. The compiler already does some flow analysis in constructors (to implement restrictions about when constructor calls can occur).
Mar 20
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 18 March 2021 at 09:07:02 UTC, Walter Bright wrote:
 On 3/16/2021 3:30 PM, deadalnix wrote:
 This is self evident. This is so obvious that I don't know how 
 to unpack it any further.
I'm sorry, I just don't understand your objection.
I hope this post will make it clearer: https://forum.dlang.org/post/bkfqchwpnonngjrtybbe forum.dlang.org It's a bit lengthy, but clearly we are not working from the same set of assumption, so it is necessary to dig deeper.
Mar 18
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/11/2021 6:06 PM, Walter Bright wrote:
 postblit bit us in the ass quite a bit. (postblit didn't do moves)
Ack. Of course it did moves.
Mar 12
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11.03.21 23:39, Walter Bright wrote:
 On 3/11/2021 10:43 AM, Timon Gehr wrote:
 I haven't had time yet to contribute a thorough review, but one thing 
 that stands out to me is that there seems to be no discussion of 
 interaction with `const`/`immutable`, etc. Given that that's a source 
 of unsoundness for postblit, maybe it deserves some explicit 
 consideration?
All the problems with const/immutable revolved around postblit. That's why this proposal has zero reliance on postblit.
My concern was that similar mistakes can be repeated. You don't ensure correctness just by avoiding the precise set of previous mistakes; it's always possible to make new, slightly different mistakes. The design of immutable and the design of postblit took place independently of each other and they ended up being incompatible. Why is the same thing not happening now? The DIP seems to assume there is no such thing as immutable, just like postblit did. A naive implementation of the DIP might therefore implicitly cast away immutable, just like postblit did.
Mar 12
parent reply Max Haughton <maxhaton gmail.com> writes:
On Friday, 12 March 2021 at 23:52:00 UTC, Timon Gehr wrote:
 On 11.03.21 23:39, Walter Bright wrote:
 On 3/11/2021 10:43 AM, Timon Gehr wrote:
 [...]
All the problems with const/immutable revolved around postblit. That's why this proposal has zero reliance on postblit.
My concern was that similar mistakes can be repeated. You don't ensure correctness just by avoiding the precise set of previous mistakes; it's always possible to make new, slightly different mistakes. The design of immutable and the design of postblit took place independently of each other and they ended up being incompatible. Why is the same thing not happening now? The DIP seems to assume there is no such thing as immutable, just like postblit did. A naive implementation of the DIP might therefore implicitly cast away immutable, just like postblit did.
Is the specific issue here a move from an immutable struct breaking the immutable-contract silently? Thank you for raising it even if not because this needs to be hashed out properly.
Mar 12
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 13.03.21 01:29, Max Haughton wrote:
 On Friday, 12 March 2021 at 23:52:00 UTC, Timon Gehr wrote:
 On 11.03.21 23:39, Walter Bright wrote:
 On 3/11/2021 10:43 AM, Timon Gehr wrote:
 [...]
All the problems with const/immutable revolved around postblit. That's why this proposal has zero reliance on postblit.
My concern was that similar mistakes can be repeated. You don't ensure correctness just by avoiding the precise set of previous mistakes; it's always possible to make new, slightly different mistakes. The design of immutable and the design of postblit took place independently of each other and they ended up being incompatible. Why is the same thing not happening now? The DIP seems to assume there is no such thing as immutable, just like postblit did. A naive implementation of the DIP might therefore implicitly cast away immutable, just like postblit did.
Is the specific issue here a move from an immutable struct breaking the immutable-contract silently? ...
Yes, one thing that could conceivably go wrong is this: safe: int* global; struct S{ int* x; this(S other){ global=other.x; this.x=other.x; } // ... } void main(){ immutable s=S(new int); immutable t=s; // s moved, so move constructor is called assert(t.x is global.x); } This is similar to what went wrong with postblit.
 Thank you for raising it even if not because this needs to be hashed out 
 properly.
Exactly, the issue is basically, even if what happens is not memory corruption, maybe the DIP should state what it is that does happen. :) E.g., I think it would be good to address questions like those somehow: Is moving an immutable object allowed? (E.g., with system variables I think you could have a trusted immutable object that's backed by malloc and deallocated on destruction, can we move that?) Is this a move constructor? struct S{ this(immutable(S) other)immutable{ ... } } On an unrelated note, I guess the DIP should also show how to actually declare a copy constructor. Is it this(ref S) ?
Mar 12
prev sibling next sibling parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of Community 
 Review of DIP 1040, "Copying, Moving, and Forwarding":

 [...]
How does it handle move constructors when a class have struct type variable that is being alias this? -Alex
Mar 11
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/11/2021 10:56 AM, 12345swordy wrote:
 How does it handle move constructors when a class have struct type variable
that 
 is being alias this?
#DIP1040 only applies to structs, not classes.
Mar 12
parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Saturday, 13 March 2021 at 03:06:35 UTC, Walter Bright wrote:
 On 3/11/2021 10:56 AM, 12345swordy wrote:
 How does it handle move constructors when a class have struct 
 type variable that is being alias this?
#DIP1040 only applies to structs, not classes.
That doesn't answer the question. How does the DIP interact with the current alias this system? If I alias this a struct variable with move schematics in struct/class definition called A, does A have the move schematics of the struct variable? Replying "only applied to structs and not classes" isn't helpful here. I am not talking about defining move schematics for classes, I am talking about the class inheriting the struct that has move schematics defined, via alias this. - Alex
Mar 13
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Saturday, 13 March 2021 at 19:47:35 UTC, 12345swordy wrote:
 On Saturday, 13 March 2021 at 03:06:35 UTC, Walter Bright wrote:
 On 3/11/2021 10:56 AM, 12345swordy wrote:
 How does it handle move constructors when a class have struct 
 type variable that is being alias this?
#DIP1040 only applies to structs, not classes.
That doesn't answer the question. How does the DIP interact with the current alias this system? If I alias this a struct variable with move schematics in struct/class definition called A, does A have the move schematics of the struct variable? Replying "only applied to structs and not classes" isn't helpful here. I am not talking about defining move schematics for classes, I am talking about the class inheriting the struct that has move schematics defined, via alias this. - Alex
Considering alias this is just an identifier resolution rule, why would you expect any interaction whatsoever with move semantics?
Mar 13
parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Saturday, 13 March 2021 at 21:09:33 UTC, deadalnix wrote:
 On Saturday, 13 March 2021 at 19:47:35 UTC, 12345swordy wrote:
 On Saturday, 13 March 2021 at 03:06:35 UTC, Walter Bright 
 wrote:
 On 3/11/2021 10:56 AM, 12345swordy wrote:
 How does it handle move constructors when a class have 
 struct type variable that is being alias this?
#DIP1040 only applies to structs, not classes.
That doesn't answer the question. How does the DIP interact with the current alias this system? If I alias this a struct variable with move schematics in struct/class definition called A, does A have the move schematics of the struct variable? Replying "only applied to structs and not classes" isn't helpful here. I am not talking about defining move schematics for classes, I am talking about the class inheriting the struct that has move schematics defined, via alias this. - Alex
Considering alias this is just an identifier resolution rule, why would you expect any interaction whatsoever with move semantics?
You are making an argument from silence fallacy, if you are asserting that there is no interaction between move semantics with alias this by the virtue of not being mentioning in the dip at all. - Alex
Mar 13
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 14 March 2021 at 01:44:19 UTC, 12345swordy wrote:
 You are making an argument from silence fallacy, if you are 
 asserting that there is no interaction between move semantics 
 with alias this by the virtue of not being mentioning in the 
 dip at all.

 - Alex
Alright, can you ensure how there is no interaction with that teapot orbiting around the sun between Mercury and Venus? Careful because if you don't, you'd be making an argument from silence fallacy, and you sure wouldn't want to do that. The rest of the sentence pretty much boils down to this: https://i.imgflip.com/51rqyi.jpg If this is what i wanted to assert, you can trust this is what I would have asserted.
Mar 15
parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Monday, 15 March 2021 at 13:21:15 UTC, deadalnix wrote:
 On Sunday, 14 March 2021 at 01:44:19 UTC, 12345swordy wrote:
 You are making an argument from silence fallacy, if you are 
 asserting that there is no interaction between move semantics 
 with alias this by the virtue of not being mentioning in the 
 dip at all.

 - Alex
Alright, can you ensure how there is no interaction with that teapot orbiting around the sun between Mercury and Venus Careful because if you don't, you'd be making an argument from silence fallacy, and you sure wouldn't want to do that.
I am not the one who is making assertions with regards to the DIP based on the virtual of not being mention, you are. Thus your comparisons fails here. - Alex
Mar 15
parent 12345swordy <alexanderheistermann gmail.com> writes:
On Monday, 15 March 2021 at 16:40:27 UTC, 12345swordy wrote:
 On Monday, 15 March 2021 at 13:21:15 UTC, deadalnix wrote:
 On Sunday, 14 March 2021 at 01:44:19 UTC, 12345swordy wrote:
 You are making an argument from silence fallacy, if you are 
 asserting that there is no interaction between move semantics 
 with alias this by the virtue of not being mentioning in the 
 dip at all.

 - Alex
Alright, can you ensure how there is no interaction with that teapot orbiting around the sun between Mercury and Venus Careful because if you don't, you'd be making an argument from silence fallacy, and you sure wouldn't want to do that.
I am not the one who is making assertions with regards to the DIP based on the virtual of not being mention, you are. Thus your comparisons fails here. - Alex
Meant to say virtue not virtual. Woops - Alex
Mar 15
prev sibling next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 13 March 2021 at 19:47:35 UTC, 12345swordy wrote:
 How does the DIP interact with the current alias this system?
 If I alias this a struct variable with move schematics in 
 struct/class definition called A, does A have the move 
 schematics of the struct variable?
According to the DIP, if A does not define explicit custom move operators, it gets implicit default field-by-field move operators:
 If a Move Constructor is not defined for a struct that has a 
 Move Assignment Operator, a default Move Constructor is defined 
 and implemented as a move for each of its fields, in lexical 
 order.
 ...
 If a Move Assignment Operator is not defined for a struct that 
 has a Move Constructor, a default Move Assignment Operator is 
 defined and implemented as a move for each of its fields, in 
 lexical order.
So, A's own move operators (whether explicit custom or implicit default) should hide any that it might otherwise pick up via alias this. The DIP should probably be updated to make this interaction with alias this explicit, though.
Mar 13
parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/13/2021 8:27 PM, tsbockman wrote:
 On Saturday, 13 March 2021 at 19:47:35 UTC, 12345swordy wrote:
 How does the DIP interact with the current alias this system?
 If I alias this a struct variable with move schematics in struct/class 
 definition called A, does A have the move schematics of the struct variable?
According to the DIP, if A does not define explicit custom move operators,
There is no such thing as a "struct/class definition". Structs and classes are different. The DIP defines move semantics for structs, not for classes.
Mar 15
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/13/2021 11:47 AM, 12345swordy wrote:
 On Saturday, 13 March 2021 at 03:06:35 UTC, Walter Bright wrote:
 On 3/11/2021 10:56 AM, 12345swordy wrote:
 How does it handle move constructors when a class have struct type variable 
 that is being alias this?
#DIP1040 only applies to structs, not classes.
That doesn't answer the question.
The question doesn't make a lot of sense, as what does it have to do with classes?
 How does the DIP interact with the current alias this system?
 If I alias this a struct variable with move schematics in struct/class 
 definition called A, does A have the move schematics of the struct variable? 
 Replying "only applied to structs and not classes" isn't helpful here. I am
not 
 talking about defining move schematics for classes, I am talking about the
class 
 inheriting the struct that has move schematics defined, via alias this.
Classes don't inherit structs. But I will guess at what you mean. We've discovered that alias this and classes produce a semantic quagmire that every proposed resolution just leads to worse weird cases. Hence: 1. I recommend NEVER using alias this in a class. 2. I'd like to make it illegal. 3. Any questions about how alias this works in a class, I refer to (1). DIP1040 does not apply to classes. If you have a class that has an alias this that refers to a struct with move semantics, you're going to be sorry :-/ I recommend NEVER using alias this in a class.
Mar 15
parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Monday, 15 March 2021 at 08:01:02 UTC, Walter Bright wrote:
 On 3/13/2021 11:47 AM, 12345swordy wrote:
 On Saturday, 13 March 2021 at 03:06:35 UTC, Walter Bright 
 wrote:
 On 3/11/2021 10:56 AM, 12345swordy wrote:
 How does it handle move constructors when a class have 
 struct type variable that is being alias this?
#DIP1040 only applies to structs, not classes.
That doesn't answer the question.
The question doesn't make a lot of sense, as what does it have to do with classes?
 How does the DIP interact with the current alias this system?
 If I alias this a struct variable with move schematics in 
 struct/class definition called A, does A have the move 
 schematics of the struct variable? Replying "only applied to 
 structs and not classes" isn't helpful here. I am not talking 
 about defining move schematics for classes, I am talking about 
 the class inheriting the struct that has move schematics 
 defined, via alias this.
Classes don't inherit structs. But I will guess at what you mean. We've discovered that alias this and classes produce a semantic quagmire that every proposed resolution just leads to worse weird cases. Hence: 1. I recommend NEVER using alias this in a class. 2. I'd like to make it illegal. 3. Any questions about how alias this works in a class, I refer to (1). DIP1040 does not apply to classes. If you have a class that has an alias this that refers to a struct with move semantics, you're going to be sorry :-/ I recommend NEVER using alias this in a class.
Then save yourself from future headaches by making it straight up illegal for classes to have alias this a struct that have move schematics. As of now, it is currently possible to have alias this in a class. Which is not going anywhere until someone create a dip that deprecated it. (Which I am 100 percent in favor of btw) -Alex
Mar 15
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Mar 15, 2021 at 04:51:19PM +0000, 12345swordy via Digitalmars-d wrote:
 On Monday, 15 March 2021 at 08:01:02 UTC, Walter Bright wrote:
[...]
 1. I recommend NEVER using alias this in a class.
 
 2. I'd like to make it illegal.
[...]
 Then save yourself from future headaches by making it straight up
 illegal for classes to have alias this a struct that have move
 schematics.
[...] I'm sure Walter would have made alias this illegal in classes a long time ago, if it were possible without problems. I'm suspecting the "don't break existing code" bugbear is among the reasons. I used to be a big fan of alias this, esp. multiple alias this. Now, after some experience with maintaining code that use alias this willy-nilly, I'm starting to agree with Walter's stance that alias this in general was a bad idea. Although there are definitely cases for which it's actually useful, the problems it brings along make it of questionable value as a general language feature. I'd also be in favor of getting rid of it, at least from classes, if not completely. (The latter is probably impossible; quite a lot of my own code relies on it, and I imagine I'm not the only one using it among the users of D.) T -- People say I'm indecisive, but I'm not sure about that. -- YHL, CONLANG
Mar 15
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/15/2021 10:18 AM, H. S. Teoh wrote:
 I'm sure Walter would have made alias this illegal in classes a long
 time ago, if it were possible without problems.  I'm suspecting the
 "don't break existing code" bugbear is among the reasons.
Right.
 I used to be a big fan of alias this, esp. multiple alias this.  Now,
 after some experience with maintaining code that use alias this
 willy-nilly, I'm starting to agree with Walter's stance that alias this
 in general was a bad idea.  Although there are definitely cases for
 which it's actually useful, the problems it brings along make it of
 questionable value as a general language feature.  I'd also be in favor
 of getting rid of it, at least from classes, if not completely. (The
 latter is probably impossible; quite a lot of my own code relies on it,
 and I imagine I'm not the only one using it among the users of D.)
Right, we're kinda stuck with it. But move constructors are a new thing, and ignoring interaction with class alias thing will not break existing code. If someone wants to use move constructors, don't mix them with class alias this. Alias this should only be used with structs, and modestly at that. Being clever with it will only annoy the dragon :-/
Mar 16
parent 12345swordy <alexanderheistermann gmail.com> writes:
On Tuesday, 16 March 2021 at 08:27:27 UTC, Walter Bright wrote:
 On 3/15/2021 10:18 AM, H. S. Teoh wrote:
 I'm sure Walter would have made alias this illegal in classes 
 a long
 time ago, if it were possible without problems.  I'm 
 suspecting the
 "don't break existing code" bugbear is among the reasons.
Right.
 I used to be a big fan of alias this, esp. multiple alias 
 this.  Now,
 after some experience with maintaining code that use alias this
 willy-nilly, I'm starting to agree with Walter's stance that 
 alias this
 in general was a bad idea.  Although there are definitely 
 cases for
 which it's actually useful, the problems it brings along make 
 it of
 questionable value as a general language feature.  I'd also be 
 in favor
 of getting rid of it, at least from classes, if not 
 completely. (The
 latter is probably impossible; quite a lot of my own code 
 relies on it,
 and I imagine I'm not the only one using it among the users of 
 D.)
Right, we're kinda stuck with it. But move constructors are a new thing, and ignoring interaction with class alias thing will not break existing code. If someone wants to use move constructors, don't mix them with class alias this. Alias this should only be used with structs, and modestly at that. Being clever with it will only annoy the dragon :-/
Here is my suggestion, deprecate alias this for classes, but the date for the removal of them to be never. With the ability to silence the deprecation if needed. There done. Simple. It won't break old code, but it will discourage new bad code from being created. -Alex
Mar 16
prev sibling parent reply bachmeier <no spam.net> writes:
On Monday, 15 March 2021 at 17:18:19 UTC, H. S. Teoh wrote:

 Although there are definitely cases for which it's actually 
 useful, the problems it brings along make it of questionable 
 value as a general language feature.  I'd also be in favor of 
 getting rid of it, at least from classes, if not completely. 
 (The latter is probably impossible; quite a lot of my own code 
 relies on it, and I imagine I'm not the only one using it among 
 the users of D.)
A feature that can't be used wrong isn't much of a feature. alias this is easy to use correctly and provides immense value. The removal of useful features is not a good solution to bad program design.
Mar 16
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Mar 16, 2021 at 03:18:49PM +0000, bachmeier via Digitalmars-d wrote:
 On Monday, 15 March 2021 at 17:18:19 UTC, H. S. Teoh wrote:
 
 Although there are definitely cases for which it's actually useful,
 the problems it brings along make it of questionable value as a
 general language feature.  I'd also be in favor of getting rid of
 it, at least from classes, if not completely. (The latter is
 probably impossible; quite a lot of my own code relies on it, and I
 imagine I'm not the only one using it among the users of D.)
A feature that can't be used wrong isn't much of a feature. alias this is easy to use correctly and provides immense value. The removal of useful features is not a good solution to bad program design.
I used to have your stance. But these days, I'm starting to realize more and more that implicit conversions are almost always a bad idea. They are convenient and fast in the beginning when you're trying to get the job done, but in the long term, they hurt readability and maintainability. I've come to realize that when the code relies too much on this kind of implicit conversion, esp. via alias this, it's often a sign of poor code structure. It's a code smell. It works, but smells bad, and eventually you realize that it *is* bad. T -- Talk is cheap. Whining is actually free. -- Lars Wirzenius
Mar 16
parent reply bachmeier <no spam.net> writes:
On Tuesday, 16 March 2021 at 16:08:46 UTC, H. S. Teoh wrote:
 On Tue, Mar 16, 2021 at 03:18:49PM +0000, bachmeier via 
 Digitalmars-d wrote:
 On Monday, 15 March 2021 at 17:18:19 UTC, H. S. Teoh wrote:
 
 Although there are definitely cases for which it's actually 
 useful, the problems it brings along make it of questionable 
 value as a general language feature.  I'd also be in favor 
 of getting rid of it, at least from classes, if not 
 completely. (The latter is probably impossible; quite a lot 
 of my own code relies on it, and I imagine I'm not the only 
 one using it among the users of D.)
A feature that can't be used wrong isn't much of a feature. alias this is easy to use correctly and provides immense value. The removal of useful features is not a good solution to bad program design.
I used to have your stance. But these days, I'm starting to realize more and more that implicit conversions are almost always a bad idea. They are convenient and fast in the beginning when you're trying to get the job done, but in the long term, they hurt readability and maintainability. I've come to realize that when the code relies too much on this kind of implicit conversion, esp. via alias this, it's often a sign of poor code structure. It's a code smell. It works, but smells bad, and eventually you realize that it *is* bad.
If you're calling into a C library for matrix operations, but you have three or more strategies for allocating the underlying memory, alias this is reasonable. Otherwise you're writing *extremely* verbose code or you need to engage in extraordinary code duplication or you're writing a bunch of ugly generic code that sits on top of the C library. alias this is a clean, trivial solution in this case.
Mar 16
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 16 March 2021 at 16:55:47 UTC, bachmeier wrote:
 [snip]

 If you're calling into a C library for matrix operations, but 
 you have three or more strategies for allocating the underlying 
 memory, alias this is reasonable. Otherwise you're writing 
 *extremely* verbose code or you need to engage in extraordinary 
 code duplication or you're writing a bunch of ugly generic code 
 that sits on top of the C library. alias this is a clean, 
 trivial solution in this case.
Generic code that sits on top of the C library sounds like the most common solution to me. You can go a long way with a generic library on top of a C library. Lots of generic libraries end up calling BLAS/LAPACK. Regardless, the burden is on those opposed to alias this to provide a solution.
Mar 16
prev sibling parent 12345swordy <alexanderheistermann gmail.com> writes:
On Tuesday, 16 March 2021 at 16:55:47 UTC, bachmeier wrote:
 On Tuesday, 16 March 2021 at 16:08:46 UTC, H. S. Teoh wrote:
 On Tue, Mar 16, 2021 at 03:18:49PM +0000, bachmeier via 
 Digitalmars-d wrote:
 On Monday, 15 March 2021 at 17:18:19 UTC, H. S. Teoh wrote:
 
 Although there are definitely cases for which it's actually 
 useful, the problems it brings along make it of 
 questionable value as a general language feature.  I'd also 
 be in favor of getting rid of it, at least from classes, if 
 not completely. (The latter is probably impossible; quite a 
 lot of my own code relies on it, and I imagine I'm not the 
 only one using it among the users of D.)
A feature that can't be used wrong isn't much of a feature. alias this is easy to use correctly and provides immense value. The removal of useful features is not a good solution to bad program design.
I used to have your stance. But these days, I'm starting to realize more and more that implicit conversions are almost always a bad idea. They are convenient and fast in the beginning when you're trying to get the job done, but in the long term, they hurt readability and maintainability. I've come to realize that when the code relies too much on this kind of implicit conversion, esp. via alias this, it's often a sign of poor code structure. It's a code smell. It works, but smells bad, and eventually you realize that it *is* bad.
If you're calling into a C library for matrix operations, but you have three or more strategies for allocating the underlying memory, alias this is reasonable. Otherwise you're writing *extremely* verbose code or you need to engage in extraordinary code duplication or you're writing a bunch of ugly generic code that sits on top of the C library. alias this is a clean, trivial solution in this case.
Meta programing is meant to solve the verbose code problem. What is preventing you from using tools such as templates and string mixins? - Alex
Mar 16
prev sibling next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of Community 
 Review of DIP 1040, "Copying, Moving, and Forwarding":
The default field-wise move operators seem to have been specified with no concern for maintaining consistency between the semantics of custom operators and default operators: From the DIP:
 If a Move Assignment Operator is not defined for a struct that
 has a Move Constructor, a default Move Assignment Operator is
 defined and implemented as a move for each of its fields, in
 lexical order.
If a custom move constructor is present for a good reason, that means that simply moving the fields one-by-one is *not* the desired move behavior. Instead, the default move assignment operator should call the struct's move constructor and destructor in one of the two valid patterns that I found earlier in this discussion (link to a full working example): https://gist.github.com/run-dlang/b789714c01905f091a44ee2666276433 // Using an alternative syntax so that this can be tested today: void moveAssign(ref S source) trusted nothrow nogc { static if(useDIPLowering) { // destroy after (the DIP's proposal): S newVal = void; newVal.moveConstruct(source); S oldVal = void; oldVal.moveConstruct(this); moveConstruct(newVal); // Implicitly destruct(oldVal). } else { // conditionally move and destroy before (my proposal): if(&source !is &this) { destruct(this); moveConstruct(source); } } } Also from the DIP:
 If a Move Constructor is not defined for a struct that has a 
 Move
 Assignment Operator, a default Move Constructor is defined and
 implemented as a move for each of its fields, in lexical order.
Again, the presence of a custom move assignment operator indicates that non-default move behavior is desired. Auto-generating a default move constructor is a missed opportunity to point out a semantic inconsistency in the user's work. Defining a custom move assignment operator without a custom move constructor to go with it should be a compile-time error, because there is no automated way to extract a valid move constructor from a custom move assignment operator. In the unlikely case that the omission of the custom move constructor is intentional, people can manually defining a field-wise move constructor (which is not difficult), or annotate that constructor with disable.
Mar 12
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/12/2021 3:47 PM, tsbockman wrote:
 ...
You make a good point, that the user should define both the Move Assignment and Move Constructor, or neither. Just one of the pair should be an error. I agree that it would be red flag if just one appeared in the code, indicative that the programmer did not think it through.
Mar 12
parent reply deadalnix <deadalnix gmail.com> writes:
On Saturday, 13 March 2021 at 03:12:02 UTC, Walter Bright wrote:
 On 3/12/2021 3:47 PM, tsbockman wrote:
 ...
You make a good point, that the user should define both the Move Assignment and Move Constructor, or neither. Just one of the pair should be an error. I agree that it would be red flag if just one appeared in the code, indicative that the programmer did not think it through.
The whole notion of move assignement is a hack inherited from C++ and don't really make sense for D. As soon as you have 2 objects, you are out of the pure move scenario, and you need to deal with this fact. C++ does so by putting the source object in a null state and then destroying it. You cannot have *2* objects and then not destroy one and expect things to not have a ton of edge cases just like postblit for copy does (because you have the exact same problem in reverse there, you had *1* object when you wanted *2*).
Mar 13
parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/13/2021 5:06 AM, deadalnix wrote:
 The whole notion of move assignement is a hack inherited from C++ and don't 
 really make sense for D.
 As soon as you have 2 objects, you are out of the pure move scenario, and you 
 need to deal with this fact. C++ does so by putting the source object in a
null 
 state and then destroying it. You cannot have *2* objects and then not destroy 
 one and expect things to not have a ton of edge cases just like postblit for 
 copy does (because you have the exact same problem in reverse there, you had
*1* 
 object when you wanted *2*).
The move assignment has access to both objects, and so can "do the right thing".
Mar 15
prev sibling parent reply vitamin <vit vit.vit> writes:
On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of Community 
 Review of DIP 1040, "Copying, Moving, and Forwarding":

 https://github.com/dlang/DIPs/blob/a9c553b0dbab1c2983a801b5e89b51c5c33d5180/DIPs/DIP1040.md
Hello, I like this dip but, must be move/copy methods ctors? Because there need to be distinctions between copy/move ctors and other ctors, all copy/move ctors are non templates and that make some problems. Something like opMoveCtor and opCopyCtor are easier differentiate from others ctors and can be template: void opMoveCtor(T, this This)(T rhs){ //traits like hasMoveConstructor can work without instantion of opMoveCtor } instead of this(typeof(this) rhs){/*...*/} this(const typeof(this) rhs)const {/*...*/} this(immutable typeof(this) rhs)immutable {/*...*/} //and all other combination including inout and sometimes shared.
Mar 16
parent reply Max Haughton <maxhaton gmail.com> writes:
On Wednesday, 17 March 2021 at 06:35:16 UTC, vitamin wrote:
 On Friday, 5 March 2021 at 12:19:54 UTC, Mike Parker wrote:
 This is the discussion thread for the first round of Community 
 Review of DIP 1040, "Copying, Moving, and Forwarding":

 https://github.com/dlang/DIPs/blob/a9c553b0dbab1c2983a801b5e89b51c5c33d5180/DIPs/DIP1040.md
Hello, I like this dip but, must be move/copy methods ctors? Because there need to be distinctions between copy/move ctors and other ctors, all copy/move ctors are non templates and that make some problems. Something like opMoveCtor and opCopyCtor are easier differentiate from others ctors and can be template: void opMoveCtor(T, this This)(T rhs){ //traits like hasMoveConstructor can work without instantion of opMoveCtor } instead of this(typeof(this) rhs){/*...*/} this(const typeof(this) rhs)const {/*...*/} this(immutable typeof(this) rhs)immutable {/*...*/} //and all other combination including inout and sometimes shared.
Some thoughts: ignore the implementation for now, but think about how the method based approach would change the language specification - the whole thesis of this particular DIP is (beyond move semantics themselves) to make move semantics (in a sense) more natural than the C++ solution, whereas with a method the decision to pass-by-move is now performed based on the presence of specific template arg to the operator. Also it's a constructor, so why not call it one as we do now. And for the specialisations, when you need to do these for the most part you end up doing them explicitly anyway (unless you are DbI-ing like a madman I guess). Anyway those are my slightly rambling thoughts for now, more coming later.
Mar 16
parent vitamin <vit vit.vit> writes:
On Wednesday, 17 March 2021 at 06:51:54 UTC, Max Haughton wrote:
 On Wednesday, 17 March 2021 at 06:35:16 UTC, vitamin wrote:
 [...]
Some thoughts: ignore the implementation for now, but think about how the method based approach would change the language specification - the whole thesis of this particular DIP is (beyond move semantics themselves) to make move semantics (in a sense) more natural than the C++ solution, whereas with a method the decision to pass-by-move is now performed based on the presence of specific template arg to the operator. [...]
Only thing I want is possibility to create template move/ctor. All other things stay same as in dip.
Mar 17