www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - funky property error with assoc array

reply "Dan" <dbdavidson yahoo.com> writes:
This fails to compile when accessing as m.pgoo() complaining 
about postblit.
What is wrong with this?

Note: If I alias as array instead of map: alias const(X)[] Map;
it compiles fine.

Thanks
Dan
-----------------------------
struct X { this(this) {} }
alias const(X)[string] Map;
 property int pgoo(ref Map x) { return 3; }

void main() {
   Map m;
   pgoo(m);
   m.pgoo();
}
Nov 14 2012
next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, November 14, 2012 20:48:57 Dan wrote:
 This fails to compile when accessing as m.pgoo() complaining
 about postblit.
 What is wrong with this?
 
 Note: If I alias as array instead of map: alias const(X)[] Map;
 it compiles fine.
 
 Thanks
 Dan
 -----------------------------
 struct X { this(this) {} }
 alias const(X)[string] Map;
  property int pgoo(ref Map x) { return 3; }
 
 void main() {
 Map m;
 pgoo(m);
 m.pgoo();
 }

Postblit doesn't work with const objects and there's not currently any way to make it work with const objects. So, any attempt to copy any element in your AA won't work. So, if the AA implementation has any code in it which requires copying an element (and it probably does), then a type with a postblit in it won't work if it's const. At present const and postblit don't get allow at all, and it sounds like the solution that Andrei and Walter have been cooking up will probably involve adding copy constructors, but nothing has actually happened yet. postblit fundamentally doesn't work with const or immutable though and can't, because you can't modify const or immutable objects, and postblit requires you to alter the object. So, as far as const and immutable go, postblit is fundamentally flawed. - Jonathan M Davis
Nov 14 2012
prev sibling next sibling parent "Dan" <dbdavidson yahoo.com> writes:
On Wednesday, 14 November 2012 at 22:07:10 UTC, Jonathan M Davis 
wrote:
 Postblit doesn't work with const objects and there's not 
 currently any way to
 make it work with const objects. So, any attempt to copy any 
 element in your
 AA won't work. So, if the AA implementation has any code in it 
 which requires
 copying an element (and it probably does), then a type with a 
 postblit in it
 won't work if it's const.

Jonathan, thanks for taking the time on this and all my other questions. Coming from C++ and trying to hammer out how to use D I've hit this postblit many times and many ways. I agree that postblit does not work with const - and my feeling is maybe it should not and I hope there is a work around (for structs). For instance: ------------ import opmix.mix; import std.stdio; struct Big { char [] c ; } void main () { const ( Big ) s = { " test " }; Big other = s.gdup; writeln(other); } ------------
 At present const and postblit don't get allow at all, and it 
 sounds like the
 solution that Andrei and Walter have been cooking up will 
 probably involve

I think that is fine they don't get along. postblit can not guarantee no data is shared after the call, so they should not. In the example here I don't access the map at all and the call works fine in non-property form.
 adding copy constructors, but nothing has actually happened yet.

I do not see how copy constructors can help. Receiving const(T) or const(T) ref to anything and hoping to copy it is the challenge and doing it from a copy constructor is no easier than a postblit. I think bearophile pointed it out best: In practice to copy something const to something that's not const you need a deep copy function, because inside the array there can be other arrays that are const, etc. Transitive const requires transitive copy, it's easy :-) If you have any extra cycles I would love feedback on (dup and Global Dup) in: https://github.com/patefacio/d-help/blob/master/doc/canonical.pdf Actually feedback on all would be great because it establishes how I want to work with structs and being new to D I'd like assurances it is safe/correct. Thanks Dan
 postblit
 fundamentally doesn't work with const or immutable though and 
 can't, because
 you can't modify const or immutable objects, and postblit 
 requires you to
 alter the object. So, as far as const and immutable go, 
 postblit is
 fundamentally flawed.

 - Jonathan M Davis

Nov 15 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, November 15, 2012 14:21:41 Dan wrote:
 I do not see how copy constructors can help. Receiving const(T)
 or const(T) ref to anything and hoping to copy it is the
 challenge and doing it from a copy constructor is no easier than
 a postblit.

Doing it from a postblit is impossible. Doing it from a copy constructor _is_ possible (assuming that they're added to the language - they don't help you any until they are). The core problem with postblit constructor is that it does a shallow copy _before_ entering the postblit constructor. This seems great at first, because then you only have to copy stuff in the postblit itself which actually needs a deep copy. But it fails completely with const and immutable, because it's illegal to alter anything which is const or immutable - even by casting them to mutable temporarily to make the change (doing so is undefined behavior). So, you can't change _anything_ in a const postblit constructor, meaning that you can't make a deep copy of anything. A copy constructor, on the other hand, does not do any copying beforehand. It all must still be constructed. The copy constructor then constructs the new object with deep copies of the data. This works with const and immutable, because it's construction rather than mutation. The postblit constructor fails because it requires you to use mutation, which doesn't work with const. So, as nice an idea as it is, the postblit constructor is a failure. And as such, I believe that Walter and Andrei intend to replace it with copy constructors in the long term (though the postblit constructor will probably stay in the language for compatibility for a very, very long time, if not permanently). Unfortunately however, nothing has been done about this on the implementation side of things (they've only discussed it, not done anything about it), so right now, we don't have a solution, since it's not like you can go and declare your own copy constructor and have it work, because it'll never be called unless you call it explicitly. Rather, the language would use the postblit constructor if it's there and complain about it not working with const if you try it, even if you have a copy constructor which would do the job. - Jonathan M Davis
Nov 15 2012
prev sibling next sibling parent "Dan" <dbdavidson yahoo.com> writes:
On Thursday, 15 November 2012 at 17:24:44 UTC, Jonathan M Davis 
wrote:
 On Thursday, November 15, 2012 14:21:41 Dan wrote:
 I do not see how copy constructors can help. Receiving const(T)
 or const(T) ref to anything and hoping to copy it is the
 challenge and doing it from a copy constructor is no easier 
 than
 a postblit.

Doing it from a postblit is impossible.

I understand and this makes sense. postblit is not a deep copy, it is only what you choose to make it.
 Doing it from a copy constructor _is_
 possible (assuming that they're added to the language - they 
 don't help you
 any until they are).

I question the need for this - at least for structs composed of structs, dynamic arrays and associative arrays. Classes are another issue.
 The core problem with postblit constructor is that it
 does a shallow copy _before_ entering the postblit constructor. 
 This seems
 great at first, because then you only have to copy stuff in the 
 postblit itself
 which actually needs a deep copy. But it fails completely with 
 const and
 immutable, because it's illegal to alter anything which is 
 const or immutable
 - even by casting them to mutable temporarily to make the 
 change (doing so is
 undefined behavior). So, you can't change _anything_ in a const 
 postblit
 constructor, meaning that you can't make a deep copy of 
 anything.

Understood. I don't think the problem with postblit is what it does _before_ I do my thing. The problem for this case is it can not guarantee what I do *in* my postblit and therefore it does not and can not know I'm leaving with no sharing. I don't see this as a failure, though. You want a copy, then you need to make a copy. Find a way other than postblit. The suggestion is use copy constructors instead. I don't see how this will help - but I have not found the proposal (do you have a link?). I have read DIP10 which was an early attempt to get at this stuff - but I think it did not go far.
 A copy constructor, on the other hand, does not do any copying 
 beforehand. It
 all must still be constructed. The copy constructor then 
 constructs the new
 object with deep copies of the data. This works with const and 
 immutable,
 because it's construction rather than mutation. The postblit 
 constructor fails
 because it requires you to use mutation, which doesn't work 
 with const.

A copy ctor does not copy anything beforehand. True. Then the idea with this new feature is to leave it to you to copy the fields from inside your new copy ctor. But you will no more be able to copy them from a copy ctor than you were from a postblit because you have a const instance as your source and something extra is needed for each field. Now if that something extra is more copy constructors all down the line - I think that would work but is unnecessary. I think that the real need is a global dup or deep copy function. It should be possible without touching our structs and I think I have something fairly close when it comes to structs, associative arrays, and dynamic arrays.
 So, as nice an idea as it is, the postblit constructor is a 
 failure.

I feel it is not a failure. It is doing its job just fine. The natural tendency, especially coming from c++ is to assume const values assigned into non-const lvalues should be ok - because we are used to deep copy semantics just working. The only problem is copy construction (and therefore postblit) is not the right tool to copy const instances with reference semantics. I think a global dup function would be the right tool and it is doable, hopefully with a reasonable performance overhead. The issue is you want to copy a const(T). Suppose I give you a function that works like this (by the way this is working code (I use 2.061)) import std.stdio; import opmix.mix; struct S { char[] c; } void foo(ref const(S) s) { // I really need a copy here auto copy = s.gdup; assert(s.c.ptr != copy.c.ptr); writeln(copy); } void main() { S s = {['a','b','c']}; foo(s); } Now if you had this gdup, all your copy ctors would look like this: struct S { char[] c; S(ref const(S) s) { swap(this, s.gdup); } } But if you had that gdup in the first place, there would be no need for copy ctors. Besides, if the compiler could *know* that my new copy ctor was really doing a deep copy - why make me write it in the first place. Just provide the implementation for me. Allowing me to do anything in the copy ctor allows me to introduce sharing which is forbidden. Does this make sense? Thanks Dan
Nov 15 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, November 15, 2012 19:30:03 Dan wrote:
 On Thursday, 15 November 2012 at 17:24:44 UTC, Jonathan M Davis
 
 wrote:
 On Thursday, November 15, 2012 14:21:41 Dan wrote:
 I do not see how copy constructors can help. Receiving const(T)
 or const(T) ref to anything and hoping to copy it is the
 challenge and doing it from a copy constructor is no easier
 than
 a postblit.

Doing it from a postblit is impossible.

I understand and this makes sense. postblit is not a deep copy, it is only what you choose to make it.

The entire reason that postblit exists is to allow deep copies, just like with a copy constructor. You can certainly create a type which doesn't do a deep copy in its postblit constructor, but the reason that it's in the language is to make it so that you can do a deep copy of member variables which normally would end up being shallowly copied. It's D's attempt at making a better copy constructor. Pure and simple. But because of issues with const, it fails at that.
 The problem for this case is it can
 not guarantee what I do *in* my postblit and therefore it does
 not and can not know I'm leaving with no sharing.

The main problem is that once you're inside the postblit construct, the new object has already been fully constructed. And since it's illegal to cast away const and modify an object, it is therefore impossible to modify the new object inside the postblit constructor, making the postblit constructor utterly pointless and making it impossible to copy that struct through any standard means. You could create your own dup method, but that doesn't help one iota with all of the places in the language where you need the language to be making copies for you (e.g. when you dup an array of that struct type). Creating a non-const copy of a const object doesn't work either, but you're totally screwed if you can't even make a deep copy of const copy of a const object. Copy constructors would solve both problems.
 The suggestion is use
 copy constructors instead. I don't see how this will helpx

A copy constructor will help, because it will make it possible to copy const objects which have member variables which are reference types and have a deep copy made because you'll be constructing the new object directly rather than copying it and then making changes to it. Right now, all you can get is a shallow copy, because postblit doesn't work with const.
 but I
 have not found the proposal (do you have a link?). I have read
 DIP10 which was an early attempt to get at this stuff - but I
 think it did not go far.

There is no DIP for it. Andrei and Walter have discussed it in private, and all they would say for a while was that they had a solution. They wouldn't give what it was. Andrei did finally mention not too long ago that it involved introducing copy constructors and phasing out postblit constructors. But nothing has happened yet, and it has not yet been discussed in detail in the newsgroup, so what exactly is involved beyond the fact that they intend to introduce copy constructors to solve the problem, I don't know.
 The only problem is
 copy construction (and therefore postblit) is not the right tool
 to copy const instances with reference semantics.

That's a separate issue from the fact that const postblits don't work. If you've given a type reference semantics, then obviously copy constructors and postblits aren't going to do deep copies. For that, you'd probably create a dup function. But many, many types need to be able to be copied as value types even though they have member variables which are reference types - which is what copy constructors and postblit constructors are for - and they can't do that in D right if they're const, because const and postblit don't mix. - Jonathan M Davis
Nov 15 2012
prev sibling next sibling parent "Dan" <dbdavidson yahoo.com> writes:
Thanks, I think I get it. const and postblit don't mix. Copy 
ctors don't yet exist but are the hoped for solution to inability 
to copy const instances. Copy ctor proposal/solution has not yet 
been written up in the newsgroups or details discussed beyond 
Andrei and Walter, but have been and maybe still are being worked.

I wish I knew more about the copy ctor proposal.

Here is the issue that haunts me: Say it is January 15 and we 
have:

module goo;
struct Goo {
   int x = 42;
}

module foo;
struct Foo {
   char[] c;
   Goo goo;
   this(ref const(Foo) other) {
     c = other.dup;
     goo = other;
   }
}

February comes around and developers of Goo want to introduce a 
new private member *int[int] ageToWeight*. Now foo won't compile, 
because Goo does not also have a copy constructor and went from 
value semantics to reference semantics in the change. This seems 
very fragile - but maybe that is not the proposal. I guess I was 
just hoping copying of const can be taken out of the struct's 
hands entirely. Contrast that situation with the state on 
February with a proper global dup. There is no copy ctor problem, 
no postblit issue. Using something other than the copy ctor and 
postblit to do the copy of the const has greatly reduced effort. 
This code below does work right now the way you would expect. I'm 
not saying it is a panacea - I've only tried it out a bit, but 
you have to admit a global dup like function that could copy 
*any* const you throw at it might put this entire issue to rest 
without the need for any change. Mine could do not that; I'm 
specifically avoiding classes. But I don't know what inherent 
limitation would prevent it. Even if you were concerned with some 
type T that had its own low level memory management, you could 
still have it provide it's own dup with a proper const qualifier, 
which gdup could call in preference to a recursive call to gdup.

You mention that possibly postblits will go away. That is 
discomforting.
In terms of priority, where does this fall?

Thanks
Dan

-----------------------------------------
import std.stdio;
import std.traits;
import opmix.mix;
struct Goo {
   private int[int] ageToWeight;
   int x = 42;
}

struct Foo {
   char[] c;
   Goo goo;
}

void main() {
   Foo foo = { ['a'], Goo([15:123, 30:180, 45:195]) };
   auto copy = foo.gdup;
   // No sharing of c and ageToWeight copied
   assert(copy.c.ptr != foo.c.ptr &&
          copy.goo.ageToWeight == foo.goo.ageToWeight);
   // No sharing of ageToWeight
   copy.goo.ageToWeight[70] = 178;
   assert(copy.goo.ageToWeight != foo.goo.ageToWeight);
   writeln(foo.gdup);
}
-------------------------------------------

Thanks,
Dan
Nov 15 2012
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, November 16, 2012 06:17:55 Dan wrote:
 You mention that possibly postblits will go away. That is
 discomforting.

It would likely stay around for a very long time for backwards compatibility and simply be discouraged, but I don't know. Walter is big on backwards compatilbility though, so it's not like postblit will suddenly be removed and break everyone's code.
 In terms of priority, where does this fall?

No idea. Clearly, it's not super high priority, or it would have been dealt with by now. But there have also been plenty of cases where nothing happens with something ages and then Walter suddenly fixes it. So who knows when it'll happen. - Jonathan M Davis
Nov 15 2012