www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Relocatable objects and internal pointers

reply Matt Elkins <notreal fake.com> writes:
Hi all, I'm a C++ programmer trying to decide whether to switch 
my main focus to D, and so I'm working on a pet project using it. 
So far I really like some of the practical aspects of the 
language (built-in contracts are great, the metaprogramming is 
very accessible, and I can't enough of these compile speeds!), 
but I keep finding myself frustrated by what seem like 
expressiveness limitations (unless, as I hope, they are just 
examples of my newbieness shining through). Case in point:

In an attempt to work around one apparent limitation (previously 
asked about here 
http://forum.dlang.org/thread/eizmagtimvetoganawrr forum.dlang.org) I came up
with an idea which would require storing internal points in a struct. A very
stripped-down but illustrative example would be something like this:

[code]
struct Foo
{
     invariant
     {
         assert(m_this == &this);
     }

      disable(this);

     this(/* arguments to populate stuff */)
     {
         m_this = &this;
         /* ... populate stuff ... */
     }

     this(this)
     {
         m_this = &this;
         /* ... do more stuff ... */
     }

     private:
         Foo* m_this;
         /* ... stuff ... */
}
[/code]

This is just a piece of what I am doing, if you are wondering why 
I am bothering to save a pointer to this. However, I was doing 
some reading on D and came across a section in TDPL which said 
internal pointers are verboten because objects must be 
relocatable. Does this mean my example is invalid (e.g., the 
invariant will not hold in all circumstances)? If it is invalid, 
does that mean there are circumstances under which the post-blit 
constructor can be elided when performing a copy or copy-like 
operation (such as a move)? I've been treating it like a sort of 
copy-constructor that lacks visibility on the copied-from object, 
but maybe that's a mistake...
Jan 29
next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/29/2016 05:07 PM, Matt Elkins wrote:

      this(/* arguments to populate stuff */)
      {
          m_this = &this;
          /* ... populate stuff ... */
      }
 a section in TDPL which said internal pointers are
 verboten because objects must be relocatable. Does this mean my example
 is invalid
Yes, D explicitly bans internal pointers.
 does that mean there are circumstances under which the
 post-blit constructor can be elided when performing a copy or copy-like
 operation (such as a move)?
Definitely so. Rvalues are moved around all the time. The following program has two rvalue moves without calling post-blits or destructors. struct Foo { this(this) { assert(false); // not expected to be called in this program } } Foo makeFoo() { return Foo(); } void takesFoo(Foo foo) { } void main() { Foo foo; foo = makeFoo(); // post-blit not called takesFoo(Foo()); // post-blit not called } Ali
Jan 29
parent reply Matt Elkins <notreal fake.com> writes:
On Saturday, 30 January 2016 at 01:18:33 UTC, Ali Çehreli wrote:
 Definitely so. Rvalues are moved around all the time. The 
 following program has two rvalue moves without calling 
 post-blits or destructors.
Oi, that makes life tough. Ok, I'll figure something else out, then... Thanks for the response!
Jan 29
parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Sat, Jan 30, 2016 at 01:21:27AM +0000, Matt Elkins via Digitalmars-d-learn
wrote:
 On Saturday, 30 January 2016 at 01:18:33 UTC, Ali Çehreli wrote:
Definitely so. Rvalues are moved around all the time. The following
program has two rvalue moves without calling post-blits or
destructors.
Oi, that makes life tough. Ok, I'll figure something else out, then...
[...] Keep in mind that D structs are conceptually different from C++ structs (even if they are similarly implemented). D structs are supposed to be value types with POD-like semantics; so when passing structs around they are bit-copied into the destination and then the postblit method (this(this)) is called to "patch up" the copy. This is unlike in C++ where you have copy ctors and dtors and operator=() to manage copying. Because there are no copy ctors, having internal pointers can be dangerous, since structs can move around in memory without any warning (e.g., returning a struct from a function generally involves copying it from the callee's stack frame into a local variable in the caller's stack frame). If you need something with internal pointers, you might want to consider classes instead. Either that, or be sure to allocate your structs on the heap instead, and work with pointers instead of the struct values directly. (Note that this is still risky, since somebody might dereference the pointer and get a stack copy of the struct, which will cause problems when it then gets passed around.) T -- I am a consultant. My job is to make your job redundant. -- Mr Tom
Jan 29
parent Matt Elkins <notreal fake.com> writes:
On Saturday, 30 January 2016 at 01:28:54 UTC, H. S. Teoh wrote:
 On Sat, Jan 30, 2016 at 01:21:27AM +0000, Matt Elkins via 
 Digitalmars-d-learn wrote:
 On Saturday, 30 January 2016 at 01:18:33 UTC, Ali Çehreli 
 wrote:
Definitely so. Rvalues are moved around all the time. The 
following program has two rvalue moves without calling 
post-blits or destructors.
Oi, that makes life tough. Ok, I'll figure something else out, then...
[...] Keep in mind that D structs are conceptually different from C++ structs (even if they are similarly implemented). D structs are supposed to be value types with POD-like semantics; so when passing structs around they are bit-copied into the destination and then the postblit method (this(this)) is called to "patch up" the copy. This is unlike in C++ where you have copy ctors and dtors and operator=() to manage copying. Because there are no copy ctors, having internal pointers can be dangerous, since structs can move around in memory without any warning (e.g., returning a struct from a function generally involves copying it from the callee's stack frame into a local variable in the caller's stack frame). If you need something with internal pointers, you might want to consider classes instead. Either that, or be sure to allocate your structs on the heap instead, and work with pointers instead of the struct values directly. (Note that this is still risky, since somebody might dereference the pointer and get a stack copy of the struct, which will cause problems when it then gets passed around.) T
Yeah, but the whole point of what I am doing is to avoid using the heap; I can think of several ways to implement this if I relax that restriction :). I'm basically trying to make C++'s std::unique_ptr for resource handles, a thin wrapper which ensures resource cleanup and allows moving the handle. Since I'm putting it in my lowest-level/most-generic library with no visibility on how it gets used, I want it very lightweight (ideally zero-cost, like I can do in C++11, or at least low-cost [sans heap] like I could do in C++98) so that I can use it with the broadest range of higher-level applications.
Jan 29
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/29/16 8:07 PM, Matt Elkins wrote:
 [snip]
 on D and came across a section in TDPL which said internal pointers are
 verboten because objects must be relocatable. Does this mean my example
 is invalid (e.g., the invariant will not hold in all circumstances)? If
 it is invalid, does that mean there are circumstances under which the
 post-blit constructor can be elided when performing a copy or copy-like
 operation (such as a move)? I've been treating it like a sort of
 copy-constructor that lacks visibility on the copied-from object, but
 maybe that's a mistake...
No, you cannot have internal pointers. But... I figured out a way to have them. You just have to guarantee you don't copy the actual "pointer" out of the struct: https://forum.dlang.org/post/mk5k4l$s5r$1 digitalmars.com -Steve
Jan 29
parent reply Matt Elkins <notreal fake.com> writes:
On Saturday, 30 January 2016 at 02:09:55 UTC, Steven 
Schveighoffer wrote:
 I figured out a way to have them. You just have to guarantee 
 you don't copy the actual "pointer" out of the struct:

 https://forum.dlang.org/post/mk5k4l$s5r$1 digitalmars.com
Unfortunately, that won't work for what I was trying to do. The stuff I elided in the comments were more pointers to other Foo instances, used to create a linked-list (of stack-allocated objects); these would still break under the conditions Ali described. I was only storing the this pointer so that blitted objects could deduce where they came from (trying to turn the post-blit constructor into a copy-constructor). Thanks, though. I'm thinking that maybe D just can't express these semantics without substantial overhead. While somewhat disappointing (I came into D with stars in my eyes :)), it's not enough by itself to make me go back to C++, at least not just yet. Not when I can just use a few static ifs to do what previously required careful template crafting that I wouldn't understand 3 months later. On the other hand, I'm falling behind on my library books since I no longer have any time for reading during compilations ;).
Jan 29
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/29/16 9:35 PM, Matt Elkins wrote:
 On Saturday, 30 January 2016 at 02:09:55 UTC, Steven Schveighoffer wrote:
 I figured out a way to have them. You just have to guarantee you don't
 copy the actual "pointer" out of the struct:

 https://forum.dlang.org/post/mk5k4l$s5r$1 digitalmars.com
Unfortunately, that won't work for what I was trying to do. The stuff I elided in the comments were more pointers to other Foo instances, used to create a linked-list (of stack-allocated objects); these would still break under the conditions Ali described. I was only storing the this pointer so that blitted objects could deduce where they came from (trying to turn the post-blit constructor into a copy-constructor).
Ah, so you were actually counting on the postblit to have an *invalid* pointer to begin with :) Yeah, that isn't going to work. In D, it's legal to do something like memcpy struct data (with no postblit), and this is done quite often in many places because of that.
 Thanks, though. I'm thinking that maybe D just can't express these
 semantics without substantial overhead. While somewhat disappointing (I
 came into D with stars in my eyes :)), it's not enough by itself to make
 me go back to C++, at least not just yet. Not when I can just use a few
 static ifs to do what previously required careful template crafting that
 I wouldn't understand 3 months later. On the other hand, I'm falling
 behind on my library books since I no longer have any time for reading
 during compilations ;).
There are some really smart people who frequent these forums, if you post your actual use case, you may get an answer that you hadn't thought of. I saw you were trying to implement something like std::unique_ptr? There is http://dlang.org/phobos/std_typecons.html#.Unique Not sure if it helps. -Steve
Jan 29
parent reply Matt Elkins <notreal fake.com> writes:
On Saturday, 30 January 2016 at 03:00:11 UTC, Steven 
Schveighoffer wrote:
 There are some really smart people who frequent these forums, 
 if you post your actual use case, you may get an answer that 
 you hadn't thought of.
Yeah, I tried that first (on the general forum, since at the time I didn't know about this one).
 I saw you were trying to implement something like 
 std::unique_ptr? There is 
 http://dlang.org/phobos/std_typecons.html#.Unique
std.typecons.Unique seems to require heap allocation, which makes it a far cry from std::unique_ptr.
Jan 29
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/29/16 10:13 PM, Matt Elkins wrote:
 On Saturday, 30 January 2016 at 03:00:11 UTC, Steven Schveighoffer wrote:
 There are some really smart people who frequent these forums, if you
 post your actual use case, you may get an answer that you hadn't
 thought of.
Yeah, I tried that first (on the general forum, since at the time I didn't know about this one).
Ah, didn't notice that. You are likely to get more eyes there, so that is good.
 I saw you were trying to implement something like std::unique_ptr?
 There is http://dlang.org/phobos/std_typecons.html#.Unique
std.typecons.Unique seems to require heap allocation, which makes it a far cry from std::unique_ptr.
I admit I'm not familiar with either's implementation. I just know that they have similar functions. -Steve
Jan 29