www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Movable resource handles

reply Matt Elkins <notreal fake.com> writes:
Hi all -- I am very new to D, coming from a background heavy on 
C++ as well as some other languages. Consequently, I am trying to 
get my head around D idioms and could use some help.

In a pet project of mine I have to deal with a lot (both many 
kinds and many instances) of handles to non-memory resources. 
These handles need deterministic destruction and are generally 
not copyable (at least, I don't want destructors run twice), but 
I do want to be able to move them ala C++'s move 
constructor/assignment. In C++ I could use std::unique_ptr with a 
custom deleter or roll my own variant of the same, but I am not 
sure how to do this in D. std.typecons.Unique doesn't appear to 
fit the bill, and my attempts to roll my own generic wrapper are 
producing results more akin to std::auto_ptr than std::unique_ptr 
(sorry for all the C++ references, it's the paradigm I know).

So to summarize, I want a struct (or something) to achieve these 
semantics:
* Stores a handle to a resource
* On destruction, cleans up the resource (custom deleter)
* Prohibits copying or in some other fashion prevents 
double-destruction
* Allows implicit moving (that is, allows destructive copy when 
the moved-from object is an rvalue)
* Is lightweight (in particular, does not touch the heap)

I have been able to implement a generic struct that addresses all 
of those except for allowing moving. Note that moving needs to be 
implicit -- that is, not require calling something like release() 
if it is an rvalue.

I apologize if this has been asked before; I found a related 
discussion from a few years ago 
(http://forum.dlang.org/post/jtogeq$2ndj$1 digitalmars.com), but 
it was focused on performance and didn't seem to address my 
specific concern.

Thanks in advance! And thanks for such a neat language!
Jan 27
parent reply ZombineDev <valid_email he.re> writes:
On Wednesday, 27 January 2016 at 23:20:27 UTC, Matt Elkins wrote:
 Hi all -- I am very new to D, coming from a background heavy on 
 C++ as well as some other languages. Consequently, I am trying 
 to get my head around D idioms and could use some help.

 In a pet project of mine I have to deal with a lot (both many 
 kinds and many instances) of handles to non-memory resources. 
 These handles need deterministic destruction and are generally 
 not copyable (at least, I don't want destructors run twice), 
 but I do want to be able to move them ala C++'s move 
 constructor/assignment. In C++ I could use std::unique_ptr with 
 a custom deleter or roll my own variant of the same, but I am 
 not sure how to do this in D. std.typecons.Unique doesn't 
 appear to fit the bill, and my attempts to roll my own generic 
 wrapper are producing results more akin to std::auto_ptr than 
 std::unique_ptr (sorry for all the C++ references, it's the 
 paradigm I know).

 So to summarize, I want a struct (or something) to achieve 
 these semantics:
 * Stores a handle to a resource
 * On destruction, cleans up the resource (custom deleter)
 * Prohibits copying or in some other fashion prevents 
 double-destruction
 * Allows implicit moving (that is, allows destructive copy when 
 the moved-from object is an rvalue)
 * Is lightweight (in particular, does not touch the heap)

 I have been able to implement a generic struct that addresses 
 all of those except for allowing moving. Note that moving needs 
 to be implicit -- that is, not require calling something like 
 release() if it is an rvalue.

 I apologize if this has been asked before; I found a related 
 discussion from a few years ago 
 (http://forum.dlang.org/post/jtogeq$2ndj$1 digitalmars.com), 
 but it was focused on performance and didn't seem to address my 
 specific concern.

 Thanks in advance! And thanks for such a neat language!
In my project I have something like this: ``` import std.experimental.allocator : make, dispose; import std.algorithm.mutation : move, moveEmplace; struct UniqueRef(T, alias allocator) { private T* handle; // I use a modified // version ofstd.typecons.Proxy. // Forwards most operations // except for opAssign to // *handle; mixin Proxy!handle; this(Args...)(auto ref Args args) { this.handle = allocator.make!T(args); } // Should work after Phobos PR 3956 // is merged. // disable this(); // Necessarry to ensure unique // ownership. disable this(this); // Steels the handle from // lvalues & rvalues void opAssign() (auto ref typeof(this) other) { destroy(this); this.handle = moveEmplace(other.handle); } ~this() { if (this.handle is null) return; allocator.dispose(this.handle); this.handle = null; } } // Initialation auto a = UniqueRed!int(41); // Transparent forwarding assert (a.handle !is null); assert (a == 41); assert (++a == 42); // Unique ownership // auto b = a; doesn't compile // Move semantics auto b = move(a); assert (b == 42); assert (a.handle is null); // Consumption void useHandle() (auto ref UniqueRef!int h) { UniqueRef!(int)[] arrayOfHandles = getArr(); arrayOfHandles[0] = move(h); } // Transfer ownership useHandle(b); // Accepts rvalues useHandle(UniqueRef!int(7)); void borrow(const ref UniqueRef!int h) { import std.stdio; int val = h + 0; writeln(val); } auto d = UniqueRef!int(12); borrow(d); assert (d.handle !is null); // and so on... ``` I don't know if it's 100% memory safe, but for me it works good enough. Sorry if there are typos/mistakes, I'm writting on a phone.
Jan 27
parent reply Matt Elkins <notreal fake.com> writes:
On Thursday, 28 January 2016 at 00:14:18 UTC, ZombineDev wrote:
 On Wednesday, 27 January 2016 at 23:20:27 UTC, Matt Elkins 
 wrote:
 [...]
In my project I have something like this: ``` import std.experimental.allocator : make, dispose; import std.algorithm.mutation : move, moveEmplace; struct UniqueRef(T, alias allocator) { private T* handle; // I use a modified // version ofstd.typecons.Proxy. // Forwards most operations // except for opAssign to // *handle; mixin Proxy!handle; this(Args...)(auto ref Args args) { this.handle = allocator.make!T(args); } // Should work after Phobos PR 3956 // is merged. // disable this(); // Necessarry to ensure unique // ownership. disable this(this); // Steels the handle from // lvalues & rvalues void opAssign() (auto ref typeof(this) other) { destroy(this); this.handle = moveEmplace(other.handle); } ~this() { if (this.handle is null) return; allocator.dispose(this.handle); this.handle = null; } } // Initialation auto a = UniqueRed!int(41); // Transparent forwarding assert (a.handle !is null); assert (a == 41); assert (++a == 42); // Unique ownership // auto b = a; doesn't compile // Move semantics auto b = move(a); assert (b == 42); assert (a.handle is null); // Consumption void useHandle() (auto ref UniqueRef!int h) { UniqueRef!(int)[] arrayOfHandles = getArr(); arrayOfHandles[0] = move(h); } // Transfer ownership useHandle(b); // Accepts rvalues useHandle(UniqueRef!int(7)); void borrow(const ref UniqueRef!int h) { import std.stdio; int val = h + 0; writeln(val); } auto d = UniqueRef!int(12); borrow(d); assert (d.handle !is null); // and so on... ``` I don't know if it's 100% memory safe, but for me it works good enough. Sorry if there are typos/mistakes, I'm writting on a phone.
Thanks for the response! I learned more about D just from studying this example. However, I'm not sure that this works for what I need. For example, if I want to move one of these handles into an array: ///// Not sure how to tag this as a code block on the forum ///// alias UR = UniqueRef!(int, theAllocator); auto a = UR(41); UR[] handles; // Both of the following lines yield: // Error: ... UniqueRef is not copyable because it is annotated with disable // handles ~= move(a); // handles ~= a; //// End code block //// This is essentially the same error I get on my attempts to implement these semantics, as well.
Jan 27
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 28 January 2016 at 03:02:34 UTC, Matt Elkins wrote:
 Thanks for the response! I learned more about D just from 
 studying this example.
Hmmm you might read the free online book, i'm going through it and finding it useful so far as a refresher. http://ddili.org/ders/d.en/index.html
 ///// Not sure how to tag this as a code block on the forum
Since the forum is text only, i like to use bbcode as [code] blocks (if there's a preferred method, I haven't one). So: [code] auto pi = 3.14159; [/code]
 /////
 alias UR = UniqueRef!(int, theAllocator);
 auto a = UR(41);
 UR[] handles;

 // Both of the following lines yield:
 // Error: ... UniqueRef is not copyable because it is annotated 
 with  disable
 // handles ~= move(a);
 // handles ~= a;
 //// End code block ////

 This is essentially the same error I get on my attempts to 
 implement these semantics, as well.
hmmm... You'd best be allocating the reference directly on the array, you should be able to add to it. [code] handles ~= UR(41); [/code]
Jan 27
next sibling parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 28 January 2016 at 03:38:32 UTC, Era Scarecrow wrote:
  hmmm... You'd best be allocating the reference directly on the 
 array, you should be able to add to it.
Forgot to mention. Since UniqueRef is likely just to ensure nothing else can copy away with the data, you don't HAVE to make it unique until the last step. So you could do all your calculations, then wrap it as a last step when returning it, say outside of the function. Then the assignment happens when the function returns and not where you generated it. [code] UR func(...) { int val; // calculates while it's in a safe/pure environment return UR(val); } [/code]
Jan 27
prev sibling parent reply Matt Elkins <notreal fake.com> writes:
On Thursday, 28 January 2016 at 03:38:32 UTC, Era Scarecrow wrote:
 On Thursday, 28 January 2016 at 03:02:34 UTC, Matt Elkins wrote:
 Thanks for the response! I learned more about D just from 
 studying this example.
Hmmm you might read the free online book, i'm going through it and finding it useful so far as a refresher. http://ddili.org/ders/d.en/index.html
Thanks. I've also got Andrei's book (The D Programming Language) which I read before starting this.
 ///// Not sure how to tag this as a code block on the forum
Since the forum is text only, i like to use bbcode as [code] blocks (if there's a preferred method, I haven't one). So: [code] auto pi = 3.14159; [/code]
Cool, thanks!
 /////
 alias UR = UniqueRef!(int, theAllocator);
 auto a = UR(41);
 UR[] handles;

 // Both of the following lines yield:
 // Error: ... UniqueRef is not copyable because it is 
 annotated with  disable
 // handles ~= move(a);
 // handles ~= a;
 //// End code block ////

 This is essentially the same error I get on my attempts to 
 implement these semantics, as well.
hmmm... You'd best be allocating the reference directly on the array, you should be able to add to it. [code] handles ~= UR(41); [/code]
Hm. Maybe I should show my original attempt to do this: [code] import std.algorithm; struct ResourceHandle(T, DestroyPolicy, T Default = T.init) { // Constructors/Destructor this(in T handle) nothrow {m_handle = handle;} disable this(this); ~this() {DestroyPolicy.destroy(m_handle);} // Operators disable void opAssign(ref ResourceHandle lvalue); ref ResourceHandle opAssign(ResourceHandle rvalue) {swap(m_handle, rvalue.m_handle); return this;} ref ResourceHandle opAssign(in T handle) {DestroyPolicy.destroy(m_handle); m_handle = handle; return this;} // Methods T get() const nothrow {return m_handle;} T release() nothrow {T result = m_handle; m_handle = Default; return result;} private: T m_handle = Default; } unittest { static uint lastDestroyed; struct Destroyer {static void destroy(uint resource) {lastDestroyed = resource;}} alias RH = ResourceHandle!(uint, Destroyer, 0u); assert (lastDestroyed == 0); {auto handle = RH(7);} assert (lastDestroyed == 7); { auto handle = RH(8); assert(handle.release() == 8); assert(handle.get() == uint.init); assert (lastDestroyed == 7); } assert (lastDestroyed == uint.init); { auto handle = RH(8); assert(handle.get() == 8); assert (lastDestroyed == uint.init); } assert (lastDestroyed == 8); auto handle1 = RH(1); auto handle2 = RH(2); assert(handle1.get() == 1); assert(handle2.get() == 2); assert (lastDestroyed == 8); handle2 = handle1.release(); assert (lastDestroyed == 2); assert (handle2.get() == 1); assert (handle1.get() == uint.init); RH[] handles; //handles ~= RH(3); // Fails because post-blit constructor is disabled } [/code] By way of comparison, in C++ I could do something like this (typing this directly into the post without checking it, so may have syntax errors: [code] // Compare to the last two lines of the unit test using RH = std::unique_ptr<unsigned int, Destroyer>; std::vector<RH> handles; handles.push_back(RH(3)); // Moves in the "3" resource [/code] Since D appears to lack an equivalent for C++'s move semantics, I wonder whether this is even possible to achieve? Maybe if there were some way to test for lvalue vs rvalue in the copy constructor...except that D has a post-blit constructor, not a copy constructor. I suspect I am handicapping myself by thinking in C++ terms.
Jan 27
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 28 January 2016 at 03:55:22 UTC, Matt Elkins wrote:
     //handles ~= RH(3); // Fails because post-blit constructor 
 is  disabled
It really comes down to that an array qualifies as as an Lvalue operator; But I _think_ this is a bug in the language since you should be able to assign a new Rvalue to a new array element just being created/added. We'll have to get input from Walter or Andrei.
Jan 27
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 28 January 2016 at 05:27:16 UTC, Era Scarecrow wrote:
 On Thursday, 28 January 2016 at 03:55:22 UTC, Matt Elkins wrote:
     //handles ~= RH(3); // Fails because post-blit constructor 
 is  disabled
It really comes down to that an array qualifies as an Lvalue operator; But I _think_ this is a bug in the language since you should be able to assign a new Rvalue to a new array element just being created/added. We'll have to get input from Walter or Andrei.
Alright here's a minimalistic test of the problem. Probably end up adding this to bugzilla. struct S { disable this(this); } unittest { // S s = S(); //normal assign, not important S a[]; a ~= S(); //doesn't assign, postblit disabled }
Jan 27
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 28 January 2016 at 05:39:55 UTC, Era Scarecrow wrote:
  It really comes down to that an array qualifies as an Lvalue 
 operator; But I _think_ this is a bug in the language since you 
 should be able to assign a new Rvalue to a new array element 
 just being created/added. We'll have to get input from Walter 
 or Andrei.
Alright, closest bug match i could find is https://issues.dlang.org/show_bug.cgi?id=7032 which is sorta the same thing, but not quite...
Jan 27
parent Matt Elkins <notreal fake.com> writes:
On Thursday, 28 January 2016 at 06:13:32 UTC, Era Scarecrow wrote:
 On Thursday, 28 January 2016 at 05:39:55 UTC, Era Scarecrow 
 wrote:
  It really comes down to that an array qualifies as an Lvalue 
 operator; But I _think_ this is a bug in the language since 
 you should be able to assign a new Rvalue to a new array 
 element just being created/added. We'll have to get input from 
 Walter or Andrei.
Alright, closest bug match i could find is https://issues.dlang.org/show_bug.cgi?id=7032 which is sorta the same thing, but not quite...
Ah, ok. A bug is better than a language design limitation -- it might get fixed at some point! I'll figure out a workaround for my use case. Thanks for all the time you spent, I appreciate the help.
Jan 29