digitalmars.D - Reference counting for resource management
- LMB (45/45) Nov 27 2009 Hello,
- LMB (2/5) Nov 27 2009 Ouch! I thought I was posting to the "learn" group. My bad, sorry!
- Denis Koroskin (52/111) Nov 27 2009 I attached my implementation of RefCounted pattern. Feel free to use and...
- Bartosz Milewski (67/129) Nov 29 2009 I don't think you need opAssign. The compiler will automatically use bit...
- Denis Koroskin (8/155) Nov 29 2009 I think RefCounted should be flexible enough to re-usable for other data...
- Bartosz Milewski (8/16) Nov 30 2009 I realize that, but I'm not sure if this would be a library bug or a use...
Hello,
I've been reading the forums for some time, but this is my first post here :-)
I am trying to create yet another D2 wrapper for SQLite. As usual with many C
libraries, SQLite provides us some "objects" and functions to create and
destroy them. So, I decided to encapsulate these SQLite objects in a struct,
and use reference counting to destroys the objects as soon as I can safely do
so.
The problem is that I can't make it work correctly in all situations. It guess
that I am not increasing the reference count on every situation in which my
object is copied, because I got "negative" reference counts in some tests. But
this is just a guess, I am not sure at all.
So, is there any complete example on how to implement reference counting in D2?
I found some discussions about ref counting in D, like Bartoz's nice post
(http://bartoszmilewski.wordpress.com/2009/08/19/the-anatomy-of-r
ference-counting/), but not a complete working example.
If there is no example around, I'd be pleased if someone could give me any
advice. This is what one of my wrappers roughly looks like (for brevity, I
removed all code not related to reference counting):
struct Database
{
public this(in string fileName)
{
sqlite3_open(toStringz(fileName), &db_);
refCount_ = cast(uint*)(malloc(uint.sizeof));
*refCount_ = 1;
}
this(this)
body
{
++(*refCount_);
}
Database opAssign(Database other)
{
db_ = other.db_;
refCount_ = other.refCount_;
++(*refCount_);
return this;
}
public ~this()
{
if (refCount_ !is null)
{
--(*refCount_);
if (*refCount_ == 0)
{
free(refCount_);
refCount_ = null;
immutable status = sqlite3_close(db_);
}
}
}
private sqlite3* db_;
private uint* refCount_;
}
Thanks!
LMB
Nov 27 2009
LMB <lmbarros gmail.com> LMB Wrote:Hello, [...]Ouch! I thought I was posting to the "learn" group. My bad, sorry!
Nov 27 2009
On Sat, 28 Nov 2009 01:27:41 +0300, LMB <lmbarros gmail.com> wrote:
Hello,
I've been reading the forums for some time, but this is my first post
here :-)
I am trying to create yet another D2 wrapper for SQLite. As usual with
many C libraries, SQLite provides us some "objects" and functions to
create and destroy them. So, I decided to encapsulate these SQLite
objects in a struct, and use reference counting to destroys the objects
as soon as I can safely do so.
The problem is that I can't make it work correctly in all situations. It
guess that I am not increasing the reference count on every situation in
which my object is copied, because I got "negative" reference counts in
some tests. But this is just a guess, I am not sure at all.
So, is there any complete example on how to implement reference counting
in D2?
I found some discussions about ref counting in D, like Bartoz's nice
post
(http://bartoszmilewski.wordpress.com/2009/08/19/the-anatomy-of-r
ference-counting/),
but not a complete working example.
If there is no example around, I'd be pleased if someone could give me
any advice. This is what one of my wrappers roughly looks like (for
brevity, I removed all code not related to reference counting):
struct Database
{
public this(in string fileName)
{
sqlite3_open(toStringz(fileName), &db_);
refCount_ = cast(uint*)(malloc(uint.sizeof));
*refCount_ = 1;
}
this(this)
body
{
++(*refCount_);
}
Database opAssign(Database other)
{
db_ = other.db_;
refCount_ = other.refCount_;
++(*refCount_);
return this;
}
public ~this()
{
if (refCount_ !is null)
{
--(*refCount_);
if (*refCount_ == 0)
{
free(refCount_);
refCount_ = null;
immutable status = sqlite3_close(db_);
}
}
}
private sqlite3* db_;
private uint* refCount_;
}
Thanks!
LMB
I attached my implementation of RefCounted pattern. Feel free to use and
improve it.
Here is an excerpt from a letter I wrote about RefCounted. It covers its
design, implementation and usage.
There are two templates:
RefCounted and RefObject (see implementation in
net/core/RefCounted.d). They are not classes/structs but rather
templates that are used for being mixed in other classes so that you
don't have to inherit from it. RefObject defines the following
methods:
final public uint referenceCounter() const;
final public void addRef();
final public void releaseRef();
(I'll probably also add final public void destroy(); methods that
would do nothing).
Once you mixin this template into your class, your class instances
become reference-counted (explicitly at this moment), so it could be
used without the other one (see below for details).
Mixins also have useful ability to override default implementation.
For example, destroy() would do nothing by default, and rely on
garbage-collector to recycle memory instead of calling "delete this;".
User may override it and free some resources immediately (e.g.
disconnect from remote host) before memory is recycled.
Automatic reference-counting is achieved using RefCounted template.
It's use is as follows:
struct DriverPtr
{
mixin RefCountedT!(Driver, DriverPtr);
}
struct LinkPtr
{
mixin RefCountedT!(Link, LinkPtr);
}
The reason is, again, flexibility. It is possible to override certain
methods when needed. I'll probably also add a default RefCounted
class, something like this:
struct RefCounted!(T)
{
mixin RefCountedT!(T, RefCounted!(T));
}
Users will be able to use it like this:
alias RefCounted!(Link) LinkPtr;
There are other also other approaches to implement ref-counting
mechanism, but I found this one to be the most flexible one.
The two classes are best used together, but you may use one without
other, two. RefCountedT relies on the following 3 methods:
uint referenceCounter();
void addRef();
void releaseRef();
and it doesn't really care the way they are implemented. The most
convenient way is to use mixin RefObject, though.
Nov 27 2009
I don't think you need opAssign. The compiler will automatically use bitblit
and postblit.
Here's my implementation of a reference counted thread id structure for
comparison:
struct Tid
{
this(HANDLE h)
{
_h = h;
Counter * cnt = cast(Counter *) core.stdc.stdlib.malloc(Counter.sizeof);
cnt._n = 1;
_cnt = cast(shared Counter *) cnt;
}
~this()
{
release();
}
this(this)
{
_cnt.inc;
}
// invalidates current Tid object
Tid transfer()
{
Tid tid = { _h, _cnt };
_cnt = null;
return tid;
}
void release()
{
if (_cnt !is null)
{
uint newCount = _cnt.dec;
if (newCount == 0)
{
CloseHandle(_h);
core.stdc.stdlib.free(cast(Counter *)_cnt);
_cnt = null;
}
}
}
void start()
{
assert(_h != INVALID_HANDLE_VALUE);
if (ResumeThread(_h) == -1)
throw new ThreadException("Error resuming thread");
}
void join(bool rethrow = true)
{
if (_h != INVALID_HANDLE_VALUE)
if (WaitForSingleObject(_h, INFINITE) != WAIT_OBJECT_0)
throw new ThreadException("Join failed");
}
void setPriority() // dummy for now
{
}
private:
// Revisit: implement using atomic add
struct Counter
{
uint inc() shared { return ++_n; }
uint dec() shared { return --_n; }
uint _n = 1;
}
private:
HANDLE _h = INVALID_HANDLE_VALUE;
shared Counter * _cnt;
}
LMB Wrote:
Hello,
I've been reading the forums for some time, but this is my first post here :-)
I am trying to create yet another D2 wrapper for SQLite. As usual with many C
libraries, SQLite provides us some "objects" and functions to create and
destroy them. So, I decided to encapsulate these SQLite objects in a struct,
and use reference counting to destroys the objects as soon as I can safely do
so.
The problem is that I can't make it work correctly in all situations. It guess
that I am not increasing the reference count on every situation in which my
object is copied, because I got "negative" reference counts in some tests. But
this is just a guess, I am not sure at all.
So, is there any complete example on how to implement reference counting in D2?
I found some discussions about ref counting in D, like Bartoz's nice post
(http://bartoszmilewski.wordpress.com/2009/08/19/the-anatomy-of-r
ference-counting/), but not a complete working example.
If there is no example around, I'd be pleased if someone could give me any
advice. This is what one of my wrappers roughly looks like (for brevity, I
removed all code not related to reference counting):
struct Database
{
public this(in string fileName)
{
sqlite3_open(toStringz(fileName), &db_);
refCount_ = cast(uint*)(malloc(uint.sizeof));
*refCount_ = 1;
}
this(this)
body
{
++(*refCount_);
}
Database opAssign(Database other)
{
db_ = other.db_;
refCount_ = other.refCount_;
++(*refCount_);
return this;
}
public ~this()
{
if (refCount_ !is null)
{
--(*refCount_);
if (*refCount_ == 0)
{
free(refCount_);
refCount_ = null;
immutable status = sqlite3_close(db_);
}
}
}
private sqlite3* db_;
private uint* refCount_;
}
Thanks!
LMB
Nov 29 2009
On Mon, 30 Nov 2009 03:19:33 +0300, Bartosz Milewski
<bartosz-nospam relisoft.com> wrote:
I don't think you need opAssign. The compiler will automatically use
bitblit and postblit.
Here's my implementation of a reference counted thread id structure for
comparison:
struct Tid
{
this(HANDLE h)
{
_h = h;
Counter * cnt = cast(Counter *)
core.stdc.stdlib.malloc(Counter.sizeof);
cnt._n = 1;
_cnt = cast(shared Counter *) cnt;
}
~this()
{
release();
}
this(this)
{
_cnt.inc;
}
// invalidates current Tid object
Tid transfer()
{
Tid tid = { _h, _cnt };
_cnt = null;
return tid;
}
void release()
{
if (_cnt !is null)
{
uint newCount = _cnt.dec;
if (newCount == 0)
{
CloseHandle(_h);
core.stdc.stdlib.free(cast(Counter *)_cnt);
_cnt = null;
}
}
}
void start()
{
assert(_h != INVALID_HANDLE_VALUE);
if (ResumeThread(_h) == -1)
throw new ThreadException("Error resuming thread");
}
void join(bool rethrow = true)
{
if (_h != INVALID_HANDLE_VALUE)
if (WaitForSingleObject(_h, INFINITE) != WAIT_OBJECT_0)
throw new ThreadException("Join failed");
}
void setPriority() // dummy for now
{
}
private:
// Revisit: implement using atomic add
struct Counter
{
uint inc() shared { return ++_n; }
uint dec() shared { return --_n; }
uint _n = 1;
}
private:
HANDLE _h = INVALID_HANDLE_VALUE;
shared Counter * _cnt;
}
LMB Wrote:
Hello,
I've been reading the forums for some time, but this is my first post
here :-)
I am trying to create yet another D2 wrapper for SQLite. As usual with
many C libraries, SQLite provides us some "objects" and functions to
create and destroy them. So, I decided to encapsulate these SQLite
objects in a struct, and use reference counting to destroys the objects
as soon as I can safely do so.
The problem is that I can't make it work correctly in all situations.
It guess that I am not increasing the reference count on every
situation in which my object is copied, because I got "negative"
reference counts in some tests. But this is just a guess, I am not sure
at all.
So, is there any complete example on how to implement reference
counting in D2?
I found some discussions about ref counting in D, like Bartoz's nice
post
(http://bartoszmilewski.wordpress.com/2009/08/19/the-anatomy-of-r
ference-counting/),
but not a complete working example.
If there is no example around, I'd be pleased if someone could give me
any advice. This is what one of my wrappers roughly looks like (for
brevity, I removed all code not related to reference counting):
struct Database
{
public this(in string fileName)
{
sqlite3_open(toStringz(fileName), &db_);
refCount_ = cast(uint*)(malloc(uint.sizeof));
*refCount_ = 1;
}
this(this)
body
{
++(*refCount_);
}
Database opAssign(Database other)
{
db_ = other.db_;
refCount_ = other.refCount_;
++(*refCount_);
return this;
}
public ~this()
{
if (refCount_ !is null)
{
--(*refCount_);
if (*refCount_ == 0)
{
free(refCount_);
refCount_ = null;
immutable status = sqlite3_close(db_);
}
}
}
private sqlite3* db_;
private uint* refCount_;
}
Thanks!
LMB
I think RefCounted should be flexible enough to re-usable for other data
structures, too (e.g. so that it could be part of Phobos).
By the way, your code has a bug:
Tid tid; // or release() and exisiting Tid
Tid tid2 = tid; // segfault since you don't check if _cnt is null in
postblit
Nov 29 2009
Denis Koroskin Wrote:I think RefCounted should be flexible enough to re-usable for other data structures, too (e.g. so that it could be part of Phobos).I agree. However this code goes into core, so it can't use Phobos.By the way, your code has a bug: Tid tid; // or release() and exisiting Tid Tid tid2 = tid; // segfault since you don't check if _cnt is null in postblitI realize that, but I'm not sure if this would be a library bug or a user bug. I wish one could just prevent such usage, e.g., by nulling the default constructor: struct Tid { this() = null; ... } This is a more general problem of the absence of user-defined default constructors for structs. This makes perfect sense for PODS, but not for structs that have a destructor (and postblit). Of course, if the default constructor is blocked, you wouldn't be able to have arrays of such structs, which, as a side effect, would eliminate the problem of exceptions thrown from default constructors and array unwinding. It's a bit of a minefield there.
Nov 30 2009