www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - To copy an auto class

reply Arcane Jill <Arcane_member pathlink.com> writes:
Auto classes would be a lot more useful if you could copy them. Now, I *do*
understand the issues here. I've written a lot of explicit memory management
stuff in C++. Basically, you need a C++-style copy-constructor.

That is, when copying an auto variable from one place to another, you must
ensure that a specially written function (copy constructor) is called in which
the resource is explicitly reference-counted. The destructor releases the
resource only when the number of explicitly counted references reaches zero.
This allows you to pass auto variables to functions, and to receive them as
return values from functions.

This sort of thing can be tedious to write - but sometimes it's the only way to
get the job done.

..but that's in C++. In D - well, you can't do it AT ALL.

So, what with auto classes not allowing copying, lazy destruction causing the
non-auto destructors not to run, and class deallocators apparently being just as
lazy as destructors, it would appear that there is NO WAY to manage a resource
that you want to be able to copy.

Short of writing my own garbage collector, that is. (And freeing module
resources at module unload time using static ~this() is an even worse solution).

I require a mechanism which will guarantee destruction of a copyable object.
There must be a way for D to provide this. I suggest either:

(1) an attribute which will cause a class's destructor to run when the gc
decides it is time to do so (that is, which requires the gc not to be lazy about
this class.
(2) a guarantee that class deallocators will always be run, or
(3) to allow an auto class to be copied if it has a copy constructor. (I'd even
be happy if the compiler required a special keyword such as
dont_blame_me_if_it_crashes, to put off the unwary).

I hope someone has the heart to do something about this.
Arcane Jill
Jun 08 2004
parent reply "Walter" <newshound digitalmars.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:ca5i7n$71o$1 digitaldaemon.com...

 Auto classes would be a lot more useful if you could copy them. Now, I


*do*

 understand the issues here. I've written a lot of explicit memory


management

 stuff in C++. Basically, you need a C++-style copy-constructor.

 That is, when copying an auto variable from one place to another, you must
 ensure that a specially written function (copy constructor) is called in


which

 the resource is explicitly reference-counted. The destructor releases the
 resource only when the number of explicitly counted references reaches


zero.

 This allows you to pass auto variables to functions, and to receive them


as

 return values from functions.

 This sort of thing can be tedious to write - but sometimes it's the only


way to

 get the job done.

 ..but that's in C++. In D - well, you can't do it AT ALL.

 So, what with auto classes not allowing copying, lazy destruction causing


the

 non-auto destructors not to run, and class deallocators apparently being


just as

 lazy as destructors, it would appear that there is NO WAY to manage a


resource

 that you want to be able to copy.

 Short of writing my own garbage collector, that is. (And freeing module
 resources at module unload time using static ~this() is an even worse


solution).

 I require a mechanism which will guarantee destruction of a copyable


object.

 There must be a way for D to provide this. I suggest either:

 (1) an attribute which will cause a class's destructor to run when the gc
 decides it is time to do so (that is, which requires the gc not to be lazy


about

 this class.
 (2) a guarantee that class deallocators will always be run, or
 (3) to allow an auto class to be copied if it has a copy constructor. (I'd


even

 be happy if the compiler required a special keyword such as
 dont_blame_me_if_it_crashes, to put off the unwary).

 I hope someone has the heart to do something about this.
 Arcane Jill


There is a misunderstanding here, as the gc *will* call the destructor when
an object is collected. What it doesn't do is guarantee that all objects
will be collected, since some random bit pattern may hold on to it.

Next, if one really does need reference counting, just use the AddRef() and
Release() protocol used with traditional COM programming. It works, it's
just not automatic.
Jun 08 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ca69nd$186h$1 digitaldaemon.com>, Walter says...


There is a misunderstanding here, as the gc *will* call the destructor when
an object is collected. What it doesn't do is guarantee that all objects
will be collected, since some random bit pattern may hold on to it.


Then the question remains. If I cannot guarantee that a non-auto class will not
be collected, the problem has not gone away.

Either I make a class auto - in which case it (currently) cannont be copied, or
I make a class non-auto, in which case it (currently) will not necessarily be
properly destroyed.

I require a class which I can copy, and for which I can absolutely guarantee
destruction. It needs to exist in an application such a server (for which module
unload may not happen for months at a time, if at all). My experiments (see
separate post) reveal that non-auto objects are quite definitely not always
destroyed, so - how do I solve this dilemma? If I have to go so far as writing
my own parallel garbage collector, that may be a step too far.




Next, if one really does need reference counting, just use the AddRef() and
Release() protocol used with traditional COM programming. It works, it's
just not automatic.


I am writing a security application Walter. It *MUST* be automatic. I will go so
far as to lock the data in memory to prevent it from being written to a swap
file. I will go so far as to request notification from the operating system
before the system hibernates for the same reason. I'm not messing about here.

I require that *EITHER* a destructor is guaranteed to be called, *OR* that a
custom deallocator is guaranteed to be called, *AND* that such objects may be
copied.

If you make auto classes copyable, then, believe it or not, that will actually
solve the problem. Right now, auto classes are useless for all but the simplest
of RAII techniques.

Jill
Jun 09 2004
parent reply "Walter" <newshound digitalmars.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:ca6tvq$26b9$1 digitaldaemon.com...

 In article <ca69nd$186h$1 digitaldaemon.com>, Walter says...

There is a misunderstanding here, as the gc *will* call the destructor




when

an object is collected. What it doesn't do is guarantee that all objects
will be collected, since some random bit pattern may hold on to it.


 Then the question remains. If I cannot guarantee that a non-auto class


will not

 be collected, the problem has not gone away.

 Either I make a class auto - in which case it (currently) cannont be


copied, or

 I make a class non-auto, in which case it (currently) will not necessarily


be

 properly destroyed.

 I require a class which I can copy, and for which I can absolutely


guarantee

 destruction. It needs to exist in an application such a server (for which


module

 unload may not happen for months at a time, if at all). My experiments


(see

 separate post) reveal that non-auto objects are quite definitely not


always

 destroyed, so - how do I solve this dilemma? If I have to go so far as


writing

 my own parallel garbage collector, that may be a step too far.


Are there any 'sync' points in your app where you know that all outstanding
security objects can be cleaned up?


Next, if one really does need reference counting, just use the AddRef()




and

Release() protocol used with traditional COM programming. It works, it's
just not automatic.


 I am writing a security application Walter. It *MUST* be automatic. I will


go so

 far as to lock the data in memory to prevent it from being written to a


swap

 file. I will go so far as to request notification from the operating


system

 before the system hibernates for the same reason. I'm not messing about


here.

I'm misunderstanding you. If you're writing it, you can completely control
the AddRef() and Release() functionality, although the compiler won't do it
for you.


 I require that *EITHER* a destructor is guaranteed to be called, *OR* that


a

 custom deallocator is guaranteed to be called, *AND* that such objects may


be

 copied.


Do you mean copy the contents of the object, or copy a reference to it?


 If you make auto classes copyable, then, believe it or not, that will


actually

 solve the problem. Right now, auto classes are useless for all but the


simplest

 of RAII techniques.
Jun 09 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ca7m1d$a47$3 digitaldaemon.com>, Walter says...


Are there any 'sync' points in your app where you know that all outstanding
security objects can be cleaned up?


You can relax now. Ben Hinkle has completely ended my misunderstanding.

But to answer your question, for any application that I write, yes, I can do
that. However, if I just supply a library and OTHER PEOPLE write apps using it,
then obviously I can't. But like I said, it doesn't matter now.



Do you mean copy the contents of the [auto] object, or copy a reference to it?


Well, it's complicated. I meant copy a reference to it. BUT - we're talking RAII
classes here, and, as you know, multiple references to the same RAII object
cause BIG problems unless you know what you're doing. However - some of us do
know what we're doing and are able to manage that complexity, or at least, we
could, if the compiler would let us. It would be necessary, when copying a RAII
reference, to call some function within the referenced class to "do the right
thing" in managing the resource. In C++, that mechanism is the following
overload:


   const T & operator=(const T &);


I /totally/ understand why you forbid this. It's the same reason that Java
forbids pointers - they're dangerous, people might get it wrong, do the wrong
thing and screw up badly. But if D is supposed to let us work under the hood,
then maybe, just maybe, you could allow the copying of auto references providing
some suitable resource management function was present in the class, and a claas
writer knew what they were doing. Just a callback which notified the class that
a reference to it was being copied would do the trick.

But if you don't want to allow this level of fine control, then, fair enough.

Jill
Jun 09 2004
parent reply "Walter" <newshound digitalmars.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:ca7nb2$chc$1 digitaldaemon.com...

 I /totally/ understand why you forbid this. It's the same reason that Java
 forbids pointers - they're dangerous, people might get it wrong, do the


wrong

 thing and screw up badly. But if D is supposed to let us work under the


hood,

 then maybe, just maybe, you could allow the copying of auto references


providing

 some suitable resource management function was present in the class, and a


claas

 writer knew what they were doing. Just a callback which notified the class


that

 a reference to it was being copied would do the trick.

 But if you don't want to allow this level of fine control, then, fair


enough.

I think I understand the problem. Let me thing about it for a while.
Sometimes, if I let things percolate, I can come up with something (I don't
think copy constructors are the right answer).
Jun 09 2004
parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
It sounds like you need reference counting, but you don't want to call 
AddRef() and RemoveRef() manually.  What you want is a reference 
counting collector. :/  How about this code?  I haven't tested it, but I 
believe that it is more or less what you want.  It implements 
refcounting, with automatic, non-lazy, guaranteed destruction, and it 
uses 'auto' objects to ensure that you don't forget to remove any 
references.

For even more security, you could declare the Handle class to be auto, 
forcing ALL instances of it to be auto.



class RefCountHandle(T) {
   RefCountMeta!(T) meta;

   this(T val) {
     this.meta = new RefCountMeta!(T);
     this.meta.val = val;
     this.meta.refCount = 1;
   }
   this(RefCountMeta!(T) meta) {
     this.meta = meta;
     meta.refCount++;
   }

   ~this() {
     meta.refCount--;
     if(meta.refCount == 0) {
       delete meta.val;
       delete meta;
     }
   }

   T val() { return meta.val; }
   RefCountMeta!(T) meta() { return meta; }
}

class RefCountMeta(T) {
   T val;
   int refCount;
}

void foo() {
   auto RefCountHandle!(MyClass) secureObj;
     secureObj = new RefCountHandle!(MyClass)(new MyClass)

   bar(secureObj.meta);
}
void bar(RefCountMeta!(MyClass) arg) {
   auto RefCountHandle!(MyClass) secureObj;
     secureObj = new RefCountHandle!(MyClass)(arg);
}

Walter wrote:

 "Arcane Jill" <Arcane_member pathlink.com> wrote in message
 news:ca7nb2$chc$1 digitaldaemon.com...
 

I /totally/ understand why you forbid this. It's the same reason that Java
forbids pointers - they're dangerous, people might get it wrong, do the


 
 wrong
 

thing and screw up badly. But if D is supposed to let us work under the


 
 hood,
 

then maybe, just maybe, you could allow the copying of auto references


 
 providing
 

some suitable resource management function was present in the class, and a


 
 claas
 

writer knew what they were doing. Just a callback which notified the class


 
 that
 

a reference to it was being copied would do the trick.

But if you don't want to allow this level of fine control, then, fair


 
 enough.
 
 I think I understand the problem. Let me thing about it for a while.
 Sometimes, if I let things percolate, I can come up with something (I don't
 think copy constructors are the right answer).
Jun 09 2004
next sibling parent Arcane Jill <Arcane_member pathlink.com> writes:
In article <ca8uhv$282p$1 digitaldaemon.com>, Russ Lewis says...
<clever stuff>

Cunning. Of course the same trick looks A LOT neater in C++ where you can
overload ->. I've done that sort of thing myself, many a time (just not in D).


    ptr<T> a;       // one of it
    ptr<T> b = a;   // two of it
    a->f();         // calls T::f() for the instance a


But D's template syntax is a little more cumbersome, and you can't overload ->.

Arcane Jill
Jun 10 2004
prev sibling parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Doesn't this just imply that 'auto' should instead be 'refcount'?  If 
the compiler implemented refcounting on specially marked variables, then 
this would be easy...
Jun 10 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <caabq3$1gpe$1 digitaldaemon.com>, Russ Lewis says...

Doesn't this just imply that 'auto' should instead be 'refcount'?


No, I'm afraid it's not going to be that simple. The reason that you can't
(currently) copy an auto class reference is because it's a very dangerous thing
to do. If you're going to do it at all (if Walter lets us), you have to know
exactly what you're doing. The compiler won't do reference counting for you. IF
that's the appropriate thing to do with a given RAII resource (and it might not
be), then you'll have to do it yourself, the hard way.



If 
the compiler implemented refcounting on specially marked variables, then 
this would be easy...


If only. Alas, no compiler can be a mind-reader. :)

For non-auto classes of course, we don't need reference counting anyway. The
normal behavior of the garbage collected does the same thing only better.

Jill
Jun 10 2004
next sibling parent Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Sorry, I guess I didn't make myself clear.  What I meant to suggest was 
that 'auto' is just a subset of 'refcount'.

If you had a 'refcount' variable that never got copied anywhere else, 
then it would work exactly like 'auto'; it would get deleted when it 
went out of scope.

On the other hand, using 'refcount' lets you copy the reference to other 
places (like passing it to a function, or even storing it someplace). 
It means that you can copy references around without the ugliness of the 
RefCountHandler class I posted above.

Russ

Arcane Jill wrote:

 In article <caabq3$1gpe$1 digitaldaemon.com>, Russ Lewis says...
 

Doesn't this just imply that 'auto' should instead be 'refcount'?


 
 
 No, I'm afraid it's not going to be that simple. The reason that you can't
 (currently) copy an auto class reference is because it's a very dangerous thing
 to do. If you're going to do it at all (if Walter lets us), you have to know
 exactly what you're doing. The compiler won't do reference counting for you. IF
 that's the appropriate thing to do with a given RAII resource (and it might not
 be), then you'll have to do it yourself, the hard way.
 
 
 

If 
the compiler implemented refcounting on specially marked variables, then 
this would be easy...


 
 
 If only. Alas, no compiler can be a mind-reader. :)
 
 For non-auto classes of course, we don't need reference counting anyway. The
 normal behavior of the garbage collected does the same thing only better.
 
 Jill
Jun 10 2004
prev sibling parent reply Roberto Mariottini <Roberto_member pathlink.com> writes:
In article <caai2r$1qk6$1 digitaldaemon.com>, Arcane Jill says...


[...]

The reason that you can't
(currently) copy an auto class reference is because it's a very dangerous thing
to do. If you're going to do it at all (if Walter lets us), you have to know
exactly what you're doing. 


I was thinking of a way to pass an auto reference to a function.
Let's suppose that exist a particular type of function parameter, the auto
parameter. An auto parameter is particular in the fact that you can't copy the
reference to it, just like normal auto references:

class MyClass 
{
..
}

int f(auto Myclass z, int n)
{
.. // here you can't take a copy of z
}

void g()
{
auto Myclass x = ...;  // auto reference
Myclass y = ...;       // normal reference
..
f(x, 2); // it's guaranteed that nobody will take a copy of x
..
f(y, 2); // this is good, don't mind if you can/cannot take a copy
..
} // here x is destroyed

The difference between a regular auto reference it's that the object it's
supposed to be already existant _before_ the function call, and it's supposed to
be disposed _after_ the function return, when it goes out of its declaration
scope.

This will permit a certain degree of auto reference copying, while keeping the
RAII-consistence.

I think that, at a theoretical level, the compiler could infer wheter a
parameter is or isn't an auto-parameter based on the following rules:
1 - if the parameter is copied, then it's non-auto
2 - if the parameter id passed as a non-auto function parameter, then it's
non-auto
3 - else: it's auto (RAII-safe)
I don't know if it can be really done in any case, nor if it's easy to do.

Ciao



 Jun 11 2004



  parent reply Arcane Jill <Arcane_member pathlink.com>  writes:


In article <cacmuo$1qr8$1 digitaldaemon.com>, Roberto Mariottini says...

I was thinking of a way to pass an auto reference to a function.


You can do that already (but don't tell Walter ;) ). 


   auto class A
   {
       uint n;
   }

   void incA(A* a)
   {
       ++a.n;
   }

   int main(char[][] args)
   {
       auto A a;
       incA(&a);
       return 0;
   }


Arcane Jill



 Jun 11 2004



  parent  "Matthew" <matthew.hat stlsoft.dot.org>  writes:


AFAIK, that's a well known technique. It's part of D's upholding of the "Spirit
of C" (http://www.comeaucomputing.com/faqs/genfaq.html#betterCgeneral),
specifically "Trust the programmer" and "Don't prevent the programmer from doing
what needs to be done".

Still, it was worth mentioning again...

"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:cad2jv$2chs$1 digitaldaemon.com...

 In article <cacmuo$1qr8$1 digitaldaemon.com>, Roberto Mariottini says...

I was thinking of a way to pass an auto reference to a function.


 You can do that already (but don't tell Walter ;) ).


   auto class A
   {
       uint n;
   }

   void incA(A* a)
   {
       ++a.n;
   }

   int main(char[][] args)
   {
       auto A a;
       incA(&a);
       return 0;
   }


 Arcane Jill





 Jun 11 2004