www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - To copy an auto class

reply Arcane Jill <Arcane_member pathlink.com> writes:
Auto classes would be a lot more useful if you could copy them. Now, I *do*
understand the issues here. I've written a lot of explicit memory management
stuff in C++. Basically, you need a C++-style copy-constructor.

That is, when copying an auto variable from one place to another, you must
ensure that a specially written function (copy constructor) is called in which
the resource is explicitly reference-counted. The destructor releases the
resource only when the number of explicitly counted references reaches zero.
This allows you to pass auto variables to functions, and to receive them as
return values from functions.

This sort of thing can be tedious to write - but sometimes it's the only way to
get the job done.

..but that's in C++. In D - well, you can't do it AT ALL.

So, what with auto classes not allowing copying, lazy destruction causing the
non-auto destructors not to run, and class deallocators apparently being just as
lazy as destructors, it would appear that there is NO WAY to manage a resource
that you want to be able to copy.

Short of writing my own garbage collector, that is. (And freeing module
resources at module unload time using static ~this() is an even worse solution).

I require a mechanism which will guarantee destruction of a copyable object.
There must be a way for D to provide this. I suggest either:

(1) an attribute which will cause a class's destructor to run when the gc
decides it is time to do so (that is, which requires the gc not to be lazy about
this class.
(2) a guarantee that class deallocators will always be run, or
(3) to allow an auto class to be copied if it has a copy constructor. (I'd even
be happy if the compiler required a special keyword such as
dont_blame_me_if_it_crashes, to put off the unwary).

I hope someone has the heart to do something about this.
Arcane Jill
Jun 08 2004
parent reply "Walter" <newshound digitalmars.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:ca5i7n$71o$1 digitaldaemon.com...
 Auto classes would be a lot more useful if you could copy them. Now, I

 understand the issues here. I've written a lot of explicit memory

 stuff in C++. Basically, you need a C++-style copy-constructor.

 That is, when copying an auto variable from one place to another, you must
 ensure that a specially written function (copy constructor) is called in

 the resource is explicitly reference-counted. The destructor releases the
 resource only when the number of explicitly counted references reaches

 This allows you to pass auto variables to functions, and to receive them

 return values from functions.

 This sort of thing can be tedious to write - but sometimes it's the only

 get the job done.

 ..but that's in C++. In D - well, you can't do it AT ALL.

 So, what with auto classes not allowing copying, lazy destruction causing

 non-auto destructors not to run, and class deallocators apparently being

 lazy as destructors, it would appear that there is NO WAY to manage a

 that you want to be able to copy.

 Short of writing my own garbage collector, that is. (And freeing module
 resources at module unload time using static ~this() is an even worse

 I require a mechanism which will guarantee destruction of a copyable

 There must be a way for D to provide this. I suggest either:

 (1) an attribute which will cause a class's destructor to run when the gc
 decides it is time to do so (that is, which requires the gc not to be lazy

 this class.
 (2) a guarantee that class deallocators will always be run, or
 (3) to allow an auto class to be copied if it has a copy constructor. (I'd

 be happy if the compiler required a special keyword such as
 dont_blame_me_if_it_crashes, to put off the unwary).

 I hope someone has the heart to do something about this.
 Arcane Jill

There is a misunderstanding here, as the gc *will* call the destructor when an object is collected. What it doesn't do is guarantee that all objects will be collected, since some random bit pattern may hold on to it. Next, if one really does need reference counting, just use the AddRef() and Release() protocol used with traditional COM programming. It works, it's just not automatic.
Jun 08 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ca69nd$186h$1 digitaldaemon.com>, Walter says...

There is a misunderstanding here, as the gc *will* call the destructor when
an object is collected. What it doesn't do is guarantee that all objects
will be collected, since some random bit pattern may hold on to it.

Then the question remains. If I cannot guarantee that a non-auto class will not be collected, the problem has not gone away. Either I make a class auto - in which case it (currently) cannont be copied, or I make a class non-auto, in which case it (currently) will not necessarily be properly destroyed. I require a class which I can copy, and for which I can absolutely guarantee destruction. It needs to exist in an application such a server (for which module unload may not happen for months at a time, if at all). My experiments (see separate post) reveal that non-auto objects are quite definitely not always destroyed, so - how do I solve this dilemma? If I have to go so far as writing my own parallel garbage collector, that may be a step too far.
Next, if one really does need reference counting, just use the AddRef() and
Release() protocol used with traditional COM programming. It works, it's
just not automatic.

I am writing a security application Walter. It *MUST* be automatic. I will go so far as to lock the data in memory to prevent it from being written to a swap file. I will go so far as to request notification from the operating system before the system hibernates for the same reason. I'm not messing about here. I require that *EITHER* a destructor is guaranteed to be called, *OR* that a custom deallocator is guaranteed to be called, *AND* that such objects may be copied. If you make auto classes copyable, then, believe it or not, that will actually solve the problem. Right now, auto classes are useless for all but the simplest of RAII techniques. Jill
Jun 09 2004
parent reply "Walter" <newshound digitalmars.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:ca6tvq$26b9$1 digitaldaemon.com...
 In article <ca69nd$186h$1 digitaldaemon.com>, Walter says...
There is a misunderstanding here, as the gc *will* call the destructor


an object is collected. What it doesn't do is guarantee that all objects
will be collected, since some random bit pattern may hold on to it.


 be collected, the problem has not gone away.

 Either I make a class auto - in which case it (currently) cannont be

 I make a class non-auto, in which case it (currently) will not necessarily

 properly destroyed.

 I require a class which I can copy, and for which I can absolutely

 destruction. It needs to exist in an application such a server (for which

 unload may not happen for months at a time, if at all). My experiments

 separate post) reveal that non-auto objects are quite definitely not

 destroyed, so - how do I solve this dilemma? If I have to go so far as

 my own parallel garbage collector, that may be a step too far.

Are there any 'sync' points in your app where you know that all outstanding security objects can be cleaned up?
Next, if one really does need reference counting, just use the AddRef()


Release() protocol used with traditional COM programming. It works, it's
just not automatic.

I am writing a security application Walter. It *MUST* be automatic. I will

 far as to lock the data in memory to prevent it from being written to a

 file. I will go so far as to request notification from the operating

 before the system hibernates for the same reason. I'm not messing about

I'm misunderstanding you. If you're writing it, you can completely control the AddRef() and Release() functionality, although the compiler won't do it for you.
 I require that *EITHER* a destructor is guaranteed to be called, *OR* that

 custom deallocator is guaranteed to be called, *AND* that such objects may

 copied.

Do you mean copy the contents of the object, or copy a reference to it?
 If you make auto classes copyable, then, believe it or not, that will

 solve the problem. Right now, auto classes are useless for all but the

 of RAII techniques.

Jun 09 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ca7m1d$a47$3 digitaldaemon.com>, Walter says...

Are there any 'sync' points in your app where you know that all outstanding
security objects can be cleaned up?

You can relax now. Ben Hinkle has completely ended my misunderstanding. But to answer your question, for any application that I write, yes, I can do that. However, if I just supply a library and OTHER PEOPLE write apps using it, then obviously I can't. But like I said, it doesn't matter now.
Do you mean copy the contents of the [auto] object, or copy a reference to it?

Well, it's complicated. I meant copy a reference to it. BUT - we're talking RAII classes here, and, as you know, multiple references to the same RAII object cause BIG problems unless you know what you're doing. However - some of us do know what we're doing and are able to manage that complexity, or at least, we could, if the compiler would let us. It would be necessary, when copying a RAII reference, to call some function within the referenced class to "do the right thing" in managing the resource. In C++, that mechanism is the following overload:
   const T & operator=(const T &);

I /totally/ understand why you forbid this. It's the same reason that Java forbids pointers - they're dangerous, people might get it wrong, do the wrong thing and screw up badly. But if D is supposed to let us work under the hood, then maybe, just maybe, you could allow the copying of auto references providing some suitable resource management function was present in the class, and a claas writer knew what they were doing. Just a callback which notified the class that a reference to it was being copied would do the trick. But if you don't want to allow this level of fine control, then, fair enough. Jill
Jun 09 2004
parent reply "Walter" <newshound digitalmars.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:ca7nb2$chc$1 digitaldaemon.com...
 I /totally/ understand why you forbid this. It's the same reason that Java
 forbids pointers - they're dangerous, people might get it wrong, do the

 thing and screw up badly. But if D is supposed to let us work under the

 then maybe, just maybe, you could allow the copying of auto references

 some suitable resource management function was present in the class, and a

 writer knew what they were doing. Just a callback which notified the class

 a reference to it was being copied would do the trick.

 But if you don't want to allow this level of fine control, then, fair

I think I understand the problem. Let me thing about it for a while. Sometimes, if I let things percolate, I can come up with something (I don't think copy constructors are the right answer).
Jun 09 2004
parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
It sounds like you need reference counting, but you don't want to call 
AddRef() and RemoveRef() manually.  What you want is a reference 
counting collector. :/  How about this code?  I haven't tested it, but I 
believe that it is more or less what you want.  It implements 
refcounting, with automatic, non-lazy, guaranteed destruction, and it 
uses 'auto' objects to ensure that you don't forget to remove any 
references.

For even more security, you could declare the Handle class to be auto, 
forcing ALL instances of it to be auto.



class RefCountHandle(T) {
   RefCountMeta!(T) meta;

   this(T val) {
     this.meta = new RefCountMeta!(T);
     this.meta.val = val;
     this.meta.refCount = 1;
   }
   this(RefCountMeta!(T) meta) {
     this.meta = meta;
     meta.refCount++;
   }

   ~this() {
     meta.refCount--;
     if(meta.refCount == 0) {
       delete meta.val;
       delete meta;
     }
   }

   T val() { return meta.val; }
   RefCountMeta!(T) meta() { return meta; }
}

class RefCountMeta(T) {
   T val;
   int refCount;
}

void foo() {
   auto RefCountHandle!(MyClass) secureObj;
     secureObj = new RefCountHandle!(MyClass)(new MyClass)

   bar(secureObj.meta);
}
void bar(RefCountMeta!(MyClass) arg) {
   auto RefCountHandle!(MyClass) secureObj;
     secureObj = new RefCountHandle!(MyClass)(arg);
}

Walter wrote:
 "Arcane Jill" <Arcane_member pathlink.com> wrote in message
 news:ca7nb2$chc$1 digitaldaemon.com...
 
I /totally/ understand why you forbid this. It's the same reason that Java
forbids pointers - they're dangerous, people might get it wrong, do the

wrong
thing and screw up badly. But if D is supposed to let us work under the

hood,
then maybe, just maybe, you could allow the copying of auto references

providing
some suitable resource management function was present in the class, and a

claas
writer knew what they were doing. Just a callback which notified the class

that
a reference to it was being copied would do the trick.

But if you don't want to allow this level of fine control, then, fair

enough. I think I understand the problem. Let me thing about it for a while. Sometimes, if I let things percolate, I can come up with something (I don't think copy constructors are the right answer).

Jun 09 2004
next sibling parent Arcane Jill <Arcane_member pathlink.com> writes:
In article <ca8uhv$282p$1 digitaldaemon.com>, Russ Lewis says...
<clever stuff>

Cunning. Of course the same trick looks A LOT neater in C++ where you can
overload ->. I've done that sort of thing myself, many a time (just not in D).

    ptr<T> a;       // one of it
    ptr<T> b = a;   // two of it
    a->f();         // calls T::f() for the instance a

But D's template syntax is a little more cumbersome, and you can't overload ->. Arcane Jill
Jun 10 2004
prev sibling parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Doesn't this just imply that 'auto' should instead be 'refcount'?  If 
the compiler implemented refcounting on specially marked variables, then 
this would be easy...
Jun 10 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <caabq3$1gpe$1 digitaldaemon.com>, Russ Lewis says...
Doesn't this just imply that 'auto' should instead be 'refcount'?

No, I'm afraid it's not going to be that simple. The reason that you can't (currently) copy an auto class reference is because it's a very dangerous thing to do. If you're going to do it at all (if Walter lets us), you have to know exactly what you're doing. The compiler won't do reference counting for you. IF that's the appropriate thing to do with a given RAII resource (and it might not be), then you'll have to do it yourself, the hard way.
If 
the compiler implemented refcounting on specially marked variables, then 
this would be easy...

If only. Alas, no compiler can be a mind-reader. :) For non-auto classes of course, we don't need reference counting anyway. The normal behavior of the garbage collected does the same thing only better. Jill
Jun 10 2004
next sibling parent Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Sorry, I guess I didn't make myself clear.  What I meant to suggest was 
that 'auto' is just a subset of 'refcount'.

If you had a 'refcount' variable that never got copied anywhere else, 
then it would work exactly like 'auto'; it would get deleted when it 
went out of scope.

On the other hand, using 'refcount' lets you copy the reference to other 
places (like passing it to a function, or even storing it someplace). 
It means that you can copy references around without the ugliness of the 
RefCountHandler class I posted above.

Russ

Arcane Jill wrote:
 In article <caabq3$1gpe$1 digitaldaemon.com>, Russ Lewis says...
 
Doesn't this just imply that 'auto' should instead be 'refcount'?

No, I'm afraid it's not going to be that simple. The reason that you can't (currently) copy an auto class reference is because it's a very dangerous thing to do. If you're going to do it at all (if Walter lets us), you have to know exactly what you're doing. The compiler won't do reference counting for you. IF that's the appropriate thing to do with a given RAII resource (and it might not be), then you'll have to do it yourself, the hard way.
If 
the compiler implemented refcounting on specially marked variables, then 
this would be easy...

If only. Alas, no compiler can be a mind-reader. :) For non-auto classes of course, we don't need reference counting anyway. The normal behavior of the garbage collected does the same thing only better. Jill

Jun 10 2004
prev sibling parent reply Roberto Mariottini <Roberto_member pathlink.com> writes:
In article <caai2r$1qk6$1 digitaldaemon.com>, Arcane Jill says...

The reason that you can't
(currently) copy an auto class reference is because it's a very dangerous thing
to do. If you're going to do it at all (if Walter lets us), you have to know
exactly what you're doing. 

I was thinking of a way to pass an auto reference to a function. Let's suppose that exist a particular type of function parameter, the auto parameter. An auto parameter is particular in the fact that you can't copy the reference to it, just like normal auto references: class MyClass { .. } int f(auto Myclass z, int n) { .. // here you can't take a copy of z } void g() { auto Myclass x = ...; // auto reference Myclass y = ...; // normal reference .. f(x, 2); // it's guaranteed that nobody will take a copy of x .. f(y, 2); // this is good, don't mind if you can/cannot take a copy .. } // here x is destroyed The difference between a regular auto reference it's that the object it's supposed to be already existant _before_ the function call, and it's supposed to be disposed _after_ the function return, when it goes out of its declaration scope. This will permit a certain degree of auto reference copying, while keeping the RAII-consistence. I think that, at a theoretical level, the compiler could infer wheter a parameter is or isn't an auto-parameter based on the following rules: 1 - if the parameter is copied, then it's non-auto 2 - if the parameter id passed as a non-auto function parameter, then it's non-auto 3 - else: it's auto (RAII-safe) I don't know if it can be really done in any case, nor if it's easy to do. Ciao
Jun 11 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cacmuo$1qr8$1 digitaldaemon.com>, Roberto Mariottini says...
I was thinking of a way to pass an auto reference to a function.

You can do that already (but don't tell Walter ;) ).
   auto class A
   {
       uint n;
   }

   void incA(A* a)
   {
       ++a.n;
   }

   int main(char[][] args)
   {
       auto A a;
       incA(&a);
       return 0;
   }

Arcane Jill
Jun 11 2004
parent "Matthew" <matthew.hat stlsoft.dot.org> writes:
AFAIK, that's a well known technique. It's part of D's upholding of the "Spirit
of C" (http://www.comeaucomputing.com/faqs/genfaq.html#betterCgeneral),
specifically "Trust the programmer" and "Don't prevent the programmer from doing
what needs to be done".

Still, it was worth mentioning again...

"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:cad2jv$2chs$1 digitaldaemon.com...
 In article <cacmuo$1qr8$1 digitaldaemon.com>, Roberto Mariottini says...
I was thinking of a way to pass an auto reference to a function.

You can do that already (but don't tell Walter ;) ).
   auto class A
   {
       uint n;
   }

   void incA(A* a)
   {
       ++a.n;
   }

   int main(char[][] args)
   {
       auto A a;
       incA(&a);
       return 0;
   }

Arcane Jill

Jun 11 2004