www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Destructor semantics

reply foo <foo bar.com> writes:
In light on recent discussions of clear() and the distructor it seems to me
that we are going backwards from one of D's great improvements over C++ - the
difference in semantics between structs and classes.

IMO, instead of enhancing class desteructors they should be completely removed
and only allowed on structs with deterministic semantics and all uses cases of
class desteructors should be replaced with structs. 
Examples:
class SocketConnection : Connection {
// struct instance allocated inline  
SocketHandle handle; 
...
}

OR:

class SocketConnection : Connection {
struct {
   this()  { acquireHandle(); }
  ~this() { releaseHandle(); }
} handle;
...
}

The suggested semantics of the above code would be that creating a 
SocketConnection object would also construct a SocketHandle as part of the
object's memory and in turn that would call the struct's ctor. 
On destruction of the object, the struct member would be also destructed and
it's d-tor is called. This is safe since the struct is part of the same memory
as the object. 

in short, struct instances should be treated just like built-in types.
Aug 10 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 10 Aug 2010 16:33:08 -0400, foo <foo bar.com> wrote:

 In light on recent discussions of clear() and the distructor it seems to  
 me that we are going backwards from one of D's great improvements over  
 C++ - the difference in semantics between structs and classes.

 IMO, instead of enhancing class desteructors they should be completely  
 removed and only allowed on structs with deterministic semantics and all  
 uses cases of class desteructors should be replaced with structs.
 Examples:
 class SocketConnection : Connection {
 // struct instance allocated inline
 SocketHandle handle;
 ...
 }

 OR:

 class SocketConnection : Connection {
 struct {
    this()  { acquireHandle(); }
   ~this() { releaseHandle(); }
 } handle;
 ...
 }

 The suggested semantics of the above code would be that creating a
 SocketConnection object would also construct a SocketHandle as part of  
 the object's memory and in turn that would call the struct's ctor.
 On destruction of the object, the struct member would be also destructed  
 and it's d-tor is called. This is safe since the struct is part of the  
 same memory as the object.

 in short, struct instances should be treated just like built-in types.

That doesn't help. deterministic destruction is not a struct-vs-class problem, its a GC-vs-manual-memory problem. A struct on the heap that is finalized by the GC has the same issues as a class destructor. In fact, struct destructors are not currently called when they are heap-allocated because the GC has no idea what is stored in those memory locations. -Steve
Aug 10 2010
parent reply foobar <foo bar.com> writes:
Steven Schveighoffer Wrote:

 On Tue, 10 Aug 2010 16:33:08 -0400, foo <foo bar.com> wrote:
 
 In light on recent discussions of clear() and the distructor it seems to  
 me that we are going backwards from one of D's great improvements over  
 C++ - the difference in semantics between structs and classes.

 IMO, instead of enhancing class desteructors they should be completely  
 removed and only allowed on structs with deterministic semantics and all  
 uses cases of class desteructors should be replaced with structs.
 Examples:
 class SocketConnection : Connection {
 // struct instance allocated inline
 SocketHandle handle;
 ...
 }

 OR:

 class SocketConnection : Connection {
 struct {
    this()  { acquireHandle(); }
   ~this() { releaseHandle(); }
 } handle;
 ...
 }

 The suggested semantics of the above code would be that creating a
 SocketConnection object would also construct a SocketHandle as part of  
 the object's memory and in turn that would call the struct's ctor.
 On destruction of the object, the struct member would be also destructed  
 and it's d-tor is called. This is safe since the struct is part of the  
 same memory as the object.

 in short, struct instances should be treated just like built-in types.

That doesn't help. deterministic destruction is not a struct-vs-class problem, its a GC-vs-manual-memory problem. A struct on the heap that is finalized by the GC has the same issues as a class destructor. In fact, struct destructors are not currently called when they are heap-allocated because the GC has no idea what is stored in those memory locations. -Steve

Let me add to the above, that the GC should NOT manage structs allocated on the heap. structs should only provide deterministic semantics.
Aug 10 2010
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 17:23:39 -0400, foobar <foo bar.com> said:

 Steven Schveighoffer Wrote:
 
 That doesn't help.  deterministic destruction is not a struct-vs-class
 problem, its a GC-vs-manual-memory problem.  A struct on the heap that is
 finalized by the GC has the same issues as a class destructor.  In fact,
 struct destructors are not currently called when they are heap-allocated
 because the GC has no idea what is stored in those memory locations.
 
 -Steve

Let me add to the above, that the GC should NOT manage structs allocated on the heap. structs should only provide deterministic semantics.

That's an interesting idea. By allowing classes only on the garbage-collected heap, and structs only in non-garbage-collected situations, we do indeed simplify the destructor problem. A struct destructor is always deterministic and a class destructor is not. Unfortunately, that's not exactly where things are headed. I'm not sure what's best, but I'm starting to believe that without this simple rule we'll have to add the complexity of having two kind of destructors in the language... would this make sense: class Test { ~this() { // disposer destructor // called by deterministic destruction // can access GC-managed members safely } ~~this() { // finalizer destructor // called during garbage collection // cannot access GC-managed members (might be dangling pointers) // unavailable in SafeD because of the dangling pointers } } -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Aug 10 2010
parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Michel Fortin wrote:

 On 2010-08-10 17:23:39 -0400, foobar <foo bar.com> said:
 
 Steven Schveighoffer Wrote:
 
 That doesn't help.  deterministic destruction is not a struct-vs-class
 problem, its a GC-vs-manual-memory problem.  A struct on the heap that is
 finalized by the GC has the same issues as a class destructor.  In fact,
 struct destructors are not currently called when they are heap-allocated
 because the GC has no idea what is stored in those memory locations.
 
 -Steve

Let me add to the above, that the GC should NOT manage structs allocated on the heap. structs should only provide deterministic semantics.

That's an interesting idea. By allowing classes only on the garbage-collected heap, and structs only in non-garbage-collected situations, we do indeed simplify the destructor problem. A struct destructor is always deterministic and a class destructor is not. Unfortunately, that's not exactly where things are headed. I'm not sure what's best, but I'm starting to believe that without this simple rule we'll have to add the complexity of having two kind of destructors in the language... would this make sense: class Test { ~this() { // disposer destructor // called by deterministic destruction // can access GC-managed members safely } ~~this() { // finalizer destructor // called during garbage collection // cannot access GC-managed members (might be dangling pointers) // unavailable in SafeD because of the dangling pointers } }

This is what I proposed, but the other way around and the semantics of ~~this implemented by an interface. If we keep ~this as the finalizer, the language need not to change so it will not impact existing code too much. Especially because delete will go away. It is not too bad, we don't need to do everything with it like .NET has to. You can always wrap a class in a struct that will destroy it, or use reference counting like File does. I think that should be preferred.
Aug 10 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 18:51:24 -0400, Lutger <lutger.blijdestijn gmail.com> said:

 This is what I proposed, but the other way around and the semantics of 
 ~~this implemented by an interface. If we keep ~this as the finalizer, 
 the language need not to change so it will not impact existing code too 
 much.

Perhaps you're right, but I wonder... Structs can be on the GC heap too, so structs also need two kinds of destructors. And I believe right now much more structs have a destructor than classes have one. So the question becomes: are most struct destructors currently intended to be run during collection cycle or they are expected to be called in a more deterministic way? I don't have the answer to that unfortunately. The only thing I can say is that if a destructor accesses the GC-heap through one of its members then it is not suitable to be run during a collection cycle. (And most classes, strings, or arrays are in GC heap.) -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Aug 10 2010
prev sibling next sibling parent reply foobar <foo bar.com> writes:
Steven Schveighoffer Wrote:

 On Tue, 10 Aug 2010 17:23:39 -0400, foobar <foo bar.com> wrote:
 
 Steven Schveighoffer Wrote:

 On Tue, 10 Aug 2010 16:33:08 -0400, foo <foo bar.com> wrote:

 In light on recent discussions of clear() and the distructor it seems  

 me that we are going backwards from one of D's great improvements over
 C++ - the difference in semantics between structs and classes.

 IMO, instead of enhancing class desteructors they should be completely
 removed and only allowed on structs with deterministic semantics and  

 uses cases of class desteructors should be replaced with structs.
 Examples:
 class SocketConnection : Connection {
 // struct instance allocated inline
 SocketHandle handle;
 ...
 }

 OR:

 class SocketConnection : Connection {
 struct {
    this()  { acquireHandle(); }
   ~this() { releaseHandle(); }
 } handle;
 ...
 }

 The suggested semantics of the above code would be that creating a
 SocketConnection object would also construct a SocketHandle as part of
 the object's memory and in turn that would call the struct's ctor.
 On destruction of the object, the struct member would be also  

 and it's d-tor is called. This is safe since the struct is part of the
 same memory as the object.

 in short, struct instances should be treated just like built-in types.

That doesn't help. deterministic destruction is not a struct-vs-class problem, its a GC-vs-manual-memory problem. A struct on the heap that is finalized by the GC has the same issues as a class destructor. In fact, struct destructors are not currently called when they are heap-allocated because the GC has no idea what is stored in those memory locations. -Steve

Let me add to the above, that the GC should NOT manage structs allocated on the heap. structs should only provide deterministic semantics.

So either you are saying that structs that are in classes are never destroyed, and you have a resource leak, or every class has an auto-generated destructor that calls the struct destructors, and we have the same determinism problem you purport to solve. If a struct is in a class, it's on the heap. You have not solved the problem. -Steve

Sorry for the confusion, my explanation wasn't good enough. Let me try again: We have 4 different cases: case 1, class contains a struct: a. the sturct is not an independent block of memory. Instead, it is part of the same memory block of the class instance's memory. b. every class has an auto-generated destructor that calls the struct destructors. This is safe because of the point a. case 2, class contains a class: this has same non-deterministic semantics as today. The containing class cannot call the dtor for the contained class. case 3, struct contains a class: struct dtor calls deterministically the auto generated dtor for the class (clean any member structs of the class) case 4, struct contains a struct c++ deterministic dtor sementics. if you allocate a struct instance on the heap you have to deallocate it (NOT GCed) and the explicit delete would trigger the deterministic dtor behavior. example: struct A { ~this(); } class B { A a; } struct C {B b; ~this(); }; auto foo = new C(); ... delete foo; // [*] when we reach [*], this is what will happen: C::~this() is called B::~this{} is called // this is ALWAYS auto generated A::~this() is called in memory we have two separate memory blocks; the first contains an instance of C (this includes a ref to a B) and the second contains an instance of B which also contains _inline_ an instance of A.
Aug 10 2010
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 18:39:03 -0400, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 It's only safe if the struct destructor does not deallocate any heap data.

To be even clearer, it's only safe if the struct destructor doesn't access any heap data through one of its members. And accessing an array or a string could potentially count as accessing heap data too (unless you can be sure it wasn't from the GC-heap). So destructors called during the collection cycle are a very tricky thing to handle. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Aug 10 2010
next sibling parent reply foobar <foo bar.com> writes:
Michel Fortin Wrote:

 On 2010-08-10 18:39:03 -0400, "Steven Schveighoffer" 
 <schveiguy yahoo.com> said:
 
 It's only safe if the struct destructor does not deallocate any heap data.

To be even clearer, it's only safe if the struct destructor doesn't access any heap data through one of its members. And accessing an array or a string could potentially count as accessing heap data too (unless you can be sure it wasn't from the GC-heap). So destructors called during the collection cycle are a very tricky thing to handle. -- Michel Fortin michel.fortin michelf.com http://michelf.com/

I agree :) In general, the rule is that destructors can only access value types. This can be enforced at compile time. Also, I was using structs and classes in my examples but this should be understood as value types vs. reference types. For example, when a dtor for a fixed-sized array is called (value type) it will in turn call dtors for its elements
Aug 10 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 19:58:08 -0400, Jonathan M Davis <jmdavisprog gmail.com> said:

 On Tuesday, August 10, 2010 16:15:33 foobar wrote:
 I agree :)
 
 In general, the rule is that destructors can only access value types. This
 can be enforced at compile time. Also, I was using structs and classes in
 my examples but this should be understood as value types vs. reference
 types.
 
 For example, when a dtor for a fixed-sized array is called (value type) it
 will in turn call dtors for its elements

If attempts to use any reference types in destructors were a compile-time error with a clear error message, that could go a long way in stopping people from trying to misuse destructors. As it is, there _are_ going to be plenty of D programmers who write destructors which access references to GC-allocated data and won't understand the weird bugs that they're getting.

Indeed. But the compiler has no way to know at compile time whether a pointer, an array, or a reference points to GC-heap or somewhere else (global data or manually allocated). That's something only the programmer may know. Perhaps this could be done for safe mode however: disallow anything that could dereference a member (assuming it might be on the GC-heap). -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Aug 10 2010
prev sibling parent Jonathan M Davis <jmdavisprog gmail.com> writes:
On Tuesday, August 10, 2010 16:15:33 foobar wrote:
 I agree :)
 
 In general, the rule is that destructors can only access value types. This
 can be enforced at compile time. Also, I was using structs and classes in
 my examples but this should be understood as value types vs. reference
 types.
 
 For example, when a dtor for a fixed-sized array is called (value type) it
 will in turn call dtors for its elements

If attempts to use any reference types in destructors were a compile-time error with a clear error message, that could go a long way in stopping people from trying to misuse destructors. As it is, there _are_ going to be plenty of D programmers who write destructors which access references to GC-allocated data and won't understand the weird bugs that they're getting. - Jonathan M Davis
Aug 10 2010
prev sibling next sibling parent reply foobar <foo bar.com> writes:
Steven Schveighoffer Wrote:

 On Tue, 10 Aug 2010 18:09:53 -0400, foobar <foo bar.com> wrote:
 
 Steven Schveighoffer Wrote:

 So either you are saying that structs that are in classes are never
 destroyed, and you have a resource leak, or every class has an
 auto-generated destructor that calls the struct destructors, and we have
 the same determinism problem you purport to solve.  If a struct is in a
 class, it's on the heap.  You have not solved the problem.

 -Steve

Sorry for the confusion, my explanation wasn't good enough. Let me try again: We have 4 different cases: case 1, class contains a struct: a. the sturct is not an independent block of memory. Instead, it is part of the same memory block of the class instance's memory. b. every class has an auto-generated destructor that calls the struct destructors. This is safe because of the point a.

It's only safe if the struct destructor does not deallocate any heap data. Here is a simple counter case Class contains struct, which in turn contains another class. The struct destructor cannot run because the class it points to may already have been cleaned up. Example: class A {} struct B { A a; ~this() {delete a;} } class C { B b; } what happens when GC destroys a C? C::~this(); // auto generated B::~this(); // so good so far A::~this(); // oops! the a is gone, program vomits bits all over itself and chokes to death. -Steve

This can only happen if you use delete on a class instance. My understanding was that this is going to be removed from D2. Without this (mis-)feature, the GC cannot remove an instance of A as long as the matching instance of C which (indirectly) contains a reference to it is still alive.
Aug 10 2010
parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 8/10/2010 16:59, foobar wrote:
 Steven Schveighoffer Wrote:
 what happens when GC destroys a C?
 
 C::~this(); // auto generated
   B::~this(); // so good so far 
     A::~this(); // oops!  the a is gone, program vomits bits all over itself
and
 chokes to death.
 
 -Steve

This can only happen if you use delete on a class instance. My understanding was that this is going to be removed from D2.

Same problem without 'delete': class A { void dispose(); } struct B { A a; ~this() { a.dispose(); } } class C { B b; } C::~this(); // auto generated B::~this(); // so good so far A::dispose(); // oops! -- Rainer Deyke - rainerd eldwood.com
Aug 10 2010
parent reply foobar <foo bar.com> writes:
Rainer Deyke Wrote:

 On 8/10/2010 16:59, foobar wrote:
 Steven Schveighoffer Wrote:
 what happens when GC destroys a C?
 
 C::~this(); // auto generated
   B::~this(); // so good so far 
     A::~this(); // oops!  the a is gone, program vomits bits all over itself
and
 chokes to death.
 
 -Steve

This can only happen if you use delete on a class instance. My understanding was that this is going to be removed from D2.

Same problem without 'delete': class A { void dispose(); } struct B { A a; ~this() { a.dispose(); } } class C { B b; } C::~this(); // auto generated B::~this(); // so good so far A::dispose(); // oops! -- Rainer Deyke - rainerd eldwood.com

I was posing late at night and hence made a mistake in my suggestion. Here's a better example of the problem: class A{} struct B{ A a; ~this(); this(ref A); } auto obj = new A(); auto first = new B(obj); auto second = new B(obj); both first and second reference the same instance of A. The correct semantics for the case of a struct containing a class (more generally, value type contains a reference type): the struct's dtor does NOT call the class dtor. The class dtor would be called either by the GC when it is collected or when it is de-allocated by the user when its memory is managed by a different memory scheme. Sorry for this confusion, it'll teach me not to post at 2AM..
Aug 11 2010
next sibling parent reply foobar <foo bar.com> writes:
Steven Schveighoffer Wrote:


 So if a struct has a class reference, it cannot clean it up?  What if the  
 class contains a struct that has an open file reference, and you want to  
 clean up that file immediately?  What if that class is private and the  
 struct knows that there are no other references to it?
 
 Your "solution" doesn't cover any new ground, we already have the issue  
 that you cannot clean up or even access heap-allocated references in  
 destructors.  The problem is, you don't know from the type system that  
 they are heap-allocated, and the compiler cannot know what is owned and  
 what is not owned, so it can't make the call to restrict you.
 
 What we need is a way to determine whether we can access those resources  
 or not in the destructor, and it has nothing to do with struct vs. class,  
 and everything to do with deterministic vs GC.  Making an artificial  
 distinction on class/struct lines doesn't help.  We have the same problem,  
 only worse restrictions (now I can't destroy a handle in a class on  
 destruction, I need to put in a struct layer around it).
 
 -Steve

I see your point above. I feel that my approach is more structured. I think that at minimum the compiler should prevent (at compile time) any access to reference types from a class dtor. I can't imagine what can be your use case for allowing such access. It seems to me that any time you feel the need for one class instance to own a different class instance you should use a struct instead to convey that owned relationship. Wouldn't you need an annotation system like the one suggested by Bartoz in order to properly implement the owned relationship for reference types?
Aug 11 2010
parent reply Jonathan M Davis <jmdavisprog gmail.com> writes:
On Wednesday, August 11, 2010 07:08:27 Steven Schveighoffer wrote:
 I agree with what Michael suggested that destructors should be restricted
 to unsafe D.

The more this whole issue gets discussed, the less it seems like I know about the issue. It's just too complicated to be immediately understandable. Personally, I think that the very fact that it's possible to access references which are no longer valid in a class destructor (and not only possible but _easily_ as well) makes it so that class destructors should be disallowed in SafeD. They just don't sound safe. If you really know what you're doing, you can use them, but other than that, forget it. That sounds precisely like why SafeD exists in the first place. Not to mention, it's looking more and more to me like if you want any kind of reasonable destruction going on, you need to be using a struct on the stack anyway. I'd suggest not only disallowing class destructors in SafeD but also disallowing structs with destructors on the heap in SafeD. It's just too messy. - Jonathan M Davis
Aug 11 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-11 13:10:08 -0400, Jonathan M Davis <jmdavisprog gmail.com> said:

 The more this whole issue gets discussed, the less it seems like I know about
 the issue. It's just too complicated to be immediately understandable.
 Personally, I think that the very fact that it's possible to access references
 which are no longer valid in a class destructor (and not only possible but
 _easily_ as well) makes it so that class destructors should be disallowed in
 SafeD. They just don't sound safe. If you really know what you're 
 doing, you can
 use them, but other than that, forget it. That sounds precisely like why SafeD
 exists in the first place.

I've made a bug report about destructors and SafeD, if anyone wants to add to it. <http://d.puremagic.com/issues/show_bug.cgi?id=4621>
 Not to mention, it's looking more and more to me like if you want any kind of
 reasonable destruction going on, you need to be using a struct on the stack
 anyway. I'd suggest not only disallowing class destructors in SafeD but also
 disallowing structs with destructors on the heap in SafeD. It's just too messy.

I'm not too sure that'll work very well. I think a better solution would be to have a way to distinguish between a struct that can be put on the GC heap and one that cannot. A struct that cannot go on the GC heap make it safe to access GC-managed members in its destructor, and thus can have a safe destructor. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Aug 11 2010
next sibling parent Joe Greer <jgreer doubletake.com> writes:
Jonathan M Davis <jmdavisprog gmail.com> wrote in
news:mailman.260.1281553632.13841.digitalmars-d puremagic.com: 

 On Wednesday, August 11, 2010 11:33:54 Michel Fortin wrote:
 I'm not too sure that'll work very well. I think a better solution
 would be to have a way to distinguish between a struct that can be
 put on the GC heap and one that cannot. A struct that cannot go on
 the GC heap make it safe to access GC-managed members in its
 destructor, and thus can have a  safe destructor.

But couldn't the fact that a struct has a destructor make it so that it can't be declared anywhere but on the heap? The destructor itself could be what distinguishes them. I don't see a need for any other attributes or whatnot to distinguish them. - Jonathan M Davis

Pardon me for sticking my nose in here. I don't use D as of yet, but I do have a bit of experience in C++ and C#. I think one of the classic ways of handling this sort of problem is to split destruction and finalization. That is, the destructor is never invoked by the GC, only explicitly or when an object on the stack goes out of scope. This means that everything referenced by the object is valid because it couldn't have been collected yet. A finalizer can be provided to be invoked by the GC for limited cleanup. Normally, this is never used, but sometimes is desirable. Obviously, in the finalizer you can't count on anything allocated by the GC to still exist, however you could still free malloced memory here if you wanted or more likely complain that the object wasn't destructed when it should have been. I just haven't seen this idea tossed about yet, so I thought I would throw it out. joe
Aug 11 2010
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-11 15:09:45 -0400, Jonathan M Davis <jmdavisprog gmail.com> said:

 On Wednesday, August 11, 2010 11:33:54 Michel Fortin wrote:
 I'm not too sure that'll work very well. I think a better solution
 would be to have a way to distinguish between a struct that can be put
 on the GC heap and one that cannot. A struct that cannot go on the GC
 heap make it safe to access GC-managed members in its destructor, and
 thus can have a  safe destructor.

But couldn't the fact that a struct has a destructor make it so that it can't be declared anywhere but on the heap? The destructor itself could be what distinguishes them. I don't see a need for any other attributes or whatnot to distinguish them.

Sure, and now you can't use std.containers.Array as a member in a Class, neither std.stdio.File, etc. Is there something left classes can do? -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Aug 11 2010
next sibling parent reply Don <nospam nospam.com> writes:
Michel Fortin wrote:
 On 2010-08-11 15:09:45 -0400, Jonathan M Davis <jmdavisprog gmail.com> 
 said:
 
 On Wednesday, August 11, 2010 11:33:54 Michel Fortin wrote:
 I'm not too sure that'll work very well. I think a better solution
 would be to have a way to distinguish between a struct that can be put
 on the GC heap and one that cannot. A struct that cannot go on the GC
 heap make it safe to access GC-managed members in its destructor, and
 thus can have a  safe destructor.

But couldn't the fact that a struct has a destructor make it so that it can't be declared anywhere but on the heap? The destructor itself could be what distinguishes them. I don't see a need for any other attributes or whatnot to distinguish them.

Sure, and now you can't use std.containers.Array as a member in a Class, neither std.stdio.File, etc. Is there something left classes can do?

As far as I can tell, the use cases for finalizers are very limited. To me, destructors don't make any sense unless they are deterministic. For example, I think having a File class is probably a bug. You may be on a system which has no limit on the number of file handles, but generally, if you need resource management, you need a guarantee that the destructor will be called. If a file handle is stored as part of a class, with a destructor that closes the file, the file might never get closed. So I don't think it's unreasonable to say that a struct with a destructor cannot be a member of a class. Personally I think destructors should be restricted to structs and scope classes. I suspect that some form of registration with the gc could do the job of finalizers. Destructor <=> deterministic destruction.
Aug 11 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Don:
 As far as I can tell, the use cases for finalizers are very limited. To 
 me, destructors don't make any sense unless they are deterministic.
 For example, I think having a File class is probably a bug. You may be 
 on a system which has no limit on the number of file handles, but 
 generally, if you need resource management, you need a guarantee that 
 the destructor will be called.

This seems similar to what I have written before: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=115028 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=115057 But if the purpose of a destructor is just to help/speedup the deallocation of a RAM resource (like to nullify the links of a tree to speed up the job of the GC) then in my opinion it is acceptable for this destructor to not run deterministically. Bye, bearophile
Aug 11 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-11 16:26:58 -0400, bearophile <bearophileHUGS lycos.com> said:

 But if the purpose of a destructor is just to help/speedup the 
 deallocation of a RAM resource (like to nullify the links of a tree to 
 speed up the job of the GC) then in my opinion it is acceptable for 
 this destructor to not run deterministically.

Indeed. So it makes sense to have both a destructor and a finalizer, because the destructor can sometime speed up things for the GC. I've made a proposal on the related bug report that reads like this: --- For instance, instead of having just destructors, we could have destructors (~this) and finalizers (~~this). A struct with neither can go anywhere, a struct with a destructor but no finalizer cannot go on the GC-heap, a struct with only a finalizer can go anywhere (the finalizer is used as the destructor), and a struct with both can go anywhere. The finalizer cannot be made safe. Doing this with structs would probably mean allowing only finalizers (~~this) for classes, which according to my syntax suggestion would break existing code for class destructors. Perhaps the syntax should be different. --- <http://d.puremagic.com/issues/show_bug.cgi?id=4621#c6> -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Aug 11 2010
prev sibling next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-11 16:14:56 -0400, Don <nospam nospam.com> said:

 Michel Fortin wrote:
 Sure, and now you can't use std.containers.Array as a member in a 
 Class, neither std.stdio.File, etc. Is there something left classes can 
 do?

As far as I can tell, the use cases for finalizers are very limited. To me, destructors don't make any sense unless they are deterministic. For example, I think having a File class is probably a bug. You may be on a system which has no limit on the number of file handles, but generally, if you need resource management, you need a guarantee that the destructor will be called. If a file handle is stored as part of a class, with a destructor that closes the file, the file might never get closed. So I don't think it's unreasonable to say that a struct with a destructor cannot be a member of a class.

Yes, this might make sense for a file handle. But not all struct destructors handle such kind of resource. The other example was std.containers.Array. Should its use be disallowed in a class?
 Personally I think destructors should be restricted to structs and 
 scope classes. I suspect that some form of registration with the gc 
 could do the job of finalizers.
 
 Destructor <=> deterministic destruction.

Having a clear distinction between destructors and finalizers certainly makes sense. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Aug 11 2010
prev sibling parent reply Joe Greer <jgreer doubletake.com> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in
news:op.vhbpkcaieav7ka localhost.localdomain: 

 
 Destructors as they are now are too limited, because they cannot be
 sure  they are being called by the GC or not, they must assume so.  So
 in one  sense I agree with you.  But in another sense, we *still* need
 a way to  clean up non-GC resources from GC-allocated items. 
 Preventing GC  allocated items from holding non-GC resources is a step
 in a very wrong  direction.
 
 I think any one of the proposals here that separates finalizers from  
 destructors should be adequate.  My backwards-compatible one is to
 pass a  parameter to the destructor indicating whether it's being
 called  deterministically or not, but that doesn't lend itself to
 preventing  safe  finalizers.  Michael also has one for using ~~this()
 and ~this().  We  could designate a method that clear uses, like
 dispose() that  deterministically disposes all resources.  I don't
 really like the  interface solution, because looking up an interface
 is expensive, plus  clear is a template so it doesn't need to use
 interfaces. 
 
 Whatever gets decided, I don't think anything should prevent them from
  being GC allocated, it's just too limiting for useful code.  The one 
 initiative the compiler could take is to prevent writing a finalizer
 in   safe code, since that can lead to memory corruption.
 
 -Steve
 

Logically speaking if an object isn't destructed, then it lives forever and if it continues to hold it's resource, then we have a programming error. The GC is for reclaiming memory, not files. It can take a long time for a GC to reclaim an object and you surely don't want a file locked for that long anymore than you want it held open forever. My point is, that it is a programming error to expect the GC be involved in reclaiming anything but memory. IMO, the best use of a finalizer is to error out if an object holding a resource hasn't been destructed, because there is obviously a programming error here and something has leaked. GCs aren't there to support sloppy programming. They are there to make your life easier and safer. joe
Aug 12 2010
next sibling parent Rainer Deyke <rainerd eldwood.com> writes:
On 8/12/2010 06:59, Joe Greer wrote:
 Logically speaking if an object isn't destructed, then it lives forever 
 and if it continues to hold it's resource, then we have a programming 
 error.  The GC is for reclaiming memory, not files.  It can take a long 
 time for a GC to reclaim an object and you surely don't want a file 
 locked for that long anymore than you want it held open forever.

Furthermore, the GC is conservative, so it isn't guaranteed to collect any particular object at all, even if manually invoked. (This also means that any D program that relies in the GC potentially leaks memory.) -- Rainer Deyke - rainerd eldwood.com
Aug 12 2010
prev sibling parent reply Joe Greer <jgreer doubletake.com> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in
news:op.vhbtwjhoeav7ka localhost.localdomain: 

 Logically speaking if an object isn't destructed, then it lives
 forever and if it continues to hold it's resource, then we have a
 programming error.  The GC is for reclaiming memory, not files.  It
 can take a long time for a GC to reclaim an object and you surely
 don't want a file locked for that long anymore than you want it held
 open forever.  My point is, that it is a programming error to expect
 the GC be involved in reclaiming anything but memory.  IMO, the best
 use of a finalizer is to error out if an object holding a resource
 hasn't been destructed, because there is obviously a programming
 error here and something has leaked. GCs aren't there to support
 sloppy programming.  They are there to make your life easier and
 safer. 

An open file maybe, but why should the compiler decide the severity of not closing the resource? What if the resource is just some C-malloc'd memory? Note, you are free to write your finalizer to do exactly what you want (print some error if it's called and the file's still open). I just don't think the compiler should be involved in the decision, because it's too ill-equipped to make one. -Steve

I think you misunderstand what I was saying. The compiler doesn't decide any of that. The programmer conserned with correctness has the option of making his finalizer complain about unfreed resources. I language shouldn't be your nanny, but it should encourage and enable you to do the right thing. There is just no substitute for knowing how to program. joe
Aug 12 2010
next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-12 15:18:33 -0400, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 On Thu, 12 Aug 2010 13:05:53 -0400, Joe Greer <jgreer doubletake.com>  wrote:
 
 I think you misunderstand what I was saying.  The compiler doesn't decide
 any of that.  The programmer conserned with correctness has the option of
 making his finalizer complain about unfreed resources.  I language
 shouldn't be your nanny, but it should encourage and enable you to do the
 right thing.  There is just no substitute for knowing how to program.

So you mark a struct something like: noheap struct File {...} That's a possible solution. I just don't like the blanket assumptions being made.

I don't like this assumption either. But short of a type system that track the owner of each memory block, all our choices will rely on the assumption that the programmer has done the finalizer right. Separating the concept of destructor and finalizer should help people realize the difference, but finalizers will still be a very tricky thing. By the way, there's currently a couple of structs in Phobos that aren't GC-friendly -- they basically have races when their destructor is called during the collection cycle (and File is one of them). I'm more and more of the opinion that the language itself should just block you from dereferencing members in the finalizer (with some sort of cast to bypass this if needed, but then you'll think twice about what you're doing). See bug 4624: <http://d.puremagic.com/issues/show_bug.cgi?id=4624> -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Aug 12 2010
prev sibling parent reply Don <nospam nospam.com> writes:
Steven Schveighoffer wrote:
 On Thu, 12 Aug 2010 13:05:53 -0400, Joe Greer <jgreer doubletake.com> 
 wrote:
 
 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in
 news:op.vhbtwjhoeav7ka localhost.localdomain:

 Logically speaking if an object isn't destructed, then it lives
 forever and if it continues to hold it's resource, then we have a
 programming error.  The GC is for reclaiming memory, not files.  It
 can take a long time for a GC to reclaim an object and you surely
 don't want a file locked for that long anymore than you want it held
 open forever.  My point is, that it is a programming error to expect
 the GC be involved in reclaiming anything but memory.  IMO, the best
 use of a finalizer is to error out if an object holding a resource
 hasn't been destructed, because there is obviously a programming
 error here and something has leaked. GCs aren't there to support
 sloppy programming.  They are there to make your life easier and
 safer.

An open file maybe, but why should the compiler decide the severity of not closing the resource? What if the resource is just some C-malloc'd memory?



That's the only example of an nearly unlimited resource which I've heard thus far, but that raises the question, why the hell would you be using malloc if you're not going to free it, when you have a language with a gc? Effectively, the gc is freeing your malloced memory. I can think of a couple of reasons for using malloc, but finalisers aren't a good solution to any of them.
 That's a possible solution.  I just don't like the blanket assumptions 
 being made.

Actually it's the absence of a use case. Hypothesis: if a finalizer is run, where it actually DOES something (as opposed to, for example, running a pile of asserts), there's always a bug. It's an extreme hypothesis, so it should be really easy to disprove. Can you come up with a counterexample?
Aug 12 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-12 16:08:17 -0400, Don <nospam nospam.com> said:

 That's the only example of an nearly unlimited resource which I've 
 heard thus far, but that raises the question, why the hell would you be 
 using malloc if you're not going to free it, when you have a language 
 with a gc? Effectively, the gc is freeing your malloced memory.
 
 I can think of a couple of reasons for using malloc, but finalisers 
 aren't a good solution to any of them.

For one thing, you could ask Andrei why he based std.containers.Array on malloc/free with a reference counter. Not being able to use Array as a member of a class would be quite drastic.
 Hypothesis: if a finalizer is run, where it actually DOES something (as 
 opposed to, for example, running a pile of asserts), there's always a 
 bug.
 
 It's an extreme hypothesis, so it should be really easy to disprove.
 Can you come up with a counterexample?

Here is a real-world example: the D/Objective-C bridge use a class destructor/finalizer to release the Objective-C counterpart of a wrapper (by calling the release method on the Objective-C object). The Objective-C object is not something I can allocate with the D GC, and the D counterpart wouldn't be very usable if it was not managed by the D GC. So finalization does the cleanup, and that works just fine. I wonder what QtD does, but it's probably something similar too. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Aug 12 2010
parent Max Samukha <spambox d-coding.com> writes:
On 08/13/2010 12:07 AM, Michel Fortin wrote:

 Here is a real-world example: the D/Objective-C bridge use a class
 destructor/finalizer to release the Objective-C counterpart of a wrapper
 (by calling the release method on the Objective-C object). The
 Objective-C object is not something I can allocate with the D GC, and
 the D counterpart wouldn't be very usable if it was not managed by the D
 GC. So finalization does the cleanup, and that works just fine.

 I wonder what QtD does, but it's probably something similar too.

It does the same. A Qt object will be destroyed when the wrapper is GCed (if the Qt object is owned by D and does not have references in C++) We tried to impose management of all QtD objects on the user but that proved to be a bad idea. The exception is QObject. In Qt QObjects are arranged in trees. When the root node is destroyed all its children are destroyed as well. GC is disabled for QObjects.
Aug 13 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisprog gmail.com> writes:
On Wednesday, August 11, 2010 13:14:56 Don wrote:
 As far as I can tell, the use cases for finalizers are very limited. To
 me, destructors don't make any sense unless they are deterministic.
 For example, I think having a File class is probably a bug. You may be
 on a system which has no limit on the number of file handles, but
 generally, if you need resource management, you need a guarantee that
 the destructor will be called.
 
 If a file handle is stored as part of a class, with a destructor that
 closes the file, the file might never get closed. So I don't think it's
 unreasonable to say that a struct with a destructor cannot be a member
 of a class.
 
 Personally I think destructors should be restricted to structs and scope
 classes. I suspect that some form of registration with the gc could do
 the job of finalizers.
 
 Destructor <=> deterministic destruction.

It would certainly make destructors more straightforward if they were guaranteed to be deterministic. And making it so that they're deterministic in some cases and not in others seems like it would make them bug-prone, since doing something with a determinstic destructor could be perfectly valid and acceptable while it could be a problem in a nondeterministic one. - Jonathan M Davis
Aug 11 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisprog gmail.com> writes:
On Wednesday, August 11, 2010 13:14:56 Don wrote:
 As far as I can tell, the use cases for finalizers are very limited. To
 me, destructors don't make any sense unless they are deterministic.
 For example, I think having a File class is probably a bug. You may be
 on a system which has no limit on the number of file handles, but
 generally, if you need resource management, you need a guarantee that
 the destructor will be called.
 
 If a file handle is stored as part of a class, with a destructor that
 closes the file, the file might never get closed. So I don't think it's
 unreasonable to say that a struct with a destructor cannot be a member
 of a class.
 
 Personally I think destructors should be restricted to structs and scope
 classes. I suspect that some form of registration with the gc could do
 the job of finalizers.
 
 Destructor <=> deterministic destruction.

It would certainly make destructors more straightforward if they were guaranteed to be deterministic. And making it so that they're deterministic in some cases and not in others seems like it would make them bug-prone, since doing something with a determinstic destructor could be perfectly valid and acceptable while it could be a problem in a nondeterministic one. - Jonathan M Davis
Aug 11 2010
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Jonathan M Davis wrote:
 This needs to be sorted out in a manner that minimizes how much pain it causes 
 for the typical programmer in their typical programming tasks. Destructors 
 should not be this much trouble.

I agree. Having too many interacting options just winds up confusing the heck out of people.
Aug 11 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-11 17:08:49 -0400, Walter Bright <newshound2 digitalmars.com> said:

 Jonathan M Davis wrote:
 This needs to be sorted out in a manner that minimizes how much pain it 
 causes for the typical programmer in their typical programming tasks. 
 Destructors should not be this much trouble.

I agree. Having too many interacting options just winds up confusing the heck out of people.

Typically, a destructor is easy to write; it's the finalizer that is tricky. Separating these two concepts would help a lot. I also filled this bug against Phobos today: <http://d.puremagic.com/issues/show_bug.cgi?id=4624> It shows how easily this kind of bug can slip in. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Aug 11 2010
prev sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-11 09:03:53 -0400, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 The problem is, you don't know from the type system that  they are 
 heap-allocated, and the compiler cannot know what is owned and  what is 
 not owned, so it can't make the call to restrict you.

Indeed, that's a real problem. And I'm beginning to be amazed by the number of issues just begging for including owner information in the type system. 1. Destructor and GC-allocated data 2. Synchronized classes members with one or more levels of indirection 3. Safe creation of immutable data through a unique reference 4. Safe moving of mutable data between threads through a unique reference Am I missing any other? In all these cases (except the first), the compiler makes the conservative assumption and when that assumption is too conservative the "solution" is always to cast. I've always found that "solution" ridiculous because, unless it's going to be very rare, adding casts everywhere is just going to make things worse. And at this point, given all the use cases that needs casts, I don't think anyone is going to say it's going to be rare. So I'm not really sure what to do about destructors. If we want it to be consistent with the rest, the compiler should take the conservative approach and disallow dereferencing members (and also calling any function that might dereference a member, ouch!). But once again, this is a fake solution to the problem: if we require casts everywhere, it'll be worse. So where do we draw the line? Well, at the very least SafeD should take the conservative route. The only "correct" solution in the end is to have owner information in the type system. But this has other issues... -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Aug 11 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 12 Aug 2010 16:08:17 -0400, Don <nospam nospam.com> wrote:

 Steven Schveighoffer wrote:
 On Thu, 12 Aug 2010 13:05:53 -0400, Joe Greer <jgreer doubletake.com>  
 wrote:

 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in
 news:op.vhbtwjhoeav7ka localhost.localdomain:

 Logically speaking if an object isn't destructed, then it lives
 forever and if it continues to hold it's resource, then we have a
 programming error.  The GC is for reclaiming memory, not files.  It
 can take a long time for a GC to reclaim an object and you surely
 don't want a file locked for that long anymore than you want it held
 open forever.  My point is, that it is a programming error to expect
 the GC be involved in reclaiming anything but memory.  IMO, the best
 use of a finalizer is to error out if an object holding a resource
 hasn't been destructed, because there is obviously a programming
 error here and something has leaked. GCs aren't there to support
 sloppy programming.  They are there to make your life easier and
 safer.

An open file maybe, but why should the compiler decide the severity of not closing the resource? What if the resource is just some C-malloc'd memory?



That's the only example of an nearly unlimited resource which I've heard thus far, but that raises the question, why the hell would you be using malloc if you're not going to free it, when you have a language with a gc? Effectively, the gc is freeing your malloced memory.

malloc/free is a lot more efficient than the GC. Partly on account of it being so mature, and partly because it doesn't have to deal with GC-like problems. Kris of Tango made an allocator for tango.container that is extremely fast based on malloc/free, but you are required to use it only for pure value types (no references). Another reason is to be able to use it in the finalizer, since malloc'd memory will be valid. If, for instance, you registered some object with a name, you need to have a malloc'd copy of the name to be able to unregister it on finalization. Finally, if the 3rd party library your using gives you malloc'd data, what do you do then?
 That's a possible solution.  I just don't like the blanket assumptions  
 being made.

Actually it's the absence of a use case. Hypothesis: if a finalizer is run, where it actually DOES something (as opposed to, for example, running a pile of asserts), there's always a bug. It's an extreme hypothesis, so it should be really easy to disprove. Can you come up with a counterexample?

Isn't it a bug to rely on finalization to tell you something is wrong with your program? Essentially, if an object is finalized, and it must close resources, you have two options: close the resources and continue silently or loudly alert the user that there is a programming bug. Considering that finalization is not guaranteed, that raises significantly the potential of bugs escaping into shipping code because the "bad" case just didn't get triggered. I'd say in most cases, the bad case isn't so bad. What if a finalizer asserted things were closed, and then closed them if they weren't. That way, you get asserts when not compiling in release mode, and in release mode, the program soldiers on as best it can without throwing seemingly random errors. It's just a thought. I don't like the idea of throwing errors in the finalizer, because the error report can possibly be completey decoupled from the source. For an example, Tango manages all I/O with classes, and I've written code that could run indefinitely opening and closing files continuously. I never ran into resource problems. If all of a sudden it started complaining that I wasn't closing the files, I'd question why someone put in that change, since it was working fine before. I agree that the GC is not the best place to rely on closing files, but it doesn't *hurt* to close resources when you realize the last reference to that resource is about to be eliminated. -Steve
Aug 13 2010
prev sibling parent Adam Ruppe <destructionator gmail.com> writes:
Perhaps relevant to this discussion is the Old New Thing posts this
week. This last one has a good sentence:

http://blogs.msdn.com/b/oldnewthing/archive/2010/08/13/10049634.aspx

"If I ruled the world, I would decree that the only thing you can do
in a finalizer is perform some tests to ensure that all the associated
external resources have already been explicitly released, and if not,
raise a fatal exception: System.Exception.Resource=ADLeak."
Aug 13 2010
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 12 Aug 2010 13:05:53 -0400, Joe Greer <jgreer doubletake.com>  
wrote:

 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in
 news:op.vhbtwjhoeav7ka localhost.localdomain:

 Logically speaking if an object isn't destructed, then it lives
 forever and if it continues to hold it's resource, then we have a
 programming error.  The GC is for reclaiming memory, not files.  It
 can take a long time for a GC to reclaim an object and you surely
 don't want a file locked for that long anymore than you want it held
 open forever.  My point is, that it is a programming error to expect
 the GC be involved in reclaiming anything but memory.  IMO, the best
 use of a finalizer is to error out if an object holding a resource
 hasn't been destructed, because there is obviously a programming
 error here and something has leaked. GCs aren't there to support
 sloppy programming.  They are there to make your life easier and
 safer.

An open file maybe, but why should the compiler decide the severity of not closing the resource? What if the resource is just some C-malloc'd memory? Note, you are free to write your finalizer to do exactly what you want (print some error if it's called and the file's still open). I just don't think the compiler should be involved in the decision, because it's too ill-equipped to make one. -Steve

I think you misunderstand what I was saying. The compiler doesn't decide any of that. The programmer conserned with correctness has the option of making his finalizer complain about unfreed resources. I language shouldn't be your nanny, but it should encourage and enable you to do the right thing. There is just no substitute for knowing how to program.

So you mark a struct something like: noheap struct File {...} That's a possible solution. I just don't like the blanket assumptions being made. -Steve
Aug 12 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 10 Aug 2010 17:23:39 -0400, foobar <foo bar.com> wrote:

 Steven Schveighoffer Wrote:

 On Tue, 10 Aug 2010 16:33:08 -0400, foo <foo bar.com> wrote:

 In light on recent discussions of clear() and the distructor it seems  

 me that we are going backwards from one of D's great improvements over
 C++ - the difference in semantics between structs and classes.

 IMO, instead of enhancing class desteructors they should be completely
 removed and only allowed on structs with deterministic semantics and  

 uses cases of class desteructors should be replaced with structs.
 Examples:
 class SocketConnection : Connection {
 // struct instance allocated inline
 SocketHandle handle;
 ...
 }

 OR:

 class SocketConnection : Connection {
 struct {
    this()  { acquireHandle(); }
   ~this() { releaseHandle(); }
 } handle;
 ...
 }

 The suggested semantics of the above code would be that creating a
 SocketConnection object would also construct a SocketHandle as part of
 the object's memory and in turn that would call the struct's ctor.
 On destruction of the object, the struct member would be also  

 and it's d-tor is called. This is safe since the struct is part of the
 same memory as the object.

 in short, struct instances should be treated just like built-in types.

That doesn't help. deterministic destruction is not a struct-vs-class problem, its a GC-vs-manual-memory problem. A struct on the heap that is finalized by the GC has the same issues as a class destructor. In fact, struct destructors are not currently called when they are heap-allocated because the GC has no idea what is stored in those memory locations. -Steve

Let me add to the above, that the GC should NOT manage structs allocated on the heap. structs should only provide deterministic semantics.

So either you are saying that structs that are in classes are never destroyed, and you have a resource leak, or every class has an auto-generated destructor that calls the struct destructors, and we have the same determinism problem you purport to solve. If a struct is in a class, it's on the heap. You have not solved the problem. -Steve
Aug 10 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 10 Aug 2010 18:09:53 -0400, foobar <foo bar.com> wrote:

 Steven Schveighoffer Wrote:

 So either you are saying that structs that are in classes are never
 destroyed, and you have a resource leak, or every class has an
 auto-generated destructor that calls the struct destructors, and we have
 the same determinism problem you purport to solve.  If a struct is in a
 class, it's on the heap.  You have not solved the problem.

 -Steve

Sorry for the confusion, my explanation wasn't good enough. Let me try again: We have 4 different cases: case 1, class contains a struct: a. the sturct is not an independent block of memory. Instead, it is part of the same memory block of the class instance's memory. b. every class has an auto-generated destructor that calls the struct destructors. This is safe because of the point a.

It's only safe if the struct destructor does not deallocate any heap data. Here is a simple counter case Class contains struct, which in turn contains another class. The struct destructor cannot run because the class it points to may already have been cleaned up. Example: class A {} struct B { A a; ~this() {delete a;} } class C { B b; } what happens when GC destroys a C? C::~this(); // auto generated B::~this(); // so good so far A::~this(); // oops! the a is gone, program vomits bits all over itself and chokes to death. -Steve
Aug 10 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 11 Aug 2010 03:12:57 -0400, foobar <foo bar.com> wrote:

 Rainer Deyke Wrote:

 On 8/10/2010 16:59, foobar wrote:
 Steven Schveighoffer Wrote:
 what happens when GC destroys a C?

 C::~this(); // auto generated
   B::~this(); // so good so far
     A::~this(); // oops!  the a is gone, program vomits bits all  


 chokes to death.

 -Steve

This can only happen if you use delete on a class instance. My understanding was that this is going to be removed from D2.

Same problem without 'delete': class A { void dispose(); } struct B { A a; ~this() { a.dispose(); } } class C { B b; } C::~this(); // auto generated B::~this(); // so good so far A::dispose(); // oops! -- Rainer Deyke - rainerd eldwood.com

I was posing late at night and hence made a mistake in my suggestion. Here's a better example of the problem: class A{} struct B{ A a; ~this(); this(ref A); } auto obj = new A(); auto first = new B(obj); auto second = new B(obj); both first and second reference the same instance of A. The correct semantics for the case of a struct containing a class (more generally, value type contains a reference type): the struct's dtor does NOT call the class dtor. The class dtor would be called either by the GC when it is collected or when it is de-allocated by the user when its memory is managed by a different memory scheme. Sorry for this confusion, it'll teach me not to post at 2AM..

So if a struct has a class reference, it cannot clean it up? What if the class contains a struct that has an open file reference, and you want to clean up that file immediately? What if that class is private and the struct knows that there are no other references to it? Your "solution" doesn't cover any new ground, we already have the issue that you cannot clean up or even access heap-allocated references in destructors. The problem is, you don't know from the type system that they are heap-allocated, and the compiler cannot know what is owned and what is not owned, so it can't make the call to restrict you. What we need is a way to determine whether we can access those resources or not in the destructor, and it has nothing to do with struct vs. class, and everything to do with deterministic vs GC. Making an artificial distinction on class/struct lines doesn't help. We have the same problem, only worse restrictions (now I can't destroy a handle in a class on destruction, I need to put in a struct layer around it). -Steve
Aug 11 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 11 Aug 2010 09:53:24 -0400, foobar <foo bar.com> wrote:

 Steven Schveighoffer Wrote:


 So if a struct has a class reference, it cannot clean it up?  What if  
 the
 class contains a struct that has an open file reference, and you want to
 clean up that file immediately?  What if that class is private and the
 struct knows that there are no other references to it?

 Your "solution" doesn't cover any new ground, we already have the issue
 that you cannot clean up or even access heap-allocated references in
 destructors.  The problem is, you don't know from the type system that
 they are heap-allocated, and the compiler cannot know what is owned and
 what is not owned, so it can't make the call to restrict you.

 What we need is a way to determine whether we can access those resources
 or not in the destructor, and it has nothing to do with struct vs.  
 class,
 and everything to do with deterministic vs GC.  Making an artificial
 distinction on class/struct lines doesn't help.  We have the same  
 problem,
 only worse restrictions (now I can't destroy a handle in a class on
 destruction, I need to put in a struct layer around it).

 -Steve

I see your point above. I feel that my approach is more structured. I think that at minimum the compiler should prevent (at compile time) any access to reference types from a class dtor. I can't imagine what can be your use case for allowing such access. It seems to me that any time you feel the need for one class instance to own a different class instance you should use a struct instead to convey that owned relationship. Wouldn't you need an annotation system like the one suggested by Bartoz in order to properly implement the owned relationship for reference types?

The point you are still not getting is that reference type != heap. The issue is heap data vs. non-heap data. Heap data is invalid (except for data contained in your current memory block), non-heap data is not. The compiler cannot tell whether a reference type is referencing heap data, so it cannot have a say in what to restrict. That is my whole point. All it can do is tell the destructor if heap references are valid or not (which it does not do now), and then it's up to the programmer who knows where the data lives (or doesn't know and assumes it could be heap data) to determine whether to access it or not. I agree with what Michael suggested that destructors should be restricted to unsafe D. -Steve
Aug 11 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisprog gmail.com> writes:
On Wednesday, August 11, 2010 11:33:54 Michel Fortin wrote:
 I'm not too sure that'll work very well. I think a better solution
 would be to have a way to distinguish between a struct that can be put
 on the GC heap and one that cannot. A struct that cannot go on the GC
 heap make it safe to access GC-managed members in its destructor, and
 thus can have a  safe destructor.

But couldn't the fact that a struct has a destructor make it so that it can't be declared anywhere but on the heap? The destructor itself could be what distinguishes them. I don't see a need for any other attributes or whatnot to distinguish them. - Jonathan M Davis
Aug 11 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisprog gmail.com> writes:
On Wednesday, August 11, 2010 12:43:10 Michel Fortin wrote:
 On 2010-08-11 15:09:45 -0400, Jonathan M Davis <jmdavisprog gmail.com> said:
 On Wednesday, August 11, 2010 11:33:54 Michel Fortin wrote:
 I'm not too sure that'll work very well. I think a better solution
 would be to have a way to distinguish between a struct that can be put
 on the GC heap and one that cannot. A struct that cannot go on the GC
 heap make it safe to access GC-managed members in its destructor, and
 thus can have a  safe destructor.

But couldn't the fact that a struct has a destructor make it so that it can't be declared anywhere but on the heap? The destructor itself could be what distinguishes them. I don't see a need for any other attributes or whatnot to distinguish them.

Sure, and now you can't use std.containers.Array as a member in a Class, neither std.stdio.File, etc. Is there something left classes can do?

This mess is just too complicated, and I'm getting more and more confused the more it gets discussed. I'll obviously need to study it closely, if I want to be able to understand it properly. Ideally, I'd be able to use structs on the heap and structs in classes with impunity. Their destructors get called when they leave scope or the class that their in gets destroyed (which may or may not be - though likely is - when it's garbage collected). Structs put directly on the heap obviously have trouble with how things are right now, though you'd think that they'd be able to have their destructors run properly as well. This needs to be sorted out in a manner that minimizes how much pain it causes for the typical programmer in their typical programming tasks. Destructors should not be this much trouble. - Jonathan M Davis
Aug 11 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 11 Aug 2010 16:14:56 -0400, Don <nospam nospam.com> wrote:

 Michel Fortin wrote:
 On 2010-08-11 15:09:45 -0400, Jonathan M Davis <jmdavisprog gmail.com>  
 said:

 On Wednesday, August 11, 2010 11:33:54 Michel Fortin wrote:
 I'm not too sure that'll work very well. I think a better solution
 would be to have a way to distinguish between a struct that can be put
 on the GC heap and one that cannot. A struct that cannot go on the GC
 heap make it safe to access GC-managed members in its destructor, and
 thus can have a  safe destructor.

But couldn't the fact that a struct has a destructor make it so that it can't be declared anywhere but on the heap? The destructor itself could be what distinguishes them. I don't see a need for any other attributes or whatnot to distinguish them.

Class, neither std.stdio.File, etc. Is there something left classes can do?

As far as I can tell, the use cases for finalizers are very limited. To me, destructors don't make any sense unless they are deterministic. For example, I think having a File class is probably a bug. You may be on a system which has no limit on the number of file handles, but generally, if you need resource management, you need a guarantee that the destructor will be called. If a file handle is stored as part of a class, with a destructor that closes the file, the file might never get closed. So I don't think it's unreasonable to say that a struct with a destructor cannot be a member of a class.

So classes are not allowed to have open files? That's too limited. Deterministic closing of the file is still possible, just not guaranteed. Here's the thing, we shouldn't be preventing smart people from writing useful code because we can't think of a way to make sure all idiot programmers close all their resources. Note also that a resource leak is not as damaging as memory corruption, which we have somewhat of an obligation to try and prevent idiots from doing.
 Personally I think destructors should be restricted to structs and scope  
 classes. I suspect that some form of registration with the gc could do  
 the job of finalizers.

 Destructor <=> deterministic destruction.

Destructors as they are now are too limited, because they cannot be sure they are being called by the GC or not, they must assume so. So in one sense I agree with you. But in another sense, we *still* need a way to clean up non-GC resources from GC-allocated items. Preventing GC allocated items from holding non-GC resources is a step in a very wrong direction. I think any one of the proposals here that separates finalizers from destructors should be adequate. My backwards-compatible one is to pass a parameter to the destructor indicating whether it's being called deterministically or not, but that doesn't lend itself to preventing safe finalizers. Michael also has one for using ~~this() and ~this(). We could designate a method that clear uses, like dispose() that deterministically disposes all resources. I don't really like the interface solution, because looking up an interface is expensive, plus clear is a template so it doesn't need to use interfaces. Whatever gets decided, I don't think anything should prevent them from being GC allocated, it's just too limiting for useful code. The one initiative the compiler could take is to prevent writing a finalizer in safe code, since that can lead to memory corruption. -Steve
Aug 12 2010
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 12 Aug 2010 08:59:31 -0400, Joe Greer <jgreer doubletake.com>  
wrote:

 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in
 news:op.vhbpkcaieav7ka localhost.localdomain:

 Destructors as they are now are too limited, because they cannot be
 sure  they are being called by the GC or not, they must assume so.  So
 in one  sense I agree with you.  But in another sense, we *still* need
 a way to  clean up non-GC resources from GC-allocated items.
 Preventing GC  allocated items from holding non-GC resources is a step
 in a very wrong  direction.

 I think any one of the proposals here that separates finalizers from
 destructors should be adequate.  My backwards-compatible one is to
 pass a  parameter to the destructor indicating whether it's being
 called  deterministically or not, but that doesn't lend itself to
 preventing  safe  finalizers.  Michael also has one for using ~~this()
 and ~this().  We  could designate a method that clear uses, like
 dispose() that  deterministically disposes all resources.  I don't
 really like the  interface solution, because looking up an interface
 is expensive, plus  clear is a template so it doesn't need to use
 interfaces.

 Whatever gets decided, I don't think anything should prevent them from
  being GC allocated, it's just too limiting for useful code.  The one
 initiative the compiler could take is to prevent writing a finalizer
 in   safe code, since that can lead to memory corruption.

 -Steve

Logically speaking if an object isn't destructed, then it lives forever and if it continues to hold it's resource, then we have a programming error. The GC is for reclaiming memory, not files. It can take a long time for a GC to reclaim an object and you surely don't want a file locked for that long anymore than you want it held open forever. My point is, that it is a programming error to expect the GC be involved in reclaiming anything but memory. IMO, the best use of a finalizer is to error out if an object holding a resource hasn't been destructed, because there is obviously a programming error here and something has leaked. GCs aren't there to support sloppy programming. They are there to make your life easier and safer.

An open file maybe, but why should the compiler decide the severity of not closing the resource? What if the resource is just some C-malloc'd memory? Note, you are free to write your finalizer to do exactly what you want (print some error if it's called and the file's still open). I just don't think the compiler should be involved in the decision, because it's too ill-equipped to make one. -Steve
Aug 12 2010