www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Finalizers, Destructors, RAII, GC

reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
It is currently an issue that people might write a class for RAII 
and then allocate in on the GC heap, that means that destructors 
can be called in the wrong order.

If D adds a finalizer() method, this could be resolved the 
following way:

Objects with non-empty destructors cannot be GC allocated.

The finalizer() method should be used for releasing all non-GC 
resources accessible from non-GC pointers/identifiers, but one 
should not follow pointers to GC-memory from a finalizer as they 
may not exist.

So a RAII object needs to call the finalizer from the destructor, 
but exactly what would the implications be for inheritance?

Anyway, if it isn't clear, the basic goal is to allow RC-based 
RAII and GC to coexist.

Or would it be better to simply forbid any cleanup in the GC? I 
kinda think so, but I also think many D users are used to letting 
the GC trigger cleanup on collection.

What do you think?
Apr 28
next sibling parent reply Guillaume Piolat <first.name spam.org> writes:
On Wednesday, 28 April 2021 at 10:04:20 UTC, Ola Fosheim Grøstad 
wrote:
 Or would it be better to simply forbid any cleanup in the GC?
Yes and I think Andrei defended the idea years ago, and it was shutdown by the ensuing discussion. With GC.inFinalizer you can use the GC as a detector of accidental correctness, so the current state is not that bad. It's just more complicated than needed.
Apr 28
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 28 April 2021 at 11:13:52 UTC, Guillaume Piolat 
wrote:
 With GC.inFinalizer you can use the GC as a detector of 
 accidental correctness, so the current state is not that bad. 
 It's just more complicated than needed.
But it is also memory unsafe isn't it? If safe is to be a meaningful concept it should at least be guaranteed for naive users who don't do anything fancy and rely on the GC for all allocations.
Apr 28
prev sibling next sibling parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Wednesday, 28 April 2021 at 10:04:20 UTC, Ola Fosheim Grøstad 
wrote:
 It is currently an issue that people might write a class for 
 RAII and then allocate in on the GC heap, that means that 
 destructors can be called in the wrong order.

 If D adds a finalizer() method, this could be resolved the 
 following way:

 Objects with non-empty destructors cannot be GC allocated.

 The finalizer() method should be used for releasing all non-GC 
 resources accessible from non-GC pointers/identifiers, but one 
 should not follow pointers to GC-memory from a finalizer as 
 they may not exist.

 So a RAII object needs to call the finalizer from the 
 destructor, but exactly what would the implications be for 
 inheritance?

 Anyway, if it isn't clear, the basic goal is to allow RC-based 
 RAII and GC to coexist.

 Or would it be better to simply forbid any cleanup in the GC? I 
 kinda think so, but I also think many D users are used to 
 letting the GC trigger cleanup on collection.

 What do you think?
Split the finalizer from the destructor and make the finalizer an interface. -Alex
Apr 28
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 28 April 2021 at 14:43:24 UTC, 12345swordy wrote:
 Split the finalizer from the destructor and make the finalizer 
 an interface.
Interfaces are expensive. I think it could be in the vtable. It should also support inheritance. One should be able to inherit a superclass which has a finalizer that supports GC and then make a RAII subclass from it. There is also a need to annotate pointers which are known to not point to GC memory, maybe nogc could be reused for that. Then you would only be allowed to read nogc pointers within a finalizer.
Apr 28
parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Wednesday, 28 April 2021 at 15:17:01 UTC, Ola Fosheim Grøstad 
wrote:
 On Wednesday, 28 April 2021 at 14:43:24 UTC, 12345swordy wrote:
 Split the finalizer from the destructor and make the finalizer 
 an interface.
Interfaces are expensive. I think it could be in the vtable. It should also support inheritance. One should be able to inherit a superclass which has a finalizer that supports GC and then make a RAII subclass from it. There is also a need to annotate pointers which are known to not point to GC memory, maybe nogc could be reused for that. Then you would only be allowed to read nogc pointers within a finalizer.
How on earth are interfaces expensive? -Alex
Apr 28
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 28 April 2021 at 21:12:08 UTC, 12345swordy wrote:
 How on earth are interfaces expensive?
Look at the implementation?
Apr 28
parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Wednesday, 28 April 2021 at 21:36:54 UTC, Ola Fosheim Grøstad 
wrote:
 On Wednesday, 28 April 2021 at 21:12:08 UTC, 12345swordy wrote:
 How on earth are interfaces expensive?
Look at the implementation?
That's an issue with the implementation not design. -Alex
Apr 28
parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Wednesday, 28 April 2021 at 23:24:38 UTC, 12345swordy wrote:
 On Wednesday, 28 April 2021 at 21:36:54 UTC, Ola Fosheim 
 Grøstad wrote:
 On Wednesday, 28 April 2021 at 21:12:08 UTC, 12345swordy wrote:
 How on earth are interfaces expensive?
Look at the implementation?
That's an issue with the implementation not design. -Alex
1. What's bad about the implementation (haven't looked)? 2. Can we fix it?
Apr 29
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 29 April 2021 at 15:11:06 UTC, Imperatorn wrote:
 1. What's bad about the implementation (haven't looked)?
From the ABI:
An interface is a pointer to a pointer to a vtbl[]. The vtbl[0] 
entry is a pointer to the corresponding instance of the 
object.Interface class. The rest of the vtbl[1..$] entries are 
pointers to the virtual functions implemented by that interface, 
in the order that they were declared.
Imperatorn:
 2. Can we fix it?
It could be made part of the vtable instead, but that requires more advanced linking. It probably won't happen.
Apr 30
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 30 April 2021 at 10:44:07 UTC, Ola Fosheim Grøstad 
wrote:
 On Thursday, 29 April 2021 at 15:11:06 UTC, Imperatorn wrote:
 1. What's bad about the implementation (haven't looked)?
From the ABI:
An interface is a pointer to a pointer to a vtbl[]. The vtbl[0] 
entry is a pointer to the corresponding instance of the 
object.Interface class. The rest of the vtbl[1..$] entries are 
pointers to the virtual functions implemented by that 
interface, in the order that they were declared.
The bad part is made more explicit here: https://dlang.org/spec/abi.html#classes As you can see, every object wastes 8 bytes per interface it supports. Ok if you only have 100 instances, not good if you have one million instances (would waste 8 megabytes per interface you support).
Apr 30
parent reply Max Haughton <maxhaton gmail.com> writes:
On Friday, 30 April 2021 at 10:52:42 UTC, Ola Fosheim Grøstad 
wrote:
 On Friday, 30 April 2021 at 10:44:07 UTC, Ola Fosheim Grøstad 
 wrote:
 On Thursday, 29 April 2021 at 15:11:06 UTC, Imperatorn wrote:
 [...]
From the ABI:
[...]
The bad part is made more explicit here: https://dlang.org/spec/abi.html#classes As you can see, every object wastes 8 bytes per interface it supports. Ok if you only have 100 instances, not good if you have one million instances (would waste 8 megabytes per interface you support).
8 Megabytes!?
Apr 30
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 30 April 2021 at 12:14:42 UTC, Max Haughton wrote:
 8 Megabytes!?
8 bytes * 1000000 per interface.
Apr 30
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
On Wednesday, 28 April 2021 at 10:04:20 UTC, Ola Fosheim Grøstad 
wrote:
 Anyway, if it isn't clear, the basic goal is to allow RC-based 
 RAII and GC to coexist.
You want synchronized RC?
Apr 28
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 28 April 2021 at 16:12:37 UTC, Kagamin wrote:
 On Wednesday, 28 April 2021 at 10:04:20 UTC, Ola Fosheim 
 Grøstad wrote:
 Anyway, if it isn't clear, the basic goal is to allow RC-based 
 RAII and GC to coexist.
You want synchronized RC?
For shared, but not for nonshared?
Apr 28
parent reply Kagamin <spam here.lot> writes:
On Wednesday, 28 April 2021 at 17:05:55 UTC, Ola Fosheim Grøstad 
wrote:
 On Wednesday, 28 April 2021 at 16:12:37 UTC, Kagamin wrote:
 On Wednesday, 28 April 2021 at 10:04:20 UTC, Ola Fosheim 
 Grøstad wrote:
 Anyway, if it isn't clear, the basic goal is to allow 
 RC-based RAII and GC to coexist.
You want synchronized RC?
For shared, but not for nonshared?
GC is multithreaded, if RC is to be compatible, it should be multithreaded too, i.e. synchronized.
Apr 29
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 29 April 2021 at 15:24:14 UTC, Kagamin wrote:
 GC is multithreaded, if RC is to be compatible, it should be 
 multithreaded too, i.e. synchronized.
You cannot change anything that is traced during collection, so synchronizing won't help? Another reason to move to per-task-GC, perhaps?
Apr 29
prev sibling parent reply sighoya <sighoya gmail.com> writes:
On Wednesday, 28 April 2021 at 10:04:20 UTC, Ola Fosheim Grøstad 
wrote:
 It is currently an issue that people might write a class for 
 RAII and then allocate in on the GC heap, that means that 
 destructors can be called in the wrong order.

 If D adds a finalizer() method, this could be resolved the 
 following way:

 Objects with non-empty destructors cannot be GC allocated.

 The finalizer() method should be used for releasing all non-GC 
 resources accessible from non-GC pointers/identifiers, but one 
 should not follow pointers to GC-memory from a finalizer as 
 they may not exist.
Doesn't suffice ownership here? I mean you surely talk about owned manually allocated memory inside a class. But why the need at all for a finalizer and instead allow non-empty destructors for gc allocated classes? Having both is confusing.
 Or would it be better to simply forbid any cleanup in the GC? I 
 kinda think so, but I also think many D users are used to 
 letting the GC trigger cleanup on collection.

 What do you think?
Probably not for gc managed resources, but what about making class members reference counted or owned instead. Owned resources should be deleted automatically when a class object is destructed and reference counted resources are deleted if refcount goes to zero, which may occurs after destruction but at least not before. Btw, I don't know how much is currently possible in D.
Apr 28
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 28 April 2021 at 22:05:31 UTC, sighoya wrote:
 Doesn't suffice ownership here? I mean you surely talk about 
 owned manually allocated memory inside a class.
The compiler currently doesn't know about ownership, but you might have a non-owning pointer to a resource-manager-object that is guaranteed to outlive the program for instance.
 But why the need at all for a finalizer and instead allow 
 non-empty destructors for gc allocated classes?
 Having both is confusing.
Because destructors should be reserved for deterministic destruction, where you can presume that there are no dangling pointers. You shouldn't be allowed to allocate such object on the GC-heap. On the other hand, you should be allowed to subclass a GC-oriented finalizer based class and extend it with deterministic destruction. I guess you could make the distinction some other way. The difference is that a finalizer is running on a partially destructed object, whereas a destructor is running on a valid object. You don't need seperate functions at runtime, I guess, but the compiler has to know whether it is typechecking a finalizer or a destructor.
 Or would it be better to simply forbid any cleanup in the GC? 
 I kinda think so, but I also think many D users are used to 
 letting the GC trigger cleanup on collection.

 What do you think?
Probably not for gc managed resources, but what about making class members reference counted or owned instead.
That is the big question, is it really necessary to allow the GC to manage other resources than memory? I guess you could do it also for file descriptors, with a precise collector. Then it would scan also file descriptors and release those that no longer are referenced. But that is kinda esoteric... :-)
 Owned resources should be deleted automatically when a class 
 object is destructed and reference counted resources are 
 deleted if refcount goes to zero, which may occurs after 
 destruction but at least not before.
Yes, but this is tricky with a GC that isn't fully precise. The more objects that reference the ref-count the more likely you are to get a leak.
Apr 28