digitalmars.D - The GC, destructors, exceptions and memory corruption
- Vladimir Panteleev (56/56) May 12 2011 Hi,
- Alexander (3/4) May 13 2011 I would say, the only expected (and correct, IMHO) behavior should be ...
- Daniel Gibson (11/18) May 13 2011 That sucks if there's no way to handle that exception (other than by try
- Vladimir Panteleev (6/10) May 13 2011 Both threads refer to destructors in RAII, though.
- Vladimir Panteleev (6/11) May 13 2011 Yes, but what if it's handled (there's a try/catch block around the
- Daniel Gibson (9/22) May 13 2011 I don't think the exception from a destructor should be thrown to an
- Vladimir Panteleev (9/12) May 13 2011 D2 throws a FinalizeError (which is an Error, so "not recoverable").
- Daniel Gibson (3/16) May 13 2011 So the program is terminated? In that case you don't really have to
- Vladimir Panteleev (8/20) May 13 2011 I guess, but this requires that the user does not use catch(Object) or
- Vladimir Panteleev (7/9) May 13 2011 Also just catch { ... }, of course (I don't understand why that's in the...
- Vladimir Panteleev (9/11) May 13 2011 After looking again at it, my patch actually does "the right thing" and ...
- Alexander (3/4) May 13 2011 Oh, that case... Sure, if GC is buggy and throw causes GC allocation i...
- Don (4/11) May 13 2011 Are you talking about *finalizers* or *destructors* ?
- Vladimir Panteleev (8/19) May 13 2011 How would you distinguish the two in a language? Class destructors =
- Vladimir Panteleev (8/19) May 13 2011 How would you distinguish the two in the language? Class destructors =
- Michel Fortin (8/10) May 13 2011 Back to bug 4621.
- Don (4/24) May 13 2011 Yes.
- dsimcha (6/7) May 13 2011 This is why I'm so interested in improving D's GC yet have shown so litt...
- Robert Jacques (2/13) May 13 2011 I use the finalizers of proxy classes to manage GPU memory.
- Vladimir Panteleev (5/16) May 13 2011 In my case, it was mainly because D1 has no struct destructors.
- Alexander (4/5) May 13 2011 Why? If object holds some external resource, which needs to be release...
- Alexander (7/10) May 13 2011 How? Destructor is called by the GC when object is deleted. When and w...
- Don (5/15) May 13 2011 The things in classes which the spec calls "destructors" are finalizers,...
- Alexander (3/4) May 13 2011 OK. Then - when finalizer is called and throws something, program must...
- Michel Fortin (12/15) May 13 2011 Really? What if you allocated the struct on the heap, as a member of a
- Jonathan M Davis (8/22) May 13 2011 Well, assuming that a struct's destructor is called when it's garbage
- Max Samukha (5/23) May 13 2011 Destructor/finalizer confusion is one of D's weakest spots. GC and
- Michel Fortin (7/17) May 13 2011 They're already broken in subtle and racy ways because of this. A few ex...
- Sean Kelly (8/18) May 13 2011 They effectively are. When a finalized throws its wrapped in a Finalizer...
Hi, A while ago, I've tracked down the cause of an insidious memory corruption problem in one of my D programs. The problem was caused by an inadvertent allocation in a destructor (called by the GC). The current GC implementation is completely unprepared to handle such a situation - an allocation during a GC run will break the GC's invariants, and will ultimately result in memory corruption. I've filed this problem as issue 5653. I've created a simple test case which illustrates the problem: ////////////////////////////////////////////////////////////////////////////// const message = "Hello, world!"; char[] s = null; class C { ~this() { s = message.dup; } } version(D_Version2) import core.memory; else import std.gc; void main() { C c; c = new C(); c = new C(); // clobber any references to first instance version(D_Version2) GC.collect(); else fullCollect(); assert(s !is null, "Destructor wasn't called"); assert(s == message, "Memory was corrupted"); } ////////////////////////////////////////////////////////////////////////////// The exact reason the above program corrupts memory is that .dup will allocate memory by taking an item from a free list. However, after the destructor returns, the GC continues on to rebuild the free list with the information it had before the .dup allocation. The first machine word of the allocated region will be overwritten with a pointer to the next item in the free list. I wrote a patch to the D1 GC to forbid allocations from destructors, and was considering to port it to D2 and wrap it in a pull request, but realized that my patch breaks the GC in case a destructor throws. However, looking at the GC code it doesn't look like the GC is prepared to handle that situation either... while I haven't noticed any ways in which it could lead to memory corruption, if the program would catch exceptions propagated through a GC run, it could lead to persistent memory leaks (inconsistency between flags and freelists) and destructors of other objects being called several times (due to free lists not being rebuilt). Thus, my question is: what's the expected behavior of D programs when a destructor throws? -- Best regards, Vladimir mailto:vladimir thecybershadow.net
May 12 2011
On 13.05.2011 06:53, Vladimir Panteleev wrote:Thus, my question is: what's the expected behavior of D programs when a destructor throws?I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception. /Alexander
May 13 2011
Am 13.05.2011 09:57, schrieb Alexander:On 13.05.2011 06:53, Vladimir Panteleev wrote:That sucks if there's no way to handle that exception (other than by try { ... } catch {} in the destructor itself).. But probably the destructor should generally be forbidden to throw, so if it does and it crashes the program it may be fine. http://www.digitalmars.com/d/archives/digitalmars/D/What_is_nothrow_for_70451.html and http://www.digitalmars.com/d/archives/digitalmars/D/What_is_throwable_86055.html discussed throwing in destructors a little bit. Cheers, - DanielThus, my question is: what's the expected behavior of D programs when a destructor throws?I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception. /Alexander
May 13 2011
On Fri, 13 May 2011 11:11:47 +0300, Daniel Gibson <metalcaedes gmail.com> wrote:http://www.digitalmars.com/d/archives/digitalmars/D/What_is_nothrow_for_70451.html and http://www.digitalmars.com/d/archives/digitalmars/D/What_is_throwable_86055.html discussed throwing in destructors a little bit.Both threads refer to destructors in RAII, though. -- Best regards, Vladimir mailto:vladimir thecybershadow.net
May 13 2011
On Fri, 13 May 2011 10:57:37 +0300, Alexander <aldem+dmars nk7.net> wrote:On 13.05.2011 06:53, Vladimir Panteleev wrote:Yes, but what if it's handled (there's a try/catch block around the allocation or fullCollect call that invoked the GC)? -- Best regards, Vladimir mailto:vladimir thecybershadow.netThus, my question is: what's the expected behavior of D programs when a destructor throws?I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception.
May 13 2011
Am 13.05.2011 10:12, schrieb Vladimir Panteleev:On Fri, 13 May 2011 10:57:37 +0300, Alexander <aldem+dmars nk7.net> wrote:I don't think the exception from a destructor should be thrown to an allocation.. and probably not fullCollect either. For clear() or delete it /may/ make sense to get exceptions thrown in the destructor, but it'd be inconsistent if the exceptions would just vanish or terminate the program otherwise. It may be sane to just define that destructors are nothrow and if they throw anyway to terminate the program. What is the current behaviour anyway? ;)On 13.05.2011 06:53, Vladimir Panteleev wrote:Yes, but what if it's handled (there's a try/catch block around the allocation or fullCollect call that invoked the GC)?Thus, my question is: what's the expected behavior of D programs when a destructor throws?I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception.
May 13 2011
On Fri, 13 May 2011 11:22:11 +0300, Daniel Gibson <metalcaedes gmail.com> wrote:It may be sane to just define that destructors are nothrow and if they throw anyway to terminate the program. What is the current behaviour anyway? ;)D2 throws a FinalizeError (which is an Error, so "not recoverable"). D1 just allows the exception to propagate through the GC to whatever caused a GC to run. Both leave the GC in an indeterminate state, as far as I can tell. -- Best regards, Vladimir mailto:vladimir thecybershadow.net
May 13 2011
Am 13.05.2011 10:33, schrieb Vladimir Panteleev:On Fri, 13 May 2011 11:22:11 +0300, Daniel Gibson <metalcaedes gmail.com> wrote:So the program is terminated? In that case you don't really have to worry about the state of the GC, right?It may be sane to just define that destructors are nothrow and if they throw anyway to terminate the program. What is the current behaviour anyway? ;)D2 throws a FinalizeError (which is an Error, so "not recoverable").D1 just allows the exception to propagate through the GC to whatever caused a GC to run. Both leave the GC in an indeterminate state, as far as I can tell.
May 13 2011
On Fri, 13 May 2011 11:41:04 +0300, Daniel Gibson <metalcaedes gmail.com> wrote:Am 13.05.2011 10:33, schrieb Vladimir Panteleev:I guess, but this requires that the user does not use catch(Object) or catch(Throwable). I don't know if this is acceptable for SafeD, for example. -- Best regards, Vladimir mailto:vladimir thecybershadow.netOn Fri, 13 May 2011 11:22:11 +0300, Daniel Gibson <metalcaedes gmail.com> wrote:So the program is terminated? In that case you don't really have to worry about the state of the GC, right?It may be sane to just define that destructors are nothrow and if they throw anyway to terminate the program. What is the current behaviour anyway? ;)D2 throws a FinalizeError (which is an Error, so "not recoverable").
May 13 2011
On Fri, 13 May 2011 12:28:01 +0300, Vladimir Panteleev <vladimir thecybershadow.net> wrote:I guess, but this requires that the user does not use catch(Object) or catch(Throwable).Also just catch { ... }, of course (I don't understand why that's in the language at all). -- Best regards, Vladimir mailto:vladimir thecybershadow.net
May 13 2011
On Fri, 13 May 2011 11:41:04 +0300, Daniel Gibson <metalcaedes gmail.com> wrote:So the program is terminated? In that case you don't really have to worry about the state of the GC, right?After looking again at it, my patch actually does "the right thing" and locks out the GC after a finalizer exception (any attempt to access the GC from that point will instantly throw), so I've sent the pull requests. Thanks for the feedback. -- Best regards, Vladimir mailto:vladimir thecybershadow.net
May 13 2011
On 13.05.2011 10:12, Vladimir Panteleev wrote:Yes, but what if it's handled (there's a try/catch block around the allocation or fullCollect call that invoked the GC)?Oh, that case... Sure, if GC is buggy and throw causes GC allocation in turn... then we have a problem, of course, but of course, this should work - normally. /Alexander
May 13 2011
Alexander wrote:On 13.05.2011 06:53, Vladimir Panteleev wrote:Are you talking about *finalizers* or *destructors* ? Throwing from inside a destructor should definitely work (unlike C++). But finalizers should probably be nothrow.Thus, my question is: what's the expected behavior of D programs when a destructor throws?I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception. /Alexander
May 13 2011
On Fri, 13 May 2011 11:25:01 +0300, Don <nospam nospam.com> wrote:Alexander wrote:How would you distinguish the two in a language? Class destructors = finalizers? Come to think of it, SafeD shouldn't allow accessing anything on the heap in destructors as well... -- Best regards, Vladimir mailto:vladimir thecybershadow.netOn 13.05.2011 06:53, Vladimir Panteleev wrote:Are you talking about *finalizers* or *destructors* ? Throwing from inside a destructor should definitely work (unlike C++). But finalizers should probably be nothrow.Thus, my question is: what's the expected behavior of D programs when a destructor throws?I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception. /Alexander
May 13 2011
On Fri, 13 May 2011 11:25:01 +0300, Don <nospam nospam.com> wrote:Alexander wrote:How would you distinguish the two in the language? Class destructors = finalizers? Come to think of it, SafeD shouldn't allow accessing anything on the heap in finalizers as well... -- Best regards, Vladimir mailto:vladimir thecybershadow.netOn 13.05.2011 06:53, Vladimir Panteleev wrote:Are you talking about *finalizers* or *destructors* ? Throwing from inside a destructor should definitely work (unlike C++). But finalizers should probably be nothrow.Thus, my question is: what's the expected behavior of D programs when a destructor throws?I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception. /Alexander
May 13 2011
On 2011-05-13 04:34:06 -0400, "Vladimir Panteleev" <vladimir thecybershadow.net> said:Come to think of it, SafeD shouldn't allow accessing anything on the heap in finalizers as well...Back to bug 4621. <http://d.puremagic.com/issues/show_bug.cgi?id=4621> -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 13 2011
Vladimir Panteleev wrote:On Fri, 13 May 2011 11:25:01 +0300, Don <nospam nospam.com> wrote:Yes. struct destructor = destructorAlexander wrote:How would you distinguish the two in the language? Class destructors = finalizers?On 13.05.2011 06:53, Vladimir Panteleev wrote:Are you talking about *finalizers* or *destructors* ? Throwing from inside a destructor should definitely work (unlike C++). But finalizers should probably be nothrow.Thus, my question is: what's the expected behavior of D programs when a destructor throws?I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception. /AlexanderCome to think of it, SafeD shouldn't allow accessing anything on the heap in finalizers as well...Yeah. It's pretty hard to come up with a use case for a finalizer.
May 13 2011
== Quote from Don (nospam nospam.com)'s articleYeah. It's pretty hard to come up with a use case for a finalizer.This is why I'm so interested in improving D's GC yet have shown so little interest in this bug. I can't even figure out what finalizers are for and basically never use them. Even in the cases where I have used them before to get around false pointer issues, I now realize that such uses were actually latent race conditions.
May 13 2011
On Fri, 13 May 2011 10:01:56 -0400, dsimcha <dsimcha yahoo.com> wrote:== Quote from Don (nospam nospam.com)'s articleI use the finalizers of proxy classes to manage GPU memory.Yeah. It's pretty hard to come up with a use case for a finalizer.This is why I'm so interested in improving D's GC yet have shown so little interest in this bug. I can't even figure out what finalizers are for and basically never use them. Even in the cases where I have used them before to get around false pointer issues, I now realize that such uses were actually latent race conditions.
May 13 2011
On Fri, 13 May 2011 17:01:56 +0300, dsimcha <dsimcha yahoo.com> wrote:== Quote from Don (nospam nospam.com)'s articleIn my case, it was mainly because D1 has no struct destructors. -- Best regards, Vladimir mailto:vladimir thecybershadow.netYeah. It's pretty hard to come up with a use case for a finalizer.This is why I'm so interested in improving D's GC yet have shown so little interest in this bug. I can't even figure out what finalizers are for and basically never use them. Even in the cases where I have used them before to get around false pointer issues, I now realize that such uses were actually latent race conditions.
May 13 2011
On 13.05.2011 15:47, Don wrote:Yeah. It's pretty hard to come up with a use case for a finalizer.Why? If object holds some external resource, which needs to be released (like file handle) - that makes perfect use case, IMHO. It *could* happen that explicit release didn't happen, that's why finalizer should do this. /Alexander
May 13 2011
On 13.05.2011 10:25, Don wrote:Are you talking about *finalizers* or *destructors* ?Destructors as defined in D spec. There are no finalizers (yet), AFAIK.Throwing from inside a destructor should definitely work (unlike C++).How? Destructor is called by the GC when object is deleted. When and where (which thread) this happens is unknown, it is done outside of regular flow of execution - so who and where can catch this exception? So, like any other uncatched exception, it will terminate the program.But finalizers should probably be nothrow.Once they are implemented :) In any case, as long as there is no try/catch around GC collection, I see little use for exceptions apart from terminating the program. If, however, we are talking about using try/catch blocks inside of destructors - then of course, this should be possible. /Alexander
May 13 2011
Alexander wrote:On 13.05.2011 10:25, Don wrote:The things in classes which the spec calls "destructors" are finalizers, not destructors.Are you talking about *finalizers* or *destructors* ?Destructors as defined in D spec. There are no finalizers (yet), AFAIK.If the GC calls it, it's a finalizer, not a destructor. Structs have destructors, and it's perfectly OK to throw inside them.Throwing from inside a destructor should definitely work (unlike C++).How? Destructor is called by the GC when object is deleted. When and where (which thread) this happens is unknown, it is done outside of regular flow of execution - so who and where can catch this exception? So, like any other uncatched exception, it will terminate the program.
May 13 2011
On 13.05.2011 15:44, Don wrote:The things in classes which the spec calls "destructors" are finalizers, not destructors.OK. Then - when finalizer is called and throws something, program must be terminated - as there is no catch. /Alexander
May 13 2011
On 2011-05-13 09:44:33 -0400, Don <nospam nospam.com> said:If the GC calls it, it's a finalizer, not a destructor. Structs have destructors, and it's perfectly OK to throw inside them.Really? What if you allocated the struct on the heap, as a member of a class, or in an array? The struct will be on the heap and its destructor will become the finalizer. What you say about structs makes sense only as long as structs are confined to the stack. In reality, structs can also be on the heap. Discussion for bug 4621's has a long discussion about this issue. <http://d.puremagic.com/issues/show_bug.cgi?id=4621> -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 13 2011
On 2011-05-13 08:06, Michel Fortin wrote:On 2011-05-13 09:44:33 -0400, Don <nospam nospam.com> said:Well, assuming that a struct's destructor is called when it's garbage collected from the heap (which as I recall, doesn't ever happen at this point), couldn't the GC just catch any exceptions that it throws and then throw the appropriate Error? Then the destructor could throw just fine while it's on the stack, but you'd get the FinalizerError (or whatever it's called) when it happens on the heap. - Jonathan M DavisIf the GC calls it, it's a finalizer, not a destructor. Structs have destructors, and it's perfectly OK to throw inside them.Really? What if you allocated the struct on the heap, as a member of a class, or in an array? The struct will be on the heap and its destructor will become the finalizer. What you say about structs makes sense only as long as structs are confined to the stack. In reality, structs can also be on the heap. Discussion for bug 4621's has a long discussion about this issue. <http://d.puremagic.com/issues/show_bug.cgi?id=4621>
May 13 2011
On 05/13/2011 04:44 PM, Don wrote:Alexander wrote:Destructor/finalizer confusion is one of D's weakest spots. GC and "delete" (now - "clear") should have never been designed to call the same function. Now that there are talks about making GC call destructors on structs, structs are at the threat of becoming as broken as classes are.On 13.05.2011 10:25, Don wrote:The things in classes which the spec calls "destructors" are finalizers, not destructors.Are you talking about *finalizers* or *destructors* ?Destructors as defined in D spec. There are no finalizers (yet), AFAIK.If the GC calls it, it's a finalizer, not a destructor. Structs have destructors, and it's perfectly OK to throw inside them.Throwing from inside a destructor should definitely work (unlike C++).How? Destructor is called by the GC when object is deleted. When and where (which thread) this happens is unknown, it is done outside of regular flow of execution - so who and where can catch this exception? So, like any other uncatched exception, it will terminate the program.
May 13 2011
On 2011-05-13 13:16:03 -0400, Max Samukha <maxter spambox.com> said:On 05/13/2011 04:44 PM, Don wrote:They're already broken in subtle and racy ways because of this. A few examples: <http://d.puremagic.com/issues/show_bug.cgi?id=4624> -- Michel Fortin michel.fortin michelf.com http://michelf.com/If the GC calls it, it's a finalizer, not a destructor. Structs have destructors, and it's perfectly OK to throw inside them.Destructor/finalizer confusion is one of D's weakest spots. GC and "delete" (now - "clear") should have never been designed to call the same function. Now that there are talks about making GC call destructors on structs, structs are at the threat of becoming as broken as classes are.
May 13 2011
They effectively are. When a finalized throws its wrapped in a FinalizerErro= r or something of the sort. This is old code from when Walter said that it s= hould be illegal for a finalizer to throw (rule should be revised with new c= haining rules, but it works for now).=20 Sent from my iPhone On May 13, 2011, at 1:25 AM, Don <nospam nospam.com> wrote:Alexander wrote:estructor throws?On 13.05.2011 06:53, Vladimir Panteleev wrote:Thus, my question is: what's the expected behavior of D programs when a d=rmination of the program because of unhandled exception.I would say, the only expected (and correct, IMHO) behavior should be te=/Alexander=20 Are you talking about *finalizers* or *destructors* ? =20 Throwing from inside a destructor should definitely work (unlike C++). But finalizers should probably be nothrow.
May 13 2011