www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Accessing memory after destroy

reply Johan Engelen <j j.nl> writes:
I'd like to check a bit of info I need for Address Sanitizer 
checking.

The spec says [1]:
Use the destroy function to finalize an object by calling its 
destructor. The memory of the object is not immediately 
deallocated, instead the GC will collect the memory of the object 
at an undetermined point after finalization:
```
   class Foo { int x; this() { x = 1; } }
   Foo foo = new Foo;
   destroy(foo);
   assert(foo.x == int.init);  // object is still accessible
```

This tells me 2 things that I'd like to verify:
1. The destroyed memory is set to the type's `.init` value. (but 
the Ctor is not called)
2. It is _valid_ to access the memory after calling destroy.

Point 2 is worrying: what if there is a thread switch right after 
destroy, in which a GC collect happens?

Thanks,
   Johan

[1] https://dlang.org/spec/class.html#deallocators
Jul 29
next sibling parent Eugene Wissner <belka caraus.de> writes:
On Saturday, 29 July 2017 at 20:44:30 UTC, Johan Engelen wrote:
 I'd like to check a bit of info I need for Address Sanitizer 
 checking.

 The spec says [1]:
 Use the destroy function to finalize an object by calling its 
 destructor. The memory of the object is not immediately 
 deallocated, instead the GC will collect the memory of the 
 object at an undetermined point after finalization:
 ```
   class Foo { int x; this() { x = 1; } }
   Foo foo = new Foo;
   destroy(foo);
   assert(foo.x == int.init);  // object is still accessible
 ```

 This tells me 2 things that I'd like to verify:
 1. The destroyed memory is set to the type's `.init` value. 
 (but the Ctor is not called)
 2. It is _valid_ to access the memory after calling destroy.

 Point 2 is worrying: what if there is a thread switch right 
 after destroy, in which a GC collect happens?

 Thanks,
   Johan

 [1] https://dlang.org/spec/class.html#deallocators
For 1) look at rt_finalize2 in rt/lifetime.d*. It is the function called by destroy. It sets the class to its init value: auto w = (*pc).initializer; p[0 .. w.length] = w[]; So it will memcpy Foo.initializer into foo. For 2) I think you're right, it seems to be unsafe to access the object after destroying. * https://github.com/dlang/druntime/blob/3485ff859a29ba44e7949bc49e62d5dd3a2a9ff0/src/rt/lifetime.d#L1402
Jul 29
prev sibling next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Saturday, 29 July 2017 at 20:44:30 UTC, Johan Engelen wrote:
 [...]
 ```
   class Foo { int x; this() { x = 1; } }
   Foo foo = new Foo;
   destroy(foo);
   assert(foo.x == int.init);  // object is still accessible
 ```
 [...]
 2. It is _valid_ to access the memory after calling destroy.

 Point 2 is worrying: what if there is a thread switch right 
 after destroy, in which a GC collect happens?
D's GC uses (conservative) stop-the-world mark-and-sweep, i.e. as long as a memory chunk is reachable via a root it won't be collected (see [1]). Since in your example the (stack) variable `foo` is still referring to the (heap) memory location of the destroyed object, if a garbage collection cycle is started between `destroy(foo);` and `assert(foo.x == int.init);` by another thread then - barring bugs - the memory location `foo` refers to will be marked as alive and not collected. [1] https://dlang.org/library/core/memory/gc.collect.html
Jul 29
parent Johan Engelen <j j.nl> writes:
On Saturday, 29 July 2017 at 22:15:41 UTC, Moritz Maxeiner wrote:
  Since in your example the (stack) variable `foo` is still 
 referring to the (heap) memory location of the destroyed object
Ah yes of course, thanks! - Johan
Jul 29
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Saturday, July 29, 2017 20:44:30 Johan Engelen via Digitalmars-d wrote:
 I'd like to check a bit of info I need for Address Sanitizer
 checking.

 The spec says [1]:
 Use the destroy function to finalize an object by calling its
 destructor. The memory of the object is not immediately
 deallocated, instead the GC will collect the memory of the object
 at an undetermined point after finalization:
 ```
    class Foo { int x; this() { x = 1; } }
    Foo foo = new Foo;
    destroy(foo);
    assert(foo.x == int.init);  // object is still accessible
 ```

 This tells me 2 things that I'd like to verify:
 1. The destroyed memory is set to the type's `.init` value. (but
 the Ctor is not called)
 2. It is _valid_ to access the memory after calling destroy.

 Point 2 is worrying: what if there is a thread switch right after
 destroy, in which a GC collect happens?

 Thanks,
    Johan

 [1] https://dlang.org/spec/class.html#deallocators
If destroy has been called on a class object, then it is a bug to access it at any point after that (IIRC, the expectation is that it will blow up in your face, because the vtable is gone - TDPL talks about this, I believe, but I don't know where my copy is at the moment, so I can't check). That being said, the memory is still valid. And as Moritz pointed out, if the memory is accessible, the GC won't free it. So, it's a bug to access the object, but it should be memory safe to do so. - Jonathan M Davis
Jul 29
parent reply Johan Engelen <j j.nl> writes:
On Saturday, 29 July 2017 at 23:09:38 UTC, Jonathan M Davis wrote:
 [1] https://dlang.org/spec/class.html#deallocators
If destroy has been called on a class object, then it is a bug to access it at any point after that (IIRC, the expectation is that it will blow up in your face, because the vtable is gone - TDPL talks about this, I believe, but I don't know where my copy is at the moment, so I can't check). That being said, the memory is still valid. And as Moritz pointed out, if the memory is accessible, the GC won't free it. So, it's a bug to access the object, but it should be memory safe to do so.
If this is the case, then we really need to improve and pin-down the spec on this point. So `destroy` only calls the destructor and puts the object in `T.initializer` (non-constructed) state, and is otherwise not related to memory deallocation. May the destructor be called again when the GC collects the memory? Why is the object put in `T.initializer` state? To make sure: My question is from a spec point of view. Not about what currently happens and what is "OK" with the current implementation.
Jul 30
next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
Johan Engelen wrote:

 If this is the case, then we really need to improve and pin-down the spec 
 on this point.

 So `destroy` only calls the destructor and puts the object in 
 `T.initializer` (non-constructed) state, and is otherwise not related to 
 memory deallocation.
 May the destructor be called again when the GC collects the memory?
 Why is the object put in `T.initializer` state?

 To make sure: My question is from a spec point of view. Not about what 
 currently happens and what is "OK" with the current implementation.
afair, somewhere in the spec there is a mention that dtor will be called at most once for each initialized object. and object state doesn't have any sense after calling dtor, but D still has to put something there, so `.init` looks like a reasonable choice. althru i'm not sure that anything in specs says that runtime *must* clear destroyed objects (they aren't really usable after calling dtor anyway, it is at least a logical bug to use object after destroying it).
Jul 30
parent Temtaime <temtaime gmail.com> writes:
On Sunday, 30 July 2017 at 07:58:19 UTC, ketmar wrote:
 Johan Engelen wrote:

 [...]
afair, somewhere in the spec there is a mention that dtor will be called at most once for each initialized object. and object state doesn't have any sense after calling dtor, but D still has to put something there, so `.init` looks like a reasonable choice. althru i'm not sure that anything in specs says that runtime *must* clear destroyed objects (they aren't really usable after calling dtor anyway, it is at least a logical bug to use object after destroying it).
Seems that zeroing it out is better choice, maybe.
Jul 30
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Sunday, 30 July 2017 at 07:45:27 UTC, Johan Engelen wrote:
 On Saturday, 29 July 2017 at 23:09:38 UTC, Jonathan M Davis 
 wrote:
 [1] https://dlang.org/spec/class.html#deallocators
If destroy has been called on a class object, then it is a bug to access it at any point after that (IIRC, the expectation is that it will blow up in your face, because the vtable is gone - TDPL talks about this, I believe, but I don't know where my copy is at the moment, so I can't check). That being said, the memory is still valid. And as Moritz pointed out, if the memory is accessible, the GC won't free it. So, it's a bug to access the object, but it should be memory safe to do so.
If this is the case, then we really need to improve and pin-down the spec on this point. So `destroy` only calls the destructor and puts the object in `T.initializer` (non-constructed) state, and is otherwise not related to memory deallocation. May the destructor be called again when the GC collects the memory? Why is the object put in `T.initializer` state?
rt_finalize2 zeroes the vptr out after assigning typeid(T).initializer, so calling the destructor more than once is not possible, unless someone manually saves the vptr and assigns it back to the object after the call to destroy / rt_finalize.
 To make sure: My question is from a spec point of view. Not 
 about what currently happens and what is "OK" with the current 
 implementation.
I guess the spec needs to be updated to state explicitly that a class destructor is called only once, if that's not already the case.
Jul 30
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/30/17 9:17 AM, Petar Kirov [ZombineDev] wrote:
 On Sunday, 30 July 2017 at 07:45:27 UTC, Johan Engelen wrote:
 On Saturday, 29 July 2017 at 23:09:38 UTC, Jonathan M Davis wrote:
 [1] https://dlang.org/spec/class.html#deallocators
If destroy has been called on a class object, then it is a bug to access it at any point after that (IIRC, the expectation is that it will blow up in your face, because the vtable is gone - TDPL talks about this, I believe, but I don't know where my copy is at the moment, so I can't check). That being said, the memory is still valid. And as Moritz pointed out, if the memory is accessible, the GC won't free it. So, it's a bug to access the object, but it should be memory safe to do so.
If this is the case, then we really need to improve and pin-down the spec on this point. So `destroy` only calls the destructor and puts the object in `T.initializer` (non-constructed) state, and is otherwise not related to memory deallocation. May the destructor be called again when the GC collects the memory? Why is the object put in `T.initializer` state?
rt_finalize2 zeroes the vptr out after assigning typeid(T).initializer, so calling the destructor more than once is not possible, unless someone manually saves the vptr and assigns it back to the object after the call to destroy / rt_finalize.
Yes, this is on purpose. It's the way the runtime knows it has already been run. In fact, destroy used to NOT do this, and that caused problems. A class instance is not usable after calling destroy. It is not intended to be usable, because classes require a constructor call to be valid, and there's no way to guarantee you can do this with destroy. Structs are different.
 To make sure: My question is from a spec point of view. Not about what 
 currently happens and what is "OK" with the current implementation.
I guess the spec needs to be updated to state explicitly that a class destructor is called only once, if that's not already the case.
I thought this was in the spec, but I can't find it. This is definitely the case. -Steve
Aug 01