www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Deterministic resource freeing - delete deprecation (again)

reply =?UTF-8?B?TWFyaXVzeiBHbGl3acWEc2tp?= <alienballance gmail.com> writes:
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hello,
I'm next person, which isn't necessarily happy about delete operator 
deprecation.
Because constructors / destructors are frequently used not only for 
application controlled memory management, how would You implement 
something like following code without delete operator?

<code>
void main(string[] args) {
     auto res = new Resource();
     auto s1 = new FirstSystem(res);
     delete s1;
     auto s2 = new SecondSystem(res);
}
class FirstSystem {
     this(Resource res) {
         res.referenced = true;
     }
     ~this() {
         res.referenced=false;
     }
}
class SecondSystem {
     this(Resource res) {
         assert(!res.referenced);
     }
}
</code>

Next questions would be:
* Are you going to drop destructor as well, or maybe destructor is going 
to be confusing feature with non-deterministic behaviour? (How would i 
know when heap allocated Socket will be destructed -> when connection 
will be closed -> if my system gonna be DoS'ed?)
* Are you going to stop supporting object oriented programming? (Well, 
if *deterministic* resource managing can be only inside function, and 
not object boundaries, that's my conclusion).

And finally, don't get me wrong. I'm just caricaturizing my current 
worries about the language, and if I'm missing something - please hint 
me what.

Sincerely,
Mariusz Gliwiński
Apr 27 2011
next sibling parent KennyTM~ <kennytm gmail.com> writes:
On Apr 27, 11 19:42, Mariusz Gliwiński wrote:
 Hello,
 I'm next person, which isn't necessarily happy about delete operator
 deprecation.

I don't get the recent resistant against deletion of 'delete' which has been decided months before (before TDPL). The operator is removed does not mean the destructor can't be invoked at all.
 Because constructors / destructors are frequently used not only for
 application controlled memory management, how would You implement
 something like following code without delete operator?

Use 'scoped' or a 'struct' if you want to constrain 's1' in a scope import std.typecons; void main(string[] args) { auto res = new Resource(); { auto s1 = scoped!FirstSystem(res); } auto s2 = new SecondSystem(res); } <or> void main(string[] args) { auto res = new Resource(); { auto s1 = FirstSystem(res); } auto s2 = new SecondSystem(res); } struct FirstSystem { private Resource res; this(Resource res) { this.res = res; res.referenced = true; } ~this() { res.referenced=false; } } class Resource { public bool referenced; } Or use 'clear' to replace 'delete' as told in many posts already void main(string[] args) { auto res = new Resource(); auto s1 = new FirstSystem(res); // many miles away clear(s1); auto s2 = new SecondSystem(res); }
 <code>

 </code>

 Next questions would be:
 * Are you going to drop destructor as well, or maybe destructor is going
 to be confusing feature with non-deterministic behaviour? (How would i
 know when heap allocated Socket will be destructed -> when connection
 will be closed -> if my system gonna be DoS'ed?)

The allocators (http://d-programming-language.org/class.html#allocators) and deallocators (http://d-programming-language.org/class.html#deallocators) are going away. The destructors (http://d-programming-language.org/class.html#destructors) are not.
 * Are you going to stop supporting object oriented programming? (Well,
 if *deterministic* resource managing can be only inside function, and
 not object boundaries, that's my conclusion).

'delete' only governs memory management. Since when deterministic *memory* management becomes a requirement of OOP? If you need deterministic resource management, you could use the dispose pattern. Java doesn't have a 'delete' operator either, that doesn't make Java a non-OOP language.
 And finally, don't get me wrong. I'm just caricaturizing my current
 worries about the language, and if I'm missing something - please hint
 me what.

 Sincerely,
 Mariusz Gliwiński

Apr 27 2011
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 07:42:22 -0400, Mariusz Gliwiński  
<alienballance gmail.com> wrote:

 Hello,
 I'm next person, which isn't necessarily happy about delete operator
 deprecation.
 Because constructors / destructors are frequently used not only for
 application controlled memory management, how would You implement
 something like following code without delete operator?

You can use clear to call the destructor of a class, and then GC.free (if you wish) to free the memory. The recommendation is to only use clear, and then let the GC clean up the memory at its leisure.
 <code>
 void main(string[] args) {
      auto res = new Resource();
      auto s1 = new FirstSystem(res);
      delete s1;
      auto s2 = new SecondSystem(res);
 }
 class FirstSystem {
      this(Resource res) {
          res.referenced = true;
      }
      ~this() {
          res.referenced=false;
      }
 }
 class SecondSystem {
      this(Resource res) {
          assert(!res.referenced);
      }
 }
 </code>

Note there is a serious flaw in your destructor. Destructors cannot access GC managed memory because if the destructor is called inside the GC, the GC managed memory being referenced may have already been destroyed.
 Next questions would be:
 * Are you going to drop destructor as well, or maybe destructor is going
 to be confusing feature with non-deterministic behaviour? (How would i
 know when heap allocated Socket will be destructed -> when connection
 will be closed -> if my system gonna be DoS'ed?)

If the Socket closes its resources in the finalizer, then use clear(). If the destructor does not close the resource (as could be the case if the resource is GC managed), you must implement a deterministic destructor member (with a different name, like close()) that you can call separately from the finalizer. There is no formal design for this, I really think there should be.
 * Are you going to stop supporting object oriented programming? (Well,
 if *deterministic* resource managing can be only inside function, and
 not object boundaries, that's my conclusion).

I assume this is rhetorical. -Steve
Apr 27 2011
next sibling parent reply Alexander <aldem+dmars nk7.net> writes:
On 27.04.2011 15:15, Steven Schveighoffer wrote:

 You can use clear to call the destructor of a class, and then GC.free (if you
wish) to free the memory. 

BTW, why it is called clear()? To me, clearing something is not destruction. For instance, some buffer object may be cleared to empty the buffer, so the action of clear() is counter-intuitive, IMHO. /Alexander
Apr 27 2011
next sibling parent Daniel Gibson <metalcaedes gmail.com> writes:
Am 27.04.2011 15:37, schrieb Alexander:
 On 27.04.2011 15:15, Steven Schveighoffer wrote:

 You can use clear to call the destructor of a class, and then GC.free (if you
wish) to free the memory.

BTW, why it is called clear()? To me, clearing something is not destruction. For instance, some buffer object may be cleared to empty the buffer, so the action of clear() is counter-intuitive, IMHO. /Alexander

clear() also works for simple types and pointers - it just resets them to type.init. Which sucks when it's a pointer to a struct and the pointer is set to null instead of calling the structs destructor. Maybe a function called "destroy()" that only works for classes and structs (with destructors? or just null other kinds of structs or something?) may help. It's a better name and it could work more like you'd expect for pointers to structs. Cheers, - Daniel
Apr 27 2011
prev sibling parent reply KennyTM~ <kennytm gmail.com> writes:
On Apr 27, 11 21:37, Alexander wrote:
 On 27.04.2011 15:15, Steven Schveighoffer wrote:

 You can use clear to call the destructor of a class, and then GC.free (if you
wish) to free the memory.

BTW, why it is called clear()? To me, clearing something is not destruction. For instance, some buffer object may be cleared to empty the buffer, so the action of clear() is counter-intuitive, IMHO. /Alexander

Because Andrei chose to call it clear() :) Well according to http://www.digitalmars.com/d/archives/digitalmars/D/clear_97688.html, because the function "clears the state of an object.". I'd rather it called destroy(), or when 'delete' is no longer a keyword, delete(), as .clear() is a container method for a different action (http://www.digitalmars.com/d/archives/digitalmars/D/Meaning_of_.clear_for_containers_126094.html).
Apr 27 2011
parent Alexander <aldem+dmars nk7.net> writes:
On 27.04.2011 15:46, KennyTM~ wrote:

 Because Andrei chose to call it clear() :)

Well, this is bad choice, IMHO... I would expect that this will zeroize all data (for arrays/structs at least), but for objects...
 ...because the function "clears the state of an object.".

...which is (from definition) doesn't imply destruction. Oh-oh... /Alexander
Apr 27 2011
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-04-27 15:15, Steven Schveighoffer wrote:
 On Wed, 27 Apr 2011 07:42:22 -0400, Mariusz Gliwiński
 <alienballance gmail.com> wrote:

 Hello,
 I'm next person, which isn't necessarily happy about delete operator
 deprecation.
 Because constructors / destructors are frequently used not only for
 application controlled memory management, how would You implement
 something like following code without delete operator?

You can use clear to call the destructor of a class, and then GC.free (if you wish) to free the memory. The recommendation is to only use clear, and then let the GC clean up the memory at its leisure.
 <code>
 void main(string[] args) {
 auto res = new Resource();
 auto s1 = new FirstSystem(res);
 delete s1;
 auto s2 = new SecondSystem(res);
 }
 class FirstSystem {
 this(Resource res) {
 res.referenced = true;
 }
 ~this() {
 res.referenced=false;
 }
 }
 class SecondSystem {
 this(Resource res) {
 assert(!res.referenced);
 }
 }
 </code>

Note there is a serious flaw in your destructor. Destructors cannot access GC managed memory because if the destructor is called inside the GC, the GC managed memory being referenced may have already been destroyed.
 Next questions would be:
 * Are you going to drop destructor as well, or maybe destructor is going
 to be confusing feature with non-deterministic behaviour? (How would i
 know when heap allocated Socket will be destructed -> when connection
 will be closed -> if my system gonna be DoS'ed?)

If the Socket closes its resources in the finalizer, then use clear(). If the destructor does not close the resource (as could be the case if the resource is GC managed), you must implement a deterministic destructor member (with a different name, like close()) that you can call separately from the finalizer. There is no formal design for this, I really think there should be.

You mean like "dispose" in Tango. That's called when "delete" or "scope" is used.
 * Are you going to stop supporting object oriented programming? (Well,
 if *deterministic* resource managing can be only inside function, and
 not object boundaries, that's my conclusion).

I assume this is rhetorical. -Steve

-- /Jacob Carlborg
Apr 27 2011
parent reply Alexander <aldem+dmars nk7.net> writes:
On 27.04.2011 16:25, Steven Schveighoffer wrote:

 You mean like "dispose" in Tango. That's called when "delete" or "scope" is
used.

Yes, that is exactly what I was thinking of (couldn't think of the name!). It's actually used I think in Java and C# as well.

Well, not exactly like this in C# - Dispose() is intended to free any unmanaged resources, which are referenced by object, but not the memory allocated to object itself. /Alexander
Apr 27 2011
parent reply Alexander <aldem+dmars nk7.net> writes:
On 27.04.2011 16:48, Steven Schveighoffer wrote:

 basically, dispose cleans up resources *knowing* that all its references are
still valid.

Right, but cleaning of resources doesn't mean that object will be destructed - it can be reused afterwards (connection reestablished, file reopened etc).
 The deallocation of object memory is a separate thing (outside dispose or the
destructor).

That's why I think that clear() is bad name choice, when it leads to destruction :) /Alexander
Apr 27 2011
next sibling parent Francisco Almeida <francisco.m.almeida gmail.com> writes:
== Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article
 On Wed, 27 Apr 2011 11:08:23 -0400, Alexander <aldem+dmars nk7.net> wrote:
 [...]
 The name choice is no longer up for debate.  It's already set in print,
 and in the language.
 -Steve

clear() is a library function which is itself having its behavior changed for the next DMD release. I think changing it to a more descriptive name such as destroy() is feasible. Especially when there are existing class methods already called clear(). Even newcomers from C++ are used to clear() as a common method name from STL classes.
Apr 27 2011
prev sibling parent reply Alexander <aldem+dmars nk7.net> writes:
On 27.04.2011 17:23, Steven Schveighoffer wrote:

 I think the only difference between the destructor (finalizer) and dispose is
 that dispose is guaranteed that all the memory is still allocated, whereas the
finalizer is not given that guarantee.

Not really. Dispose() may be called more than once, there is no guarantee that resources are still allocated.
 The name choice is no longer up for debate.  It's already set in print, and in
the language.

Well, it took me some time to find it out "in print" and "in the language", given that everywhere on site no single reference to it, and in most recent D2 it doesn't work correctly. Not to mention that it conflicts with std.container and std.array, where semantics of clear() is quite different (removes elements) - are those will be changed too? /Alexander
Apr 27 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-04-27 17:41, Alexander wrote:
 On 27.04.2011 17:23, Steven Schveighoffer wrote:

 I think the only difference between the destructor (finalizer) and dispose is
 that dispose is guaranteed that all the memory is still allocated, whereas the
finalizer is not given that guarantee.

Not really. Dispose() may be called more than once, there is no guarantee that resources are still allocated.

What I think Steven meant in this case is that when calling dispose, in Tango, any object will still be valid, i.e. the garbage collector hasn't collected it yet.
 The name choice is no longer up for debate.  It's already set in print, and in
the language.

Well, it took me some time to find it out "in print" and "in the language", given that everywhere on site no single reference to it, and in most recent D2 it doesn't work correctly. Not to mention that it conflicts with std.container and std.array, where semantics of clear() is quite different (removes elements) - are those will be changed too? /Alexander

-- /Jacob Carlborg
Apr 27 2011
parent reply Alexander <aldem+dmars nk7.net> writes:
On 27.04.2011 19:13, Steven Schveighoffer wrote:

 clear is not a keyword, it is possible to name a member clear, and also have a
clear global function.

Sure it is, though it is counter-intuitive - to use same name with quite different meaning. As for me, if this will really be in final spec, I'll always alias it to destroy(). BTW, could you please point me to "the ultimate authority"? I am referring to your "The name choice is no longer up for debate. It's already set in print, and in the language." - so far, I was thinking that DigitalMars website is the one, but your words make me wonder, if this is still true. And, since there is a phrase: "It is not governed by a corporate agenda or any overarching theory of programming. The needs and contributions of the D programming community form the direction it goes." I would like to know, how "the community" come to this choice, which is quite confusing, at least. /Alexander
Apr 27 2011
parent Francisco Almeida <francisco.m.almeida gmail.com> writes:
On 28-04-2011 17:09, Steven Schveighoffer wrote:
 On Wed, 27 Apr 2011 17:30:54 -0400, Alexander <aldem+dmars nk7.net> wrote:

 On 27.04.2011 19:13, Steven Schveighoffer wrote:

 clear is not a keyword, it is possible to name a member clear, and
 also have a clear global function.

Sure it is, though it is counter-intuitive - to use same name with quite different meaning. As for me, if this will really be in final spec, I'll always alias it to destroy().

That has been pointed out before. Yes, it is considered a bad idea to do that, but we have no choice now, clear is the name.

We have no choice? Since when is a function "part of the language"? It could still be renamed with minimal to no impact on Phobos. If it *really* cannot be renamed (which I doubt), then ok, I guess we'll have to live with it.
Apr 29 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 10:18:39 -0400, Jacob Carlborg <doob me.com> wrote:

 On 2011-04-27 15:15, Steven Schveighoffer wrote:

 If the Socket closes its resources in the finalizer, then use clear().
 If the destructor does not close the resource (as could be the case if
 the resource is GC managed), you must implement a deterministic
 destructor member (with a different name, like close()) that you can
 call separately from the finalizer. There is no formal design for this,
 I really think there should be.

You mean like "dispose" in Tango. That's called when "delete" or "scope" is used.

Yes, that is exactly what I was thinking of (couldn't think of the name!). It's actually used I think in Java and C# as well. -Steve
Apr 27 2011
prev sibling next sibling parent Peter Alexander <peter.alexander.au gmail.com> writes:
On 27/04/11 12:42 PM, Mariusz Gliwiński wrote:
 Hello,
 I'm next person, which isn't necessarily happy about delete operator
 deprecation.
 <snip>
 Sincerely,
 Mariusz Gliwiński

Perhaps someone with knowledge about the answer to this question should add it to the D FAQ. It is a frequently asked question after all. Might be good under the 'Rationale' section.
Apr 27 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 10:32:57 -0400, Alexander <aldem+dmars nk7.net> wrote:

 On 27.04.2011 16:25, Steven Schveighoffer wrote:

 You mean like "dispose" in Tango. That's called when "delete" or  
 "scope" is used.

Yes, that is exactly what I was thinking of (couldn't think of the name!). It's actually used I think in Java and C# as well.

Well, not exactly like this in C# - Dispose() is intended to free any unmanaged resources, which are referenced by object, but not the memory allocated to object itself.

dispose is supposed to be called deterministically (i.e. not by the GC), so yeah, it's the same thing. basically, dispose cleans up resources *knowing* that all its references are still valid. The destructor (finalizer) cleans up only resources it knows are not GC managed. So for instance, if you have a Socket class which contains a file descriptor, and a SocketStream class that contains a Socket, SocketStream.dispose would call Socket.dispose, which would close the file descriptor. But the destructor of SocketStream would not call Socket.dispose, since it does not know if the destructor could be called. The deallocation of object memory is a separate thing (outside dispose or the destructor). -Steve
Apr 27 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 10:48:08 -0400, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 So for instance, if you have a Socket class which contains a file  
 descriptor, and a SocketStream class that contains a Socket,  
 SocketStream.dispose would call Socket.dispose, which would close the  
 file descriptor.  But the destructor of SocketStream would not call  
 Socket.dispose, since it does not know if the destructor could be called.

This should have read "since it does not know if the Socket memory is still valid" -Steve
Apr 27 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 11:08:23 -0400, Alexander <aldem+dmars nk7.net> wrote:

 On 27.04.2011 16:48, Steven Schveighoffer wrote:

 basically, dispose cleans up resources *knowing* that all its  
 references are still valid.

Right, but cleaning of resources doesn't mean that object will be destructed - it can be reused afterwards (connection reestablished, file reopened etc).

I don't think reuse is a requirement of dispose. It might be that those resources are only allocated on construction, which would require you to re-construct the object. I think the only difference between the destructor (finalizer) and dispose is that dispose is guaranteed that all the memory is still allocated, whereas the finalizer is not given that guarantee.
 The deallocation of object memory is a separate thing (outside dispose  
 or the destructor).

That's why I think that clear() is bad name choice, when it leads to destruction :)

The name choice is no longer up for debate. It's already set in print, and in the language. -Steve
Apr 27 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 12:26:38 -0400, Jacob Carlborg <doob me.com> wrote:

 On 2011-04-27 17:41, Alexander wrote:
 On 27.04.2011 17:23, Steven Schveighoffer wrote:

 I think the only difference between the destructor (finalizer) and  
 dispose is
 that dispose is guaranteed that all the memory is still allocated,  
 whereas the finalizer is not given that guarantee.

Not really. Dispose() may be called more than once, there is no guarantee that resources are still allocated.

What I think Steven meant in this case is that when calling dispose, in Tango, any object will still be valid, i.e. the garbage collector hasn't collected it yet.

Yes, that is exactly right. I meant literally memory, not all resources.
 The name choice is no longer up for debate.  It's already set in  
 print, and in the language.

Well, it took me some time to find it out "in print" and "in the language", given that everywhere on site no single reference to it, and in most recent D2 it doesn't work correctly.


There is a recently filed report on the lack of documentation.
    Not to mention that it conflicts with std.container and std.array,  
 where semantics of clear() is quite different (removes elements) - are  
 those will be changed too?


clear is not a keyword, it is possible to name a member clear, and also have a clear global function. -Steve
Apr 27 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 17:30:54 -0400, Alexander <aldem+dmars nk7.net> wrote:

 On 27.04.2011 19:13, Steven Schveighoffer wrote:

 clear is not a keyword, it is possible to name a member clear, and also  
 have a clear global function.

Sure it is, though it is counter-intuitive - to use same name with quite different meaning. As for me, if this will really be in final spec, I'll always alias it to destroy().

That has been pointed out before. Yes, it is considered a bad idea to do that, but we have no choice now, clear is the name. I probably would consider changing the name of dcollections' function that empties all elements. Clear is a very good name for that, but maybe it should be something else to avoid confusion.
   BTW, could you please point me to "the ultimate authority"? I am  
 referring to your "The name choice is no longer up for debate.  It's  
 already set in print, and in the language." - so far, I was thinking  
 that DigitalMars website is the one, but your
 words make me wonder, if this is still true.

http://www.amazon.com/gp/reader/0321635361/ref=sib_dp_pt#reader-link
   And, since there is a phrase: "It is not governed by a corporate  
 agenda or any overarching theory of programming. The needs and  
 contributions of the D programming community form the direction it  
 goes." I would like to know, how "the community" come
 to this choice, which is quite confusing, at least.

clear is Andrei's idea, he proposed it on the newsgroups. Here was the first mention of it. http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=97688 I'm not sure if anyone really debated the name until it was already part of the language, not much was said about the name in that thread. I know many people have questioned the name of it after it was already in the language. It may have caught some by surprise. -Steve
Apr 28 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 29 Apr 2011 15:18:23 -0400, Francisco Almeida  
<francisco.m.almeida gmail.com> wrote:

 On 28-04-2011 17:09, Steven Schveighoffer wrote:
 On Wed, 27 Apr 2011 17:30:54 -0400, Alexander <aldem+dmars nk7.net>  
 wrote:

 On 27.04.2011 19:13, Steven Schveighoffer wrote:

 clear is not a keyword, it is possible to name a member clear, and
 also have a clear global function.

Sure it is, though it is counter-intuitive - to use same name with quite different meaning. As for me, if this will really be in final spec, I'll always alias it to destroy().

That has been pointed out before. Yes, it is considered a bad idea to do that, but we have no choice now, clear is the name.

We have no choice? Since when is a function "part of the language"? It could still be renamed with minimal to no impact on Phobos. If it *really* cannot be renamed (which I doubt), then ok, I guess we'll have to live with it.

It's already in the official book describing the language (IIRC, there's quite a bit of text on it). At some point, we have to stop changing parts of the spec/design so people can have a stable environment. The decision was to avoid changing things that are covered in TDPL as much as possible. Is it possible to change? Yes. But you will have to convince Andrei and Walter, and really, this is a naming issue. It's not tantamount to the language's functionality. You are more likely to see the clear function in std.container change. -Steve
Apr 29 2011