www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Deterministic resource freeing - delete deprecation (again)

reply =?UTF-8?B?TWFyaXVzeiBHbGl3acWEc2tp?= <alienballance gmail.com> writes:
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hello,
I'm next person, which isn't necessarily happy about delete operator 
deprecation.
Because constructors / destructors are frequently used not only for 
application controlled memory management, how would You implement 
something like following code without delete operator?

<code>
void main(string[] args) {
     auto res = new Resource();
     auto s1 = new FirstSystem(res);
     delete s1;
     auto s2 = new SecondSystem(res);
}
class FirstSystem {
     this(Resource res) {
         res.referenced = true;
     }
     ~this() {
         res.referenced=false;
     }
}
class SecondSystem {
     this(Resource res) {
         assert(!res.referenced);
     }
}
</code>

Next questions would be:
* Are you going to drop destructor as well, or maybe destructor is going 
to be confusing feature with non-deterministic behaviour? (How would i 
know when heap allocated Socket will be destructed -> when connection 
will be closed -> if my system gonna be DoS'ed?)
* Are you going to stop supporting object oriented programming? (Well, 
if *deterministic* resource managing can be only inside function, and 
not object boundaries, that's my conclusion).

And finally, don't get me wrong. I'm just caricaturizing my current 
worries about the language, and if I'm missing something - please hint 
me what.

Sincerely,
Mariusz Gliwiński
Apr 27 2011
next sibling parent KennyTM~ <kennytm gmail.com> writes:
On Apr 27, 11 19:42, Mariusz Gliwiński wrote:

 Hello,
 I'm next person, which isn't necessarily happy about delete operator
 deprecation.



I don't get the recent resistant against deletion of 'delete' which has 
been decided months before (before TDPL). The operator is removed does 
not mean the destructor can't be invoked at all.


 Because constructors / destructors are frequently used not only for
 application controlled memory management, how would You implement
 something like following code without delete operator?



Use 'scoped' or a 'struct' if you want to constrain 's1' in a scope

import std.typecons;
void main(string[] args) {
     auto res = new Resource();
     {
         auto s1 = scoped!FirstSystem(res);
     }
     auto s2 = new SecondSystem(res);
}

<or>

void main(string[] args) {
     auto res = new Resource();
     {
         auto s1 = FirstSystem(res);
     }
     auto s2 = new SecondSystem(res);
}
struct FirstSystem {
     private Resource res;
     this(Resource res) {
         this.res = res;
         res.referenced = true;
     }
     ~this() {
         res.referenced=false;
     }
}
class Resource {
     public bool referenced;
}

Or use 'clear' to replace 'delete' as told in many posts already

void main(string[] args) {
     auto res = new Resource();
     auto s1 = new FirstSystem(res);
     // many miles away
     clear(s1);
     auto s2 = new SecondSystem(res);
}


 <code>




 </code>

 Next questions would be:
 * Are you going to drop destructor as well, or maybe destructor is going
 to be confusing feature with non-deterministic behaviour? (How would i
 know when heap allocated Socket will be destructed -> when connection
 will be closed -> if my system gonna be DoS'ed?)



The allocators (http://d-programming-language.org/class.html#allocators) 
and deallocators 
(http://d-programming-language.org/class.html#deallocators) are going 
away. The destructors 
(http://d-programming-language.org/class.html#destructors) are not.


 * Are you going to stop supporting object oriented programming? (Well,
 if *deterministic* resource managing can be only inside function, and
 not object boundaries, that's my conclusion).



'delete' only governs memory management. Since when deterministic 
*memory* management becomes a requirement of OOP? If you need 
deterministic resource management, you could use the dispose pattern. 
Java doesn't have a 'delete' operator either, that doesn't make Java a 
non-OOP language.


 And finally, don't get me wrong. I'm just caricaturizing my current
 worries about the language, and if I'm missing something - please hint
 me what.

 Sincerely,
 Mariusz Gliwiński
Apr 27 2011
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 07:42:22 -0400, Mariusz Gliwiński  
<alienballance gmail.com> wrote:


 Hello,
 I'm next person, which isn't necessarily happy about delete operator
 deprecation.
 Because constructors / destructors are frequently used not only for
 application controlled memory management, how would You implement
 something like following code without delete operator?



You can use clear to call the destructor of a class, and then GC.free (if  
you wish) to free the memory.  The recommendation is to only use clear,  
and then let the GC clean up the memory at its leisure.


 <code>
 void main(string[] args) {
      auto res = new Resource();
      auto s1 = new FirstSystem(res);
      delete s1;
      auto s2 = new SecondSystem(res);
 }
 class FirstSystem {
      this(Resource res) {
          res.referenced = true;
      }
      ~this() {
          res.referenced=false;
      }
 }
 class SecondSystem {
      this(Resource res) {
          assert(!res.referenced);
      }
 }
 </code>



Note there is a serious flaw in your destructor.  Destructors cannot  
access GC managed memory because if the destructor is called inside the  
GC, the GC managed memory being referenced may have already been destroyed.


 Next questions would be:
 * Are you going to drop destructor as well, or maybe destructor is going
 to be confusing feature with non-deterministic behaviour? (How would i
 know when heap allocated Socket will be destructed -> when connection
 will be closed -> if my system gonna be DoS'ed?)



If the Socket closes its resources in the finalizer, then use clear().  If  
the destructor does not close the resource (as could be the case if the  
resource is GC managed), you must implement a deterministic destructor  
member (with a different name, like close()) that you can call separately  
 from the finalizer.  There is no formal design for this, I really think  
there should be.


 * Are you going to stop supporting object oriented programming? (Well,
 if *deterministic* resource managing can be only inside function, and
 not object boundaries, that's my conclusion).



I assume this is rhetorical.

-Steve
Apr 27 2011
next sibling parent reply Alexander <aldem+dmars nk7.net> writes:
On 27.04.2011 15:15, Steven Schveighoffer wrote:


 You can use clear to call the destructor of a class, and then GC.free (if you
wish) to free the memory. 



  BTW, why it is called clear()? To me, clearing something is not destruction.
For instance, some buffer object may be cleared to empty the buffer, so the
action of clear() is counter-intuitive, IMHO.

/Alexander
Apr 27 2011
next sibling parent Daniel Gibson <metalcaedes gmail.com> writes:
Am 27.04.2011 15:37, schrieb Alexander:

 On 27.04.2011 15:15, Steven Schveighoffer wrote:


 You can use clear to call the destructor of a class, and then GC.free (if you
wish) to free the memory.



    BTW, why it is called clear()? To me, clearing something is not
destruction. For instance, some buffer object may be cleared to empty the
buffer, so the action of clear() is counter-intuitive, IMHO.

 /Alexander



clear() also works for simple types and pointers - it just resets them 
to type.init.
Which sucks when it's a pointer to a struct and the pointer is set to 
null instead of calling the structs destructor.

Maybe a function called "destroy()" that only works for classes and 
structs (with destructors? or just null other kinds of structs or 
something?) may help. It's a better name and it could work more like 
you'd expect for pointers to structs.


Cheers,
- Daniel
Apr 27 2011
prev sibling parent reply KennyTM~ <kennytm gmail.com> writes:
On Apr 27, 11 21:37, Alexander wrote:

 On 27.04.2011 15:15, Steven Schveighoffer wrote:


 You can use clear to call the destructor of a class, and then GC.free (if you
wish) to free the memory.



    BTW, why it is called clear()? To me, clearing something is not
destruction. For instance, some buffer object may be cleared to empty the
buffer, so the action of clear() is counter-intuitive, IMHO.

 /Alexander



Because Andrei chose to call it clear() :)

Well according to 
http://www.digitalmars.com/d/archives/digitalmars/D/clear_97688.html, 
because the function "clears the state of
an object.".

I'd rather it called destroy(), or when 'delete' is no longer a keyword, 
delete(), as .clear() is a container method for a different action 
(http://www.digitalmars.com/d/archives/digitalmars/D/Meaning_of_.clear_for_containers_126094.html).
Apr 27 2011
parent Alexander <aldem+dmars nk7.net> writes:
On 27.04.2011 15:46, KennyTM~ wrote:


 Because Andrei chose to call it clear() :)



  Well, this is bad choice, IMHO... I would expect that this will zeroize all
data (for arrays/structs at least), but for objects...


 ...because the function "clears the state of an object.".



  ...which is (from definition) doesn't imply destruction. Oh-oh...

/Alexander
Apr 27 2011
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-04-27 15:15, Steven Schveighoffer wrote:

 On Wed, 27 Apr 2011 07:42:22 -0400, Mariusz Gliwiński
 <alienballance gmail.com> wrote:


 Hello,
 I'm next person, which isn't necessarily happy about delete operator
 deprecation.
 Because constructors / destructors are frequently used not only for
 application controlled memory management, how would You implement
 something like following code without delete operator?



 You can use clear to call the destructor of a class, and then GC.free
 (if you wish) to free the memory. The recommendation is to only use
 clear, and then let the GC clean up the memory at its leisure.


 <code>
 void main(string[] args) {
 auto res = new Resource();
 auto s1 = new FirstSystem(res);
 delete s1;
 auto s2 = new SecondSystem(res);
 }
 class FirstSystem {
 this(Resource res) {
 res.referenced = true;
 }
 ~this() {
 res.referenced=false;
 }
 }
 class SecondSystem {
 this(Resource res) {
 assert(!res.referenced);
 }
 }
 </code>



 Note there is a serious flaw in your destructor. Destructors cannot
 access GC managed memory because if the destructor is called inside the
 GC, the GC managed memory being referenced may have already been destroyed.


 Next questions would be:
 * Are you going to drop destructor as well, or maybe destructor is going
 to be confusing feature with non-deterministic behaviour? (How would i
 know when heap allocated Socket will be destructed -> when connection
 will be closed -> if my system gonna be DoS'ed?)



 If the Socket closes its resources in the finalizer, then use clear().
 If the destructor does not close the resource (as could be the case if
 the resource is GC managed), you must implement a deterministic
 destructor member (with a different name, like close()) that you can
 call separately from the finalizer. There is no formal design for this,
 I really think there should be.



You mean like "dispose" in Tango. That's called when "delete" or "scope" 
is used.


 * Are you going to stop supporting object oriented programming? (Well,
 if *deterministic* resource managing can be only inside function, and
 not object boundaries, that's my conclusion).



 I assume this is rhetorical.

 -Steve




-- 
/Jacob Carlborg
Apr 27 2011
parent reply Alexander <aldem+dmars nk7.net> writes:
On 27.04.2011 16:25, Steven Schveighoffer wrote:


 You mean like "dispose" in Tango. That's called when "delete" or "scope" is
used.



 Yes, that is exactly what I was thinking of (couldn't think of the name!). 
It's actually used I think in Java and C# as well.



  Well, not exactly like this in C# - Dispose() is intended to free any
unmanaged resources, which are referenced by object, but not the memory
allocated to object itself.

/Alexander
Apr 27 2011
parent reply Alexander <aldem+dmars nk7.net> writes:
On 27.04.2011 16:48, Steven Schveighoffer wrote:


 basically, dispose cleans up resources *knowing* that all its references are
still valid.



  Right, but cleaning of resources doesn't mean that object will be destructed
- it can be reused afterwards (connection reestablished, file reopened etc).


 The deallocation of object memory is a separate thing (outside dispose or the
destructor).



  That's why I think that clear() is bad name choice, when it leads to
destruction :)

/Alexander
Apr 27 2011
next sibling parent Francisco Almeida <francisco.m.almeida gmail.com> writes:
== Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article

 On Wed, 27 Apr 2011 11:08:23 -0400, Alexander <aldem+dmars nk7.net> wrote:
 [...]
 The name choice is no longer up for debate.  It's already set in print,
 and in the language.
 -Steve



clear() is a library function which is itself having its behavior changed for
the
next DMD release. I think changing it to a more descriptive name such as
destroy()
is feasible. Especially when there are existing class methods already called
clear(). Even newcomers from C++ are used to clear() as a common method name
from
STL classes.
Apr 27 2011
prev sibling parent reply Alexander <aldem+dmars nk7.net> writes:
On 27.04.2011 17:23, Steven Schveighoffer wrote:


 I think the only difference between the destructor (finalizer) and dispose is
 that dispose is guaranteed that all the memory is still allocated, whereas the
finalizer is not given that guarantee.



  Not really. Dispose() may be called more than once, there is no guarantee
that resources are still allocated.


 The name choice is no longer up for debate.  It's already set in print, and in
the language.



  Well, it took me some time to find it out "in print" and "in the language",
given that everywhere on site no single reference to it, and in most recent D2
it doesn't work correctly.

  Not to mention that it conflicts with std.container and std.array, where
semantics of clear() is quite different (removes elements) - are those will be
changed too?

/Alexander
Apr 27 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-04-27 17:41, Alexander wrote:

 On 27.04.2011 17:23, Steven Schveighoffer wrote:


 I think the only difference between the destructor (finalizer) and dispose is
 that dispose is guaranteed that all the memory is still allocated, whereas the
finalizer is not given that guarantee.



    Not really. Dispose() may be called more than once, there is no guarantee
that resources are still allocated.



What I think Steven meant in this case is that when calling dispose, in 
Tango, any object will still be valid, i.e. the garbage collector hasn't 
collected it yet.


 The name choice is no longer up for debate.  It's already set in print, and in
the language.



    Well, it took me some time to find it out "in print" and "in the language",
given that everywhere on site no single reference to it, and in most recent D2
it doesn't work correctly.

    Not to mention that it conflicts with std.container and std.array, where
semantics of clear() is quite different (removes elements) - are those will be
changed too?

 /Alexander




-- 
/Jacob Carlborg
Apr 27 2011
parent reply Alexander <aldem+dmars nk7.net> writes:
On 27.04.2011 19:13, Steven Schveighoffer wrote:


 clear is not a keyword, it is possible to name a member clear, and also have a
clear global function.



  Sure it is, though it is counter-intuitive - to use same name with quite
different meaning. As for me, if this will really be in final spec, I'll always
alias it to destroy().

  BTW, could you please point me to "the ultimate authority"? I am referring to
your "The name choice is no longer up for debate.  It's already set in print,
and in the language." - so far, I was thinking that DigitalMars website is the
one, but your
words make me wonder, if this is still true.

  And, since there is a phrase: "It is not governed by a corporate agenda or
any overarching theory of programming. The needs and contributions of the D
programming community form the direction it goes." I would like to know, how
"the community" come
to this choice, which is quite confusing, at least.

/Alexander
Apr 27 2011
parent Francisco Almeida <francisco.m.almeida gmail.com> writes:
On 28-04-2011 17:09, Steven Schveighoffer wrote:

 On Wed, 27 Apr 2011 17:30:54 -0400, Alexander <aldem+dmars nk7.net> wrote:


 On 27.04.2011 19:13, Steven Schveighoffer wrote:


 clear is not a keyword, it is possible to name a member clear, and
 also have a clear global function.



 Sure it is, though it is counter-intuitive - to use same name with
 quite different meaning. As for me, if this will really be in final
 spec, I'll always alias it to destroy().



 That has been pointed out before. Yes, it is considered a bad idea to do
 that, but we have no choice now, clear is the name.



We have no choice? Since when is a function "part of the language"? It 
could still be renamed with minimal to no impact on Phobos. If it 
*really* cannot be renamed (which I doubt), then ok, I guess we'll have 
to live with it.
Apr 29 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 10:18:39 -0400, Jacob Carlborg <doob me.com> wrote:


 On 2011-04-27 15:15, Steven Schveighoffer wrote:




 If the Socket closes its resources in the finalizer, then use clear().
 If the destructor does not close the resource (as could be the case if
 the resource is GC managed), you must implement a deterministic
 destructor member (with a different name, like close()) that you can
 call separately from the finalizer. There is no formal design for this,
 I really think there should be.



 You mean like "dispose" in Tango. That's called when "delete" or "scope"  
 is used.



Yes, that is exactly what I was thinking of (couldn't think of the  
name!).  It's actually used I think in Java and C# as well.

-Steve
Apr 27 2011
prev sibling next sibling parent Peter Alexander <peter.alexander.au gmail.com> writes:
On 27/04/11 12:42 PM, Mariusz Gliwiński wrote:

 Hello,
 I'm next person, which isn't necessarily happy about delete operator
 deprecation.
 <snip>
 Sincerely,
 Mariusz Gliwiński



Perhaps someone with knowledge about the answer to this question should 
add it to the D FAQ. It is a frequently asked question after all. Might 
be good under the 'Rationale' section.
Apr 27 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 10:32:57 -0400, Alexander <aldem+dmars nk7.net> wrote:


 On 27.04.2011 16:25, Steven Schveighoffer wrote:


 You mean like "dispose" in Tango. That's called when "delete" or  
 "scope" is used.



 Yes, that is exactly what I was thinking of (couldn't think of the  
 name!).  It's actually used I think in Java and C# as well.



   Well, not exactly like this in C# - Dispose() is intended to free any  
 unmanaged resources, which are referenced by object, but not the memory  
 allocated to object itself.



dispose is supposed to be called deterministically (i.e. not by the GC),  
so yeah, it's the same thing.

basically, dispose cleans up resources *knowing* that all its references  
are still valid.

The destructor (finalizer) cleans up only resources it knows are not GC  
managed.

So for instance, if you have a Socket class which contains a file  
descriptor, and a SocketStream class that contains a Socket,  
SocketStream.dispose would call Socket.dispose, which would close the file  
descriptor.  But the destructor of SocketStream would not call  
Socket.dispose, since it does not know if the destructor could be called.

The deallocation of object memory is a separate thing (outside dispose or  
the destructor).

-Steve
Apr 27 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 10:48:08 -0400, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:


 So for instance, if you have a Socket class which contains a file  
 descriptor, and a SocketStream class that contains a Socket,  
 SocketStream.dispose would call Socket.dispose, which would close the  
 file descriptor.  But the destructor of SocketStream would not call  
 Socket.dispose, since it does not know if the destructor could be called.



This should have read "since it does not know if the Socket memory is  
still valid"

-Steve
Apr 27 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 11:08:23 -0400, Alexander <aldem+dmars nk7.net> wrote:


 On 27.04.2011 16:48, Steven Schveighoffer wrote:


 basically, dispose cleans up resources *knowing* that all its  
 references are still valid.



   Right, but cleaning of resources doesn't mean that object will be  
 destructed - it can be reused afterwards (connection reestablished, file  
 reopened etc).



I don't think reuse is a requirement of dispose.  It might be that those  
resources are only allocated on construction, which would require you to  
re-construct the object.  I think the only difference between the  
destructor (finalizer) and dispose is that dispose is guaranteed that all  
the memory is still allocated, whereas the finalizer is not given that  
guarantee.


 The deallocation of object memory is a separate thing (outside dispose  
 or the destructor).



   That's why I think that clear() is bad name choice, when it leads to  
 destruction :)



The name choice is no longer up for debate.  It's already set in print,  
and in the language.

-Steve
Apr 27 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 12:26:38 -0400, Jacob Carlborg <doob me.com> wrote:


 On 2011-04-27 17:41, Alexander wrote:

 On 27.04.2011 17:23, Steven Schveighoffer wrote:


 I think the only difference between the destructor (finalizer) and  
 dispose is
 that dispose is guaranteed that all the memory is still allocated,  
 whereas the finalizer is not given that guarantee.



    Not really. Dispose() may be called more than once, there is no  
 guarantee that resources are still allocated.



 What I think Steven meant in this case is that when calling dispose, in  
 Tango, any object will still be valid, i.e. the garbage collector hasn't  
 collected it yet.



Yes, that is exactly right.  I meant literally memory, not all resources.


 The name choice is no longer up for debate.  It's already set in  
 print, and in the language.



    Well, it took me some time to find it out "in print" and "in the  
 language", given that everywhere on site no single reference to it, and  
 in most recent D2 it doesn't work correctly.






There is a recently filed report on the lack of documentation.


    Not to mention that it conflicts with std.container and std.array,  
 where semantics of clear() is quite different (removes elements) - are  
 those will be changed too?






clear is not a keyword, it is possible to name a member clear, and also  
have a clear global function.

-Steve
Apr 27 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 27 Apr 2011 17:30:54 -0400, Alexander <aldem+dmars nk7.net> wrote:


 On 27.04.2011 19:13, Steven Schveighoffer wrote:


 clear is not a keyword, it is possible to name a member clear, and also  
 have a clear global function.



   Sure it is, though it is counter-intuitive - to use same name with  
 quite different meaning. As for me, if this will really be in final  
 spec, I'll always alias it to destroy().



That has been pointed out before.  Yes, it is considered a bad idea to do  
that, but we have no choice now, clear is the name.

I probably would consider changing the name of dcollections' function that  
empties all elements.  Clear is a very good name for that, but maybe it  
should be something else to avoid confusion.


   BTW, could you please point me to "the ultimate authority"? I am  
 referring to your "The name choice is no longer up for debate.  It's  
 already set in print, and in the language." - so far, I was thinking  
 that DigitalMars website is the one, but your
 words make me wonder, if this is still true.



http://www.amazon.com/gp/reader/0321635361/ref=sib_dp_pt#reader-link


   And, since there is a phrase: "It is not governed by a corporate  
 agenda or any overarching theory of programming. The needs and  
 contributions of the D programming community form the direction it  
 goes." I would like to know, how "the community" come
 to this choice, which is quite confusing, at least.



clear is Andrei's idea, he proposed it on the newsgroups.

Here was the first mention of it.

http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=97688

I'm not sure if anyone really debated the name until it was already part  
of the language, not much was said about the name in that thread.  I know  
many people have questioned the name of it after it was already in the  
language.  It may have caught some by surprise.

-Steve
Apr 28 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 29 Apr 2011 15:18:23 -0400, Francisco Almeida  
<francisco.m.almeida gmail.com> wrote:


 On 28-04-2011 17:09, Steven Schveighoffer wrote:

 On Wed, 27 Apr 2011 17:30:54 -0400, Alexander <aldem+dmars nk7.net>  
 wrote:


 On 27.04.2011 19:13, Steven Schveighoffer wrote:


 clear is not a keyword, it is possible to name a member clear, and
 also have a clear global function.



 Sure it is, though it is counter-intuitive - to use same name with
 quite different meaning. As for me, if this will really be in final
 spec, I'll always alias it to destroy().



 That has been pointed out before. Yes, it is considered a bad idea to do
 that, but we have no choice now, clear is the name.



 We have no choice? Since when is a function "part of the language"? It  
 could still be renamed with minimal to no impact on Phobos. If it  
 *really* cannot be renamed (which I doubt), then ok, I guess we'll have  
 to live with it.



It's already in the official book describing the language (IIRC, there's  
quite a bit of text on it).  At some point, we have to stop changing parts  
of the spec/design so people can have a stable environment.  The decision  
was to avoid changing things that are covered in TDPL as much as possible.

Is it possible to change?  Yes.  But you will have to convince Andrei and  
Walter, and really, this is a naming issue.  It's not tantamount to the  
language's functionality.

You are more likely to see the clear function in std.container change.

-Steve
Apr 29 2011