www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Eliminate class allocators and deallocators?

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Hello,


D currently allows defining class allocators and deallocators. They have 
a number of problems that make them unsuitable for D 2.0. The most 
obvious issue is that D 2.0 will _not_ conflate destruction with 
deallocation anymore: invoking delete against an object will call 
~this() against it but will not recycle its memory. In contrast, class 
deallocators are designed around the idea that invoking delete calls the 
destructor and also deallocates memory.

So I'm thinking of removing at least class deallocators from the 
language. Class allocators may be marginally and occasionally useful if 
the user takes the matter of deallocation in her own hands.

A much better way to handle custom allocation of classes would be in the 
standard library.

What do you think?


Andrei
Oct 06 2009
next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Tue, 06 Oct 2009 20:01:01 +0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Hello,


 D currently allows defining class allocators and deallocators. They have  
 a number of problems that make them unsuitable for D 2.0. The most  
 obvious issue is that D 2.0 will _not_ conflate destruction with  
 deallocation anymore: invoking delete against an object will call  
 ~this() against it but will not recycle its memory. In contrast, class  
 deallocators are designed around the idea that invoking delete calls the  
 destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the  
 language. Class allocators may be marginally and occasionally useful if  
 the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be in the  
 standard library.

 What do you think?


 Andrei

I think it's okay, but I have a few points to discuss: - Is it worth a keyword (delete) just as a shortcut for obj.__dtor();? - I believe __ctor and __dtor (and __traits, too) are ugly reserved identifiers. - It's often useful to distinguish between dtor being called by user and a dtor being called by a GC. In the latter case you can't dereference any reference since they might be invalid. - How about object.d provides an IDisposable (or something like this) interface which defines a void dispose(bool finalizing); and we get rid of ~this/__dtor entirely? - obj.__ctor() is mostly used as a placement new. A new placement new mechanism would allow drop that identifier, too. - D has 2 placement new mechanisms for a struct, that are absolutely the same (but different in syntax):
 Foo foo1 = void;
 Foo foo2 = void;

 foo1 = Foo();	   // indirect
 foo2.__ctor();	  // direct

The latter one could also be used for class initialization, but not the former one. You once announced an idea of dropping the 'new' keyword altogether and make the former case also valid for classes. What's about it now? - Could you elaborate on "a much better way to handle custom allocation of classes ... in the standard library"? An example would be very appreciated.
Oct 06 2009
prev sibling next sibling parent Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Tue, Oct 6, 2009 at 12:01 PM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They have a
 number of problems that make them unsuitable for D 2.0. The most obvious
 issue is that D 2.0 will _not_ conflate destruction with deallocation
 anymore: invoking delete against an object will call ~this() against it but
 will not recycle its memory. In contrast, class deallocators are designed
 around the idea that invoking delete calls the destructor and also
 deallocates memory.

 So I'm thinking of removing at least class deallocators from the language.
 Class allocators may be marginally and occasionally useful if the user takes
 the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be in the
 standard library.

..How? Without a custom allocator to do even a simple placement new, how would you handle such a thing in the library?
Oct 06 2009
prev sibling next sibling parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 Hello,
 D currently allows defining class allocators and deallocators. They have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast, class
 deallocators are designed around the idea that invoking delete calls the
 destructor and also deallocates memory.
 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally useful if
 the user takes the matter of deallocation in her own hands.
 A much better way to handle custom allocation of classes would be in the
 standard library.
 What do you think?
 Andrei

Please give at least a little more detail. I think it's an absolute must that advanced users who really want to do manual memory management in D (even if they want to only do it selectively and without kludges like using two completely separate heaps) be able to do so. I can't really comment until I know at least roughly what a standard lib solution might look like.
Oct 06 2009
prev sibling next sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
What exactly is your suggestion?

It seems that you mean that:
delete obj;

should call a destructor but not call delete() or notify the GC that the 
memory is free.

You're saying that there is a problem, but you're not telling us what's 
wrong. Why the hell do you want to destroy an object without recycling 
its memory? Why does the inability to do so cause a problem?

It seems like a performance hack to me -- you've got an object that 
isn't valid anymore, but you want to hang on to the memory for some 
other purpose. And you could override new() and delete(), but you don't 
want to incur the performance penalty of calling the runtime to fetch 
the deallocator.

The only remaining use that I see is a way to reset a shared object 
without explicitly passing around a reference to the new version of the 
object. This seems potentially dangerous, and nothing I want for default 
behavior.
Oct 06 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Christopher Wright wrote:
 What exactly is your suggestion?
 
 It seems that you mean that:
 delete obj;
 
 should call a destructor but not call delete() or notify the GC that the 
 memory is free.

That is correct. In particular, an object remains usable after delete.
 You're saying that there is a problem, but you're not telling us what's 
 wrong. Why the hell do you want to destroy an object without recycling 
 its memory? Why does the inability to do so cause a problem?

The matter has been discussed quite a bit around here and in other places. I'm not having as much time as I'd want to explain things. In short, destroying without freeing memory avoids dangling references and preserves memory safety without impacting on other resources.
 It seems like a performance hack to me -- you've got an object that 
 isn't valid anymore, but you want to hang on to the memory for some 
 other purpose. And you could override new() and delete(), but you don't 
 want to incur the performance penalty of calling the runtime to fetch 
 the deallocator.

It's a safety hack, not a performance hack.
 The only remaining use that I see is a way to reset a shared object 
 without explicitly passing around a reference to the new version of the 
 object. This seems potentially dangerous, and nothing I want for default 
 behavior.

Well incidentally at least as of now "delete obj" puts null in obj... Andrei
Oct 06 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Leandro Lucarella wrote:
 Andrei Alexandrescu, el  6 de octubre a las 19:26 me escribiste:
 Christopher Wright wrote:
 What exactly is your suggestion?

 It seems that you mean that:
 delete obj;

 should call a destructor but not call delete() or notify the GC
 that the memory is free.


Why would you do that? What is the rationale to not notify the GC?

Because there may be other live references to the object.
 You're saying that there is a problem, but you're not telling us
 what's wrong. Why the hell do you want to destroy an object
 without recycling its memory? Why does the inability to do so
 cause a problem?

places. I'm not having as much time as I'd want to explain things. In short, destroying without freeing memory avoids dangling references and preserves memory safety without impacting on other resources.

But D is a system programming language.

Well it is but there are quite a few more things at stake. First, it is a reality that it is often desirable to distinguish between calling the destructor and reclaiming memory. D's current delete continues the bad tradition started by C++ of conflating the two.
 If you wrote delete x; the
 language should assume you know what you're doing.

I think delete should be present in SafeD and if you want manual memory management you should build on malloc and free.
 If you only want to
 "deinitialize" an object, you can write a .destroy() method for example,
 and call that. I think delete have a strong established semantic to change
 it now, and without any gain.

It has a thoroughly broken and undesired semantics. It would be a step forward to divorce it of that. In fact i'd love to simply make delete disappear as a keyword and make it a function.
 It seems like a performance hack to me -- you've got an object
 that isn't valid anymore, but you want to hang on to the memory
 for some other purpose. And you could override new() and delete(),
 but you don't want to incur the performance penalty of calling the
 runtime to fetch the deallocator.


But you shouldn't provide safety where the programmer is not expecting it. delete is for *manual* memory management. It makes no sense to guarantee that the memory is *not* freed. It makes sense not guaranteeing that it will actually be freed either. I think that's a good idea actually, because it gives more flexibility to the GC implementation.

I think we should move away from the idea that delete is for manual memory management. We should leave that to the likes of malloc and free alone.
 The only remaining use that I see is a way to reset a shared
 object without explicitly passing around a reference to the new
 version of the object. This seems potentially dangerous, and
 nothing I want for default behavior.


That's nice :)

I think it's a false sense of security. C++ beginners keep on suggesting that feature and C++ pundits keep on explaining them that it's ungainful. Andrei
Oct 06 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Leandro Lucarella wrote:
 Andrei Alexandrescu, el  6 de octubre a las 21:42 me escribiste:
 should call a destructor but not call delete() or notify the GC
 that the memory is free.




But when using delete that's exactly what it should happen. You are hiding a bug if you let that happen on purpose.

That is not hiding a bug. That's even worse than Walter's crappy argument :o).
 You're saying that there is a problem, but you're not telling us
 what's wrong. Why the hell do you want to destroy an object
 without recycling its memory? Why does the inability to do so
 cause a problem?

places. I'm not having as much time as I'd want to explain things. In short, destroying without freeing memory avoids dangling references and preserves memory safety without impacting on other resources.


is a reality that it is often desirable to distinguish between calling the destructor and reclaiming memory. D's current delete continues the bad tradition started by C++ of conflating the two.

Why is a bad idea? If you are destroying an object, the object will be in an inconsistent state. What's the point of keeping it alive. Again, you're just hiding a bug; letting the bug live longer. The language should try to expose bugs ASAP, not delay the detection.

It is a bad idea because distinguishing between release of (expensive) resources from dangerous memory recycling is the correct way to obtain deterministic resource management within the confines of safety.
 I think is a good idea not to force the GC to free the memory immediately
 with a delete, but it should if it's easy. Other protection methods as
 using mprotect to protect the objects pages it's very desirable too,
 because you can spot an access to a inconsistent (destroyed) object as
 soon as it first happen.

(mprotect is much too coarse to be useful.) With the dispose() function the state of the object will be restored to default construction: void dispose(T)(T obj) if (is(T == class) || is(typeof(*T.init))) { ... call destructor if any ... ... obliterate object with .init ... ... invoke default ctor if any ... }
 If you wrote delete x; the
 language should assume you know what you're doing.

memory management you should build on malloc and free.

If you want to introduce a new semantic, I think you should provide a new method, not change the semantic of an existent one.

Agreed. I hereby vote for deprecating delete with extreme prejudice.
 And BTW, is there any reason why this can't be implemented in the library
 instead of using an operator? Why don't you provide a "destroy()" function
 for that in Phobos?

That sounds great.
 Really, I can't see any advantages on changing the delete operator
 semantics, only problems.

I agree.
 If you only want to
 "deinitialize" an object, you can write a .destroy() method for example,
 and call that. I think delete have a strong established semantic to change
 it now, and without any gain.

forward to divorce it of that.

Why it's broken? Why it's undesired?

(See above in this message.)
 Why? Using malloc and free is a lot more trouble, you have to register the
 roots yourself for example. It's not like you do malloc() and free() and
 everything works magically. You have to have more knowledge of the GC to
 use them. Being able to manually manage the *GC* heap (if the GC support
 that, if not it can make it a NOP) is good IMHO.

We can make things a tad better with library functions, but we do need to have a garbage collected heap that guarantees safety. Andrei
Oct 07 2009
parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 It is a bad idea because distinguishing between release of (expensive)
 resources from dangerous memory recycling is the correct way to obtain
 deterministic resource management within the confines of safety.

This is based on two faulty assumptions: 1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)
Oct 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 It is a bad idea because distinguishing between release of (expensive)
 resources from dangerous memory recycling is the correct way to obtain
 deterministic resource management within the confines of safety.

This is based on two faulty assumptions: 1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)

malloc. Andrei
Oct 07 2009
next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 It is a bad idea because distinguishing between release of (expensive)
 resources from dangerous memory recycling is the correct way to obtain
 deterministic resource management within the confines of safety.

This is based on two faulty assumptions: 1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)


So for placement construction of a class, I guess it would look something like: auto x = cast(MyClass) malloc(MyClass.classinfo.init.length); x.__ctor( a, b, c ); // construct ... x.__dtor(); free( cast(void*) x ); Is that right?
Oct 07 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Sean Kelly wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 It is a bad idea because distinguishing between release of (expensive)
 resources from dangerous memory recycling is the correct way to obtain
 deterministic resource management within the confines of safety.

1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)


So for placement construction of a class, I guess it would look something like: auto x = cast(MyClass) malloc(MyClass.classinfo.init.length); x.__ctor( a, b, c ); // construct ... x.__dtor(); free( cast(void*) x ); Is that right?

Yes, I think so, but I haven't checked all the details. For example I'm not sure whether __ctor copies .init over the memory before running the user-defined constructor, or expects that to have been done already. My understanding from Walter is that __ctor(x, y, z) are simply the functions this(x, y, z) as written by the user, so you'd need to memcpy the .init by hand before calling __ctor. Aw hell I got curious so let me check. class MyClass { int x = 42; this() {} } void main() { auto x = cast(MyClass) malloc(MyClass.classinfo.init.length); x.__ctor(); writeln(x.x); writeln(x.toString); } That prints 0 and then crashes on my machine. Looks like you need to memcpy the .init before calling __ctor. I'm very glad we're starting to look into this. There are very nice opportunities for adding custom allocation support in the stdlib. Andrei
Oct 07 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Leandro Lucarella wrote:
 Andrei Alexandrescu, el  7 de octubre a las 14:16 me escribiste:
 Sean Kelly wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 It is a bad idea because distinguishing between release of (expensive)
 resources from dangerous memory recycling is the correct way to obtain
 deterministic resource management within the confines of safety.

1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)


auto x = cast(MyClass) malloc(MyClass.classinfo.init.length); x.__ctor( a, b, c ); // construct ... x.__dtor(); free( cast(void*) x ); Is that right?

I'm not sure whether __ctor copies .init over the memory before running the user-defined constructor, or expects that to have been done already. My understanding from Walter is that __ctor(x, y, z) are simply the functions this(x, y, z) as written by the user, so you'd need to memcpy the .init by hand before calling __ctor.

What I don't understand is why you're willing to make that hard to do manual memory management in D. Do you see that you're making the programmer's job deliberately for no reason? D needs conservative GC, which means slow GC; by definition. D is a system programming language, so it's expected to be fast, but because of the GC there will be often situations where you have to do manual MM. Why are you making that much harder? You know that in the search for safety you'll be making much more unsafe (or bug-prone) to do manual MM?

You seem to be asserting that without additional built-in language support, manual memory management is unduly difficult. Why so? Andrei
Oct 07 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Leandro Lucarella wrote:
 Andrei Alexandrescu, el  7 de octubre a las 15:23 me escribiste:
 You seem to be asserting that without additional built-in language
 support, manual memory management is unduly difficult. Why so?

Because of this:
 auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
 x.__ctor( a, b, c ); // construct
 ...
 x.__dtor();
 free( cast(void*) x );




:) You even forgot to register your object as a root in the GC, so if your MyClass has any pointers to the GC your program will blow in your face. If you plan to library support to ease this and avoid repetitive and bug-prone work, you can ignore my complains...

I too think it would be great to add the necessary support to the stdlib. In fact, since you have a great deal of expertise in the matter, feel free to suggest API functions! They'd need to be approved by Sean too because probably they belong to druntime. Andrei
Oct 07 2009
prev sibling parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Leandro Lucarella wrote:
 Andrei Alexandrescu, el  7 de octubre a las 15:23 me escribiste:
 Leandro Lucarella wrote:
 Andrei Alexandrescu, el  7 de octubre a las 14:16 me escribiste:
 Sean Kelly wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 It is a bad idea because distinguishing between release of (expensive)
 resources from dangerous memory recycling is the correct way to obtain
 deterministic resource management within the confines of safety.

1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)


auto x = cast(MyClass) malloc(MyClass.classinfo.init.length); x.__ctor( a, b, c ); // construct ... x.__dtor(); free( cast(void*) x ); Is that right?

I'm not sure whether __ctor copies .init over the memory before running the user-defined constructor, or expects that to have been done already. My understanding from Walter is that __ctor(x, y, z) are simply the functions this(x, y, z) as written by the user, so you'd need to memcpy the .init by hand before calling __ctor.

manual memory management in D. Do you see that you're making the programmer's job deliberately for no reason? D needs conservative GC, which means slow GC; by definition. D is a system programming language, so it's expected to be fast, but because of the GC there will be often situations where you have to do manual MM. Why are you making that much harder? You know that in the search for safety you'll be making much more unsafe (or bug-prone) to do manual MM?

support, manual memory management is unduly difficult. Why so?

Because of this:
 auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
 x.__ctor( a, b, c ); // construct
 ...
 x.__dtor();
 free( cast(void*) x );




:) You even forgot to register your object as a root in the GC, so if your MyClass has any pointers to the GC your program will blow in your face. If you plan to library support to ease this and avoid repetitive and bug-prone work, you can ignore my complains...

I agree, that manual allocation code looks plain ugly. Why should we remove delete from the language anyways? If its unsafe then don't use it and your memory will get collected someday. If you need to reclaim memory right away, or you're doing manual memory management then its there, just like the original spec said. We can't always just let the GC collect everything, its most useful for memory that travels around a lot like strings and whatnot, but it's definitely too slow and too memory hungry for performance code. From the testing I did the D garbage collector is very fast to allocate memory, but *very* slow to reclaim it by mark&sweep, yet reclaim by delete is as fast as allocating it. The argument that it may be safer doesn't count, cause you can just not call delete and have safe code. I vote to keep delete in D.
Oct 07 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jeremie Pelletier wrote:
[snip]
 I vote to keep delete in D.

But your argument builds evidence for a deallocation function, not a keyword plus a smorgasbord of language support. Andrei
Oct 07 2009
prev sibling next sibling parent reply grauzone <none example.net> writes:
Andrei Alexandrescu wrote:
 Sean Kelly wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s 
 article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s 
 article
 It is a bad idea because distinguishing between release of (expensive)
 resources from dangerous memory recycling is the correct way to obtain
 deterministic resource management within the confines of safety.

1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)


So for placement construction of a class, I guess it would look something like: auto x = cast(MyClass) malloc(MyClass.classinfo.init.length); x.__ctor( a, b, c ); // construct ... x.__dtor(); free( cast(void*) x ); Is that right?

Yes, I think so, but I haven't checked all the details. For example I'm not sure whether __ctor copies .init over the memory before running the user-defined constructor, or expects that to have been done already.

Apparently it doesn't: http://www.digitalmars.com/techtips/class_objects.html See, it's even documented. Anyway, does your statement mean that _ctor is officially supported (by all conform D compilers)? Because, quoting from the page above: "This technique goes "under the hood" of how D works, and as such it is not guaranteed to work with every D compiler. In particular, how the constructors and destructors are called is not necessarilly portable."
Oct 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
grauzone wrote:
 Andrei Alexandrescu wrote:
 Sean Kelly wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s 
 article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s 
 article
 It is a bad idea because distinguishing between release of 
 (expensive)
 resources from dangerous memory recycling is the correct way to 
 obtain
 deterministic resource management within the confines of safety.

1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)


So for placement construction of a class, I guess it would look something like: auto x = cast(MyClass) malloc(MyClass.classinfo.init.length); x.__ctor( a, b, c ); // construct ... x.__dtor(); free( cast(void*) x ); Is that right?

Yes, I think so, but I haven't checked all the details. For example I'm not sure whether __ctor copies .init over the memory before running the user-defined constructor, or expects that to have been done already.

Apparently it doesn't: http://www.digitalmars.com/techtips/class_objects.html See, it's even documented. Anyway, does your statement mean that _ctor is officially supported (by all conform D compilers)? Because, quoting from the page above: "This technique goes "under the hood" of how D works, and as such it is not guaranteed to work with every D compiler. In particular, how the constructors and destructors are called is not necessarilly portable."

That technique will be used by a library function. Andrei
Oct 07 2009
parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
Andrei Alexandrescu wrote:
 grauzone wrote:
 Andrei Alexandrescu wrote:
 Sean Kelly wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s 
 article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu 
 (SeeWebsiteForEmail erdani.org)'s article
 It is a bad idea because distinguishing between release of 
 (expensive)
 resources from dangerous memory recycling is the correct way to 
 obtain
 deterministic resource management within the confines of safety.

1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)


So for placement construction of a class, I guess it would look something like: auto x = cast(MyClass) malloc(MyClass.classinfo.init.length); x.__ctor( a, b, c ); // construct ... x.__dtor(); free( cast(void*) x ); Is that right?

Yes, I think so, but I haven't checked all the details. For example I'm not sure whether __ctor copies .init over the memory before running the user-defined constructor, or expects that to have been done already.

Apparently it doesn't: http://www.digitalmars.com/techtips/class_objects.html See, it's even documented. Anyway, does your statement mean that _ctor is officially supported (by all conform D compilers)? Because, quoting from the page above: "This technique goes "under the hood" of how D works, and as such it is not guaranteed to work with every D compiler. In particular, how the constructors and destructors are called is not necessarilly portable."

That technique will be used by a library function.

So... the library will be related somehow to the implementing compiler?
Oct 07 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Ary Borenszweig wrote:
 Andrei Alexandrescu wrote:
 grauzone wrote:
 Andrei Alexandrescu wrote:
 Sean Kelly wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s 
 article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu 
 (SeeWebsiteForEmail erdani.org)'s article
 It is a bad idea because distinguishing between release of 
 (expensive)
 resources from dangerous memory recycling is the correct way to 
 obtain
 deterministic resource management within the confines of safety.

1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)


So for placement construction of a class, I guess it would look something like: auto x = cast(MyClass) malloc(MyClass.classinfo.init.length); x.__ctor( a, b, c ); // construct ... x.__dtor(); free( cast(void*) x ); Is that right?

Yes, I think so, but I haven't checked all the details. For example I'm not sure whether __ctor copies .init over the memory before running the user-defined constructor, or expects that to have been done already.

Apparently it doesn't: http://www.digitalmars.com/techtips/class_objects.html See, it's even documented. Anyway, does your statement mean that _ctor is officially supported (by all conform D compilers)? Because, quoting from the page above: "This technique goes "under the hood" of how D works, and as such it is not guaranteed to work with every D compiler. In particular, how the constructors and destructors are called is not necessarilly portable."

That technique will be used by a library function.

So... the library will be related somehow to the implementing compiler?

I'd believe so! Andrei
Oct 07 2009
prev sibling parent =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench robertmuench.de> writes:
Picking up this old topic & state.

What's the solution at the moment? I'm a bit lost. Is there now a way 
to use / make custom alloators with D2 or not?

If yes, how to do it?


On 2009-10-07 21:16:37 +0200, Andrei Alexandrescu said:

 So for placement construction of a class, I guess it would look something like:
 
 auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
 x.__ctor( a, b, c ); // construct
 ...
 x.__dtor();
 free( cast(void*) x );
 
 Is that right?

Yes, I think so, but I haven't checked all the details. For example I'm not sure whether __ctor copies .init over the memory before running the user-defined constructor, or expects that to have been done already. My understanding from Walter is that __ctor(x, y, z) are simply the functions this(x, y, z) as written by the user, so you'd need to memcpy the .init by hand before calling __ctor. Aw hell I got curious so let me check. class MyClass { int x = 42; this() {} } void main() { auto x = cast(MyClass) malloc(MyClass.classinfo.init.length); x.__ctor(); writeln(x.x); writeln(x.toString); } That prints 0 and then crashes on my machine. Looks like you need to memcpy the .init before calling __ctor. I'm very glad we're starting to look into this. There are very nice opportunities for adding custom allocation support in the stdlib.

-- Robert M. Mnch http://www.robertmuench.de
Jun 11 2010
prev sibling next sibling parent Don <nospam nospam.com> writes:
Steven Schveighoffer wrote:
 On Thu, 08 Oct 2009 07:26:37 -0400, Denis Koroskin <2korden gmail.com> 
 wrote:
 
 On Thu, 08 Oct 2009 14:48:19 +0400, Steven Schveighoffer 
 <schveiguy yahoo.com> wrote:

 On Wed, 07 Oct 2009 17:54:35 -0400, Denis Koroskin 
 <2korden gmail.com> wrote:

 On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly 
 <sean invisibleduck.org> wrote:

 auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);

I would expect a dynamic cast to occur at this line. Which will either result in an access violation (since you are trying to cast a garbage to an object) or result in a null being returned.

malloc returns void *, so no dynamic cast. -Steve

I know malloc returns void*. I didn't know you can hijack type system that easily. But then, if no dynamic cast takes place why cast(Object)cast(void*)0 cannot be evaluated at compile time?

Your message made me test it :) import std.stdio; void *foo() { return cast(void*)0; } void main() { auto o = cast(Object)foo(); writefln("here!"); o.opEquals(o); } outputs: here! Segmentation fault So, no dynamic cast (dynamic cast would have looked at the classinfo of null, segfaulting before the output). So I would say, the fact that compile time evaluation doesn't work is a bug maybe? -Steve

CTFE is not yet supported for classes.
Oct 08 2009
prev sibling parent Max Samukha <spambox d-coding.com> writes:
On Thu, 08 Oct 2009 16:31:00 +0400, "Denis Koroskin"
<2korden gmail.com> wrote:
But you are right, casting void* to Object does a reinterpret cast instead
of dynamic cast. I'm not sure if that's a good design decision, though.

It is needed in situations where the cast is known to be safe. There is also the undocumented _d_toObject(void*). What does it do exactly?
Oct 08 2009
prev sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 It is a bad idea because distinguishing between release of (expensive)
 resources from dangerous memory recycling is the correct way to obtain
 deterministic resource management within the confines of safety.

This is based on two faulty assumptions: 1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)

Andrei

Kludge. Requires using two separate heaps (inefficient) and worrying about whether your stuff is manually freed on all code paths, not just the ones that are executed often enough for performance to matter.
Oct 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 It is a bad idea because distinguishing between release of (expensive)
 resources from dangerous memory recycling is the correct way to obtain
 deterministic resource management within the confines of safety.

1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)

Andrei

Kludge. Requires using two separate heaps (inefficient) and worrying about whether your stuff is manually freed on all code paths, not just the ones that are executed often enough for performance to matter.

Au contraire, once the GC heap becomes safe, I have less to worry about. Andrei
Oct 07 2009
parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 It is a bad idea because distinguishing between release of (expensive)
 resources from dangerous memory recycling is the correct way to obtain
 deterministic resource management within the confines of safety.

1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a




 tradeoff to add a few manual delete statements to code and sacrifice some
safety
 for making the GC run less often.)

Andrei

Kludge. Requires using two separate heaps (inefficient) and worrying about whether your stuff is manually freed on all code paths, not just the ones that are executed often enough for performance to matter.

Andrei

If you're that concerned about making the GC heap safe, here's a less destructive (to other people's programming styles) way to do it: 1. Make delete only call the d'tor and not release memory. (I'm fine with this provided the stuff below is done.) 2. Add a std. lib convenience function to core.memory that does what delete does now (calls d'tor AND frees memory). For the purposes of this discussion, we'll call it deleteFree(). There's already a std. lib. function that just frees memory, GC.free(). Keep it. 3. If you really insist on absolute heap safety even at the expense of performance, grep your code and get rid of all deleteFree() and GC.free() calls. Frankly, I consider the ability to manually free GC allocated memory to be a HUGE asset for the following reasons, which I've mentioned before but would like to distill: 1. GC is usually the best way to program, but can be a huge bottleneck in some corner cases. 2. Maintaining two separate heaps (the manually memory managed C heap and the GC'd D heap) is a massive and completely unacceptable kludge because: 1. If you just want to delete a few objects to make the GC run less often, you can just add delete statements for the common code paths, or paths where the end of an object's lifetime is obvious. You then just let the GC handle the less common code paths or cases where object lifetimes are non-trivial and gain tons of simplicity for only a small performance loss. If you have to handle all the odd code paths manually too, this is when bugs really start to seep in. 2. Heaps have overhead. Two heaps have twice the overhead. 3. addroot(), etc. is a PITA *and* adds yet another place where you have to lock on the GC mutex. Half the need for manual memory management in D is because the GC sometimes scales poorly to large numbers of threads. This would definitely not help the situation. 4. Using the C heap whenever you want the ability to manually free something doesn't play nicely w/ builtin language features such as classes, arrays, associative arrays, etc., or objects returned from library functions. Because of these 4 issues, I feel that only being allowed to do manual memory management if you use the C heap is such an unacceptably bad kludge that it is for many practical purposes akin to not being allowed to do manual memory management at all. This is unacceptable in a systems/performance language. Remember, performance/systems languages can't place excessive emphasis on safety and absolutely MUST assume the programmer knows what he/she is doing. If you want Java, you know where to find it.
Oct 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 2.  Maintaining two separate heaps (the manually memory managed C heap and the
 GC'd D heap) is a massive and completely unacceptable kludge because:

Coding in a way that requires the GC to offer manual deletion is a completely unacceptable kludge. Most GCs could NOT offer a primitive to manually release memory. Designing D around a requirement that manual deletions work on the GC is crippling pressure on GC designers.
 1.  If you just want to delete a few objects to make the GC run less often, you
 can just add delete statements for the common code paths, or paths where the
end
 of an object's lifetime is obvious.  You then just let the GC handle the less
 common code paths or cases where object lifetimes are non-trivial and gain
tons of
 simplicity for only a small performance loss.  If you have to handle all the
odd
 code paths manually too, this is when bugs really start to seep in.

Many people's famous last void were "the end of an object's lifetime is obvious".
 2.  Heaps have overhead.  Two heaps have twice the overhead.

Where did that come from?
 3.  addroot(), etc. is a PITA *and* adds yet another place where you have to
lock
 on the GC mutex.  Half the need for manual memory management in D is because
the
 GC sometimes scales poorly to large numbers of threads.  This would definitely
not
 help the situation.

So right now do you have it for free? I don't understand. What are you comparing against what?
 4.  Using the C heap whenever you want the ability to manually free something
 doesn't play nicely w/ builtin language features such as classes, arrays,
 associative arrays, etc., or objects returned from library functions.

It shouldn't too.
 Because of these 4 issues, I feel that only being allowed to do manual memory
 management if you use the C heap is such an unacceptably bad kludge that it is
for
 many practical purposes akin to not being allowed to do manual memory
management
 at all.  This is unacceptable in a systems/performance language.

I completely disagree. I believe that "unifying" safe and unsafe styles under the same umbrella is an unacceptably bad kludge that is for many practical purposes akin to not being allowed to provide the slightest guarantee about any piece of D code. That's not where D should be going.
 Remember, performance/systems languages can't place excessive emphasis on
safety
 and absolutely MUST assume the programmer knows what he/she is doing.  If you
want
 Java, you know where to find it.

We agree on D being able to provide every bit of performance control needed. But my understanding you foster a programming style that's a hodge-podge of safe and unsafe coding under the same syntactic pretense. I don't think that's a good way to go. Andrei
Oct 07 2009
parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 2.  Maintaining two separate heaps (the manually memory managed C heap and the
 GC'd D heap) is a massive and completely unacceptable kludge because:

completely unacceptable kludge. Most GCs could NOT offer a primitive to manually release memory. Designing D around a requirement that manual deletions work on the GC is crippling pressure on GC designers.

Ok, fine, you got me on one point: Manual freeing of objects only makes sense in certain GC implementations. So what? GC.free() can be defined by the runtime implementation. If you're using something like pointer bump allocation with generational, moving GC, the implementation is free to do nothing. If you're using conservative mark/sweep, it should actually free memory.
Oct 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 2.  Maintaining two separate heaps (the manually memory managed C heap and the
 GC'd D heap) is a massive and completely unacceptable kludge because:

completely unacceptable kludge. Most GCs could NOT offer a primitive to manually release memory. Designing D around a requirement that manual deletions work on the GC is crippling pressure on GC designers.

Ok, fine, you got me on one point: Manual freeing of objects only makes sense in certain GC implementations. So what? GC.free() can be defined by the runtime implementation. If you're using something like pointer bump allocation with generational, moving GC, the implementation is free to do nothing. If you're using conservative mark/sweep, it should actually free memory.

I think there is convergence! My larger point is that we can leave GC.free() with loose semantics (e.g. may or may not act on it), and that we need to remove class-level allocators and probably the delete keyword too. Andrei
Oct 07 2009
next sibling parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 2.  Maintaining two separate heaps (the manually memory managed C heap and the
 GC'd D heap) is a massive and completely unacceptable kludge because:

completely unacceptable kludge. Most GCs could NOT offer a primitive to manually release memory. Designing D around a requirement that manual deletions work on the GC is crippling pressure on GC designers.

Ok, fine, you got me on one point: Manual freeing of objects only makes sense in certain GC implementations. So what? GC.free() can be defined by the runtime implementation. If you're using something like pointer bump allocation with generational, moving GC, the implementation is free to do nothing. If you're using conservative mark/sweep, it should actually free memory.

GC.free() with loose semantics (e.g. may or may not act on it), and that we need to remove class-level allocators and probably the delete keyword too. Andrei

Perfect. I'd be happy with this proposal as long as noone makes it harder to manually free GC-allocated memory while the GC implementation is still conservative mark-sweep or something similar. I had been under the impression that you wanted to flat-out get rid of GC.free(). Making it implementation defined but requiring that it at least exist even if it does nothing makes perfect sense. If the implementation changes to some better algorithm (not likely in the short term, but fairly likely in the long run), then my whole rationale for wanting to free stuff manually in the first place may change.
Oct 07 2009
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 2.  Maintaining two separate heaps (the manually memory managed C heap and the
 GC'd D heap) is a massive and completely unacceptable kludge because:

completely unacceptable kludge. Most GCs could NOT offer a primitive to manually release memory. Designing D around a requirement that manual deletions work on the GC is crippling pressure on GC designers.

Ok, fine, you got me on one point: Manual freeing of objects only makes sense in certain GC implementations. So what? GC.free() can be defined by the runtime implementation. If you're using something like pointer bump allocation with generational, moving GC, the implementation is free to do nothing. If you're using conservative mark/sweep, it should actually free memory.

GC.free() with loose semantics (e.g. may or may not act on it), and that we need to remove class-level allocators and probably the delete keyword too.

The docs for GC.free() should already state that what actually happens is implementation-defined. If they don't it's an oversight on my part. I do agree that the presence of "delete" in D is a bit weird, and would be happy to see it replaced by a library routine. new as well.
Oct 07 2009
prev sibling next sibling parent Christopher Wright <dhasenan gmail.com> writes:
Andrei Alexandrescu wrote:
 Christopher Wright wrote:
 What exactly is your suggestion?

 It seems that you mean that:
 delete obj;

 should call a destructor but not call delete() or notify the GC that 
 the memory is free.

That is correct. In particular, an object remains usable after delete.
 You're saying that there is a problem, but you're not telling us 
 what's wrong. Why the hell do you want to destroy an object without 
 recycling its memory? Why does the inability to do so cause a problem?

The matter has been discussed quite a bit around here and in other places. I'm not having as much time as I'd want to explain things. In short, destroying without freeing memory avoids dangling references and preserves memory safety without impacting on other resources.

Memory safety, sure, but you're deleting the object. It is no longer valid. You need to add a flag to the object indicating it's invalid, and everything that uses it needs to check that flag. Instead of a probable segfault in the current system, you'll get strange errors. It sounds like a complicated way of supporting a rare use case. Why not use a library solution? Make an IDisposable interface with methods "void dispose()" and "bool disposed()"? If you don't have enough time to explain the reasoning, could you post a link to a more detailed explanation?
Oct 07 2009
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2009-10-06 20:26:48 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:

 The matter has been discussed quite a bit around here and in other 
 places. I'm not having as much time as I'd want to explain things. In 
 short, destroying without freeing memory avoids dangling references and 
 preserves memory safety without impacting on other resources.
 
 It's a safety hack, not a performance hack.

In my opinion, it's mostly an illusion of safety. If you call the destructor on an object, the object state after the call doesn't necessarily respects the object invariants and doing anything with it could result in, well, anything, from returning wrong results to falling into an infinite loop (basically undefined behaviour). What you gain is that no object will be allocated on top of the old one, and thus new objects can't get corrupted. But it's still undefined behaviour, only with less side effects and more memory consumption. I don't think it's a so bad idea on the whole, but it'd be more valuable if accessing an invalidated object could be made an error instead of undefined behaviour. If this can't be done, then we should encourage "destructors" to put the object in a clean state and not leave any dirt behind. But should that still be called a "destructor"? Perhaps we could change the paradigm a little and replace "deletion" with "recycling". Recycling an object would call the destructor and immeditately call the default constructor, so the object is never left in an invalid state. Objects with no default constructor cannot be recycled. This way you know memory is always left in a clean state, and you encourage programmers to safely reuse the memory blocks from objects they have already allocated when possible. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Oct 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Michel Fortin wrote:
 On 2009-10-06 20:26:48 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> said:
 
 The matter has been discussed quite a bit around here and in other 
 places. I'm not having as much time as I'd want to explain things. In 
 short, destroying without freeing memory avoids dangling references 
 and preserves memory safety without impacting on other resources.

 It's a safety hack, not a performance hack.

In my opinion, it's mostly an illusion of safety. If you call the destructor on an object, the object state after the call doesn't necessarily respects the object invariants and doing anything with it could result in, well, anything, from returning wrong results to falling into an infinite loop (basically undefined behaviour). What you gain is that no object will be allocated on top of the old one, and thus new objects can't get corrupted. But it's still undefined behaviour, only with less side effects and more memory consumption. I don't think it's a so bad idea on the whole, but it'd be more valuable if accessing an invalidated object could be made an error instead of undefined behaviour. If this can't be done, then we should encourage "destructors" to put the object in a clean state and not leave any dirt behind. But should that still be called a "destructor"? Perhaps we could change the paradigm a little and replace "deletion" with "recycling". Recycling an object would call the destructor and immeditately call the default constructor, so the object is never left in an invalid state. Objects with no default constructor cannot be recycled. This way you know memory is always left in a clean state, and you encourage programmers to safely reuse the memory blocks from objects they have already allocated when possible.

Yes, recycling is best and I'm considering it. I'm only worried about the extra cost. Andrei
Oct 07 2009
next sibling parent reply Craig Black <cblack ara.com> writes:
Andrei Alexandrescu Wrote:

 Michel Fortin wrote:
 On 2009-10-06 20:26:48 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> said:
 
 The matter has been discussed quite a bit around here and in other 
 places. I'm not having as much time as I'd want to explain things. In 
 short, destroying without freeing memory avoids dangling references 
 and preserves memory safety without impacting on other resources.

 It's a safety hack, not a performance hack.

In my opinion, it's mostly an illusion of safety. If you call the destructor on an object, the object state after the call doesn't necessarily respects the object invariants and doing anything with it could result in, well, anything, from returning wrong results to falling into an infinite loop (basically undefined behaviour). What you gain is that no object will be allocated on top of the old one, and thus new objects can't get corrupted. But it's still undefined behaviour, only with less side effects and more memory consumption. I don't think it's a so bad idea on the whole, but it'd be more valuable if accessing an invalidated object could be made an error instead of undefined behaviour. If this can't be done, then we should encourage "destructors" to put the object in a clean state and not leave any dirt behind. But should that still be called a "destructor"? Perhaps we could change the paradigm a little and replace "deletion" with "recycling". Recycling an object would call the destructor and immeditately call the default constructor, so the object is never left in an invalid state. Objects with no default constructor cannot be recycled. This way you know memory is always left in a clean state, and you encourage programmers to safely reuse the memory blocks from objects they have already allocated when possible.

Yes, recycling is best and I'm considering it. I'm only worried about the extra cost. Andrei

No this is a bad idea. Removing the possibility to delete data will cause serious problems with heap fragmentation in some programs. -Craig
Oct 07 2009
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2009-10-07 17:53:21 -0400, Craig Black <cblack ara.com> said:

 Yes, recycling is best and I'm considering it. I'm only worried about
 the extra cost.
 
 Andrei

No this is a bad idea. Removing the possibility to delete data will cause serious problems with heap fragmentation in some programs.

Hum, perhaps we need to review more thoroughly how memory allocation works. As Andrei said himself, we now have all the necessary parts in the language to reimplement 'new' as a library function. So let's say we ditch 'new' and 'delete' as keywords. Let's first replace the keyword 'new' with a static function of the same name in a class or a struct. It could be implemented this way: static T new(A...)(A a) { T t = GC.alloc!T(); // GC.alloc sets the T.init bits. t.__ctor(a); return t; } Usage: Foo foo = Foo.new(); That's a static function template that needs to be reimplemented for every subclass (Andrei already proposed such kind of mixins) and that returns a garbage-collected object reference. Now, if you want manual allocation: static T new(A...)(A a) { T t = GC.allocNoCollect!T(); // GC won't collect this bit. t.__ctor(a); return t; } void dispose() { this.__dtor(); GC.free(this); } Usage: Foo foo = Foo.new(); ... foo.dispose(); But then you could do much better: 'new' could return a different type: a smart reference-counted pointer struct for instance. The possibilities are endless. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Oct 07 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Michel Fortin wrote:
 On 2009-10-07 17:53:21 -0400, Craig Black <cblack ara.com> said:
 
 Yes, recycling is best and I'm considering it. I'm only worried about
 the extra cost.

 Andrei

No this is a bad idea. Removing the possibility to delete data will cause serious problems with heap fragmentation in some programs.

Hum, perhaps we need to review more thoroughly how memory allocation works. As Andrei said himself, we now have all the necessary parts in the language to reimplement 'new' as a library function. So let's say we ditch 'new' and 'delete' as keywords. Let's first replace the keyword 'new' with a static function of the same name in a class or a struct. It could be implemented this way: static T new(A...)(A a) { T t = GC.alloc!T(); // GC.alloc sets the T.init bits. t.__ctor(a); return t; } Usage: Foo foo = Foo.new(); That's a static function template that needs to be reimplemented for every subclass (Andrei already proposed such kind of mixins) and that returns a garbage-collected object reference. Now, if you want manual allocation: static T new(A...)(A a) { T t = GC.allocNoCollect!T(); // GC won't collect this bit. t.__ctor(a); return t; } void dispose() { this.__dtor(); GC.free(this); } Usage: Foo foo = Foo.new(); ... foo.dispose(); But then you could do much better: 'new' could return a different type: a smart reference-counted pointer struct for instance. The possibilities are endless.

That's just awesome. Incidentally it would dovetail nicely with the code injection feature that I recently discussed here. But then that increases the size of the language... Andrei
Oct 07 2009
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-10-07 20:11:31 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:

 That's just awesome. Incidentally it would dovetail nicely with the 
 code injection feature that I recently discussed here.

Indeed. That's what gave me the idea. :-)
 But then that increases the size of the language...

Really? Remove new and delete; add code injection. Seems like a tie to me, except the later is much less limited and will solve problems well beyond memory allocation. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Oct 07 2009
prev sibling parent reply Chris Nicholson-Sauls <ibisbasenji gmail.com> writes:
Michel Fortin wrote:
 On 2009-10-07 17:53:21 -0400, Craig Black <cblack ara.com> said:
 
 Yes, recycling is best and I'm considering it. I'm only worried about
 the extra cost.

 Andrei

No this is a bad idea. Removing the possibility to delete data will cause serious problems with heap fragmentation in some programs.

Hum, perhaps we need to review more thoroughly how memory allocation works. As Andrei said himself, we now have all the necessary parts in the language to reimplement 'new' as a library function. So let's say we ditch 'new' and 'delete' as keywords. Let's first replace the keyword 'new' with a static function of the same name in a class or a struct. It could be implemented this way: static T new(A...)(A a) { T t = GC.alloc!T(); // GC.alloc sets the T.init bits. t.__ctor(a); return t; } Usage: Foo foo = Foo.new(); That's a static function template that needs to be reimplemented for every subclass (Andrei already proposed such kind of mixins) and that returns a garbage-collected object reference. Now, if you want manual allocation: static T new(A...)(A a) { T t = GC.allocNoCollect!T(); // GC won't collect this bit. t.__ctor(a); return t; } void dispose() { this.__dtor(); GC.free(this); } Usage: Foo foo = Foo.new(); ... foo.dispose(); But then you could do much better: 'new' could return a different type: a smart reference-counted pointer struct for instance. The possibilities are endless.

Prior to this post I'd been on the side of retaining "good ole" delete, owing somewhat to my own tendency to do Evil Things with overloaded new/delete, such as transparent free-lists. I've become neutral in light of the above proposed technique, because it really doesn't break that kind of usage. In fact, it technically makes it more reliable and more flexible since the behavior of these is more predictable (not subject to compiler quality/method-of-implementation, and guaranteed to be "just another function"). That said, the stdlib (or probably druntime) needs to provide good general-case support for this, which should include some sort of IDisposable interface (as mentioned repeatedly by others) otherwise we're jumping into the abyss (of massive repetitive coding) rather than over it (into the Elysian fields). One consideration is that new(), perhaps, ought not be a static member of its class at all, but rather a global written along similar lines to tools such as "to". Given that, one could write something like: ################################################## class C {...} C new (T:C, A...) (A a) { auto c = GC.alloc!T(); c.__ctor(a); return c; } auto somevar = new! C (1, 2, 3); // free-listed class F {...} F new (T:F, A...) (A a) { return F.List.length != 0 ? F.List.pop : defaultNew! F (a) ; } ################################################## The latter examples shows my thinking: that the stdlib/druntime could easily provide a default new() that does what the current new operator does. Class designers could then overload this default new() as needed. Provide a reasonable alias for the standard new() (I used "defaultNew" above, but its probably not the best) and it can still be used as backup in custom functions, such as in the free-list example. Incidentally... does anyone else notice that, in the static-new proposal, we've once again recreated Ruby? Proposed D2: auto foo = Foo.new; Ruby: foo = Foo.new At least mine looks more like current syntax: auto foo = new! Foo; -- Christopher Nicholson-Sauls
Oct 08 2009
next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-10-08 05:00:03 -0400, Chris Nicholson-Sauls 
<ibisbasenji gmail.com> said:

 class C {...}
 
 C new (T:C, A...) (A a) {
      auto c = GC.alloc!T();
      c.__ctor(a);
      return c;
 }
 
 auto somevar = new! C (1, 2, 3);

Nice idea, and it can already work... as long as your constructor is public (or you have private access from the module the template is defined in).
 // free-listed
 class F {...}
 
 F new (T:F, A...) (A a) {
      return F.List.length != 0
          ? F.List.pop
          : defaultNew! F (a)
      ;
 }
 
 The latter examples shows my thinking: that the stdlib/druntime could 
 easily provide a default new() that does what the current new operator 
 does. Class designers could then overload this default new() as needed. 
  Provide a reasonable alias for the standard new() (I used "defaultNew" 
 above, but its probably not the best) and it can still be used as 
 backup in custom functions, such as in the free-list example.

What about 'newGarbageCollected!F' (or 'newGC!F' for short)?
 Incidentally... does anyone else notice that, in the static-new 
 proposal, we've once again recreated Ruby?
 
 Proposed D2:
 auto foo = Foo.new;
 
 Ruby:
 foo = Foo.new

Ah! I knew I had seen this pattern somewhere. Personally, I had more in mind the object instanciation pattern in Objective-C: NSString *s = [[NSString alloc] init]; and decided to combine that alloc & init pair (GC.alloc & __ctor in D) into 'new'.
 At least mine looks more like current syntax:
 auto foo = new! Foo;

I'd call this a marginal gain, but a gain nonetheless. A bigger gain of 'new!Foo' over 'Foo.new' is that it lets users invent their own allocation method without having to change any class or struct. But it'd require some changes to how protection attributes are handled in templates, because right now it just won't work with a non-public contructor. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Oct 08 2009
prev sibling parent Jeremie Pelletier <jeremiep gmail.com> writes:
Adam D. Ruppe wrote:
 On Thu, Oct 08, 2009 at 04:00:03AM -0500, Chris Nicholson-Sauls wrote:
 One consideration is that new(), perhaps, ought not be a static member of 
 its class at all, but rather a global written along similar lines to tools 
 such as "to".

Agreed. One benefit here is we can convert old code to it just by find/ replacing new -> new!. We can provide easy freestanding functions for manual management too.

sed is even better in this case :)
 new! - gc
 manualNew! - malloc() wrapper
 
 Maybe even convenience structs too:
 
 RAII! - a struct that uses the malloc() wrapper

Well if new is a template, its dead easy to use static ifs to detect if you're allocating an array, a struct or an object and initialize the memory appropriately. The only thing that bugs me is that it makes it very hard to implement overridable new/delete methods for classes, if not impossible.
Oct 08 2009
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Denis Koroskin wrote:
 I'm not sure you will convince people to use foo.recycle() instead of 
 foo.delete(). Not only it's slower, I believe recycling an object works 
 for hiding bugs: accessing a recycled object - obviously a bug - will no 
 longer be detected.

Is anyone under the illusion that today there's any detection going on? Andrei
Oct 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Denis Koroskin wrote:
 On Thu, 08 Oct 2009 04:13:12 +0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:
 
 Denis Koroskin wrote:
 I'm not sure you will convince people to use foo.recycle() instead of 
 foo.delete(). Not only it's slower, I believe recycling an object 
 works for hiding bugs: accessing a recycled object - obviously a bug 
 - will no longer be detected.

Is anyone under the illusion that today there's any detection going on? Andrei

There is none, but it's possible. It's just not implemented.

It's not possible if you allow actual memory reuse! Now I'm not sure I understand what you want. Andrei
Oct 07 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Denis Koroskin wrote:
 On Thu, 08 Oct 2009 04:39:20 +0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:
 
 Denis Koroskin wrote:
 On Thu, 08 Oct 2009 04:13:12 +0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:

 Denis Koroskin wrote:
 I'm not sure you will convince people to use foo.recycle() instead 
 of foo.delete(). Not only it's slower, I believe recycling an 
 object works for hiding bugs: accessing a recycled object - 
 obviously a bug - will no longer be detected.

Is anyone under the illusion that today there's any detection going on? Andrei


It's not possible if you allow actual memory reuse! Now I'm not sure I understand what you want. Andrei

In our custom memory management system, deallocated memory gets filled with a debug data, which is checked for consistency when memory gets allocated again. Any write to that memory we be noticed. Not immediately, but still, it's better than nothing. Microsoft C++ debug runtime does the same. Under Windows (2000 and later) you can also mark a range of memory as not accessible (by calling VirtualProtect on that memory with a PAGE_NOACCESS flag). Any read/write attempt with cause an immediate access violation exception. This is not widely used, probably because it's slow, but when you have a memory damage (caused by modifying some memory via a dangling pointer) performance is of lesser importance. I believe similar mechanisms exist for nixes, too.

There are (anyway, page-level marking is not the right level of granularity). My overall point is twofold: 1. new and delete were symmetric in C++. In D they aren't and aren't supposed to be symmetric. The delete keyword should be deprecated and the functionality of delete should be relegated to a function. 2. Mostly as a consequence of (1), class-level operators new and delete are misdesigned and should be eliminated. Object factories/pools/regions/etc. should be the way to go for custom class allocation. Heck, others are shunning new and we're clinging on to it? Andrei
Oct 07 2009
prev sibling next sibling parent Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el  7 de octubre a las 14:16 me escribiste:
 Sean Kelly wrote:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
dsimcha wrote:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
It is a bad idea because distinguishing between release of (expensive)
resources from dangerous memory recycling is the correct way to obtain
deterministic resource management within the confines of safety.

1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)


So for placement construction of a class, I guess it would look something like: auto x = cast(MyClass) malloc(MyClass.classinfo.init.length); x.__ctor( a, b, c ); // construct ... x.__dtor(); free( cast(void*) x ); Is that right?

Yes, I think so, but I haven't checked all the details. For example I'm not sure whether __ctor copies .init over the memory before running the user-defined constructor, or expects that to have been done already. My understanding from Walter is that __ctor(x, y, z) are simply the functions this(x, y, z) as written by the user, so you'd need to memcpy the .init by hand before calling __ctor.

What I don't understand is why you're willing to make that hard to do manual memory management in D. Do you see that you're making the programmer's job deliberately for no reason? D needs conservative GC, which means slow GC; by definition. D is a system programming language, so it's expected to be fast, but because of the GC there will be often situations where you have to do manual MM. Why are you making that much harder? You know that in the search for safety you'll be making much more unsafe (or bug-prone) to do manual MM? -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- Ya ni el cielo me quiere, ya ni la muerte me visita Ya ni el sol me calienta, ya ni el viento me acaricia
Oct 07 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly <sean invisibleduck.org>  
wrote:

 auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);

I would expect a dynamic cast to occur at this line. Which will either result in an access violation (since you are trying to cast a garbage to an object) or result in a null being returned.
Oct 07 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 07 Oct 2009 21:55:42 +0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Michel Fortin wrote:
 On 2009-10-06 20:26:48 -0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> said:

 The matter has been discussed quite a bit around here and in other  
 places. I'm not having as much time as I'd want to explain things. In  
 short, destroying without freeing memory avoids dangling references  
 and preserves memory safety without impacting on other resources.

 It's a safety hack, not a performance hack.

destructor on an object, the object state after the call doesn't necessarily respects the object invariants and doing anything with it could result in, well, anything, from returning wrong results to falling into an infinite loop (basically undefined behaviour). What you gain is that no object will be allocated on top of the old one, and thus new objects can't get corrupted. But it's still undefined behaviour, only with less side effects and more memory consumption. I don't think it's a so bad idea on the whole, but it'd be more valuable if accessing an invalidated object could be made an error instead of undefined behaviour. If this can't be done, then we should encourage "destructors" to put the object in a clean state and not leave any dirt behind. But should that still be called a "destructor"? Perhaps we could change the paradigm a little and replace "deletion" with "recycling". Recycling an object would call the destructor and immeditately call the default constructor, so the object is never left in an invalid state. Objects with no default constructor cannot be recycled. This way you know memory is always left in a clean state, and you encourage programmers to safely reuse the memory blocks from objects they have already allocated when possible.

Yes, recycling is best and I'm considering it. I'm only worried about the extra cost. Andrei

I rarely use delete these days (certainly not as often as in my early D days, which is a good sign IMO), and I'm afraid I'll drop its use entirely if delete will be replaced with a recycle. I mostly manage memory manually as part of performance optimization. The change you are talking about contradicts with my goals of manual object destruction. You don't even need to change a language to support your semantics: template Recyclable() { final void recycle() { this.__dtor(); memcpy(this, classinfo.init.ptr, classinfo.init.length); // ctors are not virtual auto defaultCtor = (void delegate(Object))classinfo.defaultConstructor; ctor(this); } } class Foo { mixin Recyclable!(); int i = 42; this() { i = -1; } } Foo foo = new Foo(); foo.i = 0; foo.recycle(); writeln(foo.i); // -1 And even if the proposed change will occur, old behavior will still be accessible: template Deletable() { final void delete() { this.__dtor(); GC.free(this); } } I'm not sure you will convince people to use foo.recycle() instead of foo.delete(). Not only it's slower, I believe recycling an object works for hiding bugs: accessing a recycled object - obviously a bug - will no longer be detected.
Oct 07 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 08 Oct 2009 04:13:12 +0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Denis Koroskin wrote:
 I'm not sure you will convince people to use foo.recycle() instead of  
 foo.delete(). Not only it's slower, I believe recycling an object works  
 for hiding bugs: accessing a recycled object - obviously a bug - will  
 no longer be detected.

Is anyone under the illusion that today there's any detection going on? Andrei

There is none, but it's possible. It's just not implemented.
Oct 07 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 08 Oct 2009 04:39:20 +0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Denis Koroskin wrote:
 On Thu, 08 Oct 2009 04:13:12 +0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:

 Denis Koroskin wrote:
 I'm not sure you will convince people to use foo.recycle() instead of  
 foo.delete(). Not only it's slower, I believe recycling an object  
 works for hiding bugs: accessing a recycled object - obviously a bug  
 - will no longer be detected.

Is anyone under the illusion that today there's any detection going on? Andrei


It's not possible if you allow actual memory reuse! Now I'm not sure I understand what you want. Andrei

In our custom memory management system, deallocated memory gets filled with a debug data, which is checked for consistency when memory gets allocated again. Any write to that memory we be noticed. Not immediately, but still, it's better than nothing. Microsoft C++ debug runtime does the same. Under Windows (2000 and later) you can also mark a range of memory as not accessible (by calling VirtualProtect on that memory with a PAGE_NOACCESS flag). Any read/write attempt with cause an immediate access violation exception. This is not widely used, probably because it's slow, but when you have a memory damage (caused by modifying some memory via a dangling pointer) performance is of lesser importance. I believe similar mechanisms exist for nixes, too.
Oct 07 2009
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 07 Oct 2009 17:54:35 -0400, Denis Koroskin <2korden gmail.com>  
wrote:

 On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly <sean invisibleduck.org>  
 wrote:

 auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);

I would expect a dynamic cast to occur at this line. Which will either result in an access violation (since you are trying to cast a garbage to an object) or result in a null being returned.

malloc returns void *, so no dynamic cast. -Steve
Oct 08 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 08 Oct 2009 14:48:19 +0400, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 On Wed, 07 Oct 2009 17:54:35 -0400, Denis Koroskin <2korden gmail.com>  
 wrote:

 On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly <sean invisibleduck.org>  
 wrote:

 auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);

I would expect a dynamic cast to occur at this line. Which will either result in an access violation (since you are trying to cast a garbage to an object) or result in a null being returned.

malloc returns void *, so no dynamic cast. -Steve

I know malloc returns void*. I didn't know you can hijack type system that easily. But then, if no dynamic cast takes place why cast(Object)cast(void*)0 cannot be evaluated at compile time?
Oct 08 2009
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 08 Oct 2009 07:26:37 -0400, Denis Koroskin <2korden gmail.com>  
wrote:

 On Thu, 08 Oct 2009 14:48:19 +0400, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:

 On Wed, 07 Oct 2009 17:54:35 -0400, Denis Koroskin <2korden gmail.com>  
 wrote:

 On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly  
 <sean invisibleduck.org> wrote:

 auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);

I would expect a dynamic cast to occur at this line. Which will either result in an access violation (since you are trying to cast a garbage to an object) or result in a null being returned.

malloc returns void *, so no dynamic cast. -Steve

I know malloc returns void*. I didn't know you can hijack type system that easily. But then, if no dynamic cast takes place why cast(Object)cast(void*)0 cannot be evaluated at compile time?

Your message made me test it :) import std.stdio; void *foo() { return cast(void*)0; } void main() { auto o = cast(Object)foo(); writefln("here!"); o.opEquals(o); } outputs: here! Segmentation fault So, no dynamic cast (dynamic cast would have looked at the classinfo of null, segfaulting before the output). So I would say, the fact that compile time evaluation doesn't work is a bug maybe? -Steve
Oct 08 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 08 Oct 2009 15:48:56 +0400, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 On Thu, 08 Oct 2009 07:26:37 -0400, Denis Koroskin <2korden gmail.com>  
 wrote:

 On Thu, 08 Oct 2009 14:48:19 +0400, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:

 On Wed, 07 Oct 2009 17:54:35 -0400, Denis Koroskin <2korden gmail.com>  
 wrote:

 On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly  
 <sean invisibleduck.org> wrote:

 auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);

I would expect a dynamic cast to occur at this line. Which will either result in an access violation (since you are trying to cast a garbage to an object) or result in a null being returned.

malloc returns void *, so no dynamic cast. -Steve

I know malloc returns void*. I didn't know you can hijack type system that easily. But then, if no dynamic cast takes place why cast(Object)cast(void*)0 cannot be evaluated at compile time?

Your message made me test it :) import std.stdio; void *foo() { return cast(void*)0; } void main() { auto o = cast(Object)foo(); writefln("here!"); o.opEquals(o); } outputs: here! Segmentation fault So, no dynamic cast (dynamic cast would have looked at the classinfo of null, segfaulting before the output).

No, IIRC, casting null to Object is perfectly valid and returns null. But you are right, casting void* to Object does a reinterpret cast instead of dynamic cast. I'm not sure if that's a good design decision, though.
 So I would say, the fact that compile time evaluation doesn't work is a  
 bug maybe?

 -Steve

Probably. Not only it doesn't work at compile time, it doesn't work at all! void main() { auto o = cast(Object)cast(void*)0; // Error: cannot cast void* to object.Object }
Oct 08 2009
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 08 Oct 2009 08:31:00 -0400, Denis Koroskin <2korden gmail.com>  
wrote:

 On Thu, 08 Oct 2009 15:48:56 +0400, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:

 On Thu, 08 Oct 2009 07:26:37 -0400, Denis Koroskin <2korden gmail.com>  
 wrote:

 On Thu, 08 Oct 2009 14:48:19 +0400, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:

 On Wed, 07 Oct 2009 17:54:35 -0400, Denis Koroskin  
 <2korden gmail.com> wrote:

 On Wed, 07 Oct 2009 23:00:06 +0400, Sean Kelly  
 <sean invisibleduck.org> wrote:

 auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);

I would expect a dynamic cast to occur at this line. Which will either result in an access violation (since you are trying to cast a garbage to an object) or result in a null being returned.

malloc returns void *, so no dynamic cast. -Steve

I know malloc returns void*. I didn't know you can hijack type system that easily. But then, if no dynamic cast takes place why cast(Object)cast(void*)0 cannot be evaluated at compile time?

Your message made me test it :) import std.stdio; void *foo() { return cast(void*)0; } void main() { auto o = cast(Object)foo(); writefln("here!"); o.opEquals(o); } outputs: here! Segmentation fault So, no dynamic cast (dynamic cast would have looked at the classinfo of null, segfaulting before the output).

No, IIRC, casting null to Object is perfectly valid and returns null.

Oh yeah :) I forgot. Should have returned 1 instead of 0. I did also verify via obj2asm that dynamic cast was not being called :)
 But you are right, casting void* to Object does a reinterpret cast  
 instead
 of dynamic cast. I'm not sure if that's a good design decision, though.

It's the only sane decision. void * is really the developer's way of saying "I'll handle the typing from here, thanks" to the compiler. So the compiler can't expect to interpret void * as anything special. There's no deterministic way to detect an object anyways, so the compiler can't make any assumptions without the typesystem.
 So I would say, the fact that compile time evaluation doesn't work is a  
 bug maybe?

 -Steve

Probably. Not only it doesn't work at compile time, it doesn't work at all! void main() { auto o = cast(Object)cast(void*)0; // Error: cannot cast void* to object.Object }

Hm.. strange that my example compiles and yours does not. I'd think it to be the same thing. That definitely should be flagged as a bug. I used dmd 2.033. -Steve
Oct 08 2009
prev sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Thu, Oct 08, 2009 at 04:00:03AM -0500, Chris Nicholson-Sauls wrote:
 One consideration is that new(), perhaps, ought not be a static member of 
 its class at all, but rather a global written along similar lines to tools 
 such as "to".

Agreed. One benefit here is we can convert old code to it just by find/ replacing new -> new!. We can provide easy freestanding functions for manual management too. new! - gc manualNew! - malloc() wrapper Maybe even convenience structs too: RAII! - a struct that uses the malloc() wrapper -- Adam D. Ruppe http://arsdnet.net
Oct 08 2009
prev sibling next sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el  6 de octubre a las 11:01 me escribiste:
 Hello,
 
 
 D currently allows defining class allocators and deallocators. They
 have a number of problems that make them unsuitable for D 2.0. The
 most obvious issue is that D 2.0 will _not_ conflate destruction
 with deallocation anymore: invoking delete against an object will
 call ~this() against it but will not recycle its memory.

I don't think it is a good idea (GC-wise) to say that in the specs. I think the GC implementor should be free to decide if a delete really free the memory or not. Some collectors can do this very naturally (like the current one) and some others don't (like allocators that uses pointer-bump allocation). I think the language should divide destruction and deallocation, but I don't think is a good idea not to notify the GC at all when delete is used. I think the GC should be able to do whatever it feels is good for him (so the user should not rely either on the memory being actually freed or otherwise). -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- If you don't know what direction you should take You don't know where you are
Oct 06 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Leandro Lucarella wrote:
 Andrei Alexandrescu, el  6 de octubre a las 11:01 me escribiste:
 Hello,


 D currently allows defining class allocators and deallocators. They
 have a number of problems that make them unsuitable for D 2.0. The
 most obvious issue is that D 2.0 will _not_ conflate destruction
 with deallocation anymore: invoking delete against an object will
 call ~this() against it but will not recycle its memory.

I don't think it is a good idea (GC-wise) to say that in the specs. I think the GC implementor should be free to decide if a delete really free the memory or not. Some collectors can do this very naturally (like the current one) and some others don't (like allocators that uses pointer-bump allocation). I think the language should divide destruction and deallocation, but I don't think is a good idea not to notify the GC at all when delete is used. I think the GC should be able to do whatever it feels is good for him (so the user should not rely either on the memory being actually freed or otherwise).

I agree insofar as a GC could be tipped by the compiler that no live reference of the object exists after delete. Andrei
Oct 06 2009
prev sibling next sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el  6 de octubre a las 19:26 me escribiste:
 Christopher Wright wrote:
What exactly is your suggestion?

It seems that you mean that:
delete obj;

should call a destructor but not call delete() or notify the GC
that the memory is free.

That is correct. In particular, an object remains usable after delete.

Why would you do that? What is the rationale to not notify the GC?
You're saying that there is a problem, but you're not telling us
what's wrong. Why the hell do you want to destroy an object
without recycling its memory? Why does the inability to do so
cause a problem?

The matter has been discussed quite a bit around here and in other places. I'm not having as much time as I'd want to explain things. In short, destroying without freeing memory avoids dangling references and preserves memory safety without impacting on other resources.

But D is a system programming language. If you wrote delete x; the language should assume you know what you're doing. If you only want to "deinitialize" an object, you can write a .destroy() method for example, and call that. I think delete have a strong established semantic to change it now, and without any gain.
It seems like a performance hack to me -- you've got an object
that isn't valid anymore, but you want to hang on to the memory
for some other purpose. And you could override new() and delete(),
but you don't want to incur the performance penalty of calling the
runtime to fetch the deallocator.

It's a safety hack, not a performance hack.

But you shouldn't provide safety where the programmer is not expecting it. delete is for *manual* memory management. It makes no sense to guarantee that the memory is *not* freed. It makes sense not guaranteeing that it will actually be freed either. I think that's a good idea actually, because it gives more flexibility to the GC implementation.
The only remaining use that I see is a way to reset a shared
object without explicitly passing around a reference to the new
version of the object. This seems potentially dangerous, and
nothing I want for default behavior.

Well incidentally at least as of now "delete obj" puts null in obj...

That's nice :) -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- Never let a fool kiss you, or let a kiss fool you
Oct 06 2009
parent Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el  7 de octubre a las 15:23 me escribiste:
 Leandro Lucarella wrote:
Andrei Alexandrescu, el  7 de octubre a las 14:16 me escribiste:
Sean Kelly wrote:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
dsimcha wrote:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
It is a bad idea because distinguishing between release of (expensive)
resources from dangerous memory recycling is the correct way to obtain
deterministic resource management within the confines of safety.

1. Memory is cheap. (Not if you are working with absurd amounts of data). 2. Garbage collection is never a major bottleneck. (Sometimes it's a worthwhile tradeoff to add a few manual delete statements to code and sacrifice some safety for making the GC run less often.)


auto x = cast(MyClass) malloc(MyClass.classinfo.init.length); x.__ctor( a, b, c ); // construct ... x.__dtor(); free( cast(void*) x ); Is that right?

I'm not sure whether __ctor copies .init over the memory before running the user-defined constructor, or expects that to have been done already. My understanding from Walter is that __ctor(x, y, z) are simply the functions this(x, y, z) as written by the user, so you'd need to memcpy the .init by hand before calling __ctor.

What I don't understand is why you're willing to make that hard to do manual memory management in D. Do you see that you're making the programmer's job deliberately for no reason? D needs conservative GC, which means slow GC; by definition. D is a system programming language, so it's expected to be fast, but because of the GC there will be often situations where you have to do manual MM. Why are you making that much harder? You know that in the search for safety you'll be making much more unsafe (or bug-prone) to do manual MM?

You seem to be asserting that without additional built-in language support, manual memory management is unduly difficult. Why so?

Because of this:
auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );




:) You even forgot to register your object as a root in the GC, so if your MyClass has any pointers to the GC your program will blow in your face. If you plan to library support to ease this and avoid repetitive and bug-prone work, you can ignore my complains... -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- Y tuve amores, que fue uno sólo El que me dejó de a pie y me enseñó todo...
Oct 07 2009
prev sibling next sibling parent Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el  6 de octubre a las 21:36 me escribiste:
I don't think it is a good idea (GC-wise) to say that in the specs.
I think the GC implementor should be free to decide if a delete really
free the memory or not. Some collectors can do this very naturally (like
the current one) and some others don't (like allocators that uses
pointer-bump allocation). I think the language should divide destruction
and deallocation, but I don't think is a good idea not to notify the GC at
all when delete is used. I think the GC should be able to do whatever it
feels is good for him (so the user should not rely either on the memory
being actually freed or otherwise).

I agree insofar as a GC could be tipped by the compiler that no live reference of the object exists after delete.

Great! For example, this would let me protect the object pages (if it's a large object that uses one or more full pages) when they are freed so the program segfaults as soon as a deleted object is used when it shouldn't. That could be a nice debugging feature :) -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- Hey you, with you ear against the wall Waiting for someone to call out Would you touch me?
Oct 06 2009
prev sibling next sibling parent reply downs <default_357-line yahoo.de> writes:
Andrei Alexandrescu wrote:
 Hello,
 
 
 D currently allows defining class allocators and deallocators. They have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast, class
 deallocators are designed around the idea that invoking delete calls the
 destructor and also deallocates memory.
 
 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally useful if
 the user takes the matter of deallocation in her own hands.
 
 A much better way to handle custom allocation of classes would be in the
 standard library.
 
 What do you think?
 
 
 Andrei

Do you trust the D GC to be good enough to always free everything you've allocated, without error? If your answer was 'ye- maaybe ... no actually', please rethink this.
Oct 06 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
downs wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast, class
 deallocators are designed around the idea that invoking delete calls the
 destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally useful if
 the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be in the
 standard library.

 What do you think?


 Andrei

Do you trust the D GC to be good enough to always free everything you've allocated, without error? If your answer was 'ye- maaybe ... no actually', please rethink this.

People will always be able to call functions in the garbage collector manually. The discussion on class allocators and deallocators has nothing to do with that. Andrei
Oct 06 2009
next sibling parent reply downs <default_357-line yahoo.de> writes:
Andrei Alexandrescu wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast, class
 deallocators are designed around the idea that invoking delete calls the
 destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally useful if
 the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be in the
 standard library.

 What do you think?


 Andrei

Do you trust the D GC to be good enough to always free everything you've allocated, without error? If your answer was 'ye- maaybe ... no actually', please rethink this.

People will always be able to call functions in the garbage collector manually. The discussion on class allocators and deallocators has nothing to do with that. Andrei

So you can still deallocate a class by hand, only it's not called delete anymore?
Oct 06 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
downs wrote:
 Andrei Alexandrescu wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast, class
 deallocators are designed around the idea that invoking delete calls the
 destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally useful if
 the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be in the
 standard library.

 What do you think?


 Andrei

you've allocated, without error? If your answer was 'ye- maaybe ... no actually', please rethink this.

manually. The discussion on class allocators and deallocators has nothing to do with that. Andrei

So you can still deallocate a class by hand, only it's not called delete anymore?

That is correct. Andrei
Oct 07 2009
parent reply downs <default_357-line yahoo.de> writes:
Andrei Alexandrescu wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They
 have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast, class
 deallocators are designed around the idea that invoking delete
 calls the
 destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally
 useful if
 the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be
 in the
 standard library.

 What do you think?


 Andrei

you've allocated, without error? If your answer was 'ye- maaybe ... no actually', please rethink this.

manually. The discussion on class allocators and deallocators has nothing to do with that. Andrei

So you can still deallocate a class by hand, only it's not called delete anymore?

That is correct. Andrei

Isn't that a pretty big violation of Least Surprise? http://en.wikipedia.org/wiki/Principle_of_least_astonishment : "In user interface design, programming language design, and ergonomics, the principle (or rule or law) of least astonishment (or surprise) states that, when two elements of an interface conflict, or are ambiguous, the behaviour should be that which will *least surprise* the human user or programmer at the time the conflict arises."
Oct 07 2009
parent reply Don <nospam nospam.com> writes:
downs wrote:
 Andrei Alexandrescu wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They
 have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast, class
 deallocators are designed around the idea that invoking delete
 calls the
 destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally
 useful if
 the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be
 in the
 standard library.

 What do you think?


 Andrei

you've allocated, without error? If your answer was 'ye- maaybe ... no actually', please rethink this.

manually. The discussion on class allocators and deallocators has nothing to do with that. Andrei

delete anymore?

Andrei

Isn't that a pretty big violation of Least Surprise? http://en.wikipedia.org/wiki/Principle_of_least_astonishment : "In user interface design, programming language design, and ergonomics, the principle (or rule or law) of least astonishment (or surprise) states that, when two elements of an interface conflict, or are ambiguous, the behaviour should be that which will *least surprise* the human user or programmer at the time the conflict arises."

I think the basic rule being introduced is: that every object can be managed by the gc, or manually managed. But not both. That seems reasonable to me. But if delete no longer deletes, it needs a name change.
Oct 07 2009
parent reply downs <default_357-line yahoo.de> writes:
Don wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They
 have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast,
 class
 deallocators are designed around the idea that invoking delete
 calls the
 destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally
 useful if
 the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be
 in the
 standard library.

 What do you think?


 Andrei

you've allocated, without error? If your answer was 'ye- maaybe ... no actually', please rethink this.

manually. The discussion on class allocators and deallocators has nothing to do with that. Andrei

delete anymore?

Andrei

Isn't that a pretty big violation of Least Surprise? http://en.wikipedia.org/wiki/Principle_of_least_astonishment : "In user interface design, programming language design, and ergonomics, the principle (or rule or law) of least astonishment (or surprise) states that, when two elements of an interface conflict, or are ambiguous, the behaviour should be that which will *least surprise* the human user or programmer at the time the conflict arises."

I think the basic rule being introduced is: that every object can be managed by the gc, or manually managed. But not both. That seems reasonable to me. But if delete no longer deletes, it needs a name change.

Oh, that makes more sense. Do manually managed objects still count under Mark&Sweep?
Oct 07 2009
parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
downs wrote:
 Don wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They
 have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast,
 class
 deallocators are designed around the idea that invoking delete
 calls the
 destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally
 useful if
 the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be
 in the
 standard library.

 What do you think?


 Andrei

you've allocated, without error? If your answer was 'ye- maaybe ... no actually', please rethink this.

manually. The discussion on class allocators and deallocators has nothing to do with that. Andrei

delete anymore?

Andrei

http://en.wikipedia.org/wiki/Principle_of_least_astonishment : "In user interface design, programming language design, and ergonomics, the principle (or rule or law) of least astonishment (or surprise) states that, when two elements of an interface conflict, or are ambiguous, the behaviour should be that which will *least surprise* the human user or programmer at the time the conflict arises."

that every object can be managed by the gc, or manually managed. But not both. That seems reasonable to me. But if delete no longer deletes, it needs a name change.

Oh, that makes more sense. Do manually managed objects still count under Mark&Sweep?

You have to register the memory range they cover to the GC if they contain pointers to GC memory. Otherwise the GC don't know they exist at all.
Oct 07 2009
parent downs <default_357-line yahoo.de> writes:
Jeremie Pelletier wrote:
 downs wrote:
 Don wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 downs wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators.
 They
 have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast,
 class
 deallocators are designed around the idea that invoking delete
 calls the
 destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally
 useful if
 the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be
 in the
 standard library.

 What do you think?


 Andrei

you've allocated, without error? If your answer was 'ye- maaybe ... no actually', please rethink this.

collector manually. The discussion on class allocators and deallocators has nothing to do with that. Andrei

delete anymore?

Andrei

http://en.wikipedia.org/wiki/Principle_of_least_astonishment : "In user interface design, programming language design, and ergonomics, the principle (or rule or law) of least astonishment (or surprise) states that, when two elements of an interface conflict, or are ambiguous, the behaviour should be that which will *least surprise* the human user or programmer at the time the conflict arises."

that every object can be managed by the gc, or manually managed. But not both. That seems reasonable to me. But if delete no longer deletes, it needs a name change.

Oh, that makes more sense. Do manually managed objects still count under Mark&Sweep?

You have to register the memory range they cover to the GC if they contain pointers to GC memory. Otherwise the GC don't know they exist at all.

Well I certainly wouldn't expect that! :p This sounds like something that might trip people up. I believe at least scanning objects by GC should always be the default for any object, if only because the association "D heap => GC managed" is I think a fairly core part of the language.
Oct 07 2009
prev sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 downs wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast, class
 deallocators are designed around the idea that invoking delete calls the
 destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally useful if
 the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be in the
 standard library.

 What do you think?


 Andrei

Do you trust the D GC to be good enough to always free everything you've allocated, without error? If your answer was 'ye- maaybe ... no actually', please rethink this.

manually. The discussion on class allocators and deallocators has nothing to do with that.

Right. There's no plan to eliminate GC.free().
Oct 07 2009
parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Sean Kelly wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 downs wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast, class
 deallocators are designed around the idea that invoking delete calls the
 destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally useful if
 the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be in the
 standard library.

 What do you think?


 Andrei

If your answer was 'ye- maaybe ... no actually', please rethink this.

manually. The discussion on class allocators and deallocators has nothing to do with that.

Right. There's no plan to eliminate GC.free().

But that's runtime dependent, for example on my runtime its Memory.Free(). Removing 'delete' would therefore bind the code to a certain runtime, that's not a very portable solution, and far from being as elegant as delete.
Oct 07 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jeremie Pelletier wrote:
 Sean Kelly wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s 
 article
 downs wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They 
 have
 a number of problems that make them unsuitable for D 2.0. The most
 obvious issue is that D 2.0 will _not_ conflate destruction with
 deallocation anymore: invoking delete against an object will call
 ~this() against it but will not recycle its memory. In contrast, class
 deallocators are designed around the idea that invoking delete 
 calls the
 destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the
 language. Class allocators may be marginally and occasionally 
 useful if
 the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be 
 in the
 standard library.

 What do you think?


 Andrei

you've allocated, without error? If your answer was 'ye- maaybe ... no actually', please rethink this.

manually. The discussion on class allocators and deallocators has nothing to do with that.

Right. There's no plan to eliminate GC.free().

But that's runtime dependent, for example on my runtime its Memory.Free(). Removing 'delete' would therefore bind the code to a certain runtime, that's not a very portable solution, and far from being as elegant as delete.

There's nothing elegant about delete. Andrei
Oct 07 2009
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
I don't see any problem with dispose() method (except that it doesn't nullifies
the pointer, which can be a performance issue for some GC implementations). If
you plan to go C# way, it's reasonable to adopt its techniques of destruction.
Moreover C# and C++ approaches are compatible. If the programmer doesn't
guarantee ownership of the object, it's just unreasonable to call delete, here
adding the dispose() method to the Object and using it for destruction will
help.

Your proposal is indeed better than the scheme above and it's not a pain to
implement and use destruct+free function, but delete and dispose are already
well-known idioms, as you were already told about.
Oct 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Kagamin wrote:
 I don't see any problem with dispose() method (except that it doesn't
 nullifies the pointer, which can be a performance issue for some GC
 implementations). If you plan to go C# way, it's reasonable to adopt
 its techniques of destruction. Moreover C# and C++ approaches are
 compatible. If the programmer doesn't guarantee ownership of the
 object, it's just unreasonable to call delete, here adding the
 dispose() method to the Object and using it for destruction will
 help.
 
 Your proposal is indeed better than the scheme above and it's not a
 pain to implement and use destruct+free function, but delete and
 dispose are already well-known idioms, as you were already told
 about.

You're right. It would be great to dispose of the delete keyword and define a member function and/or a free function that invokes the destructor and obliterates the object with its .init bits. At any rate: deletion + memory reclamation must go. If you want to do manual memory management, malloc/free are yours. D's native GC heap is not the right place. Andrei
Oct 07 2009
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2009-10-07 08:46:06 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:

 You're right. It would be great to dispose of the delete keyword and 
 define a member function and/or a free function that invokes the 
 destructor and obliterates the object with its .init bits.

I guess I should have read this before posting mine. :-) You're suggesting obiterating with the .init bits, but I believe this is insufficient: you need to call a constructor if you want to be sure object invariants holds. If you can't make the invariants hold, you're in undefined behaviour territory.
 At any rate: deletion + memory reclamation must go. If you want to do 
 manual memory management, malloc/free are yours. D's native GC heap is 
 not the right place.

Well, yes you're entirely right saying that. But I fail to see how this is linked to class allocators and deallocators. Class allocators and deallocators are just a way to tell the runtime (including the GC) how to allocate and deallocate a specific class of objects. There is no need to manually call delete for the allocator and deallocator to be useful. The way it is currently, if you want objects of a certain class to be allocated in one big object pool, you can encapsulate that detail in the class so clients don't have to bother about it. I've done that in C++ to speed up things without having to touch the rest of the code base and it's quite handy. At other times the client of the class that wants to manage memory, and that should be allowed too, bypassing the class's allocator and deallocator and calling directly the constructor and destructor. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Oct 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Michel Fortin wrote:
 On 2009-10-07 08:46:06 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> said:
 
 You're right. It would be great to dispose of the delete keyword and 
 define a member function and/or a free function that invokes the 
 destructor and obliterates the object with its .init bits.

I guess I should have read this before posting mine. :-) You're suggesting obiterating with the .init bits, but I believe this is insufficient: you need to call a constructor if you want to be sure object invariants holds. If you can't make the invariants hold, you're in undefined behaviour territory.

That is correct. The default constructor must be called for classes. For structs, copying .init over will do.
 At any rate: deletion + memory reclamation must go. If you want to do 
 manual memory management, malloc/free are yours. D's native GC heap is 
 not the right place.

Well, yes you're entirely right saying that. But I fail to see how this is linked to class allocators and deallocators.

Discussion took a turn.
 Class allocators and 
 deallocators are just a way to tell the runtime (including the GC) how 
 to allocate and deallocate a specific class of objects. There is no need 
 to manually call delete for the allocator and deallocator to be useful.
 
 The way it is currently, if you want objects of a certain class to be 
 allocated in one big object pool, you can encapsulate that detail in the 
 class so clients don't have to bother about it. I've done that in C++ to 
 speed up things without having to touch the rest of the code base and 
 it's quite handy.
 
 At other times the client of the class that wants to manage memory, and 
 that should be allowed too, bypassing the class's allocator and 
 deallocator and calling directly the constructor and destructor.

I agree that some would want to manage their own allocation, and see no fault with a pool that exposes factory methods a la create() and recycle() or whatever. The language has become larger and more powerful. Now we're in an odd situation: the language has become powerful enough to render obsolete some things that previously were in the language because they couldn't be expressed. Consider a factory method create(). In the olden days, there was no way to properly forward variadic arguments to an object's constructor. So repeating C++'s awful hack seemed like a reasonable thing to do. Now even the new keyword isn't that justified because a simple function could do everything new does, plus custom allocation and whatever if we so want. Walter, Don and myself are looking into ways of making the language smaller and moving some of built-in functionality to the standard library. Tomasz' post on making an in-situ class instance was a watershed point for me. I thought about it some more and realized that language size and library size aren't the same thing. (I had a feeling before that, but no good argument.) Language is not modular and doesn't have well-defined boundaries that carve subunits. Libraries do. I can always say "I will/won't use this module/package/library" but the language just comes at you in parallel. Conversely, if you see something you don't know in some code and it's in a library, you can always decide to look at that library's code and/or documentation and figure out what's what. In contrast, if I saw a highlighted keyword that I had no idea what it does I'd get quite worried. Andrei
Oct 07 2009
parent reply "Manfred_Nowak" <svv1999 hotmail.com> writes:
Andrei Alexandrescu wrote:

  if I saw a highlighted keyword that I had no idea what it does I'd
  get quite worried 

Why wouldn't you try to look at the documentation of the language---as you do with the documentation of a library? -manfred
Oct 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Manfred_Nowak wrote:
 Andrei Alexandrescu wrote:
 
  if I saw a highlighted keyword that I had no idea what it does I'd
  get quite worried 

Why wouldn't you try to look at the documentation of the language---as you do with the documentation of a library? -manfred

I didn't say I wouldn't. I just said I'd be much more worried. My point is, languages are never modular. To be even marginally effective in a language, you must have some understanding of it all. That definitely isn't the case for libraries. Andrei
Oct 07 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Leandro Lucarella wrote:
 Andrei Alexandrescu, el  7 de octubre a las 14:18 me escribiste:
 Manfred_Nowak wrote:
 Andrei Alexandrescu wrote:

 if I saw a highlighted keyword that I had no idea what it does I'd
 get quite worried

language---as you do with the documentation of a library? -manfred

My point is, languages are never modular. To be even marginally effective in a language, you must have some understanding of it all. That definitely isn't the case for libraries.

Languages are modular when they let you define new syntax, but that's another topic ;)

A topic at which no language succeeded. Andrei
Oct 07 2009
prev sibling next sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el  6 de octubre a las 21:42 me escribiste:
should call a destructor but not call delete() or notify the GC
that the memory is free.


Why would you do that? What is the rationale to not notify the GC?

Because there may be other live references to the object.

But when using delete that's exactly what it should happen. You are hiding a bug if you let that happen on purpose.
You're saying that there is a problem, but you're not telling us
what's wrong. Why the hell do you want to destroy an object
without recycling its memory? Why does the inability to do so
cause a problem?

places. I'm not having as much time as I'd want to explain things. In short, destroying without freeing memory avoids dangling references and preserves memory safety without impacting on other resources.

But D is a system programming language.

Well it is but there are quite a few more things at stake. First, it is a reality that it is often desirable to distinguish between calling the destructor and reclaiming memory. D's current delete continues the bad tradition started by C++ of conflating the two.

Why is a bad idea? If you are destroying an object, the object will be in an inconsistent state. What's the point of keeping it alive. Again, you're just hiding a bug; letting the bug live longer. The language should try to expose bugs ASAP, not delay the detection. I think is a good idea not to force the GC to free the memory immediately with a delete, but it should if it's easy. Other protection methods as using mprotect to protect the objects pages it's very desirable too, because you can spot an access to a inconsistent (destroyed) object as soon as it first happen.
If you wrote delete x; the
language should assume you know what you're doing.

I think delete should be present in SafeD and if you want manual memory management you should build on malloc and free.

If you want to introduce a new semantic, I think you should provide a new method, not change the semantic of an existent one. And BTW, is there any reason why this can't be implemented in the library instead of using an operator? Why don't you provide a "destroy()" function for that in Phobos? Really, I can't see any advantages on changing the delete operator semantics, only problems.
If you only want to
"deinitialize" an object, you can write a .destroy() method for example,
and call that. I think delete have a strong established semantic to change
it now, and without any gain.

It has a thoroughly broken and undesired semantics. It would be a step forward to divorce it of that.

Why it's broken? Why it's undesired?
 In fact i'd love to simply make delete disappear as a keyword and make
 it a function.

I agree on this one, no need for an operator (AFAIK). But again, I don't see how letting the user to use a destroyed object is any safer. It's really bad in fact.
It seems like a performance hack to me -- you've got an object
that isn't valid anymore, but you want to hang on to the memory
for some other purpose. And you could override new() and delete(),
but you don't want to incur the performance penalty of calling the
runtime to fetch the deallocator.


But you shouldn't provide safety where the programmer is not expecting it. delete is for *manual* memory management. It makes no sense to guarantee that the memory is *not* freed. It makes sense not guaranteeing that it will actually be freed either. I think that's a good idea actually, because it gives more flexibility to the GC implementation.

I think we should move away from the idea that delete is for manual memory management. We should leave that to the likes of malloc and free alone.

Why? Using malloc and free is a lot more trouble, you have to register the roots yourself for example. It's not like you do malloc() and free() and everything works magically. You have to have more knowledge of the GC to use them. Being able to manually manage the *GC* heap (if the GC support that, if not it can make it a NOP) is good IMHO.
The only remaining use that I see is a way to reset a shared
object without explicitly passing around a reference to the new
version of the object. This seems potentially dangerous, and
nothing I want for default behavior.


That's nice :)

I think it's a false sense of security.

Why it's bad for D? (I don't care that much about C++ reasons :) -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- Debemos creer en los sueños del niño. Cuando el niño sueña con tetas, se toca. -- Ricardo Vaporeso. Toulouse, 1915.
Oct 07 2009
parent Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el  7 de octubre a las 16:03 me escribiste:
 Leandro Lucarella wrote:
Andrei Alexandrescu, el  7 de octubre a las 15:23 me escribiste:
You seem to be asserting that without additional built-in language
support, manual memory management is unduly difficult. Why so?

Because of this:
auto x = cast(MyClass) malloc(MyClass.classinfo.init.length);
x.__ctor( a, b, c ); // construct
...
x.__dtor();
free( cast(void*) x );




:) You even forgot to register your object as a root in the GC, so if your MyClass has any pointers to the GC your program will blow in your face. If you plan to library support to ease this and avoid repetitive and bug-prone work, you can ignore my complains...

I too think it would be great to add the necessary support to the stdlib. In fact, since you have a great deal of expertise in the matter, feel free to suggest API functions! They'd need to be approved by Sean too because probably they belong to druntime.

I think the only API change should be adding a function to call the destructors but not GC.free() (as David suggested). Then, the other changes are only moving operators to library code. So nothing changes much there. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- No existe nada más intenso que un reloj, ni nada más flaco que una bicicleta. No intenso como el café, ni flaco como escopeta. -- Ricardo Vaporeso
Oct 08 2009
prev sibling next sibling parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Andrei Alexandrescu wrote:
 Hello,
 
 
 D currently allows defining class allocators and deallocators. They have 
 a number of problems that make them unsuitable for D 2.0. The most 
 obvious issue is that D 2.0 will _not_ conflate destruction with 
 deallocation anymore: invoking delete against an object will call 
 ~this() against it but will not recycle its memory. In contrast, class 
 deallocators are designed around the idea that invoking delete calls the 
 destructor and also deallocates memory.
 
 So I'm thinking of removing at least class deallocators from the 
 language. Class allocators may be marginally and occasionally useful if 
 the user takes the matter of deallocation in her own hands.
 
 A much better way to handle custom allocation of classes would be in the 
 standard library.
 
 What do you think?
 
 
 Andrei

I wouldn't like delete to go away at all, I use it for all my non-gc objects like this watered down example: class ManualObject : Object { new(size_t size) { return malloc(size); } delete(void* mem) { free(mem); } } And then I can easily subclass it for any objects that doesn't need the GC. I've got similar constructs for arrays and structs. malloc/free are nice, but they don't allow for elegant abstractions like new/delete does (for example if you want to use a specialized non-gc allocator you can just replace a few calls instead of every allocation). I also use delete when I no longer need large blocks of memory, I don't want them to just become uninitialized and sitting on the GC. When I want to do that I just nullify my references. If you're afraid of deleting an object that may still have valid references, use smart pointers, or don't delete it at all if it sits on the gc and just call a .destroy() method. Also in my runtime the delete implementations do free the memory, they don't just call the finalizer. In any ways, just don't remove new/delete overrides from the language please, just call it a low-level technique or something to scare the beginners away and let people who want it have it :) Jeremie
Oct 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jeremie Pelletier wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They 
 have a number of problems that make them unsuitable for D 2.0. The 
 most obvious issue is that D 2.0 will _not_ conflate destruction with 
 deallocation anymore: invoking delete against an object will call 
 ~this() against it but will not recycle its memory. In contrast, class 
 deallocators are designed around the idea that invoking delete calls 
 the destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the 
 language. Class allocators may be marginally and occasionally useful 
 if the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be in 
 the standard library.

 What do you think?


 Andrei

I wouldn't like delete to go away at all, I use it for all my non-gc objects like this watered down example: class ManualObject : Object { new(size_t size) { return malloc(size); } delete(void* mem) { free(mem); } } And then I can easily subclass it for any objects that doesn't need the GC. I've got similar constructs for arrays and structs.

Clearly you use those objects in a very different manner than GC objects. So by using new/delete with them you're fooling yourself. // untested class ManualObject { static T create(T : ManualObject)() { auto p = malloc(__traits(classInstanceSize, T)); memcpy(p, T.classinfo.init.ptr, __traits(classInstanceSize, T)); auto result = cast(T) p; result.__ctor(); return result; } static void yank(ManualObject obj) { free(cast(void*) obj); } } Looks like a fair amount of work? At some level it actually should, but we can put that kind of stuff in the standard library.
 malloc/free are nice, but they don't allow for elegant abstractions like 
 new/delete does (for example if you want to use a specialized non-gc 
 allocator you can just replace a few calls instead of every allocation).

They do if you're willing to write just a bit of scaffolding.
 I also use delete when I no longer need large blocks of memory, I don't 
 want them to just become uninitialized and sitting on the GC. When I 
 want to do that I just nullify my references.
 
 If you're afraid of deleting an object that may still have valid 
 references, use smart pointers, or don't delete it at all if it sits on 
 the gc and just call a .destroy() method.
 
 Also in my runtime the delete implementations do free the memory, they 
 don't just call the finalizer.
 
 In any ways, just don't remove new/delete overrides from the language 
 please, just call it a low-level technique or something to scare the 
 beginners away and let people who want it have it :)

I strongly believe custom new/delete must go. Andrei
Oct 07 2009
parent reply Don <nospam nospam.com> writes:
Andrei Alexandrescu wrote:
 Jeremie Pelletier wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They 
 have a number of problems that make them unsuitable for D 2.0. The 
 most obvious issue is that D 2.0 will _not_ conflate destruction with 
 deallocation anymore: invoking delete against an object will call 
 ~this() against it but will not recycle its memory. In contrast, 
 class deallocators are designed around the idea that invoking delete 
 calls the destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the 
 language. Class allocators may be marginally and occasionally useful 
 if the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be in 
 the standard library.

 What do you think?


 Andrei

I wouldn't like delete to go away at all, I use it for all my non-gc objects like this watered down example: class ManualObject : Object { new(size_t size) { return malloc(size); } delete(void* mem) { free(mem); } } And then I can easily subclass it for any objects that doesn't need the GC. I've got similar constructs for arrays and structs.

Clearly you use those objects in a very different manner than GC objects. So by using new/delete with them you're fooling yourself. // untested class ManualObject { static T create(T : ManualObject)() { auto p = malloc(__traits(classInstanceSize, T)); memcpy(p, T.classinfo.init.ptr, __traits(classInstanceSize, T)); auto result = cast(T) p; result.__ctor(); return result; } static void yank(ManualObject obj) { free(cast(void*) obj); } } Looks like a fair amount of work? At some level it actually should, but we can put that kind of stuff in the standard library.
 malloc/free are nice, but they don't allow for elegant abstractions 
 like new/delete does (for example if you want to use a specialized 
 non-gc allocator you can just replace a few calls instead of every 
 allocation).

They do if you're willing to write just a bit of scaffolding.
 I also use delete when I no longer need large blocks of memory, I 
 don't want them to just become uninitialized and sitting on the GC. 
 When I want to do that I just nullify my references.

 If you're afraid of deleting an object that may still have valid 
 references, use smart pointers, or don't delete it at all if it sits 
 on the gc and just call a .destroy() method.

 Also in my runtime the delete implementations do free the memory, they 
 don't just call the finalizer.

 In any ways, just don't remove new/delete overrides from the language 
 please, just call it a low-level technique or something to scare the 
 beginners away and let people who want it have it :)

I strongly believe custom new/delete must go. Andrei

Yes. The only reason you want them in C++ is because C++ makes constructors magical, by always glueing a memory allocation in front of them, and pretending they're not a function. Then you need to introduce placement new to avoid the memory allocation bit. Let's call a spade a spade: a constructor is just a function that establishes the invariant on a piece of memory which it recieves as a parameter. If you stop the pretense, you don't need the language machinery.
Oct 08 2009
parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Don wrote:
 Andrei Alexandrescu wrote:
 Jeremie Pelletier wrote:
 Andrei Alexandrescu wrote:
 Hello,


 D currently allows defining class allocators and deallocators. They 
 have a number of problems that make them unsuitable for D 2.0. The 
 most obvious issue is that D 2.0 will _not_ conflate destruction 
 with deallocation anymore: invoking delete against an object will 
 call ~this() against it but will not recycle its memory. In 
 contrast, class deallocators are designed around the idea that 
 invoking delete calls the destructor and also deallocates memory.

 So I'm thinking of removing at least class deallocators from the 
 language. Class allocators may be marginally and occasionally useful 
 if the user takes the matter of deallocation in her own hands.

 A much better way to handle custom allocation of classes would be in 
 the standard library.

 What do you think?


 Andrei

I wouldn't like delete to go away at all, I use it for all my non-gc objects like this watered down example: class ManualObject : Object { new(size_t size) { return malloc(size); } delete(void* mem) { free(mem); } } And then I can easily subclass it for any objects that doesn't need the GC. I've got similar constructs for arrays and structs.

Clearly you use those objects in a very different manner than GC objects. So by using new/delete with them you're fooling yourself. // untested class ManualObject { static T create(T : ManualObject)() { auto p = malloc(__traits(classInstanceSize, T)); memcpy(p, T.classinfo.init.ptr, __traits(classInstanceSize, T)); auto result = cast(T) p; result.__ctor(); return result; } static void yank(ManualObject obj) { free(cast(void*) obj); } } Looks like a fair amount of work? At some level it actually should, but we can put that kind of stuff in the standard library.
 malloc/free are nice, but they don't allow for elegant abstractions 
 like new/delete does (for example if you want to use a specialized 
 non-gc allocator you can just replace a few calls instead of every 
 allocation).

They do if you're willing to write just a bit of scaffolding.
 I also use delete when I no longer need large blocks of memory, I 
 don't want them to just become uninitialized and sitting on the GC. 
 When I want to do that I just nullify my references.

 If you're afraid of deleting an object that may still have valid 
 references, use smart pointers, or don't delete it at all if it sits 
 on the gc and just call a .destroy() method.

 Also in my runtime the delete implementations do free the memory, 
 they don't just call the finalizer.

 In any ways, just don't remove new/delete overrides from the language 
 please, just call it a low-level technique or something to scare the 
 beginners away and let people who want it have it :)

I strongly believe custom new/delete must go. Andrei

Yes. The only reason you want them in C++ is because C++ makes constructors magical, by always glueing a memory allocation in front of them, and pretending they're not a function. Then you need to introduce placement new to avoid the memory allocation bit. Let's call a spade a spade: a constructor is just a function that establishes the invariant on a piece of memory which it recieves as a parameter. If you stop the pretense, you don't need the language machinery.

Yeah I agree now after reading most of this thread, I know that these keywords just map to functions. I've seen a proposal of a global new template somewhere, I don't like that since at the global scope there are also structs, arrays and whatnot that can be allocated by 'new'. I don't like the static new either since it prevents subclasses from overriding their new/delete operations. What would then be a good way to replace new/delete operators to still have them overridable? Isn't that the convenience that first got them to be used in the first place? Other than global new/delete overrides which is plain silly in D. I've pretty much found alternatives to all my other points against taking out new/delete except for the override feature, find me an alternative for that too and I'll be voting for new/delete to be runtime function instead of language keywords, cause I can't think of anything right now.
Oct 08 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jeremie Pelletier wrote:
 Yeah I agree now after reading most of this thread, I know that these 
 keywords just map to functions.
 
 I've seen a proposal of a global new template somewhere, I don't like 
 that since at the global scope there are also structs, arrays and 
 whatnot that can be allocated by 'new'.

Well it's easy to handle all of those with conditional templates.
 I don't like the static new either since it prevents subclasses from 
 overriding their new/delete operations.
 
 What would then be a good way to replace new/delete operators to still 
 have them overridable? Isn't that the convenience that first got them to 
 be used in the first place? Other than global new/delete overrides which 
 is plain silly in D.
 
 I've pretty much found alternatives to all my other points against 
 taking out new/delete except for the override feature, find me an 
 alternative for that too and I'll be voting for new/delete to be runtime 
 function instead of language keywords, cause I can't think of anything 
 right now.

I think you'd find this article interesting: http://www.ddj.com/article/printableArticle.jhtml?articleID=184405016&dept_url=/java/ Andrei
Oct 08 2009
next sibling parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Andrei Alexandrescu wrote:
 Jeremie Pelletier wrote:
 Yeah I agree now after reading most of this thread, I know that these 
 keywords just map to functions.

 I've seen a proposal of a global new template somewhere, I don't like 
 that since at the global scope there are also structs, arrays and 
 whatnot that can be allocated by 'new'.

Well it's easy to handle all of those with conditional templates.
 I don't like the static new either since it prevents subclasses from 
 overriding their new/delete operations.

 What would then be a good way to replace new/delete operators to still 
 have them overridable? Isn't that the convenience that first got them 
 to be used in the first place? Other than global new/delete overrides 
 which is plain silly in D.

 I've pretty much found alternatives to all my other points against 
 taking out new/delete except for the override feature, find me an 
 alternative for that too and I'll be voting for new/delete to be 
 runtime function instead of language keywords, cause I can't think of 
 anything right now.

I think you'd find this article interesting: http://www.ddj.com/article/printableArticle.jhtml?articleID=18440 016&dept_url=/java/ Andrei

That was a long read, but a most interesting one! I already was familiar of these design pattens but only used them where new didn't make sense, this article really was an eye opener on that level, thanks a lot! You have my vote to remove new/delete now :)
Oct 08 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jeremie Pelletier wrote:
 Andrei Alexandrescu wrote:
 Jeremie Pelletier wrote:
 Yeah I agree now after reading most of this thread, I know that these 
 keywords just map to functions.

 I've seen a proposal of a global new template somewhere, I don't like 
 that since at the global scope there are also structs, arrays and 
 whatnot that can be allocated by 'new'.

Well it's easy to handle all of those with conditional templates.
 I don't like the static new either since it prevents subclasses from 
 overriding their new/delete operations.

 What would then be a good way to replace new/delete operators to 
 still have them overridable? Isn't that the convenience that first 
 got them to be used in the first place? Other than global new/delete 
 overrides which is plain silly in D.

 I've pretty much found alternatives to all my other points against 
 taking out new/delete except for the override feature, find me an 
 alternative for that too and I'll be voting for new/delete to be 
 runtime function instead of language keywords, cause I can't think of 
 anything right now.

I think you'd find this article interesting: http://www.ddj.com/article/printableArticle.jhtml?articleID=18440 016&dept_url=/java/ Andrei

That was a long read, but a most interesting one! I already was familiar of these design pattens but only used them where new didn't make sense, this article really was an eye opener on that level, thanks a lot! You have my vote to remove new/delete now :)

Someone convinced someone else of something on the Internets. What's this world coming to??? Andrei
Oct 08 2009
parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Andrei Alexandrescu wrote:
 Jeremie Pelletier wrote:
 Andrei Alexandrescu wrote:
 Jeremie Pelletier wrote:
 Yeah I agree now after reading most of this thread, I know that 
 these keywords just map to functions.

 I've seen a proposal of a global new template somewhere, I don't 
 like that since at the global scope there are also structs, arrays 
 and whatnot that can be allocated by 'new'.

Well it's easy to handle all of those with conditional templates.
 I don't like the static new either since it prevents subclasses from 
 overriding their new/delete operations.

 What would then be a good way to replace new/delete operators to 
 still have them overridable? Isn't that the convenience that first 
 got them to be used in the first place? Other than global new/delete 
 overrides which is plain silly in D.

 I've pretty much found alternatives to all my other points against 
 taking out new/delete except for the override feature, find me an 
 alternative for that too and I'll be voting for new/delete to be 
 runtime function instead of language keywords, cause I can't think 
 of anything right now.

I think you'd find this article interesting: http://www.ddj.com/article/printableArticle.jhtml?articleID=18440 016&dept_url=/java/ Andrei

That was a long read, but a most interesting one! I already was familiar of these design pattens but only used them where new didn't make sense, this article really was an eye opener on that level, thanks a lot! You have my vote to remove new/delete now :)

Someone convinced someone else of something on the Internets. What's this world coming to??? Andrei

World peace, open-minded societies and money-free economies where love and sharing has won over fear and competition. Maybe not, but I can dream.
Oct 08 2009
parent Craig Black <cblack ara.com> writes:
Jeremie Pelletier Wrote:

 Andrei Alexandrescu wrote:
 Jeremie Pelletier wrote:
 Andrei Alexandrescu wrote:
 Jeremie Pelletier wrote:
 Yeah I agree now after reading most of this thread, I know that 
 these keywords just map to functions.

 I've seen a proposal of a global new template somewhere, I don't 
 like that since at the global scope there are also structs, arrays 
 and whatnot that can be allocated by 'new'.

Well it's easy to handle all of those with conditional templates.
 I don't like the static new either since it prevents subclasses from 
 overriding their new/delete operations.

 What would then be a good way to replace new/delete operators to 
 still have them overridable? Isn't that the convenience that first 
 got them to be used in the first place? Other than global new/delete 
 overrides which is plain silly in D.

 I've pretty much found alternatives to all my other points against 
 taking out new/delete except for the override feature, find me an 
 alternative for that too and I'll be voting for new/delete to be 
 runtime function instead of language keywords, cause I can't think 
 of anything right now.

I think you'd find this article interesting: http://www.ddj.com/article/printableArticle.jhtml?articleID=18440 016&dept_url=/java/ Andrei

That was a long read, but a most interesting one! I already was familiar of these design pattens but only used them where new didn't make sense, this article really was an eye opener on that level, thanks a lot! You have my vote to remove new/delete now :)

Someone convinced someone else of something on the Internets. What's this world coming to??? Andrei

World peace, open-minded societies and money-free economies where love and sharing has won over fear and competition. Maybe not, but I can dream.

Perhaps not in our lifetime, but eventually. What you envision is inevitable. Kudos for not being emotionally attached to your view point. You are ahead of your time. -Craig
Oct 08 2009
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2009-10-08 10:35:01 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:

 I think you'd find this article interesting:
 
 http://www.ddj.com/article/printableArticle.jhtml?articleID=184405016&dept_url=/java/

That's
 

Smalltalk-like approach, but believe it's not really possible in a static language. But that's exactly what we can have in D by remaking 'new' as a function template. :-) Just as the author wants, with a template 'new' function it seems quite possible to change 'new' into a factory function instanciating the best class for the given arguments: void newGC(T, A...)(A args); // create a garbage-collected instance String new(T: String)(immutable(char)[] utf8Str) { return newGC!UTF8ImmutableString(utf8str); } String new(T: String)(immutable(ubyte)[] strData, int encoding) { // instanciate the best string type depending on encoding. if (encoding == UTF8) return newGC!UTF8ImmutableString(cast(string)strData); else if (encoding == ISO_LATIN_1) return newGC!OneBytePerCharImmutableString(cast(string)strData, encoding); else ... } -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Oct 08 2009
parent Yigal Chripun <yigal100 gmail.com> writes:
On 09/10/2009 04:54, Michel Fortin wrote:
 On 2009-10-08 10:35:01 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> said:

 I think you'd find this article interesting:

 http://www.ddj.com/article/printableArticle.jhtml?articleID=184405016&dept_url=/java/

That's

Smalltalk-like approach, but believe it's not really possible in a static language. But that's exactly what we can have in D by remaking 'new' as a function template. :-) Just as the author wants, with a template 'new' function it seems quite possible to change 'new' into a factory function instanciating the best class for the given arguments: void newGC(T, A...)(A args); // create a garbage-collected instance String new(T: String)(immutable(char)[] utf8Str) { return newGC!UTF8ImmutableString(utf8str); } String new(T: String)(immutable(ubyte)[] strData, int encoding) { // instanciate the best string type depending on encoding. if (encoding == UTF8) return newGC!UTF8ImmutableString(cast(string)strData); else if (encoding == ISO_LATIN_1) return newGC!OneBytePerCharImmutableString(cast(string)strData, encoding); else ... }

I like the ruby style syntax option. I'll also would like the option to encapsulate all the machinery in an allocator hierarchy conveniently provided in the stdlib. my templating skills are weak but is something like this could be possible? class Foo { this(args) {}; static Foo new(Allocator, ARGS...) (Allocator al, ARGS args) { alloc = al; return alloc.allocate!Foo(args); // alloc will also call the ctor } static Foo new(ARGS... args) { return GC.allocate!Foo(args); } Allocator alloc; } how does this affect ctors? they could become regular functions. how does this work with abstract classes?
Oct 09 2009
prev sibling next sibling parent Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el  7 de octubre a las 13:06 me escribiste:
I think is a good idea not to force the GC to free the memory immediately
with a delete, but it should if it's easy. Other protection methods as
using mprotect to protect the objects pages it's very desirable too,
because you can spot an access to a inconsistent (destroyed) object as
soon as it first happen.

(mprotect is much too coarse to be useful.) With the dispose() function the state of the object will be restored to default construction: void dispose(T)(T obj) if (is(T == class) || is(typeof(*T.init))) { ... call destructor if any ... ... obliterate object with .init ... ... invoke default ctor if any ... }

Ok, if you're going to name that dispose, is fine with me. End of discussion. With the addition of calling a constructor after destroying the object, make a little more sense too (I still find it too bug prone, you can end up with corruption if you dispose an object that other part of the program think it's not disposed yet, i.e., in a state different than the recently constructed object).
If you want to introduce a new semantic, I think you should provide a new
method, not change the semantic of an existent one.

Agreed. I hereby vote for deprecating delete with extreme prejudice.
And BTW, is there any reason why this can't be implemented in the library
instead of using an operator? Why don't you provide a "destroy()" function
for that in Phobos?

That sounds great.
Really, I can't see any advantages on changing the delete operator
semantics, only problems.

I agree.

I'm glad to see that.
Why? Using malloc and free is a lot more trouble, you have to register the
roots yourself for example. It's not like you do malloc() and free() and
everything works magically. You have to have more knowledge of the GC to
use them. Being able to manually manage the *GC* heap (if the GC support
that, if not it can make it a NOP) is good IMHO.

We can make things a tad better with library functions, but we do need to have a garbage collected heap that guarantees safety.

I don't think I understand this very well. What kind of safety? If the user disposed/freed an object before it should, it's an user bug, with unavoidable bad side effects. The best you can do is make the program blow in the user face ASAP. I don't understand what all this have to do with GC safety. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- It's not a lie, if you believe it. -- George Constanza
Oct 07 2009
prev sibling parent Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el  7 de octubre a las 14:18 me escribiste:
 Manfred_Nowak wrote:
Andrei Alexandrescu wrote:

 if I saw a highlighted keyword that I had no idea what it does I'd
 get quite worried

Why wouldn't you try to look at the documentation of the language---as you do with the documentation of a library? -manfred

I didn't say I wouldn't. I just said I'd be much more worried. My point is, languages are never modular. To be even marginally effective in a language, you must have some understanding of it all. That definitely isn't the case for libraries.

Languages are modular when they let you define new syntax, but that's another topic ;) -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- You look so tired-unhappy, bring down the government, they don't, they don't speak for us.
Oct 07 2009