www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Allocating structs with new?

reply Jonathan M Davis <jmdavisprog gmail.com> writes:
I thought that classes always went on the heap and that structs always went on 
the stack - so allocating structs with new wouldn't work. Also, I thought that 
delete was deprecated if not outright removed from D. And yet, we have a new
bug 
that Andrei reported about destructors for structs not working correctly when 
they're allocated with new (and delete is being used to destroy it in the code 
in the report).

http://d.puremagic.com/issues/show_bug.cgi?id=4442

Code from bug report:

struct S1{ ~this() { writeln("dtor"); } }
void main() {
    auto a = S1();
    auto b = new S1();
    delete b;
    auto c  = new S1();
    c = null;
    GC.collect();
}

Am I missing something here? TDPL was quite clear that classes were reference 
types and structs were value types, and here we appear to have a struct used as 
a reference type. Is this some sort of feature from outside of SafeD, or is it
a 
feature that was supposed to be removed but hasn't yet, or what? I know that 
TDPL doesn't cover everything and that what dmd does doesn't always match it 
yet, but from what I understood from reading TDPL, the code above shouldn't 
compile at all since it's using a struct like a class and is using the
supposedy 
defunct delete operator.

- Jonathan M Davis
Jul 09 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Jonathan M Davis Wrote:


 I thought that classes always went on the heap and that structs always went on 
 the stack


Structs can be allocated on the heap, and then referenced by pointer. This is
not something I can live without.
Explicit delete is a problem with a language based on GC, it has to be removed
with care.

Bye,
bearophile
Jul 09 2010
parent Justin Johansson <no spam.com> writes:
bearophile wrote:

 Jonathan M Davis Wrote:
 

 I thought that classes always went on the heap and that structs always went on 
 the stack


 
 Structs can be allocated on the heap, and then referenced by pointer. This is
not something I can live without.
 Explicit delete is a problem with a language based on GC, it has to be removed
with care.
 
 Bye,
 bearophile


With care?  That doesn't say very much except for possibly an
unintended suggestion that the OP is not trying to use care.
He is posting because he does care.

Sorry bearophile, but can you please better explain in the context
of the OP's original dilemma just what kind of care is needed to
bake this cookie.

Cheers,
Justin Johansson
Jul 10 2010
prev sibling next sibling parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Sat, 10 Jul 2010 01:06:39 -0400, Jonathan M Davis  
<jmdavisprog gmail.com> wrote:


 I thought that classes always went on the heap and that structs always  
 went on
 the stack - so allocating structs with new wouldn't work. Also, I  
 thought that
 delete was deprecated if not outright removed from D. And yet, we have a  
 new bug
 that Andrei reported about destructors for structs not working correctly  
 when
 they're allocated with new (and delete is being used to destroy it in  
 the code
 in the report).

 http://d.puremagic.com/issues/show_bug.cgi?id=4442

 Code from bug report:

 struct S1{ ~this() { writeln("dtor"); } }
 void main() {
     auto a = S1();
     auto b = new S1();
     delete b;
     auto c  = new S1();
     c = null;
     GC.collect();
 }

 Am I missing something here? TDPL was quite clear that classes were  
 reference
 types and structs were value types, and here we appear to have a struct  
 used as
 a reference type. Is this some sort of feature from outside of SafeD, or  
 is it a
 feature that was supposed to be removed but hasn't yet, or what? I know  
 that
 TDPL doesn't cover everything and that what dmd does doesn't always  
 match it
 yet, but from what I understood from reading TDPL, the code above  
 shouldn't
 compile at all since it's using a struct like a class and is using the  
 supposedy
 defunct delete operator.

 - Jonathan M Davis


A new statement can also allocate value types on the heap and return a  
pointer to them. For example, the type of b and c is S1* and the GC  
doesn't know about C's destructor. Please refer to the D spec on new  
(http://www.digitalmars.com/d/2.0/expression.html#NewExpression) or TDPL  
page 51. Both state that any type can be used with new; no limitations  
with regard to reference or non-reference types are mentioned. Also, on  
page 269 of TDPL there is an example of containing a Node*.

As for the bug, it's actually somewhat old. Issue 2834 was filed back in  
April of 09 - Struct Destructors are not called by the GC, but called on  
explicit delete. And this is a bug in the spec, not DMD; the only way to  
run struct destructors from the GC is to add vtables to them or hidden in  
the GC (i.e. to make them classes). By the way, structs inside of classes  
do get their destructors called.

As for delete and clear, the deprecation/transition is still in progress.  
clear exists, but delete is not deprecated yet.
Jul 09 2010
next sibling parent reply Jonathan M Davis <jmdavisprog gmail.com> writes:
On Friday 09 July 2010 23:31:18 Robert Jacques wrote:

 
 A new statement can also allocate value types on the heap and return a
 pointer to them. For example, the type of b and c is S1* and the GC
 doesn't know about C's destructor. Please refer to the D spec on new
 (http://www.digitalmars.com/d/2.0/expression.html#NewExpression) or TDPL
 page 51. Both state that any type can be used with new; no limitations
 with regard to reference or non-reference types are mentioned. Also, on
 page 269 of TDPL there is an example of containing a Node*.


I guess that I missed that, though it doesn't help that I've read the class and 
struct chapters more recently, and they made a big deal about how classes were 
reference type and structs were value types and may no mention whatsoever of 
using structs on the heap - that and I rarely use pointers with primitive types 
even in C++, and I pretty much never use pointers in D. I think that I was 
thinking that you had to use malloc to put primitive types no the heap though. 
It never occurred to me to put a struct on the heap. Obviously, it's not a 
feature that I use often.

I don't mind that it be possible with new, but it is potentially confusing. Oh 
well, at least I'm aware of it now. If I ever want to use new in that manner 
though, I'm obviously going to have to study up on the subject more. I'm quite 
clear about how all that works in languages like C++ or Java, but D is a bit of 
a hybrid when it comes to memory management, and I obviously haven't pick up on 
all of its subtleties yet.

- Jonathan M Davis
Jul 10 2010
parent Justin Johansson <no spam.com> writes:
Jonathan M Davis wrote:

 On Friday 09 July 2010 23:31:18 Robert Jacques wrote:

 A new statement can also allocate value types on the heap and return a
 pointer to them. For example, the type of b and c is S1* and the GC
 doesn't know about C's destructor. Please refer to the D spec on new
 (http://www.digitalmars.com/d/2.0/expression.html#NewExpression) or TDPL
 page 51. Both state that any type can be used with new; no limitations
 with regard to reference or non-reference types are mentioned. Also, on
 page 269 of TDPL there is an example of containing a Node*.


 
 I guess that I missed that, though it doesn't help that I've read the class
and 
 struct chapters more recently, and they made a big deal about how classes were 
 reference type and structs were value types and may no mention whatsoever of 
 using structs on the heap - that and I rarely use pointers with primitive
types 
 even in C++, and I pretty much never use pointers in D. I think that I was 
 thinking that you had to use malloc to put primitive types no the heap though. 
 It never occurred to me to put a struct on the heap. Obviously, it's not a 
 feature that I use often.
 
 I don't mind that it be possible with new, but it is potentially confusing. Oh 
 well, at least I'm aware of it now. If I ever want to use new in that manner 
 though, I'm obviously going to have to study up on the subject more. I'm quite 
 clear about how all that works in languages like C++ or Java, but D is a bit
of 
 a hybrid when it comes to memory management, and I obviously haven't pick up
on 
 all of its subtleties yet.
 
 - Jonathan M Davis


Don't blame it on any lack on your part to pick up on D's subtleties;
I feel your pain also.

A clean programming language should aspire to removing such subtleties.

Recall the "D manifesto" on the D home page if one can call that
a manifesto.

<quote from-url="http://www.digitalmars.com/d/">
D is a systems programming language. Its focus is on
combining the power and high performance of C and C++ with the
programmer productivity of modern languages like Ruby and Python.
Special attention is given to the needs of quality assurance,
documentation, management, portability and reliability.
</quote>

Now take challenge to that "manifesto" and ask "Who is kidding who?"

Sorry, Walter & Co, but there is a long way to go or otherwise
time to start over.  Even the Scala people have had the decency to
question themselves as to whether their language was an experiment.

See the Scala experiment:
lampwww.epfl.ch/~odersky/talks/popl06.pdf

My apologies for sometimes sounding pro-D and then sometimes
contra-D.  Overall I think that the goals of D are admirable
but then so much lost in the realization.

Cheers

Justin Johansson
Jul 10 2010
prev sibling next sibling parent reply div0 <div0 users.sourceforge.net> writes:
On 10/07/2010 07:31, Robert Jacques wrote:


 As for delete and clear, the deprecation/transition is still in
 progress. clear exists, but delete is not deprecated yet.


So what are we going to use instead? I have uses for delete.

-- 
My enormous talent is exceeded only by my outrageous laziness.
http://www.ssTk.co.uk
Jul 10 2010
next sibling parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Sat, 10 Jul 2010 10:01:52 -0400, div0 <div0 users.sourceforge.net>  
wrote:


 On 10/07/2010 07:31, Robert Jacques wrote:


 As for delete and clear, the deprecation/transition is still in
 progress. clear exists, but delete is not deprecated yet.


 So what are we going to use instead? I have uses for delete.


clear is meant to be a safe version of delete.
Jul 10 2010
parent div0 <div0 users.sourceforge.net> writes:
On 10/07/2010 18:04, Robert Jacques wrote:

 On Sat, 10 Jul 2010 10:01:52 -0400, div0 <div0 users.sourceforge.net>
 wrote:


 On 10/07/2010 07:31, Robert Jacques wrote:


 As for delete and clear, the deprecation/transition is still in
 progress. clear exists, but delete is not deprecated yet.


 So what are we going to use instead? I have uses for delete.


 clear is meant to be a safe version of delete.


Yeah I don't care about safe. Safe D can never happen at all for all I care.

If clear doesn't return the memory to the free list it's fecking useless 
to me; I have a need for deterministic memory allocation.

If new D is going to require a GC it's no longer a systems programming 
language and it's no longer either interesting or an appropriate 
replacement for C++.

-- 
My enormous talent is exceeded only by my outrageous laziness.
http://www.ssTk.co.uk
Jul 10 2010
prev sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
div0 <div0 users.sourceforge.net> wrote:


 On 10/07/2010 07:31, Robert Jacques wrote:


 As for delete and clear, the deprecation/transition is still in
 progress. clear exists, but delete is not deprecated yet.


 So what are we going to use instead? I have uses for delete.


druntime's gc.gc has extern (C) void gc_free( void* p ). That
probably is what you want.

-- 
Simen
Jul 10 2010
prev sibling next sibling parent Adam Ruppe <destructionator gmail.com> writes:
On 7/10/10, Jonathan M Davis <jmdavisprog gmail.com> wrote:

 I think that I was
 thinking that you had to use malloc to put primitive types no the heap
 though.


D is very similar to C++ here, with the big exception that classes
*must* use new, whereas new is just an option for everything else.

int* a = new int; // valid C++ and D
struct S {}
S* s = new S; // same as int case, works in C++ and D both



 I don't mind that it be possible with new, but it is potentially confusing.


The thing that might be confusing in the example is the auto keyword
hides the fact that you were working with a pointer (S*) instead of
the object itself (S). Since D doesn't have the -> operator like C++,
working with the S and the S* is the same in most ways, so they look
virtually identical.

But as far as new's functionality, that's the same as C++, returning a
pointer to a heap allocated whatever you want. Oh, there is one big
difference between D and C++ new: in D, it is garbage collected, so
you never have to call delete explicitly.
Jul 10 2010
prev sibling parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Robert Jacques, el 10 de julio a las 02:31 me escribiste:

 As for the bug, it's actually somewhat old. Issue 2834 was filed
 back in April of 09 - Struct Destructors are not called by the GC,
 but called on explicit delete. And this is a bug in the spec, not
 DMD; the only way to run struct destructors from the GC is to add
 vtables to them or hidden in the GC (i.e. to make them classes).


You don't need a vtable, because you don't have inheritance with
structs. All you need is a function pointer pointing to the finalization
function/method. This plays well with the concept of having type
information associated to blocks of memory in the GC, which means
(semi)precise heap scanning. So maybe is not a bad idea to make the
destructors of structs get called by the GC :)

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Debemos creer en los sueños del niño. Cuando el niño sueña con tetas, se
toca.
	-- Ricardo Vaporeso. Toulouse, 1915.
Jul 10 2010
parent "Robert Jacques" <sandford jhu.edu> writes:
On Sat, 10 Jul 2010 18:17:21 -0400, Leandro Lucarella <luca llucax.com.ar>  
wrote:


 Robert Jacques, el 10 de julio a las 02:31 me escribiste:

 As for the bug, it's actually somewhat old. Issue 2834 was filed
 back in April of 09 - Struct Destructors are not called by the GC,
 but called on explicit delete. And this is a bug in the spec, not
 DMD; the only way to run struct destructors from the GC is to add
 vtables to them or hidden in the GC (i.e. to make them classes).


 You don't need a vtable, because you don't have inheritance with
 structs. All you need is a function pointer pointing to the finalization
 function/method. This plays well with the concept of having type
 information associated to blocks of memory in the GC, which means
 (semi)precise heap scanning. So maybe is not a bad idea to make the
 destructors of structs get called by the GC :)


:) That's sort-of what I meant by a vtable. (Either it being one function  
or an actual table so structs could hook into the class finalization  
method.)
Jul 10 2010
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Jonathan M Davis:

 It never occurred to me to put a struct on the heap. Obviously, it's not a 
 feature that I use often.


Structs are simpler for the compiler, help you produce a smaller program,
require less memory for each instance (two words less, not counting the chunk
sizez allocated by the GC), the runtime has a bit less things to do to manage
them. Putting structs on the heap can be useful if you want to build certain
efficient data structures, like certain trees, but you may want to allocate
many of them at the same time anyway. One advantage of structs is that they are
simpler, so if you have to do simpler things, using a struct you denote that
you are doing something simple in your code :-) There's no risk of having
virtual functions or inheritance etc if you use structs.

Bye,
bearophile
Jul 11 2010
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Justin Johansson:

 With care?  That doesn't say very much except for possibly an
 unintended suggestion that the OP is not trying to use care.
 He is posting because he does care.


I probably meant something different.
What I meant is just that one of the main purposes of a GC is to free
programmers to manually release memory, this makes many other things simpler.
If you keep a way to manually delete an object in a GC-based language then the
programmer has to use care, to avoid memory leaks, etc.

In Python there you can do:
del x

but it doesn't deallocate the object x, it just decreases by one the reference
counts of x (and removes the corresponding name in the current scope), this can
lead to the deallocation of x, but it can also happen later. There is no direct
command to directly delete an object. So you need less care.

Bye,
bearophile
Jul 11 2010
parent Justin Johansson <no spam.com> writes:
bearophile wrote:

 Justin Johansson:

 With care?  That doesn't say very much except for possibly an
 unintended suggestion that the OP is not trying to use care.
 He is posting because he does care.


 
 I probably meant something different.
 What I meant is just that one of the main purposes of a GC is to free
programmers to manually release memory, this makes many other things simpler.
If you keep a way to manually delete an object in a GC-based language then the
programmer has to use care, to avoid memory leaks, etc.
 
 In Python there you can do:
 del x
 
 but it doesn't deallocate the object x, it just decreases by one the reference
counts of x (and removes the corresponding name in the current scope), this can
lead to the deallocation of x, but it can also happen later. There is no direct
command to directly delete an object. So you need less care.
 
 Bye,
 bearophile


I probably meant something different too and also remiss of me
to leave out a few smilies. :-)

"has to use care, to avoid memory leaks, etc."  Yes and presumably
"etc" includes not attempting to delete anything more than once.

Cheers
Justin
Jul 12 2010
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 10 Jul 2010 01:06:39 -0400, Jonathan M Davis  
<jmdavisprog gmail.com> wrote:


 I thought that classes always went on the heap and that structs always  
 went on
 the stack - so allocating structs with new wouldn't work. Also, I  
 thought that
 delete was deprecated if not outright removed from D. And yet, we have a  
 new bug
 that Andrei reported about destructors for structs not working correctly  
 when
 they're allocated with new (and delete is being used to destroy it in  
 the code
 in the report).

 http://d.puremagic.com/issues/show_bug.cgi?id=4442

 Code from bug report:

 struct S1{ ~this() { writeln("dtor"); } }
 void main() {
     auto a = S1();
     auto b = new S1();
     delete b;
     auto c  = new S1();
     c = null;
     GC.collect();
 }

 Am I missing something here? TDPL was quite clear that classes were  
 reference
 types and structs were value types, and here we appear to have a struct  
 used as
 a reference type. Is this some sort of feature from outside of SafeD, or  
 is it a
 feature that was supposed to be removed but hasn't yet, or what? I know  
 that
 TDPL doesn't cover everything and that what dmd does doesn't always  
 match it
 yet, but from what I understood from reading TDPL, the code above  
 shouldn't
 compile at all since it's using a struct like a class and is using the  
 supposedy
 defunct delete operator.


structs on the heap are a necessity.  Classes are too heavyweight for  
storing things like graph/tree nodes, one does not need the vtable/lock  
when one is concerned about performance.  Switching from classes to  
structs on dcollections' nodes saved quite a bit of execution time.

The problem I see here is allowing the GC to manage the memory of a struct  
and having the GC responsible for calling the destructor of something that  
it can't possibly know the destructor of.  Classes have their destructor  
stored in the vtable, which is how the GC gets at it, the GC has no  
typeinfo to go on.  Even with the proposed precise scanning, I think the  
GC only stores basic pointer maps, it does not have access to the full  
TypeInfo.

I'd say it's a program error to have a GC allocated struct with a  
destructor.  It should be made a compiler error too.  There are ways to  
have structs stored on the heap and have destructors called on them.

Also, the example given in the bug report is very simplistic, just to  
demonstrate the problem.  Does anyone have a good use case for struct  
dtors being called when allocated by the GC?  All of the struct dtors I've  
seen assume they are stack-allocated.

-Steve
Jul 12 2010
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/12/2010 07:27 AM, Steven Schveighoffer wrote:

 Also, the example given in the bug report is very simplistic, just to
 demonstrate the problem. Does anyone have a good use case for struct
 dtors being called when allocated by the GC? All of the struct dtors
 I've seen assume they are stack-allocated.


Concrete scenarios of structs ending up on the hap often involve classes 
that have structs as members, but those do end up being accounted for.

Statically disabling new for structs with dtors - that's an interesting 
idea.


Andrei
Jul 12 2010
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 Statically disabling new for structs with dtors - that's an interesting 
 idea.


In this bug report, comment 6, I have asked for a warning, but turning it into
an error can work too :-)
http://d.puremagic.com/issues/show_bug.cgi?id=2834

Bye,
bearophile
Jul 12 2010