www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - allocate array with new

reply "Namespace" <rswhite4 googlemail.com> writes:
Is there any other difference if I allocate an array with "new", 
except that it is initialized?
And must i delete it by myself, or take the GC care about that?
The documentary leaves me hanging there, unfortunately, and gives 
me no 100% answer.

Once before, thanks for the reply.
May 15 2012
next sibling parent reply "Kagamin" <spam here.lot> writes:
Difference with what?
new is a safe feature: it allocates in the GC heap
May 15 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 15.05.2012 16:27, Ondrej Pokorny wrote:
 On Tuesday, 15 May 2012 at 09:44:08 UTC, Jonathan M Davis wrote:
 On Tuesday, May 15, 2012 11:26:49 Namespace wrote:
 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:
 Difference with what?
 new is a safe feature: it allocates in the GC heap

That's what i mean. So i have to delete it yourself with "delete arr;", or not?

No. _Never_ use delete. It's going to be deprecated. The GC worries about freeing memory allocated on the GC heap, and new always allocates on the GC heap. If you don't want to allocate on the GC heap, then use malloc and free, in which case you _do_ need worry about freeing the memory. If you need to force destruction before the GC collects an object, you can call clear on that object to have its destructor called and its vtbl zeroed out, but it's memory still isn't freed. That's the GC's job. If you really have to, you can use core.memory to manipulate the GC heap (including calling GC.free), but you really shouldn't be messing with any of that unless you really need to and you know what you're doing. - Jonathan M Davis

Hi, does this hold for structs too? struct H { this(int a){writeln("ctor");} ~this(){writeln("dtor");} }; ... H* h = new H(5); clear(h); ... output: ctor seems like destructor is not called. if I change declaration of H to class H. output is following: ctor dtor I tried to create object according to RAII idiom and in case of struct H my file handle remained open and I was still able to write to file... Ondrej

H h = H(5); Same works in D. A call to clear in you code above doesn't call destructor, it only zeros out pointer. If you absolutely need pointers & heap and yet manual memory managment use: clear(*h); Explanation: clear(x) calls x.__dtor if x is struct or class, then assigns x = T.init; A better way might be to just check if x.__dtor is callbale (thus pointer to sstruct will also work). -- Dmitry Olshansky
May 15 2012
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 15.05.2012 17:12, Ondrej Pokorny wrote:
 On Tuesday, 15 May 2012 at 12:36:30 UTC, Dmitry Olshansky wrote:
 On 15.05.2012 16:27, Ondrej Pokorny wrote:
 On Tuesday, 15 May 2012 at 09:44:08 UTC, Jonathan M Davis wrote:
 On Tuesday, May 15, 2012 11:26:49 Namespace wrote:
 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:
 Difference with what?
 new is a safe feature: it allocates in the GC heap

That's what i mean. So i have to delete it yourself with "delete arr;", or not?

No. _Never_ use delete. It's going to be deprecated. The GC worries about freeing memory allocated on the GC heap, and new always allocates on the GC heap. If you don't want to allocate on the GC heap, then use malloc and free, in which case you _do_ need worry about freeing the memory. If you need to force destruction before the GC collects an object, you can call clear on that object to have its destructor called and its vtbl zeroed out, but it's memory still isn't freed. That's the GC's job. If you really have to, you can use core.memory to manipulate the GC heap (including calling GC.free), but you really shouldn't be messing with any of that unless you really need to and you know what you're doing. - Jonathan M Davis

Hi, does this hold for structs too? struct H { this(int a){writeln("ctor");} ~this(){writeln("dtor");} }; ... H* h = new H(5); clear(h); ... output: ctor seems like destructor is not called. if I change declaration of H to class H. output is following: ctor dtor I tried to create object according to RAII idiom and in case of struct H my file handle remained open and I was still able to write to file... Ondrej

H h = H(5); Same works in D. A call to clear in you code above doesn't call destructor, it only zeros out pointer. If you absolutely need pointers & heap and yet manual memory managment use: clear(*h); Explanation: clear(x) calls x.__dtor if x is struct or class, then assigns x = T.init; A better way might be to just check if x.__dtor is callbale (thus pointer to sstruct will also work).

I don't need this stuff I am playing with language and wonder why. Originally I had there auto h = H(5); and after changing H to struct I did not realized why that happened. Thanks a lot for explanation, now it makes perfect sense, but I have still little bit unpleasant felling about it because of behavior of ref to classes vs pointers to structs with clear. Both are dynamically allocated data, so I was expecting same response from clear(x).

Like I said it's easily fixable at a cost of breaking some code that relies on this clear(ptr) zeroing out pointer. I think that it's wrong code anyway. So I I'll wrap up a pull request to do just that. -- Dmitry Olshansky
May 15 2012
prev sibling next sibling parent "Namespace" <rswhite4 googlemail.com> writes:
On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:
 Difference with what?
 new is a safe feature: it allocates in the GC heap

That's what i mean. So i have to delete it yourself with "delete arr;", or not?
May 15 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, May 15, 2012 11:26:49 Namespace wrote:
 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:
 Difference with what?
 new is a safe feature: it allocates in the GC heap

That's what i mean. So i have to delete it yourself with "delete arr;", or not?

No. _Never_ use delete. It's going to be deprecated. The GC worries about freeing memory allocated on the GC heap, and new always allocates on the GC heap. If you don't want to allocate on the GC heap, then use malloc and free, in which case you _do_ need worry about freeing the memory. If you need to force destruction before the GC collects an object, you can call clear on that object to have its destructor called and its vtbl zeroed out, but it's memory still isn't freed. That's the GC's job. If you really have to, you can use core.memory to manipulate the GC heap (including calling GC.free), but you really shouldn't be messing with any of that unless you really need to and you know what you're doing. - Jonathan M Davis
May 15 2012
prev sibling next sibling parent "Namespace" <rswhite4 googlemail.com> writes:
On Tuesday, 15 May 2012 at 09:44:08 UTC, Jonathan M Davis wrote:
 On Tuesday, May 15, 2012 11:26:49 Namespace wrote:
 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:
 Difference with what?
 new is a safe feature: it allocates in the GC heap

That's what i mean. So i have to delete it yourself with "delete arr;", or not?

No. _Never_ use delete. It's going to be deprecated. The GC worries about freeing memory allocated on the GC heap, and new always allocates on the GC heap. If you don't want to allocate on the GC heap, then use malloc and free, in which case you _do_ need worry about freeing the memory. If you need to force destruction before the GC collects an object, you can call clear on that object to have its destructor called and its vtbl zeroed out, but it's memory still isn't freed. That's the GC's job. If you really have to, you can use core.memory to manipulate the GC heap (including calling GC.free), but you really shouldn't be messing with any of that unless you really need to and you know what you're doing. - Jonathan M Davis

Understood. Never use delete. Fine. :)
May 15 2012
prev sibling next sibling parent "Ondrej Pokorny" <pokorny.ondrej gmail.com> writes:
On Tuesday, 15 May 2012 at 09:44:08 UTC, Jonathan M Davis wrote:
 On Tuesday, May 15, 2012 11:26:49 Namespace wrote:
 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:
 Difference with what?
 new is a safe feature: it allocates in the GC heap

That's what i mean. So i have to delete it yourself with "delete arr;", or not?

No. _Never_ use delete. It's going to be deprecated. The GC worries about freeing memory allocated on the GC heap, and new always allocates on the GC heap. If you don't want to allocate on the GC heap, then use malloc and free, in which case you _do_ need worry about freeing the memory. If you need to force destruction before the GC collects an object, you can call clear on that object to have its destructor called and its vtbl zeroed out, but it's memory still isn't freed. That's the GC's job. If you really have to, you can use core.memory to manipulate the GC heap (including calling GC.free), but you really shouldn't be messing with any of that unless you really need to and you know what you're doing. - Jonathan M Davis

Hi, does this hold for structs too? struct H { this(int a){writeln("ctor");} ~this(){writeln("dtor");} }; ... H* h = new H(5); clear(h); ... output: ctor seems like destructor is not called. if I change declaration of H to class H. output is following: ctor dtor I tried to create object according to RAII idiom and in case of struct H my file handle remained open and I was still able to write to file... Ondrej
May 15 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 15/05/12 11:43, Jonathan M Davis wrote:
 No. _Never_ use delete. It's going to be deprecated.

What if you turn the GC off?
May 15 2012
prev sibling next sibling parent "Ondrej Pokorny" <pokorny.ondrej gmail.com> writes:
On Tuesday, 15 May 2012 at 12:36:30 UTC, Dmitry Olshansky wrote:
 On 15.05.2012 16:27, Ondrej Pokorny wrote:
 On Tuesday, 15 May 2012 at 09:44:08 UTC, Jonathan M Davis 
 wrote:
 On Tuesday, May 15, 2012 11:26:49 Namespace wrote:
 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:
 Difference with what?
 new is a safe feature: it allocates in the GC heap

That's what i mean. So i have to delete it yourself with "delete arr;", or not?

No. _Never_ use delete. It's going to be deprecated. The GC worries about freeing memory allocated on the GC heap, and new always allocates on the GC heap. If you don't want to allocate on the GC heap, then use malloc and free, in which case you _do_ need worry about freeing the memory. If you need to force destruction before the GC collects an object, you can call clear on that object to have its destructor called and its vtbl zeroed out, but it's memory still isn't freed. That's the GC's job. If you really have to, you can use core.memory to manipulate the GC heap (including calling GC.free), but you really shouldn't be messing with any of that unless you really need to and you know what you're doing. - Jonathan M Davis

Hi, does this hold for structs too? struct H { this(int a){writeln("ctor");} ~this(){writeln("dtor");} }; ... H* h = new H(5); clear(h); ... output: ctor seems like destructor is not called. if I change declaration of H to class H. output is following: ctor dtor I tried to create object according to RAII idiom and in case of struct H my file handle remained open and I was still able to write to file... Ondrej

allocation): H h = H(5); Same works in D. A call to clear in you code above doesn't call destructor, it only zeros out pointer. If you absolutely need pointers & heap and yet manual memory managment use: clear(*h); Explanation: clear(x) calls x.__dtor if x is struct or class, then assigns x = T.init; A better way might be to just check if x.__dtor is callbale (thus pointer to sstruct will also work).

I don't need this stuff I am playing with language and wonder why. Originally I had there auto h = H(5); and after changing H to struct I did not realized why that happened. Thanks a lot for explanation, now it makes perfect sense, but I have still little bit unpleasant felling about it because of behavior of ref to classes vs pointers to structs with clear. Both are dynamically allocated data, so I was expecting same response from clear(x). Ondrej
May 15 2012
prev sibling next sibling parent "Namespace" <rswhite4 googlemail.com> writes:
And what if i use an array of structs? Also clear instead of
delete?
May 15 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 5/15/12, Jonathan M Davis <jmdavisProg gmx.com> wrote:
 If you're going to use anything, use clear.

He might as well just use this if he wants the GC to take care of things: arr = null; clear(arr) is a little bit of a misnomer. If you have another slice pointing to the same array clear() won't really release memory at all, it will just make the first slice null. so using "arr = null" is good enough. Also good to note is that clear/null assign is very different from what delete does. delete won't care that you have other slices pointing to the same array and you'll end up with slices that point to garbage after a call to delete.
May 15 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 15 May 2012 05:43:45 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Tuesday, May 15, 2012 11:26:49 Namespace wrote:
 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:
 Difference with what?
 new is a safe feature: it allocates in the GC heap

That's what i mean. So i have to delete it yourself with "delete arr;", or not?

No. _Never_ use delete. It's going to be deprecated. The GC worries about freeing memory allocated on the GC heap, and new always allocates on the GC heap. If you don't want to allocate on the GC heap, then use malloc and free, in which case you _do_ need worry about freeing the memory. If you need to force destruction before the GC collects an object, you can call clear on that object to have its destructor called and its vtbl zeroed out, but it's memory still isn't freed. That's the GC's job. If you really have to, you can use core.memory to manipulate the GC heap (including calling GC.free), but you really shouldn't be messing with any of that unless you really need to and you know what you're doing.

Well, the D GC is conservative, and will always be at least partly conservative. So freeing memory manually is not really a crime. I agree you should avoid delete, due to its possible deprecation. -Steve
May 16 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, May 16, 2012 11:51:59 Steven Schveighoffer wrote:
 Well, the D GC is conservative, and will always be at least partly
 conservative. So freeing memory manually is not really a crime.

The idea with a GC is that it's supposed to manage the memory allocated through it. So, if you're worrying about freeing GC memory, you're fighting the GC and how it works. And unless you're keeping very close track of the number of reference to a particular chunk of memory, freeing it manually is dangerous. So, anyone looking to use delete is generally going about things the wrong way. However, it _is_ true that sometimes you need to intervene in order to get sections of code as performant as they need to be (particularly since D's current GC is not the most performant), which is part of the reason that core.memory provides what it does. But it should generally be something used when it's clear that you need that extra performance, not as a matter of course, because using it is inherently unsafe. If there's something that you know ahead of time needs to have more deterministic deallocation, then it's probably better to be use malloc and free with ref-counting than trying to manually manage GC memory. - Jonathan M Davis
May 16 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 16 May 2012 12:52:33 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Wednesday, May 16, 2012 11:51:59 Steven Schveighoffer wrote:
 Well, the D GC is conservative, and will always be at least partly
 conservative. So freeing memory manually is not really a crime.

The idea with a GC is that it's supposed to manage the memory allocated through it. So, if you're worrying about freeing GC memory, you're fighting the GC and how it works. And unless you're keeping very close track of the number of reference to a particular chunk of memory, freeing it manually is dangerous. So, anyone looking to use delete is generally going about things the wrong way. However, it _is_ true that sometimes you need to intervene in order to get sections of code as performant as they need to be (particularly since D's current GC is not the most performant), which is part of the reason that core.memory provides what it does. But it should generally be something used when it's clear that you need that extra performance, not as a matter of course, because using it is inherently unsafe. If there's something that you know ahead of time needs to have more deterministic deallocation, then it's probably better to be use malloc and free with ref-counting than trying to manually manage GC memory.

Where the GC gets into trouble is with allocation of large chunks of data. If the data 'contains pointers', it creates a very good chance that the data keeps some unknown number of blocks from being deallocated. And just by the sheer fact of how big the data is, it's bound to be pointed at by some piece of stack data, global data, or TLS data. So you can go a long way by manually managing larger chunks of data. I think it's worth worrying about as you get into large chunks (on the order of 10MB or more). What really really sucks about this is, as you allocate larger chunks of data, the chances that the GC incorrectly keeps garbage memory get larger. In other words, the more memory you allocate, the less likely it is that the GC will give you some of it back to you! Precise GC will help this *tremendously*, but I'm unsure if we'll ever get to the point of having a fully-precise GC. There will always be a need to do manual freeing of memory. -Steve
May 16 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, May 16, 2012 13:52:12 Steven Schveighoffer wrote:
 Where the GC gets into trouble is with allocation of large chunks of data.
 
 If the data 'contains pointers', it creates a very good chance that the
 data keeps some unknown number of blocks from being deallocated.
 
 And just by the sheer fact of how big the data is, it's bound to be
 pointed at by some piece of stack data, global data, or TLS data.
 
 So you can go a long way by manually managing larger chunks of data. I
 think it's worth worrying about as you get into large chunks (on the order
 of 10MB or more).
 
 What really really sucks about this is, as you allocate larger chunks of
 data, the chances that the GC incorrectly keeps garbage memory get
 larger. In other words, the more memory you allocate, the less likely it
 is that the GC will give you some of it back to you!

Using 64-bit helps with this.
 Precise GC will help this *tremendously*, but I'm unsure if we'll ever get
 to the point of having a fully-precise GC.

Any improvement to the GC is a welcome improvement.
 There will always be a need to
 do manual freeing of memory.

Yes, but my point is that it's something you do when the need arises due to performance concerns in a particular section of code. It's not something that you do as a matter of course in your code in general. If you're always trying to free memory as soon as you don't need it anymore rather than letting the GC do its job (or try to anyway), then you might as well be using malloc and free. - Jonathan M Davis
May 16 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, May 16, 2012 at 02:20:47PM -0400, Jonathan M Davis wrote:
 On Wednesday, May 16, 2012 13:52:12 Steven Schveighoffer wrote:
 Where the GC gets into trouble is with allocation of large chunks of
 data.
 
 If the data 'contains pointers', it creates a very good chance that
 the data keeps some unknown number of blocks from being deallocated.
 
 And just by the sheer fact of how big the data is, it's bound to be
 pointed at by some piece of stack data, global data, or TLS data.
 
 So you can go a long way by manually managing larger chunks of data.
 I think it's worth worrying about as you get into large chunks (on
 the order of 10MB or more).
 
 What really really sucks about this is, as you allocate larger
 chunks of data, the chances that the GC incorrectly keeps garbage
 memory get larger. In other words, the more memory you allocate, the
 less likely it is that the GC will give you some of it back to you!

Using 64-bit helps with this.

Yeah, IIRC somebody did some tests comparing false pointer incidence in 32-bit vs. 64-bit environments, and found that false pointer incidence is much lower in 64-bit. But of course, a large-enough chunk of memory will still have a non-zero chance of a false pointer into it, which is bad because it will just keep sticking around.
 Precise GC will help this *tremendously*, but I'm unsure if we'll
 ever get to the point of having a fully-precise GC.

Any improvement to the GC is a welcome improvement.

Didn't Walter say that we're pushing toward precise scanning? Of course, the fact that we interface directly with C (and have union types) means that there will always be _some_ place that can't be fully precise. But having more precise info, such as which references are slices and which are free pointers, would help a lot. For example, if the 10MB chunk is referenced by a slice of 10 bytes, the GC could in theory release everything except those 10 bytes. You're still out of luck with free pointers, but in regular D code those should be quite rare.
 There will always be a need to
 do manual freeing of memory.

Yes, but my point is that it's something you do when the need arises due to performance concerns in a particular section of code. It's not something that you do as a matter of course in your code in general. If you're always trying to free memory as soon as you don't need it anymore rather than letting the GC do its job (or try to anyway), then you might as well be using malloc and free.

It would be nice if there was an easy way of switching allocators, so that you can, for example, indicate that all allocations done inside a particular function should use malloc/free (including, say, AA's, arrays, etc.), and have the runtime automatically switch back to the GC on scope exit. T -- Laissez-faire is a French term commonly interpreted by Conservatives to mean 'lazy fairy,' which is the belief that if governments are lazy enough, the Good Fairy will come down from heaven and do all their work for them.
May 16 2012
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, May 16, 2012 12:05:02 H. S. Teoh wrote:
 It would be nice if there was an easy way of switching allocators, so
 that you can, for example, indicate that all allocations done inside a
 particular function should use malloc/free (including, say, AA's,
 arrays, etc.), and have the runtime automatically switch back to the GC
 on scope exit.

Once we have custom allocators, that should be easy enough as long as you passing the allocator around. But new will always use the GC, and malloc will always do what it does in C (it _is_ the C function after all), so anything you want in terms of custom allocation is going to have to go through custom allocator objects. However, since the custom allocators are going to be runtime entities (rather than having classes templated on their allocators like the STL does), it would be easy enough to choose the appropriate allocator at runtime and have each function use a different allocator if it so chooses. - Jonathan M Davis
May 16 2012