www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Partial arrays reclaimed?

reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
Suppose an array is being used like a FIFO:

-----------------------
T[] slice;

// Add:
slice ~= T();

// Remove:
slice = slice[1..$];
-----------------------

Assuming of course there's no other references to the memory, as this 
gets used, does the any of the memory from the removed elements ever get 
GC'd?

Also, if this is a long-running process, isn't there a potential danger 
in the array just marching through the address space and running out of 
room? (ie either running out of of continuous space, or hitting 0xFFF....)
Jan 27
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 27 January 2017 at 23:22:17 UTC, Nick Sabalausky wrote:
 Suppose an array is being used like a FIFO:

 -----------------------
 T[] slice;

 // Add:
 slice ~= T();

 // Remove:
 slice = slice[1..$];
 -----------------------

 Assuming of course there's no other references to the memory, 
 as this gets used, does the any of the memory from the removed 
 elements ever get GC'd?

 Also, if this is a long-running process, isn't there a 
 potential danger in the array just marching through the address 
 space and running out of room? (ie either running out of of 
 continuous space, or hitting 0xFFF....)
It should reclaim the memory from the beginning of the array. But your could just have written a small tool that does this and look at the memory use over time. Generally a good idea, to perform experiments with things you are interested in. And there is always the risk of running out of memory on machines which have no mmu.
Jan 27
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 27 January 2017 at 23:36:58 UTC, Stefan Koch wrote:
 It should reclaim the memory from the beginning of the array.
I doubt the current implementation will though, you should check.
Jan 27
prev sibling next sibling parent Jonathan M Davis via Digitalmars-d-learn writes:
On Friday, January 27, 2017 18:22:17 Nick Sabalausky via Digitalmars-d-learn 
wrote:
 Suppose an array is being used like a FIFO:

 -----------------------
 T[] slice;

 // Add:
 slice ~= T();

 // Remove:
 slice = slice[1..$];
 -----------------------

 Assuming of course there's no other references to the memory, as this
 gets used, does the any of the memory from the removed elements ever get
 GC'd?

 Also, if this is a long-running process, isn't there a potential danger
 in the array just marching through the address space and running out of
 room? (ie either running out of of continuous space, or hitting 0xFFF....)
Given that a dynamic array refers to a block of memory, whether that's the whole block of memory or just a portion of it, and that as you shrink the array, it's just moving where the pointer refers to in the block, there's no way that it can release the memory. A reallocation would have to take place so that the elements would be copied to a new block of memory and the dynamic array would then refer to the new block. And the only times that a reallocation is going to take place are when you append to the array or when you call reserve, and the array doesn't have enough capacity to grow in place to fulfill that reserve. So, if all you're doing is shrinking the array, then there's no way that any of its memory is going to be freed. You'd pretty much have to dup it at some point and assign the new array back to it if you want to make that possible. - Jonathan M Davis
Jan 29
prev sibling next sibling parent ag0aep6g <anonymous example.com> writes:
On Friday, 27 January 2017 at 23:22:17 UTC, Nick Sabalausky wrote:
 Suppose an array is being used like a FIFO:

 -----------------------
 T[] slice;

 // Add:
 slice ~= T();

 // Remove:
 slice = slice[1..$];
 -----------------------

 Assuming of course there's no other references to the memory, 
 as this gets used, does the any of the memory from the removed 
 elements ever get GC'd?
If the array has no additional capacity, then appending will relocate the data. I.e., copy it to a larger allocation. The old data can then be collected. Since the old first element is not part of the new array, it's doesn't get copied over. So the allocation doesn't grow indefinitely.
 Also, if this is a long-running process, isn't there a 
 potential danger in the array just marching through the address 
 space and running out of room? (ie either running out of of 
 continuous space, or hitting 0xFFF....)
If you append and pop the front over and over, the program should reuse old locations, cycling through them.
Jan 29
prev sibling parent Ivan Kazmenko <gassa mail.ru> writes:
On Friday, 27 January 2017 at 23:22:17 UTC, Nick Sabalausky wrote:
 Suppose an array is being used like a FIFO:

 -----------------------
 T[] slice;

 // Add:
 slice ~= T();

 // Remove:
 slice = slice[1..$];
 -----------------------

 Assuming of course there's no other references to the memory, 
 as this gets used, does the any of the memory from the removed 
 elements ever get GC'd?
As I see it, the line slice = slice[1..$]; effectively ends slice's in-place appending capabilities. So each append after remove will likely reallocate. You have to use assumeSafeAppend to re-enable appending in place. Here [1] is an old thread about the caveats of using built-in arrays as queues and stacks. If not in a hurry, the better option is perhaps to just write the respective wrapper structs which explicitly store indices, instead of using built-in slices and assumeSafeAppend all over the place. Ivan Kazmenko. [1] http://forum.dlang.org/post/yrxspdrpusrrijmfyldc forum.dlang.org
Jan 31