www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - length = 0 clears reserve

reply Jethro <qyzz gr.ff> writes:
arrays have the ability to reserve but when setting the length to 
0, it removes the reserve!! ;/

char[] buf;
buf.reserve = 1000;
buf.length = 0;
assert(buf.capacity == 0);

But I simply want to clear the buffer, not change it's 
reserve/capacity.

I've tried to hack by setting the length to 0 through a pointer, 
but that still clears the capacity!

I want to do this because I want to be able to reuse the array 
without ever reallocating(I'll set the capacity to the max that 
will ever be used, I don't have to worry about conflicts since it 
will always be ran serially).

It is a bug if dmd is setting the capacity to 0 and reallocating 
the array when the length is set to 0. It should either do it 
lazily or not at all(or, say setting the length to -1 does not 
clear the capacity, only sets the length to 0).
Apr 10
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Tuesday, April 11, 2017 01:42:32 Jethro via Digitalmars-d-learn wrote:
 arrays have the ability to reserve but when setting the length to
 0, it removes the reserve!! ;/

 char[] buf;
 buf.reserve = 1000;
 buf.length = 0;
 assert(buf.capacity == 0);

 But I simply want to clear the buffer, not change it's
 reserve/capacity.

 I've tried to hack by setting the length to 0 through a pointer,
 but that still clears the capacity!

 I want to do this because I want to be able to reuse the array
 without ever reallocating(I'll set the capacity to the max that
 will ever be used, I don't have to worry about conflicts since it
 will always be ran serially).

 It is a bug if dmd is setting the capacity to 0 and reallocating
 the array when the length is set to 0. It should either do it
 lazily or not at all(or, say setting the length to -1 does not
 clear the capacity, only sets the length to 0).
You can't reuse the memory of a dynamic array by simply setting its length to 0. If that were allowed, it would risk allow dynamic arrays to stomp on each others memory (since there is no guarantee that there are no other dynamic arrays referring to the same memory). However, if you know that there are no other dynamic arrays referrin to the same memory, then you can call assumeSafeAppend on the dynamic array, and then the runtime will assume that there are no other dynamic arrays referring to the same memory. If you haven't already, I would advise reading http://dlang.org/d-array-article.html It uses the wrong terminology, but otherwise, it's an excellent, informative article. The article refers to T[] as a slice, and the GC-allocated block of memory as a dynamic array. However, the official terminology is that T[] is a dynamic array which refers to a block of memory which may or may not be GC-allocated, and there is no special term for a GC-allocated block of memory. T[] _is_ also a slice of whatever memory it refers to, but it's the dynamic array, not the block of memory. And in fact, all of the dynamic array operations work perfectly fine regardless of whether the block of memory is GC-allocated or not (it's just that if it's not, the capacity is 0, so appending will always result in a reallocation). In any case, while the terminology in the article isn't quite right, it gives a good overview of how dynamic arrays work in D, and it explains what's going with the capacity of the array and why setting the length of a dynamic array to 0 results in the capacity being 0. - Jonathan M Davis
Apr 10
parent reply Jon Degenhardt <jond noreply.com> writes:
On Tuesday, 11 April 2017 at 01:59:57 UTC, Jonathan M Davis wrote:
 On Tuesday, April 11, 2017 01:42:32 Jethro via 
 Digitalmars-d-learn wrote:
 arrays have the ability to reserve but when setting the length 
 to 0, it removes the reserve!! ;/

 char[] buf;
 buf.reserve = 1000;
 buf.length = 0;
 assert(buf.capacity == 0);

 But I simply want to clear the buffer, not change it's 
 reserve/capacity.

 I've tried to hack by setting the length to 0 through a 
 pointer, but that still clears the capacity!

 I want to do this because I want to be able to reuse the array 
 without ever reallocating(I'll set the capacity to the max 
 that will ever be used, I don't have to worry about conflicts 
 since it will always be ran serially).

 [snip]
You can't reuse the memory of a dynamic array by simply setting its length to 0. If that were allowed, it would risk allow dynamic arrays to stomp on each others memory (since there is no guarantee that there are no other dynamic arrays referring to the same memory). However, if you know that there are no other dynamic arrays referrin to the same memory, then you can call assumeSafeAppend on the dynamic array, and then the runtime will assume that there are no other dynamic arrays referring to the same memory. [snip]
Another technique that works for many cases is to use an Appender (std.array). Appender supports reserve and clear, the latter setting the length to zero without reallocating. A typical use case is an algorithm doing a series of appends, then setting the length to zero and starts appending again. --Jon
Apr 10
parent reply Jethro <qyzz gr.ff> writes:
On Tuesday, 11 April 2017 at 03:00:29 UTC, Jon Degenhardt wrote:
 On Tuesday, 11 April 2017 at 01:59:57 UTC, Jonathan M Davis 
 wrote:
 On Tuesday, April 11, 2017 01:42:32 Jethro via 
 Digitalmars-d-learn wrote:
 [...]
You can't reuse the memory of a dynamic array by simply setting its length to 0. If that were allowed, it would risk allow dynamic arrays to stomp on each others memory (since there is no guarantee that there are no other dynamic arrays referring to the same memory). However, if you know that there are no other dynamic arrays referrin to the same memory, then you can call assumeSafeAppend on the dynamic array, and then the runtime will assume that there are no other dynamic arrays referring to the same memory. [snip]
Another technique that works for many cases is to use an Appender (std.array). Appender supports reserve and clear, the latter setting the length to zero without reallocating. A typical use case is an algorithm doing a series of appends, then setting the length to zero and starts appending again. --Jon
Appender reports clear? Are you sure? Seems appender is no different than string, maybe worse? string as assumeSafeAppend, reserve and clear(although clear necessarily reallocates. They should have a function called empty, which resets the length to zero but doesn't reallocate.
Apr 11
parent Jon Degenhardt <jond noreply.com> writes:
On Tuesday, 11 April 2017 at 20:00:48 UTC, Jethro wrote:
 On Tuesday, 11 April 2017 at 03:00:29 UTC, Jon Degenhardt wrote:
 On Tuesday, 11 April 2017 at 01:59:57 UTC, Jonathan M Davis 
 wrote:
 On Tuesday, April 11, 2017 01:42:32 Jethro via 
 Digitalmars-d-learn wrote:
 [...]
You can't reuse the memory of a dynamic array by simply setting its length to 0. If that were allowed, it would risk allow dynamic arrays to stomp on each others memory (since there is no guarantee that there are no other dynamic arrays referring to the same memory). However, if you know that there are no other dynamic arrays referrin to the same memory, then you can call assumeSafeAppend on the dynamic array, and then the runtime will assume that there are no other dynamic arrays referring to the same memory. [snip]
Another technique that works for many cases is to use an Appender (std.array). Appender supports reserve and clear, the latter setting the length to zero without reallocating. A typical use case is an algorithm doing a series of appends, then setting the length to zero and starts appending again. --Jon
Appender reports clear? Are you sure? Seems appender is no different than string, maybe worse? string as assumeSafeAppend, reserve and clear(although clear necessarily reallocates. They should have a function called empty, which resets the length to zero but doesn't reallocate.
See the Appender.clear documentation (https://dlang.org/phobos/std_array.html#.Appender.clear), the key piece being: Clears the managed array. This allows the elements of the array to be reused for appending. I've tried using both dynamic array and appender in this way, setting the length of the dynamic array to zero vs using Appender.clear, in this cycle of fill-the-array by appending, operate on the array, clearing, and repeating. Appender is dramatically faster. And, if you look at GC reports you find that setting a dynamic array to zero creates garbage to collect, while Appender.clear does not. (Use the --DRT-gcopt=profile:1 command line option to see GC reports, described here: https://dlang.org/spec/garbage.html#gc_config).
Apr 11