www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - object.reserve() and array size

reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
Is object.reserve() only useful for large arrays, or should I always use 
it if I intend to append to an array?

Let's say I want to read some data, and I expect there to be ~100 bytes 
of data.  Is there any point in using reserve() first, or will there 
always be that much memory available to an array?

  byte[] buffer;
  buffer.reserve(100);
  foreach(byte b; dataSource) buffer ~= b;

-Lars
Jul 14 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 14 Jul 2010 06:01:10 -0400, Lars T. Kyllingstad  
<public kyllingen.nospamnet> wrote:

 Is object.reserve() only useful for large arrays, or should I always use
 it if I intend to append to an array?

 Let's say I want to read some data, and I expect there to be ~100 bytes
 of data.  Is there any point in using reserve() first, or will there
 always be that much memory available to an array?

   byte[] buffer;
   buffer.reserve(100);
   foreach(byte b; dataSource) buffer ~= b;
Yes, you should always reserve if you know how much data is coming. If you do not here is what happens: Upon adding the first byte, a block of 16 bytes is allocated Upon adding the 16th byte (there is one byte for padding), a *new* block of 32 bytes is allocated, and the 15 previous bytes are copied to the 32-byte block. The original 16 byte block is left allocated because there could be other aliases to the data. Upon adding the 32nd byte, a block of 64 bytes is allocated, rinse and repeat. Upon adding the 64th byte, a block of 128 bytes is allocated, same deal. Then the block will hold 100 bytes. If you reserve 100 bytes, then a block of 128 bytes is allocated, and your data all goes in there. OT, I assume you realize that reading data in this way is horribly inefficient :) You should read a block at once if possible. -Steve
Jul 14 2010
parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Wed, 14 Jul 2010 10:35:19 -0400, Steven Schveighoffer wrote:

 On Wed, 14 Jul 2010 06:01:10 -0400, Lars T. Kyllingstad
 <public kyllingen.nospamnet> wrote:
 
 Is object.reserve() only useful for large arrays, or should I always
 use it if I intend to append to an array?

 Let's say I want to read some data, and I expect there to be ~100 bytes
 of data.  Is there any point in using reserve() first, or will there
 always be that much memory available to an array?

   byte[] buffer;
   buffer.reserve(100);
   foreach(byte b; dataSource) buffer ~= b;
Yes, you should always reserve if you know how much data is coming. If you do not here is what happens: Upon adding the first byte, a block of 16 bytes is allocated Upon adding the 16th byte (there is one byte for padding), a *new* block of 32 bytes is allocated, and the 15 previous bytes are copied to the 32-byte block. The original 16 byte block is left allocated because there could be other aliases to the data. Upon adding the 32nd byte, a block of 64 bytes is allocated, rinse and repeat. Upon adding the 64th byte, a block of 128 bytes is allocated, same deal. Then the block will hold 100 bytes. If you reserve 100 bytes, then a block of 128 bytes is allocated, and your data all goes in there. OT, I assume you realize that reading data in this way is horribly inefficient :) You should read a block at once if possible.
Yeah, that was just an artificial example. :) The actual use case was building a string from substrings. Thanks! -Lars
Jul 14 2010