www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Why is this increasing memory consumption on every iteration?

reply "Geancarlo Rocha" <asdf mailinator.com> writes:
I expected the memory to ramp up in the first couple iterations 
and eventually reach a stable point, but for some reason, windows 
task manager shows it increases on every iteration. Compiling 
with -m64 doesn't seem to change this issue.

http://pastebin.com/G5JXR9AA
Jun 15 2013
parent reply "Geancarlo Rocha" <asdf mailinator.com> writes:
On Sunday, 16 June 2013 at 02:12:02 UTC, Geancarlo Rocha wrote:

 I expected the memory to ramp up in the first couple iterations 
 and eventually reach a stable point, but for some reason, 
 windows task manager shows it increases on every iteration. 
 Compiling with -m64 doesn't seem to change this issue.

 http://pastebin.com/G5JXR9AA


I think it is safe to assume the current GC is unreliable, booo!
Jun 16 2013
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/16/2013 07:20 PM, Geancarlo Rocha wrote:

 On Sunday, 16 June 2013 at 02:12:02 UTC, Geancarlo Rocha wrote:

 I expected the memory to ramp up in the first couple iterations and
 eventually reach a stable point, but for some reason, windows task
 manager shows it increases on every iteration. Compiling with -m64
 doesn't seem to change this issue.

 http://pastebin.com/G5JXR9AA


 I think it is safe to assume


This way of reasoning is flawed.


 the current GC is unreliable, booo!


No it is not. You are not even allocating GC-controlled memory during 
the loop.

This is a std.container.BinaryHeap issue. It never shrinks the 
underlying container.

http://d.puremagic.com/issues/
Jun 16 2013
parent reply "Geancarlo Rocha" <asdf mailinator.com> writes:
On Sunday, 16 June 2013 at 18:02:26 UTC, Timon Gehr wrote:

 On 06/16/2013 07:20 PM, Geancarlo Rocha wrote:

 On Sunday, 16 June 2013 at 02:12:02 UTC, Geancarlo Rocha wrote:

 I expected the memory to ramp up in the first couple 
 iterations and
 eventually reach a stable point, but for some reason, windows 
 task
 manager shows it increases on every iteration. Compiling with 
 -m64
 doesn't seem to change this issue.

 http://pastebin.com/G5JXR9AA


 I think it is safe to assume


 This way of reasoning is flawed.


Agreed, but hey...it caught someone's attention. Mission 
accomplished :)


 the current GC is unreliable, booo!


 No it is not. You are not even allocating GC-controlled memory 
 during the loop.

 This is a std.container.BinaryHeap issue. It never shrinks the 
 underlying container.

 http://d.puremagic.com/issues/


If that is the case, I don't think that's something a beginner 
would figure out unless he went to look the source code.

the BinaryHeap is weird to me. I also don't understand this:

void toy()
{
     Array!(int) store;
     auto minheap = new BinaryHeap!(Array!(int), "a > b")(store);
     auto rng = new Random(unpredictableSeed);

     assert( store.length == 0 );
     writeln("inserting...");

     for( uint i = 0 ; i < 100_000 ; i++)
     {
         minheap.insert(1);
         assert( store.length == 0 );
     }
     writeln("done.");
     assert( store.length == 0 );

     writeln("removing");
     while(!minheap.empty)
     {
         minheap.removeFront();
         assert( store.length == 0 );
     }
     writeln("done.");
     assert( store.length == 0 );
}

I thought the BinaryHeap would modify store's buffer AND it's 
properties, but it doesn't seem to be the case. How is store 
being used? Again this is the sort of thing that makes a 
beginner's learning curve steeper.


Thanks for your attention
Jun 16 2013
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/16/2013 08:37 PM, Geancarlo Rocha wrote:

 ...

 the current GC is unreliable, booo!


 No it is not. You are not even allocating GC-controlled memory during
 the loop.

 This is a std.container.BinaryHeap issue. It never shrinks the
 underlying container.

 http://d.puremagic.com/issues/


 If that is the case, I don't think that's something a beginner would
 figure out unless he went to look the source code.


That's what I have done. I consider the behaviour entirely unreasonable. 
You may report it using the link above.



 BinaryHeap is weird to me.


You mean coming from the System.Collections namespace of the .NET 
framework. BinaryHeap is a library artefact.


 I also don't understand this:

 void toy()
 {
      Array!(int) store;
      auto minheap = new BinaryHeap!(Array!(int), "a > b")(store);
      auto rng = new Random(unpredictableSeed);

      assert( store.length == 0 );
      writeln("inserting...");

      for( uint i = 0 ; i < 100_000 ; i++)
      {
          minheap.insert(1);
          assert( store.length == 0 );
      }
      writeln("done.");
      assert( store.length == 0 );

      writeln("removing");
      while(!minheap.empty)
      {
          minheap.removeFront();
          assert( store.length == 0 );
      }
      writeln("done.");
      assert( store.length == 0 );
 }

 I thought the BinaryHeap would modify store's buffer AND it's
 properties, but it doesn't seem to be the case. How is store being used?


I'd guess a copy of store is used, since Array is a value type (using a 
ref-counted payload).


 Again this is the sort of thing that makes a beginner's learning curve
 steeper.


I wouldn't spend too much time on std.container. A sane collections 
library currently is not part of the standard distribution.


 Thanks for your attention
Jun 16 2013