www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - Memory leak in rare cases using foreach and parallel

reply Guillaume Lathoud <gsub glat.info> writes:
Hello,

In a number-crunching app that tries many different configuration 
parameters, I observed in some rare cases a memory leak when 
doing:


     class Cfg { /* ...configuration parameters...*/ }

     auto cfg_range; // range that spits out the many 
configurations (instances of Cfg)

     foreach (cfg; parallel( cfg_range ))
     {
        // Do one computation for the set of parameters `cfg`
     }

using LDC 1.10.0 (based on dmd 2.080.1) on a linux machine.

This was difficult to reduce to a simple use case. The memory 
leak happened in rare cases, but in those cases, it always 
happened.

The workaround I found looked like this:

     foreach (i; parallel( cfg_range.length.iota ))
     {
       auto cfg = cfg_range[ i ];
       // Do one computation for the set of parameters `cfg`
     }

Hopefully this helps anyone encountering a similar issue and/or 
LDC developers.

Best regards,
Guillaume
Oct 13 2019
next sibling parent reply "David Nadlinger" <code klickverbot.at> writes:
Hi Guillaume,

On 14 Oct 2019, at 8:52, Guillaume Lathoud via digitalmars-d-ldc wrote:

 This was difficult to reduce to a simple use case. The memory leak 
 happened in rare cases, but in those cases, it always happened.


Just for future reference, do you know when the leak happened – once 
per execution? Once per thread? On every iteration?

And to clarify, by leak do you mean actual leaked memory, or excessive 
consumption of GC memory?

Best,
David
Oct 15 2019
parent reply Guillaume Lathoud <gsub glat.info> writes:
On Tuesday, 15 October 2019 at 20:36:05 UTC, David Nadlinger 
wrote:


 Just for future reference, do you know when the leak happened – 
 once per execution? Once per thread? On every iteration?

 And to clarify, by leak do you mean actual leaked memory, or 
 excessive consumption of GC memory?


In those particular cases, I observed in htop a growing memory 
usage for the whole executable. I recall that all threads were 
concerned. As soon as I switched to the iota implementation, the 
problem disappeared. I cannot say much more than this.

Best regards,
Guillaume
Oct 15 2019
parent reply Seb <seb wilzba.ch> writes:
On Wednesday, 16 October 2019 at 05:39:22 UTC, Guillaume Lathoud 
wrote:

 On Tuesday, 15 October 2019 at 20:36:05 UTC, David Nadlinger 
 wrote:


 Just for future reference, do you know when the leak happened 
 – once per execution? Once per thread? On every iteration?

 And to clarify, by leak do you mean actual leaked memory, or 
 excessive consumption of GC memory?


 In those particular cases, I observed in htop a growing memory 
 usage for the whole executable. I recall that all threads were 
 concerned. As soon as I switched to the iota implementation, 
 the problem disappeared. I cannot say much more than this.

 Best regards,
 Guillaume


Do you have by any chance a small reproducible example?
Oct 16 2019
parent reply Guillaume Lathoud <gsub glat.info> writes:
On Thursday, 17 October 2019 at 06:43:38 UTC, Seb wrote:


 Do you have by any chance a small reproducible example?


No. The use case was difficult to reduce, to reproduce the bug in 
a usable manner
If I manage to get one I'll definitely post it!

Best,
Guillaume
Oct 17 2019
parent Guillaume Lathoud <gsub glat.info> writes:
On Thursday, 17 October 2019 at 14:44:51 UTC, Guillaume Lathoud 
wrote:

 On Thursday, 17 October 2019 at 06:43:38 UTC, Seb wrote:


 Do you have by any chance a small reproducible example?


 No. The use case was difficult to reduce, to reproduce the bug 
 in a usable manner
 If I manage to get one I'll definitely post it!


I still could not reproduce this in a small example.

But here is an additional hint that helped me as well, and could 
help someone having similar memory usage issues, in the case of 
running long tasks using `parallel()`: run

core.memory.GC.collect();
core.memory.GC.minimize();

at the end of each task. See the example below.

Best regards,
Guillaume
.



import core.memory;
import core.thread;
import std.parallelism;
import std.range;
import std.stdio;

double[] do_one()
{
   auto arr = new double[ 50_000_000 ];

   Thread.sleep( 500.msecs );

   return arr;
}

void main()
{
   foreach (i; parallel( 100.iota ))
     {
       do_one; // Some long task (several seconds or minutes)

       // The next two lines divided the peak memory usage by a
       // factor of about 2 or 3
       core.memory.GC.collect();
       core.memory.GC.minimize();
     }

   writeln( "Done. Sleeping so that you can get the peak memory 
usage using `grep VmHWM /proc/<id>/status or similar..." );
   Thread.sleep( 10.seconds );
}
Dec 04 2019
prev sibling parent reply Kagamin <spam here.lot> writes:
On Monday, 14 October 2019 at 06:52:32 UTC, Guillaume Lathoud 
wrote:

 This was difficult to reduce to a simple use case. The memory 
 leak happened in rare cases, but in those cases, it always 
 happened.


By rare cases you mean that if a particular compiled executable 
always leaked? If it was recompiled (or relinked) from exactly 
the same source, the problem could disappear?
Dec 06 2019
parent reply Guillaume Lathoud <gsub glat.info> writes:
On Saturday, 7 December 2019 at 07:19:44 UTC, Kagamin wrote:


 By rare cases you mean that if a particular compiled executable 
 always leaked? If it was recompiled (or relinked) from exactly 
 the same source, the problem could disappear?


No, from what I remember the problem remained after recompiling 
everything. In those cases I ended up doing the change mentioned 
in the OP.
Dec 09 2019
parent leeja201912 <leeja201912 gmail.com> writes:
On Monday, 9 December 2019 at 13:18:29 UTC, Guillaume Lathoud 
wrote:

 On Saturday, 7 December 2019 at 07:19:44 UTC, Kagamin wrote:


 By rare cases you mean that if a particular compiled 
 executable always leaked? If it was recompiled (or relinked) 
 from exactly the same source, the problem could disappear?


 No, from what I remember the problem remained after recompiling 
 everything. In those cases I ended up doing the change 
 mentioned in the OP.
Dec 19 2019