www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Using closure causes GC allocation

reply Vino.B <vino.bheeman hotmail.com> writes:
Hi All,

    Request your help on how to solve the issue in the below code 
as when i execute the program with -vgc it state as below:

NewTD.d(21): vgc: using closure causes GC allocation
NewTD.d(25): vgc: array literal may cause GC allocation

void logClean (string[] Lglst, int LogAge) {   //Line 21
	if (!Lglst[0].exists) { mkdir(Lglst[0]); }
	auto ct1 = Clock.currTime();
	auto st1 = ct1 + days(-LogAge);
	auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a 
=> a.exists && a.isFile && a.timeCreated < st1).map!(a 
=>[a.name]).array;  // Line 25
	  dFiles.each!(f => f[0].remove);
}

From,
Vino.B
Sep 02
parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Saturday, 2 September 2017 at 17:43:08 UTC, Vino.B wrote:
 Hi All,

    Request your help on how to solve the issue in the below 
 code as when i execute the program with -vgc it state as below:

 NewTD.d(21): vgc: using closure causes GC allocation
 NewTD.d(25): vgc: array literal may cause GC allocation

 void logClean (string[] Lglst, int LogAge) {   //Line 21
 	if (!Lglst[0].exists) { mkdir(Lglst[0]); }
 	auto ct1 = Clock.currTime();
 	auto st1 = ct1 + days(-LogAge);
 	auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a 
 => a.exists && a.isFile && a.timeCreated < st1).map!(a 
 =>[a.name]).array;  // Line 25
 	  dFiles.each!(f => f[0].remove);
 }
Line 25 happens because of `[a.name]`. You request a new array: the memory for this has to be allocated (the reason why the compiler says "may" is because sometimes, e.g. if the array literal itself contains only literals, the allocations needn't happen at runtime and no GC call is necessary). Since you don't actually use the array, get rid of it: --- void logClean (string[] Lglst, int LogAge) { //Line 21 if (!Lglst[0].exists) { mkdir(Lglst[0]); } auto ct1 = Clock.currTime(); auto st1 = ct1 + days(-LogAge); auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a => a.exists && a.isFile && a.timeCreated < st1).array; // Line 25 dFiles.each!(f => f.remove); } --- I cannot reproduce the line 21 report, though. Since you use `timeCreated` I assume you're on Windows, but what's your D compiler, which D frontend version are you using, etc. (all the things needed to attempt to reproduce the error).
Sep 02
parent reply vino.b <vino.bheeman hotmail.com> writes:
On Saturday, 2 September 2017 at 18:02:06 UTC, Moritz Maxeiner 
wrote:
 On Saturday, 2 September 2017 at 17:43:08 UTC, Vino.B wrote:
 [...]
Line 25 happens because of `[a.name]`. You request a new array: the memory for this has to be allocated (the reason why the compiler says "may" is because sometimes, e.g. if the array literal itself contains only literals, the allocations needn't happen at runtime and no GC call is necessary). Since you don't actually use the array, get rid of it: [...]
Hi, Thank you for your help and the DMD version that i am using is DMD 2.076.0 and yes I am on windows. From, Vino.B
Sep 02
parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Saturday, 2 September 2017 at 18:08:19 UTC, vino.b wrote:
 On Saturday, 2 September 2017 at 18:02:06 UTC, Moritz Maxeiner 
 wrote:
 On Saturday, 2 September 2017 at 17:43:08 UTC, Vino.B wrote:
 [...]
Line 25 happens because of `[a.name]`. You request a new array: the memory for this has to be allocated (the reason why the compiler says "may" is because sometimes, e.g. if the array literal itself contains only literals, the allocations needn't happen at runtime and no GC call is necessary). Since you don't actually use the array, get rid of it: [...]
Hi, Thank you for your help and the DMD version that i am using is DMD 2.076.0 and yes I am on windows.
Please post a compilable, minimal example including how that function gets called that yields you that compiler output.
Sep 02
parent reply Vino.B <vino.bheeman hotmail.com> writes:
On Saturday, 2 September 2017 at 18:32:55 UTC, Moritz Maxeiner 
wrote:
 On Saturday, 2 September 2017 at 18:08:19 UTC, vino.b wrote:
 On Saturday, 2 September 2017 at 18:02:06 UTC, Moritz Maxeiner 
 wrote:
 On Saturday, 2 September 2017 at 17:43:08 UTC, Vino.B wrote:
 [...]
Line 25 happens because of `[a.name]`. You request a new array: the memory for this has to be allocated (the reason why the compiler says "may" is because sometimes, e.g. if the array literal itself contains only literals, the allocations needn't happen at runtime and no GC call is necessary). Since you don't actually use the array, get rid of it: [...]
Hi, Thank you for your help and the DMD version that i am using is DMD 2.076.0 and yes I am on windows.
Please post a compilable, minimal example including how that function gets called that yields you that compiler output.
Hi, Please find the example code below, import std.stdio: File,writeln; import std.datetime.systime: Clock, days, SysTime; import std.file: SpanMode, dirEntries, exists, isFile, mkdir, remove; import std.typecons: tuple; import std.algorithm: filter, map, each; import std.array: array; void logClean (string[] Lglst, int LogAge) { if (!Lglst[0].exists) { mkdir(Lglst[0]); } auto ct1 = Clock.currTime(); auto st1 = ct1 + days(-LogAge); auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a => a.exists && a.isFile && a.timeCreated < st1).map!(a => tuple(a.name)).array; dFiles.each!(a => a[0].remove); } void main () { string[] LogDir = ["C:\\Users\\bheev1\\Desktop\\Current\\Script\\D\\Logs"]; int LogAge = 1; logClean(LogDir,LogAge); } Another similar issue : I removed the [a.name] and the issue in line 25 has resolved, but for another function i am getting the same error string[][] cleanFiles(string FFs, string Step) { auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a => a.isFile).map!(a =>[a.name , a.timeCreated.toSimpleString[0 .. 20]]).array; -> Issue in this line if (Step == "run") dFiles.each!(a => a[0].remove); return dFiles; } if the replace the line in error as below then i am getting the error "Error: cannot implicitly convert expression dFiles of type Tuple!(string, string)[] to string[][]" auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a => a.isFile).map!(a => tuple(a.name, a.timeCreated.toSimpleString[0 .. 20])).array;
Sep 02
parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Saturday, 2 September 2017 at 18:59:30 UTC, Vino.B wrote:
 On Saturday, 2 September 2017 at 18:32:55 UTC, Moritz Maxeiner 
 wrote:
 On Saturday, 2 September 2017 at 18:08:19 UTC, vino.b wrote:
 On Saturday, 2 September 2017 at 18:02:06 UTC, Moritz 
 Maxeiner wrote:
 On Saturday, 2 September 2017 at 17:43:08 UTC, Vino.B wrote:
 [...]
Line 25 happens because of `[a.name]`. You request a new array: the memory for this has to be allocated (the reason why the compiler says "may" is because sometimes, e.g. if the array literal itself contains only literals, the allocations needn't happen at runtime and no GC call is necessary). Since you don't actually use the array, get rid of it: [...]
Hi, Thank you for your help and the DMD version that i am using is DMD 2.076.0 and yes I am on windows.
Please post a compilable, minimal example including how that function gets called that yields you that compiler output.
Hi, Please find the example code below, [...]
Cannot reproduce under Linux with dmd 2.076.0 (with commented out Windows-only check). I'll try to see what happens on Windows once I have a VM setup.
 Another similar issue :
  I removed the [a.name] and the issue in line 25 has resolved, 
 but for another function i am getting the same error

 string[][] cleanFiles(string FFs, string Step) {
 	auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a => 
 a.isFile).map!(a =>[a.name , a.timeCreated.toSimpleString[0 .. 
 20]]).array;  -> Issue in this line
     if (Step == "run")
         dFiles.each!(a => a[0].remove);
 		return dFiles;
 }

 if the replace the line in error as below then i am getting the 
 error "Error: cannot implicitly convert expression dFiles of 
 type Tuple!(string, string)[] to string[][]"

 auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a => 
 a.isFile).map!(a => tuple(a.name, 
 a.timeCreated.toSimpleString[0 .. 20])).array;
You changed the type of dFiles, which you return from cleanFiles, without changing the return type of cleanFiles. Change the return type of cleanFiles to the type the compiler error above tells you it should be (`Tuple!(string, string)[]` instead of `string[][]`), or let the compiler infer it via auto (`auto cleanFiles(...`).
Sep 02
parent reply Vino.B <vino.bheeman hotmail.com> writes:
On Saturday, 2 September 2017 at 20:10:58 UTC, Moritz Maxeiner 
wrote:
 On Saturday, 2 September 2017 at 18:59:30 UTC, Vino.B wrote:
 [...]
Cannot reproduce under Linux with dmd 2.076.0 (with commented out Windows-only check). I'll try to see what happens on Windows once I have a VM setup.
 [...]
You changed the type of dFiles, which you return from cleanFiles, without changing the return type of cleanFiles. Change the return type of cleanFiles to the type the compiler error above tells you it should be (`Tuple!(string, string)[]` instead of `string[][]`), or let the compiler infer it via auto (`auto cleanFiles(...`).
Hi, Thank you very much, was able to resolve the second code issue by changing the return type of the function.
Sep 02
parent reply Vino.B <vino.bheeman hotmail.com> writes:
On Saturday, 2 September 2017 at 20:54:03 UTC, Vino.B wrote:
 On Saturday, 2 September 2017 at 20:10:58 UTC, Moritz Maxeiner 
 wrote:
 On Saturday, 2 September 2017 at 18:59:30 UTC, Vino.B wrote:
 [...]
Cannot reproduce under Linux with dmd 2.076.0 (with commented out Windows-only check). I'll try to see what happens on Windows once I have a VM setup.
 [...]
You changed the type of dFiles, which you return from cleanFiles, without changing the return type of cleanFiles. Change the return type of cleanFiles to the type the compiler error above tells you it should be (`Tuple!(string, string)[]` instead of `string[][]`), or let the compiler infer it via auto (`auto cleanFiles(...`).
Hi, Thank you very much, was able to resolve the second code issue by changing the return type of the function.
Hi, In order to resolve the issue "Using closure causes GC allocation" it was stated that we need to use delegates, can you please help me on how to as i have not gone that much far in D programming. import std.stdio: File,writeln; import std.datetime.systime: Clock, days, SysTime; import std.file: SpanMode, dirEntries, exists, isFile, mkdir, remove; import std.typecons: tuple, Tuple; import std.algorithm: filter, map, each; import std.array: array; Tuple!(string)[] logClean (string[] Lglst, int LogAge) { if (!Lglst[0].exists) { mkdir(Lglst[0]); } auto ct1 = Clock.currTime(); auto st1 = ct1 + days(-LogAge); auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a => a.exists && a.isFile && a.timeCreated < st1).map!(a => tuple(a.name)).array; dFiles.each!(a => a[0].remove); return dFiles; } void main () { string[] LogDir = ["C:\\Users\\bheev1\\Desktop\\Current\\Script\\D\\Logs"]; int LogAge = 1; logClean(LogDir,LogAge); } From, Vino.B
Sep 03
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 4 September 2017 at 05:45:18 UTC, Vino.B wrote:
 On Saturday, 2 September 2017 at 20:54:03 UTC, Vino.B wrote:
 On Saturday, 2 September 2017 at 20:10:58 UTC, Moritz Maxeiner 
 wrote:
 On Saturday, 2 September 2017 at 18:59:30 UTC, Vino.B wrote:
 [...]
Cannot reproduce under Linux with dmd 2.076.0 (with commented out Windows-only check). I'll try to see what happens on Windows once I have a VM setup.
 [...]
You changed the type of dFiles, which you return from cleanFiles, without changing the return type of cleanFiles. Change the return type of cleanFiles to the type the compiler error above tells you it should be (`Tuple!(string, string)[]` instead of `string[][]`), or let the compiler infer it via auto (`auto cleanFiles(...`).
Hi, Thank you very much, was able to resolve the second code issue by changing the return type of the function.
Hi, In order to resolve the issue "Using closure causes GC allocation" it was stated that we need to use delegates, can you please help me on how to as i have not gone that much far in D programming. import std.stdio: File,writeln; import std.datetime.systime: Clock, days, SysTime; import std.file: SpanMode, dirEntries, exists, isFile, mkdir, remove; import std.typecons: tuple, Tuple; import std.algorithm: filter, map, each; import std.array: array; Tuple!(string)[] logClean (string[] Lglst, int LogAge) { if (!Lglst[0].exists) { mkdir(Lglst[0]); } auto ct1 = Clock.currTime(); auto st1 = ct1 + days(-LogAge); auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a => a.exists && a.isFile && a.timeCreated < st1).map!(a => tuple(a.name)).array; dFiles.each!(a => a[0].remove); return dFiles; } void main () { string[] LogDir = ["C:\\Users\\bheev1\\Desktop\\Current\\Script\\D\\Logs"]; int LogAge = 1; logClean(LogDir,LogAge); } From, Vino.B
If you are getting Using closure causes GC allocation from the above, it will be because you reference a local in the template delegate argument to filter: filter!(a => a.exists && a.isFile && a.timeCreated < st1) ^^^ because of the local (st1) it needs a closure. I can't remember how to transform delegates to take parameters and then call with the parameter set to the local, someone else will have to fill you in on that one.
Sep 04
prev sibling parent reply Azi Hassan <azi.hassan live.fr> writes:
On Monday, 4 September 2017 at 05:45:18 UTC, Vino.B wrote:
   In order to resolve the issue "Using closure causes GC 
 allocation" it was stated that we need to use delegates
Alternatively you can drop the functional style and use a foreach loop that doesn't require delegates, but you'd still need the GC to store the result in an array. And even then you could still use Array (from std.container).
Sep 04
parent Vino.B <vino.bheeman hotmail.com> writes:
On Monday, 4 September 2017 at 14:42:45 UTC, Azi Hassan wrote:
 On Monday, 4 September 2017 at 05:45:18 UTC, Vino.B wrote:
   In order to resolve the issue "Using closure causes GC 
 allocation" it was stated that we need to use delegates
Alternatively you can drop the functional style and use a foreach loop that doesn't require delegates, but you'd still need the GC to store the result in an array. And even then you could still use Array (from std.container).
Hi All, Was able to resolve this issue, thank you for your help, below is the changes that i did to resolve the issue. import std.stdio: File,writeln; import std.datetime.systime: Clock, days, SysTime; import std.file: SpanMode, dirEntries, exists, isFile, mkdir, remove; import std.typecons: tuple, Tuple; import std.algorithm: filter, map, each; import std.array: array; Tuple!(string)[] logClean (string[] Lglst, int LogAge) { if (!Lglst[0].exists) { mkdir(Lglst[0]); } auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a =a.exists && a.isFile && a.timeCreated < dtLogAge).map!(a =tuple(a.name)).array; dFiles.each!(a =a[0].remove); return dFiles; } SysTime dtLogAge () { int LogAge = mParams[1]; auto ct2 = Clock.currTime(); auto st2 = ct2 + days(-LogAge); return st2; } void main () { string[] LogDir = ["C:\\Users\\admin\\Desktop\\Current\\Script\\D\\Logs"]; logClean(LogDir); } "mParams" is another function that reads the value from the configuration file and returns the value for the LogAge as defined in the configuration file. From, Vino.B
Sep 05