www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - strange memory dmd bug

reply Timon Gehr <timon.gehr gmx.ch> writes:
void main(){
     mixin({
             string r;
             foreach(i;0..12000) r~=q{mixin(q{{enum x="";}});};
             return r;
         }());
}

When I compile this program, DMD sometimes uses about 1GB of memory, 
sometimes about 2GB of memory and sometimes it makes my OS crash.

Can anyone reproduce this?
Sep 09 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/09/2011 10:30 PM, Timon Gehr wrote:
 void main(){
 mixin({
 string r;
 foreach(i;0..12000) r~=q{mixin(q{{enum x="";}});};
 return r;
 }());
 }

 When I compile this program, DMD sometimes uses about 1GB of memory,
 sometimes about 2GB of memory and sometimes it makes my OS crash.

 Can anyone reproduce this?

Similarly this: template TT(string s){enum TT=s;} void main(){ mixin({ string r; foreach(i;0..12000) r~=q{{enum x=TT!"";};}; return r; }()); } That does use a somewhat smaller amount of memory, but it is still huge and non-deterministic in size.
Sep 09 2011
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 10.09.2011 0:55, Timon Gehr wrote:
 On 09/09/2011 10:30 PM, Timon Gehr wrote:
 void main(){
 mixin({
 string r;
 foreach(i;0..12000) r~=q{mixin(q{{enum x="";}});};
 return r;
 }());
 }

 When I compile this program, DMD sometimes uses about 1GB of memory,
 sometimes about 2GB of memory and sometimes it makes my OS crash.

 Can anyone reproduce this?

Similarly this: template TT(string s){enum TT=s;} void main(){ mixin({ string r; foreach(i;0..12000) r~=q{{enum x=TT!"";};}; return r; }()); } That does use a somewhat smaller amount of memory, but it is still huge and non-deterministic in size.

I guess issue 6498, i.e. it's CTFE doing memory allocation on each append in foreach loop. http://d.puremagic.com/issues/show_bug.cgi?id=6498 As for as non-determinism, maybe heap fragmentation? -- Dmitry Olshansky
Sep 09 2011
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/09/2011 11:44 PM, Dmitry Olshansky wrote:
 On 10.09.2011 0:55, Timon Gehr wrote:
 On 09/09/2011 10:30 PM, Timon Gehr wrote:
 void main(){
 mixin({
 string r;
 foreach(i;0..12000) r~=q{mixin(q{{enum x="";}});};
 return r;
 }());
 }

 When I compile this program, DMD sometimes uses about 1GB of memory,
 sometimes about 2GB of memory and sometimes it makes my OS crash.

 Can anyone reproduce this?

Similarly this: template TT(string s){enum TT=s;} void main(){ mixin({ string r; foreach(i;0..12000) r~=q{{enum x=TT!"";};}; return r; }()); } That does use a somewhat smaller amount of memory, but it is still huge and non-deterministic in size.

I guess issue 6498, i.e. it's CTFE doing memory allocation on each append in foreach loop. http://d.puremagic.com/issues/show_bug.cgi?id=6498

Well, I don't get the issue if I don't mixin mixins or template instantiations. void main(){ mixin({ string r; foreach(i;0..12000) r~="{}{}{}{}{}{}{}{}"; return r; }()); }
 As for as non-determinism, maybe heap fragmentation?

That might well be it. Thanks!
Sep 09 2011
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, September 09, 2011 23:58:56 Timon Gehr wrote:
 On 09/09/2011 11:44 PM, Dmitry Olshansky wrote:
 On 10.09.2011 0:55, Timon Gehr wrote:
 On 09/09/2011 10:30 PM, Timon Gehr wrote:
 void main(){
 mixin({
 string r;
 foreach(i;0..12000) r~=q{mixin(q{{enum x="";}});};
 return r;
 }());
 }
 
 When I compile this program, DMD sometimes uses about 1GB of memory,
 sometimes about 2GB of memory and sometimes it makes my OS crash.
 
 Can anyone reproduce this?

Similarly this: template TT(string s){enum TT=s;} void main(){ mixin({ string r; foreach(i;0..12000) r~=q{{enum x=TT!"";};}; return r; }()); } That does use a somewhat smaller amount of memory, but it is still huge and non-deterministic in size.

I guess issue 6498, i.e. it's CTFE doing memory allocation on each append in foreach loop. http://d.puremagic.com/issues/show_bug.cgi?id=6498

Well, I don't get the issue if I don't mixin mixins or template instantiations. void main(){ mixin({ string r; foreach(i;0..12000) r~="{}{}{}{}{}{}{}{}"; return r; }()); }
 As for as non-determinism, maybe heap fragmentation?

That might well be it. Thanks!

Excessive memory consumption during compilation does not surprise me. While some bugs in that area have been fixed, there are still several issues with dmd using way too much memory (one of the main issues being that it never actually releases any of it). The non-determinism, however, is a bit odd. Heap fragmentation may be a good explanation though. I don't know. - Jonathan M Davis
Sep 09 2011