digitalmars.D.bugs - [Issue 11021] New: Huge GC leak leads to crash; GC uses 7x more memory
- d-bugmail puremagic.com (28/28) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (15/15) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (9/16) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (7/7) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (14/17) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (6/6) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (10/11) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (12/20) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (17/32) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (18/26) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (11/38) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (8/17) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (6/6) Sep 13 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
- d-bugmail puremagic.com (12/13) Oct 03 2013 http://d.puremagic.com/issues/show_bug.cgi?id=11021
http://d.puremagic.com/issues/show_bug.cgi?id=11021 Summary: Huge GC leak leads to crash; GC uses 7x more memory Product: D Version: D2 Platform: All OS/Version: All Status: NEW Severity: blocker Priority: P2 Component: druntime AssignedTo: nobody puremagic.com ReportedBy: temtaime gmail.com import core.memory; import std.traits; void main() { while(true) { GC.collect(); new ubyte[40 * 1024 * 1024]; } } Application's memory usage grows and it crashes with message: core.exception.OutOfMemoryError If i comment import std.traits; everything goes OK, but now application uses 300 MM of memory. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=11021 monarchdodra gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |monarchdodra gmail.com Are you, by any chance, on win32? What is your metric for "GC uses 7x more memory"? Also, you say: "but now application uses 300 MM of memory." But is that a problem? I don't know what to make about your "import" issue, as I can't reproduce on linux. I'll try on windows. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=11021Are you, by any chance, on win32?I can reproduce on win32, all version of dmd.What is your metric for "GC uses 7x more memory"? Also, you say: "but now application uses 300 MM of memory." But is that a problem?Never mind, I understood what you meant.I don't know what to make about your "import" issue, as I can't reproduce on linux. I'll try on windows.I don't know how you did it, but I run out of memory regardless. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=11021 Yes, it's on win32. I think app should use about 50 mb as i'm allocating 40 mb array, not 300. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=11021 Orvid King <blah38621 gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |blah38621 gmail.comYes, it's on win32. I think app should use about 50 mb as i'm allocating 40 mb array, not 300.I would actually expect it to use 90mb, your allocating a 40mb array inside a loop, the previous 40mb array will still be referenced when you go to allocate the new 40mb array, meaning that it can't be collected if the second allocation triggers a GC. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=11021 In loop i calling collect, so gc should frees the memory. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=11021In loop i calling collect, so gc should frees the memory.You should really be clearing the existing array rather than allocating a new one, but that's beside the point. When you call the GC manually, the array is still referenced in local variable, if you were to reset that local variable before calling the GC, then it should collect it correctly. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=11021I would actually expect it to use 90mb, your allocating a 40mb array inside a loop, the previous 40mb array will still be referenced when you go to allocate the new 40mb array, meaning that it can't be collected if the second allocation triggers a GC.You should really be clearing the existing array rather than allocating a new one, but that's beside the point. When you call the GC manually, the array is still referenced in local variable, if you were to reset that local variable before calling the GC, then it should collect it correctly.He keeps no references to the array, so it should be collected immediatly on each call to GC.collect(). OR at least, there is no reason for it not to be collected. That said, I'm unsure "GC.collect()" promises it'll acutally *do* anything, it's more of a hint "now would be a good time to run a collection" (AFAIK). -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=11021 Dmitry Olshansky <dmitry.olsh gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dmitry.olsh gmail.com 11:51:16 PDT ---import core.memory; import std.traits; void main() { while(true) { GC.collect(); new ubyte[40 * 1024 * 1024]; } } Application's memory usage grows and it crashes with message: core.exception.OutOfMemoryError If i comment import std.traits; everything goes OK, but now application uses 300 MM of memory.32bits? Allocating large blocks in a loop? Good luck with that... sooner or later they would be pinned by something that looks like a pointer to GC. We are back to false pointers and precise collection topic. How std.traits affects that is interesting thing, maybe inflates executable size. Anyway this is game of chance so any slight change may avoid it or bring it back. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=1102132bits? Allocating large blocks in a loop? Good luck with that... sooner or later they would be pinned by something that looks like a pointer to GC. We are back to false pointers and precise collection topic. How std.traits affects that is interesting thing, maybe inflates executable size. Anyway this is game of chance so any slight change may avoid it or bring it back.2 things I don't understand: 1. New initializes, so the block contains nothing but 0x00's. How could that contain a false pointers? EDIT: Maybe the size/capacity info is wrongly seen as pointer? 2. Besides, the GC knows about NO_SCAN, doesn't it? We are explicitly asking for new ubytes, which is a non-pointer type. Why would the GC scan any of that at all anywas? If replace the call with GC.malloc(BIG_SIZE, GC.BlkAttr.NO_SCAN), then my win32 runs forever... -------- TLDR, I don't see how the false pointer issue in 32 bit environment would produce this failure. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=11021 13:22:18 PDT ---False pointers that HAPPEN TO POINT INTO THAT BLOCK. I can't imagine we are having this conversation again. The larger the block the better target it makes.32bits? Allocating large blocks in a loop? Good luck with that... sooner or later they would be pinned by something that looks like a pointer to GC. We are back to false pointers and precise collection topic. How std.traits affects that is interesting thing, maybe inflates executable size. Anyway this is game of chance so any slight change may avoid it or bring it back.2 things I don't understand: 1. New initializes, so the block contains nothing but 0x00's. How could that contain a false pointers? EDIT: Maybe the size/capacity info is wrongly seen aspointer? 2. Besides, the GC knows about NO_SCAN, doesn't it? We are explicitly asking for new ubytes, which is a non-pointer type. Why would the GC scan any of that at all anywas?Yes it SHOULDN'T scan THAT BLOCK.If replace the call with GC.malloc(BIG_SIZE, GC.BlkAttr.NO_SCAN), then my win32 runs forever...Or change something else and it might not... ? This is an erratic issue.-------- TLDR, I don't see how the false pointer issue in 32 bit environment would produce this failure.-- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=11021Ah. Right. I apologize for not being there the last time you had this conversation... -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------2 things I don't understand: 1. New initializes, so the block contains nothing but 0x00's. How could that contain a false pointers? EDIT: Maybe the size/capacity info is wrongly seen asFalse pointers that HAPPEN TO POINT INTO THAT BLOCK. I can't imagine we are having this conversation again. The larger the block the better target it makes.
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=11021 It's array of ubytes, not pointers. GC mustn't scan its content. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2013
http://d.puremagic.com/issues/show_bug.cgi?id=11021 safety0ff.bugz gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |safety0ff.bugz gmail.comIt's array of ubytes, not pointers. GC mustn't scan its content.I think you misunderstand, something in std.traits gets scanned and it is treated as a pointer into the array of ubytes. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 03 2013