www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Unable to use tradional tools to find memory leaks

reply "Flamaros" <flamaros.xavier gmail.com> writes:
I tried to used Valgrind (Linux) and Dr Memory (Windows) without 
success to find a big leak I have in my application.
But both tools can't launch my application without make it crash.

Do I need do something particular, to have a chance to see one of 
those tool working fine with my application?

It can be really hard to find leaks manually, maybe some option 
of the gc can help me?
Sep 21 2013
next sibling parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 21 September 2013 at 14:30:19 UTC, Flamaros wrote:
 I tried to used Valgrind (Linux) and Dr Memory (Windows) 
 without success to find a big leak I have in my application.
 But both tools can't launch my application without make it 
 crash.
Is application crashing without these tools? Probably you have some bug which is detected by valgrind. By the way, some pitfalls of using valgrind for testing D code: 1) Its implementation of float numbers at compile time is buggy (for example, there may be static asserts which are true when running under native envorionment and false under valgrind) 2) Dmd's codegen is sometimes not supported by valgrind (cannot execute some instructions which is rare case when something from D beats some tool outside) 3) It has some false positives regarding using uninitialized values especially during execution of GC code.
 Do I need do something particular, to have a chance to see one 
 of those tool working fine with my application?
If you run into #2 then you cannot fix it.
 It can be really hard to find leaks manually, maybe some option 
 of the gc can help me?
If you look into gc sources, you can found some testing code, but it needs druntime recompilation.
Sep 21 2013
next sibling parent "Flamaros" <flamaros.xavier gmail.com> writes:
On Saturday, 21 September 2013 at 16:01:17 UTC, Maxim Fomin wrote:
 On Saturday, 21 September 2013 at 14:30:19 UTC, Flamaros wrote:
 I tried to used Valgrind (Linux) and Dr Memory (Windows) 
 without success to find a big leak I have in my application.
 But both tools can't launch my application without make it 
 crash.
Is application crashing without these tools? Probably you have some bug which is detected by valgrind.
Application run correctly without Valgring or Dr Memory.
 By the way, some pitfalls of using valgrind for testing D code:

 1) Its implementation of float numbers at compile time is buggy 
 (for example, there may be static asserts which are true when 
 running under native envorionment and false under valgrind)
 2) Dmd's codegen is sometimes not supported by valgrind (cannot 
 execute some instructions which is rare case when something 
 from D beats some tool outside)
 3) It has some false positives regarding using uninitialized 
 values especially during execution of GC code.
Yes, I think it's normal, I got also some false leaks from graphics drivers. Sadly Valgrind make my application crash before the initialization is finished. I think I am in you 2 point.
 Do I need do something particular, to have a chance to see one 
 of those tool working fine with my application?
If you run into #2 then you cannot fix it.
 It can be really hard to find leaks manually, maybe some 
 option of the gc can help me?
If you look into gc sources, you can found some testing code, but it needs druntime recompilation.
Sep 21 2013
prev sibling parent reply Brad Roberts <braddr puremagic.com> writes:
On 9/21/13 9:01 AM, Maxim Fomin wrote:
 On Saturday, 21 September 2013 at 14:30:19 UTC, Flamaros wrote:
 I tried to used Valgrind (Linux) and Dr Memory (Windows) without success to
find a big leak I have
 in my application.
 But both tools can't launch my application without make it crash.
Is application crashing without these tools? Probably you have some bug which is detected by valgrind. By the way, some pitfalls of using valgrind for testing D code: 1) Its implementation of float numbers at compile time is buggy (for example, there may be static asserts which are true when running under native envorionment and false under valgrind) 2) Dmd's codegen is sometimes not supported by valgrind (cannot execute some instructions which is rare case when something from D beats some tool outside) 3) It has some false positives regarding using uninitialized values especially during execution of GC code.
 Do I need do something particular, to have a chance to see one of those tool
working fine with my
 application?
If you run into #2 then you cannot fix it.
That's wrong. Both DMD and Valgrind are software, both of which can be debugged and changed. Please file appropriate bug reports, hopefully with nicely minimized test cases.
Sep 21 2013
next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Saturday, 21 September 2013 at 17:03:14 UTC, Brad Roberts 
wrote:
 That's wrong.  Both DMD and Valgrind are software, both of 
 which can be debugged and changed. Please file appropriate bug 
 reports, hopefully with nicely minimized test cases.
I ran into an issue like this once where DMD would generate a strange instruction prefix on Linux x86_64 that was not supported by Valgrind. Never went on to check whether it was actually legal though. If I recall correctly, Valgrind reports details about the problematic instruction on the console before it terminates. If its an instruction encoding issue, just disassemble your program at/around the given address (e.g. using objdump -d) and post the result together with the error message. David
Sep 21 2013
prev sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 21 September 2013 at 17:03:14 UTC, Brad Roberts 
wrote:
 On 9/21/13 9:01 AM, Maxim Fomin wrote:
 On Saturday, 21 September 2013 at 14:30:19 UTC, Flamaros wrote:
 I tried to used Valgrind (Linux) and Dr Memory (Windows) 
 without success to find a big leak I have
 in my application.
 But both tools can't launch my application without make it 
 crash.
Is application crashing without these tools? Probably you have some bug which is detected by valgrind. By the way, some pitfalls of using valgrind for testing D code: 1) Its implementation of float numbers at compile time is buggy (for example, there may be static asserts which are true when running under native envorionment and false under valgrind) 2) Dmd's codegen is sometimes not supported by valgrind (cannot execute some instructions which is rare case when something from D beats some tool outside) 3) It has some false positives regarding using uninitialized values especially during execution of GC code.
 Do I need do something particular, to have a chance to see 
 one of those tool working fine with my
 application?
If you run into #2 then you cannot fix it.
That's wrong. Both DMD and Valgrind are software, both of which can be debugged and changed. Please file appropriate bug reports, hopefully with nicely minimized test cases.
No, that's true. DMD does produce some code which is executed by native envoronment but not by valrgind which is likely to be a valgrind bug. Sure, you can fix the issue belonging to a valgrind code if you are a valring developer, but this is not an option for most users.
Sep 21 2013
prev sibling parent reply "Flamaros" <flamaros.xavier gmail.com> writes:
On Saturday, 21 September 2013 at 14:30:19 UTC, Flamaros wrote:
 I tried to used Valgrind (Linux) and Dr Memory (Windows) 
 without success to find a big leak I have in my application.
 But both tools can't launch my application without make it 
 crash.

 Do I need do something particular, to have a chance to see one 
 of those tool working fine with my application?

 It can be really hard to find leaks manually, maybe some option 
 of the gc can help me?
Here is my Valkyrie output : ===valkyrie:3014=== DEBUG: utils/vk_config.cpp#253: static bool VkCfg::checkConfigDir(const QString&): ===valkyrie:3014=== No existing config dir '/tmp/valkyrie_flamaros/' => creating. ===valkyrie:3014=== ===valkyrie:3014=== DEBUG: utils/vk_config.cpp#702: void VkCfgProj::openProject(const QString&): ===valkyrie:3014=== Can't open project: File doesn't exist, or is not readable: '/home/flamaros/development/personnal/dquick/data/dquick.cfg' ===valkyrie:3014=== MainWindow::runTool( tool: 0, proc: 0 ) ===valkyrie:3014=== DEBUG: utils/vk_config.cpp#636: void VkCfgProj::replaceConfig(QSettings*): ===valkyrie:3014=== Replacing current config with: /home/flamaros/development/personnal/dquick/dquick.cfg ===valkyrie:3014=== MainWindow::runTool( tool: 0, proc: 0 ) Valkyrie::runTool( 0, 0) Current path: /home/flamaros/development/personnal/dquick/. [OpenGLContext] OpenGL Version: 2.1 Mesa 9.1.4 rootItem main vex amd64->IR: unhandled instruction bytes: 0x48 0xDF 0x1C 0x24 0x48 0xD9 0x6C 0x24 vex amd64->IR: REX=1 REX.W=1 REX.R=0 REX.X=0 REX.B=0 vex amd64->IR: VEX=0 VEX.L=0 VEX.nVVVV=0x0 ESC=NONE vex amd64->IR: PFX.66=0 PFX.F2=0 PFX.F3=0 ==3017== valgrind: Unrecognised instruction at address 0x5bad9a. ==3017== Your program just tried to execute an instruction that Valgrind ==3017== did not recognise. There are two possible reasons for this. ==3017== 1. Your program has a bug and erroneously jumped to a non-code ==3017== location. If you are running Memcheck and you just saw a ==3017== warning about a bad jump, it's probably your program's fault. ==3017== 2. The instruction is legitimate but Valgrind doesn't handle it, ==3017== i.e. it's Valgrind's fault. If you think this is the case or ==3017== you are not sure, please let us know and we'll try to fix it. ==3017== Either way, Valgrind will now raise a SIGILL signal which will ==3017== probably kill your program. ToolObject::processDone( 0, 1 ) ===valkyrie:3014=== DEBUG: objects/tool_object.cpp#687: void ToolObject::processDone(int, QProcess::ExitStatus): ===valkyrie:3014=== Error running VgProcess: process failed (1) ===valkyrie:3014=== ===valkyrie:3014=== DEBUG: objects/tool_object.cpp#714: void ToolObject::processDone(int, QProcess::ExitStatus): ===valkyrie:3014=== VgProcess finished with error: stop VgReader ===valkyrie:3014=== ===valkyrie:3014=== DEBUG: objects/tool_object.cpp#440: void ToolObject::stopProcess(): ===valkyrie:3014=== Stopping VgProcess ===valkyrie:3014=== ===valkyrie:3014=== DEBUG: objects/tool_object.cpp#488: void ToolObject::stopProcess(): ===valkyrie:3014=== VgProcess already stopped (or never started). ===valkyrie:3014=== PS : Code was generated with gdc, my application crash exactly like with dmd, so it doesn't seems to be a linker issue. Maybe a D spec issue or Valgrind one.
Sep 23 2013
parent reply "Flamaros" <flamaros.xavier gmail.com> writes:
On Monday, 23 September 2013 at 20:06:20 UTC, Flamaros wrote:
 On Saturday, 21 September 2013 at 14:30:19 UTC, Flamaros wrote:
 I tried to used Valgrind (Linux) and Dr Memory (Windows) 
 without success to find a big leak I have in my application.
 But both tools can't launch my application without make it 
 crash.

 Do I need do something particular, to have a chance to see one 
 of those tool working fine with my application?

 It can be really hard to find leaks manually, maybe some 
 option of the gc can help me?
Here is my Valkyrie output : ===valkyrie:3014=== DEBUG: utils/vk_config.cpp#253: static bool VkCfg::checkConfigDir(const QString&): ===valkyrie:3014=== No existing config dir '/tmp/valkyrie_flamaros/' => creating. ===valkyrie:3014=== ===valkyrie:3014=== DEBUG: utils/vk_config.cpp#702: void VkCfgProj::openProject(const QString&): ===valkyrie:3014=== Can't open project: File doesn't exist, or is not readable: '/home/flamaros/development/personnal/dquick/data/dquick.cfg' ===valkyrie:3014=== MainWindow::runTool( tool: 0, proc: 0 ) ===valkyrie:3014=== DEBUG: utils/vk_config.cpp#636: void VkCfgProj::replaceConfig(QSettings*): ===valkyrie:3014=== Replacing current config with: /home/flamaros/development/personnal/dquick/dquick.cfg ===valkyrie:3014=== MainWindow::runTool( tool: 0, proc: 0 ) Valkyrie::runTool( 0, 0) Current path: /home/flamaros/development/personnal/dquick/. [OpenGLContext] OpenGL Version: 2.1 Mesa 9.1.4 rootItem main vex amd64->IR: unhandled instruction bytes: 0x48 0xDF 0x1C 0x24 0x48 0xD9 0x6C 0x24 vex amd64->IR: REX=1 REX.W=1 REX.R=0 REX.X=0 REX.B=0 vex amd64->IR: VEX=0 VEX.L=0 VEX.nVVVV=0x0 ESC=NONE vex amd64->IR: PFX.66=0 PFX.F2=0 PFX.F3=0 ==3017== valgrind: Unrecognised instruction at address 0x5bad9a. ==3017== Your program just tried to execute an instruction that Valgrind ==3017== did not recognise. There are two possible reasons for this. ==3017== 1. Your program has a bug and erroneously jumped to a non-code ==3017== location. If you are running Memcheck and you just saw a ==3017== warning about a bad jump, it's probably your program's fault. ==3017== 2. The instruction is legitimate but Valgrind doesn't handle it, ==3017== i.e. it's Valgrind's fault. If you think this is the case or ==3017== you are not sure, please let us know and we'll try to fix it. ==3017== Either way, Valgrind will now raise a SIGILL signal which will ==3017== probably kill your program. ToolObject::processDone( 0, 1 ) ===valkyrie:3014=== DEBUG: objects/tool_object.cpp#687: void ToolObject::processDone(int, QProcess::ExitStatus): ===valkyrie:3014=== Error running VgProcess: process failed (1) ===valkyrie:3014=== ===valkyrie:3014=== DEBUG: objects/tool_object.cpp#714: void ToolObject::processDone(int, QProcess::ExitStatus): ===valkyrie:3014=== VgProcess finished with error: stop VgReader ===valkyrie:3014=== ===valkyrie:3014=== DEBUG: objects/tool_object.cpp#440: void ToolObject::stopProcess(): ===valkyrie:3014=== Stopping VgProcess ===valkyrie:3014=== ===valkyrie:3014=== DEBUG: objects/tool_object.cpp#488: void ToolObject::stopProcess(): ===valkyrie:3014=== VgProcess already stopped (or never started). ===valkyrie:3014=== PS : Code was generated with gdc, my application crash exactly like with dmd, so it doesn't seems to be a linker issue. Maybe a D spec issue or Valgrind one.
I found some leaks, but not the critical one. It seems my objects aren't correctly released when I simply set variables on a class to null. Calling explicitly clear() force immediate call of destructors. I certainly need dig into that way to find my issue. PS : MTuner can launch my application without make it crash, but it doesn't see memory of GC.
Sep 23 2013
parent reply "Flamaros" <flamaros.xavier gmail.com> writes:
On Monday, 23 September 2013 at 21:35:21 UTC, Flamaros wrote:
 On Monday, 23 September 2013 at 20:06:20 UTC, Flamaros wrote:
 On Saturday, 21 September 2013 at 14:30:19 UTC, Flamaros wrote:
 I tried to used Valgrind (Linux) and Dr Memory (Windows) 
 without success to find a big leak I have in my application.
 But both tools can't launch my application without make it 
 crash.

 Do I need do something particular, to have a chance to see 
 one of those tool working fine with my application?

 It can be really hard to find leaks manually, maybe some 
 option of the gc can help me?
Here is my Valkyrie output : ===valkyrie:3014=== DEBUG: utils/vk_config.cpp#253: static bool VkCfg::checkConfigDir(const QString&): ===valkyrie:3014=== No existing config dir '/tmp/valkyrie_flamaros/' => creating. ===valkyrie:3014=== ===valkyrie:3014=== DEBUG: utils/vk_config.cpp#702: void VkCfgProj::openProject(const QString&): ===valkyrie:3014=== Can't open project: File doesn't exist, or is not readable: '/home/flamaros/development/personnal/dquick/data/dquick.cfg' ===valkyrie:3014=== MainWindow::runTool( tool: 0, proc: 0 ) ===valkyrie:3014=== DEBUG: utils/vk_config.cpp#636: void VkCfgProj::replaceConfig(QSettings*): ===valkyrie:3014=== Replacing current config with: /home/flamaros/development/personnal/dquick/dquick.cfg ===valkyrie:3014=== MainWindow::runTool( tool: 0, proc: 0 ) Valkyrie::runTool( 0, 0) Current path: /home/flamaros/development/personnal/dquick/. [OpenGLContext] OpenGL Version: 2.1 Mesa 9.1.4 rootItem main vex amd64->IR: unhandled instruction bytes: 0x48 0xDF 0x1C 0x24 0x48 0xD9 0x6C 0x24 vex amd64->IR: REX=1 REX.W=1 REX.R=0 REX.X=0 REX.B=0 vex amd64->IR: VEX=0 VEX.L=0 VEX.nVVVV=0x0 ESC=NONE vex amd64->IR: PFX.66=0 PFX.F2=0 PFX.F3=0 ==3017== valgrind: Unrecognised instruction at address 0x5bad9a. ==3017== Your program just tried to execute an instruction that Valgrind ==3017== did not recognise. There are two possible reasons for this. ==3017== 1. Your program has a bug and erroneously jumped to a non-code ==3017== location. If you are running Memcheck and you just saw a ==3017== warning about a bad jump, it's probably your program's fault. ==3017== 2. The instruction is legitimate but Valgrind doesn't handle it, ==3017== i.e. it's Valgrind's fault. If you think this is the case or ==3017== you are not sure, please let us know and we'll try to fix it. ==3017== Either way, Valgrind will now raise a SIGILL signal which will ==3017== probably kill your program. ToolObject::processDone( 0, 1 ) ===valkyrie:3014=== DEBUG: objects/tool_object.cpp#687: void ToolObject::processDone(int, QProcess::ExitStatus): ===valkyrie:3014=== Error running VgProcess: process failed (1) ===valkyrie:3014=== ===valkyrie:3014=== DEBUG: objects/tool_object.cpp#714: void ToolObject::processDone(int, QProcess::ExitStatus): ===valkyrie:3014=== VgProcess finished with error: stop VgReader ===valkyrie:3014=== ===valkyrie:3014=== DEBUG: objects/tool_object.cpp#440: void ToolObject::stopProcess(): ===valkyrie:3014=== Stopping VgProcess ===valkyrie:3014=== ===valkyrie:3014=== DEBUG: objects/tool_object.cpp#488: void ToolObject::stopProcess(): ===valkyrie:3014=== VgProcess already stopped (or never started). ===valkyrie:3014=== PS : Code was generated with gdc, my application crash exactly like with dmd, so it doesn't seems to be a linker issue. Maybe a D spec issue or Valgrind one.
I found some leaks, but not the critical one. It seems my objects aren't correctly released when I simply set variables on a class to null. Calling explicitly clear() force immediate call of destructors. I certainly need dig into that way to find my issue. PS : MTuner can launch my application without make it crash, but it doesn't see memory of GC.
OK I just found my leak. That was not a real leak!!! A piece of code was turn off the GC and never put it back on.
Sep 23 2013
parent reply "growler" <growlercab gmail.com> writes:
Might be related to or even the same issue reported here:

http://forum.dlang.org/thread/bug-10054-3 http.d.puremagic.com/issues/

This is a Valgrind issue though and not DMD related.
Sep 23 2013
parent "Flamaros" <flamaros.xavier gmail.com> writes:
On Tuesday, 24 September 2013 at 01:09:29 UTC, growler wrote:
 Might be related to or even the same issue reported here:

 http://forum.dlang.org/thread/bug-10054-3 http.d.puremagic.com/issues/

 This is a Valgrind issue though and not DMD related.
It seems I had the same issue, but for Dr Memory there is no explanation because under Windows my application is build in 32bit.
Sep 23 2013