www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - GC being too eager?

reply "Paulo Pinto" <pjmlp progtools.org> writes:
In a toy project I am working on with D v2.065, I came to the 
following situation:

Node path = solver.find (map, start, end);
if (path !is null) {
     path.writeContents();  <-- Access Violation
}

Apparently between the test and trying to use the class, the 
reference becomes null.

Quite strange in a single threaded application.

Any idea what might be happening or should I delve into assembly?

--
Paulo
Apr 10 2014
next sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:

 In a toy project I am working on with D v2.065, I came to the 
 following situation:

 Node path = solver.find (map, start, end);
 if (path !is null) {
     path.writeContents();  <-- Access Violation
 }

 Apparently between the test and trying to use the class, the 
 reference becomes null.

 Quite strange in a single threaded application.

 Any idea what might be happening or should I delve into 
 assembly?

 --
 Paulo


Delve delve delve. Is this in an optimised build?
Apr 10 2014
next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 10 April 2014 at 09:31:30 UTC, John Colvin wrote:

 On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:

 In a toy project I am working on with D v2.065, I came to the 
 following situation:

 Node path = solver.find (map, start, end);
 if (path !is null) {
    path.writeContents();  <-- Access Violation
 }

 Apparently between the test and trying to use the class, the 
 reference becomes null.

 Quite strange in a single threaded application.

 Any idea what might be happening or should I delve into 
 assembly?

 --
 Paulo


 Delve delve delve. Is this in an optimised build?


No, debug build.
Apr 10 2014
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 10 April 2014 at 09:31:30 UTC, John Colvin wrote:

 On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:

 In a toy project I am working on with D v2.065, I came to the 
 following situation:

 Node path = solver.find (map, start, end);
 if (path !is null) {
    path.writeContents();  <-- Access Violation
 }

 Apparently between the test and trying to use the class, the 
 reference becomes null.

 Quite strange in a single threaded application.

 Any idea what might be happening or should I delve into 
 assembly?

 --
 Paulo


 Delve delve delve. Is this in an optimised build?


Well, I got into something but it will require more research.

The above code is


0x0040223a e8ed360100       call        solver.find (0041592c)
0x0040223f 8945f0           mov         dword ptr [path],eax
0x00402242 83c40c           add         esp,0c
0x00402245 85c0             test        eax,eax
0x00402247 7408             je          D main+00000241 (00402251)
0x00402249 8b45f0           mov         eax,dword ptr [path]
0x0040224c 8b08             mov         ecx,dword ptr [eax]
0x0040224e ff5138           call        dword ptr [ecx+38]
0x00402251 ff3564e14400     push        dword ptr 
[__xt_z+000000f4 (0044e164)]
0x00402257 ff3560e14400     push        dword ptr 
[__xt_z+000000f0 (0044e160)]
0x0040225d 837df000         cmp         dword ptr [path],00
0x00402261 0f95c0           setne       al
0x00402264 e86b8d0000       call        
std.stdio.writefln!(string, bool).writefln (0040afd4)

After find() gets called, eax = 0x0030f900 which points to 
nowhere valid.

Hence, ecx will contain a null address and [ecx + 38] as vtable 
lookup points to 0x00000038, which lands in no man's land and 
boom.

Trying to get deep into this will require a bit more time.

I wonder if this is caused by the nodes being stored into a 
BinaryHeap with an Array store, coupled with a RedBlackTree as 
well. Maybe some interaction between the manual memory management 
and the GC.

More info later on.

--
Paulo
Apr 10 2014
prev sibling parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 10 April 2014 at 09:31:30 UTC, John Colvin wrote:

 On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:

 In a toy project I am working on with D v2.065, I came to the 
 following situation:

 Node path = solver.find (map, start, end);
 if (path !is null) {
    path.writeContents();  <-- Access Violation
 }

 Apparently between the test and trying to use the class, the 
 reference becomes null.

 Quite strange in a single threaded application.

 Any idea what might be happening or should I delve into 
 assembly?

 --
 Paulo


 Delve delve delve. Is this in an optimised build?


I found out the cause, apparently std.container.Array destroys 
the memory used by reference types as well (e.g. classes).

The small example below reproduces the error.

Apparently the way Array manages the memory on its own 
invalidates the reference pointed by node, as I assume bad == 
node, but its memory is invalid.

import std.stdio;
import std.container;


class Node {
}

Node indentity (Node n)
{
     return n;
}

Node badIndentity (Node n)
{
     Array!Node data;
     data.insert(n);
     return data.front();
}

int  main(string[] args)
{
     auto node = new Node();
     auto n = indentity (node);
     writefln("Hello the address is %s", n);
     auto bad = badIndentity (node);
     writefln("Hello the address is %s", bad);

     return 0;
}
Apr 10 2014
parent reply "Chris" <wendlec tcd.ie> writes:
On Friday, 11 April 2014 at 06:51:01 UTC, Paulo Pinto wrote:

 On Thursday, 10 April 2014 at 09:31:30 UTC, John Colvin wrote:

 On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:

 In a toy project I am working on with D v2.065, I came to the 
 following situation:

 Node path = solver.find (map, start, end);
 if (path !is null) {
   path.writeContents();  <-- Access Violation
 }

 Apparently between the test and trying to use the class, the 
 reference becomes null.

 Quite strange in a single threaded application.

 Any idea what might be happening or should I delve into 
 assembly?

 --
 Paulo


 Delve delve delve. Is this in an optimised build?


 I found out the cause, apparently std.container.Array destroys 
 the memory used by reference types as well (e.g. classes).

 The small example below reproduces the error.

 Apparently the way Array manages the memory on its own 
 invalidates the reference pointed by node, as I assume bad == 
 node, but its memory is invalid.

 import std.stdio;
 import std.container;


 class Node {
 }

 Node indentity (Node n)
 {
     return n;
 }

 Node badIndentity (Node n)
 {
     Array!Node data;
     data.insert(n);
     return data.front();
 }

 int  main(string[] args)
 {
     auto node = new Node();
     auto n = indentity (node);
     writefln("Hello the address is %s", n);
     auto bad = badIndentity (node);
     writefln("Hello the address is %s", bad);

     return 0;
 }


This reminds me of the question I had about the use of appender. 
(http://forum.dlang.org/thread/xfnvtlzyolmtncsmmqqi forum.dlang.org)

The internal memory management of containers and appender can be 
the source of subtle bugs. For cases like your badIndentity 
function, Objective-C has autorelease, so it's only released when 
the program is done with the value.
Out of interest, why would you write a function like badIndentity 
this way? Or is it just a proof of concept?
Apr 11 2014
parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Friday, 11 April 2014 at 09:32:29 UTC, Chris wrote:

 On Friday, 11 April 2014 at 06:51:01 UTC, Paulo Pinto wrote:

 On Thursday, 10 April 2014 at 09:31:30 UTC, John Colvin wrote:

 On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:

 In a toy project I am working on with D v2.065, I came to 
 the following situation:

 Node path = solver.find (map, start, end);
 if (path !is null) {
  path.writeContents();  <-- Access Violation
 }

 Apparently between the test and trying to use the class, the 
 reference becomes null.

 Quite strange in a single threaded application.

 Any idea what might be happening or should I delve into 
 assembly?

 --
 Paulo


 Delve delve delve. Is this in an optimised build?


 I found out the cause, apparently std.container.Array destroys 
 the memory used by reference types as well (e.g. classes).

 The small example below reproduces the error.

 Apparently the way Array manages the memory on its own 
 invalidates the reference pointed by node, as I assume bad == 
 node, but its memory is invalid.

 import std.stdio;
 import std.container;


 class Node {
 }

 Node indentity (Node n)
 {
    return n;
 }

 Node badIndentity (Node n)
 {
    Array!Node data;
    data.insert(n);
    return data.front();
 }

 int  main(string[] args)
 {
    auto node = new Node();
    auto n = indentity (node);
    writefln("Hello the address is %s", n);
    auto bad = badIndentity (node);
    writefln("Hello the address is %s", bad);

    return 0;
 }


 This reminds me of the question I had about the use of 
 appender. 
 (http://forum.dlang.org/thread/xfnvtlzyolmtncsmmqqi forum.dlang.org)

 The internal memory management of containers and appender can 
 be the source of subtle bugs. For cases like your badIndentity 
 function, Objective-C has autorelease, so it's only released 
 when the program is done with the value.
 Out of interest, why would you write a function like 
 badIndentity this way? Or is it just a proof of concept?



This was only to show you how to reproduce the error.

On my application I use Array as a backing store for a BinaryHeap 
used to store the nodes that are available on the A* open list.

If this usage of Array is correct and the destroy method is 
misbehaving, I can create a bug for it.

I assume Array should manage its own memory but not touch the 
memory that belongs to reference types stored on it. At least the 
documentation doesn't say otherwise.

--
Paulo
Apr 11 2014
next sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 11 April 2014 at 11:25:16 UTC, Paulo Pinto wrote:

 This was only to show you how to reproduce the error.

 On my application I use Array as a backing store for a 
 BinaryHeap used to store the nodes that are available on the A* 
 open list.

 If this usage of Array is correct and the destroy method is 
 misbehaving, I can create a bug for it.

 I assume Array should manage its own memory but not touch the 
 memory that belongs to reference types stored on it. At least 
 the documentation doesn't say otherwise.


Yes, it's bug. Trivial to fix too. Please file it.
Apr 11 2014
parent reply Nils =?UTF-8?B?Qm/Dn3VuZw==?= <nilsbossung googlemail.com> writes:
On Freitag, 11. April 2014 13:44, monarch_dodra wrote:


 Yes, it's bug. Trivial to fix too. Please file it.


I think I already filed it quite some time ago: 
https://issues.dlang.org/show_bug.cgi?id=6998

See also: http://stackoverflow.com/questions/22624021
Apr 11 2014
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 11 April 2014 at 12:52:52 UTC, Nils Boßung wrote:

 On Freitag, 11. April 2014 13:44, monarch_dodra wrote:


 Yes, it's bug. Trivial to fix too. Please file it.


 I think I already filed it quite some time ago:
 https://issues.dlang.org/show_bug.cgi?id=6998


I am appalled that such a trivial bug, one that leads to 
undefined behavior, could be left open for so long :/

No one to blame I guess, but... damn.
Apr 11 2014
parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Friday, 11 April 2014 at 13:02:01 UTC, monarch_dodra wrote:

 On Friday, 11 April 2014 at 12:52:52 UTC, Nils Boßung wrote:

 On Freitag, 11. April 2014 13:44, monarch_dodra wrote:


 Yes, it's bug. Trivial to fix too. Please file it.


 I think I already filed it quite some time ago:
 https://issues.dlang.org/show_bug.cgi?id=6998


 I am appalled that such a trivial bug, one that leads to 
 undefined behavior, could be left open for so long :/

 No one to blame I guess, but... damn.


No need to create duplicate entries then :)
Apr 11 2014
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 11 Apr 2014 07:25:15 -0400, Paulo Pinto <pjmlp progtools.org>  
wrote:


 On Friday, 11 April 2014 at 09:32:29 UTC, Chris wrote:

 On Friday, 11 April 2014 at 06:51:01 UTC, Paulo Pinto wrote:

 On Thursday, 10 April 2014 at 09:31:30 UTC, John Colvin wrote:

 On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:

 In a toy project I am working on with D v2.065, I came to the  
 following situation:

 Node path = solver.find (map, start, end);
 if (path !is null) {
  path.writeContents();  <-- Access Violation
 }

 Apparently between the test and trying to use the class, the  
 reference becomes null.

 Quite strange in a single threaded application.

 Any idea what might be happening or should I delve into assembly?

 --
 Paulo


 Delve delve delve. Is this in an optimised build?


 I found out the cause, apparently std.container.Array destroys the  
 memory used by reference types as well (e.g. classes).

 The small example below reproduces the error.

 Apparently the way Array manages the memory on its own invalidates the  
 reference pointed by node, as I assume bad == node, but its memory is  
 invalid.

 import std.stdio;
 import std.container;


 class Node {
 }

 Node indentity (Node n)
 {
    return n;
 }

 Node badIndentity (Node n)
 {
    Array!Node data;
    data.insert(n);
    return data.front();
 }

 int  main(string[] args)
 {
    auto node = new Node();
    auto n = indentity (node);
    writefln("Hello the address is %s", n);
    auto bad = badIndentity (node);
    writefln("Hello the address is %s", bad);

    return 0;
 }


 This reminds me of the question I had about the use of appender.  
 (http://forum.dlang.org/thread/xfnvtlzyolmtncsmmqqi forum.dlang.org)

 The internal memory management of containers and appender can be the  
 source of subtle bugs. For cases like your badIndentity function,  
 Objective-C has autorelease, so it's only released when the program is  
 done with the value.
 Out of interest, why would you write a function like badIndentity this  
 way? Or is it just a proof of concept?



 This was only to show you how to reproduce the error.

 On my application I use Array as a backing store for a BinaryHeap used  
 to store the nodes that are available on the A* open list.

 If this usage of Array is correct and the destroy method is misbehaving,  
 I can create a bug for it.

 I assume Array should manage its own memory but not touch the memory  
 that belongs to reference types stored on it. At least the documentation  
 doesn't say otherwise.


This is a side effect of classes being first-class reference types. Array  
needs to call destroy on structs, but not on classes. Sort of a "shallow  
destroy."

It's an interesting bug.

-Steve
Apr 11 2014
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 11 April 2014 at 12:26:14 UTC, Steven Schveighoffer 
wrote:

 This is a side effect of classes being first-class reference 
 types. Array needs to call destroy on structs, but not on 
 classes. Sort of a "shallow destroy."

 It's an interesting bug.

 -Steve


Yeah, the issue "transcends" the language. Things like 
"FieldTypeTuple" on a class (directly) also tell you what the 
class is made of.

Ideally, the call should have been "protected" by a 
"hasElaborateDestructor" to begin with. Not only does it make the 
code faster for types that don't have destructors, 
"hasElaborateDestructor" replies false for classes.

User  denis-sh had complained about "destroy" before for this 
very reason, and had proposed a "destruct":
https://github.com/D-Programming-Language/phobos/pull/929
Apr 11 2014
prev sibling parent reply "Rene Zwanenburg" <renezwanenburg gmail.com> writes:
On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:

 In a toy project I am working on with D v2.065, I came to the 
 following situation:

 Node path = solver.find (map, start, end);
 if (path !is null) {
     path.writeContents();  <-- Access Violation
 }

 Apparently between the test and trying to use the class, the 
 reference becomes null.

 Quite strange in a single threaded application.

 Any idea what might be happening or should I delve into 
 assembly?

 --
 Paulo


Can you post writeContents?
Apr 10 2014
parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 10 April 2014 at 10:37:58 UTC, Rene Zwanenburg wrote:

 On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:

 In a toy project I am working on with D v2.065, I came to the 
 following situation:

 Node path = solver.find (map, start, end);
 if (path !is null) {
    path.writeContents();  <-- Access Violation
 }

 Apparently between the test and trying to use the class, the 
 reference becomes null.

 Quite strange in a single threaded application.

 Any idea what might be happening or should I delve into 
 assembly?

 --
 Paulo


 Can you post writeContents?


Sure, but only today's evening. At work now.

This is a simple A* search program I was porting from C++ to D, 
and I plan to make it visible on Github anyway.

But it shouldn't be related to writeContents, I guess, as the 
access violation takes place regardless of the operation I try to 
apply to path.


--
Paulo
Apr 10 2014