www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 2927] New: Ignore Interior GC attribute

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2927

           Summary: Ignore Interior GC attribute
           Product: D
           Version: 2.029
          Platform: PC
        OS/Version: Windows
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: DMD
        AssignedTo: bugzilla digitalmars.com
        ReportedBy: dsimcha yahoo.com


A very easy, but often very effective, way to deal with false pointer issues in
the GC, would be to include an attribute in GC.BlkAttr that allows for interior
pointers to a block of memory to be ignored and only pointers to the root to be
able to keep the object alive.  

An example use case is when a class or struct object stores a large array that
never escapes the scope of the object.  The array will therefore always have a
pointer to its root if it is referenced.  Counting only pointers to the root as
references to the object would drastically lessen the probability that large
arrays are kept around due to false pointers.


-- 
May 03 2009
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2927





------- Comment #1 from dsimcha yahoo.com  2009-05-03 12:48 -------
Created an attachment (id=344)
 --> (http://d.puremagic.com/issues/attachment.cgi?id=344&action=view)
Adds interior pointer ignoring feature to gcx.d


-- 
May 03 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2927





------- Comment #2 from dsimcha yahoo.com  2009-05-03 12:49 -------
Created an attachment (id=345)
 --> (http://d.puremagic.com/issues/attachment.cgi?id=345&action=view)
Makes core.memory interface with modified gcx.d.


-- 
May 03 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2927





------- Comment #3 from dsimcha yahoo.com  2009-05-03 12:50 -------
I've submitted patches that implement this feature.  If the BlkAttr.NO_INTERIOR
flag is set, then only pointers to the base address of a GC-allocated block can
keep that block alive.  Here's a small test program demonstrating how this
works:

import std.stdio, core.memory;

void main() {
    // Test for < PAGESIZE
    test(32, cast(GC.BlkAttr) 0, 0);
    test(32, cast(GC.BlkAttr) 0, 1);
    test(32, GC.BlkAttr.NO_INTERIOR, 0);
    test(32, GC.BlkAttr.NO_INTERIOR, 1);

    // Test for > PAGESIZE
    enum megabyte = 1024 * 1024;
    test(megabyte, cast(GC.BlkAttr) 0, 0);
    test(megabyte, cast(GC.BlkAttr) 0, 1);
    test(megabyte, GC.BlkAttr.NO_INTERIOR, 0);
    test(megabyte, GC.BlkAttr.NO_INTERIOR, 1);
}

// In this test case, we expect the GC to allocate the same address more than
// once only when NO_INTERIOR is set and we store an interior pointer in ptrs
// instead of a pointer to the root address.
void test(size_t size, GC.BlkAttr attr, uint toAdd) {
    uint[] oldPtrs = new uint[10];
    void*[10] ptrs;
    foreach(i; 0..10) {
        auto ptr = GC.malloc(size, attr);
        ptrs[i] = ptr + toAdd;

        foreach(oldPtr; oldPtrs) {
            if(oldPtr == cast(uint) ptr) {
                writeln("GC re-allocated address ", oldPtr, ".  (Parameters: 
",
                        size, " ", attr, "  ", toAdd, ")");
                return;
            }
        }
        oldPtrs[i] = cast(uint) ptr;
        GC.collect();
    }
}


-- 
May 03 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2927


dsimcha yahoo.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #344 is|0                           |1
           obsolete|                            |




------- Comment #4 from dsimcha yahoo.com  2009-05-03 16:57 -------
Created an attachment (id=347)
 --> (http://d.puremagic.com/issues/attachment.cgi?id=347&action=view)
Fix a few things I missed in the first patch.


-- 
May 03 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2927



--- Comment #5 from Walter Bright <bugzilla digitalmars.com> 2009-11-02
23:44:29 PST ---
The problem is, I don't see how an interior pointer could be prevented. You can
always take a slice to the interior array, so there is an interior pointer. I
think it's a big risk to assume or tell the user not to slice the array,
because they will, and then will suffer from mysterious memory errors.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 02 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2927



--- Comment #6 from David Simcha <dsimcha yahoo.com> 2009-11-03 05:51:22 PST ---
I know this isn't completely safe, but neither is any alternative, such as
using the C heap, using the GC but deleting stuff manually, or having your
program leak memory like crazy.  In some cases, it may be the least bad
solution.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 03 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2927


Leandro Lucarella <llucax gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |llucax gmail.com


--- Comment #7 from Leandro Lucarella <llucax gmail.com> 2009-11-03 06:57:17
PST ---
(In reply to comment #5)
 The problem is, I don't see how an interior pointer could be prevented. You can
 always take a slice to the interior array, so there is an interior pointer. I
 think it's a big risk to assume or tell the user not to slice the array,
 because they will, and then will suffer from mysterious memory errors.

But this shouldn't be set on any array, only on some private data where you can ensure that no interior pointers exist. Example: ---- class A { private void* losts_of_pointers; this() { lots_of_pointers = new void*[100_000_000]; } } It would be nice to be able to tell the GC "trust me, nobody will ever store an interior pointer to this". -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Nov 03 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2927



--- Comment #8 from David Simcha <dsimcha yahoo.com> 2009-11-03 07:06:24 PST ---
Yes, and the killer use case would be associative arrays, which leak memory
like  a seive.  They are currently (and probably quite often) implemented as
something like:

struct Node {
    Node* left;
    Node* right;
    hash_t hash;
    K key;
    V value;
}

struct AA {
    Node*[] table;  // Can be huge, will always have a pointer to head as
                    // long as the AA is alive, fully owned by the AA 
                    // instance.  Perfect candidate for NO_INTERIOR.
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 03 2009
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2927



--- Comment #9 from David Simcha <dsimcha yahoo.com> 2010-11-04 07:19:16 PDT ---
Now that we have the concept of sealed containers, I think it's time to
reconsider this patch.  If necessary I'll redo it so it works with the current
GC.  One of the advantages of sealed containers is that, in some cases, they
could safely use ignore interior on their internal data structures.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 04 2010