www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 10593] New: array's reserve/capacity go haywire if length has been changed prior

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10593

           Summary: array's reserve/capacity go haywire if length has been
                    changed prior
           Product: D
           Version: unspecified
          Platform: x86
        OS/Version: Windows
            Status: NEW
          Severity: critical
          Priority: P2
         Component: druntime
        AssignedTo: nobody puremagic.com
        ReportedBy: monarchdodra gmail.com


--- Comment #0 from monarchdodra gmail.com 2013-07-10 03:45:32 PDT ---
On a win32 install on win7 64.

Array size must be bigger than 2047.

Not sure who is failing here (reserve? capacity?) but there is a big
discrepancy between the two. It may and/or may not also make subsequent appends
fail, or behave erratically.

My tests show capacity failing to report correctly. relocation being
indeterminate (I know it is not determinate behavior, but there's stink in my
tests), and reserve going out of control...

This is my test program:

//----
import std.array, std.stdio, core.memory;

void main()
{
    enum M = 2047;
    enum N = 4080;
    //Basic case, everything works correctly
    {
        ubyte[] a = new ubyte[](M);
        writefln("a.capacity before: %s", a.capacity);      //4079
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //8175
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //8175
        writefln("a.capacity after:  %s", a.capacity);      //8175
        auto b = a;
        b.length = N;
        writefln("Relocation after append?  %s", a.ptr !is b.ptr); //false
    }
    writeln();

    {
        ubyte[] a = new ubyte[](M);
        a ~= 1; // <= This little bastard here >:(
        writefln("a.capacity before: %s", a.capacity); //4079
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //8175
        writefln("a.capacity after: %s", a.capacity); //4079 !!!
        auto b = a;
        b.length = N;
        writefln("Relocation after append? %s", a.ptr !is b.ptr); //false
    }
    writeln();

    {
        ubyte[] a;
        a.length = M; // <= This little bastard here >:(
        writefln("a.capacity before: %s", a.capacity); //4079
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //8175
        writefln("a.capacity after: %s", a.capacity); //4079 !!!
        auto b = a;
        b.length = N;
        writefln("Relocation after append? %s", a.ptr !is b.ptr); //false
    }
    writeln();

    {
        ubyte[] a;
        a.length = M;
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //8175
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //12271 !!!
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //16367 !!!
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //20463 !!!
        writefln("a.capacity after: %s", a.capacity); //4079 !!!
    }
    writeln();
}
//----

And corresponding output (!!! added manually).

//----
a.capacity before: 4079
a.reserve(4080): 8175
a.reserve(4080): 8175
a.capacity after:  8175
Relocation after append?  false

a.capacity before: 4079
a.reserve(4080): 8175
a.capacity after: 4079
Relocation after append? false

a.capacity before: 4079
a.reserve(4080): 8175
a.capacity after: 4079
Relocation after append? false

a.reserve(4080): 8175
a.reserve(4080): 12271
a.reserve(4080): 16367
a.reserve(4080): 20463
a.capacity after: 4079
//----

To trigger, M must be at least as big as 2047, and N must be enough to require
work.

Marking as critical, as this is really a core feature of D, and *really*
getting in the way of some of my array/appender fixes.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 10 2013
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10593



--- Comment #1 from monarchdodra gmail.com 2013-07-10 05:05:14 PDT ---
Note: The same behavior happens with GC.extend too: extend will succeed, but
the reported capacity will not be updated.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 10 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10593


monarchdodra gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Platform|x86                         |All
         OS/Version|Windows                     |All
           Severity|critical                    |blocker


--- Comment #2 from monarchdodra gmail.com 2013-07-15 13:40:35 PDT ---
I'm marking this as blocker.

The fact that a reserve could fail, while still doing something, is highly
problematic.

I also believe this is the issue creating the failures here:
https://github.com/D-Programming-Language/phobos/pull/1413

Will provide more information as I can.

Also: Confirmed on all platforms.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 15 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10593


Martin Nowak <code dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code dawg.eu


--- Comment #3 from Martin Nowak <code dawg.eu> 2013-07-20 09:58:49 PDT ---
It seems that reserve doesn't update the capacity.
If bug 6372 is not the reason for this you might want to look at the array
blockinfo cache.

reduced test case
---
cat > bug.d << CODE
void main()
{
    enum M = 4079;
    ubyte[] a = new ubyte[](M - 1);
    a ~= 1;
    immutable u = a.reserve(M+1);
    assert(u == a.capacity);
}
CODE
dmd -run bug
---

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 20 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10593


Rainer Schuetze <r.sagitario gmx.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |r.sagitario gmx.de


--- Comment #4 from Rainer Schuetze <r.sagitario gmx.de> 2013-07-21 00:34:21
PDT ---
There is an update missing to the blkinfo cache in the array extending:

https://github.com/D-Programming-Language/druntime/pull/548

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 21 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10593



--- Comment #5 from github-bugzilla puremagic.com 2013-07-21 00:45:27 PDT ---
Commit pushed to master at https://github.com/D-Programming-Language/druntime

https://github.com/D-Programming-Language/druntime/commit/4f0e73f1fdcd8c8c3898089f3ae1707f77ff857d
Merge pull request #548 from rainers/fix10593

fix  Issue 10593 - array's reserve/capacity go haywire if length has been
changed prior

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 21 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10593


monarchdodra gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 21 2013