www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 429] New: Unable to distinguish between empty and uninitialized dynamic arrays

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=429

           Summary: Unable to distinguish between empty and uninitialized
                    dynamic arrays
           Product: D
           Version: unspecified
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: minor
          Priority: P4
         Component: Phobos
        AssignedTo: bugzilla digitalmars.com
        ReportedBy: ddparnell bigpond.com


When setting the length of a dynamic array to zero, phobos (gc) also frees the
RAM for that array. This means that applications cannot tell an uninitialized
array from an empty array. It also means that an application cannot reserve RAM
in the generic case. 

Note that reducing the length of an array to a value greater than zero does not
cause this behaviour, instead all the RAM for the array is reusable when the
application later increases the length.

Example:
   int[] test;
   test.length = 10;
   // Show address of array start and its length (10)
   writefln("%s %s", cast(uint)test.ptr, test.length);

   test.length = 1;
   // Show address of array start and its length (1)
   writefln("%s %s", cast(uint)test.ptr, test.length);

   test.length = 8;
   // Show address of array start and its length (8)
   writefln("%s %s", cast(uint)test.ptr, test.length);

   test.length = 0;
   // Shows 0 and 0!
   writefln("%s %s", cast(uint)test.ptr, test.length);

If an application needs to reset an array to the uninitialized state then it
can do 'arrayname = null;'.

The change to gc.d could be done as a two-line change ...

extern (C)
byte[] _d_arraysetlength2(size_t newlength, size_t sizeelem, Array *p, ...)
in
{
    assert(sizeelem);
    assert(!p.length || p.data);
}
body
{
    byte* newdata;

    debug(PRINTF)
    {
        printf("_d_arraysetlength2(p = %p, sizeelem = %d, newlength = %d)\n",
p, sizeelem, newlength);
        if (p)
            printf("\tp.data = %p, p.length = %d\n", p.data, p.length);
    }

    if (newlength)
    {
        version (D_InlineAsm_X86)
        {
            size_t newsize = void;

            asm
            {
                mov     EAX,newlength   ;
                mul     EAX,sizeelem    ;
                mov     newsize,EAX     ;
                jc      Loverflow       ;
            }
        }
        else
        {
            size_t newsize = sizeelem * newlength;

            if (newsize / newlength != sizeelem)
                goto Loverflow;
        }
        //printf("newsize = %x, newlength = %x\n", newsize, newlength);

        size_t size = p.length * sizeelem;
        if (p.data)  // <<-CHG
        {
            newdata = p.data;
            if (newlength > p.length)
            {
                size_t cap = _gc.capacity(p.data);

                if (cap <= newsize)
                {
                    newdata = cast(byte *)_gc.malloc(newsize + 1);
                    newdata[0 .. size] = p.data[0 .. size];
                }
            }
        }
        else
        {
            newdata = cast(byte *)_gc.malloc(newsize + 1);
        }

        va_list q;
        va_start!(Array *)(q, p);       // q is pointer to initializer

        if (newsize > size)
        {
            if (sizeelem == 1)
            {
                //printf("newdata = %p, size = %d, newsize = %d, *q = %d\n",
newdata, size, newsize, *cast(byte*)q);
                newdata[size .. newsize] = *(cast(byte*)q);
            }
            else
            {
                for (size_t u = size; u < newsize; u += sizeelem)
                {
                    memcpy(newdata + u, q, sizeelem);
                }
            }
        }
    }
    else
    {
      newdata = p.data;  //<<-CHG
    }

    p.data = newdata;
    p.length = newlength;
    return newdata[0 .. newlength];

Loverflow:
    _d_OutOfMemory();
}


-- 
Oct 11 2006
next sibling parent "Ameer Armaly" <ameer_armaly hotmail.com> writes:
<d-bugmail puremagic.com> wrote in message 
news:bug-429-3 http.d.puremagic.com/issues/...
 http://d.puremagic.com/issues/show_bug.cgi?id=429

           Summary: Unable to distinguish between empty and uninitialized
                    dynamic arrays
           Product: D
           Version: unspecified
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: minor
          Priority: P4
         Component: Phobos
        AssignedTo: bugzilla digitalmars.com
        ReportedBy: ddparnell bigpond.com


 When setting the length of a dynamic array to zero, phobos (gc) also frees 
 the
 RAM for that array. This means that applications cannot tell an 
 uninitialized
 array from an empty array. It also means that an application cannot 
 reserve RAM
 in the generic case.

 Note that reducing the length of an array to a value greater than zero 
 does not
 cause this behaviour, instead all the RAM for the array is reusable when 
 the
 application later increases the length.

 Example:
   int[] test;
   test.length = 10;
   // Show address of array start and its length (10)
   writefln("%s %s", cast(uint)test.ptr, test.length);

   test.length = 1;
   // Show address of array start and its length (1)
   writefln("%s %s", cast(uint)test.ptr, test.length);

   test.length = 8;
   // Show address of array start and its length (8)
   writefln("%s %s", cast(uint)test.ptr, test.length);

   test.length = 0;
   // Shows 0 and 0!
   writefln("%s %s", cast(uint)test.ptr, test.length);

 If an application needs to reset an array to the uninitialized state then 
 it
 can do 'arrayname = null;'.

 The change to gc.d could be done as a two-line change ...

 extern (C)
 byte[] _d_arraysetlength2(size_t newlength, size_t sizeelem, Array *p, 
 ...)
 in
 {
    assert(sizeelem);
    assert(!p.length || p.data);
 }
 body
 {
    byte* newdata;

    debug(PRINTF)
    {
        printf("_d_arraysetlength2(p = %p, sizeelem = %d, newlength = 
 %d)\n",
 p, sizeelem, newlength);
        if (p)
            printf("\tp.data = %p, p.length = %d\n", p.data, p.length);
    }

    if (newlength)
    {
        version (D_InlineAsm_X86)
        {
            size_t newsize = void;

            asm
            {
                mov     EAX,newlength   ;
                mul     EAX,sizeelem    ;
                mov     newsize,EAX     ;
                jc      Loverflow       ;
            }
        }
        else
        {
            size_t newsize = sizeelem * newlength;

            if (newsize / newlength != sizeelem)
                goto Loverflow;
        }
        //printf("newsize = %x, newlength = %x\n", newsize, newlength);

        size_t size = p.length * sizeelem;
        if (p.data)  // <<-CHG
        {
            newdata = p.data;
            if (newlength > p.length)
            {
                size_t cap = _gc.capacity(p.data);

                if (cap <= newsize)
                {
                    newdata = cast(byte *)_gc.malloc(newsize + 1);
                    newdata[0 .. size] = p.data[0 .. size];
                }
            }
        }
        else
        {
            newdata = cast(byte *)_gc.malloc(newsize + 1);
        }

        va_list q;
        va_start!(Array *)(q, p);       // q is pointer to initializer

        if (newsize > size)
        {
            if (sizeelem == 1)
            {
                //printf("newdata = %p, size = %d, newsize = %d, *q = 
 %d\n",
 newdata, size, newsize, *cast(byte*)q);
                newdata[size .. newsize] = *(cast(byte*)q);
            }
            else
            {
                for (size_t u = size; u < newsize; u += sizeelem)
                {
                    memcpy(newdata + u, q, sizeelem);
                }
            }
        }
    }
    else
    {
      newdata = p.data;  //<<-CHG
    }

    p.data = newdata;
    p.length = newlength;
    return newdata[0 .. newlength];

 Loverflow:
    _d_OutOfMemory();
 }


 -- 

setting length to 0 does nothing, but setting capacity to 0 frees memmory. Capacity would contain the maximum amount an array could hold without a new allocation, and if length is set greater than capacity a new allocation occurs and capacity is adjusted accordingly. This way the size of the memmory block and the actual array are two different things if you wanted to use them that way , but at the same time you can treat them as the same thing.
Oct 14 2006
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=429


bugzilla digitalmars.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED




------- Comment #2 from bugzilla digitalmars.com  2006-10-18 13:30 -------
Fixed DMD 0.170


-- 
Oct 18 2006
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=429


mk krej.cz changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |




------- Comment #3 from mk krej.cz  2006-11-08 15:36 -------
Linux DMD 0.173 seems to still have the original issue, Windows version works
ok.


-- 
Nov 08 2006
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=429


bugzilla digitalmars.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |FIXED




------- Comment #4 from bugzilla digitalmars.com  2006-11-25 03:52 -------
Fixed DMD 0.175


-- 
Nov 25 2006