www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 4400] New: D2 GC doesn't allocate with 16 bytes alignment

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4400

           Summary: D2 GC doesn't allocate with 16 bytes alignment
           Product: D
           Version: D2
          Platform: x86
        OS/Version: Windows
            Status: NEW
          Severity: normal
          Priority: P2
         Component: druntime
        AssignedTo: sean invisibleduck.org
        ReportedBy: bearophile_hugs eml.cc



I think the D GC is supposed to allocate dynamic arrays with a 16 alignment,
but this program on dmd 2.047-Windows shows they are aligned to 8 if length >
255:


void main() {
    auto a = new double[256];
    assert((cast(size_t)a.ptr % 16) == 8);
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 28 2010
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4400


Steven Schveighoffer <schveiguy yahoo.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
                 CC|                            |schveiguy yahoo.com
         AssignedTo|sean invisibleduck.org      |schveiguy yahoo.com



05:52:58 PDT ---
If that is the case, I can change it.  Where does the spec state the GC must
align to 16 bytes?  If that isn't stated, should it be?  I.e. is there a reason
you need 16 byte alignment vs. 8 byte?

BTW, the size it must be above is supposed to be PAGESIZE/2, i.e. 2048 bytes. 
Actually the true size is 2047 because of how I store the length of the block
for appending.

The reason for doing so has to do with the ability of the GC to append
consecutive empty pages to an array pages.  I normally store the length of the
array at the end of the block, but if I did that, the "end of the block" may
change if such an append occurs, which raises the possibility of reading
arbitrary data as the length, so I store the length at the beginning of the
block.  For all blocks <= 2046 bytes, an append that expands the size will do a
copy of the data to a new block, so the block length remains correct.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 28 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4400




I have not found the thread where this was announced, you can ask to other
people about this. I remember a "recent" change in the D GC to ensure the 16
byte alignment of the dynamic arrays and structs.

The 16 bytes alignment was introduced because instructions like the SSE2 movapd
need 16 byte alignment:
http://en.wikipedia.org/wiki/MOVAPD

I have recently used it here:
http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=112670

And some other SSE* instructions work with 8 byte alignment too, but they are
slower (future CPUs can remove such alignment requirements, some of it has
being removed already, so in that future the GC can go back giving 8 bytes
aligned memory).

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 28 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4400




10:59:08 PDT ---
It was changed when I added the append patch.

By default the GC was 16-byte aligned, because all allocations below a page
were in some power of 2 increments, starting with 16 bytes.

The append patch needs to store the "used length" of the block somewhere in the
block itself.  What I discovered is that the array runtime functions stored a
padding byte at the end of every array.  This was to prevent pointers that
point to the end of the array from being considered as part of the next block
of memory.

So I used that byte to store the length.  I used it for blocks up to and
including 256 bytes (255 usable bytes).  For blocks 512 to 2048 bytes, I use 2
bytes to store the length.

For blocks page size and above, I use a full size_t.  However, it is possible
to append to an array and have it just commandeer empty pages after the used
pages, thereby changing the allocated block length.  This means that if I
stored the length at the end of the block, the length would have to be moved
when adding more pages, and the caching scheme could get messed up.  So I
decided to store the length at the front of the block.  In order to satisfy
alignment requirements, I had to add some padding to get the alignment to work.
 I asked around and was told that 8-byte alignment is sufficient.  If that is
wrong or makes sense to change, it is an easy fix.

I will bring up your concerns on the phobos mailing list and see what people
say.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 28 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4400




14:21:29 PDT ---
I'll change it to 16 bytes when I get a chance.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 28 2010
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4400


Steven Schveighoffer <schveiguy yahoo.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED



07:52:23 PDT ---
Changeset 328 (http://www.dsource.org/projects/druntime/changeset/328)

Don't use this changeset yet for production, I have made some alterations that
aren't finished, but the 16-byte alignment should work.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 13 2010