digitalmars.D.learn - Regarding emplace, arrays, and helper functions
- Andrej Mitrovic (139/139) Aug 29 2013 The emplace docs state that the chunk where to store the class object
- bearophile (4/4) Aug 29 2013 A little related:
- =?UTF-8?B?QWxpIMOHZWhyZWxp?= (27/60) Aug 29 2013 I had experimented with this in a chapter (not in English yet):
- Andrej Mitrovic (14/16) Aug 30 2013 Excellent.
- =?UTF-8?B?QWxpIMOHZWhyZWxp?= (19/29) Aug 30 2013 I don't know the answer but I don't think it is specified even for C++.
- Namespace (6/6) Aug 30 2013 I hate this "magic" library fix.. Why cannot we use 'scope' for
- Andrej Mitrovic (3/9) Aug 30 2013 It would also avoid this nasty issue:
- Namespace (2/11) Aug 30 2013 Yes. :)
The emplace docs state that the chunk where to store the class object
instance needs to be aligned to the class type alignment. But it
doesn't say much on how to get this alignment from a class (we could
add a note about using the classInstanceAlignment template), or even
how to use it to create e.g. an array of class objects (not an array
of references).
Here's how one can run into a problem:
-----
import std.conv;
class C
{
this(ubyte b) { _b = b; }
ubyte _b;
}
void main()
{
// 9 bytes (x32), it's not going to align itself properly
enum Size = __traits(classInstanceSize, C);
ubyte[Size][2] buffer;
// off-topic: cast needed -> yet another emplace bug
auto obj1 = emplace!C(buffer[0], cast(ubyte)20);
assert(obj1._b == 20);
auto obj2 = emplace!C(buffer[1], cast(ubyte)20); // Boom!
assert(obj2._b == 20);
}
-----
On the second emplace call, an exception is thrown:
-----
std.conv.ConvException std\conv.d(3832): emplace: Misaligned memory
block (0x18FD41): it must be 4-byte aligned for type C
-----
So one has to figure out how to do alignment properly, but there's no
real documentation on how to do it. I had a look at the scoped()
implementation for reference, but it's quite complex, here's a
snippet:
-----
template scoped(T)
if (is(T == class))
{
// _d_newclass now use default GC alignment (looks like
(void*).sizeof * 2 for
// small objects). We will just use the maximum of filed alignments.
alias classInstanceAlignment!T alignment;
alias _alignUp!alignment aligned;
static struct Scoped
{
// Addition of `alignment` is required as `Scoped_store` can
be misaligned in memory.
private void[aligned(__traits(classInstanceSize, T) +
size_t.sizeof) + alignment] Scoped_store = void;
property inout(T) Scoped_payload() inout
{
void* alignedStore = cast(void*) aligned(cast(size_t)
Scoped_store.ptr);
// As `Scoped` can be unaligned moved in memory class
instance should be moved accordingly.
immutable size_t d = alignedStore - Scoped_store.ptr;
size_t* currD = cast(size_t*) &Scoped_store[$ - size_t.sizeof];
if(d != *currD)
{
import core.stdc.string;
memmove(alignedStore, Scoped_store.ptr + *currD,
__traits(classInstanceSize, T));
*currD = d;
}
return cast(inout(T)) alignedStore;
}
alias Scoped_payload this;
disable this();
disable this(this);
~this()
{
// `destroy` will also write .init but we have no
functions in druntime
// for deterministic finalization and memory releasing for now.
.destroy(Scoped_payload);
}
}
/// Returns the scoped object
system auto scoped(Args...)(auto ref Args args)
{
Scoped result = void;
void* alignedStore = cast(void*) aligned(cast(size_t)
result.Scoped_store.ptr);
immutable size_t d = alignedStore - result.Scoped_store.ptr;
*cast(size_t*) &result.Scoped_store[$ - size_t.sizeof] = d;
emplace!(Unqual!T)(result.Scoped_store[d .. $ - size_t.sizeof], args);
return result;
}
}
----
It uses a private _alignUp helper template as well:
-----
private size_t _alignUp(size_t alignment)(size_t n)
if(alignment > 0 && !((alignment - 1) & alignment))
{
enum badEnd = alignment - 1; // 0b11, 0b111, ...
return (n + badEnd) & ~badEnd;
}
-----
There is a lot of magic here that's hard to understand.
Anyway, since we already have emplace() as a public Phobos function, I
thought it might be good if we had another overload which created an
array of objects, or even a static array of objects wrapped in a
convenient random access range on top. Otherwise guaranteeing
alignment by hand seems to be complicated (judging from the scoped
template implementation).
I'm just pseudocoding here:
-----
class Point
{
this(int x, int y)
{
_x = x;
_y = y;
}
int _x;
int _y;
}
void main()
{
// arr is a struct instance which holds the aligned static array,
// which itself holds the Point objects.
// The struct has opIndex, alias this, etc, to simulate
// an array of references.
auto arr = emplaceStaticArray!(Point, 2); // 2 items
// note: one /could/ also allow calling the ctors in the above call,
// although the syntax could be tricky
arr[0].initialize(1, 2); // destroys existing state (if any),
calls the ctor
arr[1].initialize(3, 4); // ditto
Point point = arr[0]; // get the object reference via alias this
(just like in scoped)
assert(point.x == 1 && point.y == 2);
}
-----
An equivalent emplaceArray for dynamic (resizable) arrays could also
be implemented, where the memory can expand and shrink as necessary.
Thoughts? I know bearophile is probably interested. :)
Aug 29 2013
"bearophile" <bearophileHUGS lycos.com> A little related: http://d.puremagic.com/issues/show_bug.cgi?id=8873 Bye, bearophile
Aug 29 2013
On 08/29/2013 05:20 AM, Andrej Mitrovic wrote:
The emplace docs state that the chunk where to store the class object
instance needs to be aligned to the class type alignment. But it
doesn't say much on how to get this alignment from a class (we could
add a note about using the classInstanceAlignment template), or even
how to use it to create e.g. an array of class objects (not an array
of references).
Here's how one can run into a problem:
-----
import std.conv;
class C
{
this(ubyte b) { _b = b; }
ubyte _b;
}
void main()
{
// 9 bytes (x32), it's not going to align itself properly
enum Size = __traits(classInstanceSize, C);
ubyte[Size][2] buffer;
// off-topic: cast needed -> yet another emplace bug
auto obj1 = emplace!C(buffer[0], cast(ubyte)20);
assert(obj1._b == 20);
auto obj2 = emplace!C(buffer[1], cast(ubyte)20); // Boom!
assert(obj2._b == 20);
}
-----
On the second emplace call, an exception is thrown:
-----
std.conv.ConvException std\conv.d(3832): emplace: Misaligned memory
block (0x18FD41): it must be 4-byte aligned for type C
-----
So one has to figure out how to do alignment properly, but there's no
real documentation on how to do it.
I had experimented with this in a chapter (not in English yet):
http://ddili.org/ders/d/bellek_yonetimi.html
Here are two functions that I have just translated from that page (I see
that alignedAddress should better be alignUp):
T * alignedAddress(T)(T * candidateAddress)
out (result)
{
assert((cast(size_t)result % T.alignof) == 0);
}
body
{
return cast(T*)((cast(size_t)candidateAddress + T.alignof - 1)
/ T.alignof * T.alignof);
}
size_t paddedSize(T)()
{
static if (is (T == class)) {
size_t size = __traits(classInstanceSize, T);
} else {
size_t size = T.sizeof;
}
return cast(size_t)alignedAddress(cast(T*)size);
}
Now your program works with a single change:
enum Size = paddedSize!C();
Ali
Aug 29 2013
On 8/30/13, Ali =C7ehreli <acehreli yahoo.com> wrote:
Now your program works with a single change:
enum Size =3D paddedSize!C();
Excellent.
However will the compiler align all static arrays so their memory
begins at a proper offset?
Maybe a more appropriate question is: Is all stack data guaranteed to
be properly aligned, and on all platforms? For example:
enum Size =3D paddedSize!C;
ubyte[1] preBuffer;
ubyte[Size][2] buffer;
Is 'buffer' guaranteed to be aligned so its memory begins at a good
offset? There could be any number of bytes before 'buffer', such as
the preBuffer above it.
Tests show that they are indeed aligned, but I wonder if this is
something you can guarantee on all platforms?
Aug 30 2013
On 08/30/2013 07:02 AM, Andrej Mitrovic wrote:However will the compiler align all static arrays so their memory begins at a proper offset?The compiler considers only the element type of the static array.Maybe a more appropriate question is: Is all stack data guaranteed to be properly aligned, and on all platforms?I don't know the answer but I don't think it is specified even for C++.For example: enum Size = paddedSize!C; ubyte[1] preBuffer; ubyte[Size][2] buffer; Is 'buffer' guaranteed to be aligned so its memory begins at a good offset?I don't think so because the elements are ubytes and they can be placed any odd address. Perhaps it works on the stack but the following align(1) struct demonstrates that your buffer can be at an odd address: import std.stdio; align(1) struct S { ubyte[1] preBuffer; ubyte[16][2] buffer; } void main() { writeln(S.preBuffer.offsetof); // prints 0 writeln(S.buffer.offsetof); // prints 1 } Ali
Aug 30 2013
I hate this "magic" library fix.. Why cannot we use 'scope' for this? I know currently it is an unsafe feature that was also proposed for rejection, but it would look better, feel better, and the compiler could take the dirty work for us. And maybe better optimized.
Aug 30 2013
On 8/30/13, Namespace <rswhite4 googlemail.com> wrote:I hate this "magic" library fix.. Why cannot we use 'scope' for this? I know currently it is an unsafe feature that was also proposed for rejection, but it would look better, feel better, and the compiler could take the dirty work for us. And maybe better optimized.It would also avoid this nasty issue: http://d.puremagic.com/issues/show_bug.cgi?id=10921
Aug 30 2013
On Friday, 30 August 2013 at 14:45:40 UTC, Andrej Mitrovic wrote:On 8/30/13, Namespace <rswhite4 googlemail.com> wrote:Yes. :)I hate this "magic" library fix.. Why cannot we use 'scope' for this? I know currently it is an unsafe feature that was also proposed for rejection, but it would look better, feel better, and the compiler could take the dirty work for us. And maybe better optimized.It would also avoid this nasty issue: http://d.puremagic.com/issues/show_bug.cgi?id=10921
Aug 30 2013