www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D needs emplacement new

reply "Namespace" <rswhite4 gmail.com> writes:
Consider both of these scripts: test_gc_new.d 
(http://dpaste.dzfl.pl/4f36b165c502) and test_gc_emplace_new.d 
(http://dpaste.dzfl.pl/ff4e3c35479f)

and these results:

test_gc_new:

test_gc_new.exe "--DRT-gcopt=profile:1"
Number of collections: 140 Total GC prep time: 0 milliseconds Total mark time: 5 milliseconds Total sweep time: 9 milliseconds Total page recovery time: 2 milliseconds Max Pause Time: 0 milliseconds Grand total GC time: 18 milliseconds GC summary: 5 MB, 140 GC 18 ms, Pauses 6 ms < 0 ms test_gc_emplace_new:
test_gc_emplace_new.exe "--DRT-gcopt=profile:1"
Number of collections: 2 Total GC prep time: 0 milliseconds Total mark time: 0 milliseconds Total sweep time: 0 milliseconds Total page recovery time: 0 milliseconds Max Pause Time: 0 milliseconds Grand total GC time: 0 milliseconds GC summary: 5 MB, 2 GC 0 ms, Pauses 0 ms < 0 ms As you can see, thanks to the emplacement, I have 138 less GC collections. In C++ you can do new (f) Foo(...) instead of my New!(Foo)(f, ...) The C++ version is a lot more readable, nicer and safer. Therefore D should (since it says it is a system language like C++) add an equal syntax or, at least, implement a library version like mine to allow people a nice and easy way to reuse their object. We already have emplace (which my implementation uses) but to write emplace((cast(void*) obj)[0 .. __traits(classInstanceSize, Foo)], ...) is not nice, not intuitive and not safe and can be improved.
Apr 25 2015
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
You could just call the constructor again to reuse an object. I 
guess you should also reinitialize the memory, but that's pretty 
easy too.

Instead of placement new to reuse an object, make a function like 
reset() or reinitialize() that destructs the first, then copies 
the init back over and calls the constructor again.



Something like this:

void reinitialize(ClassName, CtorArgs...)(ClassName obj, CtorArgs 
args) {
     assert(typeid(obj) == typeid(ClassName),
         "Don't use this on interfaces or base classes!");
     static if(__traits(hasMember, obj, "__dtor"))
         obj.__dtor();
     (cast(void*) obj)[0 .. typeid(obj).init.length] = 
typeid(obj).init[];
     static if(__traits(hasMember, obj, "__ctor"))
         obj.__ctor(args);
}


Then, you create it however up front and just reinitialize it 
when you're all set. I think that is a bit clearer in meaning 
than placement new as well.
Apr 25 2015
next sibling parent reply "Namespace" <rswhite4 gmail.com> writes:
On Saturday, 25 April 2015 at 18:46:52 UTC, Adam D. Ruppe wrote:
 You could just call the constructor again to reuse an object. I 
 guess you should also reinitialize the memory, but that's 
 pretty easy too.

 Instead of placement new to reuse an object, make a function 
 like reset() or reinitialize() that destructs the first, then 
 copies the init back over and calls the constructor again.



 Something like this:

 void reinitialize(ClassName, CtorArgs...)(ClassName obj, 
 CtorArgs args) {
     assert(typeid(obj) == typeid(ClassName),
         "Don't use this on interfaces or base classes!");
     static if(__traits(hasMember, obj, "__dtor"))
         obj.__dtor();
     (cast(void*) obj)[0 .. typeid(obj).init.length] = 
 typeid(obj).init[];
     static if(__traits(hasMember, obj, "__ctor"))
         obj.__ctor(args);
 }


 Then, you create it however up front and just reinitialize it 
 when you're all set. I think that is a bit clearer in meaning 
 than placement new as well.
Nice name, fits better. But that should be in phobos, don't you think?
Apr 25 2015
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 25 April 2015 at 18:50:59 UTC, Namespace wrote:
 Nice name, fits better. But that should be in phobos, don't you 
 think?
meh, it might be nice to have, but not being in phobos isn't a big deal to me for little utilities like this.
Apr 25 2015
parent reply "Namespace" <rswhite4 gmail.com> writes:
On Saturday, 25 April 2015 at 18:57:13 UTC, Adam D. Ruppe wrote:
 On Saturday, 25 April 2015 at 18:50:59 UTC, Namespace wrote:
 Nice name, fits better. But that should be in phobos, don't 
 you think?
meh, it might be nice to have, but not being in phobos isn't a big deal to me for little utilities like this
I'm sure newcomers (especially those coming from C++) would be delighted if they had the possibility to reuse their memory. In addition, it could be impossible for them to write such "hack" itself. I already hear the cries: "D wasted memory" and: "I can not reuse my memory". And then it would be less attractive for such people, to switch to D. I would suggest something like: ---- T emplace(T, U...)(ref T obj, auto ref U args) if (is(T == class)) { if (!obj) return null; import std.conv : emplace; // temporary, as long it is not in std.conv enum ClassSizeOf = __traits(classInstanceSize, T); void[] buf = (cast(void*) obj)[0 .. ClassSizeOf]; return emplace!(T)(buf, args); } T emplaceOrNew(T, U...)(ref T obj, auto ref U args) if (is(T == class)) { if (!obj) return new T(args); return emplace(obj, args); } ----
Apr 25 2015
parent reply "weaselcat" <weaselcat gmail.com> writes:
On Saturday, 25 April 2015 at 19:09:15 UTC, Namespace wrote:
 On Saturday, 25 April 2015 at 18:57:13 UTC, Adam D. Ruppe wrote:
 On Saturday, 25 April 2015 at 18:50:59 UTC, Namespace wrote:
 Nice name, fits better. But that should be in phobos, don't 
 you think?
meh, it might be nice to have, but not being in phobos isn't a big deal to me for little utilities like this
I'm sure newcomers (especially those coming from C++) would be delighted if they had the possibility to reuse their memory. In addition, it could be impossible for them to write such "hack" itself. I already hear the cries: "D wasted memory" and: "I can not reuse my memory". And then it would be less attractive for such people, to switch to D. I would suggest something like: ---- T emplace(T, U...)(ref T obj, auto ref U args) if (is(T == class)) { if (!obj) return null; import std.conv : emplace; // temporary, as long it is not in std.conv enum ClassSizeOf = __traits(classInstanceSize, T); void[] buf = (cast(void*) obj)[0 .. ClassSizeOf]; return emplace!(T)(buf, args); } T emplaceOrNew(T, U...)(ref T obj, auto ref U args) if (is(T == class)) { if (!obj) return new T(args); return emplace(obj, args); } ----
hmm... http://dlang.org/phobos/std_conv.html#.emplace
Apr 25 2015
next sibling parent "weaselcat" <weaselcat gmail.com> writes:
On Saturday, 25 April 2015 at 19:14:41 UTC, weaselcat wrote:
 On Saturday, 25 April 2015 at 19:09:15 UTC, Namespace wrote:
 On Saturday, 25 April 2015 at 18:57:13 UTC, Adam D. Ruppe 
 wrote:
 On Saturday, 25 April 2015 at 18:50:59 UTC, Namespace wrote:
 Nice name, fits better. But that should be in phobos, don't 
 you think?
meh, it might be nice to have, but not being in phobos isn't a big deal to me for little utilities like this
I'm sure newcomers (especially those coming from C++) would be delighted if they had the possibility to reuse their memory. In addition, it could be impossible for them to write such "hack" itself. I already hear the cries: "D wasted memory" and: "I can not reuse my memory". And then it would be less attractive for such people, to switch to D. I would suggest something like: ---- T emplace(T, U...)(ref T obj, auto ref U args) if (is(T == class)) { if (!obj) return null; import std.conv : emplace; // temporary, as long it is not in std.conv enum ClassSizeOf = __traits(classInstanceSize, T); void[] buf = (cast(void*) obj)[0 .. ClassSizeOf]; return emplace!(T)(buf, args); } T emplaceOrNew(T, U...)(ref T obj, auto ref U args) if (is(T == class)) { if (!obj) return new T(args); return emplace(obj, args); } ----
hmm... http://dlang.org/phobos/std_conv.html#.emplace
woops, accidentally posted before finishing I was going to say that this should probably just be an overload of emplace if it isn't already, seems odd if it isn't.
Apr 25 2015
prev sibling parent reply "Namespace" <rswhite4 gmail.com> writes:
 hmm...
 http://dlang.org/phobos/std_conv.html#.emplace
 constructs an object of non-class type T at that address.
Non-Class. ;)
Apr 25 2015
parent reply "weaselcat" <weaselcat gmail.com> writes:
On Saturday, 25 April 2015 at 19:16:21 UTC, Namespace wrote:
 hmm...
 http://dlang.org/phobos/std_conv.html#.emplace
 constructs an object of non-class type T at that address.
Non-Class. ;)
There's a class overload 3 down T emplace(T, Args...)(void[] chunk, auto ref Args args) if (is(T == class));
Apr 25 2015
parent reply "Namespace" <rswhite4 gmail.com> writes:
On Saturday, 25 April 2015 at 19:21:28 UTC, weaselcat wrote:
 On Saturday, 25 April 2015 at 19:16:21 UTC, Namespace wrote:
 hmm...
 http://dlang.org/phobos/std_conv.html#.emplace
 constructs an object of non-class type T at that address.
Non-Class. ;)
There's a class overload 3 down T emplace(T, Args...)(void[] chunk, auto ref Args args) if (is(T == class));
Which accepts a void[] chunk: ---- Foo f = new Foo(...); emplace!(Foo)((cast(void*) f)[0 .. __traits(classInstanceSize, Foo)], ...); ---- That doesn't look like a "simple and friendly" manner for me. And it is also still not intuitive, not nice and not safe.
Apr 25 2015
parent reply "Namespace" <rswhite4 gmail.com> writes:
Since I'm unable to rebuild phobos on my Windows 8.1 PC, would 
someone else be interested, to apply a PR? The code (including 
comment) would be this:

----
/**
Given an existing object $(D obj), reinitialize the object of $(D 
class)
type $(D T) at that address. The constructor is passed the 
arguments
$(D Args), if any.

Returns: The reinitialized object
  */
T emplace(T, Args...)(ref T obj, auto ref Args args)
     if (is(T == class))
{
     enforce!ConvException(obj !is null, "emplace: Object is null 
and cannot be reinitialized");

     enum classSize = __traits(classInstanceSize, T);
     void[] buf = (cast(void*) obj)[0 .. classSize];
     return emplace!T(buf, args);
}
----

Unfortunately, I do not have much time to deal with the 
phobos-rebuilding problem, but I still want to improve phobos.
Apr 26 2015
parent reply "Namespace" <rswhite4 gmail.com> writes:
Since nobody wants to take that work, can at least someone 
explain me what is going on if I don't reinitialize the memorY?
I create 1000 Foo's 1000 times. After the first iteration there 
are 1000 unused Foo objects and the GC wants to reallocate 
another 1000 Foo's. Now, what happen? The GC sees that the 
previous 1000 objects are unused, remove them and allocate new 
and fresh memory for the current 1000 Foo's? Or does the GC see 
1000 unused Foo's and reuse the memory? How can I observe if the 
later is true? If I store the pointer to the object to compare 
them, the GC will recognize that and will not finalize the object.
Apr 27 2015
parent reply "anonymous" <anonymous example.com> writes:
On Monday, 27 April 2015 at 11:47:46 UTC, Namespace wrote:
 I create 1000 Foo's 1000 times. After the first iteration there 
 are 1000 unused Foo objects and the GC wants to reallocate 
 another 1000 Foo's. Now, what happen? The GC sees that the 
 previous 1000 objects are unused, remove them and allocate new 
 and fresh memory for the current 1000 Foo's? Or does the GC see 
 1000 unused Foo's and reuse the memory?
Collected memory will be reused for new allocations. The GC won't necessarily reuse the exact memory of the 1000 old Foos for the 1000 new Foos, but chances are it actually will.
 How can I observe if the later is true? If I store the pointer 
 to the object to compare them, the GC will recognize that and 
 will not finalize the object.
You can store the pointer as a size_t on the GC heap, and the GC will not regard it as a pointer. Alternatively, you can store it on the C heap, which is also ignored by the GC. Using size_t: ---- void main() { import std.stdio; import core.memory: GC; auto pointerInDisguise = new size_t; *pointerInDisguise = cast(size_t) cast(void*) new Object; /* Not sure why stomping is necessary, but without this, the first try below fails. */ static void stomp() {ubyte[1024] x;} stomp(); GC.collect(); writeln(cast(size_t) cast(void*) new Object == *pointerInDisguise); /* prints "true" => memory is reused */ GC.collect(); writeln(cast(size_t) cast(void*) new Object == *pointerInDisguise); /* prints "true" => memory is reused */ } ----
Apr 27 2015
next sibling parent reply "Casper =?UTF-8?B?RsOmcmdlbWFuZCI=?= <shorttail hotmail.com> writes:
On Monday, 27 April 2015 at 13:12:51 UTC, anonymous wrote:
 You can store the pointer as a size_t on the GC heap, and the 
 GC will not regard it as a pointer.
Doesn't the GC regard every 4/8 sequence of bytes as a pointer?
Apr 27 2015
parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 27 April 2015 at 13:29:03 UTC, Casper Færgemand wrote:
 Doesn't the GC regard every 4/8 sequence of bytes as a pointer?
Only on the stack. On the heap, it knows what type the data is and if it has a pointer or not.
Apr 27 2015
prev sibling parent "anonymous" <anonymous example.com> writes:
On Monday, 27 April 2015 at 13:12:51 UTC, anonymous wrote:
 ----
 void main()
 {
     import std.stdio;
     import core.memory: GC;

     auto pointerInDisguise = new size_t;
     *pointerInDisguise = cast(size_t) cast(void*) new Object;

     /* Not sure why stomping is necessary, but without this, 
 the first
     try below fails. */
Probably issue 9614 - Uninitialized holes in function stack frames confuses GC - https://issues.dlang.org/show_bug.cgi?id=9614
     static void stomp() {ubyte[1024] x;}
     stomp();
     GC.collect();
     writeln(cast(size_t) cast(void*) new Object == 
 *pointerInDisguise);
         /* prints "true" => memory is reused */

     GC.collect();
     writeln(cast(size_t) cast(void*) new Object == 
 *pointerInDisguise);
         /* prints "true" => memory is reused */
 }
 ----
May 01 2015
prev sibling parent reply "weaselcat" <weaselcat gmail.com> writes:
On Saturday, 25 April 2015 at 18:46:52 UTC, Adam D. Ruppe wrote:
 You could just call the constructor again to reuse an object. I 
 guess you should also reinitialize the memory, but that's 
 pretty easy too.

 Instead of placement new to reuse an object, make a function 
 like reset() or reinitialize() that destructs the first, then 
 copies the init back over and calls the constructor again.



 Something like this:

 void reinitialize(ClassName, CtorArgs...)(ClassName obj, 
 CtorArgs args) {
     assert(typeid(obj) == typeid(ClassName),
         "Don't use this on interfaces or base classes!");
     static if(__traits(hasMember, obj, "__dtor"))
         obj.__dtor();
     (cast(void*) obj)[0 .. typeid(obj).init.length] = 
 typeid(obj).init[];
     static if(__traits(hasMember, obj, "__ctor"))
         obj.__ctor(args);
 }


 Then, you create it however up front and just reinitialize it 
 when you're all set. I think that is a bit clearer in meaning 
 than placement new as well.
doesn't this have the issue of not calling the ctor/dtor of nested objects?
Apr 25 2015
parent "Adam D. Ruppe" <destructionator gmail.com> writes:
  On Saturday, 25 April 2015 at 19:08:52 UTC, weaselcat wrote:
 doesn't this have the issue of not calling the ctor/dtor of 
 nested objects?
You could also use .destroy(copy_of_obj); instead of __dtor which should handle that, I think. Constructors are fine because there's no default construction in D anyway - the object's this() would be calling them anyway.
Apr 25 2015