digitalmars.D.learn - Better way to append to array than ~= ?
- Vladimirs Nordholm (30/30) Apr 03 2018 Hello people.
- Meta (8/38) Apr 03 2018 In this specific case, since you know the length of `Args`, you
- Vladimirs Nordholm (6/15) Apr 03 2018 I don't think I know the size of the arguments.
- Meta (11/29) Apr 03 2018 You know the static types of the arguments, so it is not
- Alex (8/26) Apr 03 2018 In my try, I iterate the args twice. The first time to calculate
- Vladimirs Nordholm (3/24) Apr 04 2018 Yeah, I will probably need to iterate twice.
- Jordan Wilson (14/44) Apr 04 2018 I believe you are right to question the ~= in regards to
Hello people.
I currently have a function which multiple times per second takes
in arguments, and appends the argument as my special type. The
following code should explain what I do more properly:
struct MySpecialType { char c; }
auto foo(Args...)(Args args)
{
MySpecialType[] bar;
foreach(ref arg; args)
{
static if(is(typeof(arg) == MySpecialType))
{
bar ~= arg;
}
else
{
foreach(c; to!string(arg))
{
bar ~= MySpecialType(c);
}
}
}
// do more stuff
}
Now, from my trace.log, some of the topmost things on the timing
list are `std.array.Appender!(immutable(char).<more stuff>`. I
also remember reading some years ago that ~= isn't optimal for
speed.
So my question is: Is there a better and/or faster way of doing
this, or is this the best approach?
Apr 03 2018
On Tuesday, 3 April 2018 at 19:02:25 UTC, Vladimirs Nordholm
wrote:
Hello people.
I currently have a function which multiple times per second
takes in arguments, and appends the argument as my special
type. The following code should explain what I do more properly:
struct MySpecialType { char c; }
auto foo(Args...)(Args args)
{
MySpecialType[] bar;
foreach(ref arg; args)
{
static if(is(typeof(arg) == MySpecialType))
{
bar ~= arg;
}
else
{
foreach(c; to!string(arg))
{
bar ~= MySpecialType(c);
}
}
}
// do more stuff
}
Now, from my trace.log, some of the topmost things on the
timing list are `std.array.Appender!(immutable(char).<more
stuff>`. I also remember reading some years ago that ~= isn't
optimal for speed.
So my question is: Is there a better and/or faster way of doing
this, or is this the best approach?
In this specific case, since you know the length of `Args`, you
can pre-allocate an array of that size and loop through it doing
your initialization.
However, if you want really performant code, you should allocate
a static array on the stack outside of the function and pass it
in as a buffer.
Apr 03 2018
On Tuesday, 3 April 2018 at 19:53:11 UTC, Meta wrote:On Tuesday, 3 April 2018 at 19:02:25 UTC, Vladimirs Nordholm wrote:I don't think I know the size of the arguments. If I pass in "123" and MySpecialType('a'), the result should be: assert(foo("123", MySpecialType('a')) == [MySpecialType('1'), MySpecialType('2'), MySpecialType('3'), MySpecialType('a')]); What should the length of the pre-allocated array be?[...]In this specific case, since you know the length of `Args`, you can pre-allocate an array of that size and loop through it doing your initialization. However, if you want really performant code, you should allocate a static array on the stack outside of the function and pass it in as a buffer.
Apr 03 2018
Meta <jared771 gmail.com> On Tuesday, 3 April 2018 at 20:02:46 UTC, Vladimirs Nordholm wrote:On Tuesday, 3 April 2018 at 19:53:11 UTC, Meta wrote:You know the static types of the arguments, so it is not impossible. However, the more flexible you need to be, the more complex your code will have to be, and it probably won't be worth the added complexity. Anyway, sorry for derailing a bit. That's not really your question. Using Appender!MySpecialType might be marginally faster than ~=, but from the looks of it you are already using Appender. I don't know if there's much more you can do to speed up appending without just rolling your own solution or restructuring your code.On Tuesday, 3 April 2018 at 19:02:25 UTC, Vladimirs Nordholm wrote:I don't think I know the size of the arguments. If I pass in "123" and MySpecialType('a'), the result should be: assert(foo("123", MySpecialType('a')) == [MySpecialType('1'), MySpecialType('2'), MySpecialType('3'), MySpecialType('a')]); What should the length of the pre-allocated array be?[...]In this specific case, since you know the length of `Args`, you can pre-allocate an array of that size and loop through it doing your initialization. However, if you want really performant code, you should allocate a static array on the stack outside of the function and pass it in as a buffer.
Apr 03 2018
On Tuesday, 3 April 2018 at 20:02:46 UTC, Vladimirs Nordholm wrote:On Tuesday, 3 April 2018 at 19:53:11 UTC, Meta wrote:In my try, I iterate the args twice. The first time to calculate the number of elements, then preallocate and then iterate them again and constructing the proper objects. It is not nice, but about 1/3 of time, compared to original version, compiled in release mode. https://run.dlang.io/is/E6ckogOn Tuesday, 3 April 2018 at 19:02:25 UTC, Vladimirs Nordholm wrote:I don't think I know the size of the arguments. If I pass in "123" and MySpecialType('a'), the result should be: assert(foo("123", MySpecialType('a')) == [MySpecialType('1'), MySpecialType('2'), MySpecialType('3'), MySpecialType('a')]); What should the length of the pre-allocated array be?[...]In this specific case, since you know the length of `Args`, you can pre-allocate an array of that size and loop through it doing your initialization. However, if you want really performant code, you should allocate a static array on the stack outside of the function and pass it in as a buffer.
Apr 03 2018
On Tuesday, 3 April 2018 at 20:41:01 UTC, Alex wrote:On Tuesday, 3 April 2018 at 20:02:46 UTC, Vladimirs Nordholm wrote:Yeah, I will probably need to iterate twice. Thanks a bunch 👍On Tuesday, 3 April 2018 at 19:53:11 UTC, Meta wrote:In my try, I iterate the args twice. The first time to calculate the number of elements, then preallocate and then iterate them again and constructing the proper objects. It is not nice, but about 1/3 of time, compared to original version, compiled in release mode. https://run.dlang.io/is/E6ckog[...]I don't think I know the size of the arguments. If I pass in "123" and MySpecialType('a'), the result should be: assert(foo("123", MySpecialType('a')) == [MySpecialType('1'), MySpecialType('2'), MySpecialType('3'), MySpecialType('a')]); What should the length of the pre-allocated array be?
Apr 04 2018
On Tuesday, 3 April 2018 at 19:02:25 UTC, Vladimirs Nordholm
wrote:
Hello people.
I currently have a function which multiple times per second
takes in arguments, and appends the argument as my special
type. The following code should explain what I do more properly:
struct MySpecialType { char c; }
auto foo(Args...)(Args args)
{
MySpecialType[] bar;
foreach(ref arg; args)
{
static if(is(typeof(arg) == MySpecialType))
{
bar ~= arg;
}
else
{
foreach(c; to!string(arg))
{
bar ~= MySpecialType(c);
}
}
}
// do more stuff
}
Now, from my trace.log, some of the topmost things on the
timing list are `std.array.Appender!(immutable(char).<more
stuff>`. I also remember reading some years ago that ~= isn't
optimal for speed.
So my question is: Is there a better and/or faster way of doing
this, or is this the best approach?
I believe you are right to question the ~= in regards to
performance. So, what I would do is replace this loop:
foreach(c; to!string(arg))
{
bar ~= MySpecialType(c);
}
with this one liner:
bar ~= arg.map!(a => MySpecialType(a.to!char)).array;
To my mind, you replace multiple appends with just the one append
(to be fair, I don't know what the .array is doing internally,
but I'm sure whatever it does is nice and optimised).
Jordan
Apr 04 2018