www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Weird performance degradation when using vector ops

reply Andrej Mitrovic <none none.none> writes:
I was testing out array concatenation and reassignment performance when I ran
into some weird performance degradation when using vector ops.

I have two dchar[][] buffers, one is initialized via vector ops, the other via
a foreach loop. Both are initialized to the same values, and they compare equal
via the assert.

After that, I compare concatenation and reassignment of both buffers. The
buffer which was initialized via vector ops operates much slower for some

On my machine, the following code gives these average results:
buffer1 done in usecs: 19820.
buffer2 done in usecs: 650.

Download: https://gist.github.com/965883
Pasted here:

import std.stdio;
import std.algorithm;
import std.array;
import std.datetime;

void testPerf(ref dchar[][] buffer)
    buffer = buffer[0..buffer.length/2] ~ "test"d.dup ~ buffer[buffer.length/2
.. $];

void main()
    dchar[][] buffer1;
    dchar[][] buffer2;
    buffer1.length = 100_000;
    buffer2.length = 100_000;
    buffer1[] = "foo"d.replicate(100).dup;
    foreach (ref buf; buffer2)
        buf = "foo"d.replicate(100).dup;
    assert(buffer1 == buffer2);    // will take a few seconds..
    StopWatch sw;
    writefln("buffer1 done in usecs: %s.", sw.peek().usecs);
    writefln("buffer2 done in usecs: %s.", sw.peek().usecs);

The actual initialization is much faster for vector ops than using the foreach
loop, but that's not what is being tested here.
May 10 2011
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Sorry, I don't this has anything to do with vector ops. Concatenation
always triggers memory allocation for new arrays, right? This could
explain the slowdown. In fact if I just test the performance of the
foreach-initialized buffer over 50 runs, I get results like:

buffer done in usecs: 608.
buffer done in usecs: 604.
buffer done in usecs: 607.
buffer done in usecs: 604.
buffer done in usecs: 608.
buffer done in usecs: 614.
buffer done in usecs: 613.
buffer done in usecs: 601.
buffer done in usecs: 596.
buffer done in usecs: 606.
buffer done in usecs: 607.
buffer done in usecs: 24353. <- boom

So this probably has something to do with memory and paging. :)
May 10 2011