digitalmars.D - Disappointing performance from DMD/Phobos
- Manu (32/32) Jun 25 2018 Some code:
- David Bennett (12/25) Jun 25 2018 Last time I checked, dmd's inliner would give up as soon as it
- Nicholas Wilson (5/45) Jun 25 2018 Then use LDC! ;)
- Manu (3/4) Jun 26 2018 Keep LDC up to date with DMD master daily! ;)
- Iain Buclaw (4/9) Jun 26 2018 Like what GDC is doing (almost) ;-)
- Manu (9/18) Jun 26 2018 Orly? Have you gotten GDC to a point where you can actively keep up to
- Eugene Wissner (19/41) Jun 26 2018 D frontend was merged into GDC master. It is version 2.076 now.
- Iain Buclaw (5/45) Jun 26 2018 Well, once in sync with dmd/stable. A weekly (or more frequent) check
- Iain Buclaw (4/21) Jun 26 2018 Not quite, there are some funny bugs I'm looking at, some of which
- Manu (4/27) Jun 26 2018 You love it! They're spectacular patches! :P
- Per =?UTF-8?B?Tm9yZGzDtnc=?= (2/3) Jun 27 2018 Agreed!
Some code:
---------------------------------
struct Entity
{
enum NumSystems = 4;
struct SystemData
{
uint start, length;
}
SystemData[NumSystems] systemData;
property uint systemBits() const { return systemData[].map!(e =>
e.length).sum; }
}
Entity e;
e.systemBits(); // <- call the function, notice the codegen
---------------------------------
This property sum's 4 ints... that should be insanely fast. It should
also be something like 5-8 lines of asm.
Turns out, that call to sum() is eating 2.5% of my total perf
(significant among a substantial workload), and the call tree is quite
deep.
Basically, inliner tried, but failed to seal the deal, and leaves a
call stack 7 levels deep.
Pipeline programming is hip and also *recommended* D usage. The
optimiser must do a good job. This is such a trivial workloop, and
with constant length (4).
I expect 3 integer adds to unroll and inline. A call-tree 7 levels
deep is quite a ways from the mark.
Maybe this is another instance of Walter's "phobos begat madness" observation?
The unoptimised callstack is mental. Compiling with -O trims most of
the noise in the call tree, but it fails to inline the remaining work
which ends up 7-levels down a redundant call-tree.
Jun 25 2018
David Bennett <davidbennett bravevision.com>
On Tuesday, 26 June 2018 at 02:10:17 UTC, Manu wrote:
[snip]
property uint systemBits() const { return
systemData[].map!(e =>
e.length).sum; } [snip]
This property sum's 4 ints... that should be insanely fast. It
should
also be something like 5-8 lines of asm.
Turns out, that call to sum() is eating 2.5% of my total perf
(significant among a substantial workload), and the call tree
is quite
deep.
Basically, inliner tried, but failed to seal the deal, and
leaves a call stack 7 levels deep.
Last time I checked, dmd's inliner would give up as soon as it
sees any type of loop, even the simplest while loop... then the
unroller and optimiser later on have less to work with.
So I would expect it's the loop in `sumPairwise()` [0] or
`sumKahan()` [1] that's the main source of your problems.
If we could get that to inline better using `dmd -inline` it
would probably speed up quite a lot of code.
[0]
https://github.com/dlang/phobos/blob/master/std/algorithm/iteration.d#L5483
[1]
https://github.com/dlang/phobos/blob/master/std/algorithm/iteration.d#L5578
Jun 25 2018
On Tuesday, 26 June 2018 at 02:10:17 UTC, Manu wrote:
Some code:
---------------------------------
struct Entity
{
enum NumSystems = 4;
struct SystemData
{
uint start, length;
}
SystemData[NumSystems] systemData;
property uint systemBits() const { return
systemData[].map!(e =>
e.length).sum; }
}
Entity e;
e.systemBits(); // <- call the function, notice the codegen
---------------------------------
This property sum's 4 ints... that should be insanely fast. It
should
also be something like 5-8 lines of asm.
Turns out, that call to sum() is eating 2.5% of my total perf
(significant among a substantial workload), and the call tree
is quite
deep.
Basically, inliner tried, but failed to seal the deal, and
leaves a call stack 7 levels deep.
Pipeline programming is hip and also *recommended* D usage. The
optimiser must do a good job. This is such a trivial workloop,
and
with constant length (4).
I expect 3 integer adds to unroll and inline. A call-tree 7
levels
deep is quite a ways from the mark.
Maybe this is another instance of Walter's "phobos begat
madness" observation?
The unoptimised callstack is mental. Compiling with -O trims
most of
the noise in the call tree, but it fails to inline the
remaining work
which ends up 7-levels down a redundant call-tree.
Then use LDC! ;)
But seriously, DMD's inliner is a) in the wrong spot in the
compilation pipeline (at the AST level) and b) is timid to say
the least.
Jun 25 2018
On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via Digitalmars-d <digitalmars-d puremagic.com> wrote:Then use LDC! ;)Keep LDC up to date with DMD master daily! ;)
Jun 26 2018
On 26 June 2018 at 19:41, Manu via Digitalmars-d <digitalmars-d puremagic.com> wrote:On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via Digitalmars-d <digitalmars-d puremagic.com> wrote:Like what GDC is doing (almost) ;-) Iain.Then use LDC! ;)Keep LDC up to date with DMD master daily! ;)
Jun 26 2018
On Tue, 26 Jun 2018 at 10:43, Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> wrote:On 26 June 2018 at 19:41, Manu via Digitalmars-d <digitalmars-d puremagic.com> wrote:Orly? Have you gotten GDC to a point where you can actively keep up to date? You should definitely advertise that better, I think most users still presume that GDC is a few revisions behind latest. Do you have nightly builds that are close to the DMD nightlies? Sadly we're working with the MSVC ABI, and I don't think GCC has a backend for that arch do they? LLVM has done a lot of work to interop with MSVC.On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via Digitalmars-d <digitalmars-d puremagic.com> wrote:Like what GDC is doing (almost) ;-)Then use LDC! ;)Keep LDC up to date with DMD master daily! ;)
Jun 26 2018
On Tuesday, 26 June 2018 at 18:07:56 UTC, Manu wrote:On Tue, 26 Jun 2018 at 10:43, Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> wrote:D frontend was merged into GDC master. It is version 2.076 now. The older version was merged to make the switch easy and just replace the C++ frontend with the same version of the D frontend. There is a pull request to merge 2.080-beta into master: https://github.com/D-Programming-GDC/GDC/pull/683 But some tests are still fail. There is no nightly builds for DMD. It is currently more important to keep GDC up to date with GCC master (I'm updating GDC to to work with GCC weekly snapshots), because it would help to get GDC merged into GCC. I can't currently promise anything but I wanted to build GDC binaries (gcc-7 or gcc-8) for me with a new frontend (2.080) and for Ubuntu, so other people can use it. Not sure how difficult will it be to update to the newer D-frontend versions (2.081 ... and so on), so not sure if I'll be able to help Iain much with that. I should probably write some announcement about the latest work on GDC :).On 26 June 2018 at 19:41, Manu via Digitalmars-d <digitalmars-d puremagic.com> wrote:Orly? Have you gotten GDC to a point where you can actively keep up to date? You should definitely advertise that better, I think most users still presume that GDC is a few revisions behind latest. Do you have nightly builds that are close to the DMD nightlies? Sadly we're working with the MSVC ABI, and I don't think GCC has a backend for that arch do they? LLVM has done a lot of work to interop with MSVC.On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via Digitalmars-d <digitalmars-d puremagic.com> wrote:Like what GDC is doing (almost) ;-)Then use LDC! ;)Keep LDC up to date with DMD master daily! ;)
Jun 26 2018
On 26 June 2018 at 20:26, Eugene Wissner via Digitalmars-d <digitalmars-d puremagic.com> wrote:On Tuesday, 26 June 2018 at 18:07:56 UTC, Manu wrote:Well, once in sync with dmd/stable. A weekly (or more frequent) check would all that would be needed. A weekly merge would be trivial to maintain.On Tue, 26 Jun 2018 at 10:43, Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> wrote:D frontend was merged into GDC master. It is version 2.076 now. The older version was merged to make the switch easy and just replace the C++ frontend with the same version of the D frontend. There is a pull request to merge 2.080-beta into master: https://github.com/D-Programming-GDC/GDC/pull/683 But some tests are still fail. There is no nightly builds for DMD. It is currently more important to keep GDC up to date with GCC master (I'm updating GDC to to work with GCC weekly snapshots), because it would help to get GDC merged into GCC. I can't currently promise anything but I wanted to build GDC binaries (gcc-7 or gcc-8) for me with a new frontend (2.080) and for Ubuntu, so other people can use it. Not sure how difficult will it be to update to the newer D-frontend versions (2.081 ... and so on), so not sure if I'll be able to help Iain much with that. I should probably write some announcement about the latest work on GDC :).On 26 June 2018 at 19:41, Manu via Digitalmars-d <digitalmars-d puremagic.com> wrote:Orly? Have you gotten GDC to a point where you can actively keep up to date? You should definitely advertise that better, I think most users still presume that GDC is a few revisions behind latest. Do you have nightly builds that are close to the DMD nightlies? Sadly we're working with the MSVC ABI, and I don't think GCC has a backend for that arch do they? LLVM has done a lot of work to interop with MSVC.On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via > Digitalmars-d <digitalmars-d puremagic.com> wrote:Like what GDC is doing (almost) ;-)Then use LDC! ;)Keep LDC up to date with DMD master daily! ;)
Jun 26 2018
On 26 June 2018 at 20:07, Manu via Digitalmars-d <digitalmars-d puremagic.com> wrote:On Tue, 26 Jun 2018 at 10:43, Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> wrote:Not quite, there are some funny bugs I'm looking at, some of which were features introduced by you. :-)On 26 June 2018 at 19:41, Manu via Digitalmars-d <digitalmars-d puremagic.com> wrote:Orly? Have you gotten GDC to a point where you can actively keep up to date? You should definitely advertise that better, I think most users still presume that GDC is a few revisions behind latest. Do you have nightly builds that are close to the DMD nightlies?On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via Digitalmars-d <digitalmars-d puremagic.com> wrote:Like what GDC is doing (almost) ;-)Then use LDC! ;)Keep LDC up to date with DMD master daily! ;)
Jun 26 2018
On Tue., 26 Jun. 2018, 11:45 am Iain Buclaw via Digitalmars-d, < digitalmars-d puremagic.com> wrote:On 26 June 2018 at 20:07, Manu via Digitalmars-d <digitalmars-d puremagic.com> wrote:You love it! They're spectacular patches! :P This is some seriously good news for GDC. Awesome stuff guys!On Tue, 26 Jun 2018 at 10:43, Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> wrote:Not quite, there are some funny bugs I'm looking at, some of which were features introduced by you. :-)On 26 June 2018 at 19:41, Manu via Digitalmars-d <digitalmars-d puremagic.com> wrote:Orly? Have you gotten GDC to a point where you can actively keep up to date? You should definitely advertise that better, I think most users still presume that GDC is a few revisions behind latest. Do you have nightly builds that are close to the DMD nightlies?On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via Digitalmars-d <digitalmars-d puremagic.com> wrote:Like what GDC is doing (almost) ;-)Then use LDC! ;)Keep LDC up to date with DMD master daily! ;)
Jun 26 2018
On Wednesday, 27 June 2018 at 06:47:46 UTC, Manu wrote:This is some seriously good news for GDC. Awesome stuff guys!Agreed!
Jun 27 2018