www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DMD backend quality (Was: Re: DIP 1031--Deprecate Brace-Style Struct

reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
[Moving this to separate thread because it's tangential to DIP 1031
discussion.]

On Sat, Feb 15, 2020 at 03:59:39PM -0800, Walter Bright via Digitalmars-d wrote:
 On 2/15/2020 9:58 AM, jxel wrote:
 how poor quality the backend code is.
Baloney. It's written in an older style, sure, but it's remarkably bug free considering how complex it is. The complexity is a result of the fiendishly complex x86 instructions set. The proof of its quality is it has successfully been extended from a purely 16 bit generator to 32 and extended again to 64 bit, and has been extended to support multiple ABIs and object file formats. The DFA optimizer has also proven to be fast and very robust.
Unfortunately, the GDC and LDC optimizers consistently produce code that outperforms code generated by the DMD backend by 20-30%, sometimes as high as 40% for CPU-intensive code. DMD also tends to have more codegen bugs when run with -O -inline (which, ironically, still produces inferior code to what LDC/GDC produces with -O). I'm sure, given enough time and effort, you could bring it to par, but after >5 years of waiting, I've essentially given up on the DMD backend for performance-sensitive code. I now use LDC for my larger projects, and would probably start new projects on LDC rather than DMD. LDC can also cross-compile to Windows from Linux, which allows me the very convenient setup of a single build script that builds both Linux and Windows executables simultaneously. With DMD I need a VM or at least Wine (with its very slow and resource-intensive startup times) to be able to do this. And LDC can target a *lot* more architectures than x86 on Linux/Windows, such as WebAssembly, Android, etc.. Again, DMD loses out here. Of course, DMD is still very fast at compiling and useful for smaller utilities and script-like programs where performance isn't critical, or for running unittests for isolated modules during development due to fast turnaround times. But codegen quality is definitely not among the reasons I still use DMD. T -- Computers shouldn't beep through the keyhole.
Feb 15
next sibling parent reply NaN <divide by.zero> writes:
On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
 [Moving this to separate thread because it's tangential to DIP 
 1031 discussion.]

 On Sat, Feb 15, 2020 at 03:59:39PM -0800, Walter Bright via 
 Digitalmars-d wrote:
 On 2/15/2020 9:58 AM, jxel wrote:
 how poor quality the backend code is.
Baloney. It's written in an older style, sure, but it's remarkably bug free considering how complex it is. The complexity is a result of the fiendishly complex x86 instructions set. The proof of its quality is it has successfully been extended from a purely 16 bit generator to 32 and extended again to 64 bit, and has been extended to support multiple ABIs and object file formats. The DFA optimizer has also proven to be fast and very robust.
Unfortunately, the GDC and LDC optimizers consistently produce code that outperforms code generated by the DMD backend by 20-30%, sometimes as high as 40% for CPU-intensive code. DMD also tends to have more codegen bugs when run with -O -inline (which, ironically, still produces inferior code to what LDC/GDC produces with -O).
I was seeing worse than that last time i used DMD, it was mostly int code, no fpu, and it was around 60% slower with DMD than LDC. Haven't used it since. SIMD support with DMD was awful at the time too, dont know if thats improved.
Feb 16
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Sunday, 16 February 2020 at 10:52:33 UTC, NaN wrote:
 On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
 [Moving this to separate thread because it's tangential to DIP 
 1031 discussion.]

 On Sat, Feb 15, 2020 at 03:59:39PM -0800, Walter Bright via 
 Digitalmars-d wrote:
 On 2/15/2020 9:58 AM, jxel wrote:
 how poor quality the backend code is.
Baloney. It's written in an older style, sure, but it's remarkably bug free considering how complex it is. The complexity is a result of the fiendishly complex x86 instructions set. The proof of its quality is it has successfully been extended from a purely 16 bit generator to 32 and extended again to 64 bit, and has been extended to support multiple ABIs and object file formats. The DFA optimizer has also proven to be fast and very robust.
Unfortunately, the GDC and LDC optimizers consistently produce code that outperforms code generated by the DMD backend by 20-30%, sometimes as high as 40% for CPU-intensive code. DMD also tends to have more codegen bugs when run with -O -inline (which, ironically, still produces inferior code to what LDC/GDC produces with -O).
I was seeing worse than that last time i used DMD, it was mostly int code, no fpu, and it was around 60% slower with DMD than LDC. Haven't used it since. SIMD support with DMD was awful at the time too, dont know if thats improved.
This discussion has veered off topic. Let's please stick to discussion the DIP. Thanks!
Feb 16
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 16 February 2020 at 10:55:26 UTC, Mike Parker wrote:
 This discussion has veered off topic. Let's please stick to 
 discussion the DIP. Thanks!
Hence why it was split off into a different thread...
Feb 16
parent Mike Parker <aldacron gmail.com> writes:
On Sunday, 16 February 2020 at 11:06:53 UTC, Nicholas Wilson 
wrote:
 On Sunday, 16 February 2020 at 10:55:26 UTC, Mike Parker wrote:
 This discussion has veered off topic. Let's please stick to 
 discussion the DIP. Thanks!
Hence why it was split off into a different thread...
Didn't catch that. Thanks.
Feb 16
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2020 2:52 AM, NaN wrote:
 I was seeing worse than that last time i used DMD, it was mostly int code, no 
 fpu, and it was around 60% slower with DMD than LDC. Haven't used it since.
SIMD 
 support with DMD was awful at the time too, dont know if thats improved.
The SIMD code was substantially improved a year or two ago. I added common subexpression support and improved the register allocator for it. The main problem is I made a mistake in putting the inliner in the DMD front end. It should be in the back end, where it'll be a lot more effective.
Feb 16
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 17/02/2020 11:15 AM, Walter Bright wrote:
 On 2/16/2020 2:52 AM, NaN wrote:
 I was seeing worse than that last time i used DMD, it was mostly int 
 code, no fpu, and it was around 60% slower with DMD than LDC. Haven't 
 used it since. SIMD support with DMD was awful at the time too, dont 
 know if thats improved.
The SIMD code was substantially improved a year or two ago. I added common subexpression support and improved the register allocator for it. The main problem is I made a mistake in putting the inliner in the DMD front end. It should be in the back end, where it'll be a lot more effective.
Perhaps you could mentor somebody in say SaoC to build that up? A chance to do that for an inspiring compiler dev would be invaluable.
Feb 16
parent RazvanN <razvan.nitu1305 gmail.com> writes:
On Sunday, 16 February 2020 at 22:29:01 UTC, rikki cattermole 
wrote:
 On 17/02/2020 11:15 AM, Walter Bright wrote:
 On 2/16/2020 2:52 AM, NaN wrote:
 I was seeing worse than that last time i used DMD, it was 
 mostly int code, no fpu, and it was around 60% slower with 
 DMD than LDC. Haven't used it since. SIMD support with DMD 
 was awful at the time too, dont know if thats improved.
The SIMD code was substantially improved a year or two ago. I added common subexpression support and improved the register allocator for it. The main problem is I made a mistake in putting the inliner in the DMD front end. It should be in the back end, where it'll be a lot more effective.
Perhaps you could mentor somebody in say SaoC to build that up? A chance to do that for an inspiring compiler dev would be invaluable.
Or it could be a good project for GSOC if we get accepted, but Walter should be willing to mentor the student.
Feb 16
prev sibling parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Sunday, 16 February 2020 at 22:15:22 UTC, Walter Bright wrote:
 On 2/16/2020 2:52 AM, NaN wrote:
 I was seeing worse than that last time i used DMD, it was 
 mostly int code, no fpu, and it was around 60% slower with DMD 
 than LDC. Haven't used it since. SIMD support with DMD was 
 awful at the time too, dont know if thats improved.
The SIMD code was substantially improved a year or two ago. I added common subexpression support and improved the register allocator for it. The main problem is I made a mistake in putting the inliner in the DMD front end. It should be in the back end, where it'll be a lot more effective.
Replying here so as not to pollute the other thread. Since PR are welcome, can you please take care of https://github.com/dlang/dmd/pull/10200 which hasn't been reviewed for 6+ months ? This is the main concern I have with the DMD backend: while it's an incredible one-man effort (and I don't think people are giving you enough credit for this achievement), it *is* a one-man effort, with a bus factor of 1. It does produce slower code than other backends, generate worse debug infos (DWARF debug infos are completely broken on OSX), does not support cross-compilation except for `-m32`/`-m64` and has a few ABI issues which haven't been solved in years. And that's completely understandable, given that amount of work required to make things work. That being said, DMD is still my default compiler for its compilation speed (although it's loosing a bit more with every release).
Feb 16
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2020 6:07 PM, Mathias Lang wrote:
 (DWARF debug infos are completely broken on OSX),
I hear this now and then about the DWARF output, but there are ZERO bug reports on it. I use gdb myself on OSX and Linux, and while I don't much care for gdb, it works.
Feb 16
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 17 February 2020 at 02:54:09 UTC, Walter Bright wrote:
 I hear this now and then about the DWARF output, but there are 
 ZERO bug reports on it.
I just searched "dwarf" on bugzilla and several came up https://issues.dlang.org/buglist.cgi?quicksearch=dwarf&list_id=230166 from 2016 https://issues.dlang.org/show_bug.cgi?id=15747 from 2020 https://issues.dlang.org/show_bug.cgi?id=20510 and many more. I haven't confirmed these myself but it is pretty obviously not zero in bugzilla. (my personal experience btw is that bugzilla is where issues go to be ignored, lost, or forgotten for years.)
Feb 16
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2020 7:02 PM, Adam D. Ruppe wrote:
 (my personal experience btw is that bugzilla is where issues go to be ignored, 
 lost, or forgotten for years.)
Anyone can submit PRs to fix bugzilla issues, including you.
Feb 16
parent reply jxel <jxel gmall.com> writes:
On Monday, 17 February 2020 at 05:11:14 UTC, Walter Bright wrote:
 On 2/16/2020 7:02 PM, Adam D. Ruppe wrote:
 (my personal experience btw is that bugzilla is where issues 
 go to be ignored, lost, or forgotten for years.)
Anyone can submit PRs to fix bugzilla issues, including you.
That doesn't mean they will get merged. There's lots of PR open that have been open for years without any sort of feedback. Why would anybody waste their free time to simply have their work ignored?
Feb 17
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2020 10:45 AM, jxel wrote:
 That doesn't mean they will get merged. There's lots of PR open that have been 
 open for years without any sort of feedback. Why would anybody waste their
free 
 time to simply have their work ignored?
The dmd compiler itself has 24,254 commits from 195 contributors. https://github.com/dlang/dmd Phobos has 17,927 commits from 367 contributors. https://github.com/dlang/phobos Druntime has 7,023 commits from 198 contributors. https://github.com/dlang/druntime
Feb 17
next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Monday, 17 February 2020 at 19:35:53 UTC, Walter Bright wrote:
 On 2/17/2020 10:45 AM, jxel wrote:
 That doesn't mean they will get merged. There's lots of PR 
 open that have been open for years without any sort of 
 feedback. Why would anybody waste their free time to simply 
 have their work ignored?
The dmd compiler itself has 24,254 commits from 195 contributors. https://github.com/dlang/dmd Phobos has 17,927 commits from 367 contributors. https://github.com/dlang/phobos Druntime has 7,023 commits from 198 contributors. https://github.com/dlang/druntime
This gives somewhat an idea but actually these numbers, as provided by GH, are innaccurate, they don't include people who have left GH. If you really want to give numbers then you must use git to find the unique authors by email.
Feb 17
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2020 6:07 PM, Basile B. wrote:
 On Monday, 17 February 2020 at 19:35:53 UTC, Walter Bright wrote:
 On 2/17/2020 10:45 AM, jxel wrote:
 That doesn't mean they will get merged. There's lots of PR open that have 
 been open for years without any sort of feedback. Why would anybody waste 
 their free time to simply have their work ignored?
The dmd compiler itself has 24,254 commits from 195 contributors.   https://github.com/dlang/dmd Phobos has 17,927 commits from 367 contributors.   https://github.com/dlang/phobos Druntime has 7,023 commits from 198 contributors.   https://github.com/dlang/druntime
This gives somewhat an idea but actually these numbers, as provided by GH, are innaccurate, they don't include people who have left GH. If you really want to give numbers then you must use git to find the unique authors by email.
When it says 195 contributors, what's inaccurate about that? Are you saying some are counted multiple times because they submit PRs under multiple aliases?
Feb 17
parent reply Basile B. <b2.temp gmx.com> writes:
On Tuesday, 18 February 2020 at 07:20:17 UTC, Walter Bright wrote:
 On 2/17/2020 6:07 PM, Basile B. wrote:
 On Monday, 17 February 2020 at 19:35:53 UTC, Walter Bright 
 wrote:
 [...]
This gives somewhat an idea but actually these numbers, as provided by GH, are innaccurate, they don't include people who have left GH. If you really want to give numbers then you must use git to find the unique authors by email.
When it says 195 contributors, what's inaccurate about that? Are you saying some are counted multiple times because they submit PRs under multiple aliases?
No I say that some are not counted because they left github. For example me, I'm not in the list, despite of 80+ commits in DMD.
Feb 17
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2020 11:57 PM, Basile B. wrote:
 No I say that some are not counted because they left github. For example me,
I'm 
 not in the list, despite of 80+ commits in DMD.
Oh, I see. So it's an undercount, the actual number of contributors is higher. Anyhow, I can see becoming an inactive github user, but why leave, especially since github contributions can be a great asset for your resume? A number of significant D contributors have been able to leverage that into getting well compensated positions, a result I'm pretty proud of. I like to see you guys being successful.
Feb 18
prev sibling parent jxel <jxel gmall.com> writes:
On Monday, 17 February 2020 at 19:35:53 UTC, Walter Bright wrote:
 On 2/17/2020 10:45 AM, jxel wrote:
 That doesn't mean they will get merged. There's lots of PR 
 open that have been open for years without any sort of 
 feedback. Why would anybody waste their free time to simply 
 have their work ignored?
The dmd compiler itself has 24,254 commits from 195 contributors. https://github.com/dlang/dmd Phobos has 17,927 commits from 367 contributors. https://github.com/dlang/phobos Druntime has 7,023 commits from 198 contributors. https://github.com/dlang/druntime
The contributions drop to single digits fairly quickly. People aren't coming back to contribute, I wonder why. Pointing to numbers doesn't mean there's are countless PR that are being neglected. All you are showing is that only the handful of people that can just ask someone else to merge their work gets it merged regularly.
Feb 17
prev sibling parent Basile B. <b2.temp gmx.com> writes:
On Monday, 17 February 2020 at 18:45:25 UTC, jxel wrote:
 On Monday, 17 February 2020 at 05:11:14 UTC, Walter Bright 
 wrote:
 On 2/16/2020 7:02 PM, Adam D. Ruppe wrote:
 (my personal experience btw is that bugzilla is where issues 
 go to be ignored, lost, or forgotten for years.)
Anyone can submit PRs to fix bugzilla issues, including you.
That doesn't mean they will get merged. There's lots of PR open that have been open for years without any sort of feedback. Why would anybody waste their free time to simply have their work ignored?
This is not true. If a lot of PR are opened it's not because they are ignored. There are plenty of valid reasons and none matches to what your reductive POV. - additional changes are requested but the author does not make these changes ; - the changes cannot be merged because they need to be rebased ; - the changes cannot be merged until other another PR get merged (label BLOCKED) ; - the changes cannot be merged because they doesn't pass the test suite ; - the changes are not accepted, they don't follow the language direction ; - the PR is a draft or a WIP, so it get a bit ignored because the author ask for it ; - the changes are too big, hard to understand ; - the author doesn't care anymore ; - the changes were not approved explicitly but the author doesn't close the PR ; etc. see https://github.com/dlang/dmd/pulls?q=is%3Apr+is%3Aopen+sort%3Acomments-asc There is always at least one comment (excluding the one auto generated by the bot)
Feb 18
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2020 7:02 PM, Adam D. Ruppe wrote:
 I just searched "dwarf" on bugzilla and several came up
 
 https://issues.dlang.org/buglist.cgi?quicksearch=dwarf&list_id=230166
I see the situation has changed fairly recently. That's good. I haven't looked at dwarf for a while.
Feb 16
prev sibling next sibling parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Monday, 17 February 2020 at 02:54:09 UTC, Walter Bright wrote:
 On 2/16/2020 6:07 PM, Mathias Lang wrote:
 (DWARF debug infos are completely broken on OSX),
I hear this now and then about the DWARF output, but there are ZERO bug reports on it. I use gdb myself on OSX and Linux, and while I don't much care for gdb, it works.
Really ? - I mentioned last time in the Skype meeting; - we put a bounty on it (blog post: https://dlang.org/blog/2019/08/17/bug-bounties-have-arrived/ and bounty itself: https://www.flipcause.com/secure/cause_pdetails/NjI2NjQ=) - there is a bugzilla issue: https://issues.dlang.org/show_bug.cgi?id=20460 - I opened a PR detailing the issue in great details: https://github.com/dlang/dmd/pull/10722 And people are not going to mention "DWARF" in bugzilla, they're going to say "stack traces are broken but I can't reproduce with a small example" (in this case it depends on the order the root modules are passed to the compiler). I was lucky to be familiar enough with the compiler to notice the pattern, but it did take me a while. Also which OS version / gdb version are you using ? On a relatively recent Mac, GDB barely works and LLDB is a much better place to get started.
Feb 16
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2020 9:44 PM, Mathias Lang wrote:
 On Monday, 17 February 2020 at 02:54:09 UTC, Walter Bright wrote:
 I use gdb myself on OSX and Linux, and while I don't much care for gdb, it
works.
Really ?
Well, somehow I debugged the compiler on OSX.
 Also which OS version / gdb version are you using ?
Don't recall, but old.
Feb 16
next sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Monday, 17 February 2020 at 07:11:47 UTC, Walter Bright wrote:
 On 2/16/2020 9:44 PM, Mathias Lang wrote:
 On Monday, 17 February 2020 at 02:54:09 UTC, Walter Bright 
 wrote:
 I use gdb myself on OSX and Linux, and while I don't much 
 care for gdb, it works.
Really ?
Well, somehow I debugged the compiler on OSX. [..]
I think Mathias was replying to this part:
 I hear this now and then about the DWARF output, but there are 
 ZERO bug reports on it.
Feb 17
prev sibling parent Jacob Carlborg <doob me.com> writes:
On Monday, 17 February 2020 at 07:11:47 UTC, Walter Bright wrote:

 Don't recall, but old.
You can run `sw_vers -productVersion` in the terminal to find out. -- /Jacob Carlborg
Feb 17
prev sibling parent Jacob Carlborg <doob me.com> writes:
On Monday, 17 February 2020 at 05:44:32 UTC, Mathias Lang wrote:

 Really ?
Yeah, I've tested with DMD combined with GDB and it works. But it doesn't work with LLDB. LDC of course work together with LLDB. -- /Jacob Carlborg
Feb 17
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On Monday, 17 February 2020 at 02:54:09 UTC, Walter Bright wrote:

 I hear this now and then about the DWARF output, but there are 
 ZERO bug reports on it.
As others have mentioned, the bug reports might not include the word "dwarf".
 I use gdb myself on OSX and Linux, and while I don't much care 
 for gdb, it works.
This explains a lot, and is also what I suspected. I'm guessing you haven't updated your tools since the original port of DMD to macOS. Nobody cares about GDB on macOS anymore. Apple has since long abandoned the GCC toolchain and now relies exclusively on the LLVM toolchain, as I've mentioned in several places on several occasions. That means Clang and LLDB. There are several issues reported for this: https://issues.dlang.org/show_bug.cgi?id=18527 https://issues.dlang.org/show_bug.cgi?id=18612 You might want to search for "LLDB" as well: https://issues.dlang.org/buglist.cgi?quicksearch=lldb -- /Jacob Carlborg
Feb 17
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2020 12:25 AM, Jacob Carlborg wrote:
 Apple has since long abandoned the GCC toolchain and now 
 relies exclusively on the LLVM toolchain, as I've mentioned in several places
on 
 several occasions. That means Clang and LLDB.
 
 There are several issues reported for this:
 
 https://issues.dlang.org/show_bug.cgi?id=18527
 https://issues.dlang.org/show_bug.cgi?id=18612
 
 You might want to search for "LLDB" as well: 
 https://issues.dlang.org/buglist.cgi?quicksearch=lldb
Thank you, you're right, I've never used LLDB. It never occurred to me that it would fail where gdb succeeds. They're both supposed to be compatible with dwarf. Anyhow, do you want to take a look at the dwarf output and see what's different? The dumpobj tool will pretty-print it.
Feb 17
parent reply Jacob Carlborg <doob me.com> writes:
On 2020-02-17 20:44, Walter Bright wrote:

 Anyhow, do you want to take a look at the dwarf output and see what's 
 different? The dumpobj tool will pretty-print it.
I added the output of `dwarfdump` to the issue [1], both of DMD and LDC. I also extracted the actual differences in a separate comment [2]. [1] https://issues.dlang.org/show_bug.cgi?id=18527 [2] https://issues.dlang.org/show_bug.cgi?id=18527#c5 -- /Jacob Carlborg
Feb 19
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/19/2020 5:21 AM, Jacob Carlborg wrote:
 On 2020-02-17 20:44, Walter Bright wrote:
 Anyhow, do you want to take a look at the dwarf output and see what's 
 different?
[1] https://issues.dlang.org/show_bug.cgi?id=18527 [2] https://issues.dlang.org/show_bug.cgi?id=18527#c5
Thank you.
Feb 21
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2020 6:07 PM, Mathias Lang wrote:
 Since PR are welcome, can you please take care of 
 https://github.com/dlang/dmd/pull/10200 which hasn't been reviewed for 6+
months ?
It fails all tests on Posix: ERROR: dmd/target.d(408): Error: no property `isPOSIX` for type `Param` Also, it's pretty hard to deduce the strategy from the low level detail.
Feb 16
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
 Of course, DMD is still very fast at compiling and useful for 
 smaller utilities and script-like programs where performance 
 isn't critical, or for running unittests for isolated modules 
 during development due to fast turnaround times. But codegen 
 quality is definitely not among the reasons I still use DMD.
I still happily use DMD for most development because it still builds faster. (On Windows, try `dub -a x86 --compiler dmd` and see) But yes adding -O -inline frightens me. I think DMD should play its strength of generating code very quickly :) even at the expense of output performance.
Feb 17
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2020 1:14 PM, Guillaume Piolat wrote:
 I still happily use DMD for most development because it still builds faster.
 (On Windows, try `dub -a x86 --compiler dmd` and see)
 
 But yes adding -O -inline frightens me.
 I think DMD should play its strength of generating code very quickly :) even
at 
 the expense of output performance.
No worries, there are no plans to change the fast path through the backend.
Feb 17
parent reply jxel <jxel gmall.com> writes:
On Monday, 17 February 2020 at 22:25:17 UTC, Walter Bright wrote:
 On 2/17/2020 1:14 PM, Guillaume Piolat wrote:
 I still happily use DMD for most development because it still 
 builds faster.
 (On Windows, try `dub -a x86 --compiler dmd` and see)
 
 But yes adding -O -inline frightens me.
 I think DMD should play its strength of generating code very 
 quickly :) even at the expense of output performance.
No worries, there are no plans to change the fast path through the backend.
If DMD used LDC to build itself, it'd run even faster. There's a PR open for that iirc.
Feb 17
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Tuesday, 18 February 2020 at 04:11:51 UTC, jxel wrote:
 On Monday, 17 February 2020 at 22:25:17 UTC, Walter Bright 
 wrote:
 On 2/17/2020 1:14 PM, Guillaume Piolat wrote:
 I still happily use DMD for most development because it still 
 builds faster.
 (On Windows, try `dub -a x86 --compiler dmd` and see)
 
 But yes adding -O -inline frightens me.
 I think DMD should play its strength of generating code very 
 quickly :) even at the expense of output performance.
No worries, there are no plans to change the fast path through the backend.
If DMD used LDC to build itself, it'd run even faster. There's a PR open for that iirc.
Yes please !
Feb 18
parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 18 February 2020 at 15:00:47 UTC, Guillaume Piolat 
wrote:
 Yes please !
https://github.com/dlang/installer/pull/425
Feb 18
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Feb 17, 2020 at 09:14:31PM +0000, Guillaume Piolat via Digitalmars-d
wrote:
 On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
 Of course, DMD is still very fast at compiling and useful for
 smaller utilities and script-like programs where performance isn't
 critical, or for running unittests for isolated modules during
 development due to fast turnaround times. But codegen quality is
 definitely not among the reasons I still use DMD.
I still happily use DMD for most development because it still builds faster. (On Windows, try `dub -a x86 --compiler dmd` and see)
I don't use dub because it increases my build times by like 2X or more. I do still use DMD for when I'm working on unittesting a single module and don't need the entire program linked (`dmd -unittest -i -main -run mymodule.d` is amazing). But for my main program I prefer LDC, because I'm frightened of discrepancies between compilers coming back to bite me in the behind, plus for small utilities I sometimes just use debug builds for "production" use, and LDC gives me basically a free 20% performance boost for no extra effort. Plus, it's annoying to have to switch between different compiler options syntax just so I can use both dmd and ldc2 for my builds.
 But yes adding -O -inline frightens me.
I've encountered quite a few obscure codegen bugs related to -O -inline. It's great that they are fixed now, but from a user's POV it's a rather unpleasant experience to have to deal with compiler bugs that only show up with -O -inline. It exarcerbates the problem of dmd producing slow code, because now I'm forced to live with *not* using -O -inline. It's yet another reason to use LDC instead.
 I think DMD should play its strength of generating code very quickly
 :) even at the expense of output performance.
The thing is, for all the effort Walter put into making dmd as fast as possible, when you're writing template-heavy code it's *still* slow. Well, you say, then just write less templates. But the problem is, sometimes these compiler performance problems aren't obvious until your project grows larger, and by then the templated code is already heavily ingrained in your codebase and would be too onerous to rewrite. So if I'm going to have to wait several seconds for my template-heavy code to compile anyway, why not use a compiler that gives me better codegen in return? T -- It is impossible to make anything foolproof because fools are so ingenious. -- Sammy
Feb 18
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 18 February 2020 at 18:00:03 UTC, H. S. Teoh wrote:
 

 Plus, it's annoying to have to switch between different 
 compiler options syntax just so I can use both dmd and ldc2 for 
 my builds.
Why not use ldmd2 then?
Feb 18
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 18, 2020 at 06:21:21PM +0000, jmh530 via Digitalmars-d wrote:
 On Tuesday, 18 February 2020 at 18:00:03 UTC, H. S. Teoh wrote:
 
 Plus, it's annoying to have to switch between different compiler
 options syntax just so I can use both dmd and ldc2 for my builds.
Why not use ldmd2 then?
Because some LDC flags are not accessible through the ldmd2 interface. T -- Error: Keyboard not attached. Press F1 to continue. -- Yoon Ha Lee, CONLANG
Feb 18
parent reply kinke <noone nowhere.com> writes:
On Tuesday, 18 February 2020 at 18:40:34 UTC, H. S. Teoh wrote:
 On Tue, Feb 18, 2020 at 06:21:21PM +0000, jmh530 via
 Why not use ldmd2 then?
Because some LDC flags are not accessible through the ldmd2 interface.
Just try it - unknown flags are forwarded to ldc2.exe.
Feb 18
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 18, 2020 at 07:44:49PM +0000, kinke via Digitalmars-d wrote:
 On Tuesday, 18 February 2020 at 18:40:34 UTC, H. S. Teoh wrote:
 On Tue, Feb 18, 2020 at 06:21:21PM +0000, jmh530 via
 Why not use ldmd2 then?
Because some LDC flags are not accessible through the ldmd2 interface.
Just try it - unknown flags are forwarded to ldc2.exe.
It works!! Thanks for the tip! T -- Food and laptops don't mix.
Feb 18
prev sibling next sibling parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
 ...
 Of course, DMD is still very fast at compiling and useful for 
 smaller utilities and script-like programs where performance 
 isn't critical, or for running unittests for isolated modules 
 during development due to fast turnaround times. But codegen 
 quality is definitely not among the reasons I still use DMD.
 ...
I do not know about others, but I personally do not expect DMD to be as performant as people would like it to be. As a REFERENCE COMPILER I expect it to be as close to the current language specification as possible. DMD does that beautifully, and on top of that does a pretty good job at compiling code quite fast. I can't ask for more! - If they optimize the codegen to generate better code, that is fine, but not essential, as I use GCC (others may prefer LDC) for release builds.
Feb 18
parent reply jxel <jxel gmall.com> writes:
On Tuesday, 18 February 2020 at 13:04:36 UTC, Dejan Lekic wrote:
 On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
 ...
 Of course, DMD is still very fast at compiling and useful for 
 smaller utilities and script-like programs where performance 
 isn't critical, or for running unittests for isolated modules 
 during development due to fast turnaround times. But codegen 
 quality is definitely not among the reasons I still use DMD.
 ...
I do not know about others, but I personally do not expect DMD to be as performant as people would like it to be. As a REFERENCE COMPILER I expect it to be as close to the current language specification as possible. DMD does that beautifully, and on top of that does a pretty good job at compiling code quite fast. I can't ask for more! - If they optimize the codegen to generate better code, that is fine, but not essential, as I use GCC (others may prefer LDC) for release builds.
The problem is that efforts are divided as a result. Work still has to be maintained on the backend. As someone showed there's multiple PR that have stagnated because no one is willing to look at them that modify the backend to comply with the C ABI. Bugs that have been open for years. Bugs that don't exist in LDC, even though that's basically being maintained by one person. If people want a fast backend, you can build a custom backend for performance in LLVM, a few projects have done this already. But honestly that's a waste of time. The slowest part of D is the frontend and CTFE not the backend in LDC. That's not even mentioning the constant out of memory problems I experience and have to try to optimize the compiler for with my code.
Feb 18
next sibling parent reply bachmeier <no spam.net> writes:
On Tuesday, 18 February 2020 at 15:30:57 UTC, jxel wrote:
 On Tuesday, 18 February 2020 at 13:04:36 UTC, Dejan Lekic wrote:
 On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
 ...
 Of course, DMD is still very fast at compiling and useful for 
 smaller utilities and script-like programs where performance 
 isn't critical, or for running unittests for isolated modules 
 during development due to fast turnaround times. But codegen 
 quality is definitely not among the reasons I still use DMD.
 ...
I do not know about others, but I personally do not expect DMD to be as performant as people would like it to be. As a REFERENCE COMPILER I expect it to be as close to the current language specification as possible. DMD does that beautifully, and on top of that does a pretty good job at compiling code quite fast. I can't ask for more! - If they optimize the codegen to generate better code, that is fine, but not essential, as I use GCC (others may prefer LDC) for release builds.
The problem is that efforts are divided as a result. Work still has to be maintained on the backend. As someone showed there's multiple PR that have stagnated because no one is willing to look at them that modify the backend to comply with the C ABI. Bugs that have been open for years. Bugs that don't exist in LDC, even though that's basically being maintained by one person. If people want a fast backend, you can build a custom backend for performance in LLVM, a few projects have done this already. But honestly that's a waste of time. The slowest part of D is the frontend and CTFE not the backend in LDC. That's not even mentioning the constant out of memory problems I experience and have to try to optimize the compiler for with my code.
DMD already exists, nobody's forced to use it, and I doubt it's sucking any resources away from LDC.
Feb 18
parent IGotD- <nise nise.com> writes:
On Tuesday, 18 February 2020 at 15:41:36 UTC, bachmeier wrote:
 DMD already exists, nobody's forced to use it, and I doubt it's 
 sucking any resources away from LDC.
Is there one thing DMD does better, that is being able to debug with Visual Studio. With DMD you get full debugging information including locals, almost as you were debugging a C++ program. When I compile with LDC I just get basic debugging information. I presume that this is because LDC emits a different debugging format.
Feb 18
prev sibling parent reply drug <drug2004 bk.ru> writes:
On 2/18/20 6:30 PM, jxel wrote:
 
 The problem is that efforts are divided as a result. Work still has to 
 be maintained on the backend. As someone showed there's multiple PR that 
 have stagnated because no one is willing to look at them that modify the 
 backend to comply with the C ABI. Bugs that have been open for years. 
 Bugs that don't exist in LDC, even though that's basically being 
 maintained by one person. If people want a fast backend, you can build a 
 custom backend for performance in LLVM, a few projects have done this 
 already. But honestly that's a waste of time. The slowest part of D is 
 the frontend and CTFE not the backend in LDC. That's not even mentioning 
 the constant out of memory problems I experience and have to try to 
 optimize the compiler for with my code.
 
IMO you are wrong if you think that dropping dmd will increase man power in ldc/gdc land. Who wants to contribute to ldc/gdc already doing it. It is open source - people contribute to projects they like, you can not tell them what to do. Efforts are not divided - if dmd would be dropped people who doesn't contribute to ldc/gdc won't start doing that.
Feb 18
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 18, 2020 at 07:12:34PM +0300, drug via Digitalmars-d wrote:
 On 2/18/20 6:30 PM, jxel wrote:
 
 The problem is that efforts are divided as a result.
[...] That's the fallacy described in "The Mythical Man-Month".
 IMO you are wrong if you think that dropping dmd will increase man
 power in ldc/gdc land. Who wants to contribute to ldc/gdc already
 doing it. It is open source - people contribute to projects they like,
 you can not tell them what to do. Efforts are not divided - if dmd
 would be dropped people who doesn't contribute to ldc/gdc won't start
 doing that.
Yeah, this is not a commercial endeavour where people are paid to work on stuff they didn't choose to work on. The people currently working on dmd do so because they *want* to work on dmd. If we drop dmd, they may end up going elsewhere to find something else they're interested instead. There's no reason to expect that they will suddenly acquire a fresh interest in ldc/gdc -- if they already had such an interest they'd already be working on ldc/gdc.
 On 2/18/20 6:30 PM, jxel wrote:
[...]
 The slowest part of D is the frontend and CTFE not the backend in
 LDC.
?! That's clearly not true. I'm compiling the same codebase with dmd, and it's measurably faster than ldc2. Since it's identical source code, and the two compilers share the same frontend, the only possible difference in compile times must be due to the different backends. Of course, compile speed isn't the only criterion for choosing between compilers, and that's why I still prefer using LDC in spite of DMD being faster at compiling. The deciding factor is the fact that the LDC backend, in spite of being slower, emits better code. So I'm gaining more runtime performance by paying with slightly longer compile times. The increase in compile times is (currently) outweighed by the superior performance of the resulting executable.
 That's not even mentioning the constant out of memory problems
 I experience and have to try to optimize the compiler for with my
 code.
The frontend's memory issues are mainly because Walter switched to a bump-the-pointer allocator in interest of compile speed. Even with -lowmem there are still fundamental problems that are unsolvable when running a low-memory system. I've essentially given up using dmd on low-memory systems. Fast compile times are worthless when the compiler never finishes running before it runs out of memory. It's an embarrassment. (And this is why the whole "fast code fast" slogan makes me cringe. It epitomizes one of the worst aspects of D in my experience, in spite of everything else I love about D.) T -- "640K ought to be enough" -- Bill G. (allegedly), 1984. "The Internet is not a primary goal for PC usage" -- Bill G., 1995. "Linux has no impact on Microsoft's strategy" -- Bill G., 1999.
Feb 18
parent reply jxel <jxel gmall.com> writes:
On Tuesday, 18 February 2020 at 18:14:56 UTC, H. S. Teoh wrote:
 On Tue, Feb 18, 2020 at 07:12:34PM +0300, drug via 
 Digitalmars-d wrote:
 On 2/18/20 6:30 PM, jxel wrote:
 
 The problem is that efforts are divided as a result.
[...] That's the fallacy described in "The Mythical Man-Month".
Not really, how many people do you think work on and use LLVM? How many other compilers use it and as a result bugs and optimizations all happen " for free". Its not just the few people involved with D, and so much so the thousands involved with LLVM (including corporate funding).
 IMO you are wrong if you think that dropping dmd will increase 
 man power in ldc/gdc land.
Who said that? That's your misinterpretation.
Feb 18
parent reply drug <drug2004 bk.ru> writes:
On 2/18/20 11:00 PM, jxel wrote:
 IMO you are wrong if you think that dropping dmd will increase man 
 power in ldc/gdc land.
Who said that? That's your misinterpretation.
Experience says that again and again. If someone contributes to project A why he should start contributing to project B if project A would be completed? He can easily start contributing to project C, D etc. You have no warranty at all.
Feb 18
next sibling parent Basile B. <b2.temp gmx.com> writes:
On Wednesday, 19 February 2020 at 07:27:39 UTC, drug wrote:
 On 2/18/20 11:00 PM, jxel wrote:
 IMO you are wrong if you think that dropping dmd will 
 increase man power in ldc/gdc land.
Who said that? That's your misinterpretation.
Experience says that again and again. If someone contributes to project A why he should start contributing to project B if project A would be completed? He can easily start contributing to project C, D etc. You have no warranty at all.
I think there's been a misunderstanding in this branch of the conversation. Dropping DMD means nothing... the front end is shared between all compilers after all.
Feb 18
prev sibling parent reply jxel <jxel gmall.com> writes:
On Wednesday, 19 February 2020 at 07:27:39 UTC, drug wrote:
 On 2/18/20 11:00 PM, jxel wrote:
 IMO you are wrong if you think that dropping dmd will 
 increase man power in ldc/gdc land.
Who said that? That's your misinterpretation.
Experience says that again and again. If someone contributes to project A why he should start contributing to project B if project A would be completed? He can easily start contributing to project C, D etc. You have no warranty at all.
Like I said, that's your misinterpretation, read the first paragraph.
Feb 19
parent drug <drug2004 bk.ru> writes:
On 2/19/20 7:29 PM, jxel wrote:
 On Wednesday, 19 February 2020 at 07:27:39 UTC, drug wrote:
 On 2/18/20 11:00 PM, jxel wrote:
 IMO you are wrong if you think that dropping dmd will increase man 
 power in ldc/gdc land.
Who said that? That's your misinterpretation.
Experience says that again and again. If someone contributes to project A why he should start contributing to project B if project A would be completed? He can easily start contributing to project C, D etc. You have no warranty at all.
Like I said, that's your misinterpretation, read the first paragraph.
Well, I responded to your "The problem is that efforts are divided as a result" statement. But probably I misinterpreted indeed.
Feb 19
prev sibling parent reply NaN <divide by.zero> writes:
On Tuesday, 18 February 2020 at 16:12:34 UTC, drug wrote:
 On 2/18/20 6:30 PM, jxel wrote:

 IMO you are wrong if you think that dropping dmd will increase 
 man power in ldc/gdc land. Who wants to contribute to ldc/gdc 
 already doing it. It is open source - people contribute to 
 projects they like, you can not tell them what to do. Efforts 
 are not divided - if dmd would be dropped people who doesn't 
 contribute to ldc/gdc won't start doing that.
Yes dropping DMD wont suddenly move a bunch of developers over to LDC. But over time if LDC is the official compiler, new users, developers, contributors are more likely to work on LDC related issues if thats where they've been guided from day 1. Yes people work on what they need or interests them, but change the focus to LDC, and eventually what people need and are interested in doing is more likely to be there than on DMD. I cant believe people are so short sighted on this.
Feb 19
parent reply bachmeier <no spam.net> writes:
On Wednesday, 19 February 2020 at 19:08:45 UTC, NaN wrote:
 On Tuesday, 18 February 2020 at 16:12:34 UTC, drug wrote:
 On 2/18/20 6:30 PM, jxel wrote:

 IMO you are wrong if you think that dropping dmd will increase 
 man power in ldc/gdc land. Who wants to contribute to ldc/gdc 
 already doing it. It is open source - people contribute to 
 projects they like, you can not tell them what to do. Efforts 
 are not divided - if dmd would be dropped people who doesn't 
 contribute to ldc/gdc won't start doing that.
Yes dropping DMD wont suddenly move a bunch of developers over to LDC. But over time if LDC is the official compiler, new users, developers, contributors are more likely to work on LDC related issues if thats where they've been guided from day 1. Yes people work on what they need or interests them, but change the focus to LDC, and eventually what people need and are interested in doing is more likely to be there than on DMD. I cant believe people are so short sighted on this.
What does "dropping DMD" mean? Are you going to go around breaking kneecaps if you catch someone working on DMD? It's an open source project and people who want to work on it are going to work on it.
Feb 19
next sibling parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Wednesday, 19 February 2020 at 19:36:38 UTC, bachmeier wrote:
 On Wednesday, 19 February 2020 at 19:08:45 UTC, NaN wrote:
 On Tuesday, 18 February 2020 at 16:12:34 UTC, drug wrote:
 On 2/18/20 6:30 PM, jxel wrote:

 IMO you are wrong if you think that dropping dmd will 
 increase man power in ldc/gdc land. Who wants to contribute 
 to ldc/gdc already doing it. It is open source - people 
 contribute to projects they like, you can not tell them what 
 to do. Efforts are not divided - if dmd would be dropped 
 people who doesn't contribute to ldc/gdc won't start doing 
 that.
Yes dropping DMD wont suddenly move a bunch of developers over to LDC. But over time if LDC is the official compiler, new users, developers, contributors are more likely to work on LDC related issues if thats where they've been guided from day 1. Yes people work on what they need or interests them, but change the focus to LDC, and eventually what people need and are interested in doing is more likely to be there than on DMD. I cant believe people are so short sighted on this.
What does "dropping DMD" mean?
It means dropping the backend DMD code generator, and use llvm as a backend instead. Personally I think the frontend and the backend need to be separate projects. It quite clear that the backend is walter baby, and it better for him to work and improve on it on his own accord without have to wait for pr approval from other people. -Alex
Feb 19
next sibling parent bachmeier <no spam.net> writes:
On Wednesday, 19 February 2020 at 20:04:22 UTC, 12345swordy wrote:

 What does "dropping DMD" mean?
It means dropping the backend DMD code generator, and use llvm as a backend instead.
That doesn't explain how you're going to force others (Walter) to stop working on it, which is required in order to drop it.
Feb 19
prev sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 20/02/2020 9:04 AM, 12345swordy wrote:
 Personally I think the frontend and the backend need to be separate 
 projects. It quite clear that the backend is walter baby, and it better 
 for him to work and improve on it on his own accord without have to wait 
 for pr approval from other people.
Given the end goal of dmd as a library, this makes a lot more sense today than it did 5 years ago. I am all for splitting the frontend from dmd into a new repository.
Feb 19
prev sibling parent reply NaN <divide by.zero> writes:
On Wednesday, 19 February 2020 at 19:36:38 UTC, bachmeier wrote:
 On Wednesday, 19 February 2020 at 19:08:45 UTC, NaN wrote:
 On Tuesday, 18 February 2020 at 16:12:34 UTC, drug wrote:
 On 2/18/20 6:30 PM, jxel wrote:

 IMO you are wrong if you think that dropping dmd will 
 increase man power in ldc/gdc land. Who wants to contribute 
 to ldc/gdc already doing it. It is open source - people 
 contribute to projects they like, you can not tell them what 
 to do. Efforts are not divided - if dmd would be dropped 
 people who doesn't contribute to ldc/gdc won't start doing 
 that.
Yes dropping DMD wont suddenly move a bunch of developers over to LDC. But over time if LDC is the official compiler, new users, developers, contributors are more likely to work on LDC related issues if thats where they've been guided from day 1. Yes people work on what they need or interests them, but change the focus to LDC, and eventually what people need and are interested in doing is more likely to be there than on DMD. I cant believe people are so short sighted on this.
What does "dropping DMD" mean? Are you going to go around breaking kneecaps if you catch someone working on DMD? It's an open source project and people who want to work on it are going to work on it.
I assume "dropping DMD" just means making LDC the main reference compiler instead of DMD. It's not about forcing anyone to do anything, it's about perception, for example new users will almost always start with the official compiler. So you start to build more user base around LDC, and over time, if the d core team is focused in that direction, most people will follow suit.
Feb 19
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Feb 20, 2020 at 01:11:08AM +0000, NaN via Digitalmars-d wrote:
 On Wednesday, 19 February 2020 at 19:36:38 UTC, bachmeier wrote:
[...]
 What does "dropping DMD" mean? Are you going to go around breaking
 kneecaps if you catch someone working on DMD? It's an open source
 project and people who want to work on it are going to work on it.
I assume "dropping DMD" just means making LDC the main reference compiler instead of DMD. It's not about forcing anyone to do anything, it's about perception, for example new users will almost always start with the official compiler. So you start to build more user base around LDC, and over time, if the d core team is focused in that direction, most people will follow suit.
[...] Good luck getting Walter to agree to abandoning DMD and working on LDC instead. T -- Bare foot: (n.) A device for locating thumb tacks on the floor.
Feb 19
parent reply jxel <jxel gmall.com> writes:
On Thursday, 20 February 2020 at 01:54:45 UTC, H. S. Teoh wrote:
 On Thu, Feb 20, 2020 at 01:11:08AM +0000, NaN via Digitalmars-d 
 wrote:
 On Wednesday, 19 February 2020 at 19:36:38 UTC, bachmeier 
 wrote:
[...]
 What does "dropping DMD" mean? Are you going to go around 
 breaking kneecaps if you catch someone working on DMD? It's 
 an open source project and people who want to work on it are 
 going to work on it.
I assume "dropping DMD" just means making LDC the main reference compiler instead of DMD. It's not about forcing anyone to do anything, it's about perception, for example new users will almost always start with the official compiler. So you start to build more user base around LDC, and over time, if the d core team is focused in that direction, most people will follow suit.
[...] Good luck getting Walter to agree to abandoning DMD and working on LDC instead. T
No body expect him to. Just how the official DMD isn't built with LDC, why there isn't a 64 bit version of DMD on Windows, why Optlink is still even a thing, why restrictions are placed in DMD to workaround an Optlink bug instead of fixing it. No one expects these things to change. What seems like common sense means nothing to someone stubborn and blinded by pride.
Feb 19
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 20/02/2020 4:35 PM, jxel wrote:
 Optlink is still even a thing
Which is going the way of the dodo soon enough. The alternative needs to mature in production for a while before we swap.
Feb 19
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/19/2020 8:40 PM, rikki cattermole wrote:
 On 20/02/2020 4:35 PM, jxel wrote:
 Optlink is still even a thing
Which is going the way of the dodo soon enough. The alternative needs to mature in production for a while before we swap.
32 bit Windows code is becoming irrelevant, and optlink is only for 32 bit Windows. If you have more than 32,000 symbols in the executable, I wonder why you're building a 32 bit executable.
Feb 20
prev sibling next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Thursday, 20 February 2020 at 03:35:13 UTC, jxel wrote:

 one expects these things to change. What seems like common 
 sense means nothing to someone stubborn and blinded by pride.
There are historical reasons why Optlink is still around and they have nothing to do with stubbornness or pride. Please make your points without going ad hominem.
Feb 19
next sibling parent reply Seb <seb wilzba.ch> writes:
On Thursday, 20 February 2020 at 05:54:13 UTC, Mike Parker wrote:
 On Thursday, 20 February 2020 at 03:35:13 UTC, jxel wrote:

 one expects these things to change. What seems like common 
 sense means nothing to someone stubborn and blinded by pride.
There are historical reasons why Optlink is still around and they have nothing to do with stubbornness or pride. Please make your points without going ad hominem.
That's not true _anymore_. Walter has rejected any kind of PR that would introduce sane defaults in a modern world outright (which is what people commonly understand as stubbornness). "Funnily" we changed the defaults for dub (one can't build anything with optlink, no vibe.d, no phobos etc. anyhow) more than a year ago and people are fine with dub using LLD (like LDC) and thus being able to have more than 32k symbols in their entire program, getting a 64-bit binary by default and not being linked with an ancient, full of well-known and decade-old bugs custom-brew Digitalmars libc.
Feb 19
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Thursday, 20 February 2020 at 06:08:03 UTC, Seb wrote:
 "Funnily" we changed the defaults for dub (one can't build 
 anything with optlink, no vibe.d, no phobos etc. anyhow) more 
 than a year ago and people are fine with dub using LLD (like 
 LDC) and thus being able to have more than 32k symbols in their 
 entire program, getting a 64-bit binary by default and not 
 being linked with an ancient, full of well-known and decade-old 
 bugs custom-brew Digitalmars libc.
Hello, Reading the discussion, it seems something undervalued is the morale of contributors. Getting and _keeping_ contributors is of paramount importance to open-source software. Some say that metric dwarf the importance of _having users_, and is the only real health metrics of a project. Please (D leadership) consider keeping contributors happy, even if that means dropping a backend or OPTLINK or both. Projects that nly person understand can be "unbabyfied" with documentation/Wiki/years of contribution, but not having the problem in the first place is much easier.
Feb 20
next sibling parent NaN <divide by.zero> writes:
On Thursday, 20 February 2020 at 13:11:21 UTC, Guillaume Piolat 
wrote:
 On Thursday, 20 February 2020 at 06:08:03 UTC, Seb wrote:

 Please (D leadership) consider keeping contributors happy, even 
 if that means dropping a backend or OPTLINK or both. Projects 
 that nly person understand can be "unbabyfied" with 
 documentation/Wiki/years of contribution, but not having the 
 problem in the first place is much easier.
If what Seb says is true, that almost all current contributors would like to switch and it's not even being considered, it doesnt bode well.
Feb 20
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 20 February 2020 at 13:11:21 UTC, Guillaume Piolat 
wrote:
 

 Please (D leadership) consider keeping contributors happy, even 
 if that means dropping a backend or OPTLINK or both. Projects 
 that nly person understand can be "unbabyfied" with 
 documentation/Wiki/years of contribution, but not having the 
 problem in the first place is much easier.
I think you need to be a bit more concrete in explaining how dropping a backend would keep contributors happy. In the time I've been following D, the people I've seen complain about multiple backends are rarely participants very long at all. To the point, where I would almost suggest putting something up on the wiki about the motivation here and then just always linking to that when people bring it up every few months. By contrast, the people I've observed who were more active participants and then stop contributing usually complain about other things, like their pull requests being ignored.
Feb 20
next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Thursday, 20 February 2020 at 14:31:44 UTC, jmh530 wrote:
 I think you need to be a bit more concrete in explaining how 
 dropping a backend would keep contributors happy.
Have you read Seb suggestions?
Feb 20
parent jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 20 February 2020 at 14:37:36 UTC, Guillaume Piolat 
wrote:
 On Thursday, 20 February 2020 at 14:31:44 UTC, jmh530 wrote:
 I think you need to be a bit more concrete in explaining how 
 dropping a backend would keep contributors happy.
Have you read Seb suggestions?
Had to go back to re-read it. It's more of a list of issues with supporting multiple compilers than a list of suggestions, but it does address my point about keeping contributors happy. It sounds like a tricky management problem, to be honest. However, I think there is an important trade-off to consider between the value of keeping DMD and the cost of maintaining support for it. For instance, I don't know how much more work it is to maintain CIs for three compilers rather than two. It doesn't sound like much more work, but I really have no idea. Similarly, I would expect that the other backends would need to support new features, even if DMD is put out to pasture. That cost doesn't go away. Regardless, I would hope that Walter considers it is possible to reduce the cost for some of those issues.
Feb 20
prev sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Thursday, 20 February 2020 at 14:31:44 UTC, jmh530 wrote:
 By contrast, the people I've observed who were more active 
 participants and then stop contributing usually complain about 
 other things, like their pull requests being ignored.
Now imagine if Linux spent all his time implementing Linux drivers instead of spending all its time reviewing. He would probably say "there is no time for review".
Feb 20
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/20/2020 6:39 AM, Guillaume Piolat wrote:
 Now imagine if Linux spent all his time implementing Linux drivers instead of 
 spending all its time reviewing. He would probably say "there is no time for 
 review".
There are 24 members of Team DMD, which have pull privileges: https://github.com/orgs/dlang/teams/team-dmd/members Team Phobos has 34 members: https://github.com/orgs/dlang/teams/team-phobos/members If you feel your PRs are being ignored, make some noise to these people. It's what they're there for.
Feb 20
parent reply Paul Backus <snarwin gmail.com> writes:
On Thursday, 20 February 2020 at 21:10:31 UTC, Walter Bright 
wrote:
 On 2/20/2020 6:39 AM, Guillaume Piolat wrote:
 Now imagine if Linux spent all his time implementing Linux 
 drivers instead of spending all its time reviewing. He would 
 probably say "there is no time for review".
There are 24 members of Team DMD, which have pull privileges: https://github.com/orgs/dlang/teams/team-dmd/members Team Phobos has 34 members: https://github.com/orgs/dlang/teams/team-phobos/members If you feel your PRs are being ignored, make some noise to these people. It's what they're there for.
Both of those links show a 404 page for me. I don't think this information is visible to users who aren't part of the Github dlang organization.
Feb 20
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/20/2020 2:42 PM, Paul Backus wrote:
 Both of those links show a 404 page for me. I don't think this information is 
 visible to users who aren't part of the Github dlang organization.
Team DMD: dnadlinger braddr CyberShadow DmitryOlshansky MartinNowak jacob-carlborg ibuclaw andralex WalterBright yebblies schveiguy rainers Geod24 don-clugston-sociomantic PetarKirov wilzbach atilaneves AndrewEdwards auto-tester thewilsonator SSoulaimane AndrejMitrovic dlang-bot RazvanN7 Team Phobos: burner burner repeatedly repeatedly repeatedly dnadlinger dnadlinger dnadlinger braddr braddr braddr alexrp alexrp alexrp JakobOvrum JakobOvrum JakobOvrum CyberShadow CyberShadow CyberShadow kyllingstad kyllingstad kyllingstad DmitryOlshansky DmitryOlshansky DmitryOlshansky MartinNowak MartinNowak MartinNowak Maintainer ibuclaw ibuclaw ibuclaw quickfur quickfur quickfur andralex andralex andralex WalterBright WalterBright WalterBright jmdavis jmdavis jmdavis yebblies yebblies yebblies schveiguy schveiguy schveiguy JackStouffer JackStouffer JackStouffer rainers rainers rainers 9il 9il 9il Hackerpilot Hackerpilot Hackerpilot UplinkCoder UplinkCoder UplinkCoder PetarKirov PetarKirov PetarKirov edi33416 edi33416 edi33416 wilzbach wilzbach wilzbach MetaLang MetaLang MetaLang atilaneves atilaneves atilaneves AndrewEdwards AndrewEdwards AndrewEdwards auto-tester auto-tester auto-tester jercaianu jercaianu jercaianu
Feb 20
prev sibling parent Mathias Lang <pro.mathias.lang gmail.com> writes:
On Thursday, 20 February 2020 at 13:11:21 UTC, Guillaume Piolat 
wrote:
 On Thursday, 20 February 2020 at 06:08:03 UTC, Seb wrote:
 "Funnily" we changed the defaults for dub (one can't build 
 anything with optlink, no vibe.d, no phobos etc. anyhow) more 
 than a year ago and people are fine with dub using LLD (like 
 LDC) and thus being able to have more than 32k symbols in 
 their entire program, getting a 64-bit binary by default and 
 not being linked with an ancient, full of well-known and 
 decade-old bugs custom-brew Digitalmars libc.
Hello, Reading the discussion, it seems something undervalued is the morale of contributors. Getting and _keeping_ contributors is of paramount importance to open-source software. Some say that metric dwarf the importance of _having users_, and is the only real health metrics of a project. Please (D leadership) consider keeping contributors happy, even if that means dropping a backend or OPTLINK or both. Projects that nly person understand can be "unbabyfied" with documentation/Wiki/years of contribution, but not having the problem in the first place is much easier.
To be the devil's advocate, no one has contributed to D longer than Walter, and in the 20+ year he's been working on D, I'm sure he's seen countless contributors coming and going, many of them proposed ideas they considered improvements. Everyone has a their own, often conflicting, vision of what is "the right way". We had many people saying "deprecate scope class" (myself included), we let it happen, and then one of the biggest commercial user steps in and said "Hey guys we're using this feature, like, a lot...". It surely makes you reflect on the silent majority. This is one example out of many where we went one way, and things didn't quite work out as expected. As you cannot satisfy everyone, keeping people happy is not a reliable metric. This is not to say that I think everything is going as it should. I do think that there is a lot to gain with dropping homegrown alternative to mainstream utilities (DM Make, optlink, DMD's backend...). But we made immense progress over the years, as some very large projects have been tackled (conversion of DMD to D), and others have been approved but not yet championed (moving from Bugzilla to DMD). Perhaps improving communication among core contributors could help with reaching agreement on such large project. From what I've seen, this is not a problem that is unique to D.
Feb 20
prev sibling parent jxel <jxel gmall.com> writes:
On Thursday, 20 February 2020 at 05:54:13 UTC, Mike Parker wrote:
 On Thursday, 20 February 2020 at 03:35:13 UTC, jxel wrote:

 one expects these things to change. What seems like common 
 sense means nothing to someone stubborn and blinded by pride.
There are historical reasons why Optlink is still around and they have nothing to do with stubbornness or pride. Please make your points without going ad hominem.
Being around for backwards compatibility and STILL BEING USED IN RELEASES are two very different things. Historical is the only thing that should describe Optlink. With all the breaking changes that are already happening, one can't really argue keeping it to maintain backwards compatibility either. Anyways it seems we are one step closer to that, if the PR doesn't get reversed before release.
Feb 20
prev sibling next sibling parent reply Seb <seb wilzba.ch> writes:
On Thursday, 20 February 2020 at 03:35:13 UTC, jxel wrote:
 On Thursday, 20 February 2020 at 01:54:45 UTC, H. S. Teoh wrote:
 On Thu, Feb 20, 2020 at 01:11:08AM +0000, NaN via 
 Digitalmars-d wrote:
 On Wednesday, 19 February 2020 at 19:36:38 UTC, bachmeier 
 wrote:
[...]
 What does "dropping DMD" mean? Are you going to go around 
 breaking kneecaps if you catch someone working on DMD? It's 
 an open source project and people who want to work on it 
 are going to work on it.
I assume "dropping DMD" just means making LDC the main reference compiler instead of DMD. It's not about forcing anyone to do anything, it's about perception, for example new users will almost always start with the official compiler. So you start to build more user base around LDC, and over time, if the d core team is focused in that direction, most people will follow suit.
[...] Good luck getting Walter to agree to abandoning DMD and working on LDC instead. T
No body expect him to. Just how the official DMD isn't built with LDC, why there isn't a 64 bit version of DMD on Windows, why Optlink is still even a thing, why restrictions are placed in DMD to workaround an Optlink bug instead of fixing it. No one expects these things to change. What seems like common sense means nothing to someone stubborn and blinded by pride.
People often don't see how much effort is wasted on maintaining multiple compilers, but how much time i.e. - the release process (especially with stable releases) - keeping CIs up, configured and running - actively triaging bugs - having multiple test suite solutions - maintaining automations (e.g. dlang-bot) consumes is hidden to non-insiders. Furthermore, - supporting every new feature in each backend (ask Ian or kinke about the recent multiple inner pointer change) - maintaining multiple installers - nasty DMD backend bugs (a few people apart from Walter actually do try to fix things in the backend) - working around obscure infra problems (IIRC Digitalmars libc still isn't thread-safe and people have run in all kinds of problems with it and needed to come up with ingenious solutions, even DMD's build.d shows signs of this) - working with Digitalmars Make - maintaining a modern set of features per compiler (address sanitization, ...) - keeping other compilers up-to-date with upstream (every release introduces regressions) - maintaining support for every OS for every compiler (e.g.FreeBSD) - having an n-times bigger surface for bugs ... And I haven't even mentioned the bottleneck of any major DMD PR needing to be approved by Walter. In my opinion DMD is only the default because switching development efforts to LDC would be a rather big initial cost (time investment) to setup/change the infrastructure and no one wants to head this effort.
 if the d core team is focused in that direction, most people 
 will follow suit.
I would switch in a heartbeat and I am fairly confident to estimate that the same applies to almost all remaining active contributors who haven't left yet because of frustration.
Feb 19
next sibling parent drug <drug2004 bk.ru> writes:
On 2/20/20 9:34 AM, Seb wrote:
 snipped
 
The reason you listed above get a hot response in my soul. That maintenance things are really annoying. As for me they really are the reasons to change the thing order. Probably we can start from making a new separate repository for the frontend? It seems to me that then things will fall into place themselves. But I'm afraid of gdc in case we switch on ldc. Gdc is important thing.
Feb 19
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/19/2020 10:34 PM, Seb wrote:
 And I haven't even mentioned the bottleneck of any major DMD PR needing to be 
 approved by Walter.
Given the current regression list: https://issues.dlang.org/buglist.cgi?bug_severity=regression&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&list_id=230215&query_format=advanced I'm probably being too permissive. I'm usually the one who gets to fix them. I asked in this forum for some help isolating which PRs caused the regressions, to no avail. Major PRs *should* be regarded with skepticism. For example, I recently fixed a regression caused by a 2 or 3 hundred line PR, where the substance of the PR was one line and the rest of it was refactoring. Folks, with PRs, smaller, tightly focused PRs are better. Do not: 1. lump multiple issues into one PR 2. include your favorite refactorings in with it 3. refactors must not change observable behavior Do: 1. minimize the PR even if the result of the PR suggests a refactoring 2. do that refactor SEPARATELY AFTER THE PR WAS MERGED
Feb 20
parent jxel <jxel gmall.com> writes:
On Thursday, 20 February 2020 at 10:58:36 UTC, Walter Bright 
wrote:
 On 2/19/2020 10:34 PM, Seb wrote:
 And I haven't even mentioned the bottleneck of any major DMD 
 PR needing to be approved by Walter.
Given the current regression list: https://issues.dlang.org/buglist.cgi?bug_severity=regression&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&list_id=230215&query_format=advanced I'm probably being too permissive. I'm usually the one who gets to fix them. I asked in this forum for some help isolating which PRs caused the regressions, to no avail. Major PRs *should* be regarded with skepticism. For example, I recently fixed a regression caused by a 2 or 3 hundred line PR, where the substance of the PR was one line and the rest of it was refactoring. Folks, with PRs, smaller, tightly focused PRs are better. Do not: 1. lump multiple issues into one PR 2. include your favorite refactorings in with it 3. refactors must not change observable behavior Do: 1. minimize the PR even if the result of the PR suggests a refactoring 2. do that refactor SEPARATELY AFTER THE PR WAS MERGED
I've fixed a regression caused by one of your PRs. It was a small change to the actual code, couldn't have been more than 10 lines, but it changed a few of the test case outputs. Funnily enough there were valid test cases with error messages that were just nuked over blindly (automatically probably). Errors happen, even for very simple things, if you don't take care it doesn't matter what the size of the PR is. People have trouble getting one PR merged, and you want them to try and split them up and somehow have that be coordinated properly when the PR have to be merged in a certain order and across 3 different projects? Yah, no thanks.
Feb 20
prev sibling parent reply Dibyendu Majumdar <d.majumdar gmail.com> writes:
On Thursday, 20 February 2020 at 06:34:45 UTC, Seb wrote:

 In my opinion DMD is only the default because switching 
 development efforts to LDC would be a rather big initial cost 
 (time investment) to setup/change the infrastructure and no one 
 wants to head this effort.

 if the d core team is focused in that direction, most people 
 will follow suit.
I would switch in a heartbeat and I am fairly confident to estimate that the same applies to almost all remaining active contributors who haven't left yet because of frustration.
I used to think that the backend should be abandoned in favour of LLVM but no more. I think it is worth having a fast backend. I was surprised how quickly dmd can be compiled compared to LLVM. It is excellent for a language designer to have full control over the implementation too as it helps play with ideas without having to fight with external libraries. Regards
Feb 20
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/20/2020 12:35 PM, Dibyendu Majumdar wrote:
 I used to think that the backend should be abandoned in favour of LLVM but no 
 more. I think it is worth having a fast backend. I was surprised how quickly
dmd 
 can be compiled compared to LLVM.
 
 It is excellent for a language designer to have full control over the 
 implementation too as it helps play with ideas without having to fight with 
 external libraries.
I might add that having a fast edit-compile-debug loop is indeed transformative. Sometimes tracking down a problem may require hundreds of times through that loop. On my dev machine, a full debug build of dmd takes 43 seconds. An incremental build (one file change) takes 9 seconds. (The build compiler is a Windows 32 bit dmd built with dmd.)
Feb 20
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 20 February 2020 at 21:04:16 UTC, Walter Bright 
wrote:
 On 2/20/2020 12:35 PM, Dibyendu Majumdar wrote:
 I used to think that the backend should be abandoned in favour 
 of LLVM but no more. I think it is worth having a fast 
 backend. I was surprised how quickly dmd can be compiled 
 compared to LLVM.
 
 It is excellent for a language designer to have full control 
 over the implementation too as it helps play with ideas 
 without having to fight with external libraries.
I might add that having a fast edit-compile-debug loop is indeed transformative. Sometimes tracking down a problem may require hundreds of times through that loop. On my dev machine, a full debug build of dmd takes 43 seconds. An incremental build (one file change) takes 9 seconds. (The build compiler is a Windows 32 bit dmd built with dmd.)
43 seconds? Are you still on an hdd?
Feb 20
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/20/2020 1:07 PM, Stefan Koch wrote:
 43 seconds?
 Are you still on an hdd?
No. But my computer is an older one.
Feb 20
prev sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 20 February 2020 at 03:35:13 UTC, jxel wrote:
 [..]
 Just how the official DMD isn't built with LDC, why there
 isn't a 64 bit version of DMD on Windows [..]
Technically not true anymore! https://github.com/dlang/installer/pull/425 https://github.com/dlang/installer/pull/417 See also: https://github.com/dlang/installer/pulls?q=is%3Apr+is%3Aclosed ;)
Feb 20
prev sibling parent reply matheus <matheus gmail.com> writes:
On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
 Unfortunately, the GDC and LDC optimizers consistently produce 
 code that outperforms code generated by the DMD backend by 
 20-30%, sometimes as high as 40% for CPU-intensive code...
Well these numbers are huge, I never thought the difference would be higher than 10% between these compilers, thanks for pointing this out. I use old DMD for my hobby projects in D, but now I'll test both GDC/LDC. Matheus.
Feb 18
parent reply kinke <noone nowhere.com> writes:
On Tuesday, 18 February 2020 at 18:50:44 UTC, matheus wrote:
 On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
 Unfortunately, the GDC and LDC optimizers consistently produce 
 code that outperforms code generated by the DMD backend by 
 20-30%, sometimes as high as 40% for CPU-intensive code...
Well these numbers are huge, I never thought the difference would be higher than 10% between these compilers, thanks for pointing this out.
Once you know that the DMD backend is a formidable one-man project, it should be clear as day that that cannot compete with huge eco-systems like LLVM and GCC with millions of man-hours, in terms of architecture support, optimizations and flexibility. It's nice for fast unoptimized codegen if you are only targeting x86, and probably features better debuginfos than LDC at the moment (well, ecept for Mac apparently), but to me, the DMD *backend* is clearly a dead end in the long run. DMD itself runs faster by 58% when compiled with LDC for a random compilation test case, see https://github.com/dlang/installer/pull/425#issuecomment-580868218. For number crunching, you can definitely expect much higher speed-ups if the auto-vectorizer kicks in. Whole-program optimization via LTO, even across C(++) and D, and PGO can improve your runtime further. On Tuesday, 18 February 2020 at 15:30:57 UTC, jxel wrote:
 The problem is that efforts are divided as a result.
As others have stated, that's not true. The vast majority of contributions to the dmd repo affect the front-end and thus implicitly benefit all compilers.
Feb 18
next sibling parent reply matheus <matheus gmail.com> writes:
On Tuesday, 18 February 2020 at 20:10:57 UTC, kinke wrote:
 On Tuesday, 18 February 2020 at 18:50:44 UTC, matheus wrote:
 On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
 Unfortunately, the GDC and LDC optimizers consistently 
 produce code that outperforms code generated by the DMD 
 backend by 20-30%, sometimes as high as 40% for CPU-intensive 
 code...
Well these numbers are huge, I never thought the difference would be higher than 10% between these compilers, thanks for pointing this out.
Once you know that the DMD backend is a formidable one-man project, it should be clear as day that that cannot compete with huge eco-systems like LLVM and GCC with millions of man-hours, in terms of architecture support, optimizations and flexibility. It's nice for fast unoptimized codegen if you are only targeting x86, and probably features better debuginfos than LDC at the moment (well, ecept for Mac apparently), but to me, the DMD *backend* is clearly a dead end in the long run.
Just to be clear, I am NOT diminishing any work, I just wasn't aware of these differences, because 20 to 30 or maybe 40% on runtime is something that you can't just ignore. I think that maybe is possible to balance things, like developing with DMD and generate final code with GDC/LDC to get faster runtime, well maybe some problems will happen mixing compilers and final result, but this is something that I will considerate for now on. Matheus.
Feb 18
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 18, 2020 at 09:22:18PM +0000, matheus via Digitalmars-d wrote:
[...]
 Just to be clear, I am NOT diminishing any work, I just wasn't aware
 of these differences, because 20 to 30 or maybe 40% on runtime is
 something that you can't just ignore.
It depends on your use case. YMMV, as they say. IME, the difference is most pronounced for CPU-intensive code where you have a lot of nested constructs, esp. in range-based code with extensive use of UFCS chains. Some years ago I analysed the assembly output to understand why this is so. According to my observations, one big factor is that the DMD inliner is rather anemic: it gives up at the slightest complication, thus missing out on what's commonly a domino-effect of optimizations: inlining the innermost function call opens up new optimization opportunities which causes the next outer level to be inlineable, etc., until most of the call stack has been inlined. But stop somewhere in between and you miss out the rest of the entire call chain's worth of optimizations. In range-based code, most of the method calls are very small, so quite often a long chain of nested calls can be reduced to just a few inlined instructions, so this is where LDC's very aggressive inliner really shines. In such cases, you can easily see 20-30% almost-guaranteed performance difference, and sometimes I've seen even up to 40%. I just ran a quick test again, as I posted in another reply, and again I see a >30% performance boost just from using ldc2 instead of dmd. The LLVM optimizer has been known to optimize entire call chains into the equivalent of a `return constValue;` by executing LLVM bytecode at compile-time to optimize away an entire subtree of function calls into a constant value. DMD's inliner, in comparison, balks at the sight of things as trivial as writing: auto myFunc(...) { if (cond) return result1; return result2; } vs. auto myFunc(...) { if (cond) return result1; else return result2; } so it tends to give up long before the code has been transformed into a state where it can be inlined one level further up the call chain. Thus, an entire domino-chain of optimizations is missed, and you end up with several nested function calls where it could have been reduced to just a few inlined instructions. When this happens in the inner loop, a 30% performance drop is pretty much expected.
 I think that maybe is possible to balance things, like developing with
 DMD and generate final code with GDC/LDC to get faster runtime, well
 maybe some problems will happen mixing compilers and final result, but
 this is something that I will considerate for now on.
[...] As somebody pointed out, using dmd for development and ldc2 for release build is definitely a viable approach. Recent LDC releases have been tracking DMD releases very closely, so excepting rare corner cases and bugs, functionality-wise the two compilers should pretty much be on par. And now that I've learned that ldmd2 doesn't preclude passing LDC-specific flags on the command-line, I might actually just adopt this approach. (The trouble with different command-line syntaxes is that compilers are not drop-in replacements for each other, which makes supporting multiple compilers in the same build script a pain. Certainly possible, and not even hard, but nonetheless a pain to write and maintain. With ldmd2 I can just standardize on DMD command-line syntax and have everything Just Work(tm). Best of both worlds.) T -- People demand freedom of speech to make up for the freedom of thought which they avoid. -- Soren Aabye Kierkegaard (1813-1855)
Feb 18
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/18/2020 1:52 PM, H. S. Teoh wrote:
 DMD's inliner, in comparison, balks at the sight of
 things as trivial as writing:
 
 	auto myFunc(...) {
 		if (cond) return result1;
 		return result2;
 	}
 
 vs.
 
 	auto myFunc(...) {
 		if (cond) return result1;
 		else return result2;
 	}
With: int fooa(int a, int b, int c) { if (a) return b; return c; } int foob(int a, int b, int c) { if (a) return b; else return c; } int test(int a, int b, int c) { return fooa(a, b, c) + foob(a, b, c); } the generated code for test() compiled with -inline -O is: push EAX cmp dword ptr 0Ch[ESP],0 je LE mov EAX,8[ESP] jmp short L11 LE: mov EAX,[ESP] L11: add EAX,EAX pop ECX ret 8 so it is inlining both forms. If you do have trivial functions that aren't being inlined, please let me know. Thanks!
Feb 18
parent reply Max Samukha <maxsamukha gmail.com> writes:
On Wednesday, 19 February 2020 at 05:42:19 UTC, Walter Bright 
wrote:

 so it is inlining both forms. If you do have trivial functions 
 that aren't being inlined, please let me know. Thanks!
Anything with a 'switch': void foo() { switch(true) { default: } }
Feb 18
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/18/2020 10:10 PM, Max Samukha wrote:
 On Wednesday, 19 February 2020 at 05:42:19 UTC, Walter Bright wrote:
 
 so it is inlining both forms. If you do have trivial functions that aren't 
 being inlined, please let me know. Thanks!
Anything with a 'switch': void foo() {     switch(true) {         default:     } }
That's true, it's also true for loops. Though trivial functions with switches seem to me to be rare. Keep in mind that the inliner is part of the dmd front end, not the back end. The inliner would be more effective if it was part of the back end, but that comes with a major memory/time penalty (intermediate code would have to be generated for all functions, even ones not being compiled, and kept around in memory).
Feb 19
next sibling parent reply Max Samukha <maxsamukha gmail.com> writes:
On Thursday, 20 February 2020 at 00:52:02 UTC, Walter Bright 
wrote:

 
 Anything with a 'switch':
 
 void foo() {
      switch(true) {
          default:
      }
 }
That's true, it's also true for loops. Though trivial functions with switches seem to me to be rare.
We encountered a case recently where we wanted to inline one. It was trivial to work around, though.
 Keep in mind that the inliner is part of the dmd front end, not 
 the back end. The inliner would be more effective if it was 
 part of the back end, but that comes with a major memory/time 
 penalty (intermediate code would have to be generated for all 
 functions, even ones not being compiled, and kept around in 
 memory).
Yes, I know about that. People are often willing to pay the cost for highly optimized builds.
Feb 20
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/20/2020 2:08 AM, Max Samukha wrote:
 We encountered a case recently where we wanted to inline one. It was trivial
to 
 work around, though.
I'm interested in evaluating common cases that don't inline but should.
Feb 20
parent reply Max Samukha <maxsamukha gmail.com> writes:
On Thursday, 20 February 2020 at 10:36:35 UTC, Walter Bright 
wrote:
 On 2/20/2020 2:08 AM, Max Samukha wrote:
 We encountered a case recently where we wanted to inline one. 
 It was trivial to work around, though.
I'm interested in evaluating common cases that don't inline but should.
Trivial functions that use a 'switch' to check membership in a small set of constants are still quite common I think, such as: bool isReserved(char c) { switch (c) { case ':', '/', '?'...: return true; default: return false; } }
Feb 20
parent Walter Bright <newshound2 digitalmars.com> writes:
Thank you.
Feb 20
prev sibling parent reply NaN <divide by.zero> writes:
On Thursday, 20 February 2020 at 00:52:02 UTC, Walter Bright 
wrote:
 On 2/18/2020 10:10 PM, Max Samukha wrote:
 On Wednesday, 19 February 2020 at 05:42:19 UTC, Walter Bright 
 wrote:
 
 so it is inlining both forms. If you do have trivial 
 functions that aren't being inlined, please let me know. 
 Thanks!
Anything with a 'switch': void foo() {     switch(true) {         default:     } }
That's true, it's also true for loops. Though trivial functions with switches seem to me to be rare. Keep in mind that the inliner is part of the dmd front end, not the back end. The inliner would be more effective if it was part of the back end, but that comes with a major memory/time penalty (intermediate code would have to be generated for all functions, even ones not being compiled, and kept around in memory).
Would moving the inliner to the back end still slow down the compile times even when inlining is turned off?
Feb 20
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/20/2020 3:17 AM, NaN wrote:
 Would moving the inliner to the back end still slow down the compile times
even 
 when inlining is turned off?
If it was done right, probably not.
Feb 20
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 18, 2020 at 08:10:57PM +0000, kinke via Digitalmars-d wrote:
[...]
 Once you know that the DMD backend is a formidable one-man project, it
 should be clear as day that that cannot compete with huge eco-systems
 like LLVM and GCC with millions of man-hours, in terms of architecture
 support, optimizations and flexibility. It's nice for fast unoptimized
 codegen if you are only targeting x86, and probably features better
 debuginfos than LDC at the moment (well, ecept for Mac apparently),
 but to me, the DMD *backend* is clearly a dead end in the long run.
Yeah, not to mention Walter himself has been mostly working on the front end these days, as he himself said he hasn't been keeping the backend up-to-date. A single person only has so many hours a day, no matter what a brilliant genius Walter is, there's still a limit as to what can be accomplished by a single person. And if DMD's very limited target arch support is already languishing from lack of sufficient time to keep up-to-date, support for other targets like Android is pretty much never going to happen, whereas using LDC gives you that support *today*. Add to that LDC's recent ability to cross-compile from Linux to Windows without needing to rebuild the entire toolchain, vs. dmd's inability to cross-compile at all (AFAIK), and the best choice from the user's POV is beyond obvious.
 DMD itself runs faster by 58% when compiled with LDC for a random
 compilation test case, see
 https://github.com/dlang/installer/pull/425#issuecomment-580868218.
IMNSHO, we should use LDC to compile official DMD releases, as someone has already mentioned. ;-)
 For number crunching, you can definitely expect much higher speed-ups
 if the auto-vectorizer kicks in. Whole-program optimization via LTO,
 even across C(++) and D, and PGO can improve your runtime further.
[...] Yeah, LDC's optimizer is light-years ahead of DMD's. I just tested it again on one of my latest projects, with a custom main() that runs a CPU-intensive part of the code in a loop 100 times. With `dmd -O -inline`, a typical run is about 25-26 seconds. With `ldmd2 -O3`, a typical run is about 16-17 seconds. We're looking at a ~30% increase in performance here, on exactly the same code and maximum optimization flags on both compilers. The numbers speak for themselves, really. As for compile times, `dmd -O -inline` typically takes about 6 seconds, whereas `ldmd2 -O3` typically takes about 8 seconds, which is about a 25% slowdown in compile times. Is a 2-second compile-time slowdown worth a 30% performance increase in the resulting executable? To me, it's an unquestionable yes. T -- Democracy: The triumph of popularity over principle. -- C.Bond
Feb 18
parent reply kinke <noone nowhere.com> writes:
On Tuesday, 18 February 2020 at 21:26:14 UTC, H. S. Teoh wrote:
 With `dmd -O -inline`, a typical run is about 25-26 seconds.  
 With `ldmd2 -O3`, a typical run is about 16-17 seconds.  We're 
 looking at a ~30% increase in performance here
Take care with such numbers - this sounds more like a 50% increase in performance. A speed-up by 4 (300% performance *increase*) results in 1/4th of the runtime, i.e., a runtime *decrease* by 75%.
Feb 18
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 18, 2020 at 10:14:54PM +0000, kinke via Digitalmars-d wrote:
 On Tuesday, 18 February 2020 at 21:26:14 UTC, H. S. Teoh wrote:
 With `dmd -O -inline`, a typical run is about 25-26 seconds.  With
 `ldmd2 -O3`, a typical run is about 16-17 seconds.  We're looking at
 a ~30% increase in performance here
Take care with such numbers - this sounds more like a 50% increase in performance. A speed-up by 4 (300% performance *increase*) results in 1/4th of the runtime, i.e., a runtime *decrease* by 75%.
Even better then. :-D T -- "A one-question geek test. If you get the joke, you're a geek: Seen on a California license plate on a VW Beetle: 'FEATURE'..." -- Joshua D. Wachs - Natural Intelligence, Inc.
Feb 18
prev sibling parent reply jxel <jxel gmall.com> writes:
On Tuesday, 18 February 2020 at 20:10:57 UTC, kinke wrote:
 but to me, the DMD *backend* is clearly a dead end in the long 
 run.
Its already at that point for me. PR sit stagnant for so long waiting on Walter by the time he looks at it it no longer compiles because of changes to the compiler, then proceeds to complain that it doesn't compile. It did 6 months ago.
Feb 18
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/18/2020 8:17 PM, jxel wrote:
 Its already at that point for me.
We all have to rebase our PRs from time to time. It comes with the territory, it's nothing terrible. I bet you could rebase whatever you've done in much less time than you spend complaining. If you don't know how to rebase with git, there are many people here who can lead you through it. Keep in mind that one of the reasons I'm not eager for major refactoring PRs is they cause everything in the queue to be rebased. Mine included.
Feb 20
parent reply jxel <jxel gmall.com> writes:
On Friday, 21 February 2020 at 06:05:15 UTC, Walter Bright wrote:
 On 2/18/2020 8:17 PM, jxel wrote:
 Its already at that point for me.
We all have to rebase our PRs from time to time. It comes with the territory, it's nothing terrible. I bet you could rebase whatever you've done in much less time than you spend complaining. If you don't know how to rebase with git, there are many people here who can lead you through it. Keep in mind that one of the reasons I'm not eager for major refactoring PRs is they cause everything in the queue to be rebased. Mine included.
So how do you get that I have a problem with rebasing from the DMD backend being buggy, a damn mess of code with no documentation, and there's really only one person maintaining it but they are too busy doing other things to bother doing work on the backend and related PRs, and that it should be replaced with LLVM because of a long list of benefits that outweighs the cons? My mind is shattered with the ki d of mental gymnastics that you have had to pull to get that point, did you even read the comment? Evidentially not. The troft is over there, not here -- cause that's about as relevant.
Feb 21
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/21/2020 11:16 AM, jxel wrote:
 So how do you get that I have a problem with rebasing from the DMD backend
being 
 buggy, a damn mess of code with no documentation, and there's really only one 
 person maintaining it but they are too busy doing other things to bother doing 
 work on the backend and related PRs, and that it should be replaced with LLVM 
 because of a long list of benefits that outweighs the cons?
You can always start contributing in a positive manner by addressing the issues that matter to you. This is an open source project. If you want currency, contribute. Not contributing coupled with relentless complaining moves you to the bottom of the list of people we listen to.
 My mind is shattered with the ki d of mental gymnastics that you have had to
pull
More friendly advice: berating people isn't going to work here. It's not like you're paying anyone.
Feb 21