www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DMD backend quality (Was: Re: DIP 1031--Deprecate Brace-Style Struct

reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
[Moving this to separate thread because it's tangential to DIP 1031
discussion.]

On Sat, Feb 15, 2020 at 03:59:39PM -0800, Walter Bright via Digitalmars-d wrote:

 On 2/15/2020 9:58 AM, jxel wrote:

 how poor quality the backend code is.


 
 Baloney.
 
 It's written in an older style, sure, but it's remarkably bug free
 considering how complex it is. The complexity is a result of the
 fiendishly complex x86 instructions set.
 
 The proof of its quality is it has successfully been extended from a
 purely 16 bit generator to 32 and extended again to 64 bit, and has
 been extended to support multiple ABIs and object file formats.
 
 The DFA optimizer has also proven to be fast and very robust.


Unfortunately, the GDC and LDC optimizers consistently produce code that
outperforms code generated by the DMD backend by 20-30%, sometimes as
high as 40% for CPU-intensive code. DMD also tends to have more codegen
bugs when run with -O -inline (which, ironically, still produces
inferior code to what LDC/GDC produces with -O).

I'm sure, given enough time and effort, you could bring it to par, but
after >5 years of waiting, I've essentially given up on the DMD backend
for performance-sensitive code.  I now use LDC for my larger projects,
and would probably start new projects on LDC rather than DMD.

LDC can also cross-compile to Windows from Linux, which allows me the
very convenient setup of a single build script that builds both Linux
and Windows executables simultaneously.  With DMD I need a VM or at
least Wine (with its very slow and resource-intensive startup times) to
be able to do this. And LDC can target a *lot* more architectures than
x86 on Linux/Windows, such as WebAssembly, Android, etc.. Again, DMD
loses out here.

Of course, DMD is still very fast at compiling and useful for smaller
utilities and script-like programs where performance isn't critical, or
for running unittests for isolated modules during development due to
fast turnaround times. But codegen quality is definitely not among the
reasons I still use DMD.


T

-- 
Computers shouldn't beep through the keyhole.
Feb 15 2020
next sibling parent reply NaN <divide by.zero> writes:
On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:

 [Moving this to separate thread because it's tangential to DIP 
 1031 discussion.]

 On Sat, Feb 15, 2020 at 03:59:39PM -0800, Walter Bright via 
 Digitalmars-d wrote:

 On 2/15/2020 9:58 AM, jxel wrote:

 how poor quality the backend code is.


 
 Baloney.
 
 It's written in an older style, sure, but it's remarkably bug 
 free considering how complex it is. The complexity is a result 
 of the fiendishly complex x86 instructions set.
 
 The proof of its quality is it has successfully been extended 
 from a purely 16 bit generator to 32 and extended again to 64 
 bit, and has been extended to support multiple ABIs and object 
 file formats.
 
 The DFA optimizer has also proven to be fast and very robust.


 Unfortunately, the GDC and LDC optimizers consistently produce 
 code that outperforms code generated by the DMD backend by 
 20-30%, sometimes as high as 40% for CPU-intensive code. DMD 
 also tends to have more codegen bugs when run with -O -inline 
 (which, ironically, still produces inferior code to what 
 LDC/GDC produces with -O).


I was seeing worse than that last time i used DMD, it was mostly 
int code, no fpu, and it was around 60% slower with DMD than LDC. 
Haven't used it since. SIMD support with DMD was awful at the 
time too, dont know if thats improved.
Feb 16 2020
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Sunday, 16 February 2020 at 10:52:33 UTC, NaN wrote:

 On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:

 [Moving this to separate thread because it's tangential to DIP 
 1031 discussion.]

 On Sat, Feb 15, 2020 at 03:59:39PM -0800, Walter Bright via 
 Digitalmars-d wrote:

 On 2/15/2020 9:58 AM, jxel wrote:

 how poor quality the backend code is.


 
 Baloney.
 
 It's written in an older style, sure, but it's remarkably bug 
 free considering how complex it is. The complexity is a 
 result of the fiendishly complex x86 instructions set.
 
 The proof of its quality is it has successfully been extended 
 from a purely 16 bit generator to 32 and extended again to 64 
 bit, and has been extended to support multiple ABIs and 
 object file formats.
 
 The DFA optimizer has also proven to be fast and very robust.


 Unfortunately, the GDC and LDC optimizers consistently produce 
 code that outperforms code generated by the DMD backend by 
 20-30%, sometimes as high as 40% for CPU-intensive code. DMD 
 also tends to have more codegen bugs when run with -O -inline 
 (which, ironically, still produces inferior code to what 
 LDC/GDC produces with -O).


 I was seeing worse than that last time i used DMD, it was 
 mostly int code, no fpu, and it was around 60% slower with DMD 
 than LDC. Haven't used it since. SIMD support with DMD was 
 awful at the time too, dont know if thats improved.


This discussion has veered off topic. Let's please stick to 
discussion the DIP. Thanks!
Feb 16 2020
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 16 February 2020 at 10:55:26 UTC, Mike Parker wrote:

 This discussion has veered off topic. Let's please stick to 
 discussion the DIP. Thanks!


Hence why it was split off into a different thread...
Feb 16 2020
parent Mike Parker <aldacron gmail.com> writes:
On Sunday, 16 February 2020 at 11:06:53 UTC, Nicholas Wilson 
wrote:

 On Sunday, 16 February 2020 at 10:55:26 UTC, Mike Parker wrote:

 This discussion has veered off topic. Let's please stick to 
 discussion the DIP. Thanks!


 Hence why it was split off into a different thread...


Didn't catch that. Thanks.
Feb 16 2020
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2020 2:52 AM, NaN wrote:

 I was seeing worse than that last time i used DMD, it was mostly int code, no 
 fpu, and it was around 60% slower with DMD than LDC. Haven't used it since.
SIMD 
 support with DMD was awful at the time too, dont know if thats improved.


The SIMD code was substantially improved a year or two ago. I added common 
subexpression support and improved the register allocator for it.

The main problem is I made a mistake in putting the inliner in the DMD front 
end. It should be in the back end, where it'll be a lot more effective.
Feb 16 2020
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 17/02/2020 11:15 AM, Walter Bright wrote:

 On 2/16/2020 2:52 AM, NaN wrote:

 I was seeing worse than that last time i used DMD, it was mostly int 
 code, no fpu, and it was around 60% slower with DMD than LDC. Haven't 
 used it since. SIMD support with DMD was awful at the time too, dont 
 know if thats improved.


 
 The SIMD code was substantially improved a year or two ago. I added 
 common subexpression support and improved the register allocator for it.
 
 The main problem is I made a mistake in putting the inliner in the DMD 
 front end. It should be in the back end, where it'll be a lot more 
 effective.


Perhaps you could mentor somebody in say SaoC to build that up?

A chance to do that for an inspiring compiler dev would be invaluable.
Feb 16 2020
parent RazvanN <razvan.nitu1305 gmail.com> writes:
On Sunday, 16 February 2020 at 22:29:01 UTC, rikki cattermole 
wrote:

 On 17/02/2020 11:15 AM, Walter Bright wrote:

 On 2/16/2020 2:52 AM, NaN wrote:

 I was seeing worse than that last time i used DMD, it was 
 mostly int code, no fpu, and it was around 60% slower with 
 DMD than LDC. Haven't used it since. SIMD support with DMD 
 was awful at the time too, dont know if thats improved.


 
 The SIMD code was substantially improved a year or two ago. I 
 added common subexpression support and improved the register 
 allocator for it.
 
 The main problem is I made a mistake in putting the inliner in 
 the DMD front end. It should be in the back end, where it'll 
 be a lot more effective.


 Perhaps you could mentor somebody in say SaoC to build that up?

 A chance to do that for an inspiring compiler dev would be 
 invaluable.


Or it could be a good project for GSOC if we get accepted, but 
Walter should be willing to mentor the student.
Feb 16 2020
prev sibling parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Sunday, 16 February 2020 at 22:15:22 UTC, Walter Bright wrote:

 On 2/16/2020 2:52 AM, NaN wrote:

 I was seeing worse than that last time i used DMD, it was 
 mostly int code, no fpu, and it was around 60% slower with DMD 
 than LDC. Haven't used it since. SIMD support with DMD was 
 awful at the time too, dont know if thats improved.


 The SIMD code was substantially improved a year or two ago. I 
 added common subexpression support and improved the register 
 allocator for it.

 The main problem is I made a mistake in putting the inliner in 
 the DMD front end. It should be in the back end, where it'll be 
 a lot more effective.


Replying here so as not to pollute the other thread.
Since PR are welcome, can you please take care of 
https://github.com/dlang/dmd/pull/10200 which hasn't been 
reviewed for 6+ months ?

This is the main concern I have with the DMD backend: while it's 
an incredible one-man effort (and I don't think people are giving 
you enough credit for this achievement), it *is* a one-man 
effort, with a bus factor of 1.

It does produce slower code than other backends, generate worse 
debug infos (DWARF debug infos are completely broken on OSX), 
does not support cross-compilation except for `-m32`/`-m64` and 
has a few ABI issues which haven't been solved in years. And 
that's completely understandable, given that amount of work 
required to make things work.

That being said, DMD is still my default compiler for its 
compilation speed (although it's loosing a bit more with every 
release).
Feb 16 2020
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2020 6:07 PM, Mathias Lang wrote:

 (DWARF debug infos are completely broken on OSX),


I hear this now and then about the DWARF output, but there are ZERO bug reports 
on it.

I use gdb myself on OSX and Linux, and while I don't much care for gdb, it
works.
Feb 16 2020
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 17 February 2020 at 02:54:09 UTC, Walter Bright wrote:

 I hear this now and then about the DWARF output, but there are 
 ZERO bug reports on it.


I just searched "dwarf" on bugzilla and several came up

https://issues.dlang.org/buglist.cgi?quicksearch=dwarf&list_id=230166

from 2016
https://issues.dlang.org/show_bug.cgi?id=15747

from 2020
https://issues.dlang.org/show_bug.cgi?id=20510


and many more.

I haven't confirmed these myself but it is pretty obviously not 
zero in bugzilla.

(my personal experience btw is that bugzilla is where issues go 
to be ignored, lost, or forgotten for years.)
Feb 16 2020
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2020 7:02 PM, Adam D. Ruppe wrote:

 (my personal experience btw is that bugzilla is where issues go to be ignored, 
 lost, or forgotten for years.)


Anyone can submit PRs to fix bugzilla issues, including you.
Feb 16 2020
parent reply jxel <jxel gmall.com> writes:
On Monday, 17 February 2020 at 05:11:14 UTC, Walter Bright wrote:

 On 2/16/2020 7:02 PM, Adam D. Ruppe wrote:

 (my personal experience btw is that bugzilla is where issues 
 go to be ignored, lost, or forgotten for years.)


 Anyone can submit PRs to fix bugzilla issues, including you.


That doesn't mean they will get merged. There's lots of PR open 
that have been open for years without any sort of feedback. Why 
would anybody waste their free time to simply have their work 
ignored?
Feb 17 2020
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2020 10:45 AM, jxel wrote:

 That doesn't mean they will get merged. There's lots of PR open that have been 
 open for years without any sort of feedback. Why would anybody waste their
free 
 time to simply have their work ignored?


The dmd compiler itself has 24,254 commits from 195 contributors.

   https://github.com/dlang/dmd

Phobos has 17,927 commits from 367 contributors.

   https://github.com/dlang/phobos

Druntime has 7,023 commits from 198 contributors.

   https://github.com/dlang/druntime
Feb 17 2020
next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Monday, 17 February 2020 at 19:35:53 UTC, Walter Bright wrote:

 On 2/17/2020 10:45 AM, jxel wrote:

 That doesn't mean they will get merged. There's lots of PR 
 open that have been open for years without any sort of 
 feedback. Why would anybody waste their free time to simply 
 have their work ignored?


 The dmd compiler itself has 24,254 commits from 195 
 contributors.

   https://github.com/dlang/dmd

 Phobos has 17,927 commits from 367 contributors.

   https://github.com/dlang/phobos

 Druntime has 7,023 commits from 198 contributors.

   https://github.com/dlang/druntime


This gives somewhat an idea but actually these numbers, as 
provided by GH, are innaccurate, they don't include people who 
have left GH. If you really want to give numbers then you must 
use git to find the unique authors by email.
Feb 17 2020
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2020 6:07 PM, Basile B. wrote:

 On Monday, 17 February 2020 at 19:35:53 UTC, Walter Bright wrote:

 On 2/17/2020 10:45 AM, jxel wrote:

 That doesn't mean they will get merged. There's lots of PR open that have 
 been open for years without any sort of feedback. Why would anybody waste 
 their free time to simply have their work ignored?


 The dmd compiler itself has 24,254 commits from 195 contributors.

   https://github.com/dlang/dmd

 Phobos has 17,927 commits from 367 contributors.

   https://github.com/dlang/phobos

 Druntime has 7,023 commits from 198 contributors.

   https://github.com/dlang/druntime


 
 This gives somewhat an idea but actually these numbers, as provided by GH, are 
 innaccurate, they don't include people who have left GH. If you really want to 
 give numbers then you must use git to find the unique authors by email.


When it says 195 contributors, what's inaccurate about that? Are you saying
some 
are counted multiple times because they submit PRs under multiple aliases?
Feb 17 2020
parent reply Basile B. <b2.temp gmx.com> writes:
On Tuesday, 18 February 2020 at 07:20:17 UTC, Walter Bright wrote:

 On 2/17/2020 6:07 PM, Basile B. wrote:

 On Monday, 17 February 2020 at 19:35:53 UTC, Walter Bright 
 wrote:

 [...]


 
 This gives somewhat an idea but actually these numbers, as 
 provided by GH, are innaccurate, they don't include people who 
 have left GH. If you really want to give numbers then you must 
 use git to find the unique authors by email.


 When it says 195 contributors, what's inaccurate about that? 
 Are you saying some are counted multiple times because they 
 submit PRs under multiple aliases?


No I say that some are not counted because they left github. For 
example me, I'm not in the list, despite of 80+ commits in DMD.
Feb 17 2020
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2020 11:57 PM, Basile B. wrote:

 No I say that some are not counted because they left github. For example me,
I'm 
 not in the list, despite of 80+ commits in DMD.


Oh, I see. So it's an undercount, the actual number of contributors is higher.

Anyhow, I can see becoming an inactive github user, but why leave, especially 
since github contributions can be a great asset for your resume? A number of 
significant D contributors have been able to leverage that into getting well 
compensated positions, a result I'm pretty proud of. I like to see you guys 
being successful.
Feb 18 2020
prev sibling parent jxel <jxel gmall.com> writes:
On Monday, 17 February 2020 at 19:35:53 UTC, Walter Bright wrote:

 On 2/17/2020 10:45 AM, jxel wrote:

 That doesn't mean they will get merged. There's lots of PR 
 open that have been open for years without any sort of 
 feedback. Why would anybody waste their free time to simply 
 have their work ignored?


 The dmd compiler itself has 24,254 commits from 195 
 contributors.

   https://github.com/dlang/dmd

 Phobos has 17,927 commits from 367 contributors.

   https://github.com/dlang/phobos

 Druntime has 7,023 commits from 198 contributors.

   https://github.com/dlang/druntime


The contributions drop to single digits fairly quickly. People 
aren't coming back to contribute, I wonder why. Pointing to 
numbers doesn't mean there's are countless PR that are being 
neglected. All you are showing is that only the handful of people 
that can just ask someone else to merge their work gets it merged 
regularly.
Feb 17 2020
prev sibling parent Basile B. <b2.temp gmx.com> writes:
On Monday, 17 February 2020 at 18:45:25 UTC, jxel wrote:

 On Monday, 17 February 2020 at 05:11:14 UTC, Walter Bright 
 wrote:

 On 2/16/2020 7:02 PM, Adam D. Ruppe wrote:

 (my personal experience btw is that bugzilla is where issues 
 go to be ignored, lost, or forgotten for years.)


 Anyone can submit PRs to fix bugzilla issues, including you.


 That doesn't mean they will get merged. There's lots of PR open 
 that have been open for years without any sort of feedback. Why 
 would anybody waste their free time to simply have their work 
 ignored?


This is not true. If a lot of PR are opened it's not because they 
are ignored. There are plenty of valid reasons and none matches 
to what your reductive POV.

- additional changes are requested but the author does not make 
these changes ;
- the changes cannot be merged because they need to be rebased ;
- the changes cannot be merged until other another PR get merged 
(label BLOCKED) ;
- the changes cannot be merged because they doesn't pass the test 
suite ;
- the changes are not accepted, they don't follow the language 
direction ;
- the PR is a draft or a WIP, so it get a bit ignored because the 
author ask for it ;
- the changes are too big, hard to understand ;
- the author doesn't care anymore ;
- the changes were not approved explicitly but the author doesn't 
close the PR ;
etc.

see 
https://github.com/dlang/dmd/pulls?q=is%3Apr+is%3Aopen+sort%3Acomments-asc

There is always at least one comment (excluding the one auto 
generated by the bot)
Feb 18 2020
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2020 7:02 PM, Adam D. Ruppe wrote:

 I just searched "dwarf" on bugzilla and several came up
 
 https://issues.dlang.org/buglist.cgi?quicksearch=dwarf&list_id=230166


I see the situation has changed fairly recently. That's good. I haven't looked 
at dwarf for a while.
Feb 16 2020
prev sibling next sibling parent reply Mathias Lang <pro.mathias.lang gmail.com> writes:
On Monday, 17 February 2020 at 02:54:09 UTC, Walter Bright wrote:

 On 2/16/2020 6:07 PM, Mathias Lang wrote:

 (DWARF debug infos are completely broken on OSX),


 I hear this now and then about the DWARF output, but there are 
 ZERO bug reports on it.

 I use gdb myself on OSX and Linux, and while I don't much care 
 for gdb, it works.


Really ?

- I mentioned last time in the Skype meeting;
- we put a bounty on it (blog post: 
https://dlang.org/blog/2019/08/17/bug-bounties-have-arrived/ and 
bounty itself: 
https://www.flipcause.com/secure/cause_pdetails/NjI2NjQ=)
- there is a bugzilla issue: 
https://issues.dlang.org/show_bug.cgi?id=20460
- I opened a PR detailing the issue in great details: 
https://github.com/dlang/dmd/pull/10722

And people are not going to mention "DWARF" in bugzilla, they're 
going to say "stack traces are broken but I can't reproduce with 
a small example" (in this case it depends on the order the root 
modules are passed to the compiler). I was lucky to be familiar 
enough with the compiler to notice the pattern, but it did take 
me a while.

Also which OS version / gdb version are you using ? On a 
relatively recent Mac, GDB barely works and LLDB is a much better 
place to get started.
Feb 16 2020
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2020 9:44 PM, Mathias Lang wrote:

 On Monday, 17 February 2020 at 02:54:09 UTC, Walter Bright wrote:

 I use gdb myself on OSX and Linux, and while I don't much care for gdb, it
works.


 
 Really ?


Well, somehow I debugged the compiler on OSX.



 Also which OS version / gdb version are you using ?


Don't recall, but old.
Feb 16 2020
next sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Monday, 17 February 2020 at 07:11:47 UTC, Walter Bright wrote:

 On 2/16/2020 9:44 PM, Mathias Lang wrote:

 On Monday, 17 February 2020 at 02:54:09 UTC, Walter Bright 
 wrote:

 I use gdb myself on OSX and Linux, and while I don't much 
 care for gdb, it works.


 
 Really ?


 Well, somehow I debugged the compiler on OSX.

 [..]


I think Mathias was replying to this part:


 I hear this now and then about the DWARF output, but there are 
 ZERO bug reports on it.
Feb 17 2020
prev sibling parent Jacob Carlborg <doob me.com> writes:
On Monday, 17 February 2020 at 07:11:47 UTC, Walter Bright wrote:


 Don't recall, but old.


You can run `sw_vers -productVersion` in the terminal to find out.

--
/Jacob Carlborg
Feb 17 2020
prev sibling parent Jacob Carlborg <doob me.com> writes:
On Monday, 17 February 2020 at 05:44:32 UTC, Mathias Lang wrote:


 Really ?


Yeah, I've tested with DMD combined with GDB and it works. But it 
doesn't work with LLDB. LDC of course work together with LLDB.

--
/Jacob Carlborg
Feb 17 2020
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On Monday, 17 February 2020 at 02:54:09 UTC, Walter Bright wrote:


 I hear this now and then about the DWARF output, but there are 
 ZERO bug reports on it.


As others have mentioned, the bug reports might not include the 
word "dwarf".


 I use gdb myself on OSX and Linux, and while I don't much care 
 for gdb, it works.


This explains a lot, and is also what I suspected. I'm guessing 
you haven't updated your tools since the original port of DMD to 
macOS. Nobody cares about GDB on macOS anymore. Apple has since 
long abandoned the GCC toolchain and now relies exclusively on 
the LLVM toolchain, as I've mentioned in several places on 
several occasions. That means Clang and LLDB.

There are several issues reported for this:

https://issues.dlang.org/show_bug.cgi?id=18527
https://issues.dlang.org/show_bug.cgi?id=18612

You might want to search for "LLDB" as well: 
https://issues.dlang.org/buglist.cgi?quicksearch=lldb

--
/Jacob Carlborg
Feb 17 2020
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2020 12:25 AM, Jacob Carlborg wrote:

 Apple has since long abandoned the GCC toolchain and now 
 relies exclusively on the LLVM toolchain, as I've mentioned in several places
on 
 several occasions. That means Clang and LLDB.
 
 There are several issues reported for this:
 
 https://issues.dlang.org/show_bug.cgi?id=18527
 https://issues.dlang.org/show_bug.cgi?id=18612
 
 You might want to search for "LLDB" as well: 
 https://issues.dlang.org/buglist.cgi?quicksearch=lldb


Thank you, you're right, I've never used LLDB. It never occurred to me that it 
would fail where gdb succeeds. They're both supposed to be compatible with
dwarf.

Anyhow, do you want to take a look at the dwarf output and see what's
different? 
The dumpobj tool will pretty-print it.
Feb 17 2020
parent reply Jacob Carlborg <doob me.com> writes:
On 2020-02-17 20:44, Walter Bright wrote:


 Anyhow, do you want to take a look at the dwarf output and see what's 
 different? The dumpobj tool will pretty-print it.


I added the output of `dwarfdump` to the issue [1], both of DMD and LDC. 
I also extracted the actual differences in a separate comment [2].

[1] https://issues.dlang.org/show_bug.cgi?id=18527
[2] https://issues.dlang.org/show_bug.cgi?id=18527#c5

-- 
/Jacob Carlborg
Feb 19 2020
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/19/2020 5:21 AM, Jacob Carlborg wrote:

 On 2020-02-17 20:44, Walter Bright wrote:

 Anyhow, do you want to take a look at the dwarf output and see what's 
 different?


 [1] https://issues.dlang.org/show_bug.cgi?id=18527
 [2] https://issues.dlang.org/show_bug.cgi?id=18527#c5


Thank you.
Feb 21 2020
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/16/2020 6:07 PM, Mathias Lang wrote:

 Since PR are welcome, can you please take care of 
 https://github.com/dlang/dmd/pull/10200 which hasn't been reviewed for 6+
months ?


It fails all tests on Posix:

ERROR: dmd/target.d(408): Error: no property `isPOSIX` for type `Param`

Also, it's pretty hard to deduce the strategy from the low level detail.
Feb 16 2020
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:

 Of course, DMD is still very fast at compiling and useful for 
 smaller utilities and script-like programs where performance 
 isn't critical, or for running unittests for isolated modules 
 during development due to fast turnaround times. But codegen 
 quality is definitely not among the reasons I still use DMD.


I still happily use DMD for most development because it still 
builds faster.
(On Windows, try `dub -a x86 --compiler dmd` and see)

But yes adding -O -inline frightens me.
I think DMD should play its strength of generating code very 
quickly :) even at the expense of output performance.
Feb 17 2020
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2020 1:14 PM, Guillaume Piolat wrote:

 I still happily use DMD for most development because it still builds faster.
 (On Windows, try `dub -a x86 --compiler dmd` and see)
 
 But yes adding -O -inline frightens me.
 I think DMD should play its strength of generating code very quickly :) even
at 
 the expense of output performance.


No worries, there are no plans to change the fast path through the backend.
Feb 17 2020
parent reply jxel <jxel gmall.com> writes:
On Monday, 17 February 2020 at 22:25:17 UTC, Walter Bright wrote:

 On 2/17/2020 1:14 PM, Guillaume Piolat wrote:

 I still happily use DMD for most development because it still 
 builds faster.
 (On Windows, try `dub -a x86 --compiler dmd` and see)
 
 But yes adding -O -inline frightens me.
 I think DMD should play its strength of generating code very 
 quickly :) even at the expense of output performance.


 No worries, there are no plans to change the fast path through 
 the backend.


If DMD used LDC to build itself, it'd run even faster. There's a 
PR open for that iirc.
Feb 17 2020
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Tuesday, 18 February 2020 at 04:11:51 UTC, jxel wrote:

 On Monday, 17 February 2020 at 22:25:17 UTC, Walter Bright 
 wrote:

 On 2/17/2020 1:14 PM, Guillaume Piolat wrote:

 I still happily use DMD for most development because it still 
 builds faster.
 (On Windows, try `dub -a x86 --compiler dmd` and see)
 
 But yes adding -O -inline frightens me.
 I think DMD should play its strength of generating code very 
 quickly :) even at the expense of output performance.


 No worries, there are no plans to change the fast path through 
 the backend.


 If DMD used LDC to build itself, it'd run even faster. There's 
 a PR open for that iirc.


Yes please !
Feb 18 2020
parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 18 February 2020 at 15:00:47 UTC, Guillaume Piolat 
wrote:

 Yes please !


https://github.com/dlang/installer/pull/425
Feb 18 2020
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Feb 17, 2020 at 09:14:31PM +0000, Guillaume Piolat via Digitalmars-d
wrote:

 On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:

 Of course, DMD is still very fast at compiling and useful for
 smaller utilities and script-like programs where performance isn't
 critical, or for running unittests for isolated modules during
 development due to fast turnaround times. But codegen quality is
 definitely not among the reasons I still use DMD.


 
 I still happily use DMD for most development because it still builds
 faster.  (On Windows, try `dub -a x86 --compiler dmd` and see)


I don't use dub because it increases my build times by like 2X or more.
I do still use DMD for when I'm working on unittesting a single module
and don't need the entire program linked (`dmd -unittest -i -main -run
mymodule.d` is amazing).  But for my main program I prefer LDC, because
I'm frightened of discrepancies between compilers coming back to bite me
in the behind, plus for small utilities I sometimes just use debug
builds for "production" use, and LDC gives me basically a free 20%
performance boost for no extra effort.  Plus, it's annoying to have to
switch between different compiler options syntax just so I can use both
dmd and ldc2 for my builds.



 But yes adding -O -inline frightens me.


I've encountered quite a few obscure codegen bugs related to -O -inline.
It's great that they are fixed now, but from a user's POV it's a rather
unpleasant experience to have to deal with compiler bugs that only show
up with -O -inline.  It exarcerbates the problem of dmd producing slow
code, because now I'm forced to live with *not* using -O -inline.  It's
yet another reason to use LDC instead.



 I think DMD should play its strength of generating code very quickly
 :) even at the expense of output performance.


The thing is, for all the effort Walter put into making dmd as fast as
possible, when you're writing template-heavy code it's *still* slow.
Well, you say, then just write less templates.  But the problem is,
sometimes these compiler performance problems aren't obvious until your
project grows larger, and by then the templated code is already heavily
ingrained in your codebase and would be too onerous to rewrite.

So if I'm going to have to wait several seconds for my template-heavy
code to compile anyway, why not use a compiler that gives me better
codegen in return?


T

-- 
It is impossible to make anything foolproof because fools are so ingenious. --
Sammy
Feb 18 2020
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 18 February 2020 at 18:00:03 UTC, H. S. Teoh wrote:

 

 Plus, it's annoying to have to switch between different 
 compiler options syntax just so I can use both dmd and ldc2 for 
 my builds.


Why not use ldmd2 then?
Feb 18 2020
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 18, 2020 at 06:21:21PM +0000, jmh530 via Digitalmars-d wrote:

 On Tuesday, 18 February 2020 at 18:00:03 UTC, H. S. Teoh wrote:

 
 Plus, it's annoying to have to switch between different compiler
 options syntax just so I can use both dmd and ldc2 for my builds.


 
 Why not use ldmd2 then?


Because some LDC flags are not accessible through the ldmd2 interface.


T

-- 
Error: Keyboard not attached. Press F1 to continue. -- Yoon Ha Lee, CONLANG
Feb 18 2020
parent reply kinke <noone nowhere.com> writes:
On Tuesday, 18 February 2020 at 18:40:34 UTC, H. S. Teoh wrote:

 On Tue, Feb 18, 2020 at 06:21:21PM +0000, jmh530 via

 Why not use ldmd2 then?


 Because some LDC flags are not accessible through the ldmd2 
 interface.


Just try it - unknown flags are forwarded to ldc2.exe.
Feb 18 2020
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 18, 2020 at 07:44:49PM +0000, kinke via Digitalmars-d wrote:

 On Tuesday, 18 February 2020 at 18:40:34 UTC, H. S. Teoh wrote:

 On Tue, Feb 18, 2020 at 06:21:21PM +0000, jmh530 via

 Why not use ldmd2 then?


 
 Because some LDC flags are not accessible through the ldmd2
 interface.


 
 Just try it - unknown flags are forwarded to ldc2.exe.


It works!!  Thanks for the tip!


T

-- 
Food and laptops don't mix.
Feb 18 2020
prev sibling next sibling parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:

 ...
 Of course, DMD is still very fast at compiling and useful for 
 smaller utilities and script-like programs where performance 
 isn't critical, or for running unittests for isolated modules 
 during development due to fast turnaround times. But codegen 
 quality is definitely not among the reasons I still use DMD.
 ...


I do not know about others, but I personally do not expect DMD to 
be as performant as people would like it to be. As a REFERENCE 
COMPILER I expect it to be as close to the current language 
specification as possible. DMD does that beautifully, and on top 
of that does a pretty good job at compiling code quite fast. I 
can't ask for more! - If they optimize the codegen to generate 
better code, that is fine, but not essential, as I use GCC 
(others may prefer LDC) for release builds.
Feb 18 2020
parent reply jxel <jxel gmall.com> writes:
On Tuesday, 18 February 2020 at 13:04:36 UTC, Dejan Lekic wrote:

 On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:

 ...
 Of course, DMD is still very fast at compiling and useful for 
 smaller utilities and script-like programs where performance 
 isn't critical, or for running unittests for isolated modules 
 during development due to fast turnaround times. But codegen 
 quality is definitely not among the reasons I still use DMD.
 ...


 I do not know about others, but I personally do not expect DMD 
 to be as performant as people would like it to be. As a 
 REFERENCE COMPILER I expect it to be as close to the current 
 language specification as possible. DMD does that beautifully, 
 and on top of that does a pretty good job at compiling code 
 quite fast. I can't ask for more! - If they optimize the 
 codegen to generate better code, that is fine, but not 
 essential, as I use GCC (others may prefer LDC) for release 
 builds.


The problem is that efforts are divided as a result. Work still 
has to be maintained on the backend. As someone showed there's 
multiple PR that have stagnated because no one is willing to look 
at them that modify the backend to comply with the C ABI. Bugs 
that have been open for years. Bugs that don't exist in LDC, even 
though that's basically being maintained by one person. If people 
want a fast backend, you can build a custom backend for 
performance in LLVM, a few projects have done this already. But 
honestly that's a waste of time. The slowest part of D is the 
frontend and CTFE not the backend in LDC. That's not even 
mentioning the constant out of memory problems I experience and 
have to try to optimize the compiler for with my code.
Feb 18 2020
next sibling parent reply bachmeier <no spam.net> writes:
On Tuesday, 18 February 2020 at 15:30:57 UTC, jxel wrote:

 On Tuesday, 18 February 2020 at 13:04:36 UTC, Dejan Lekic wrote:

 On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:

 ...
 Of course, DMD is still very fast at compiling and useful for 
 smaller utilities and script-like programs where performance 
 isn't critical, or for running unittests for isolated modules 
 during development due to fast turnaround times. But codegen 
 quality is definitely not among the reasons I still use DMD.
 ...


 I do not know about others, but I personally do not expect DMD 
 to be as performant as people would like it to be. As a 
 REFERENCE COMPILER I expect it to be as close to the current 
 language specification as possible. DMD does that beautifully, 
 and on top of that does a pretty good job at compiling code 
 quite fast. I can't ask for more! - If they optimize the 
 codegen to generate better code, that is fine, but not 
 essential, as I use GCC (others may prefer LDC) for release 
 builds.


 The problem is that efforts are divided as a result. Work still 
 has to be maintained on the backend. As someone showed there's 
 multiple PR that have stagnated because no one is willing to 
 look at them that modify the backend to comply with the C ABI. 
 Bugs that have been open for years. Bugs that don't exist in 
 LDC, even though that's basically being maintained by one 
 person. If people want a fast backend, you can build a custom 
 backend for performance in LLVM, a few projects have done this 
 already. But honestly that's a waste of time. The slowest part 
 of D is the frontend and CTFE not the backend in LDC. That's 
 not even mentioning the constant out of memory problems I 
 experience and have to try to optimize the compiler for with my 
 code.


DMD already exists, nobody's forced to use it, and I doubt it's 
sucking any resources away from LDC.
Feb 18 2020
parent IGotD- <nise nise.com> writes:
On Tuesday, 18 February 2020 at 15:41:36 UTC, bachmeier wrote:

 DMD already exists, nobody's forced to use it, and I doubt it's 
 sucking any resources away from LDC.


Is there one thing DMD does better, that is being able to debug 
with Visual Studio. With DMD you get full debugging information 
including locals, almost as you were debugging a C++ program. 
When I compile with LDC I just get basic debugging information. I 
presume that this is because LDC emits a different debugging 
format.
Feb 18 2020
prev sibling parent reply drug <drug2004 bk.ru> writes:
On 2/18/20 6:30 PM, jxel wrote:

 
 The problem is that efforts are divided as a result. Work still has to 
 be maintained on the backend. As someone showed there's multiple PR that 
 have stagnated because no one is willing to look at them that modify the 
 backend to comply with the C ABI. Bugs that have been open for years. 
 Bugs that don't exist in LDC, even though that's basically being 
 maintained by one person. If people want a fast backend, you can build a 
 custom backend for performance in LLVM, a few projects have done this 
 already. But honestly that's a waste of time. The slowest part of D is 
 the frontend and CTFE not the backend in LDC. That's not even mentioning 
 the constant out of memory problems I experience and have to try to 
 optimize the compiler for with my code.
 


IMO you are wrong if you think that dropping dmd will increase man power 
in ldc/gdc land. Who wants to contribute to ldc/gdc already doing it. It 
is open source - people contribute to projects they like, you can not 
tell them what to do. Efforts are not divided - if dmd would be dropped 
people who doesn't contribute to ldc/gdc won't start doing that.
Feb 18 2020
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 18, 2020 at 07:12:34PM +0300, drug via Digitalmars-d wrote:

 On 2/18/20 6:30 PM, jxel wrote:

 
 The problem is that efforts are divided as a result.




[...]

That's the fallacy described in "The Mythical Man-Month".



 IMO you are wrong if you think that dropping dmd will increase man
 power in ldc/gdc land. Who wants to contribute to ldc/gdc already
 doing it. It is open source - people contribute to projects they like,
 you can not tell them what to do. Efforts are not divided - if dmd
 would be dropped people who doesn't contribute to ldc/gdc won't start
 doing that.


Yeah, this is not a commercial endeavour where people are paid to work
on stuff they didn't choose to work on.  The people currently working on
dmd do so because they *want* to work on dmd.  If we drop dmd, they may
end up going elsewhere to find something else they're interested
instead. There's no reason to expect that they will suddenly acquire a
fresh interest in ldc/gdc -- if they already had such an interest they'd
already be working on ldc/gdc.



 On 2/18/20 6:30 PM, jxel wrote:


[...]

 The slowest part of D is the frontend and CTFE not the backend in
 LDC.




?!  That's clearly not true.  I'm compiling the same codebase with dmd,
and it's measurably faster than ldc2.  Since it's identical source code,
and the two compilers share the same frontend, the only possible
difference in compile times must be due to the different backends.

Of course, compile speed isn't the only criterion for choosing between
compilers, and that's why I still prefer using LDC in spite of DMD being
faster at compiling.  The deciding factor is the fact that the LDC
backend, in spite of being slower, emits better code.  So I'm gaining
more runtime performance by paying with slightly longer compile times.
The increase in compile times is (currently) outweighed by the superior
performance of the resulting executable.



 That's not even mentioning the constant out of memory problems
 I experience and have to try to optimize the compiler for with my
 code.




The frontend's memory issues are mainly because Walter switched to a
bump-the-pointer allocator in interest of compile speed.  Even with
-lowmem there are still fundamental problems that are unsolvable when
running a low-memory system.  I've essentially given up using dmd on
low-memory systems.  Fast compile times are worthless when the compiler
never finishes running before it runs out of memory. It's an
embarrassment.  (And this is why the whole "fast code fast" slogan makes
me cringe. It epitomizes one of the worst aspects of D in my
experience, in spite of everything else I love about D.)


T

-- 
"640K ought to be enough" -- Bill G. (allegedly), 1984. "The Internet is not a
primary goal for PC usage" -- Bill G., 1995. "Linux has no impact on
Microsoft's strategy" -- Bill G., 1999.
Feb 18 2020
parent reply jxel <jxel gmall.com> writes:
On Tuesday, 18 February 2020 at 18:14:56 UTC, H. S. Teoh wrote:

 On Tue, Feb 18, 2020 at 07:12:34PM +0300, drug via 
 Digitalmars-d wrote:

 On 2/18/20 6:30 PM, jxel wrote:

 
 The problem is that efforts are divided as a result.




 [...]

 That's the fallacy described in "The Mythical Man-Month".



Not really, how many people do you think work on and use LLVM? 
How many other compilers use it and as a result bugs and 
optimizations all happen " for free". Its not just the few people 
involved with D, and so much so the thousands involved with LLVM 
(including corporate funding).


 IMO you are wrong if you think that dropping dmd will increase 
 man power in ldc/gdc land.




Who said that? That's your misinterpretation.
Feb 18 2020
parent reply drug <drug2004 bk.ru> writes:
On 2/18/20 11:00 PM, jxel wrote:

 IMO you are wrong if you think that dropping dmd will increase man 
 power in ldc/gdc land.




 
 Who said that? That's your misinterpretation.


Experience says that again and again. If someone contributes to project 
A why he should start contributing to project B if project A would be 
completed? He can easily start contributing to project C, D etc. You 
have no warranty at all.
Feb 18 2020
next sibling parent Basile B. <b2.temp gmx.com> writes:
On Wednesday, 19 February 2020 at 07:27:39 UTC, drug wrote:

 On 2/18/20 11:00 PM, jxel wrote:

 IMO you are wrong if you think that dropping dmd will 
 increase man power in ldc/gdc land.




 
 Who said that? That's your misinterpretation.


 Experience says that again and again. If someone contributes to 
 project A why he should start contributing to project B if 
 project A would be completed? He can easily start contributing 
 to project C, D etc. You have no warranty at all.


I think there's been a misunderstanding in this branch of the 
conversation. Dropping DMD means nothing... the front end is 
shared between all compilers after all.
Feb 18 2020
prev sibling parent reply jxel <jxel gmall.com> writes:
On Wednesday, 19 February 2020 at 07:27:39 UTC, drug wrote:

 On 2/18/20 11:00 PM, jxel wrote:

 IMO you are wrong if you think that dropping dmd will 
 increase man power in ldc/gdc land.




 
 Who said that? That's your misinterpretation.


 Experience says that again and again. If someone contributes to 
 project A why he should start contributing to project B if 
 project A would be completed? He can easily start contributing 
 to project C, D etc. You have no warranty at all.


Like I said, that's your misinterpretation, read the first 
paragraph.
Feb 19 2020
parent drug <drug2004 bk.ru> writes:
On 2/19/20 7:29 PM, jxel wrote:

 On Wednesday, 19 February 2020 at 07:27:39 UTC, drug wrote:

 On 2/18/20 11:00 PM, jxel wrote:

 IMO you are wrong if you think that dropping dmd will increase man 
 power in ldc/gdc land.




 Who said that? That's your misinterpretation.


 Experience says that again and again. If someone contributes to 
 project A why he should start contributing to project B if project A 
 would be completed? He can easily start contributing to project C, D 
 etc. You have no warranty at all.


 
 Like I said, that's your misinterpretation, read the first paragraph.


Well, I responded to your "The problem is that efforts are divided as a 
result" statement. But probably I misinterpreted indeed.
Feb 19 2020
prev sibling parent reply NaN <divide by.zero> writes:
On Tuesday, 18 February 2020 at 16:12:34 UTC, drug wrote:

 On 2/18/20 6:30 PM, jxel wrote:


 IMO you are wrong if you think that dropping dmd will increase 
 man power in ldc/gdc land. Who wants to contribute to ldc/gdc 
 already doing it. It is open source - people contribute to 
 projects they like, you can not tell them what to do. Efforts 
 are not divided - if dmd would be dropped people who doesn't 
 contribute to ldc/gdc won't start doing that.


Yes dropping DMD wont suddenly move a bunch of developers over to 
LDC. But over time if LDC is the official compiler, new users, 
developers, contributors are more likely to work on LDC related 
issues if thats where they've been guided from day 1. Yes people 
work on what they need or interests them, but change the focus to 
LDC, and eventually what people need and are interested in doing 
is more likely to be there than on DMD.

I cant believe people are so short sighted on this.



 Feb 19 2020



  parent reply bachmeier <no spam.net>  writes:


On Wednesday, 19 February 2020 at 19:08:45 UTC, NaN wrote:

 On Tuesday, 18 February 2020 at 16:12:34 UTC, drug wrote:

 On 2/18/20 6:30 PM, jxel wrote:


 IMO you are wrong if you think that dropping dmd will increase 
 man power in ldc/gdc land. Who wants to contribute to ldc/gdc 
 already doing it. It is open source - people contribute to 
 projects they like, you can not tell them what to do. Efforts 
 are not divided - if dmd would be dropped people who doesn't 
 contribute to ldc/gdc won't start doing that.


 Yes dropping DMD wont suddenly move a bunch of developers over 
 to LDC. But over time if LDC is the official compiler, new 
 users, developers, contributors are more likely to work on LDC 
 related issues if thats where they've been guided from day 1. 
 Yes people work on what they need or interests them, but change 
 the focus to LDC, and eventually what people need and are 
 interested in doing is more likely to be there than on DMD.

 I cant believe people are so short sighted on this.


What does "dropping DMD" mean? Are you going to go around 
breaking kneecaps if you catch someone working on DMD? It's an 
open source project and people who want to work on it are going 
to work on it.



 Feb 19 2020



 next sibling parent reply 12345swordy <alexanderheistermann gmail.com>  writes:


On Wednesday, 19 February 2020 at 19:36:38 UTC, bachmeier wrote:

 On Wednesday, 19 February 2020 at 19:08:45 UTC, NaN wrote:

 On Tuesday, 18 February 2020 at 16:12:34 UTC, drug wrote:

 On 2/18/20 6:30 PM, jxel wrote:


 IMO you are wrong if you think that dropping dmd will 
 increase man power in ldc/gdc land. Who wants to contribute 
 to ldc/gdc already doing it. It is open source - people 
 contribute to projects they like, you can not tell them what 
 to do. Efforts are not divided - if dmd would be dropped 
 people who doesn't contribute to ldc/gdc won't start doing 
 that.


 Yes dropping DMD wont suddenly move a bunch of developers over 
 to LDC. But over time if LDC is the official compiler, new 
 users, developers, contributors are more likely to work on LDC 
 related issues if thats where they've been guided from day 1. 
 Yes people work on what they need or interests them, but 
 change the focus to LDC, and eventually what people need and 
 are interested in doing is more likely to be there than on DMD.

 I cant believe people are so short sighted on this.


 What does "dropping DMD" mean?


It means dropping the backend DMD code generator, and use llvm as 
a backend instead.

Personally I think the frontend and the backend need to be 
separate projects. It quite clear that the backend is walter 
baby, and it better for him to work and improve on it on his own 
accord without have to wait for pr approval from other people.

-Alex



 Feb 19 2020



 next sibling parent  bachmeier <no spam.net>  writes:


On Wednesday, 19 February 2020 at 20:04:22 UTC, 12345swordy wrote:


 What does "dropping DMD" mean?


 It means dropping the backend DMD code generator, and use llvm 
 as a backend instead.


That doesn't explain how you're going to force others (Walter) to 
stop working on it, which is required in order to drop it.



 Feb 19 2020



prev sibling  parent  rikki cattermole <rikki cattermole.co.nz>  writes:


On 20/02/2020 9:04 AM, 12345swordy wrote:

 Personally I think the frontend and the backend need to be separate 
 projects. It quite clear that the backend is walter baby, and it better 
 for him to work and improve on it on his own accord without have to wait 
 for pr approval from other people.


Given the end goal of dmd as a library, this makes a lot more sense 
today than it did 5 years ago.

I am all for splitting the frontend from dmd into a new repository.



 Feb 19 2020



prev sibling  parent reply NaN <divide by.zero>  writes:


On Wednesday, 19 February 2020 at 19:36:38 UTC, bachmeier wrote:

 On Wednesday, 19 February 2020 at 19:08:45 UTC, NaN wrote:

 On Tuesday, 18 February 2020 at 16:12:34 UTC, drug wrote:

 On 2/18/20 6:30 PM, jxel wrote:


 IMO you are wrong if you think that dropping dmd will 
 increase man power in ldc/gdc land. Who wants to contribute 
 to ldc/gdc already doing it. It is open source - people 
 contribute to projects they like, you can not tell them what 
 to do. Efforts are not divided - if dmd would be dropped 
 people who doesn't contribute to ldc/gdc won't start doing 
 that.


 Yes dropping DMD wont suddenly move a bunch of developers over 
 to LDC. But over time if LDC is the official compiler, new 
 users, developers, contributors are more likely to work on LDC 
 related issues if thats where they've been guided from day 1. 
 Yes people work on what they need or interests them, but 
 change the focus to LDC, and eventually what people need and 
 are interested in doing is more likely to be there than on DMD.

 I cant believe people are so short sighted on this.


 What does "dropping DMD" mean? Are you going to go around 
 breaking kneecaps if you catch someone working on DMD? It's an 
 open source project and people who want to work on it are going 
 to work on it.


I assume "dropping DMD" just means making LDC the main reference 
compiler instead of DMD.

It's not about forcing anyone to do anything, it's about 
perception, for example new users will almost always start with 
the official compiler. So you start to build more user base 
around LDC, and over time, if the d core team is focused in that 
direction, most people will follow suit.



 Feb 19 2020



  parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Thu, Feb 20, 2020 at 01:11:08AM +0000, NaN via Digitalmars-d wrote:

 On Wednesday, 19 February 2020 at 19:36:38 UTC, bachmeier wrote:


[...]

 What does "dropping DMD" mean? Are you going to go around breaking
 kneecaps if you catch someone working on DMD? It's an open source
 project and people who want to work on it are going to work on it.


 
 I assume "dropping DMD" just means making LDC the main reference
 compiler instead of DMD.
 
 It's not about forcing anyone to do anything, it's about perception,
 for example new users will almost always start with the official
 compiler. So you start to build more user base around LDC, and over
 time, if the d core team is focused in that direction, most people
 will follow suit.


[...]

Good luck getting Walter to agree to abandoning DMD and working on LDC
instead.


T

-- 
Bare foot: (n.) A device for locating thumb tacks on the floor.



 Feb 19 2020



  parent reply jxel <jxel gmall.com>  writes:


On Thursday, 20 February 2020 at 01:54:45 UTC, H. S. Teoh wrote:

 On Thu, Feb 20, 2020 at 01:11:08AM +0000, NaN via Digitalmars-d 
 wrote:

 On Wednesday, 19 February 2020 at 19:36:38 UTC, bachmeier 
 wrote:


 [...]

 What does "dropping DMD" mean? Are you going to go around 
 breaking kneecaps if you catch someone working on DMD? It's 
 an open source project and people who want to work on it are 
 going to work on it.


 
 I assume "dropping DMD" just means making LDC the main 
 reference compiler instead of DMD.
 
 It's not about forcing anyone to do anything, it's about 
 perception, for example new users will almost always start 
 with the official compiler. So you start to build more user 
 base around LDC, and over time, if the d core team is focused 
 in that direction, most people will follow suit.


 [...]

 Good luck getting Walter to agree to abandoning DMD and working 
 on LDC instead.


 T


No body expect him to. Just how the official DMD isn't built with 
LDC, why there isn't a 64 bit version of DMD on Windows, why 
Optlink is still even a thing, why restrictions are placed in DMD 
to workaround an Optlink bug instead of fixing it. No one expects 
these things to change. What seems like common sense means 
nothing to someone stubborn and blinded by pride.



 Feb 19 2020



 next sibling parent reply rikki cattermole <rikki cattermole.co.nz>  writes:


On 20/02/2020 4:35 PM, jxel wrote:

 Optlink is still even a thing


Which is going the way of the dodo soon enough.

The alternative needs to mature in production for a while before we swap.



 Feb 19 2020



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 2/19/2020 8:40 PM, rikki cattermole wrote:

 On 20/02/2020 4:35 PM, jxel wrote:

 Optlink is still even a thing


 
 Which is going the way of the dodo soon enough.
 
 The alternative needs to mature in production for a while before we swap.


32 bit Windows code is becoming irrelevant, and optlink is only for 32 bit 
Windows. If you have more than 32,000 symbols in the executable, I wonder why 
you're building a 32 bit executable.



 Feb 20 2020



prev sibling next sibling parent reply Mike Parker <aldacron gmail.com>  writes:


On Thursday, 20 February 2020 at 03:35:13 UTC, jxel wrote:


 one expects these things to change. What seems like common 
 sense means nothing to someone stubborn and blinded by pride.


There are historical reasons why Optlink is still around and they 
have nothing to do with stubbornness or pride. Please make your 
points without going ad hominem.



 Feb 19 2020



 next sibling parent reply Seb <seb wilzba.ch>  writes:


On Thursday, 20 February 2020 at 05:54:13 UTC, Mike Parker wrote:

 On Thursday, 20 February 2020 at 03:35:13 UTC, jxel wrote:


 one expects these things to change. What seems like common 
 sense means nothing to someone stubborn and blinded by pride.


 There are historical reasons why Optlink is still around and 
 they have nothing to do with stubbornness or pride. Please make 
 your points without going ad hominem.


That's not true _anymore_. Walter has rejected any kind of PR 
that would introduce sane defaults in a modern world outright 
(which is what people commonly understand as stubbornness).

"Funnily" we changed the defaults for dub (one can't build 
anything with optlink, no vibe.d, no phobos etc. anyhow) more 
than a year ago and people are fine with dub using LLD (like LDC) 
and thus being able to have more than 32k symbols in their entire 
program, getting a 64-bit binary by default and not being linked 
with an ancient, full of well-known and decade-old bugs 
custom-brew Digitalmars libc.



 Feb 19 2020



  parent reply Guillaume Piolat <first.last gmail.com>  writes:


On Thursday, 20 February 2020 at 06:08:03 UTC, Seb wrote:

 "Funnily" we changed the defaults for dub (one can't build 
 anything with optlink, no vibe.d, no phobos etc. anyhow) more 
 than a year ago and people are fine with dub using LLD (like 
 LDC) and thus being able to have more than 32k symbols in their 
 entire program, getting a 64-bit binary by default and not 
 being linked with an ancient, full of well-known and decade-old 
 bugs custom-brew Digitalmars libc.


Hello,

Reading the discussion, it seems something undervalued is the 
morale of contributors.

Getting and _keeping_ contributors is of paramount importance to 
open-source software. Some say that metric dwarf the importance 
of _having users_, and is the only real health metrics of a 
project.

Please (D leadership) consider keeping contributors happy, even 
if that means dropping a backend or OPTLINK or both. Projects 
that nly person understand can be "unbabyfied" with 
documentation/Wiki/years of contribution, but not having the 
problem in the first place is much easier.



 Feb 20 2020



 next sibling parent  NaN <divide by.zero>  writes:


On Thursday, 20 February 2020 at 13:11:21 UTC, Guillaume Piolat 
wrote:

 On Thursday, 20 February 2020 at 06:08:03 UTC, Seb wrote:


 Please (D leadership) consider keeping contributors happy, even 
 if that means dropping a backend or OPTLINK or both. Projects 
 that nly person understand can be "unbabyfied" with 
 documentation/Wiki/years of contribution, but not having the 
 problem in the first place is much easier.


If what Seb says is true, that almost all current contributors 
would like to switch and it's not even being considered, it 
doesnt bode well.



 Feb 20 2020



prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com>  writes:


On Thursday, 20 February 2020 at 13:11:21 UTC, Guillaume Piolat 
wrote:

 

 Please (D leadership) consider keeping contributors happy, even 
 if that means dropping a backend or OPTLINK or both. Projects 
 that nly person understand can be "unbabyfied" with 
 documentation/Wiki/years of contribution, but not having the 
 problem in the first place is much easier.


I think you need to be a bit more concrete in explaining how 
dropping a backend would keep contributors happy. In the time 
I've been following D, the people I've seen complain about 
multiple backends are rarely participants very long at all. To 
the point, where I would almost suggest putting something up on 
the wiki about the motivation here and then just always linking 
to that when people bring it up every few months. By contrast, 
the people I've observed who were more active participants and 
then stop contributing usually complain about other things, like 
their pull requests being ignored.



 Feb 20 2020



 next sibling parent reply Guillaume Piolat <first.last gmail.com>  writes:


On Thursday, 20 February 2020 at 14:31:44 UTC, jmh530 wrote:

 I think you need to be a bit more concrete in explaining how 
 dropping a backend would keep contributors happy.


Have you read Seb suggestions?



 Feb 20 2020



  parent  jmh530 <john.michael.hall gmail.com>  writes:


On Thursday, 20 February 2020 at 14:37:36 UTC, Guillaume Piolat 
wrote:

 On Thursday, 20 February 2020 at 14:31:44 UTC, jmh530 wrote:

 I think you need to be a bit more concrete in explaining how 
 dropping a backend would keep contributors happy.


 Have you read Seb suggestions?


Had to go back to re-read it. It's more of a list of issues with 
supporting multiple compilers than a list of suggestions, but it 
does address my point about keeping contributors happy. It sounds 
like a tricky management problem, to be honest. However, I think 
there is an important trade-off to consider between the value of 
keeping DMD and the cost of maintaining support for it. For 
instance, I don't know how much more work it is to maintain CIs 
for three compilers rather than two. It doesn't sound like much 
more work, but I really have no idea. Similarly, I would expect 
that the other backends would need to support new features, even 
if DMD is put out to pasture. That cost doesn't go away. 
Regardless, I would hope that Walter considers it is possible to 
reduce the cost for some of those issues.



 Feb 20 2020



prev sibling  parent reply Guillaume Piolat <first.last gmail.com>  writes:


On Thursday, 20 February 2020 at 14:31:44 UTC, jmh530 wrote:

 By contrast, the people I've observed who were more active 
 participants and then stop contributing usually complain about 
 other things, like their pull requests being ignored.


Now imagine if Linux spent all his time implementing Linux 
drivers instead of spending all its time reviewing. He would 
probably say "there is no time for review".



 Feb 20 2020



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 2/20/2020 6:39 AM, Guillaume Piolat wrote:

 Now imagine if Linux spent all his time implementing Linux drivers instead of 
 spending all its time reviewing. He would probably say "there is no time for 
 review".


There are 24 members of Team DMD, which have pull privileges:

   https://github.com/orgs/dlang/teams/team-dmd/members

Team Phobos has 34 members:

   https://github.com/orgs/dlang/teams/team-phobos/members

If you feel your PRs are being ignored, make some noise to these people. It's 
what they're there for.



 Feb 20 2020



  parent reply Paul Backus <snarwin gmail.com>  writes:


On Thursday, 20 February 2020 at 21:10:31 UTC, Walter Bright 
wrote:

 On 2/20/2020 6:39 AM, Guillaume Piolat wrote:

 Now imagine if Linux spent all his time implementing Linux 
 drivers instead of spending all its time reviewing. He would 
 probably say "there is no time for review".


 There are 24 members of Team DMD, which have pull privileges:

   https://github.com/orgs/dlang/teams/team-dmd/members

 Team Phobos has 34 members:

   https://github.com/orgs/dlang/teams/team-phobos/members

 If you feel your PRs are being ignored, make some noise to 
 these people. It's what they're there for.


Both of those links show a 404 page for me. I don't think this 
information is visible to users who aren't part of the Github 
dlang organization.



 Feb 20 2020



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 2/20/2020 2:42 PM, Paul Backus wrote:

 Both of those links show a 404 page for me. I don't think this information is 
 visible to users who aren't part of the Github dlang organization.


Team DMD:

	 dnadlinger
	 braddr
	 CyberShadow
	 DmitryOlshansky
	 MartinNowak
	 jacob-carlborg
	 ibuclaw
	 andralex
	 WalterBright
	 yebblies
	 schveiguy
	 rainers
	 Geod24
	 don-clugston-sociomantic
	 PetarKirov
	 wilzbach
	 atilaneves
	 AndrewEdwards
	 auto-tester	
	 thewilsonator
	 SSoulaimane
	 AndrejMitrovic
	 dlang-bot
	 RazvanN7

Team Phobos:

         burner burner
	 repeatedly	repeatedly repeatedly
	 dnadlinger	dnadlinger dnadlinger
	 braddr	braddr braddr
	 alexrp	alexrp alexrp
	 JakobOvrum	JakobOvrum JakobOvrum
	 CyberShadow	CyberShadow CyberShadow
	 kyllingstad	kyllingstad kyllingstad
	 DmitryOlshansky	DmitryOlshansky DmitryOlshansky
	 MartinNowak	MartinNowak MartinNowak Maintainer
	 ibuclaw	ibuclaw ibuclaw
	 quickfur	quickfur quickfur
	 andralex	andralex andralex
	 WalterBright	WalterBright WalterBright
	 jmdavis	jmdavis jmdavis
	 yebblies	yebblies yebblies
	 schveiguy	schveiguy schveiguy
	 JackStouffer	JackStouffer JackStouffer
	 rainers	rainers rainers
	 9il	9il 9il
	 Hackerpilot	Hackerpilot Hackerpilot
	 UplinkCoder	UplinkCoder UplinkCoder
	 PetarKirov	PetarKirov PetarKirov
	 edi33416	edi33416 edi33416
	 wilzbach	wilzbach wilzbach
	 MetaLang	MetaLang MetaLang
	 atilaneves	atilaneves atilaneves
	 AndrewEdwards	AndrewEdwards AndrewEdwards
	 auto-tester	auto-tester auto-tester
	 jercaianu	jercaianu jercaianu



 Feb 20 2020



prev sibling  parent  Mathias Lang <pro.mathias.lang gmail.com>  writes:


On Thursday, 20 February 2020 at 13:11:21 UTC, Guillaume Piolat 
wrote:

 On Thursday, 20 February 2020 at 06:08:03 UTC, Seb wrote:

 "Funnily" we changed the defaults for dub (one can't build 
 anything with optlink, no vibe.d, no phobos etc. anyhow) more 
 than a year ago and people are fine with dub using LLD (like 
 LDC) and thus being able to have more than 32k symbols in 
 their entire program, getting a 64-bit binary by default and 
 not being linked with an ancient, full of well-known and 
 decade-old bugs custom-brew Digitalmars libc.


 Hello,

 Reading the discussion, it seems something undervalued is the 
 morale of contributors.

 Getting and _keeping_ contributors is of paramount importance 
 to open-source software. Some say that metric dwarf the 
 importance of _having users_, and is the only real health 
 metrics of a project.

 Please (D leadership) consider keeping contributors happy, even 
 if that means dropping a backend or OPTLINK or both. Projects 
 that nly person understand can be "unbabyfied" with 
 documentation/Wiki/years of contribution, but not having the 
 problem in the first place is much easier.


To be the devil's advocate, no one has contributed to D longer 
than Walter, and in the 20+ year he's been working on D, I'm sure 
he's seen countless contributors coming and going, many of them 
proposed ideas they considered improvements.

Everyone has a their own, often conflicting, vision of what is 
"the right way". We had many people saying "deprecate scope 
class" (myself included), we let it happen, and then one of the 
biggest commercial user steps in and said "Hey guys we're using 
this feature, like, a lot...". It surely makes you reflect on the 
silent majority. This is one example out of many where we went 
one way, and things didn't quite work out as expected.
As you cannot satisfy everyone, keeping people happy is not a 
reliable metric.

This is not to say that I think everything is going as it should. 
I do think that there is a lot to gain with dropping homegrown 
alternative to mainstream utilities (DM Make, optlink, DMD's 
backend...).
But we made immense progress over the years, as some very large 
projects have been tackled (conversion of DMD to D), and others 
have been approved but not yet championed (moving from Bugzilla 
to DMD).

Perhaps improving communication among core contributors could 
help with reaching agreement on such large project. From what 
I've seen, this is not a problem that is unique to D.



 Feb 20 2020



prev sibling  parent  jxel <jxel gmall.com>  writes:


On Thursday, 20 February 2020 at 05:54:13 UTC, Mike Parker wrote:

 On Thursday, 20 February 2020 at 03:35:13 UTC, jxel wrote:


 one expects these things to change. What seems like common 
 sense means nothing to someone stubborn and blinded by pride.


 There are historical reasons why Optlink is still around and 
 they have nothing to do with stubbornness or pride. Please make 
 your points without going ad hominem.


Being around for backwards compatibility and STILL BEING USED IN 
RELEASES are two very different things. Historical is the only 
thing that should describe Optlink. With all the breaking changes 
that are already happening, one can't really argue keeping it to 
maintain backwards compatibility either.

Anyways it seems we are one step closer to that, if the PR 
doesn't get reversed before release.



 Feb 20 2020



prev sibling next sibling parent reply Seb <seb wilzba.ch>  writes:


On Thursday, 20 February 2020 at 03:35:13 UTC, jxel wrote:

 On Thursday, 20 February 2020 at 01:54:45 UTC, H. S. Teoh wrote:

 On Thu, Feb 20, 2020 at 01:11:08AM +0000, NaN via 
 Digitalmars-d wrote:

 On Wednesday, 19 February 2020 at 19:36:38 UTC, bachmeier 
 wrote:


 [...]

 What does "dropping DMD" mean? Are you going to go around 
 breaking kneecaps if you catch someone working on DMD? It's 
 an open source project and people who want to work on it 
 are going to work on it.


 
 I assume "dropping DMD" just means making LDC the main 
 reference compiler instead of DMD.
 
 It's not about forcing anyone to do anything, it's about 
 perception, for example new users will almost always start 
 with the official compiler. So you start to build more user 
 base around LDC, and over time, if the d core team is focused 
 in that direction, most people will follow suit.


 [...]

 Good luck getting Walter to agree to abandoning DMD and 
 working on LDC instead.


 T


 No body expect him to. Just how the official DMD isn't built 
 with LDC, why there isn't a 64 bit version of DMD on Windows, 
 why Optlink is still even a thing, why restrictions are placed 
 in DMD to workaround an Optlink bug instead of fixing it. No 
 one expects these things to change. What seems like common 
 sense means nothing to someone stubborn and blinded by pride.


People often don't see how much effort is wasted on maintaining 
multiple compilers, but how much time i.e.
- the release process (especially with stable releases)
- keeping CIs up, configured and running
- actively triaging bugs
- having multiple test suite solutions
- maintaining automations (e.g. dlang-bot)

consumes is hidden to non-insiders. Furthermore,

- supporting every new feature in each backend (ask Ian or kinke 
about the recent multiple inner pointer change)
- maintaining multiple installers
- nasty DMD backend bugs (a few people apart from Walter actually 
do try to fix things in the backend)
- working around obscure infra problems (IIRC Digitalmars libc 
still isn't thread-safe and people have run in all kinds of 
problems with it and needed to come up with ingenious solutions, 
even DMD's build.d shows signs of this)
- working with Digitalmars Make
- maintaining a modern set of features per compiler (address 
sanitization, ...)
- keeping other compilers up-to-date with upstream (every release 
introduces regressions)
- maintaining support for every OS for every compiler 
(e.g.FreeBSD)
- having an n-times bigger surface for bugs
...

And I haven't even mentioned the bottleneck of any major DMD PR 
needing to be approved by Walter.

In my opinion DMD is only the default because switching 
development efforts to LDC would be a rather big initial cost 
(time investment) to setup/change the infrastructure and no one 
wants to head this effort.


 if the d core team is focused in that direction, most people 
 will follow suit.


I would switch in a heartbeat and I am fairly confident to 
estimate that the same applies to almost all remaining active 
contributors who haven't left yet because of frustration.



 Feb 19 2020



 next sibling parent  drug <drug2004 bk.ru>  writes:


On 2/20/20 9:34 AM, Seb wrote:

 snipped
 


The reason you listed above get a hot response in my soul. That 
maintenance things are really annoying. As for me they really are the 
reasons to change the thing order. Probably we can start from making a 
new separate repository for the frontend? It seems to me that then 
things will fall into place themselves.
But I'm afraid of gdc in case we switch on ldc. Gdc is important thing.



 Feb 19 2020



prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 2/19/2020 10:34 PM, Seb wrote:

 And I haven't even mentioned the bottleneck of any major DMD PR needing to be 
 approved by Walter.


Given the current regression list:

https://issues.dlang.org/buglist.cgi?bug_severity=regression&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&list_id=230215&query_format=advanced

I'm probably being too permissive. I'm usually the one who gets to fix them. I 
asked in this forum for some help isolating which PRs caused the regressions,
to 
no avail.

Major PRs *should* be regarded with skepticism. For example, I recently fixed a 
regression caused by a 2 or 3 hundred line PR, where the substance of the PR
was 
one line and the rest of it was refactoring.

Folks, with PRs, smaller, tightly focused PRs are better. Do not:

1. lump multiple issues into one PR

2. include your favorite refactorings in with it

3. refactors must not change observable behavior

Do:

1. minimize the PR even if the result of the PR suggests a refactoring

2. do that refactor SEPARATELY AFTER THE PR WAS MERGED



 Feb 20 2020



  parent  jxel <jxel gmall.com>  writes:


On Thursday, 20 February 2020 at 10:58:36 UTC, Walter Bright 
wrote:

 On 2/19/2020 10:34 PM, Seb wrote:

 And I haven't even mentioned the bottleneck of any major DMD 
 PR needing to be approved by Walter.


 Given the current regression list:

 https://issues.dlang.org/buglist.cgi?bug_severity=regression&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&list_id=230215&query_format=advanced

 I'm probably being too permissive. I'm usually the one who gets 
 to fix them. I asked in this forum for some help isolating 
 which PRs caused the regressions, to no avail.

 Major PRs *should* be regarded with skepticism. For example, I 
 recently fixed a regression caused by a 2 or 3 hundred line PR, 
 where the substance of the PR was one line and the rest of it 
 was refactoring.

 Folks, with PRs, smaller, tightly focused PRs are better. Do 
 not:

 1. lump multiple issues into one PR

 2. include your favorite refactorings in with it

 3. refactors must not change observable behavior

 Do:

 1. minimize the PR even if the result of the PR suggests a 
 refactoring

 2. do that refactor SEPARATELY AFTER THE PR WAS MERGED


I've fixed a regression caused by one of your PRs. It was a small 
change to the actual code, couldn't have been more than 10 lines, 
but it changed a few of the test case outputs. Funnily enough 
there were valid test cases with error messages that were just 
nuked over blindly (automatically probably). Errors happen, even 
for very simple things, if you don't take care it doesn't matter 
what the size of the PR is.

People have trouble getting one PR merged, and you want them to 
try and split them up and somehow have that be coordinated 
properly when the PR have to be merged in a certain order and 
across 3 different projects? Yah, no thanks.



 Feb 20 2020



prev sibling  parent reply Dibyendu Majumdar <d.majumdar gmail.com>  writes:


On Thursday, 20 February 2020 at 06:34:45 UTC, Seb wrote:


 In my opinion DMD is only the default because switching 
 development efforts to LDC would be a rather big initial cost 
 (time investment) to setup/change the infrastructure and no one 
 wants to head this effort.


 if the d core team is focused in that direction, most people 
 will follow suit.


 I would switch in a heartbeat and I am fairly confident to 
 estimate that the same applies to almost all remaining active 
 contributors who haven't left yet because of frustration.



I used to think that the backend should be abandoned in favour of 
LLVM but no more. I think it is worth having a fast backend. I 
was surprised how quickly dmd can be compiled compared to LLVM.

It is excellent for a language designer to have full control over 
the implementation too as it helps play with ideas without having 
to fight with external libraries.

Regards



 Feb 20 2020



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 2/20/2020 12:35 PM, Dibyendu Majumdar wrote:

 I used to think that the backend should be abandoned in favour of LLVM but no 
 more. I think it is worth having a fast backend. I was surprised how quickly
dmd 
 can be compiled compared to LLVM.
 
 It is excellent for a language designer to have full control over the 
 implementation too as it helps play with ideas without having to fight with 
 external libraries.


I might add that having a fast edit-compile-debug loop is indeed
transformative. 
Sometimes tracking down a problem may require hundreds of times through that
loop.

On my dev machine, a full debug build of dmd takes 43 seconds.

An incremental build (one file change) takes 9 seconds.

(The build compiler is a Windows 32 bit dmd built with dmd.)



 Feb 20 2020



  parent reply Stefan Koch <uplink.coder googlemail.com>  writes:


On Thursday, 20 February 2020 at 21:04:16 UTC, Walter Bright 
wrote:

 On 2/20/2020 12:35 PM, Dibyendu Majumdar wrote:

 I used to think that the backend should be abandoned in favour 
 of LLVM but no more. I think it is worth having a fast 
 backend. I was surprised how quickly dmd can be compiled 
 compared to LLVM.
 
 It is excellent for a language designer to have full control 
 over the implementation too as it helps play with ideas 
 without having to fight with external libraries.


 I might add that having a fast edit-compile-debug loop is 
 indeed transformative. Sometimes tracking down a problem may 
 require hundreds of times through that loop.

 On my dev machine, a full debug build of dmd takes 43 seconds.

 An incremental build (one file change) takes 9 seconds.

 (The build compiler is a Windows 32 bit dmd built with dmd.)


43 seconds?
Are you still on an hdd?



 Feb 20 2020



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 2/20/2020 1:07 PM, Stefan Koch wrote:

 43 seconds?
 Are you still on an hdd?


No. But my computer is an older one.



 Feb 20 2020



prev sibling  parent  Petar Kirov [ZombineDev] <petar.p.kirov gmail.com>  writes:


On Thursday, 20 February 2020 at 03:35:13 UTC, jxel wrote:

 [..]
 Just how the official DMD isn't built with LDC, why there
 isn't a 64 bit version of DMD on Windows [..]


Technically not true anymore!

https://github.com/dlang/installer/pull/425
https://github.com/dlang/installer/pull/417

See also:
https://github.com/dlang/installer/pulls?q=is%3Apr+is%3Aclosed

;)



 Feb 20 2020



prev sibling  parent reply matheus <matheus gmail.com>  writes:


On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:

 Unfortunately, the GDC and LDC optimizers consistently produce 
 code that outperforms code generated by the DMD backend by 
 20-30%, sometimes as high as 40% for CPU-intensive code...


Well these numbers are huge, I never thought the difference would 
be higher than 10% between these compilers, thanks for pointing 
this out.

I use old DMD for my hobby projects in D, but now I'll test both 
GDC/LDC.

Matheus.



 Feb 18 2020



  parent reply kinke <noone nowhere.com>  writes:


On Tuesday, 18 February 2020 at 18:50:44 UTC, matheus wrote:

 On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:

 Unfortunately, the GDC and LDC optimizers consistently produce 
 code that outperforms code generated by the DMD backend by 
 20-30%, sometimes as high as 40% for CPU-intensive code...


 Well these numbers are huge, I never thought the difference 
 would be higher than 10% between these compilers, thanks for 
 pointing this out.


Once you know that the DMD backend is a formidable one-man 
project, it should be clear as day that that cannot compete with 
huge eco-systems like LLVM and GCC with millions of man-hours, in 
terms of architecture support, optimizations and flexibility. 
It's nice for fast unoptimized codegen if you are only targeting 
x86, and probably features better debuginfos than LDC at the 
moment (well, ecept for Mac apparently), but to me, the DMD 
*backend* is clearly a dead end in the long run.

DMD itself runs faster by 58% when compiled with LDC for a random 
compilation test case, see 
https://github.com/dlang/installer/pull/425#issuecomment-580868218. For number
crunching, you can definitely expect much higher speed-ups if the
auto-vectorizer kicks in. Whole-program optimization via LTO, even across C(++)
and D, and PGO can improve your runtime further.

On Tuesday, 18 February 2020 at 15:30:57 UTC, jxel wrote:

 The problem is that efforts are divided as a result.


As others have stated, that's not true. The vast majority of 
contributions to the dmd repo affect the front-end and thus 
implicitly benefit all compilers.



 Feb 18 2020



 next sibling parent reply matheus <matheus gmail.com>  writes:


On Tuesday, 18 February 2020 at 20:10:57 UTC, kinke wrote:

 On Tuesday, 18 February 2020 at 18:50:44 UTC, matheus wrote:

 On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:

 Unfortunately, the GDC and LDC optimizers consistently 
 produce code that outperforms code generated by the DMD 
 backend by 20-30%, sometimes as high as 40% for CPU-intensive 
 code...


 Well these numbers are huge, I never thought the difference 
 would be higher than 10% between these compilers, thanks for 
 pointing this out.


 Once you know that the DMD backend is a formidable one-man 
 project, it should be clear as day that that cannot compete 
 with huge eco-systems like LLVM and GCC with millions of 
 man-hours, in terms of architecture support, optimizations and 
 flexibility. It's nice for fast unoptimized codegen if you are 
 only targeting x86, and probably features better debuginfos 
 than LDC at the moment (well, ecept for Mac apparently), but to 
 me, the DMD *backend* is clearly a dead end in the long run.


Just to be clear, I am NOT diminishing any work, I just wasn't 
aware of these differences, because 20 to 30 or maybe 40% on 
runtime is something that you can't just ignore.

I think that maybe is possible to balance things, like developing 
with DMD and generate final code with GDC/LDC to get faster 
runtime, well maybe some problems will happen mixing compilers 
and final result, but this is something that I will considerate 
for now on.

Matheus.



 Feb 18 2020



  parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Tue, Feb 18, 2020 at 09:22:18PM +0000, matheus via Digitalmars-d wrote:
[...]

 Just to be clear, I am NOT diminishing any work, I just wasn't aware
 of these differences, because 20 to 30 or maybe 40% on runtime is
 something that you can't just ignore.


It depends on your use case.  YMMV, as they say.

IME, the difference is most pronounced for CPU-intensive code where you
have a lot of nested constructs, esp. in range-based code with extensive
use of UFCS chains.

Some years ago I analysed the assembly output to understand why this is
so.  According to my observations, one big factor is that the DMD
inliner is rather anemic: it gives up at the slightest complication,
thus missing out on what's commonly a domino-effect of optimizations:
inlining the innermost function call opens up new optimization
opportunities which causes the next outer level to be inlineable, etc.,
until most of the call stack has been inlined. But stop somewhere in
between and you miss out the rest of the entire call chain's worth of
optimizations.  In range-based code, most of the method calls are very
small, so quite often a long chain of nested calls can be reduced to
just a few inlined instructions, so this is where LDC's very aggressive
inliner really shines.  In such cases, you can easily see 20-30%
almost-guaranteed performance difference, and sometimes I've seen even
up to 40%.  I just ran a quick test again, as I posted in another reply,
and again I see a >30% performance boost just from using ldc2 instead of
dmd.

The LLVM optimizer has been known to optimize entire call chains into
the equivalent of a `return constValue;` by executing LLVM bytecode at
compile-time to optimize away an entire subtree of function calls into a
constant value. DMD's inliner, in comparison, balks at the sight of
things as trivial as writing:

	auto myFunc(...) {
		if (cond) return result1;
		return result2;
	}

vs.

	auto myFunc(...) {
		if (cond) return result1;
		else return result2;
	}

so it tends to give up long before the code has been transformed into a
state where it can be inlined one level further up the call chain.
Thus, an entire domino-chain of optimizations is missed, and you end up
with several nested function calls where it could have been reduced to
just a few inlined instructions. When this happens in the inner loop, a
30% performance drop is pretty much expected.



 I think that maybe is possible to balance things, like developing with
 DMD and generate final code with GDC/LDC to get faster runtime, well
 maybe some problems will happen mixing compilers and final result, but
 this is something that I will considerate for now on.


[...]

As somebody pointed out, using dmd for development and ldc2 for release
build is definitely a viable approach.  Recent LDC releases have been
tracking DMD releases very closely, so excepting rare corner cases and
bugs, functionality-wise the two compilers should pretty much be on par.

And now that I've learned that ldmd2 doesn't preclude passing
LDC-specific flags on the command-line, I might actually just adopt this
approach.  (The trouble with different command-line syntaxes is that
compilers are not drop-in replacements for each other, which makes
supporting multiple compilers in the same build script a pain. Certainly
possible, and not even hard, but nonetheless a pain to write and
maintain. With ldmd2 I can just standardize on DMD command-line syntax
and have everything Just Work(tm). Best of both worlds.)


T

-- 
People demand freedom of speech to make up for the freedom of thought which
they avoid. -- Soren Aabye Kierkegaard (1813-1855)



 Feb 18 2020



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 2/18/2020 1:52 PM, H. S. Teoh wrote:

 DMD's inliner, in comparison, balks at the sight of
 things as trivial as writing:
 
 	auto myFunc(...) {
 		if (cond) return result1;
 		return result2;
 	}
 
 vs.
 
 	auto myFunc(...) {
 		if (cond) return result1;
 		else return result2;
 	}


With:

   int fooa(int a, int b, int c) {
     if (a)
	return b;
     return c;
   }

   int foob(int a, int b, int c) {
     if (a)
	return b;
     else
	return c;
   }

   int test(int a, int b, int c) {
     return fooa(a, b, c) + foob(a, b, c);
   }

the generated code for test() compiled with -inline -O is:

                 push    EAX
                 cmp     dword ptr 0Ch[ESP],0
                 je      LE
                 mov     EAX,8[ESP]
                 jmp short       L11
   LE:           mov     EAX,[ESP]
   L11:          add     EAX,EAX
                 pop     ECX
                 ret     8

so it is inlining both forms. If you do have trivial functions that aren't
being 
inlined, please let me know. Thanks!



 Feb 18 2020



  parent reply Max Samukha <maxsamukha gmail.com>  writes:


On Wednesday, 19 February 2020 at 05:42:19 UTC, Walter Bright 
wrote:


 so it is inlining both forms. If you do have trivial functions 
 that aren't being inlined, please let me know. Thanks!


Anything with a 'switch':

void foo() {
     switch(true) {
         default:
     }
}



 Feb 18 2020



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 2/18/2020 10:10 PM, Max Samukha wrote:

 On Wednesday, 19 February 2020 at 05:42:19 UTC, Walter Bright wrote:
 

 so it is inlining both forms. If you do have trivial functions that aren't 
 being inlined, please let me know. Thanks!


 
 Anything with a 'switch':
 
 void foo() {
      switch(true) {
          default:
      }
 }


That's true, it's also true for loops. Though trivial functions with switches 
seem to me to be rare.

Keep in mind that the inliner is part of the dmd front end, not the back end. 
The inliner would be more effective if it was part of the back end, but that 
comes with a major memory/time penalty (intermediate code would have to be 
generated for all functions, even ones not being compiled, and kept around in 
memory).



 Feb 19 2020



 next sibling parent reply Max Samukha <maxsamukha gmail.com>  writes:


On Thursday, 20 February 2020 at 00:52:02 UTC, Walter Bright 
wrote:


 
 Anything with a 'switch':
 
 void foo() {
      switch(true) {
          default:
      }
 }


 That's true, it's also true for loops. Though trivial functions 
 with switches seem to me to be rare.


We encountered a case recently where we wanted to inline one. It 
was trivial to work around, though.


 Keep in mind that the inliner is part of the dmd front end, not 
 the back end. The inliner would be more effective if it was 
 part of the back end, but that comes with a major memory/time 
 penalty (intermediate code would have to be generated for all 
 functions, even ones not being compiled, and kept around in 
 memory).


Yes, I know about that. People are often willing to pay the cost 
for highly optimized builds.



 Feb 20 2020



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 2/20/2020 2:08 AM, Max Samukha wrote:

 We encountered a case recently where we wanted to inline one. It was trivial
to 
 work around, though.


I'm interested in evaluating common cases that don't inline but should.



 Feb 20 2020



  parent reply Max Samukha <maxsamukha gmail.com>  writes:


On Thursday, 20 February 2020 at 10:36:35 UTC, Walter Bright 
wrote:

 On 2/20/2020 2:08 AM, Max Samukha wrote:

 We encountered a case recently where we wanted to inline one. 
 It was trivial to work around, though.


 I'm interested in evaluating common cases that don't inline but 
 should.


Trivial functions that use a 'switch' to check membership in a 
small set of constants are still quite common I think, such as:

bool isReserved(char c) {
         switch (c) {
             case ':', '/', '?'...:
                 return true;
             default:
                 return false;
         }
}



 Feb 20 2020



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


Thank you.



 Feb 20 2020



prev sibling  parent reply NaN <divide by.zero>  writes:


On Thursday, 20 February 2020 at 00:52:02 UTC, Walter Bright 
wrote:

 On 2/18/2020 10:10 PM, Max Samukha wrote:

 On Wednesday, 19 February 2020 at 05:42:19 UTC, Walter Bright 
 wrote:
 

 so it is inlining both forms. If you do have trivial 
 functions that aren't being inlined, please let me know. 
 Thanks!


 
 Anything with a 'switch':
 
 void foo() {
      switch(true) {
          default:
      }
 }


 That's true, it's also true for loops. Though trivial functions 
 with switches seem to me to be rare.

 Keep in mind that the inliner is part of the dmd front end, not 
 the back end. The inliner would be more effective if it was 
 part of the back end, but that comes with a major memory/time 
 penalty (intermediate code would have to be generated for all 
 functions, even ones not being compiled, and kept around in 
 memory).


Would moving the inliner to the back end still slow down the 
compile times even when inlining is turned off?



 Feb 20 2020



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 2/20/2020 3:17 AM, NaN wrote:

 Would moving the inliner to the back end still slow down the compile times
even 
 when inlining is turned off?


If it was done right, probably not.



 Feb 20 2020



prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Tue, Feb 18, 2020 at 08:10:57PM +0000, kinke via Digitalmars-d wrote:
[...]

 Once you know that the DMD backend is a formidable one-man project, it
 should be clear as day that that cannot compete with huge eco-systems
 like LLVM and GCC with millions of man-hours, in terms of architecture
 support, optimizations and flexibility. It's nice for fast unoptimized
 codegen if you are only targeting x86, and probably features better
 debuginfos than LDC at the moment (well, ecept for Mac apparently),
 but to me, the DMD *backend* is clearly a dead end in the long run.


Yeah, not to mention Walter himself has been mostly working on the front
end these days, as he himself said he hasn't been keeping the backend
up-to-date.  A single person only has so many hours a day, no matter
what a brilliant genius Walter is, there's still a limit as to what can
be accomplished by a single person.  And if DMD's very limited target
arch support is already languishing from lack of sufficient time to keep
up-to-date, support for other targets like Android is pretty much never
going to happen, whereas using LDC gives you that support *today*.

Add to that LDC's recent ability to cross-compile from Linux to Windows
without needing to rebuild the entire toolchain, vs. dmd's inability to
cross-compile at all (AFAIK), and the best choice from the user's POV is
beyond obvious.



 DMD itself runs faster by 58% when compiled with LDC for a random
 compilation test case, see
 https://github.com/dlang/installer/pull/425#issuecomment-580868218.


IMNSHO, we should use LDC to compile official DMD releases, as someone
has already mentioned. ;-)



 For number crunching, you can definitely expect much higher speed-ups
 if the auto-vectorizer kicks in. Whole-program optimization via LTO,
 even across C(++) and D, and PGO can improve your runtime further.


[...]

Yeah, LDC's optimizer is light-years ahead of DMD's.  I just tested it
again on one of my latest projects, with a custom main() that runs a
CPU-intensive part of the code in a loop 100 times.  With `dmd -O
-inline`, a typical run is about 25-26 seconds.  With `ldmd2 -O3`, a
typical run is about 16-17 seconds.  We're looking at a ~30% increase in
performance here, on exactly the same code and maximum optimization
flags on both compilers.

The numbers speak for themselves, really.

As for compile times, `dmd -O -inline` typically takes about 6 seconds,
whereas `ldmd2 -O3` typically takes about 8 seconds, which is about a
25% slowdown in compile times.  Is a 2-second compile-time slowdown
worth a 30% performance increase in the resulting executable?  To me,
it's an unquestionable yes.


T

-- 
Democracy: The triumph of popularity over principle. -- C.Bond



 Feb 18 2020



  parent reply kinke <noone nowhere.com>  writes:


On Tuesday, 18 February 2020 at 21:26:14 UTC, H. S. Teoh wrote:

 With `dmd -O -inline`, a typical run is about 25-26 seconds.  
 With `ldmd2 -O3`, a typical run is about 16-17 seconds.  We're 
 looking at a ~30% increase in performance here


Take care with such numbers - this sounds more like a 50% 
increase in performance. A speed-up by 4 (300% performance 
*increase*) results in 1/4th of the runtime, i.e., a runtime 
*decrease* by 75%.



 Feb 18 2020



  parent  "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Tue, Feb 18, 2020 at 10:14:54PM +0000, kinke via Digitalmars-d wrote:

 On Tuesday, 18 February 2020 at 21:26:14 UTC, H. S. Teoh wrote:

 With `dmd -O -inline`, a typical run is about 25-26 seconds.  With
 `ldmd2 -O3`, a typical run is about 16-17 seconds.  We're looking at
 a ~30% increase in performance here


 
 Take care with such numbers - this sounds more like a 50% increase in
 performance. A speed-up by 4 (300% performance *increase*) results in
 1/4th of the runtime, i.e., a runtime *decrease* by 75%.


Even better then. :-D


T

-- 
"A one-question geek test. If you get the joke, you're a geek: Seen on a
California license plate on a VW Beetle: 'FEATURE'..." -- Joshua D. Wachs -
Natural Intelligence, Inc.



 Feb 18 2020



prev sibling  parent reply jxel <jxel gmall.com>  writes:


On Tuesday, 18 February 2020 at 20:10:57 UTC, kinke wrote:

 but to me, the DMD *backend* is clearly a dead end in the long 
 run.


Its already at that point for me. PR sit stagnant for so long 
waiting on Walter by the time he looks at it it no longer 
compiles because of changes to the compiler, then proceeds to 
complain that it doesn't compile. It did 6 months ago.



 Feb 18 2020



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 2/18/2020 8:17 PM, jxel wrote:

 Its already at that point for me.


We all have to rebase our PRs from time to time. It comes with the territory, 
it's nothing terrible. I bet you could rebase whatever you've done in much less 
time than you spend complaining. If you don't know how to rebase with git,
there 
are many people here who can lead you through it.

Keep in mind that one of the reasons I'm not eager for major refactoring PRs is 
they cause everything in the queue to be rebased. Mine included.



 Feb 20 2020



  parent reply jxel <jxel gmall.com>  writes:


On Friday, 21 February 2020 at 06:05:15 UTC, Walter Bright wrote:

 On 2/18/2020 8:17 PM, jxel wrote:

 Its already at that point for me.


 We all have to rebase our PRs from time to time. It comes with 
 the territory, it's nothing terrible. I bet you could rebase 
 whatever you've done in much less time than you spend 
 complaining. If you don't know how to rebase with git, there 
 are many people here who can lead you through it.

 Keep in mind that one of the reasons I'm not eager for major 
 refactoring PRs is they cause everything in the queue to be 
 rebased. Mine included.


So how do you get that I have a problem with rebasing from the 
DMD backend being buggy, a damn mess of code with no 
documentation, and there's really only one person maintaining it 
but they are too busy doing other things to bother doing work on 
the backend and related PRs, and that it should be replaced with 
LLVM because of a long list of benefits that outweighs the cons? 
My mind is shattered with the ki d of mental gymnastics that you 
have had to pull to get that point, did you even read the 
comment? Evidentially not.

The troft is over there, not here -- cause that's about as 
relevant.



 Feb 21 2020



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 2/21/2020 11:16 AM, jxel wrote:

 So how do you get that I have a problem with rebasing from the DMD backend
being 
 buggy, a damn mess of code with no documentation, and there's really only one 
 person maintaining it but they are too busy doing other things to bother doing 
 work on the backend and related PRs, and that it should be replaced with LLVM 
 because of a long list of benefits that outweighs the cons?


You can always start contributing in a positive manner by addressing the issues 
that matter to you. This is an open source project. If you want currency, 
contribute. Not contributing coupled with relentless complaining moves you to 
the bottom of the list of people we listen to.



 My mind is shattered with the ki d of mental gymnastics that you have had to
pull


More friendly advice: berating people isn't going to work here. It's not like 
you're paying anyone.



 Feb 21 2020