www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Release D 2.077.0

reply Martin Nowak <code dawg.eu> writes:
Glad to announce D 2.077.0.

This release comes with a new, more compact mangling, templated 
vector operations, reproducible dmd builds, and various fixes.
Thanks to everyone involved in this 👏.

http://downloads.dlang.org/releases/2.x/2.077.0/
http://dlang.org/changelog/2.077.0.html
The dlang.org website will get updated soon.

-Martin
Nov 02
next sibling parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 11/02/2017 06:35 PM, Martin Nowak wrote:
 reproducible dmd builds,
Curiosity: What exactly was preventing this before? Order of source files altering the order of output? Something else? Also, just musing...Regarding the matter of __TIME__(etc) breaking this guarantee (breaking it for obvious reasons). Seems to me it would be helpful to have compiler switches to force lexer symbols to specific values in order to bring at least some level of reproducablity even to projects that do use such symbols.
Nov 02
prev sibling next sibling parent reply Daniel Kozak <kozzi11 gmail.com> writes:
On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote:
 Glad to announce D 2.077.0.

 This release comes with a new, more compact mangling, templated 
 vector operations, reproducible dmd builds, and various fixes.
 Thanks to everyone involved in this 👏.

 http://downloads.dlang.org/releases/2.x/2.077.0/
 http://dlang.org/changelog/2.077.0.html
 The dlang.org website will get updated soon.

 -Martin
How should I compile my program to enable array vectorization? I have tried dmd -march=native, -march=avx2 as changlog suggest but does not work I have tried even just -march=native or -march=avx2 but still does not compile Error: unrecognized switch '-march=avx2' Error: unrecognized switch '-march=native'
Nov 03
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/3/2017 2:28 AM, Daniel Kozak wrote:
 How should I compile my program to enable array vectorization?
dmd doesn't do what is known as "auto-vectorization". https://en.wikipedia.org/wiki/Automatic_vectorization What D does is have vector data types, and when those are used vector instructions are generated for them. https://dlang.org/spec/simd.html
Nov 03
parent reply Mike Parker <aldacron gmail.com> writes:
On Friday, 3 November 2017 at 09:33:31 UTC, Walter Bright wrote:
 On 11/3/2017 2:28 AM, Daniel Kozak wrote:
 How should I compile my program to enable array vectorization?
dmd doesn't do what is known as "auto-vectorization". https://en.wikipedia.org/wiki/Automatic_vectorization What D does is have vector data types, and when those are used vector instructions are generated for them. https://dlang.org/spec/simd.html
For clarity, where the changeling says that GDC & LDC use auto-vectorization, that's actually happening with the array operations and core.simd is not required, correct?
Nov 03
next sibling parent Joakim <dlang joakim.fea.st> writes:
On Friday, 3 November 2017 at 10:02:18 UTC, Mike Parker wrote:
 On Friday, 3 November 2017 at 09:33:31 UTC, Walter Bright wrote:
 On 11/3/2017 2:28 AM, Daniel Kozak wrote:
 How should I compile my program to enable array vectorization?
dmd doesn't do what is known as "auto-vectorization". https://en.wikipedia.org/wiki/Automatic_vectorization What D does is have vector data types, and when those are used vector instructions are generated for them. https://dlang.org/spec/simd.html
For clarity, where the changeling says that GDC & LDC use auto-vectorization, that's actually happening with the array operations and core.simd is not required, correct?
Yes, at least with ldc.
Nov 03
prev sibling next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Friday, 3 November 2017 at 10:02:18 UTC, Mike Parker wrote:
 On Friday, 3 November 2017 at 09:33:31 UTC, Walter Bright wrote:
 On 11/3/2017 2:28 AM, Daniel Kozak wrote:
 How should I compile my program to enable array vectorization?
dmd doesn't do what is known as "auto-vectorization". https://en.wikipedia.org/wiki/Automatic_vectorization What D does is have vector data types, and when those are used vector instructions are generated for them. https://dlang.org/spec/simd.html
For clarity, where the changeling says that GDC & LDC use auto-vectorization, that's actually happening with the array operations and core.simd is not required, correct?
OK, I'm a bit confused here. This gives the impression that the vectorization happens automatically with array operations: "Array operations have been converted from dedicated assembly routines for some array operations to a generic template implementation for all array operations. This provides huge performance increases (2-4x higher throughput) for array operations that were not previously vectorized. Furthermore the implementation makes better use of vectorization even for short arrays to heavily reduce latency for some operations (up to 4x)." Where does core.simd fit in?
Nov 03
parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 3 November 2017 at 10:07:25 UTC, Mike Parker wrote:
 On Friday, 3 November 2017 at 10:02:18 UTC, Mike Parker wrote:
 [...]
OK, I'm a bit confused here. This gives the impression that the vectorization happens automatically with array operations: "Array operations have been converted from dedicated assembly routines for some array operations to a generic template implementation for all array operations. This provides huge performance increases (2-4x higher throughput) for array operations that were not previously vectorized. Furthermore the implementation makes better use of vectorization even for short arrays to heavily reduce latency for some operations (up to 4x)." Where does core.simd fit in?
See the linked druntime pull, core.simd is only imported for dmd: https://github.com/dlang/druntime/pull/1891/files#diff-c17bbc97c8719ab709a4a54e2f6924ceR67
Nov 03
parent Mike Parker <aldacron gmail.com> writes:
On Friday, 3 November 2017 at 10:14:27 UTC, Joakim wrote:
 See the linked druntime pull, core.simd is only imported for 
 dmd:

 https://github.com/dlang/druntime/pull/1891/files#diff-c17bbc97c8719ab709a4a54e2f6924ceR67
Ah, I see. I misunderstood Walter to be saying the user needed core.simd to get the vectorization. Thanks!
Nov 03
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/3/2017 3:02 AM, Mike Parker wrote:
 For clarity, where the changeling says that GDC & LDC use auto-vectorization, 
 that's actually happening with the array operations and core.simd is not 
 required, correct?
I think that GDC and LDC do do auto-vectorization, but I haven't verified it myself. Auto-vectorization is a fundamentally bizarre feature. It takes low level code and reverse-engineers it back into a higher level construct, and then proceeds to generate code for that higher level construct. Everything else a compiler does is start from a high level construct and then generate low level code. The trouble with AV is whether it succeeds or not depends on peculiarities (and I mean peculiarities) of the particular vector instruction set target. It can decided to not vectorize based on seemingly trivial and innocuous changes to the loop. The only way to tell is to benchmark it or look at the object file - methods that are unreliable (benchmarking) or do not scale (manually looking at the object file). What's needed is a language feature that is straightforwardly vectorizable. That would be D's array operations.
Nov 03
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Friday, 3 November 2017 at 19:46:58 UTC, Walter Bright wrote:
 On 11/3/2017 3:02 AM, Mike Parker wrote:
 For clarity, where the changeling says that GDC & LDC use 
 auto-vectorization, that's actually happening with the array 
 operations and core.simd is not required, correct?
I think that GDC and LDC do do auto-vectorization, but I haven't verified it myself. Auto-vectorization is a fundamentally bizarre feature. It takes low level code and reverse-engineers it back into a higher level construct, and then proceeds to generate code for that higher level construct. Everything else a compiler does is start from a high level construct and then generate low level code. The trouble with AV is whether it succeeds or not depends on peculiarities (and I mean peculiarities) of the particular vector instruction set target. It can decided to not vectorize based on seemingly trivial and innocuous changes to the loop.
I’ll share an anecdotal experience from a time I worked in reasearch lab of a well known tech giant. 2 senior researchers spent literally 2 weeks trying to coerce compiler into vectorizing an inner loop of a non-trivial matrix algorithm. The only diagnostic from compiler was “loop form is not correct”. Thankfully it did tell them it failed, else they’d have to disassemble it each time. I think eventually they either rewritten it to fit heuristic or just carried on with explicit intrinsics.
 What's needed is a language feature that is straightforwardly 
 vectorizable. That would be D's array operations.
Sadly array ops would be insufficient for said problem. It wasn’t a direct element wise expression.
Nov 03
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/3/2017 1:20 PM, Dmitry Olshansky wrote:
 Sadly array ops would be insufficient for said problem. It wasn’t a direct 
 element wise expression.
That sounds like that might be why it failed vectorization :-) If you recall the expression, it would be interesting to see it.
Nov 04
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Saturday, 4 November 2017 at 08:19:17 UTC, Walter Bright wrote:
 On 11/3/2017 1:20 PM, Dmitry Olshansky wrote:
 Sadly array ops would be insufficient for said problem. It 
 wasn’t a direct element wise expression.
That sounds like that might be why it failed vectorization :-)
As I recall it there were no trivial loops there. Usually these 2 magicians could make compiler eat it in a few hours of shuffling the code. They vectorized about half a dozen loops that way. The last one took 10 times more then the others taken together ;)
 If you recall the expression, it would be interesting to see it.
Even if I had it saved somewhere the place was NDA-ed to death. I traded 3 months of intellectual work (and property) for a modest amount of money. Interesting experience but no illusions about R&D centers anymore.
Nov 04
prev sibling parent Martin Nowak <code dawg.eu> writes:
On Friday, 3 November 2017 at 09:28:37 UTC, Daniel Kozak wrote:
 How should I compile my program to enable array vectorization?
Array operations refers to https://dlang.org/spec/arrays.html#array-operations.
 I have tried dmd -march=native, -march=avx2 as changlog suggest
It's -mcpu= not -march= for dmd, my bad. Unfortunate that dmd uses different switches than gcc. If you're compiling for 64-bit, you'll get SSE2 by default.
Nov 03
prev sibling next sibling parent ANtlord <antlord92 gmail.com> writes:
On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote:
 Glad to announce D 2.077.0.

 This release comes with a new, more compact mangling, templated 
 vector operations, reproducible dmd builds, and various fixes.
 Thanks to everyone involved in this 👏.

 http://downloads.dlang.org/releases/2.x/2.077.0/
 http://dlang.org/changelog/2.077.0.html
 The dlang.org website will get updated soon.

 -Martin
It's time to update documentation on devdocs
Nov 03
prev sibling next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote:
 Glad to announce D 2.077.0.

 This release comes with a new, more compact mangling, templated 
 vector operations, reproducible dmd builds, and various fixes.
 Thanks to everyone involved in this 👏.

 http://downloads.dlang.org/releases/2.x/2.077.0/
 http://dlang.org/changelog/2.077.0.html
 The dlang.org website will get updated soon.

 -Martin
Blog: https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/ Reddit: https://www.reddit.com/r/programming/comments/7ajg71/dmd_20770_released/
Nov 03
next sibling parent reply Arun Chandrasekaran <aruncxy gmail.com> writes:
On Friday, 3 November 2017 at 13:47:55 UTC, Mike Parker wrote:
 On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak 
 wrote:
 Glad to announce D 2.077.0.

 This release comes with a new, more compact mangling, 
 templated vector operations, reproducible dmd builds, and 
 various fixes.
 Thanks to everyone involved in this 👏.

 http://downloads.dlang.org/releases/2.x/2.077.0/
 http://dlang.org/changelog/2.077.0.html
 The dlang.org website will get updated soon.

 -Martin
Blog: https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/
Mike, thanks for the blog post. Few lines about how the name mangling issue was addressed would've been interesting know on the blog.
 Reddit:
 https://www.reddit.com/r/programming/comments/7ajg71/dmd_20770_released/
Nov 03
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Friday, 3 November 2017 at 14:00:38 UTC, Arun Chandrasekaran 
wrote:

 Mike, thanks for the blog post. Few lines about how the name 
 mangling issue was addressed would've been interesting know on 
 the blog.


There's a link in the post to the documentation describing the enhancement. As for how Rainer settled on that solution, I'm hoping to get a guest post out of him (though I haven't asked him yet, so shhhh!). [1] https://dlang.org/spec/abi.html#back_ref
Nov 03
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/3/17 10:00 AM, Arun Chandrasekaran wrote:
 On Friday, 3 November 2017 at 13:47:55 UTC, Mike Parker wrote:
 On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote:
 Glad to announce D 2.077.0.

 This release comes with a new, more compact mangling, templated 
 vector operations, reproducible dmd builds, and various fixes.
 Thanks to everyone involved in this 👏.

 http://downloads.dlang.org/releases/2.x/2.077.0/
 http://dlang.org/changelog/2.077.0.html
 The dlang.org website will get updated soon.

 -Martin
Blog: https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/
Mike, thanks for the blog post. Few lines about how the name mangling issue was addressed would've been interesting know on the blog.
A blog post I wrote about the issue itself (and a workaround that I employed to achieve the same result) is here: http://www.schveiguy.com/blog/2016/05/have-your-voldemort-types-and-keep-your-disk-space-too/ I hope Rainer agrees to the blog post as well. While I understand the concept, a detailed description of how the back references work would be very interesting. -Steve
Nov 03
parent Daniel Kozak <kozzi11 gmail.com> writes:
I have SIGSEGV when using DMD and simd types. This code works ok with GDC
and LDC fine, but SIGSEGV with latest DMD (maybe even with previous
versions I do not know)

https://dpaste.dzfl.pl/5476f5bef828

On Fri, Nov 3, 2017 at 5:04 PM, Steven Schveighoffer via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On 11/3/17 10:00 AM, Arun Chandrasekaran wrote:

 On Friday, 3 November 2017 at 13:47:55 UTC, Mike Parker wrote:

 On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote:

 Glad to announce D 2.077.0.

 This release comes with a new, more compact mangling, templated vector
 operations, reproducible dmd builds, and various fixes.
 Thanks to everyone involved in this =F0=9F=91=8F.

 http://downloads.dlang.org/releases/2.x/2.077.0/
 http://dlang.org/changelog/2.077.0.html
 The dlang.org website will get updated soon.

 -Martin
Blog: https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/
Mike, thanks for the blog post. Few lines about how the name mangling issue was addressed would've been interesting know on the blog.
A blog post I wrote about the issue itself (and a workaround that I employed to achieve the same result) is here: http://www.schveiguy.com/blog/2016/05/have-your-voldemort-ty pes-and-keep-your-disk-space-too/ I hope Rainer agrees to the blog post as well. While I understand the concept, a detailed description of how the back references work would be very interesting. -Steve
Nov 03
prev sibling parent Seb <seb wilzba.ch> writes:
On Friday, 3 November 2017 at 14:00:38 UTC, Arun Chandrasekaran 
wrote:
 Mike, thanks for the blog post. Few lines about how the name 
 mangling issue was addressed would've been interesting know on 
 the blog.
The regarding main PR contains a lot of info: https://github.com/dlang/dmd/pull/5855 For example, some quick stats on the reduced size of Phobos: https://github.com/dlang/dmd/pull/5855#issuecomment-315565256
Nov 03
prev sibling parent Joakim <dlang joakim.fea.st> writes:
On Friday, 3 November 2017 at 13:47:55 UTC, Mike Parker wrote:
 On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak 
 wrote:
 Glad to announce D 2.077.0.

 This release comes with a new, more compact mangling, 
 templated vector operations, reproducible dmd builds, and 
 various fixes.
 Thanks to everyone involved in this 👏.

 http://downloads.dlang.org/releases/2.x/2.077.0/
 http://dlang.org/changelog/2.077.0.html
 The dlang.org website will get updated soon.

 -Martin
Blog: https://dlang.org/blog/2017/11/03/dmd-2-077-0-released/ Reddit: https://www.reddit.com/r/programming/comments/7ajg71/dmd_20770_released/
Typo: particulary "case so that," -> "case, so that" (I'd also remove the comma after that)
Nov 03
prev sibling parent reply Gerald <gerald.b.nunn gmail.com> writes:
On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak wrote:
 Glad to announce D 2.077.0.

 This release comes with a new, more compact mangling, templated 
 vector operations, reproducible dmd builds, and various fixes.
 Thanks to everyone involved in this 👏.

 http://downloads.dlang.org/releases/2.x/2.077.0/
 http://dlang.org/changelog/2.077.0.html
 The dlang.org website will get updated soon.

 -Martin
Great release, in Tilix the dmd executable was quite large due to all the symbols generated in GtkD for event handling (mea culpa since I did that PR). The new version is much better with only a 6 MB difference between the stripped and non-stripped versions.
Nov 03
parent crimaniak <crimaniak gmail.com> writes:
On Friday, 3 November 2017 at 14:41:06 UTC, Gerald wrote:
 On Thursday, 2 November 2017 at 22:35:03 UTC, Martin Nowak 
 wrote:
 Glad to announce D 2.077.0.
... The new version is much better with only a 6 MB difference between the stripped and non-stripped versions.
My vibe-d application debug build reduced from 56Mb to 44Mb (release = 19Mb).
Nov 03