www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - A little Py Vs C++

reply "bearophile" <bearophileHUGS lycos.com> writes:
Maybe this Reddit thread should be completed with a nice D 
version :-)

http://www.reddit.com/r/programming/comments/12ecq5/c11_and_boost_succinct_like_python/

The thread contains some sad comments:

http://www.reddit.com/r/programming/comments/12ecq5/c11_and_boost_succinct_like_python/c6uihbi

http://www.reddit.com/r/programming/comments/12ecq5/c11_and_boost_succinct_like_python/c6ugw8h

Bye,
bearophile
Oct 31 2012
next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Wednesday, 31 October 2012 at 23:04:15 UTC, bearophile wrote:
 The thread contains some sad comments:

It's unfortunate that there's still bad press circulating about a situation that is long gone. I suppose you just have to try and ignore those people. A more interesting comment is this one: "But the real problem here is that in order to achieve even that, the complexity and amount of concepts you have to deal with in C++11 is mind boggling." The same is true in D. Well-written D code often does look rather elegant, but the amount of understanding needed to write beautiful D code is staggering.
Nov 01 2012
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/1/2012 12:19 PM, Paulo Pinto wrote:> On the other hand, except for the 
programming drones, most D concepts are also
 available in most mainstream languages.

Most of them are also well established in one form or another, with proven value.
Nov 01 2012
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/1/2012 2:20 PM, bearophile wrote:
 Some complexity comes from the
 desire to do more and more. As example see this recent request from Manu,

What about all your feature requests? I think you've made more than anyone, by a factor of 10 at least! :-) As for Manu's request http://d.puremagic.com/issues/show_bug.cgi?id=8108 I've gone over with him why he needs it, and there's no other reasonable way. He needs it for real code in a real application.
Nov 01 2012
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-11-01 23:51, Walter Bright wrote:

 What about all your feature requests? I think you've made more than
 anyone, by a factor of 10 at least!

 :-)

 As for Manu's request

 http://d.puremagic.com/issues/show_bug.cgi?id=8108

 I've gone over with him why he needs it, and there's no other reasonable
 way. He needs it for real code in a real application.

This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented. Although I think something like AST macros could possible solve issue 8108 and a whole bunch of other features, a few already present in the language. -- /Jacob Carlborg
Nov 02 2012
next sibling parent reply Don Clugston <dac nospam.com> writes:
On 02/11/12 09:07, Jacob Carlborg wrote:
 On 2012-11-01 23:51, Walter Bright wrote:

 What about all your feature requests? I think you've made more than
 anyone, by a factor of 10 at least!

 :-)

 As for Manu's request

 http://d.puremagic.com/issues/show_bug.cgi?id=8108

 I've gone over with him why he needs it, and there's no other reasonable
 way. He needs it for real code in a real application.

This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented.

He just knows how to convince Walter. (Hint: use real-world use cases. Arguments from theoretical computer science carry almost no weight with Walter).
 Although I think something like
 AST macros could possible solve issue 8108 and a whole bunch of other
 features, a few already present in the language.

Yes, and they could cure cancer. "AST macros" can do anything, because they are completely undefined. Without even a vague proposal, it seems like a rather meaningless term.
Nov 02 2012
next sibling parent reply Don Clugston <dac nospam.com> writes:
On 02/11/12 10:12, Jens Mueller wrote:
 Don Clugston wrote:
 On 02/11/12 09:07, Jacob Carlborg wrote:
 On 2012-11-01 23:51, Walter Bright wrote:

 What about all your feature requests? I think you've made more than
 anyone, by a factor of 10 at least!

 :-)

 As for Manu's request

 http://d.puremagic.com/issues/show_bug.cgi?id=8108

 I've gone over with him why he needs it, and there's no other reasonable
 way. He needs it for real code in a real application.

This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented.

He just knows how to convince Walter. (Hint: use real-world use cases. Arguments from theoretical computer science carry almost no weight with Walter).

I think there are lots of issues in the bug tracker which describe real use cases. The point is that these are more like bugs whereas Manu's proposals are more like feature requests. Jens

The SIMD stuff has no workarounds. I don't know of many other feature requests in that category.
Nov 02 2012
parent Don Clugston <dac nospam.com> writes:
On 02/11/12 11:57, Jens Mueller wrote:
 Peter Alexander wrote:
 On Friday, 2 November 2012 at 10:24:34 UTC, Jens Mueller wrote:
 Then I have a serious misunderstanding.
 I thought D introduced array operations to allow the compiler to
 generate efficient vector operations (in the long run), i.e.
 generate
 SIMD code. Why is this not working out?

It works fine for large vectors. For small vectors, it is horrendously slow. The syntax a[] += b[] essentially calls a function which is designed to work for large vectors. It has to determine alignment, load everything from memory, do the operations, then store it back. The SIMD extensions allow you to define variables that are guaranteed to be aligned and will probably be in the right registers to start with. Using them, the vectors ops don't need to determine alignment, and often don't need to do lots of loads/stores. Both have their purposes.

I see. But can't the alignment problem be solved by using align. Then have the compiler emits a call that checks for alignment if none was specified else use a faster version.

No. For SIMD, you cannot afford to have even a single machine instruction inserted, or the benefit is entirely lost.
Nov 02 2012
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-11-02 09:38, Don Clugston wrote:

 Yes, and they could cure cancer. "AST macros" can do anything, because
 they are completely undefined. Without even a vague proposal, it seems
 like a rather meaningless term.

I know, I know. It's be brought up before and without any more detailed specification/definition is hard to do anything about it. I've done some research in the subject in the hope I can write down something useful that could be a proposal for D, but I don't have anything yet. -- /Jacob Carlborg
Nov 02 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-11-02 19:03, Dejan Lekic wrote:

 How do you think people came up with those bug reports? By some magic?
 :) You think they did not actually *pull their hair off* trying to
 figure out why their REAL LIFE program does not work, and when we could
 not find the reason we naturally started thinking "oh, it might be yet
 another missing thing in D" or "oh, yet another DMD/phobos/druntime
 bug!"...
 Or, another scenario (a very typical one) - after DMD/phobos/druntime is
 updated, my production application no longer compiles...

Good point. When you do put the whole real life program in the bug report "it's too big, you need to create a small test case". When you do create a small test case for the feature/bug it's suddenly not "real life" code. -- /Jacob Carlborg
Nov 02 2012
prev sibling next sibling parent reply Don Clugston <dac nospam.com> writes:
On 02/11/12 10:01, Jens Mueller wrote:
 Jacob Carlborg wrote:
 On 2012-11-01 23:51, Walter Bright wrote:

 What about all your feature requests? I think you've made more than
 anyone, by a factor of 10 at least!

 :-)

 As for Manu's request

 http://d.puremagic.com/issues/show_bug.cgi?id=8108

 I've gone over with him why he needs it, and there's no other reasonable
 way. He needs it for real code in a real application.

This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented. Although I think something like AST macros could possible solve issue 8108 and a whole bunch of other features, a few already present in the language.

I had the same thought when reading this. Very disappointing. An issue with zero votes is fixed instead of more important ones. Why do I vote anyway? Regarding SIMD I have the feeling that because it is built into the compiler static vectors have actually failed what they promised. I thought D proposed a portable way of vector operations such that you write float[4] = a[] + b[] and the compiler generates SIMD code for you.

Not for short vectors. They are more like the builtin operations in Fortran, ie designed for large vectors. More for scientific kinds of applications than games. (The two applications look superficially similar, but in practice they have little in common).
Nov 02 2012
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/2/2012 3:50 AM, Jens Mueller wrote:
 Okay. For me they look the same. Can you elaborate, please? Assume I
 want to add two float vectors which is common in both games and
 scientific computing. The only difference is in games their length is
 usually 3 or 4 whereas in scientific computing they are of arbitrary
 length. Why do I need instrinsics to support the game setting?

Another excellent question. Most languages have taken the "auto-vectorization" approach of reverse engineering loops to turn them into high level constructs, and then compiling the code into special SIMD instructions. How to do this is explained in detail in the (rare) book "The Software Vectorization Handbook" by Bik, which I fortunately was able to obtain a copy of. This struck me as a terrible approach, however. It just seemed stupid to try to teach the compiler to reverse engineer low level code into high level code. A better design would be to start with high level code. Hence, the appearance of D vector operations. The trouble with D vector operations, however, is that they are too general purpose. The SIMD instructions are very quirky, and it's easy to unwittingly and silently cause the compiler to generate absolutely terribly slow code. The reasons for that are the alignment requirements, coupled with the SIMD instructions not being orthogonal - some operations work for some types and not for others, in a way that is unintuitive unless you're carefully reading the SIMD specs. Just saying align(16) isn't good enough, as the vector ops work on slices and those slices aren't always aligned. So each one has to check alignment at runtime, which is murder on performance. If a particular vector op for a particular type has no SIMD support, then the compiler has to generate workaround code. This can also have terrible performance consequences. So the user writes vector code, benchmarks it, finds zero improvement, and the reasons why will be elusive to anyone but an expert SIMD programmer. (Auto-vectorizing technology has similar issues, pretty much meaning you won't get fast code out of it unless you've got a habit of examining the assembler output and tweaking as necessary.) Enter Manu, who has a lot of experience making SIMD work for games. His proposal was: 1. Have native SIMD types. This will guarantee alignment, and will guarantee a compile time error for SIMD types that are not supported by the CPU. 2. Have the compiler issue an error for SIMD operations that are not supported by the CPU, rather than silently generating inefficient workaround code. 3. There are all kinds of weird but highly useful SIMD instructions that don't have a straightforward representation in high level code, such as saturated arithmetic. Manu's answer was to expose these instructions via intrinsics, so the user can string them together, be sure that they will generate real SIMD instructions, while the compiler can deal with register allocation. This approach works, is inlineable, generates code as good as hand-built assembler, and is useable by regular programmers. I won't say there aren't better approaches, but this one we know works.
Nov 02 2012
parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/2/2012 3:10 PM, Jens Mueller wrote:
 I see. Thanks for clarifying.
 If I want fast vector operations I have to use core.simd. The built-in
 vector operations won't fit the bill.

At the moment, yes. However, Manu is working on developing a higher order layer.
Nov 02 2012
prev sibling next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 11/02/2012 09:07 AM, Jacob Carlborg wrote:
 On 2012-11-01 23:51, Walter Bright wrote:

 What about all your feature requests? I think you've made more than
 anyone, by a factor of 10 at least!

 :-)

 As for Manu's request

 http://d.puremagic.com/issues/show_bug.cgi?id=8108

 I've gone over with him why he needs it, and there's no other reasonable
 way. He needs it for real code in a real application.

This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented.

SIMD support is necessary and the fix for 8108 is very simple. If the way default arguments work shall be improved though, it would add additional ways that the compiler can screw up conditional compilation.
 Although I think something like
 AST macros could possible solve issue 8108 and a whole bunch of other
 features, a few already present in the language.

A sufficiently well/badly designed macro feature can potentially replace most other language features.
Nov 02 2012
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/2/2012 2:01 AM, Jens Mueller wrote:
 I had the same thought when reading this. Very disappointing. An issue
 with zero votes is fixed instead of more important ones.

It's a very fair question. Manu works for Remedy Games, a developer of hit PC games, such as Max Payne and Alan Wake. He champions a team that is investigating committing to D for a major new game. This would be a huge design win for D, and an enormous boost for D. I view the most effective use of my time at the moment is to ensure that they can bet on D and win. Remedy's use of D for a high profile product will prove that D is ready and able for the big time, and that others can develop using D with confidence. Nearly all of what he needs the D community needs anyway, such as the big push for Win64 support and the SIMD support (more on that in another post).
Nov 02 2012
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/2/2012 10:22 AM, Walter Bright wrote:
 such as the big push for Win64 support

I want to give a shoutout here for Rainer Schuetze who has been a big help behind the scenes in getting the Win64 symbolic debug support working. He's saved me a ton of time.
Nov 02 2012
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/2/12 1:22 PM, Walter Bright wrote:
 On 11/2/2012 2:01 AM, Jens Mueller wrote:
  > I had the same thought when reading this. Very disappointing. An issue
  > with zero votes is fixed instead of more important ones.

 It's a very fair question.

 Manu works for Remedy Games, a developer of hit PC games, such as Max
 Payne and Alan Wake. He champions a team that is investigating
 committing to D for a major new game. This would be a huge design win
 for D, and an enormous boost for D. I view the most effective use of my
 time at the moment is to ensure that they can bet on D and win. Remedy's
 use of D for a high profile product will prove that D is ready and able
 for the big time, and that others can develop using D with confidence.

 Nearly all of what he needs the D community needs anyway, such as the
 big push for Win64 support and the SIMD support (more on that in another
 post).

I should add that I'm also totally behind this. When Walter jumped into implementing SIMD support on a hunch, I completely disagreed, but that was a great decision. Andrei
Nov 02 2012
next sibling parent deadalnix <deadalnix gmail.com> writes:
Le 02/11/2012 18:46, Andrei Alexandrescu a écrit :
 On 11/2/12 1:22 PM, Walter Bright wrote:
 On 11/2/2012 2:01 AM, Jens Mueller wrote:
 I had the same thought when reading this. Very disappointing. An issue
 with zero votes is fixed instead of more important ones.

It's a very fair question. Manu works for Remedy Games, a developer of hit PC games, such as Max Payne and Alan Wake. He champions a team that is investigating committing to D for a major new game. This would be a huge design win for D, and an enormous boost for D. I view the most effective use of my time at the moment is to ensure that they can bet on D and win. Remedy's use of D for a high profile product will prove that D is ready and able for the big time, and that others can develop using D with confidence. Nearly all of what he needs the D community needs anyway, such as the big push for Win64 support and the SIMD support (more on that in another post).

I should add that I'm also totally behind this. When Walter jumped into implementing SIMD support on a hunch, I completely disagreed, but that was a great decision. Andrei

I still don't understand the benefice over align(16) float[4] .
Nov 02 2012
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-11-02 18:46, Andrei Alexandrescu wrote:

 I should add that I'm also totally behind this. When Walter jumped into
 implementing SIMD support on a hunch, I completely disagreed, but that
 was a great decision.

I can absolutely understand why he did it but it would be really nice if you (Walter, Andrei and probably others as well) could be more transparent about things like these. I think this would really help the community. -- /Jacob Carlborg
Nov 02 2012
next sibling parent deadalnix <deadalnix gmail.com> writes:
Le 02/11/2012 19:19, Jacob Carlborg a écrit :
 On 2012-11-02 18:46, Andrei Alexandrescu wrote:

 I should add that I'm also totally behind this. When Walter jumped into
 implementing SIMD support on a hunch, I completely disagreed, but that
 was a great decision.

I can absolutely understand why he did it but it would be really nice if you (Walter, Andrei and probably others as well) could be more transparent about things like these. I think this would really help the community.

I couldn't agree more.
Nov 02 2012
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/2/2012 11:19 AM, Jacob Carlborg wrote:
 I can absolutely understand why he did it but it would be really nice if you
 (Walter, Andrei and probably others as well) could be more transparent about
 things like these. I think this would really help the community.

I apologize for being circumspect about this, but I have to respect peoples' privacy and I cleared what I posted about Remedy with them before posting it.
Nov 02 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-11-02 19:36, Walter Bright wrote:

 I apologize for being circumspect about this, but I have to respect
 peoples' privacy and I cleared what I posted about Remedy with them
 before posting it.

This has nothing to do with what Manu does for a living (sure, mentioning Remedy gives it more weight, at least for me). It's more in the line of creating a post/starting a new thread saying something like: "After a throughout discussion with Manu (or 'a fellow D programmer' if he/she prefers to be anonymous) we have decided it would be in best interest of D if we implement this particular feature. From now on this is where I will focus most most of my time". Something like this would be far better then suddenly seeing commits regarding SIMD (or whatever feature it might be) for out of the blue. -- /Jacob Carlborg
Nov 02 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-11-02 21:22, jerro wrote:

 SIMD support was discussed here at length in the days before it was
 implemented. See this thread:

Yeah, I know it's been talked about and discussed, but it's the final decision that's missing. -- /Jacob Carlborg
Nov 02 2012
prev sibling next sibling parent deadalnix <deadalnix gmail.com> writes:
Le 02/11/2012 19:25, mist a écrit :
 On Friday, 2 November 2012 at 18:19:54 UTC, Jacob Carlborg wrote:
 On 2012-11-02 18:46, Andrei Alexandrescu wrote:

 I should add that I'm also totally behind this. When Walter jumped into
 implementing SIMD support on a hunch, I completely disagreed, but that
 was a great decision.

I can absolutely understand why he did it but it would be really nice if you (Walter, Andrei and probably others as well) could be more transparent about things like these. I think this would really help the community.

So true. Strong mid-term vision of rationales of main language developers really helps to reason about it.

Yeah, and avoid noise and help people concentrate on main points.
Nov 02 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-11-02 20:22, Manu wrote:

 This is probably my fault, and a matter of corporate transparency. We
 didn't want to make a noise about it until we reached a particular level
 of confidence.
 We're fairly invested now, and quietly hopeful it will go ahead from here.

This has nothing to do with corporate transparency. It has to do with transparency to the community, see my reply to Walter: http://forum.dlang.org/thread/jauixhakwvpgsghapzvz forum.dlang.org?page=6#post-k718iu:242iu6:241:40digitalmars.com -- /Jacob Carlborg
Nov 02 2012
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/2/2012 10:32 AM, Andrej Mitrovic wrote:
 Manu if you know of any trivial-type bugs you'd like to get fixed (for
 example error messages) let me know so I can prioritize them. There's
 a ton of open issues and I usually select the ones for fixing by
 random. I've managed to fix a few so far. :)

Thanks, any help would be appreciated. I'll ask Manu for a list of his priority issues, and even minor ones. What will also help is trying out the beta Win64 target. I've been posting nightly new betas for it.
Nov 02 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-11-02 18:22, Walter Bright wrote:

 It's a very fair question.

 Manu works for Remedy Games, a developer of hit PC games, such as Max
 Payne and Alan Wake. He champions a team that is investigating
 committing to D for a major new game. This would be a huge design win
 for D, and an enormous boost for D. I view the most effective use of my
 time at the moment is to ensure that they can bet on D and win. Remedy's
 use of D for a high profile product will prove that D is ready and able
 for the big time, and that others can develop using D with confidence.

 Nearly all of what he needs the D community needs anyway, such as the
 big push for Win64 support and the SIMD support (more on that in another
 post).

I can absolutely see the point in this. Thanks for taking the time and explaining this. -- /Jacob Carlborg
Nov 02 2012
prev sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Simen Kjaeraas wrote:
 On 2012-39-03 12:11, Jens Mueller <jens.k.mueller gmx.de> wrote:
 
 
I have a fork; some people are using it already. It still needs a lot of
work though; some compilers missing parts, platforms not supported.
That said, it's not an effort to address D's natural vector
syntax, the key
goal is to provide a hardware SIMD API that is as orthogonal as possible
and portable (with confidence it will run reasonably well).
I wonder if druntime could be enhanced to use the SIMD stuff
though in the
functions that perform the natural vector operations, might
offer some nice
little boosts.

Cool. It'll be nice though if D's vector operations could be expressed on top of it. I mean a[] + b[] is so much easier to read.

Perhaps I'm the confused one here, but as I understand it, that's the whole point of the latter part of the paragraph - array operations are implemented in druntime, and that implementation might benefit from using SIMD instructions.

Sorry. I'm the one being confused. You're right Simen. Jens
Nov 03 2012
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/1/12 6:51 PM, Walter Bright wrote:
 On 11/1/2012 2:20 PM, bearophile wrote:
 Some complexity comes from the
 desire to do more and more. As example see this recent request from Manu,

What about all your feature requests? I think you've made more than anyone, by a factor of 10 at least! :-) As for Manu's request http://d.puremagic.com/issues/show_bug.cgi?id=8108 I've gone over with him why he needs it, and there's no other reasonable way. He needs it for real code in a real application.

I'd argue this actually is part of a category of features that does not increase the complexity of the language (quite the contrary in fact). This is because stating that declaration+definition in the same file won't work takes actually more cognitive load than just allowing it. By the consistency principle, one should infer unknown parts of a complex system from knowing the others. Consider then this setup: * Declarations (without definition) of functions are allowed. * Definitions are allowed. * Declarations and definitions are allowed in distinct files in the same project as long as they match. At this point, it is more tenuous to argue that "same file" is a special case that should prevent declarations and definitions to coexist, than to just let it happen and let the consistency principle take care of explaining it. Andrei
Nov 02 2012
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 1 November 2012 at 18:06:21 UTC, Peter Alexander 
wrote:
 On Wednesday, 31 October 2012 at 23:04:15 UTC, bearophile wrote:
 The thread contains some sad comments:

It's unfortunate that there's still bad press circulating about a situation that is long gone. I suppose you just have to try and ignore those people. A more interesting comment is this one: "But the real problem here is that in order to achieve even that, the complexity and amount of concepts you have to deal with in C++11 is mind boggling." The same is true in D. Well-written D code often does look rather elegant, but the amount of understanding needed to write beautiful D code is staggering.

I have to agree having to deal with lots of concepts. On the other hand, except for the programming drones, most D concepts are also available in most mainstream languages. -- Paulo
Nov 01 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Nov 01, 2012 at 08:19:56PM +0100, Paulo Pinto wrote:
 On Thursday, 1 November 2012 at 18:06:21 UTC, Peter Alexander wrote:

A more interesting comment is this one:

"But the real problem here is that in order to achieve even that,
the complexity and amount of concepts you have to deal with in
C++11 is mind boggling."

The same is true in D. Well-written D code often does look rather
elegant, but the amount of understanding needed to write beautiful
D code is staggering.

I have to agree having to deal with lots of concepts.

I don't see it as a problem, unless one is a programmer of the drone persuasion. Many of D's concepts are liberatingly powerful, and very potent in combination.
 On the other hand, except for the programming drones, most D
 concepts are also available in most mainstream languages.

If we want to minimize the number of concepts, we should program using Lambda calculus. ;-) We already have lambda-syntax for delegates, after all. Now just restrict all statements to only lambda expressions, get rid of difficult concepts like arithmetic operators, variables and imperative programming, and we have a winner on our hands. Seriously, though, imagining that one can program effectively without learning new concepts is a preposterous proposition to me. I just don't understand the unwillingness to learn. T -- Computers shouldn't beep through the keyhole.
Nov 01 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Thursday, 1 November 2012 at 18:06:21 UTC, Peter Alexander 
wrote:
 The same is true in D. Well-written D code often does look 
 rather elegant, but the amount of understanding needed to write 
 beautiful D code is staggering.

I'll second that! I'm finally making good progress with D after a whole lot of effort, and there were (and still are) periods where I felt like giving up, although not so much anymore. The number one missing item that I would love to get my hands on is comprehensive documentaion. What we have now leaves out far too much detail, and may be missing entire features, and what is available is scattered all over the place. Reading the TDPL book is a huge help for understanding D, it's essential reading for sure, but it's only an overview rather than a full text on the subject. The TDPL lacks many of the fine details that one needs to know in order to write proper D code. I find myself experimenting a lot in order to figure out what the various obscure details are, but without comprehensive documentation, I cannot be certain if the observed behaviours are actually correct or fully complete under all situations. --rt
Nov 01 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Nov 01, 2012 at 08:49:40PM +0100, Rob T wrote:
 On Thursday, 1 November 2012 at 18:06:21 UTC, Peter Alexander wrote:
The same is true in D. Well-written D code often does look rather
elegant, but the amount of understanding needed to write beautiful
D code is staggering.

I'll second that! I'm finally making good progress with D after a whole lot of effort, and there were (and still are) periods where I felt like giving up, although not so much anymore. The number one missing item that I would love to get my hands on is comprehensive documentaion. What we have now leaves out far too much detail, and may be missing entire features, and what is available is scattered all over the place. Reading the TDPL book is a huge help for understanding D, it's essential reading for sure, but it's only an overview rather than a full text on the subject. The TDPL lacks many of the fine details that one needs to know in order to write proper D code. I find myself experimenting a lot in order to figure out what the various obscure details are, but without comprehensive documentation, I cannot be certain if the observed behaviours are actually correct or fully complete under all situations.

Ali's D book might be helpful: http://ddili.org/ders/d.en/index.html But yeah, the current documentation needs work. Unfortunately, wishing for it to happen won't make it happen, so if you contribute (use the "improve this page" link) that would be great, and suggestions are also helpful. I recently revised the docs for std.range to explain briefly what ranges are and why we even need them, but this is only one piece of the entire standard library. Many other modules need improved docs. And there is still the need for a general overview to put everything in perspective. T -- It is impossible to make anything foolproof because fools are so ingenious. -- Sammy
Nov 01 2012
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 1 November 2012 at 19:44:04 UTC, H. S. Teoh wrote:
 On Thu, Nov 01, 2012 at 08:19:56PM +0100, Paulo Pinto wrote:
 On Thursday, 1 November 2012 at 18:06:21 UTC, Peter Alexander 
 wrote:

A more interesting comment is this one:

"But the real problem here is that in order to achieve even 
that,
the complexity and amount of concepts you have to deal with in
C++11 is mind boggling."

The same is true in D. Well-written D code often does look 
rather
elegant, but the amount of understanding needed to write 
beautiful
D code is staggering.

I have to agree having to deal with lots of concepts.

I don't see it as a problem, unless one is a programmer of the drone persuasion. Many of D's concepts are liberatingly powerful, and very potent in combination.
 On the other hand, except for the programming drones, most D
 concepts are also available in most mainstream languages.

If we want to minimize the number of concepts, we should program using Lambda calculus. ;-) We already have lambda-syntax for delegates, after all. Now just restrict all statements to only lambda expressions, get rid of difficult concepts like arithmetic operators, variables and imperative programming, and we have a winner on our hands. Seriously, though, imagining that one can program effectively without learning new concepts is a preposterous proposition to me. I just don't understand the unwillingness to learn. T

It is not the unwillingness to learn, rather the standard HR way of getting replaceable programming drones in most enterprises. This was already discussed a few times. -- Paulo
Nov 01 2012
prev sibling next sibling parent "Tavi Cacina" <octavian.cacina outlook.com> writes:
On Thursday, 1 November 2012 at 19:49:42 UTC, Rob T wrote:
 On Thursday, 1 November 2012 at 18:06:21 UTC, Peter Alexander 
 wrote:

The number one missing item that I would love to get my hands on is comprehensive documentaion. What we have now leaves out far too much detail, and may be missing entire features, and what is available is scattered all over the place.

Workaround: I find it useful to check the code of a 'std' module that interests me. A lot of unittests and comments are quite usefull. For ex. about the RAII/ref-counted stuff I checked the stdio.d File.
Nov 01 2012
prev sibling next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Peter Alexander:

 The same is true in D. Well-written D code often does look 
 rather elegant, but the amount of understanding needed to write 
 beautiful D code is staggering.

It's almost impossible to keep languages simple. Some complexity comes from the desire to do more and more. As example see this recent request from Manu, it's simple, but it also increases the complexity of D: http://d.puremagic.com/issues/show_bug.cgi?id=8108 Bye, bearophile
Nov 01 2012
next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Don Clugston wrote:
 On 02/11/12 11:57, Jens Mueller wrote:
Peter Alexander wrote:
On Friday, 2 November 2012 at 10:24:34 UTC, Jens Mueller wrote:
Then I have a serious misunderstanding.
I thought D introduced array operations to allow the compiler to
generate efficient vector operations (in the long run), i.e.
generate
SIMD code. Why is this not working out?

It works fine for large vectors. For small vectors, it is horrendously slow. The syntax a[] += b[] essentially calls a function which is designed to work for large vectors. It has to determine alignment, load everything from memory, do the operations, then store it back. The SIMD extensions allow you to define variables that are guaranteed to be aligned and will probably be in the right registers to start with. Using them, the vectors ops don't need to determine alignment, and often don't need to do lots of loads/stores. Both have their purposes.

I see. But can't the alignment problem be solved by using align. Then have the compiler emits a call that checks for alignment if none was specified else use a faster version.

No. For SIMD, you cannot afford to have even a single machine instruction inserted, or the benefit is entirely lost.

But the compiler knows about the alignment, doesn't it? align(16) float[4] a; vs float[4] a; In the former case the compiler can generate better code and it should. The above syntax is not supported. But my point is all the compiler cares about is the alignment which can be specified in the code somehow. Sorry for being stubborn. Jens
Nov 02 2012
prev sibling next sibling parent "Kapps" <opantm2+spam gmail.com> writes:
On Friday, 2 November 2012 at 14:22:34 UTC, Jens Mueller wrote:
 But the compiler knows about the alignment, doesn't it?

 align(16) float[4] a;
 vs
 float[4] a;

 In the former case the compiler can generate better code and it 
 should.
 The above syntax is not supported. But my point is all the 
 compiler
 cares about is the alignment which can be specified in the code 
 somehow.
 Sorry for being stubborn.

 Jens

Note: My knowledge of SIMD/SSE is fairly limited, and may be somewhat out of date. In other words, some of this may be flat out wrong. First, just because you have something that can have SIMD operations performed on it, doesn't mean you necessarily want to. SSE instructions for example have to store things in the XMM registers, and accessing the actual values of individual elements in the vector is expensive. When using SSE, you want to avoid accessing individual elements as much as possible. Not following this tends to hurt performance quite badly. Yet when you just have a float[4], you may or may not be frequently or infrequently accessing individual elements. The compiler can't know whether you use it as a single SIMD vector more often, or use it to simply store 4 elements more often. You could be aligning it for any reason, so it's not too fair a way of determining it. Secondly, you can't really know which SIMD instructions are supported by your target CPU. It's safe to say SSE2 is supported for pretty much all x86 CPUs at this point, but something like SSE4.2 instructions may not be. Just because the compiler knows that the CPU compiling it supports it doesn't mean that the CPU running the program will have those instructions. Lastly, we'd still need SIMD intrinsics. It may be simple to tell that a float[4] + float[4] operation could use addps, but it would be more difficult to determine when to use something like dotps (dot product across two SIMD vectors), and various other instructions. Not to mention, non-x86 architectures.
Nov 02 2012
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Manu wrote:
 On 3 November 2012 01:41, Walter Bright <newshound2 digitalmars.com> wrote:
 
 On 11/2/2012 3:10 PM, Jens Mueller wrote:

 I see. Thanks for clarifying.
 If I want fast vector operations I have to use core.simd. The built-in
 vector operations won't fit the bill.


operations..."; this is not automatically faster by nature, it requires strict self-control in terms of application, and very careful attention if you want your code to be portable. At the moment, yes.
 However, Manu is working on developing a higher order layer.

I have a fork; some people are using it already. It still needs a lot of work though; some compilers missing parts, platforms not supported. That said, it's not an effort to address D's natural vector syntax, the key goal is to provide a hardware SIMD API that is as orthogonal as possible and portable (with confidence it will run reasonably well). I wonder if druntime could be enhanced to use the SIMD stuff though in the functions that perform the natural vector operations, might offer some nice little boosts.

Cool. It'll be nice though if D's vector operations could be expressed on top of it. I mean a[] + b[] is so much easier to read. Jens
Nov 03 2012
prev sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On 2012-39-03 12:11, Jens Mueller <jens.k.mueller gmx.de> wrote:


 I have a fork; some people are using it already. It still needs a lot of
 work though; some compilers missing parts, platforms not supported.
 That said, it's not an effort to address D's natural vector syntax, the  
 key
 goal is to provide a hardware SIMD API that is as orthogonal as possible
 and portable (with confidence it will run reasonably well).
 I wonder if druntime could be enhanced to use the SIMD stuff though in  
 the
 functions that perform the natural vector operations, might offer some  
 nice
 little boosts.

Cool. It'll be nice though if D's vector operations could be expressed on top of it. I mean a[] + b[] is so much easier to read.

Perhaps I'm the confused one here, but as I understand it, that's the whole point of the latter part of the paragraph - array operations are implemented in druntime, and that implementation might benefit from using SIMD instructions. -- Simen
Nov 03 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Thursday, 1 November 2012 at 20:10:47 UTC, H. S. Teoh wrote:
 But yeah, the current documentation needs work. Unfortunately, 
 wishing
 for it to happen won't make it happen, so if you contribute 
 (use the
 "improve this page" link) that would be great

I never noticed that link before. When I click on it, I go to github showing me a webpage with "404" on it. The text over link says I need a github account and to submit pull requests. I'm not sure how I'm supposed to make changes with that. I know about the regular Wiki and have made a contribution there, that's very simple to do. --rt
Nov 01 2012
prev sibling next sibling parent "Jesse Phillips" <jessekphillips+D gmail.com> writes:
On Thursday, 1 November 2012 at 23:56:00 UTC, Rob T wrote:
 On Thursday, 1 November 2012 at 20:10:47 UTC, H. S. Teoh wrote:
 But yeah, the current documentation needs work. Unfortunately, 
 wishing
 for it to happen won't make it happen, so if you contribute 
 (use the
 "improve this page" link) that would be great

I never noticed that link before. When I click on it, I go to github showing me a webpage with "404" on it. The text over link says I need a github account and to submit pull requests. I'm not sure how I'm supposed to make changes with that. I know about the regular Wiki and have made a contribution there, that's very simple to do. --rt

All the documentation is written in DDoc. The website is hosted on github. Github provides an online text editor usable against your repository. Your repository is created as a fork of the original. Github uses pull requests to move changes upstream. It isn't as simple as a wiki but it isn't too bad. Hope that is enough information to get started.
Nov 01 2012
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Jacob Carlborg wrote:
 On 2012-11-01 23:51, Walter Bright wrote:
 
What about all your feature requests? I think you've made more than
anyone, by a factor of 10 at least!

:-)

As for Manu's request

http://d.puremagic.com/issues/show_bug.cgi?id=8108

I've gone over with him why he needs it, and there's no other reasonable
way. He needs it for real code in a real application.

This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented. Although I think something like AST macros could possible solve issue 8108 and a whole bunch of other features, a few already present in the language.

I had the same thought when reading this. Very disappointing. An issue with zero votes is fixed instead of more important ones. Why do I vote anyway? Regarding SIMD I have the feeling that because it is built into the compiler static vectors have actually failed what they promised. I thought D proposed a portable way of vector operations such that you write float[4] = a[] + b[] and the compiler generates SIMD code for you. Jens
Nov 02 2012
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Don Clugston wrote:
 On 02/11/12 09:07, Jacob Carlborg wrote:
On 2012-11-01 23:51, Walter Bright wrote:

What about all your feature requests? I think you've made more than
anyone, by a factor of 10 at least!

:-)

As for Manu's request

http://d.puremagic.com/issues/show_bug.cgi?id=8108

I've gone over with him why he needs it, and there's no other reasonable
way. He needs it for real code in a real application.

This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented.

He just knows how to convince Walter. (Hint: use real-world use cases. Arguments from theoretical computer science carry almost no weight with Walter).

I think there are lots of issues in the bug tracker which describe real use cases. The point is that these are more like bugs whereas Manu's proposals are more like feature requests. Jens
Nov 02 2012
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Don Clugston wrote:
 On 02/11/12 10:12, Jens Mueller wrote:
Don Clugston wrote:
On 02/11/12 09:07, Jacob Carlborg wrote:
On 2012-11-01 23:51, Walter Bright wrote:

What about all your feature requests? I think you've made more than
anyone, by a factor of 10 at least!

:-)

As for Manu's request

http://d.puremagic.com/issues/show_bug.cgi?id=8108

I've gone over with him why he needs it, and there's no other reasonable
way. He needs it for real code in a real application.

This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented.

He just knows how to convince Walter. (Hint: use real-world use cases. Arguments from theoretical computer science carry almost no weight with Walter).

I think there are lots of issues in the bug tracker which describe real use cases. The point is that these are more like bugs whereas Manu's proposals are more like feature requests. Jens

The SIMD stuff has no workarounds. I don't know of many other feature requests in that category.

Then I have a serious misunderstanding. I thought D introduced array operations to allow the compiler to generate efficient vector operations (in the long run), i.e. generate SIMD code. Why is this not working out? Jens
Nov 02 2012
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Friday, 2 November 2012 at 10:24:34 UTC, Jens Mueller wrote:
 Then I have a serious misunderstanding.
 I thought D introduced array operations to allow the compiler to
 generate efficient vector operations (in the long run), i.e. 
 generate
 SIMD code. Why is this not working out?

It works fine for large vectors. For small vectors, it is horrendously slow. The syntax a[] += b[] essentially calls a function which is designed to work for large vectors. It has to determine alignment, load everything from memory, do the operations, then store it back. The SIMD extensions allow you to define variables that are guaranteed to be aligned and will probably be in the right registers to start with. Using them, the vectors ops don't need to determine alignment, and often don't need to do lots of loads/stores. Both have their purposes.
Nov 02 2012
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Don Clugston wrote:
 On 02/11/12 10:01, Jens Mueller wrote:
Jacob Carlborg wrote:
On 2012-11-01 23:51, Walter Bright wrote:

What about all your feature requests? I think you've made more than
anyone, by a factor of 10 at least!

:-)

As for Manu's request

http://d.puremagic.com/issues/show_bug.cgi?id=8108

I've gone over with him why he needs it, and there's no other reasonable
way. He needs it for real code in a real application.

This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented. Although I think something like AST macros could possible solve issue 8108 and a whole bunch of other features, a few already present in the language.

I had the same thought when reading this. Very disappointing. An issue with zero votes is fixed instead of more important ones. Why do I vote anyway? Regarding SIMD I have the feeling that because it is built into the compiler static vectors have actually failed what they promised. I thought D proposed a portable way of vector operations such that you write float[4] = a[] + b[] and the compiler generates SIMD code for you.

Not for short vectors. They are more like the builtin operations in Fortran, ie designed for large vectors. More for scientific kinds of applications than games. (The two applications look superficially similar, but in practice they have little in common).

Okay. For me they look the same. Can you elaborate, please? Assume I want to add two float vectors which is common in both games and scientific computing. The only difference is in games their length is usually 3 or 4 whereas in scientific computing they are of arbitrary length. Why do I need instrinsics to support the game setting? Jens
Nov 02 2012
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Peter Alexander wrote:
 On Friday, 2 November 2012 at 10:24:34 UTC, Jens Mueller wrote:
Then I have a serious misunderstanding.
I thought D introduced array operations to allow the compiler to
generate efficient vector operations (in the long run), i.e.
generate
SIMD code. Why is this not working out?

It works fine for large vectors. For small vectors, it is horrendously slow. The syntax a[] += b[] essentially calls a function which is designed to work for large vectors. It has to determine alignment, load everything from memory, do the operations, then store it back. The SIMD extensions allow you to define variables that are guaranteed to be aligned and will probably be in the right registers to start with. Using them, the vectors ops don't need to determine alignment, and often don't need to do lots of loads/stores. Both have their purposes.

I see. But can't the alignment problem be solved by using align. Then have the compiler emits a call that checks for alignment if none was specified else use a faster version. Jens
Nov 02 2012
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Friday, 2 November 2012 at 10:50:56 UTC, Jens Mueller wrote:
 Don Clugston wrote:
 On 02/11/12 10:01, Jens Mueller wrote:
Jacob Carlborg wrote:
On 2012-11-01 23:51, Walter Bright wrote:

What about all your feature requests? I think you've made 
more than
anyone, by a factor of 10 at least!

:-)

As for Manu's request

http://d.puremagic.com/issues/show_bug.cgi?id=8108

I've gone over with him why he needs it, and there's no 
other reasonable
way. He needs it for real code in a real application.

This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented. Although I think something like AST macros could possible solve issue 8108 and a whole bunch of other features, a few already present in the language.

I had the same thought when reading this. Very disappointing. An issue with zero votes is fixed instead of more important ones. Why do I vote anyway? Regarding SIMD I have the feeling that because it is built into the compiler static vectors have actually failed what they promised. I thought D proposed a portable way of vector operations such that you write float[4] = a[] + b[] and the compiler generates SIMD code for you.

Not for short vectors. They are more like the builtin operations in Fortran, ie designed for large vectors. More for scientific kinds of applications than games. (The two applications look superficially similar, but in practice they have little in common).

Okay. For me they look the same. Can you elaborate, please? Assume I want to add two float vectors which is common in both games and scientific computing. The only difference is in games their length is usually 3 or 4 whereas in scientific computing they are of arbitrary length. Why do I need instrinsics to support the game setting? Jens

The auto vectorization code in Visual Studio 2012 seems to work pretty well.
Nov 02 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 11/2/12, Walter Bright <newshound2 digitalmars.com> wrote:
 Manu works for Remedy Games, a developer of hit PC games, such as Max Payne
 and
 Alan Wake. He champions a team that is investigating committing to D for a
 major
 new game.

Wow that's really cool! Manu if you know of any trivial-type bugs you'd like to get fixed (for example error messages) let me know so I can prioritize them. There's a ton of open issues and I usually select the ones for fixing by random. I've managed to fix a few so far. :)
Nov 02 2012
prev sibling next sibling parent "Dejan Lekic" <dejan.lekic gmail.com> writes:
On Thursday, 1 November 2012 at 18:06:21 UTC, Peter Alexander 
wrote:
 On Wednesday, 31 October 2012 at 23:04:15 UTC, bearophile wrote:
 The thread contains some sad comments:

It's unfortunate that there's still bad press circulating about a situation that is long gone. I suppose you just have to try and ignore those people. A more interesting comment is this one: "But the real problem here is that in order to achieve even that, the complexity and amount of concepts you have to deal with in C++11 is mind boggling." The same is true in D. Well-written D code often does look rather elegant, but the amount of understanding needed to write beautiful D code is staggering.

I think that is the case of any multi-paradigm language. It is already enough to allow both functional and OO style of programming to have hundreds of things to think about... That is exactly one of the major reasons why people prefer Java over other languages...
Nov 02 2012
prev sibling next sibling parent "Dejan Lekic" <dejan.lekic gmail.com> writes:
On Friday, 2 November 2012 at 08:38:21 UTC, Don Clugston wrote:
 On 02/11/12 09:07, Jacob Carlborg wrote:
 On 2012-11-01 23:51, Walter Bright wrote:

 What about all your feature requests? I think you've made 
 more than
 anyone, by a factor of 10 at least!

 :-)

 As for Manu's request

 http://d.puremagic.com/issues/show_bug.cgi?id=8108

 I've gone over with him why he needs it, and there's no other 
 reasonable
 way. He needs it for real code in a real application.

This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented.

He just knows how to convince Walter. (Hint: use real-world use cases. Arguments from theoretical computer science carry almost no weight with Walter).
 Although I think something like
 AST macros could possible solve issue 8108 and a whole bunch 
 of other
 features, a few already present in the language.

Yes, and they could cure cancer. "AST macros" can do anything, because they are completely undefined. Without even a vague proposal, it seems like a rather meaningless term.

How do you think people came up with those bug reports? By some magic? :) You think they did not actually *pull their hair off* trying to figure out why their REAL LIFE program does not work, and when we could not find the reason we naturally started thinking "oh, it might be yet another missing thing in D" or "oh, yet another DMD/phobos/druntime bug!"... Or, another scenario (a very typical one) - after DMD/phobos/druntime is updated, my production application no longer compiles...
Nov 02 2012
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Fri, 02 Nov 2012 18:10:39 -0000, deadalnix <deadalnix gmail.com> wrot=
e:

 Le 02/11/2012 18:46, Andrei Alexandrescu a =E9crit :
 On 11/2/12 1:22 PM, Walter Bright wrote:
 On 11/2/2012 2:01 AM, Jens Mueller wrote:
 I had the same thought when reading this. Very disappointing. An  =




 issue
 with zero votes is fixed instead of more important ones.

It's a very fair question. Manu works for Remedy Games, a developer of hit PC games, such as Ma=



 Payne and Alan Wake. He champions a team that is investigating
 committing to D for a major new game. This would be a huge design wi=



 for D, and an enormous boost for D. I view the most effective use of=



 time at the moment is to ensure that they can bet on D and win.  =



 Remedy's
 use of D for a high profile product will prove that D is ready and a=



 for the big time, and that others can develop using D with confidenc=



 Nearly all of what he needs the D community needs anyway, such as th=



 big push for Win64 support and the SIMD support (more on that in  =



 another
 post).

I should add that I'm also totally behind this. When Walter jumped in=


 implementing SIMD support on a hunch, I completely disagreed, but tha=


 was a great decision.

 Andrei

I still don't understand the benefice over align(16) float[4] .

http://forum.dlang.org/thread/jauixhakwvpgsghapzvz forum.dlang.org?page=3D= 4#post-k711rd:242786:242:40digitalmars.com R -- = Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 02 2012
prev sibling next sibling parent "mist" <none none.none> writes:
On Friday, 2 November 2012 at 18:19:54 UTC, Jacob Carlborg wrote:
 On 2012-11-02 18:46, Andrei Alexandrescu wrote:

 I should add that I'm also totally behind this. When Walter 
 jumped into
 implementing SIMD support on a hunch, I completely disagreed, 
 but that
 was a great decision.

I can absolutely understand why he did it but it would be really nice if you (Walter, Andrei and probably others as well) could be more transparent about things like these. I think this would really help the community.

So true. Strong mid-term vision of rationales of main language developers really helps to reason about it.
Nov 02 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--bcaec5014c71f17edc04cd87bffa
Content-Type: text/plain; charset=UTF-8

Uh oh, I just caught wind of this thread! ;)

If there's a single argument I'd like to make in defense of the changes
that have gone through as a result of my motivating, it's that they're
either big tickets and mutually beneficial for the whole community anyway
(DMD-win64), or relatively trivial (function prototypes + definitions)
details that enable us to realistically consider proceeding with D
commercially.

The SIMD types support may be considered a little grey, but I've
encountered loads of general D users who appreciate the SIMD work already,
and while it's certainly not trivial, it's nowhere the mammoth scale of
task auto-vectorisation would be. I can't see it as a loss for the language
or the community, and again, it has further enabled our consideration of D
commercially; which I'd like to think is a general goal for the language.

I know plenty of you don't care about me or my industry, but I maintain
that it's an entire industry in desperate need of salvation from C++,
there's a lot of potential in the games industry to get fantastic value
from using D, and I'm interested in proving that it's a realistic
consideration.

I'd also like to note that we are also very conscious of the time Walter
and other contributors have kindly offered to our support, and we are
hopeful to be able to give reasonable value back to the community should it
all go well for us, in whatever way that manifests...

On 2 November 2012 11:01, Jens Mueller <jens.k.mueller gmx.de> wrote:

 Jacob Carlborg wrote:
 On 2012-11-01 23:51, Walter Bright wrote:

What about all your feature requests? I think you've made more than
anyone, by a factor of 10 at least!

:-)

As for Manu's request

http://d.puremagic.com/issues/show_bug.cgi?id=8108

I've gone over with him why he needs it, and there's no other reasonable
way. He needs it for real code in a real application.

This is quite interesting. Manu comes in from basically nowhere and fairly quickly manage to convince Walter to implement at least two feature requests, this one and the SIMD support. I'm not saying that they shouldn't have been implemented. Although I think something like AST macros could possible solve issue 8108 and a whole bunch of other features, a few already present in the language.

I had the same thought when reading this. Very disappointing. An issue with zero votes is fixed instead of more important ones. Why do I vote anyway? Regarding SIMD I have the feeling that because it is built into the compiler static vectors have actually failed what they promised. I thought D proposed a portable way of vector operations such that you write float[4] = a[] + b[] and the compiler generates SIMD code for you.

As I mentioned above, I think that is a MUCH larger task, and probably fairly unrealistic in the near-term regardless. But secondly, it's just not that simple. (I can see Walter has already addressed it in a previous post) It's worth considering that a very significant portion of the silicon on modern processors (well in excess of 50% on some chips) is dedicated to hardware acceleration of SIMD/media functions. Until recently, D simply offered no mechanism at all to interact with half of the silicone in your box. That's a massive language hole. At least as a starting point, low level access to this hardware is vital. Portable libraries can be built using this technology, which are immediately useful. The definition of hardware SIMD types doesn't rule out possible future auto-vectorisation either. And you can probably access auto-vectorising backends right now if you use GDC or LDC. --bcaec5014c71f17edc04cd87bffa Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div>Uh oh, I just caught wind of this thread! ;)<div><br></div><div>If the= re&#39;s a single argument I&#39;d like to make in defense of the changes t= hat have gone through as a result of my motivating, it&#39;s that they&#39;= re either big tickets and mutually beneficial for the whole community anywa= y (DMD-win64), or relatively trivial (function prototypes + definitions) de= tails that enable us to realistically consider proceeding with D commercial= ly.</div> <div><br></div><div>The SIMD types support may be considered a little grey,= but I&#39;ve encountered loads of general D users who appreciate the SIMD = work already, and while it&#39;s certainly not trivial, it&#39;s nowhere th= e=C2=A0mammoth=C2=A0scale of task auto-vectorisation would be. I can&#39;t = see it as a loss for the language or the community, and again, it has furth= er enabled our consideration of D commercially; which I&#39;d like to think= is a general goal for the language.</div> <div><br></div><div>I know plenty of you don&#39;t care about me or my indu= stry, but I maintain that it&#39;s an entire industry in desperate need of = salvation from C++, there&#39;s a lot of potential in the games industry to= get fantastic value from using D, and I&#39;m interested in proving that i= t&#39;s a realistic consideration.</div> <div><br></div><div>I&#39;d also like to note that we are also very conscio= us of the time Walter and other contributors have kindly offered to our sup= port, and we are hopeful to be able to give reasonable value back to the co= mmunity should it all go well for us, in whatever way that manifests...</di= v> </div><div><br></div>On 2 November 2012 11:01, Jens Mueller <span dir=3D"lt= r">&lt;<a href=3D"mailto:jens.k.mueller gmx.de" target=3D"_blank">jens.k.mu= eller gmx.de</a>&gt;</span> wrote:<br><div class=3D"gmail_quote"><blockquot= e class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc sol= id;padding-left:1ex"> <div class=3D"HOEnZb"><div class=3D"h5">Jacob Carlborg wrote:<br> &gt; On 2012-11-01 23:51, Walter Bright wrote:<br> &gt;<br> &gt; &gt;What about all your feature requests? I think you&#39;ve made more= than<br> &gt; &gt;anyone, by a factor of 10 at least!<br> &gt; &gt;<br> &gt; &gt;:-)<br> &gt; &gt;<br> &gt; &gt;As for Manu&#39;s request<br> &gt; &gt;<br> &gt; &gt;<a href=3D"http://d.puremagic.com/issues/show_bug.cgi?id=3D8108" t= arget=3D"_blank">http://d.puremagic.com/issues/show_bug.cgi?id=3D8108</a><b= r> &gt; &gt;<br> &gt; &gt;I&#39;ve gone over with him why he needs it, and there&#39;s no ot= her reasonable<br> &gt; &gt;way. He needs it for real code in a real application.<br> &gt;<br> &gt; This is quite interesting. Manu comes in from basically nowhere and<br=

&gt; feature requests, this one and the SIMD support. I&#39;m not saying th= at<br> &gt; they shouldn&#39;t have been implemented. Although I think something<b= r> &gt; like AST macros could possible solve issue 8108 and a whole bunch of<b= r> &gt; other features, a few already present in the language.<br> <br> </div></div>I had the same thought when reading this. Very disappointing. A= n issue<br> with zero votes is fixed instead of more important ones. Why do I vote<br> anyway?<br> Regarding SIMD I have the feeling that because it is built into the<br> compiler static vectors have actually failed what they promised. I<br> thought D proposed a portable way of vector operations such that you<br> write<br> float[4] =3D a[] + b[]<br> and the compiler generates SIMD code for you.<br></blockquote><div><br></di= v><div>As I mentioned above, I think that is a MUCH larger task, and probab= ly fairly unrealistic in the near-term regardless.</div><div>But secondly, = it&#39;s just not that simple. (I can see Walter has already addressed it i= n a previous post)</div> <div><br></div><div>It&#39;s worth considering that a very significant port= ion of the silicon on modern processors (well in excess of 50% on some chip= s) is dedicated to hardware acceleration of SIMD/media functions. Until rec= ently, D simply offered no mechanism at all to interact with half of the si= licone in your box. That&#39;s a massive language hole.</div> <div>At least as a starting point, low level access to this hardware is vit= al. Portable libraries can be built using this technology, which are immedi= ately useful.</div><div><br></div><div>The definition of hardware SIMD type= s doesn&#39;t rule out possible future auto-vectorisation either. And you c= an probably access auto-vectorising backends right now if you use GDC or LD= C.</div> </div> --bcaec5014c71f17edc04cd87bffa--
Nov 02 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--047d7b67002d86b91804cd87cc87
Content-Type: text/plain; charset=UTF-8

On 2 November 2012 19:32, Andrej Mitrovic <andrej.mitrovich gmail.com>wrote:

 On 11/2/12, Walter Bright <newshound2 digitalmars.com> wrote:
 Manu works for Remedy Games, a developer of hit PC games, such as Max

 and
 Alan Wake. He champions a team that is investigating committing to D for

 major
 new game.

Wow that's really cool! Manu if you know of any trivial-type bugs you'd like to get fixed (for example error messages) let me know so I can prioritize them. There's a ton of open issues and I usually select the ones for fixing by random. I've managed to fix a few so far. :)

Hey cheers man! :) Actually, one thing that does consistently bite me are nonsensical error messages. I'll start keeping tabs on them. --047d7b67002d86b91804cd87cc87 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 2 November 2012 19:32, Andrej Mitrovic <span dir=3D"ltr">&lt;<a href=3D"= mailto:andrej.mitrovich gmail.com" target=3D"_blank">andrej.mitrovich gmail= .com</a>&gt;</span> wrote:<br><div class=3D"gmail_quote"><blockquote class= =3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd= ing-left:1ex"> <div class=3D"im">On 11/2/12, Walter Bright &lt;<a href=3D"mailto:newshound= 2 digitalmars.com">newshound2 digitalmars.com</a>&gt; wrote:<br> &gt; Manu works for Remedy Games, a developer of hit PC games, such as Max = Payne<br> &gt; and<br> &gt; Alan Wake. He champions a team that is investigating committing to D f= or a<br> &gt; major<br> &gt; new game.<br> <br> </div>Wow that&#39;s really cool!<br> <br> Manu if you know of any trivial-type bugs you&#39;d like to get fixed (for<= br> example error messages) let me know so I can prioritize them. There&#39;s<b= r> a ton of open issues and I usually select the ones for fixing by<br> random. I&#39;ve managed to fix a few so far. :)<br> </blockquote></div><br><div>Hey cheers man! :)</div><div>Actually, one thin= g that does consistently bite me are nonsensical error messages. I&#39;ll s= tart keeping tabs on them.</div> --047d7b67002d86b91804cd87cc87--
Nov 02 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--20cf307f3a9e0a792404cd87fba0
Content-Type: text/plain; charset=UTF-8

On 2 November 2012 20:02, Walter Bright <newshound2 digitalmars.com> wrote:

 On 11/2/2012 3:50 AM, Jens Mueller wrote:
 Okay. For me they look the same. Can you elaborate, please? Assume I
 want to add two float vectors which is common in both games and
 scientific computing. The only difference is in games their length is
 usually 3 or 4 whereas in scientific computing they are of arbitrary
 length. Why do I need instrinsics to support the game setting?

Another excellent question. Most languages have taken the "auto-vectorization" approach of reverse engineering loops to turn them into high level constructs, and then compiling the code into special SIMD instructions. How to do this is explained in detail in the (rare) book "The Software Vectorization Handbook" by Bik, which I fortunately was able to obtain a copy of. This struck me as a terrible approach, however. It just seemed stupid to try to teach the compiler to reverse engineer low level code into high level code. A better design would be to start with high level code. Hence, the appearance of D vector operations. The trouble with D vector operations, however, is that they are too general purpose. The SIMD instructions are very quirky, and it's easy to unwittingly and silently cause the compiler to generate absolutely terribly slow code. The reasons for that are the alignment requirements, coupled with the SIMD instructions not being orthogonal - some operations work for some types and not for others, in a way that is unintuitive unless you're carefully reading the SIMD specs. Just saying align(16) isn't good enough, as the vector ops work on slices and those slices aren't always aligned. So each one has to check alignment at runtime, which is murder on performance. If a particular vector op for a particular type has no SIMD support, then the compiler has to generate workaround code. This can also have terrible performance consequences. So the user writes vector code, benchmarks it, finds zero improvement, and the reasons why will be elusive to anyone but an expert SIMD programmer. (Auto-vectorizing technology has similar issues, pretty much meaning you won't get fast code out of it unless you've got a habit of examining the assembler output and tweaking as necessary.) Enter Manu, who has a lot of experience making SIMD work for games. His proposal was: 1. Have native SIMD types. This will guarantee alignment, and will guarantee a compile time error for SIMD types that are not supported by the CPU. 2. Have the compiler issue an error for SIMD operations that are not supported by the CPU, rather than silently generating inefficient workaround code. 3. There are all kinds of weird but highly useful SIMD instructions that don't have a straightforward representation in high level code, such as saturated arithmetic. Manu's answer was to expose these instructions via intrinsics, so the user can string them together, be sure that they will generate real SIMD instructions, while the compiler can deal with register allocation.

Well, I wouldn't claim any credit for the approach ;) .. I think this is the standard for maximum performance, and also very well understood. But the thing that excites me most is the potential quality of libraries that can be built on top. D has so much potential to extend on this SIMD foundation with it's templates being able to intelligently handle far more context specific situations. What we do already in other languages will be far more convenient, more portable, and possibly even produce better code in D. And the biggest bonus, it will be readable! :) I think it's a low risk investment, and it doesn't prohibit higher level support in the future. This approach works, is inlineable, generates code as good as hand-built
 assembler, and is useable by regular programmers.

 I won't say there aren't better approaches, but this one we know works.

Aye, and it's relatively un-intrusive too. Some new types and a few intrinsics, build useful libraries on top. It shouldn't have complex side effects, and if offers something that was sorely missing from the language today. --20cf307f3a9e0a792404cd87fba0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 2 November 2012 20:02, Walter Bright <span dir=3D"ltr">&lt;<a href=3D"ma= ilto:newshound2 digitalmars.com" target=3D"_blank">newshound2 digitalmars.c= om</a>&gt;</span> wrote:<br><div class=3D"gmail_quote"><blockquote class=3D= "gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding= -left:1ex"> <div class=3D"im">On 11/2/2012 3:50 AM, Jens Mueller wrote:<br> &gt; Okay. For me they look the same. Can you elaborate, please? Assume I<b= r> &gt; want to add two float vectors which is common in both games and<br> &gt; scientific computing. The only difference is in games their length is<= br> &gt; usually 3 or 4 whereas in scientific computing they are of arbitrary<b= r> &gt; length. Why do I need instrinsics to support the game setting?<br> <br></div> Another excellent question.<br> <br> Most languages have taken the &quot;auto-vectorization&quot; approach of re= verse engineering loops to turn them into high level constructs, and then c= ompiling the code into special SIMD instructions.<br> <br> How to do this is explained in detail in the (rare) book &quot;The Software= Vectorization Handbook&quot; by Bik, which I fortunately was able to obtai= n a copy of.<br> <br> This struck me as a terrible approach, however. It just seemed stupid to tr= y to teach the compiler to reverse engineer low level code into high level = code. A better design would be to start with high level code. Hence, the ap= pearance of D vector operations.<br> <br> The trouble with D vector operations, however, is that they are too general= purpose. The SIMD instructions are very quirky, and it&#39;s easy to unwit= tingly and silently cause the compiler to generate absolutely terribly slow= code. The reasons for that are the alignment requirements, coupled with th= e SIMD instructions not being orthogonal - some operations work for some ty= pes and not for others, in a way that is unintuitive unless you&#39;re care= fully reading the SIMD specs.<br> <br> Just saying align(16) isn&#39;t good enough, as the vector ops work on slic= es and those slices aren&#39;t always aligned. So each one has to check ali= gnment at runtime, which is murder on performance.<br> <br> If a particular vector op for a particular type has no SIMD support, then t= he compiler has to generate workaround code. This can also have terrible pe= rformance consequences.<br> <br> So the user writes vector code, benchmarks it, finds zero improvement, and = the reasons why will be elusive to anyone but an expert SIMD programmer.<br=

(Auto-vectorizing technology has similar issues, pretty much meaning you wo= n&#39;t get fast code out of it unless you&#39;ve got a habit of examining = the assembler output and tweaking as necessary.)<br> <br> Enter Manu, who has a lot of experience making SIMD work for games. His pro= posal was:<br> <br> 1. Have native SIMD types. This will guarantee alignment, and will guarante= e a compile time error for SIMD types that are not supported by the CPU.<br=

2. Have the compiler issue an error for SIMD operations that are not suppor= ted by the CPU, rather than silently generating inefficient workaround code= .<br> <br> 3. There are all kinds of weird but highly useful SIMD instructions that do= n&#39;t have a straightforward representation in high level code, such as s= aturated arithmetic. Manu&#39;s answer was to expose these instructions via= intrinsics, so the user can string them together, be sure that they will g= enerate real SIMD instructions, while the compiler can deal with register a= llocation.<br> </blockquote><div><br></div><div>Well, I wouldn&#39;t claim any credit for = the approach ;) .. I think this is the standard for maximum performance, an= d also very well understood.</div><div>But the thing that excites me most i= s the potential quality of libraries that can be built on top. D has so muc= h potential to extend on this SIMD foundation with it&#39;s templates being= able to intelligently handle far more context specific situations.</div> <div>What we do already in other languages will be far more convenient, mor= e portable, and possibly even produce better code in D. And the biggest bon= us, it will be readable! :)</div><div><br></div><div>I think it&#39;s a low= risk investment, and it doesn&#39;t prohibit higher level support in the f= uture.</div> <div><br></div><div><br></div><blockquote class=3D"gmail_quote" style=3D"ma= rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">This approach = works, is inlineable, generates code as good as hand-built assembler, and i= s useable by regular programmers.<br> <br> I won&#39;t say there aren&#39;t better approaches, but this one we know wo= rks.<br></blockquote><div><br></div><div>Aye, and it&#39;s relatively un-in= trusive too. Some new types and a few intrinsics, build useful libraries on= top. It shouldn&#39;t have complex side effects, and if offers something t= hat was sorely missing from the language today.</div> </div> --20cf307f3a9e0a792404cd87fba0--
Nov 02 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--20cf307811d0cc4c6e04cd88071f
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 2 November 2012 20:10, deadalnix <deadalnix gmail.com> wrote:

 Le 02/11/2012 18:46, Andrei Alexandrescu a =C3=A9crit :

  On 11/2/12 1:22 PM, Walter Bright wrote:
 On 11/2/2012 2:01 AM, Jens Mueller wrote:
 I had the same thought when reading this. Very disappointing. An issu=




 with zero votes is fixed instead of more important ones.

It's a very fair question. Manu works for Remedy Games, a developer of hit PC games, such as Max Payne and Alan Wake. He champions a team that is investigating committing to D for a major new game. This would be a huge design win for D, and an enormous boost for D. I view the most effective use of my time at the moment is to ensure that they can bet on D and win. Remedy'=



 use of D for a high profile product will prove that D is ready and able
 for the big time, and that others can develop using D with confidence.

 Nearly all of what he needs the D community needs anyway, such as the
 big push for Win64 support and the SIMD support (more on that in anothe=



 post).

I should add that I'm also totally behind this. When Walter jumped into implementing SIMD support on a hunch, I completely disagreed, but that was a great decision. Andrei

I still don't understand the benefice over align(16) float[4] .

It's a mechanism to insist on register placement and usage semantics. On the vast majority of architectures, floats and simd are absolutely incompatible. It's awfully dangerous to describe them both with a single type when they are mutually exclusive... --20cf307811d0cc4c6e04cd88071f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 2 November 2012 20:10, deadalnix <span dir=3D"ltr">&lt;<a href=3D"mailto= :deadalnix gmail.com" target=3D"_blank">deadalnix gmail.com</a>&gt;</span> = wrote:<br><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" styl= e=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Le 02/11/2012 18:46, Andrei Alexandrescu a =C3=A9crit :<div><div class=3D"h= 5"><br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On 11/2/12 1:22 PM, Walter Bright wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On 11/2/2012 2:01 AM, Jens Mueller wrote:<br> &gt; I had the same thought when reading this. Very disappointing. An issue= <br> &gt; with zero votes is fixed instead of more important ones.<br> <br> It&#39;s a very fair question.<br> <br> Manu works for Remedy Games, a developer of hit PC games, such as Max<br> Payne and Alan Wake. He champions a team that is investigating<br> committing to D for a major new game. This would be a huge design win<br> for D, and an enormous boost for D. I view the most effective use of my<br> time at the moment is to ensure that they can bet on D and win. Remedy&#39;= s<br> use of D for a high profile product will prove that D is ready and able<br> for the big time, and that others can develop using D with confidence.<br> <br> Nearly all of what he needs the D community needs anyway, such as the<br> big push for Win64 support and the SIMD support (more on that in another<br=

</blockquote> <br> I should add that I&#39;m also totally behind this. When Walter jumped into= <br> implementing SIMD support on a hunch, I completely disagreed, but that<br> was a great decision.<br> <br> Andrei<br> </blockquote> <br></div></div> I still don&#39;t understand the benefice over align(16) float[4] .<br> </blockquote></div><br><div>It&#39;s a mechanism to insist on register plac= ement and usage semantics. On the vast majority of architectures, floats an= d simd are absolutely incompatible. It&#39;s awfully dangerous to describe = them both with a single type when they are mutually exclusive...</div> --20cf307811d0cc4c6e04cd88071f--
Nov 02 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--20cf307f3a9ee5035904cd880df4
Content-Type: text/plain; charset=UTF-8

On 2 November 2012 20:19, Jacob Carlborg <doob me.com> wrote:

 On 2012-11-02 18:46, Andrei Alexandrescu wrote:

  I should add that I'm also totally behind this. When Walter jumped into
 implementing SIMD support on a hunch, I completely disagreed, but that
 was a great decision.

I can absolutely understand why he did it but it would be really nice if you (Walter, Andrei and probably others as well) could be more transparent about things like these. I think this would really help the community.

This is probably my fault, and a matter of corporate transparency. We didn't want to make a noise about it until we reached a particular level of confidence. We're fairly invested now, and quietly hopeful it will go ahead from here. --20cf307f3a9ee5035904cd880df4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 2 November 2012 20:19, Jacob Carlborg <span dir=3D"ltr">&lt;<a href=3D"m= ailto:doob me.com" target=3D"_blank">doob me.com</a>&gt;</span> wrote:<br><= div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin= :0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div class=3D"im">On 2012-11-02 18:46, Andrei Alexandrescu wrote:<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I should add that I&#39;m also totally behind this. When Walter jumped into= <br> implementing SIMD support on a hunch, I completely disagreed, but that<br> was a great decision.<br> </blockquote> <br></div> I can absolutely understand why he did it but it would be really nice if yo= u (Walter, Andrei and probably others as well) could be more transparent ab= out things like these. I think this would really help the community.</block= quote> <div><br></div><div>This is probably my fault, and a matter of corporate tr= ansparency. We didn&#39;t want to make a noise about it until we reached a = particular level of confidence.</div><div>We&#39;re fairly invested now, an= d quietly hopeful it will go ahead from here.</div> </div> --20cf307f3a9ee5035904cd880df4--
Nov 02 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--20cf307811d0c1365e04cd88218a
Content-Type: text/plain; charset=UTF-8

On 2 November 2012 20:36, Walter Bright <newshound2 digitalmars.com> wrote:

 On 11/2/2012 11:19 AM, Jacob Carlborg wrote:

 I can absolutely understand why he did it but it would be really nice if
 you
 (Walter, Andrei and probably others as well) could be more transparent
 about
 things like these. I think this would really help the community.

I apologize for being circumspect about this, but I have to respect peoples' privacy and I cleared what I posted about Remedy with them before posting it.

That said, I think we'd perhaps appreciate that it doesn't appear all over the internets just yet. It would be much more interesting, and probably have a lot more impact if we made such an announcement alongside something to show. --20cf307811d0c1365e04cd88218a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 2 November 2012 20:36, Walter Bright <span dir=3D"ltr">&lt;<a href=3D"ma= ilto:newshound2 digitalmars.com" target=3D"_blank">newshound2 digitalmars.c= om</a>&gt;</span> wrote:<br><div class=3D"gmail_quote"><blockquote class=3D= "gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding= -left:1ex"> <div class=3D"im">On 11/2/2012 11:19 AM, Jacob Carlborg wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I can absolutely understand why he did it but it would be really nice if yo= u<br> (Walter, Andrei and probably others as well) could be more transparent abou= t<br> things like these. I think this would really help the community.<br> </blockquote> <br></div> I apologize for being circumspect about this, but I have to respect peoples= &#39; privacy and I cleared what I posted about Remedy with them before pos= ting it.<br></blockquote><div><br></div><div>That said, I think we&#39;d pe= rhaps appreciate that it doesn&#39;t appear all over the internets just yet= . It would be much more interesting, and probably have a lot more impact if= we made such an announcement alongside something to show.</div> </div> --20cf307811d0c1365e04cd88218a--
Nov 02 2012
prev sibling next sibling parent "jerro" <a a.com> writes:
 "After a throughout discussion with Manu (or 'a fellow D 
 programmer' if he/she prefers to be anonymous) we have decided 
 it would be in best interest of D if we implement this 
 particular feature. From now on this is where I will focus most 
 most of my time".

 Something like this would be far better then suddenly seeing 
 commits regarding SIMD (or whatever feature it might be) for 
 out of the blue.

SIMD support was discussed here at length in the days before it was implemented. See this thread: http://forum.dlang.org/thread/jdhb57$10vf$1 digitalmars.com#post-wdjdcrkiaakmkzqtdhxu:40dfeed.kimsufi.thecybershadow.net and this thread: http://forum.dlang.org/post/mailman.76.1325814175.16222.digitalmars-d puremagic.com
Nov 02 2012
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Walter Bright wrote:
 On 11/2/2012 3:50 AM, Jens Mueller wrote:
 Okay. For me they look the same. Can you elaborate, please? Assume I
 want to add two float vectors which is common in both games and
 scientific computing. The only difference is in games their length is
 usually 3 or 4 whereas in scientific computing they are of arbitrary
 length. Why do I need instrinsics to support the game setting?

Another excellent question. Most languages have taken the "auto-vectorization" approach of reverse engineering loops to turn them into high level constructs, and then compiling the code into special SIMD instructions. How to do this is explained in detail in the (rare) book "The Software Vectorization Handbook" by Bik, which I fortunately was able to obtain a copy of. This struck me as a terrible approach, however. It just seemed stupid to try to teach the compiler to reverse engineer low level code into high level code. A better design would be to start with high level code. Hence, the appearance of D vector operations. The trouble with D vector operations, however, is that they are too general purpose. The SIMD instructions are very quirky, and it's easy to unwittingly and silently cause the compiler to generate absolutely terribly slow code. The reasons for that are the alignment requirements, coupled with the SIMD instructions not being orthogonal - some operations work for some types and not for others, in a way that is unintuitive unless you're carefully reading the SIMD specs. Just saying align(16) isn't good enough, as the vector ops work on slices and those slices aren't always aligned. So each one has to check alignment at runtime, which is murder on performance. If a particular vector op for a particular type has no SIMD support, then the compiler has to generate workaround code. This can also have terrible performance consequences. So the user writes vector code, benchmarks it, finds zero improvement, and the reasons why will be elusive to anyone but an expert SIMD programmer. (Auto-vectorizing technology has similar issues, pretty much meaning you won't get fast code out of it unless you've got a habit of examining the assembler output and tweaking as necessary.) Enter Manu, who has a lot of experience making SIMD work for games. His proposal was: 1. Have native SIMD types. This will guarantee alignment, and will guarantee a compile time error for SIMD types that are not supported by the CPU. 2. Have the compiler issue an error for SIMD operations that are not supported by the CPU, rather than silently generating inefficient workaround code. 3. There are all kinds of weird but highly useful SIMD instructions that don't have a straightforward representation in high level code, such as saturated arithmetic. Manu's answer was to expose these instructions via intrinsics, so the user can string them together, be sure that they will generate real SIMD instructions, while the compiler can deal with register allocation. This approach works, is inlineable, generates code as good as hand-built assembler, and is useable by regular programmers. I won't say there aren't better approaches, but this one we know works.

I see. Thanks for clarifying. If I want fast vector operations I have to use core.simd. The built-in vector operations won't fit the bill. I was of the opinion that a vector operation in D should (at some point) generate vectorized code. Jens
Nov 02 2012
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Walter Bright wrote:
 On 11/2/2012 2:01 AM, Jens Mueller wrote:
 I had the same thought when reading this. Very disappointing. An issue
 with zero votes is fixed instead of more important ones.

It's a very fair question. Manu works for Remedy Games, a developer of hit PC games, such as Max Payne and Alan Wake. He champions a team that is investigating committing to D for a major new game. This would be a huge design win for D, and an enormous boost for D. I view the most effective use of my time at the moment is to ensure that they can bet on D and win. Remedy's use of D for a high profile product will prove that D is ready and able for the big time, and that others can develop using D with confidence. Nearly all of what he needs the D community needs anyway, such as the big push for Win64 support and the SIMD support (more on that in another post).

Now I can understand why you made these decisions. It makes sense to me. But I'd like to know which direction the ship is sailing. The current rationale is to proof that D is ready for prime time. Though I fear that some issues in the bug tracker may give a bad impression. But in general the decision process should be as open and transparent as possible. In Jacob's words: We need a road map. Jens
Nov 02 2012
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Andrei Alexandrescu wrote:
 On 11/2/12 1:22 PM, Walter Bright wrote:
On 11/2/2012 2:01 AM, Jens Mueller wrote:
 I had the same thought when reading this. Very disappointing. An issue
 with zero votes is fixed instead of more important ones.

It's a very fair question. Manu works for Remedy Games, a developer of hit PC games, such as Max Payne and Alan Wake. He champions a team that is investigating committing to D for a major new game. This would be a huge design win for D, and an enormous boost for D. I view the most effective use of my time at the moment is to ensure that they can bet on D and win. Remedy's use of D for a high profile product will prove that D is ready and able for the big time, and that others can develop using D with confidence. Nearly all of what he needs the D community needs anyway, such as the big push for Win64 support and the SIMD support (more on that in another post).

I should add that I'm also totally behind this. When Walter jumped into implementing SIMD support on a hunch, I completely disagreed, but that was a great decision.

From the inside you can tell it is a great decision. From the outside it

Jens
Nov 02 2012
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Walter Bright wrote:
 On 11/2/2012 11:19 AM, Jacob Carlborg wrote:
I can absolutely understand why he did it but it would be really nice if you
(Walter, Andrei and probably others as well) could be more transparent about
things like these. I think this would really help the community.

I apologize for being circumspect about this, but I have to respect peoples' privacy and I cleared what I posted about Remedy with them before posting it.

There is no need to reveal anything private just because you have a road map. Jens
Nov 02 2012
prev sibling parent Manu <turkeyman gmail.com> writes:
--bcaec51d25d407538404cd94f633
Content-Type: text/plain; charset=UTF-8

On 3 November 2012 01:41, Walter Bright <newshound2 digitalmars.com> wrote:

 On 11/2/2012 3:10 PM, Jens Mueller wrote:

 I see. Thanks for clarifying.
 If I want fast vector operations I have to use core.simd. The built-in
 vector operations won't fit the bill.


operations..."; this is not automatically faster by nature, it requires strict self-control in terms of application, and very careful attention if you want your code to be portable. At the moment, yes.
 However, Manu is working on developing a higher order layer.

I have a fork; some people are using it already. It still needs a lot of work though; some compilers missing parts, platforms not supported. That said, it's not an effort to address D's natural vector syntax, the key goal is to provide a hardware SIMD API that is as orthogonal as possible and portable (with confidence it will run reasonably well). I wonder if druntime could be enhanced to use the SIMD stuff though in the functions that perform the natural vector operations, might offer some nice little boosts. --bcaec51d25d407538404cd94f633 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_extra">On 3 November 2012 01:41, Walter Bright <span di= r=3D"ltr">&lt;<a href=3D"mailto:newshound2 digitalmars.com" target=3D"_blan= k">newshound2 digitalmars.com</a>&gt;</span> wrote:<br><div class=3D"gmail_= quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"><div class=3D"im">On 11/2/2012 3:10 PM, Jens Mueller wrote= :<br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"> I see. Thanks for clarifying.<br> If I want fast vector operations I have to use core.simd. The built-in<br> vector operations won&#39;t fit the bill.<br></blockquote></div></blockquot= e><div><br></div><div>I think a better quote would be &quot;If i want=C2=A0= <u>HARDWARE</u> vector operations...&quot;; this is not automatically faste= r by nature, it requires strict self-control in terms of application, and v= ery careful attention if you want your code to be portable.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-lef= t-style:solid;padding-left:1ex"> At the moment, yes.<br> <br> However, Manu is working on developing a higher order layer.<br></blockquot= e><div><br></div>I have a fork; some people are using it already. It still = needs a lot of work though; some compilers missing parts, platforms not sup= ported.<div> That said, it&#39;s not an effort to address D&#39;s natural vector syntax,= the key goal is to provide a hardware SIMD API that is as orthogonal as po= ssible and portable (with confidence it will run reasonably well).</div> <div>I wonder if druntime could be enhanced to use the SIMD stuff though in= the functions that perform the natural vector operations, might offer some= nice little boosts.</div></div></div> --bcaec51d25d407538404cd94f633--
Nov 03 2012