www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [OT] compiler optimisations

reply "John Colvin" <john.loughran.colvin gmail.com> writes:
Why can no compiler I try optimise this toy example as I would 
expect?

// uncomment if using a C compiler
// typedef unsigned int uint;
uint foo(uint a)
{
   if (a < 5)
     return (a * 3) / 3;
   else
     return 0;
}

So, I would expect the compiler to be able to see that it is 
equivalent to

uint foo(uint a)
{
   return (a < 5) ? a : 0;
}

But apparently not a single modern compiler I tried can do this 
optimisation, unless it's hidden in some obscure flag I'm not 
aware of.

An even more striking example can be found if you replace the / 
with %, where the result of the function is then unconditionally 
zero, but every compiler i tried still spat out multiplication 
instructions.

Is there a good reason for this, or is it just " * and / aren't 
always inverses, so never mind all the cases where they are"?

Now I know that this seems like a unrealistic example, but when 
you're in complicated meta-programming situations code like this 
can and will appear.
Apr 23 2015
next sibling parent reply "rumbu" <rumbu rumbu.ro> writes:
On Thursday, 23 April 2015 at 08:33:56 UTC, John Colvin wrote:
 Why can no compiler I try optimise this toy example as I would 
 expect?

 // uncomment if using a C compiler
 // typedef unsigned int uint;
 uint foo(uint a)
 {
   if (a < 5)
     return (a * 3) / 3;
   else
     return 0;
 }

 So, I would expect the compiler to be able to see that it is 
 equivalent to

 uint foo(uint a)
 {
   return (a < 5) ? a : 0;
 }

 But apparently not a single modern compiler I tried can do this 
 optimisation, unless it's hidden in some obscure flag I'm not 
 aware of.
I think because of the potential overflow in a * 3 (if we ignore the a < 5 condition). To optimize this, a compiler must figure out that there is no overflow for any a < 5. If you change the condition to a > 5 and call foo(uint.max), the two expression above are not equivalent. int foo(uint a) { if (a > 5) return (a * 3) / 3; else return 0; } int foo_optimized(uint a) { return (a > 5) ? a : 0; } assert(foo(uint.max) == foo_optimized(uint.max)) // -> fail.
Apr 23 2015
next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Thursday, 23 April 2015 at 08:56:38 UTC, rumbu wrote:
 On Thursday, 23 April 2015 at 08:33:56 UTC, John Colvin wrote:
 Why can no compiler I try optimise this toy example as I would 
 expect?

 // uncomment if using a C compiler
 // typedef unsigned int uint;
 uint foo(uint a)
 {
  if (a < 5)
    return (a * 3) / 3;
  else
    return 0;
 }

 So, I would expect the compiler to be able to see that it is 
 equivalent to

 uint foo(uint a)
 {
  return (a < 5) ? a : 0;
 }

 But apparently not a single modern compiler I tried can do 
 this optimisation, unless it's hidden in some obscure flag I'm 
 not aware of.
I think because of the potential overflow in a * 3 (if we ignore the a < 5 condition). To optimize this, a compiler must figure out that there is no overflow for any a < 5. If you change the condition to a > 5 and call foo(uint.max), the two expression above are not equivalent. int foo(uint a) { if (a > 5) return (a * 3) / 3; else return 0; } int foo_optimized(uint a) { return (a > 5) ? a : 0; } assert(foo(uint.max) == foo_optimized(uint.max)) // -> fail.
Yes, the three things making * and / not inverses of each other on unsigned integer types are: truncation of integer division div by 0 overflow But it still amazes me that the compiler can't use the a < 5 condition here. I regularly watch compilers do what appear to be magical transformations and simplifications to my code, but here they are tripping up on some very simple maths.
Apr 23 2015
prev sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Thursday, 23 April 2015 at 08:56:38 UTC, rumbu wrote:
 I think because of the potential overflow in a * 3 (if we 
 ignore the a < 5 condition). To optimize this, a compiler must 
 figure out that there is no overflow for any a < 5.
Yes, it is because of modular artithmetics which is a D design flaw. In C++ this only applies to unsigned integers, signed integers are monothonic in C++. I think Rust uses non-modular for both and Ada allows you to specify it. Compiled using ICC: int foo(int a) { if (a > 5) return (a * 3) / 3; else return 0; } yields: xorl %edx, %edx cmpl $5, %edi cmovle %edx, %edi movl %edi, %eax ret --------------------------------- int foo(unsigned int a) { if (a > 5) return (a * 3) / 3; else return 0; } yields: cmpl $5, %edi jbe ..B1.3 movl $-1431655765, %eax lea (%rdi,%rdi,2), %ecx mull %ecx shrl $1, %edx movl %edx, %eax ret ..B1.3: xorl %eax, %eax ret
Apr 23 2015
next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
Interestingly only clang understands that an upcast will prevent 
the overflow:

int foo(unsigned int a)
{
	if (a > 5)
		return ((unsigned long long)a * 3) / 3;
	else
		return 0;
}

Is compiled to compact code in clang:

	xorl	%eax, %eax
	cmpl	$5, %edi
	cmoval	%edi, %eax
	retq
Apr 23 2015
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Thursday, 23 April 2015 at 12:37:12 UTC, Ola Fosheim Grøstad 
wrote:
 On Thursday, 23 April 2015 at 08:56:38 UTC, rumbu wrote:
 I think because of the potential overflow in a * 3 (if we 
 ignore the a < 5 condition). To optimize this, a compiler must 
 figure out that there is no overflow for any a < 5.
Yes, it is because of modular artithmetics which is a D design flaw. In C++ this only applies to unsigned integers, signed integers are monothonic in C++. I think Rust uses non-modular for both and Ada allows you to specify it. Compiled using ICC: int foo(int a) { if (a > 5) return (a * 3) / 3; else return 0; } yields: xorl %edx, %edx cmpl $5, %edi cmovle %edx, %edi movl %edi, %eax ret --------------------------------- int foo(unsigned int a) { if (a > 5) return (a * 3) / 3; else return 0; } yields: cmpl $5, %edi jbe ..B1.3 movl $-1431655765, %eax lea (%rdi,%rdi,2), %ecx mull %ecx shrl $1, %edx movl %edx, %eax ret ..B1.3: xorl %eax, %eax ret
Just to confirm this, all C compilers I tried were able to use the undefined behaviour of signed overflow to avoid the multiplication. D compilers of course do not do this, as signed overflow is defined. Nonetheless, I maintain that the compiler should be able to propagate the value range of a and perform the optimisation regardless.
Apr 23 2015
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/23/2015 5:37 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 Yes, it is because of modular artithmetics which is a D design flaw.
Out of the innumerable posts you write, I can't recall one which didn't assert that whatever D does is wrong.
Apr 23 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 23 Apr 2015 19:05:06 -0700, Walter Bright wrote:

 On 4/23/2015 5:37 AM, "Ola Fosheim =3D?UTF-8?B?R3LDuHN0YWQi?=3D
 <ola.fosheim.grostad+dlang gmail.com>" wrote:
 Yes, it is because of modular artithmetics which is a D design flaw.
=20 Out of the innumerable posts you write, I can't recall one which didn't assert that whatever D does is wrong.
that's 'cause he don't talking about features done right. ;-)=
Apr 23 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 24 April 2015 at 02:33:19 UTC, ketmar wrote:
 On Thu, 23 Apr 2015 19:05:06 -0700, Walter Bright wrote:

 On 4/23/2015 5:37 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?=
 <ola.fosheim.grostad+dlang gmail.com>" wrote:
 Yes, it is because of modular artithmetics which is a D 
 design flaw.
Out of the innumerable posts you write, I can't recall one which didn't assert that whatever D does is wrong.
that's 'cause he don't talking about features done right. ;-)
Oh, but I have!! I've pointed out that the vision for D1 was right, but D2 ruined it by adding cruft without fixing the flaws... ;-) Walter got A LOT right in his original _vision_ as represented on his original website for D1: - Taking current practice for C++ and building a better syntax for the most common patterns. - Clearly stating that a programming language should encourage you to write code that is aesthetically pleasing on the screen and make that easy. - Clearly stating that language semantics should be so simple that you didn't need a long specification for it. - Clearly stating that performance was imperative as a goal for the language and that D would not aim to replace higher level languages like C#. I can applaud to this, anyone who has exposed themselves to the annoyances of C++ can applaud to this! And D1 was a step in the right direction. A good start. The vision was lost on the way to D2, and most unfortunately the market for programming languages is a Winner Takes It All market. D2 is only a marginal improvement on C++, and worse in some areas. That can't win. I find it worrying that the evangelical D users are perceiving D as a compiled scripting language and claim it is similar to Python... D semantics are not at all like Python. That can't win. I find it worrying that the people who say they want to use D as a system programming language are into games, yet the projected vision for the D leadership now is to make it a web programming language that should ship with vibe.d. That can't win. I find it worrying that so many people attracted to D system level programming are into games, yet game development needs are ignored. That can't win. D is lucky that Rust is annoying, Go is marginal, and Nim is unknown, so people are stuck with ugly look C++ code. There is a need to move towards something beautiful, and that's not in Andrei's vision, but in the original D1 vision + the improvements proposed by Bearophile, Timon Gehr and others. Or swing 100% to Andrei's direction and improve significantly on meta programming by adding pattern matching and partial evaluation, so that you have something significantly better than C++. …but move... Remember: It's a winner takes it all game.
Apr 24 2015
next sibling parent reply "weaselcat" <weaselcat gmail.com> writes:
On Friday, 24 April 2015 at 07:04:10 UTC, Ola Fosheim Grøstad 
wrote:
 I find it worrying that so many people attracted to D system 
 level programming are into games, yet game development needs 
 are ignored. That can't win.

 D is lucky that Rust is annoying, Go is marginal, and Nim is 
 unknown, so people are stuck with ugly look C++ code.
+1 to all of this, D needs better ownership semantics to be useful for game development. Many of the recent improvement requests have felt like ugly hacks at best(ref return is useless for classes, not sure if that was intended.) I'm biased because I do essentially zero webdev though, so when I see a lot of changes for std.json or text processing, I don't get too excited. D has a lot of sugar but missing many essential things you'd expect if you want to compete with C++. for example, typecons.unique has been in an unusable state for... ever? It's just another one of those things in D that feels like it will never be finished. It's hard to attract people to a language then tell them they need to fix the standard library if they want to use it. I'm aware that work is being done on master, but it's just one example(and there's tons of blockers...)
 There is a need to move towards something beautiful, and that's 
 not in Andrei's vision, but in the original D1 vision + the 
 improvements proposed by Bearophile, Timon Gehr and others. Or 
 swing 100% to Andrei's direction and improve significantly on 
 meta programming by adding pattern matching and partial 
 evaluation, so that you have something significantly better 
 than C++.
I think pattern matching and better meta programming in general would make the language better either way. C++ has mach7 for pattern matching, all done in metaprogramming AFAIK. I doubt anything like that could be done in D without being far uglier.
Apr 24 2015
next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 24 April 2015 at 07:33:42 UTC, weaselcat wrote:
 I'm biased because I do essentially zero webdev though, so when 
 I see a lot of changes for std.json or text processing, I don't 
 get too excited. D has a lot of sugar but missing many 
 essential things you'd expect if you want to compete with C++.
I do webdev, but I would not use D for it. Integration with the surrounding infrastructure is more important in most webdev applications (e.g. frontends, caching, load balancing, distributed databases/file systems don't have to be application specific). D could fit for a game server, but json and text processing? Nah. The only C++ text processing I do are file paths and the like. D has a slight edge on C++ when it comes to arrays and slicing. Like I spent several days writing my own type safe array slice library to improve my C++ codebase, but then I realized that a library solution is more flexible than a builtin for reference types like slices (template matching on any parameter you want: fixed size, alignment, unique typing, strides, 2D slicing, special casing for void slices). So overall, not sure if it makes sense to have it as a builtin if you also have meta programming features.
 for example, typecons.unique has been in an unusable state 
 for... ever? It's just another one of those things in D that
The memory model needs work and uniformity, not a quick fix. Good unique ownership and stack allocation is critical. I don't recall the last time I used reference counting in C++ and I cringe every time I do heap allocation by reference (rather than embedding).
 I think pattern matching and better meta programming in general 
 would make the language better either way.
Yes, that can be an advantage if you also simplify the core language when you can express the syntactical sugar using the meta programming features. Ideally the core language should just be a high level VM, which you use to verify memory safety, separate compilation, high level optimization etc.
Apr 24 2015
prev sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 24 April 2015 at 07:33:42 UTC, weaselcat wrote:
 C++ has mach7 for pattern matching, all done in metaprogramming 
 AFAIK. I doubt anything like that could be done in D without 
 being far uglier.
Ran across this paper that describes C++ patternmatching, haven't studied it closely, but worth a look: http://www.stroustrup.com/OpenPatternMatching.pdf
May 02 2015
prev sibling next sibling parent reply "bachmeier" <no spam.com> writes:
On Friday, 24 April 2015 at 07:04:10 UTC, Ola Fosheim Grøstad 
wrote:
 the market for programming languages is a Winner Takes It All 
 market.
/*Scratching my head*/ I don't see how anyone could possibly describe the current landscape as "winner takes it all". Scala, Clojure, D, Go, Haskell, C#, Objective C, Swift, Ceylon, Python, Ruby, PHP, Julia...these are just a few of the languages that I've watched develop in recent years.
Apr 24 2015
next sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 24 April 2015 at 14:41:13 UTC, bachmeier wrote:
 On Friday, 24 April 2015 at 07:04:10 UTC, Ola Fosheim Grøstad 
 wrote:
 the market for programming languages is a Winner Takes It All 
 market.
/*Scratching my head*/ I don't see how anyone could possibly describe the current landscape as "winner takes it all". Scala, Clojure, D, Go, Haskell, C#, Objective C, Swift, Ceylon, Python, Ruby, PHP, Julia...these are just a few of the languages that I've watched develop in recent years.
General system level languages: C/C++, Ada. The rest are marginal. On the JVM you get some "big" marginal languages in addition to Java because you have a stable VM, thus reducing risk/retain interoperability. But they are marginal compared to Java. Objective-C/Swift would be dead without Cocoa. They are framework languages and marginal outside the framework. C# would be dead without Windows. It is a framework language that is being pushed outside the framework, but would still die without it. Web dev is a fashion industry, not an engineering discipline. The hot frameworks shift constantly: Perl is dying, Php is dying, Ruby may be next, Go is still marginal and could be a fad. Haskell is a marginal language, even though it is big within the FP community as it is backed by programming language research (you can say the same about ML). Adoption of programming languages are by nature tied to their eco system (libraries, frameworks, and eductional resources), so you have potential for exponential growth and lock in, hence the winner takes it all. There are _lots_ of languages, most are close to dead, some are lingering, some are clinging to a niche... but only a few ones gain momentum.
Apr 24 2015
parent reply "bachmeier" <no spam.com> writes:
On Friday, 24 April 2015 at 16:40:03 UTC, Ola Fosheim Grøstad 
wrote:

 There are _lots_ of languages, most are close to dead, some are 
 lingering, some are clinging to a niche... but only a few ones 
 gain momentum.
Well, I don't want to get into a big debate about how you define those things. I'll just say that I'm not concerned about D disappearing. I write some D code, drop it into an R package along with a Makefile, and my coauthors and I are using D. None of the things you claim as design flaws are a problem for us. As always, when it comes to programming languages, it really depends on what you're trying to do. Not that long ago someone around here was claiming Python is a niche language like Haskell. On Reddit, garbage collection is often called a design flaw. YMMV applies more to programming languages than about anything else.
Apr 24 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 24 April 2015 at 20:50:17 UTC, bachmeier wrote:
 along with a Makefile, and my coauthors and I are using D. None 
 of the things you claim as design flaws are a problem for us.
Sounds like your usage fall into the category "compiled scripting language", but there you have many alternatives. So, you may use D for such, but I'd question if that is a rational direction. For system programming a solid unmanaged memory model, strong typing, verification and near optimal performance matters. The requirements are much more demanding.
 As always, when it comes to programming languages, it really 
 depends on what you're trying to do. Not that long ago someone 
 around here was claiming Python is a niche language like 
 Haskell.
Which is wrong. Python and Haskell are opposites. Python is a versatile general dynamic imperative _scripting_ language, suitable for connecting components top-down. Haskell is a statically typed functional programming language where you design bottom-up. Haskell has a small following (but big within FP). Python has a wide following, extensively documented, to the level where it is difficult to find a question unanswered when using Google.
On Reddit, garbage collection is often called a design
 flaw. YMMV applies more to programming languages than about 
 anything else.
C++ would have been dead if the memory model was based on a Boehm GC. Many people have tried and left D due to compiler quality and GC. If those two issues had been given the highest priority (over new features) D would have taken a larger market share a long time ago. (And no Tango/Phobos was not a big deal, just a minor annoyance.)
Apr 25 2015
parent reply "Lucas" <lgoss007 yahoo.com> writes:
On Saturday, 25 April 2015 at 07:51:38 UTC, Ola Fosheim Grøstad 
wrote:
 C++ would have been dead if the memory model was based on a 
 Boehm GC. Many people have tried and left D due to compiler 
 quality and GC. If those two issues had been given the highest 
 priority (over new features) D would have taken a larger market 
 share a long time ago.

 (And no Tango/Phobos was not a big deal, just a minor 
 annoyance.)
That's me. I looked at D a while back and started playing around with it some, but it seemed at the time D was still working out it's design (v2 was being discussed) and the GC seemed too integral in the libraries. I came back recently to see how its progressed and the focus seems to be like it wants to be a lower level scripting language. I can just use Java or C# for such things, both have a wider range of supported platforms and perform pretty well for having a GC. D does seem nice for shell scripting on *nix though. But the GC is annoying when making games, it's like a network lag, very noticeable, even with tuning. I use C++ as C with classes. If C had namespaces, strings, templates with a good syntax and was all in one file it would be a dream (classes are a bonus). That was my initial impression of D... until I learned of the GC. Then I thought of it as Java/C# without the VM. The lack of supported platforms was also a consideration. At a minimum I'd want 64 bit desktop support for Linux, OSX and Windows and mobile support for iOS and Android. LDC is tempting.
Apr 25 2015
parent reply "jdeath" <jdeath jdeath.com> writes:
i couldn't agree more with Ola Fosheim Grøstad. the d1 vision was 
great and d2 has become an abomination of a scripting language. i 
left after the demise of d1 and the lot of andrei's ideas.
now come every once in while and see whats going on - and hope 
maybe in d3 it will go back to the vision of d1.
no offense intended
Apr 25 2015
parent reply "Lucas" <lgoss007 yahoo.com> writes:
I think Andrei has some good ideas and I like seeing his and 
others perspectives. Everyone has different experiences and can 
bring something to the table, which can cause some interesting 
disagreements, but more viewpoints add perspective. The 
difficulty is that everyone has their own agenda of why they want 
their change, and value is different depending on what kind of 
work you do and what domains your familiar with.

I actually do like that D2 wasn't afraid to break things. I 
actually like breaking changes if done for good reason, but I 
can't say I know all the changes in D2.
Apr 25 2015
parent "weaselcat" <weaselcat gmail.com> writes:
On Saturday, 25 April 2015 at 17:00:28 UTC, Lucas wrote:
 I think Andrei has some good ideas and I like seeing his and 
 others perspectives. Everyone has different experiences and can 
 bring something to the table, which can cause some interesting 
 disagreements, but more viewpoints add perspective. The 
 difficulty is that everyone has their own agenda of why they 
 want their change, and value is different depending on what 
 kind of work you do and what domains your familiar with.

 I actually do like that D2 wasn't afraid to break things. I 
 actually like breaking changes if done for good reason, but I 
 can't say I know all the changes in D2.
This, not sure what's up with all the blame on Andrei for D's issues. Especially blaming him for D not being enough of a systems language when Andrei is most known for his c++ work. I think D is moving in the right direction, and the community seems more lively now than ever since I lurked this NG(~2012) The focus on working without the GC is _still_ very fresh, yet so much has been done in only months. The refusal to acknowledge that D should be usable without a GC was pretty bad though.
Apr 25 2015
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Fri, 24 Apr 2015 14:41:11 +0000, bachmeier wrote:

 On Friday, 24 April 2015 at 07:04:10 UTC, Ola Fosheim Gr=C3=B8stad wrote:
 the market for programming languages is a Winner Takes It All market.
=20 /*Scratching my head*/ =20 I don't see how anyone could possibly describe the current landscape as "winner takes it all". Scala, Clojure, D, Go, Haskell, C#, Objective C, Swift, Ceylon, Python, Ruby, PHP, Julia...these are just a few of the languages that I've watched develop in recent years.
yet how much of them are really used for something serious? C and C++ -- thanks to libraries PHP -- thanks to monkeys C# -- thanks to m$ Obj-C -- thanks to apple sometimes Ruby and nodejs with js (oh god, why?!). that's all. you can find some niche projects on other languages, but that=20 doesn't really matters.=
Apr 25 2015
prev sibling next sibling parent reply "Laeeth Isharc" <nospamlaeeth nospamlaeeth.com> writes:
Hi Ola.

You have been in this community for much longer than me, and I 
always learn from your posts technically.

 I find it worrying that the evangelical D users are perceiving 
 D as a compiled scripting language and claim it is similar to 
 Python... D semantics are not at all like Python. That can't 
 win.
Why does it worry you? What bad things will happen? People say all kinds of things and have all kinds of delusions about reality, yet the sky doesn't fall. To an academic computer scientist, no doubt Python and D could not be more different. To someone in the commercial world (hedge funds are what I know best), what tool do you pick up next to process your data when Python starts to choke? Both the Eurostar and the plane will get you from London to Berlin - they work on very different principles technically, but am I a fool for considering them substitutes?
 I find it worrying that the people who say they want to use D 
 as a system programming language are into games, yet the 
 projected vision for the D leadership now is to make it a web 
 programming language that should ship with vibe.d. That can't 
 win.
Perhaps it's obvious to someone more familiar with the specific questions, but I think your argument would be more effective if you explained why shipping vibe.d somehow detracts from D's potential as a systems programming language, or for games in particular. There are games devs here, and although they are spirited and opinionated, I don't recall them arguing against incuding vibe.d.
 I find it worrying that so many people attracted to D system 
 level programming are into games, yet game development needs 
 are ignored. That can't win.
Really? You have a man with the expertise and experience of Walter Bright devoting his time to rewriting string processing parts of the standard library himself, in service of the goal of making Phobos GC free, and you say that game development needs are ignored? (Not that games need strings, but that either a library is GC free or it isn't, and this is something games people seem to care about). Plus all the work on refcounting etc. I am sure there are many other aspects, and games themselves don't interest me, but that doesn't strike me as a balanced perspective. It's odd to mention D's role as a systems language without discussing its use in embedded systems. The pioneer who spoke at dconf a couple years back undertook a valiant effort, but it was too much for him to manage in one go. (And of course Adam Ruppe's highly entertaining presentation on bare metal programming). But although the pioneers may have the arrows in the back and not always be so happy about the situation, they do clear the way for others - these now seem to be picking this up, and I would be really surprised if D is not pretty usable on a decent subset of embedded systems in a couple of years. Similarly the work on ARM/Android/iOS, which seems to be coming along.
 There is a need to move towards something beautiful, and that's 
 not in Andrei's vision, but in the original D1 vision + the 
 improvements proposed by Bearophile, Timon Gehr and others.
I appreciate you may not have time, but if you had any links to stuff if they are gathered in documents rather than myriad fragments, I would be curious to see.
 …but move...
It's not for me to talk about strategy. But it strikes me that you are calling for a further massive shift, when people have their plates too full already. It may not be exciting, but full C++ support, making Phobos GC free, rounding out refcounting and introducing custom allocators seem to me to be likely to have a more immediate and more powerful impact than what you suggest, whatever may be the longer-term merits. (I would like to understand these)
 Remember: It's a winner takes it all game.
I notcie that you keep asserting this, without taking the trouble (that I have seen) to argue the reasons for this belief. The winner takes all idea can be traced to the work of Vilfredo Pareto, but he spends some time speaking about the circulation of the elites. In other words the top dog is not static - this applies to income of a relatively free-market nation, and it also applies to other aspects of this phenomenon in human society. It's a human tendency to lack imagination and to believe that the present state of things is how they intrinsically are and can never change. This tendency is developed in our era by the mental habits that go with the way we use technology (see Iain Macgilchrist's work), and yet it also creates tremendous opportunity for those with imagination and perceptiveness. You might as well have said To Honda, Toyota, Nissan, Hyundai etc that "it's a winner takes all game" when their products were demonstrably inferior with little prospect ever of competing with US cars - the idea of that would have been laughable. But beyond being mistaken, this would not have been helpful advice. I don't follow the news closely day to day, and just saw there are now 8Tb drives for a couple of hundred pounds. These days you can download not just data for US equities, but the majority of all non-intraday data for all publicly listed stocks, futures, and currencies and economic series for free - this used to cost $100k pa plus. So data sets keep growing - CPU performance continues to improve, but in a less convenient way, but as I understand it memory perf lags. Which means in the future you may be increasingly irritated by people speaking of using D for scripting purposes... Laeeth.
Apr 25 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Saturday, 25 April 2015 at 14:48:41 UTC, Laeeth Isharc wrote:
 I find it worrying that the evangelical D users are perceiving 
 D as a compiled scripting language and claim it is similar to 
 Python... D semantics are not at all like Python. That can't 
 win.
Why does it worry you? What bad things will happen?
Bad things that could happen is that D never can be like Python and if you try to make it such you no longer have a system programming contender.
 questions, but I think your argument would be more effective if 
 you explained why shipping vibe.d somehow detracts from D's
Because it shifts the focus towards an application area where D will have trouble to gain significant ground. That means the language will be evaluated up to that application area. There is a limit in the market as new projects will gravitate towards the most promising language in their application area. And there are many languages pitching in the web domain.
 Really?  You have a man with the expertise and experience of 
 Walter Bright devoting his time to rewriting string processing 
 parts of the standard library himself, in service of the goal
Which essentially is escapism from a language development point of view. Languages are not judged by their libraries, unless they lack functionality due to flaws in language semantics.
 needs are ignored?  (Not that games need strings, but that 
 either a library is GC free or it isn't, and this is something 
 games people seem to care about).
nogc was a good addition. If D is the best option in an application domain where you have long running projects people will build frameworks for it that covers the ground that the existing libraries are missing. I have no concerns related to Phobos whatsoever. It is inconsequential. This is different in a scripting language which often is used in contexts where you cannot predict your needs ahead of time. I.e. you are prototyping and are exploring new directions or are just covering your needs day by day. If you are doing that in a long running predictable project you are in a bad shape (aka fire fighting).
 Plus all the work on refcounting etc.  I am sure there are many 
 other aspects, and games themselves don't interest me, but that 
 doesn't strike me as a balanced perspective.
Games is just something that is being brought up because people interested in it come to D looking for something less tedious than C++. It is just an exemplar of system level programming (when the games run after loading). You could say the same things about interactive audio software, embedded programming, memory constrained high throughput servers etc.
 It's odd to mention D's role as a systems language without 
 discussing its use in embedded systems.  The pioneer who spoke 
 at dconf a couple years back undertook a valiant effort, but it 
 was too much for him to manage in one go.  (And of course Adam 
 Ruppe's highly entertaining presentation on bare metal 
 programming).
Adam is a great guy, but he is probably more patient than most with figuring out workarounds ;-).
 Similarly the work on ARM/Android/iOS, which seems to be coming 
 along.
Maybe, I do iOS work and it is very convenient to just use Objective-C++ everywhere I need something that cannot be done in C++. Add to this that Apple keeps mutating their libraries and Apples IDE becomes kind of irreplacable. You need something a lot better than C++ to encourage a switch there...
 There is a need to move towards something beautiful, and 
 that's not in Andrei's vision, but in the original D1 vision + 
 the improvements proposed by Bearophile, Timon Gehr and others.
I appreciate you may not have time, but if you had any links to stuff if they are gathered in documents rather than myriad fragments, I would be curious to see.
I don't think so, but it is mostly a fairly standard stance about programming language ideals. (Which C does not adhere to, and D leans heavily on C.)
 It's not for me to talk about strategy.  But it strikes me that 
 you are calling for a further massive shift, when people have 
 their plates too full already.
Not at all. I've argued that D2 should stay with the GC, and focus on doing what it does really well, basically catering to the market I think you are in. Changing the semantics slightly so that you only touch the cachelines that need to be traced when scanning live objects. Then rework the memory model, which is a lot of work if done well, to a D3 version of the language. Fudging it with reference counting hacks makes D not very attractive beyond "compiled scripting", but "compiled scripting" is better off with a good GC than unmanaged memory handling and ref counting by default... So the proposed solutions have a very low potential for increasing market share. In fact some of the proposed changes would probably make the language hard to analyze which has a bad effect on future tooling and a programmer's ability to keep a sane model of the language in his head. C++ combats this with good IDEs (that complain when you forgot to add "typename" or ".template" in a templated method call (rather ugly). That's obviously just an emergency solution for a language that's beyond "cleaning up". And it's where D is heading, not in one go, but drip, drip, drip... like C++.
 circulation of the elites.  In other words the top dog is not 
 static - this applies to income of a relatively free-market
I takes ~10 years for a language to get big, so it is not like we will be overrun by surprises anytime soon... However the "winner takes it all" effect has become a lot stronger now that you have so many excellent free libraries. When you had to purchase your libraries the situation was way different. In the 80s you could fund a decent dev environment, but that gets harder and harder unless you own the platform (like Apple). Critical mass is a very strong phenomenon in this domain. To the extent that the commercial sector has more or less given up and sell IDEs only. If this was not so, the commercial sector would be much more active in language development (inventing new languages).
 You might as well have said To Honda, Toyota, Nissan, Hyundai 
 etc that "it's a winner takes all game" when their products
Nah, there is plenty of commercial activity in the car industry.
 this used to cost $100k pa plus.  So data sets keep growing - 
 CPU performance continues to improve, but in a less convenient 
 way, but as I understand it memory perf lags.  Which means in 
 the future you may be increasingly irritated by people speaking 
 of using D for scripting purposes...
I'm not irritated by it. It just does not represent system level programming, so unless D stops claiming to be a system level programming language (like Go) it should not be the primary long term target. Pervasive reference counting is a scripting language solution. System level programming means you control memory layout, memory usage etc. For instance, in my current project it would perhaps be easier to use ref counting, but since I generate/load arrays that could easily consume 40% of the memory I better be sure that the memory is released before loading the next set. Otherwise real time performance will suffer (audio playback). That level of control is overkill for processing historical data, but the most promising solution for that application area are distributed cloud solutions. Like Google Big Query that AFAIK can do brute force SQLish quries over very large datasets fast (using some kind of built in query optimization). Cheers!
Apr 25 2015
parent reply "Laeeth Isharc" <nospamlaeeth nospam.laeeth.com> writes:
On Saturday, 25 April 2015 at 22:05:05 UTC, Ola Fosheim Grøstad 
wrote:
 On Saturday, 25 April 2015 at 14:48:41 UTC, Laeeth Isharc wrote:
 I find it worrying that the evangelical D users are 
 perceiving D as a compiled scripting language and claim it is 
 similar to Python... D semantics are not at all like Python. 
 That can't win.
Why does it worry you? What bad things will happen?
Bad things that could happen is that D never can be like Python and if you try to make it such you no longer have a system programming contender.
So because some people have found it useful in that domain and have shared their positive feelings, there is a risk that this hijacks the direction of the language away from what would ultimately be to its greatest benefit (and perhaps to yours, anyway)? "Nobody goes there anymore - that place is too popular". Conceivable, but you can hardly control what people do with and say about their use of a programming language, even of a closed source commercial product. I guess one can submit pull requests that take the language in the direction one favours though, and maybe you do this.
 questions, but I think your argument would be more effective 
 if you explained why shipping vibe.d somehow detracts from D's
Because it shifts the focus towards an application area where D will have trouble to gain significant ground. That means the language will be evaluated up to that application area. There is a limit in the market as new projects will gravitate towards the most promising language in their application area. And there are many languages pitching in the web domain.
It's very hard to know what people ultimately end up doing with a tool that you bring into the world, and one may be the master of computer science and language design and still be surprised by what takes off. The world is a big place and changing rapidly. If one has a set idea about what something should and shouldn't do, one may find oneself eventually overcome by Nature, who is more powerful - I have given up trying to fight her.
 Which essentially is escapism from a language development point 
 of view. Languages are not judged by their libraries, unless 
 they lack functionality due to flaws in language semantics.
It depends on who is doing the judging, and what they are trying to do. The decision by a commercial user to adopt a language framework surely does depend on the cost of accomplishing her goals using that framework, and this surely depends for many domains on the implementations and libraries available. It's a funny thing I notice people do to pretend that decisions about language adoption are based on the merits of the pure language itself, when only for a subset (I suspect a minority) is that truly the case. [I am not sure if it is escapism to listen to your market and do what you need to to address the biggest concerns].
 This is different in a scripting language which often is used 
 in contexts where you cannot predict your needs ahead of time. 
 I.e. you are prototyping and are exploring new directions or 
 are just covering your needs day by day. If you are doing that 
 in a long running predictable project you are in a bad shape 
 (aka fire fighting).
Fair point, although I suspect this is a feature of the domain not the language. One might write a bond analytics framework in C++, but that doesn't mean one knows how it ultimately is going to be used. The world is a big place, and one doesn't necessarily understand the needs of others. Of course it is frustrating if the world charges ahead in a direction that one doesn't find interesting, but submitting code probably has more influence than telling people they shouldn't do this. This whole scripting vs system language thing suffers from reification of a distinction that once mapped to something crisp in reality, but no longer does. AHL (one of the largest systematic fund managers) have all their trading systems in Python, for example, so the connotations of lack of robustness and difficulty of building large and complex systems that perhaps were once associated with the idea of a scripting language perhaps apply less today. (Which isn't to say that you will get me to love dynamic typing for serious work). Is Go a scripting language or a systems language? On the one hand, the D front page no longer positions D as a systems language (which I think is the right move); on the other, people are using it for low-level stuff. So why get hung up on labels: technology is a tool for solving problems, and the question is how well adapted a particular tool is to the particular problems one faces (and how easily it can get there with a bit of work).
 Adam is a great guy, but he is probably more patient than most 
 with figuring out workarounds ;-).
Yes - which is why he (and his unique kind of way of being in the world) is so valuable. Someone needs to break the ground, and by doing so makes it easier for everyone that follows. See the very interesting work on embedded systems for ARM Cortex. Adam's technical contribution is large in itself, but the effect of his inspiration on others may well be larger.
 Similarly the work on ARM/Android/iOS, which seems to be 
 coming along.
Maybe, I do iOS work and it is very convenient to just use Objective-C++ everywhere I need something that cannot be done in C++. Add to this that Apple keeps mutating their libraries and Apples IDE becomes kind of irreplacable. You need something a lot better than C++ to encourage a switch there...
Let's see what happens - I am very interested to find out. I am looking at a project that might involve D on the server and PC client side, and it is very nice to know that by the time I need it, probably it will be viable for the analytics on mobile (even if you glue it together with something else).
 There is a need to move towards something beautiful, and 
 that's not in Andrei's vision, but in the original D1 vision 
 + the improvements proposed by Bearophile, Timon Gehr and 
 others.
I appreciate you may not have time, but if you had any links to stuff if they are gathered in documents rather than myriad fragments, I would be curious to see.
I don't think so, but it is mostly a fairly standard stance about programming language ideals. (Which C does not adhere to, and D leans heavily on C.)
If you have time, I would very much appreciate any book suggestions, and the like. I am returning to programming after a twenty year break, and apparently thinking has moved on a little since I was away ;)
 Then rework the memory model, which is a lot of work if done 
 well, to a D3 version of the language.
Have you written anything on what this should look like?
 Fudging it with reference counting hacks makes D not very 
 attractive beyond "compiled scripting", but "compiled 
 scripting" is better off with a good GC than unmanaged memory 
 handling and ref counting by default...
I didn't know anyone wanted to change the defaults, rather than to offer some more choices.
 So the proposed solutions have a very low potential for 
 increasing market share.
But do you think that you are looking at it the right way? Must it be the case that we are all in a battle to the death for a share of a limited pie? I personally tend to agree with Peter Thiel that it is a destructive and false belief to think that as an entrepreneur (whether in the commercial or open source worlds) one should think of competition as a positive thing. It's much better for oneself to strive for a monopoly, but a monopoly gained through creating something valuable. And it's probably better for everyone else, too. http://www.wsj.com/articles/peter-thiel-competition-is-for-losers-1410535536 "Americans mythologize competition and credit it with saving us from socialist bread lines. Actually, capitalism and competition are opposites. Capitalism is premised on the accumulation of capital, but under perfect competition, all profits get competed away. The lesson for entrepreneurs is clear: If you want to create and capture lasting value, don't build an undifferentiated commodity business. ... So a monopoly is good for everyone on the inside, but what about everyone on the outside? Do outsize profits come at the expense of the rest of society? Actually, yes: Profits come out of customers' wallets, and monopolies deserve their bad reputation—but only in a world where nothing changes. In a static world, a monopolist is just a rent collector. If you corner the market for something, you can jack up the price; others will have no choice but to buy from you. Think of the famous board game: Deeds are shuffled around from player to player, but the board never changes. There is no way to win by inventing a better kind of real-estate development. The relative values of the properties are fixed for all time, so all you can do is try to buy them up. But the world we live in is dynamic: We can invent new and better things. Creative monopolists give customers more choices by adding entirely new categories of abundance to the world. Creative monopolies aren't just good for the rest of society; they're powerful engines for making it better." Speaking as a commercial user that has an intellectual curiosity about languages, and the enthusiasm of a craftsman for a tool that can help do a job well and efficiently, what matters as a user of D isn't its market share, but whether one can bet on it being around in 5-10 years, whether one can find people capable of helping one as one grows, whether its robust enough for ones application (not all compiler bugs are equal), power, efficiency, productivity, and the existence of and ease of porting frameworks needed to accomplish one's ends. So as regards adoption, the following chart is much more interesting to me than market share - and I think it should be to you too, perhaps! https://qph.is.quoracdn.net/main-qimg-ee719ae3a29d4523251255a604a16a6d?convert_to_webp=true
 In fact some of the proposed changes would probably make the 
 language hard to analyze which has a bad effect on future 
 tooling and a programmer's ability to keep a sane model of the 
 language in his head.
Seriously? One is hardly going to need to bother with the allocation stuff and reference counting if one doesn't need it (I am not sure what other factors you mean). And it's the coherence or lack of it that leads to difficulty fitting things in one's head - if things are based on principles you don't need to learn a whole bunch of rules. I personally found D quicker to pick up to reach a level where I can be decently productive than Python, and I don't think that would have been true of C++. So there is plenty of spare cognitive budget to spend on a bit of optional complexity if it's done right to serve a real need. Plus I personally wouldn't be short (bet against) quality of tooling if you look at how things have developed and what is in the pipeline.
 However the "winner takes it all" effect has become a lot 
 stronger now that you have so many excellent free libraries.
I thought languages weren't judged by libraries ;) But I take your point, and agree with it - which is why C++ interop is so satisfying to see develop. And perhaps it is not necessary to have libraries written natively, but merely the key components, with bindings/wrappers being quite satisfactory for everything else. Empirically though, as someone else here said, the number of languages in decently wide usage does not seem to fit your view of a winner take all phenomenon (I am not sure if you intend this to apply to all languages, or just to systems languages). That's what Knuth called for in the talk I posted from decades back - he said that because use cases differ and because of the diversity of cognitive styles, one size fits all in language design was not the right way.
 You might as well have said To Honda, Toyota, Nissan, Hyundai 
 etc that "it's a winner takes all game" when their products
Nah, there is plenty of commercial activity in the car industry.
Perhaps I was unclear. Japanese auto sector used to be a joke, and market share was tiny. They achieved adoption at the fringes, and then used this as a launching pad to move into more impressive domains. Similar stories with newer ways of making steel, for example. Thiel talks about this. Ie you don't win by beating your opponent head-on in the area where he is strongest - that's just suicidal. So in my view it's totally irrelevant to speak about what would be needed to get a core heavy C++ guy to switch - you want to persuade the guy who has unusual needs, who is unhappy with his existing options, who has more freedom to try things, and the like. Then if you achieve big wins in small areas, other people will slowly take notice.
 I'm not irritated by it. It just does not represent system 
 level programming, so unless D stops claiming to be a system 
 level programming language (like Go) it should not be the 
 primary long term target. Pervasive reference counting is a 
 scripting language solution.
Okay. From what I can see D these days claims to be D - not even in essence a better C++ (although that is one facet of it). Not quintessentially a scripting language, although it can do that very well in many cases, and not quintessentially exclusively a systems language. Does it matter what label you pin on it ? It might be more constructive to say: here is the problem I am grappling with in building this embedded system/doing this audio processing/etc, and there are these little things in the runtime/library etc that get in the way.
 System level programming means you control memory layout, 
 memory usage etc. For instance, in my current project it would 
 perhaps be easier to use ref counting, but since I 
 generate/load arrays that could easily consume 40% of the 
 memory I better be sure that the memory is released before 
 loading the next set. Otherwise real time performance will 
 suffer (audio playback).
Yes - I am just now having the same problem, although luckily for me it's not real time, and I can just allocate a static buffer once and re-use it.
 That level of control is overkill for processing historical 
 data, but the most promising solution for that application area 
 are distributed cloud solutions. Like Google Big Query that 
 AFAIK can do brute force SQLish quries over very large datasets 
 fast (using some kind of built in query optimization).
There is a lot inbetween the area where Python chokes, and where you want to have the complexity, hassle and expense of a managed cluster. Also, in this environment, it's so much easier to prototype something where one doesn't require justifying a budget than to have to do it in an industrial style at vast scale from day one. I brought up the 8Tb drive as just one straw in the wind for something that will surely be playing out in a fractal way. I'll leave it there as per Andrei's request about focusing on the hackathon. Always interesting to exchange perspectives. Laeeth.
Apr 26 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/26/15 4:33 AM, Laeeth Isharc wrote:
 I'll leave it there as per Andrei's request about focusing on the
 hackathon.
Thanks! -- Andrei
Apr 26 2015
parent "jack death" <jdeath jdeath.con> writes:
On Sunday, 26 April 2015 at 15:40:11 UTC, Andrei Alexandrescu 
wrote:
 On 4/26/15 4:33 AM, Laeeth Isharc wrote:
 I'll leave it there as per Andrei's request about focusing on 
 the
 hackathon.
Thanks! -- Andrei
thanks for what? need statistics that you make? well i download and try, play --- and don't use. your statics are worth NOTHING. please clean up the language - d1 style and make it stable.
Apr 26 2015
prev sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Sunday, 26 April 2015 at 11:33:07 UTC, Laeeth Isharc wrote:
 Conceivable, but you can hardly control what people do with and 
 say about their use of a programming language, even of a closed 
 source commercial product.  I guess one can submit pull 
 requests that take the language in the direction one favours 
 though, and maybe you do this.
You are downplaying the underlying issue. When you design a language you have layers that build upon each other. The memory model is a foundational design layer (not implementation) that affects all layers above it. It's not a patchable entity.
 by what takes off.  The world is a big place and changing 
 rapidly.  If one has a set idea about what something should and 
 shouldn't do, one may find oneself eventually overcome by 
 Nature, who is more powerful - I have given up trying to fight 
 her.
That would take a paradigm shift. Which could happen, but not any time soon. (e.g. program synthesis). D/C++/Java are all built on Simula's model. Nothing surprisingly new. And no, the uptake of programming languages over the past 30 years have not been full of surprises... It's been surprisingly predictable. Installed base and critical mass across the board.
 available.  It's a funny thing I notice people do to pretend 
 that decisions about language adoption are based on the merits 
 of the pure language itself, when only for a subset (I suspect 
 a minority) is that truly the case.
As usual in these technological social dynamics you have: 1. early adopters 2. hubs (people/artifacts that connect to lots of new nodes in the network) 3. mainstream adoption (commercial project management) At first you need the early adopters. Like independent game devs/enthusiast that pave the road by building attractive infrastructure, like state of the art frameworks. Then the frameworks/people (if they are good) will connect to the more pragmatic audience. But that means you need excellent programmers/designers in the first place, and that means you need something that excellent programmers/designers find attractive. If Rust overcomes the complexity of linear typing, then they are in a good spot (in terms of social dynamics), but that's a big "if". The mainstream will go with what is perceived as low risk/reduced costs. And that takes a lot more than a downloadable compiler or three.
 [I am not sure if it is escapism to listen to your market and 
 do what you need to to address the biggest concerns].
You mean like fixing the language so that you can either have a fast GC, efficient ownership handling or both? Surely you need at least one?
 necessarily understand the needs of others.  Of course it is 
 frustrating if the world charges ahead in a direction that one 
 doesn't find interesting, but submitting code probably has more 
 influence than telling people they shouldn't do this.
Nope. As long as there are intelligent people involved informing is always the most efficient strategy, since you need people to pull in the same direction. If not, then forking is the only solution. Submitting code has very low effect when the key issues are design issues or knowledge related. You now have nogc and a modest effort on improving the GC. That came about as an outcome of persistent debate. Without persistent debate, nothing would have come in that direction.
 This whole scripting vs system language thing suffers from 
 reification of a distinction that once mapped to something 
 crisp in reality, but no longer does.
Uhm... This is completely wrong. It's like conflating perl and Ada.
 Is Go a scripting language or a systems language?
It is an application level programming language geared towards servers using a particular implementation strategy (CSP/GC). Go is a very opinionated language.
  On the one hand, the D front page no longer positions D as a 
 systems language (which I think is the right move); on the 
 other, people are using it for low-level stuff.
Andrei and Walter both keep calling it a system level programming language. Walter repeatedly states that even 1% performance loss is a big issue for him. So obviously they will need to focus on backing that up, or make a statement that there is a shift in direction.
 Adam's technical contribution is large in itself, but the 
 effect of his inspiration on others may well be larger.
Probably, but workaround makes for unmaintainable code. Tricks are cute, but you should avoid using tricks in production.
 Have you written anything on what this should look like?
Could, not should. Read up on linear typing and you see what the perimeter looks like. Read up on C++ and you know what current practice looks like. Read up on dependent and behavioural typing and you may catch a glimpse of the horizon. As far as I am concerned, C99/C++ semantics are good enough as a common ground (if made orthogonal). The main issues C++ struggles with are syntactical/textual. That doesn't mean you shouldn't try to do better, of course.
 Must it be the case that we are all in a battle to the death 
 for a share of a limited pie?
As long as the resources are limited, yes. As long as the programming language theory status quo is limited, yes. The vast majority resources are not poured into the language development, it is put into the software written with it.
 expense of the rest of society? Actually, yes: Profits come out 
 of customers' wallets, and monopolies deserve their bad 
 reputation—but only in a world where nothing changes.
English speaking countries have huge savings on not maintaining a small scale language. The only advantage in speaking Norwegian is social dynamics and group borders, but those niche benefits have a high cost. In Norway we have 3 official languages unique to this country, with a rather small population. That's great for maintaining group identity, but there are costs involved and no measurable advantages, they are cultural. You forget that imperative Simula-style languages are more or less the same, but the devil is in getting the details right. And CS is a field that gives you a lot of handholding to get the quirks out.
 But the world we live in is dynamic: We can invent new and 
 better things.
Language invention by and large happens in academia. Very little significant invention happens outside it. Neither C++/D or Java qualify as being inventive. That's ok, we are getting different blends of the same. Then you need to find the right blend, and CS and language history provides guidance.
 matters as a user of D isn't its market share, but whether one 
 can bet on it being around in 5-10 years, whether one can find
 people capable of helping one as one grows, whether its robust 
 enough for ones application (not all compiler bugs are equal), 
 power, efficiency, productivity, and the existence of and ease 
 of porting frameworks needed to accomplish one's ends.
What matters is that projects can pick the language because it reduces risks and saves costs. It is currently difficult to make a case for D/Rust/Nim without factoring in emotional factors (such as personal fun). Go might have a case in some scenarios.
 So as regards adoption, the following chart is much more 
 interesting to me than market share - and I think it should be 
 to you too, perhaps!
No, because retention is the only metric that matters. So github and stackoverflow presence is a more significant measure. Here's D's github presence: https://github.com/trending?l=d&since=monthly
 need to learn a whole bunch of rules.  I personally found D 
 quicker to pick up to reach a level where I can be decently 
 productive than Python, and I don't think that would have been 
 true of C++.
C++ suffers from a lot of textual noise. Python's model is quite simple, much simpler than D. What matters long-term is the core language. E.g. the semantical language that you are dealing with after removing all the sugar.
 Plus I personally wouldn't be short (bet against) quality of 
 tooling if you look at how things have developed and what is in 
 the pipeline.
There is no language spec.
 However the "winner takes it all" effect has become a lot 
 stronger now that you have so many excellent free libraries.
I thought languages weren't judged by libraries ;) But I take your point, and agree with it - which is why C++ interop is so
Languages should not be judged by libraries, but libraries are judged by project managers.
 Empirically though, as someone else here said, the number of 
 languages in decently wide usage does not seem to fit your view 
 of a winner take all phenomenon (I am not sure if you intend 
 this to apply to all languages, or just to systems languages).
It applies to all general programming languages.
 That's what Knuth called for in the talk I posted from decades 
 back - he said that because use cases differ and because of the 
 diversity of cognitive styles, one size fits all in language 
 design was not the right way.
I sincerely doubt he was arguing for having a dozen different Simula descendants with a little bit of ad-hoc generic programming bolted on. He probably referred to different programming paradigms. C++ and D are the same paradigm.
 don't win by beating your opponent head-on in the area where he 
 is strongest - that's just suicidal.  So in my view it's 
 totally irrelevant to speak about what would be needed to get a 
 core heavy C++ guy to switch - you want to persuade the guy who 
 has unusual needs, who is unhappy with his existing options, 
 who has more freedom to try things, and the like.  Then if you 
 achieve big wins in small areas, other people will slowly take 
 notice.
But then you need to excel in at least one area?! Competing compilers are free, the price point can't be lowered. So your only option is to compete on quality, marketing or bundling (e.g. swift/C#).
 can do that very well in many cases, and not quintessentially 
 exclusively a systems language.
But _A_ systems language. And that comes with lot of requirements if it is to be taken seriously.
 pin on it ?  It might be more constructive to say: here is the 
 problem I am grappling with in building this embedded 
 system/doing this audio processing/etc, and there are these 
 little things in the runtime/library etc that get in the way.
If D was mature and competitive yes, but I already know the answer. C++ is currently the better option, even if it is a syntactical mess. That does not mean that I favour C++, or would do hobby projects in it. It means that I don't find D a rational choice for commercial use in system level programming.
 area are distributed cloud solutions. Like Google Big Query 
 that AFAIK can do brute force SQLish quries over very large 
 datasets fast (using some kind of built in query optimization).
There is a lot inbetween the area where Python chokes, and where you want to have the complexity, hassle and expense of a managed cluster
BigQuery is on demand and $5/TB for processing. BigQuery and Google Cloud Dataflow look like reasonable alternatives to evaluate IMO.
  Also, in this environment, it's so much easier to prototype 
 something where one doesn't require justifying a budget than to 
 have to do it in an industrial style at vast scale from day one.
Maybe. An external query engine sounds like the better solution for prototyping to me, though.
 I'll leave it there as per Andrei's request about focusing on 
 the hackathon.
Oh well, I don't think these debates have any effect on that. People probably have an ability to set their own priorities whether it is family or hackatons.
Apr 26 2015
parent reply "weaselcat" <weaselcat gmail.com> writes:
On Sunday, 26 April 2015 at 22:41:22 UTC, Ola Fosheim Grøstad 
wrote:
 Oh well, I don't think these debates have any effect on that. 
 People probably have an ability to set their own priorities 
 whether it is family or hackatons.
I think it's worth respecting Andrei's wishes for the week, we can resume arguing over D in May. : )
Apr 26 2015
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Sunday, 26 April 2015 at 23:05:28 UTC, weaselcat wrote:
 On Sunday, 26 April 2015 at 22:41:22 UTC, Ola Fosheim Grøstad 
 wrote:
 Oh well, I don't think these debates have any effect on that. 
 People probably have an ability to set their own priorities 
 whether it is family or hackatons.
I think it's worth respecting Andrei's wishes for the week, we can resume arguing over D in May. : )
Sure, _you_ are free to do what _you_ want to do in the next week. :) Telling other people what to do before they go do bed is over-the-top (unless they are aggressive/going ad-hominem). Closing a longwinded rebuttal with "oh, I'm out of here now cause of Andrei" is downright impolite. ;-] But Andrei should then surely shut down threads that distract the population towards hacker news too: http://forum.dlang.org/thread/mh8sq4$2npp$1 digitalmars.com Oh, now, wait. That's different, because that thread is geared towards marketing how much better D is than C++... Surely that is much more important than fixing your own issues...
Apr 27 2015
prev sibling parent reply "bigsandwich" <bigsandwich gmail.com> writes:
On Friday, 24 April 2015 at 07:04:10 UTC, Ola Fosheim Grøstad 
wrote:
 On Friday, 24 April 2015 at 02:33:19 UTC, ketmar wrote:
 On Thu, 23 Apr 2015 19:05:06 -0700, Walter Bright wrote:

 On 4/23/2015 5:37 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?=
 <ola.fosheim.grostad+dlang gmail.com>" wrote:
 Yes, it is because of modular artithmetics which is a D 
 design flaw.
Out of the innumerable posts you write, I can't recall one which didn't assert that whatever D does is wrong.
that's 'cause he don't talking about features done right. ;-)
Oh, but I have!! I've pointed out that the vision for D1 was right, but D2 ruined it by adding cruft without fixing the flaws... ;-) Walter got A LOT right in his original _vision_ as represented on his original website for D1: - Taking current practice for C++ and building a better syntax for the most common patterns. - Clearly stating that a programming language should encourage you to write code that is aesthetically pleasing on the screen and make that easy. - Clearly stating that language semantics should be so simple that you didn't need a long specification for it. - Clearly stating that performance was imperative as a goal for the language and that D would not aim to replace higher level languages like C#. I can applaud to this, anyone who has exposed themselves to the annoyances of C++ can applaud to this! And D1 was a step in the right direction. A good start. The vision was lost on the way to D2, and most unfortunately the market for programming languages is a Winner Takes It All market. D2 is only a marginal improvement on C++, and worse in some areas. That can't win. I find it worrying that the evangelical D users are perceiving D as a compiled scripting language and claim it is similar to Python... D semantics are not at all like Python. That can't win. I find it worrying that the people who say they want to use D as a system programming language are into games, yet the projected vision for the D leadership now is to make it a web programming language that should ship with vibe.d. That can't win. I find it worrying that so many people attracted to D system level programming are into games, yet game development needs are ignored. That can't win. D is lucky that Rust is annoying, Go is marginal, and Nim is unknown, so people are stuck with ugly look C++ code. There is a need to move towards something beautiful, and that's not in Andrei's vision, but in the original D1 vision + the improvements proposed by Bearophile, Timon Gehr and others. Or swing 100% to Andrei's direction and improve significantly on meta programming by adding pattern matching and partial evaluation, so that you have something significantly better than C++. …but move... Remember: It's a winner takes it all game.
I get so tired of non game devs spouting off about what they think gamedevs do. Let me give you a clue, we are aware of the internet. We do process strings and JSON. Not only that but we usually do this stuff in C++, and it often sucks to write. There is really only a small fraction of game code that tends to look like low level C. There are people who spend all their time writing this kind of code, but there are tons of other programers doing other things. Farming this work out to C# isn't a realistic option at runtime, and at tool time it requires maintaining bindings. That's part of the reason D is attractive to me as a gamedev. I WANT all those high level features, I want them to be performant, and I want the ability to write low level code when necessary. D1 just doesn't cut it.
Apr 25 2015
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Saturday, 25 April 2015 at 16:55:18 UTC, bigsandwich wrote:
 I get so tired of non game devs spouting off about what they 
 think gamedevs do.  Let me give you a clue, we are aware of the 
 internet.  We do process strings and JSON.  Not only that but 
 we usually do this stuff in C++, and it often sucks to write.
Those are library issues and not language issues, even in C++. You are making to many assumptions about what "game dev" is all about as it is very different from project to proejct, but in this context it refers to the real time aspects.
Apr 25 2015
prev sibling next sibling parent reply "Andrea Fontana" <nospam example.com> writes:
On Thursday, 23 April 2015 at 08:33:56 UTC, John Colvin wrote:
 Why can no compiler I try optimise this toy example as I would 
 expect?

 // uncomment if using a C compiler
 // typedef unsigned int uint;
 uint foo(uint a)
 {
   if (a < 5)
     return (a * 3) / 3;
   else
     return 0;
 }

 So, I would expect the compiler to be able to see that it is 
 equivalent to

 uint foo(uint a)
 {
   return (a < 5) ? a : 0;
 }

 But apparently not a single modern compiler I tried can do this 
 optimisation, unless it's hidden in some obscure flag I'm not 
 aware of.

 An even more striking example can be found if you replace the / 
 with %, where the result of the function is then 
 unconditionally zero, but every compiler i tried still spat out 
 multiplication instructions.

 Is there a good reason for this, or is it just " * and / aren't 
 always inverses, so never mind all the cases where they are"?

 Now I know that this seems like a unrealistic example, but when 
 you're in complicated meta-programming situations code like 
 this can and will appear.
If I'm right, there's a website where I can see assembly generated by d compiler. And it's not dpaste... any hint?
Apr 23 2015
parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 23/04/2015 10:02 p.m., Andrea Fontana wrote:
 On Thursday, 23 April 2015 at 08:33:56 UTC, John Colvin wrote:
 Why can no compiler I try optimise this toy example as I would expect?

 // uncomment if using a C compiler
 // typedef unsigned int uint;
 uint foo(uint a)
 {
   if (a < 5)
     return (a * 3) / 3;
   else
     return 0;
 }

 So, I would expect the compiler to be able to see that it is
 equivalent to

 uint foo(uint a)
 {
   return (a < 5) ? a : 0;
 }

 But apparently not a single modern compiler I tried can do this
 optimisation, unless it's hidden in some obscure flag I'm not aware of.

 An even more striking example can be found if you replace the / with
 %, where the result of the function is then unconditionally zero, but
 every compiler i tried still spat out multiplication instructions.

 Is there a good reason for this, or is it just " * and / aren't always
 inverses, so never mind all the cases where they are"?

 Now I know that this seems like a unrealistic example, but when you're
 in complicated meta-programming situations code like this can and will
 appear.
If I'm right, there's a website where I can see assembly generated by d compiler. And it's not dpaste... any hint?
asm.dlang.org
Apr 23 2015
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Thursday, 23 April 2015 at 10:04:47 UTC, Rikki Cattermole 
wrote:
 On 23/04/2015 10:02 p.m., Andrea Fontana wrote:
 On Thursday, 23 April 2015 at 08:33:56 UTC, John Colvin wrote:
 Why can no compiler I try optimise this toy example as I 
 would expect?

 // uncomment if using a C compiler
 // typedef unsigned int uint;
 uint foo(uint a)
 {
  if (a < 5)
    return (a * 3) / 3;
  else
    return 0;
 }

 So, I would expect the compiler to be able to see that it is
 equivalent to

 uint foo(uint a)
 {
  return (a < 5) ? a : 0;
 }

 But apparently not a single modern compiler I tried can do 
 this
 optimisation, unless it's hidden in some obscure flag I'm not 
 aware of.

 An even more striking example can be found if you replace the 
 / with
 %, where the result of the function is then unconditionally 
 zero, but
 every compiler i tried still spat out multiplication 
 instructions.

 Is there a good reason for this, or is it just " * and / 
 aren't always
 inverses, so never mind all the cases where they are"?

 Now I know that this seems like a unrealistic example, but 
 when you're
 in complicated meta-programming situations code like this can 
 and will
 appear.
If I'm right, there's a website where I can see assembly generated by d compiler. And it's not dpaste... any hint?
asm.dlang.org
and d.godbolt.org This isn't a D-specific question though, so gcc.godbolt.org would allow you to test a wider range of backends.
Apr 23 2015
parent reply "Andrea Fontana" <nospam example.com> writes:
On Thursday, 23 April 2015 at 10:08:24 UTC, John Colvin wrote:
 asm.dlang.org
and d.godbolt.org This isn't a D-specific question though, so gcc.godbolt.org would allow you to test a wider range of backends.
I was wondering if compilers can optimize this: uint foo3(uint a) { return a*!(a/5); } That actually gives the same results.
Apr 23 2015
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/23/2015 3:23 AM, Andrea Fontana wrote:
 I was wondering if compilers can optimize this:

 uint foo3(uint a)
 {
    return a*!(a/5);
 }

 That actually gives the same results.
A fun article about these sorts of things: http://www.davespace.co.uk/blog/20150131-branchless-sequences.html
Apr 23 2015
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 23 April 2015 at 10:23:57 UTC, Andrea Fontana wrote:
 On Thursday, 23 April 2015 at 10:08:24 UTC, John Colvin wrote:
 asm.dlang.org
and d.godbolt.org This isn't a D-specific question though, so gcc.godbolt.org would allow you to test a wider range of backends.
I was wondering if compilers can optimize this: uint foo3(uint a) { return a*!(a/5); } That actually gives the same results.
That is cool ! However, careful, division can stall the pipeline.
Apr 23 2015
next sibling parent "Andrea Fontana" <nospam example.com> writes:
On Friday, 24 April 2015 at 06:29:55 UTC, deadalnix wrote:
 On Thursday, 23 April 2015 at 10:23:57 UTC, Andrea Fontana 
 wrote:
 On Thursday, 23 April 2015 at 10:08:24 UTC, John Colvin wrote:
 asm.dlang.org
and d.godbolt.org This isn't a D-specific question though, so gcc.godbolt.org would allow you to test a wider range of backends.
I was wondering if compilers can optimize this: uint foo3(uint a) { return a*!(a/5); } That actually gives the same results.
That is cool ! However, careful, division can stall the pipeline.
I'm not an assembly expert but it seems that some compilers (using godbolt, asm.dlang.org) optimize it as: return a*(a<=4) gcc: foo3(unsigned int): xor eax, eax cmp edi, 4 setbe al imul eax, edi ret gdc: uint foo2(uint a) { return a*(a<5); } uint foo3(uint a) { return a*!(a/5); } uint example.foo2(uint): xorl %eax, %eax cmpl $4, %edi setbe %al imull %edi, %eax ret uint example.foo3(uint): movl %edi, %eax movl $-858993459, %edx mull %edx xorl %eax, %eax shrl $2, %edx testl %edx, %edx sete %al imull %edi, %eax ret
Apr 24 2015
prev sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Friday, 24 April 2015 at 06:29:55 UTC, deadalnix wrote:
 On Thursday, 23 April 2015 at 10:23:57 UTC, Andrea Fontana 
 wrote:
 On Thursday, 23 April 2015 at 10:08:24 UTC, John Colvin wrote:
 asm.dlang.org
and d.godbolt.org This isn't a D-specific question though, so gcc.godbolt.org would allow you to test a wider range of backends.
I was wondering if compilers can optimize this: uint foo3(uint a) { return a*!(a/5); } That actually gives the same results.
That is cool ! However, careful, division can stall the pipeline.
No optimiser worth it's salt is going to emit a idiv instruction there.
Apr 24 2015
prev sibling next sibling parent "J" <j a.de> writes:
On Thursday, 23 April 2015 at 08:33:56 UTC, John Colvin wrote:
 Why can no compiler I try optimise this toy example as I would 
 expect?

 // uncomment if using a C compiler
 // typedef unsigned int uint;
 uint foo(uint a)
 {
   if (a < 5)
     return (a * 3) / 3;
   else
     return 0;
 }
It is interesting to see how things do get optimized when e.g. changing the multiply * into a plus +.
Apr 23 2015
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/23/2015 1:33 AM, John Colvin wrote:
 Is there a good reason for this,
Modern compilers check for literally thousands of such patterns, and more are constantly added. There's an infinitely "long tail" of these patterns possible, and at some point, you have to ship the compiler. The ones that get added are the ones that, unsurprisingly, are brought to the compiler writers' attention, usually via a benchmark example that they are losing :-)
Apr 23 2015