www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - GoingNative 6: The D Episode with Walter Bright and Andrei Alexandrescu

reply Walter Bright <newshound2 digitalmars.com> writes:
http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu
Feb 21 2012
next sibling parent Caligo <iteronvexor gmail.com> writes:
Thanks Walter.  Thanks Andrei.  You too, smiley face.

On Tue, Feb 21, 2012 at 6:39 PM, Walter Bright
<newshound2 digitalmars.com> wrote:

 http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu
Feb 21 2012
prev sibling next sibling parent Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Walter Bright wrote:

 http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu


Great talk!
Feb 21 2012
prev sibling next sibling parent Juan Manuel Cabo <juanmanuel.cabo gmail.com> writes:
 We want to have many users.


dUsers ~= (juanManuel);

:-) :-) :-)

--jm

On 02/21/2012 09:39 PM, Walter Bright wrote:

 http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu
Feb 21 2012
prev sibling next sibling parent reply Caligo <iteronvexor gmail.com> writes:
I thought GDC was going to be part of GCC 4.7, but Andrei in the video
said 4.8.  That's another year, :-(

On Tue, Feb 21, 2012 at 6:39 PM, Walter Bright
<newshound2 digitalmars.com> wrote:

 http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu
Feb 21 2012
parent "Jesse Phillips" <jessekphillips+D gmail.com> writes:
On Wednesday, 22 February 2012 at 02:16:31 UTC, Caligo wrote:

 I thought GDC was going to be part of GCC 4.7, but Andrei in 
 the video
 said 4.8.  That's another year, :-(


Didn`t have it ready for 4.7 so the work is planned for 4.8, but 
still not certain.
Feb 22 2012
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/21/12 6:39 PM, Walter Bright wrote:

 http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu


On reddit: 
http://www.reddit.com/r/programming/comments/q09su/c9goingnative_6_the_d_episode_with_walter_bright/


Andrei
Feb 21 2012
prev sibling next sibling parent "Nick Sabalausky" <a a.a> writes:
"Walter Bright" <newshound2 digitalmars.com> wrote in message 
news:ji1dfl$17ha$1 digitalmars.com...

 http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu


Great video.  It's espcially nice to see an entire one of these videos 
dedicated to D, not just a little cameo mention here or there. D's really 
moving forward.  :)
Feb 21 2012
prev sibling next sibling parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Walter Bright" <newshound2 digitalmars.com> wrote in message 
news:ji1dfl$17ha$1 digitalmars.com...

 http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu


Sounds like I should finish my extern(C++) branch...
Feb 22 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 22/02/2012 13:58, Daniel Murphy a écrit :

 "Walter Bright"<newshound2 digitalmars.com>  wrote in message
 news:ji1dfl$17ha$1 digitalmars.com...

 http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu


 Sounds like I should finish my extern(C++) branch...


Pleaso do ! And ask help if you feel it is needed. I'd be happy to help 
with that for instance (but ATM I'm still struggling to understand dmd 
source code).
Feb 22 2012
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter:


 http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu


Andrei says that some new languages suffer because they have a poor
implementation, because creating the base for a language is a lot of work.
Today this is issue is much less of a problem, new languages are implemented on
the JavaVM, DotNetVM. System languages are implemented with LLVM.
And maybe PyPy will be a good base to create efficient dynamic languages
quickly:
  http://tratt.net/laurie/tech_articles/articles/fast_enough_vms_in_fast_enough_time


Regarding the comparison between dynamic languages like Python or Ruby and D:
what Andrei has said is not fully fair. A simple common scripting task: read
the lines of a text file and put them in a hash. This is probably faster in
Python compared to D. I am willing to write a benchmark too, if asked.


Regarding what Walter has said, that most of the code of D applications will be
 safe: if safe code gets (or has to get) so common as he says, then it will be
good for the D compiler to learn some tricks to avoid (optimize away) array
bound tests in some cases.

Bye,
bearophile
Feb 22 2012
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/22/2012 7:51 PM, bearophile wrote:

 Andrei says that some new languages suffer because they have a poor
 implementation, because creating the base for a language is a lot of work.
 Today this is issue is much less of a problem, new languages are implemented
 on the JavaVM,


Using the JVM forces your program into Java semantics. For example, there are
no 
structs in the JVM bytecode. No pointers, either. Nor unsigned types. Your new 
language is fairly boxed in to being a rehash of Java semantics.



 DotNetVM.


Cristi's D compiler on .NET had large problems because array slicing was not 
expressible in the .NET intermediate code. It's not nearly as bad as the JVM in 
that regard, but it's still limited.



 System languages are implemented with LLVM.


That works if your language is expressible as C, because LLVM is a C/C++ back 
end. If your language has different semantics (like how Go does stacks), using 
LLVM can be a large problem.

Note that early C++ compilers suffered badly when they were forced into using C 
back ends, because C++ wanted new features (like COMDATs) which are not 
expressible in C and so not supported by C back ends. Ditto for any language 
feature that needs something in the back end that C/C++ doesn't need.



 Regarding the comparison between dynamic languages like Python or Ruby and D:
 what Andrei has said is not fully fair. A simple common scripting task: read
 the lines of a text file and put them in a hash. This is probably faster in
 Python compared to D. I am willing to write a benchmark too, if asked.


That would be comparing library code, not language code. Much of Python's 
implementation is in C, not Python.



 Regarding what Walter has said, that most of the code of D applications will
 be  safe: if safe code gets (or has to get) so common as he says, then it
 will be good for the D compiler to learn some tricks to avoid (optimize away)
 array bound tests in some cases.


I looked into this years ago. Very little of array bounds checking can be 
optimized away. I've been working on optimizers for 25 years now, including a 
native code generating Java compiler, and I do know a few things about how to
do 
arrays.

Clang has some pretty good ideas, like the spell checker on undefined 
identifiers. But others talked about in the spiel at GoingNative have been in 
compilers for 30 years.
Feb 22 2012
parent reply =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <xtzgzorex gmail.com> writes:
On 23-02-2012 07:51, Walter Bright wrote:

 On 2/22/2012 7:51 PM, bearophile wrote:

 Andrei says that some new languages suffer because they have a poor
 implementation, because creating the base for a language is a lot of
 work.
 Today this is issue is much less of a problem, new languages are
 implemented
 on the JavaVM,


 Using the JVM forces your program into Java semantics. For example,
 there are no structs in the JVM bytecode. No pointers, either. Nor
 unsigned types. Your new language is fairly boxed in to being a rehash
 of Java semantics.


I still cannot fathom how the Scala guys thought using the JVM was a 
good idea.


 DotNetVM.


 Cristi's D compiler on .NET had large problems because array slicing was
 not expressible in the .NET intermediate code. It's not nearly as bad as
 the JVM in that regard, but it's still limited.


I assume the problem here was the .ptr property? I can't think of 
anything else about array slices that would be problematic in CIL.


  > System languages are implemented with LLVM.

 That works if your language is expressible as C, because LLVM is a C/C++
 back end. If your language has different semantics (like how Go does
 stacks), using LLVM can be a large problem.



ActionScript, Python, Java, Lua, Haskell, and many others have been 
compiled with LLVM successfully.

LLVM is very much engineered for C and C++, but it has many other 
features that those languages don't make use of at all (see for example 
the precise GC support; and this is not even something Apple uses, as 
their Obj-C GC is conservative). There is also the language-specific 
support for OCaml's precise stack maps. Also, LLVM has segmented stacks 
on x86 these days. That said, LLVM is definitely not as easy to use for 
managed languages as it is for systems/native languages.

So, while the design of LLVM certainly is driven by Clang primarily, 
it's not as if they don't welcome non-C family features.


 Note that early C++ compilers suffered badly when they were forced into
 using C back ends, because C++ wanted new features (like COMDATs) which
 are not expressible in C and so not supported by C back ends. Ditto for
 any language feature that needs something in the back end that C/C++
 doesn't need.



 Regarding the comparison between dynamic languages like Python or Ruby
 and D:
 what Andrei has said is not fully fair. A simple common scripting
 task: read
 the lines of a text file and put them in a hash. This is probably
 faster in
 Python compared to D. I am willing to write a benchmark too, if asked.


 That would be comparing library code, not language code. Much of
 Python's implementation is in C, not Python.



 Regarding what Walter has said, that most of the code of D
 applications will
 be  safe: if safe code gets (or has to get) so common as he says, then it
 will be good for the D compiler to learn some tricks to avoid
 (optimize away)
 array bound tests in some cases.


 I looked into this years ago. Very little of array bounds checking can
 be optimized away. I've been working on optimizers for 25 years now,
 including a native code generating Java compiler, and I do know a few
 things about how to do arrays.


In Mono, we found that ABC removal was actually beneficial in some code; 
consider for example allocating small static arrays locally and indexing 
them with constants (or very simple expressions).


 Clang has some pretty good ideas, like the spell checker on undefined
 identifiers. But others talked about in the spiel at GoingNative have
 been in compilers for 30 years.


-- 
- Alex
Feb 23 2012
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/23/2012 8:57 AM, Alex Rønne Petersen wrote:

 System languages are implemented with LLVM.


 That works if your language is expressible as C, because LLVM is a C/C++
 back end. If your language has different semantics (like how Go does
 stacks), using LLVM can be a large problem.



 Python, Java, Lua, Haskell, and many others have been compiled with LLVM
 successfully.

 LLVM is very much engineered for C and C++, but it has many other features that
 those languages don't make use of at all (see for example the precise GC
 support; and this is not even something Apple uses, as their Obj-C GC is
 conservative). There is also the language-specific support for OCaml's precise
 stack maps. Also, LLVM has segmented stacks on x86 these days. That said, LLVM
 is definitely not as easy to use for managed languages as it is for
 systems/native languages.

 So, while the design of LLVM certainly is driven by Clang primarily, it's not
as
 if they don't welcome non-C family features.


What you're saying is they add support here and there for non-C languages, 
meaning someone has to do it if it isn't there already.



 In Mono, we found that ABC removal was actually beneficial in some code;
 consider for example allocating small static arrays locally and indexing them
 with constants (or very simple expressions).


This is done already by standard data flow optimization techniques.
Feb 23 2012
prev sibling parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 02/23/2012 11:57 AM, Alex Rønne Petersen wrote:

 I still cannot fathom how the Scala guys thought using the JVM was a
 good idea.


It gave them a good garbage collector (an area that has held D's 
performance back for years), a large library via Java compatibility, and 
a population of users to appeal to. Scala probably wouldn't have 
succeeded if it was just another object-oriented language without ties 
to the JVM.
Feb 23 2012
next sibling parent reply =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <xtzgzorex gmail.com> writes:
On 24-02-2012 05:06, Jeff Nowakowski wrote:

 On 02/23/2012 11:57 AM, Alex Rønne Petersen wrote:

 I still cannot fathom how the Scala guys thought using the JVM was a
 good idea.


 It gave them a good garbage collector (an area that has held D's
 performance back for years), a large library via Java compatibility, and
 a population of users to appeal to. Scala probably wouldn't have
 succeeded if it was just another object-oriented language without ties
 to the JVM.


They could have used the CLI. Both MS.NET and Mono have good garbage 
collectors, and Mono in particular has SIMD intrinsics, an LLVM back 
end, etc.

I've heard a lot of people complain that Scala doesn't run on .NET.

-- 
- Alex
Feb 23 2012
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/12 11:09 PM, Alex Rønne Petersen wrote:

 On 24-02-2012 05:06, Jeff Nowakowski wrote:

 On 02/23/2012 11:57 AM, Alex Rønne Petersen wrote:

 I still cannot fathom how the Scala guys thought using the JVM was a
 good idea.


 It gave them a good garbage collector (an area that has held D's
 performance back for years), a large library via Java compatibility, and
 a population of users to appeal to. Scala probably wouldn't have
 succeeded if it was just another object-oriented language without ties
 to the JVM.


 They could have used the CLI. Both MS.NET and Mono have good garbage
 collectors, and Mono in particular has SIMD intrinsics, an LLVM back
 end, etc.

 I've heard a lot of people complain that Scala doesn't run on .NET.


I think Scala has positioned itself as a Java replacement with a smooth 
transition. The JVM works on more platforms than .NET so there was a 
simple argument by numbers to be made. Probably most importantly, 
Odersky already was a Java expert motivated by needs and desires on that 
particular platform.

Andrei
Feb 24 2012
prev sibling parent Jeff Nowakowski <jeff dilacero.org> writes:
On 02/24/2012 12:09 AM, Alex Rønne Petersen wrote:

 They could have used the CLI. Both MS.NET and Mono have good garbage
 collectors, and Mono in particular has SIMD intrinsics, an LLVM back
 end, etc.

 I've heard a lot of people complain that Scala doesn't run on .NET.


Scala initially ran on .NET, but there was more traction with the Java 
crowd, so they stopped supporting it. As I understand it, .NET support 
has recently been funded by Microsoft and is actively being worked on.

There was, and still is, a much greater need for Scala on the JVM than 

getting big improvements. As for Mono, there's a lot of mistrust against 
following Microsoft's lead from the Linux crowd. So while there's some 
interest in Scala on .NET, it pales in comparison to the JVM.
Feb 24 2012
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/12 10:06 PM, Jeff Nowakowski wrote:

 On 02/23/2012 11:57 AM, Alex Rønne Petersen wrote:

 I still cannot fathom how the Scala guys thought using the JVM was a
 good idea.


 It gave them a good garbage collector (an area that has held D's
 performance back for years)


And still is, unfortunately.

Andrei
Feb 24 2012
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/22/12 9:51 PM, bearophile wrote:

 Regarding the comparison between dynamic languages like Python or
 Ruby and D: what Andrei has said is not fully fair. A simple common
 scripting task: read the lines of a text file and put them in a hash.
 This is probably faster in Python compared to D.


I assume you're referring to the point starting around 16:50. I don't 
see how fairness is an issue here. Anyway, the point there is that in my 
experience performance of short scripts in dynamic languages generally 
decays quickly when things get interesting. Of course, it comes without 
saying that choosing one particular script that exercises a very small, 
well-honed area of a toolchain and that is essentially dominated by 
library calls may show excellent performance of a scripting language.


 I am willing to write a benchmark too, if asked.


What would be better would be investigating what particular aspects of D 
we could improve (and better yet, contribute the improvements themselves).

If one were to corroborate this post with this other one:

http://forum.dlang.org/post/jhn1ub$2tni$1 digitalmars.com

the conclusion is there's a conspiracy afoot targeted at sabotaging D 
scripting :o).


Andrei
Feb 23 2012
prev sibling next sibling parent reply "MattCodr" <mattcoder hotmail.com> writes:
On Wednesday, 22 February 2012 at 00:39:17 UTC, Walter Bright 
wrote:

 http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu


You want 1 million users? Simple... is just Andrei start saying 
that he use D on FB and you will get it 10 millions instantly ! :D

Overall it's was a great Episode and you're on the right track.
Feb 23 2012
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/12 6:22 AM, MattCodr wrote:

 On Wednesday, 22 February 2012 at 00:39:17 UTC, Walter Bright wrote:

 http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-6-The-D-Episode-with-Walter-Bright-and-Andrei-Alexandrescu


 You want 1 million users? Simple... is just Andrei start saying that he
 use D on FB and you will get it 10 millions instantly ! :D

 Overall it's was a great Episode and you're on the right track.


Incidentally, we have a diff out as of yesterday that upgrades FB's dmd 
installation to 2.058. Things are set up and interest does exist, but 
there's no actual use of D at Facebook yet.

Andrei
Feb 23 2012
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Sorry for the slow reply, I was busy.

Walter:


 Using the JVM forces your program into Java semantics.


Right, a JVM doesn't give all the freedom needed by a system language.



 For example, there are no
 structs in the JVM bytecode. No pointers, either. Nor unsigned types.


The little emscripten compiler compiles LLVM "bytecode" (IR code) to
JavaScript, and despite JavaScript has no structs, it's able to compile them
efficiently, lowering each one of them to a bunch of single variables. The
result seems efficient enough (taking into account the target language is JS):
https://github.com/kripken/emscripten
This can't be used to return structs from a function, but I think allows to
avoid few heap allocations (when the struct is a function argument, or it's
used locally. The Oracle JavaVM nowadays performs escape analysis, I presume
with similar effects).

Despite there are no unsigned integers in the JVM, something is moving:
https://blogs.oracle.com/darcy/entry/unsigned_api



 Your new language is fairly boxed in to being a rehash of Java semantics.


The semantics of languages like Jruby and Scala are different from Java
semantics. Scala type system is not comparable to Java one. But I understand
what you are saying. I think there is more than one meaning for "language
semantics".



 That works if your language is expressible as C, because LLVM is a C/C++ back
 end. If your language has different semantics (like how Go does stacks), using
 LLVM can be a large problem.


LLVM is not infinitely flexible, I agree. But LLVM is already quite wide and
growing. So even for strange languages LLVM seems able to implement a large
percentage of the semantics out of the box. Recently they have implemented a
Haskell back-end using LLVM and it was not able to implement everything needed.
So the Haskell devs have written a small patch (2000 lines or so), to implement
the missing semantics and the LLVM have added it to the main LLVM code.

In the discussion Andrei was talking about the amount of code needed to
implement the efficient base of a new language. Even if LLVM is sometimes not
able to implement 100% of the semantics of your code, I think writing just the
few missing parts reduces the work needed a lot.



 I looked into this years ago. Very little of array bounds checking can be
 optimized away.


This paper (that's about Java) shows nice results in terms of percentage of
array bound checks eliminated (but as expected the speedup is visible only on
code that uses array heavily, like scientific-style code):
http://ssw.jku.at/Research/Papers/Wuerthinger07/

So there are any low-hanging fruits here?



 I've been working on optimizers for 25 years now, including a
 native code generating Java compiler, and I do know a few things about how to
do
 arrays.


You are one of my few programming heroes :-) But there's always some more to
invent and learn.



 Clang has some pretty good ideas, like the spell checker on undefined
identifiers.


This is another idea I'd like in D:
http://d.puremagic.com/issues/show_bug.cgi?id=5004

Bye,
bearophile
Feb 26 2012