www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Jai compiles 80,000 lines of code in under a second

reply aliak <something something.com> writes:
Alo!

I just watched this talk from Jonathan Blow [0] about his 
programming language called Jai, and he can now compile an 80,000 
line game in about 1.5 seconds on a laptop (of course I have no 
idea what laptop he's using), under 1 second on a desktop.

And he claims it's he wants to eventually hit compilation of 
1,000,000 lines per second and that he thinks that's a realistic 
goal.

I just tried compiling an optional utility library that has a 
grand total of 1794 lines in .d files and:

time dmd -c -debug -g -unittest -w -Isource/ -Itests  
-I../../.dub/packages/bolts-0.7.1/bolts/source/ 
source/optional/dispatch.d source/optional/internal.d 
source/optional/notnull.d source/optional/optional.d 
source/optional/package.d source/optional/traits.d 
tests/dispatch.d tests/match.d tests/notnull.d tests/optional.d 
tests/orelse.d tests/unwrap.d

1.92s user 0.23s system 99% cpu 2.157 total (2.6ghz, 4 cores, 
plenty ram)

Have compile times gotten worse in D over the years or better or 
just stayed the same? And is there anyway to get even near the 
performance of Jai when it comes to compilations (parallelize 
stuff here and there maybe)? Or is DMD in a state where that is 
just not feasible?

On a related note: He also mentions some really cool compilation 
features like having compiler hooks that tell you when 
compilation is done, when executable and where it will be written 
so you can create your build recipe inside the program itself. 
Also allows you do do things like:

whenCompilationFinishes(exeLocation) => 
loadExecutableIcon(myIcon, exeLocation)

During the build!

Your source knows how to build itself as a concept is awesome! 
There's actually a D runner [1] that kind of allows for source 
files to set stuff up.

[0] https://www.youtube.com/watch?v=uZgbKrDEzAs
[1] https://github.com/marler8997/rund
Sep 20 2018
next sibling parent Neia Neutuladh <neia ikeran.org> writes:
On Thursday, 20 September 2018 at 23:13:38 UTC, aliak wrote:

 Alo!

 I just watched this talk from Jonathan Blow [0] about his 
 programming language called Jai, and he can now compile an 
 80,000 line game in about 1.5 seconds on a laptop (of course I 
 have no idea what laptop he's using), under 1 second on a 
 desktop.


Jai is in the hands of maybe a dozen people and , so it's hard to 
compare. But with a sufficiently simple language with no 
metaprogramming, 80k lines of code in 1.5 seconds seems doable.

And in that situation, dmd does just fine -- 0.73 seconds to 
compile 84k lines of simple generated code on i5 2400, or 0.20 
seconds with -c -o-.

It's just that D code tends toward heavy metaprogramming. That's 
a lot safer (consider C-style varargs writefln versus the 
template version), and it's slower to compile.


 On a related note: He also mentions some really cool 
 compilation features like having compiler hooks that tell you 
 when compilation is done, when executable and where it will be 
 written so you can create your build recipe inside the program 
 itself. Also allows you do do things like:

 whenCompilationFinishes(exeLocation) => 
 loadExecutableIcon(myIcon, exeLocation)

 During the build!

 Your source knows how to build itself as a concept is awesome! 
 There's actually a D runner [1] that kind of allows for source 
 files to set stuff up.


It's awesome for demos and terrible otherwise. It takes "it 
builds on my machine" to a new level.
Sep 20 2018
prev sibling next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 20 September 2018 at 23:13:38 UTC, aliak wrote:

 he can now compile an 80,000 line game in about 1.5 seconds on 
 a laptop


D can compile similar amounts of code in half the time.

For example, the entire D1 runtime and standard library can be 
built (compiled and linked!) in 0.6 seconds on my computer, and 
it is about 82,000 lines of code.

A good chunk of my gui libs in D: terminal.d, simpledisplay.d, 
minigui.d, nanovega.d, color.d, and dom.d for good measure, can 
be compiled in 1 second on my computer. That is ~49,000 lines of 
code. Back when I didn't use phobos in them, it compiled in about 
1/3 that time - see, that's the hidden cost of builds: you 
frequently need to compile parts of the standard library too. In 
C++, this is caused by #include (sort of, C++ mitigates it in 
practice though). In D, it is templates. Any templates you use 
from the stdlib will be compiled and instantiated too - and this 
is slow.


Of course, D can also take ages to compile one line of code. It 
all depends on that that line is doing... ctfe and templates are 
slow. C or Java style code compiling in D is very fast.


 Have compile times gotten worse in D over the years or better 
 or just stayed the same?


Well, if you still compile D code written in and older style - 
like my example of the D1 phobos - it still builds exceedingly 
quickly. Just that older style is less common  nowadays - using 
Phobos function is frequently the slowest part of compiling my 
code with D2, whereas D1 it was all written C style, and would 
build in the blink of an eye.

So dmd hasn't gotten much slower, but the typical D style has 
moved toward using more of the slower parts of the compiler 
instead of the faster parts.
Sep 20 2018
parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 21 September 2018 at 00:47:27 UTC, Adam D. Ruppe wrote:

 Of course, D can also take ages to compile one line of code. It 
 all depends on that that line is doing... ctfe and templates 
 are slow. C or Java style code compiling in D is very fast.


I was going to say this too, ie how much of that Jai code is run 
at compile-time, how much is uninstantiated templates that is 
just skipped over like D does, and how much is templates 
instantiated many times? Lines of code is not a good enough 
measure with those programming constructs.

I was just building the stdlib tests with LDC yesterday and they 
took so much memory on a new Linux/x64 VPS with 2GB of RAM that I 
had spun up that I couldn't even ssh in anymore. I eventually had 
to restart the VPS and add a swapfile, which I usually have but 
simply hadn't bothered with yet for this new Ubuntu 18.04 VPS. 
The stdlib tests instantiate a ton of templates.
Sep 20 2018
parent reply aliak <something something.com> writes:
On Friday, 21 September 2018 at 01:04:51 UTC, Joakim wrote:

 On Friday, 21 September 2018 at 00:47:27 UTC, Adam D. Ruppe 
 wrote:

 Of course, D can also take ages to compile one line of code. 
 It all depends on that that line is doing... ctfe and 
 templates are slow. C or Java style code compiling in D is 
 very fast.


 I was going to say this too, ie how much of that Jai code is 
 run at compile-time, how much is uninstantiated templates that 
 is just skipped over like D does, and how much is templates 
 instantiated many times? Lines of code is not a good enough 
 measure with those programming constructs.

 I was just building the stdlib tests with LDC yesterday and 
 they took so much memory on a new Linux/x64 VPS with 2GB of RAM 
 that I had spun up that I couldn't even ssh in anymore. I 
 eventually had to restart the VPS and add a swapfile, which I 
 usually have but simply hadn't bothered with yet for this new 
 Ubuntu 18.04 VPS. The stdlib tests instantiate a ton of 
 templates.


Sure, all true, but from what I've seen of Jai, it's not a simple 
language, and it does a decent amount of compile time stuff, but 
who knows, maybe the code is simple indeed. I remember a demo 
where he ran a game at compile time and was also fast AFAIR. I 
think that his goal is to keep it fast regardless of which 
features are used though. I hope.

Regardless, you can't really claim X compiles fast if that's only 
true on a subset of the language features. Cause otherwise the 
statement "X compiles fast" is, well, just not true ;)
Sep 21 2018
parent Neia Neutuladh <neia ikeran.org> writes:
On Friday, 21 September 2018 at 13:28:47 UTC, aliak wrote:

 Sure, all true, but from what I've seen of Jai, it's not a 
 simple language, and it does a decent amount of compile time 
 stuff, but who knows, maybe the code is simple indeed. I 
 remember a demo where he ran a game at compile time and was 
 also fast AFAIR. I think that his goal is to keep it fast 
 regardless of which features are used though. I hope.


We don't have access to the source code being tested. We don't 
have access to the compiler. Until the language is actually made 
public, we can't make any substantive conclusions about its speed.
Sep 21 2018
prev sibling next sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 09/20/2018 07:13 PM, aliak wrote:

 
 On a related note: He also mentions some really cool compilation 
 features like having compiler hooks that tell you when compilation is 
 done, when executable and where it will be written so you can create 
 your build recipe inside the program itself. Also allows you do do 
 things like:
 
 whenCompilationFinishes(exeLocation) => loadExecutableIcon(myIcon, 
 exeLocation)
 
 During the build!
 
 Your source knows how to build itself as a concept is awesome! There's 
 actually a D runner [1] that kind of allows for source files to set 
 stuff up.
 
 [0] https://www.youtube.com/watch?v=uZgbKrDEzAs
 [1] https://github.com/marler8997/rund


Just watched that part of the video. It sounds like it really just boils 
down to these three things:

1. He did away with C/C++'s awful header system, just like every other 
language out there.

(It's a good thing, but it's also a bare minimum expectation in ANY 
language that isn't C or C++. Yawn.)

2. He created a standard (and presumably mandatory) build tool to go 
along with his language, but is avoiding *calling* it a build 
tool...even though it's clearly just a build tool that happens to be 
built into the compiler.

(I've already used a language that did that: Haxe. There are definitely 
some nice things about it, but the problem is: What happens when the 
official buildsystem doesn't fit your needs? Your pretty much SOL. Just 
imagine if DUB was the ONLY way to compile D and you get the picture of 
why this isn't as great as it sounds. And even if his compiler DOES let 
you opt-out of the official built-in buildsystem, it's ultimately still 
just yet another buildsystem, nothing inherently special or new except 
for...)

3. You can embed your buildscript into one of your project's existing 
source files, instead of putting it in a dedicated buildscript file.

(We can do that in D too, by utilizing version identifiers, but we don't 
because its messy and mostly pointless. Just like you wouldn't put your 
graphics code in an XML-parsing module, or your JSON serializer in an 
audio-mixing module. They each get their own file or dir. Basic 
separation of concerns. For one-file scripts, this IS actually really, 
really nice though, which is why I like both rund and DUB's embedded 
dub.sdl. But aside from one-file scripts, there's not much point to 
avoiding basic separation-of-concerns.)
Sep 20 2018
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/20/2018 7:44 PM, Nick Sabalausky (Abscissa) wrote:

 3. You can embed your buildscript into one of your project's existing source 
 files, instead of putting it in a dedicated buildscript file.
 
 (We can do that in D too, by utilizing version identifiers, but we don't
because 
 its messy and mostly pointless. Just like you wouldn't put your graphics code
in 
 an XML-parsing module, or your JSON serializer in an audio-mixing module. They 
 each get their own file or dir. Basic separation of concerns. For one-file 
 scripts, this IS actually really, really nice though, which is why I like both 
 rund and DUB's embedded dub.sdl. But aside from one-file scripts, there's not 
 much point to avoiding basic separation-of-concerns.)


In D1, you could embed D code in HTML files. It was a cool, and completely 
useless, feature.
Sep 20 2018
prev sibling parent reply mate <aiueo aiueo.aiueo> writes:
On Friday, 21 September 2018 at 02:44:57 UTC, Nick Sabalausky 
(Abscissa) wrote:

 2. He created a standard (and presumably mandatory) build tool 
 to go along with his language, but is avoiding *calling* it a 
 build tool...even though it's clearly just a build tool that 
 happens to be built into the compiler.



 3. You can embed your buildscript into one of your project's 
 existing source files, instead of putting it in a dedicated 
 buildscript file.


I understand that the build tool is the compiler itself, 
presumably providing some functions/hooks for the build script, 
which is part of program itself written in the language itself. 
It seems to me that this approach is superior to the usual build 
systems in many ways: complete control of the build, no external 
program/lib needed apart from compiler, no other language to 
learn.

Note that the build can be done at compile time because the 
metaprogramming capabilities of the language are not limited in 
terms of system calls.
Sep 20 2018
next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Friday, 21 September 2018 at 05:11:32 UTC, mate wrote:

 Note that the build can be done at compile time because the 
 metaprogramming capabilities of the language are not limited in 
 terms of system calls.


Good luck bisecting that code base when any version of it did 
anything even mildly specific to the author's PC.
Sep 20 2018
next sibling parent reply mate <aiueo aiueo.aiueo> writes:
On Friday, 21 September 2018 at 05:39:35 UTC, Vladimir Panteleev 
wrote:

 On Friday, 21 September 2018 at 05:11:32 UTC, mate wrote:

 Note that the build can be done at compile time because the 
 metaprogramming capabilities of the language are not limited 
 in terms of system calls.


 Good luck bisecting that code base when any version of it did 
 anything even mildly specific to the author's PC.


Indeed. I am actually not sure if there really are no limitations 
to Jai’s CTFE, in its current state. There are probably 
facilities in the stdlib to avoid the need for doing system 
specific things; also the build instructions would hopefully be 
contained in some function/file either by convention or as 
required by the compiler, limiting the scope of build debugging.

Moreover, I got the feeling that the language is geared towards 
“good programmers” and is less concerned by mistakes happening 
because the author did something stupid.
Sep 20 2018
parent Guillaume Piolat <spam smam.org> writes:
On Friday, 21 September 2018 at 06:02:26 UTC, mate wrote:

 I am actually not sure if there really are no limitations to 
 Jai’s CTFE, in its current state.


What I like with unrestricted CTFE is that it makes something 
that was completely safe a security problem.
Sep 21 2018
prev sibling parent aliak <something something.com> writes:
On Friday, 21 September 2018 at 05:39:35 UTC, Vladimir Panteleev 
wrote:

 On Friday, 21 September 2018 at 05:11:32 UTC, mate wrote:

 Note that the build can be done at compile time because the 
 metaprogramming capabilities of the language are not limited 
 in terms of system calls.


 Good luck bisecting that code base when any version of it did 
 anything even mildly specific to the author's PC.


Where your build system lives makes zero difference to bisecting. 
You can have author-PC specific behavior in the build recipe 
whether that's in a source file or a "build script".

I guess it would be more compartmentalized though. But being able 
to say "the code here needs this feature" (which is not something 
you can do when the code doesn't know how to compile itself) 
could seems pretty useful.
Sep 21 2018
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/20/2018 10:11 PM, mate wrote:

 Note that the build can be done at compile time because the metaprogramming 
 capabilities of the language are not limited in terms of system calls.


Back in the naive olden days, Microsoft released ActiveX, where a web page
could 
load executable objects (!) from the internet and run them in the browser.

It quickly became apparent that this was a disaster, as lots of people on the 
internet aren't to be trusted.

CTFE on D doesn't allow making any system calls. This is on purpose.
Sep 20 2018
parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Friday, 21 September 2018 at 06:30:40 UTC, Walter Bright wrote:

 On 9/20/2018 10:11 PM, mate wrote:

 Note that the build can be done at compile time because the 
 metaprogramming capabilities of the language are not limited 
 in terms of system calls.


 Back in the naive olden days, Microsoft released ActiveX, where 
 a web page could load executable objects (!) from the internet 
 and run them in the browser.

 It quickly became apparent that this was a disaster, as lots of 
 people on the internet aren't to be trusted.

 CTFE on D doesn't allow making any system calls. This is on 
 purpose.


The usual argument against this is that source code distributions 
already usually include some sort of build or installation script 
(be it in the form of "configure", or a makefile, or a Visual 
Studio project), which can already execute arbitrary commands.

The problem with putting it in the compiler is that it 
invalidates many contracts (and, thus, use cases) about what 
invoking the compiler can do. This means you can't bisect or 
reduce (as with Dustmite) the source code reliably. Reproducible 
builds are out too, as the produced object file is no longer 
purely a function of the source code and compiler version.
Sep 20 2018
next sibling parent reply mate <aiueo aiueo.aiueo> writes:
On Friday, 21 September 2018 at 06:34:47 UTC, Vladimir Panteleev 
wrote:

 The problem with putting it in the compiler is that it 
 invalidates many contracts (and, thus, use cases) about what 
 invoking the compiler can do. This means you can't bisect or 
 reduce (as with Dustmite) the source code reliably.


I am not able to see the difference it makes. Normally when you 
bisect you build the program to test using the build system. Is 
not it equivalent to what the Jai compiler would do?
What cases do you have in mind?


 Reproducible builds are out too, as the produced object file is 
 no longer purely a function of the source code and compiler 
 version.


It depends on the developer not doing anything stupid in the 
build instructions, be it compiler-executed or not. Doesn’t it?
Sep 21 2018
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/21/2018 12:19 AM, mate wrote:

 It depends on the developer not doing anything stupid


Aye, there's the rub!
Sep 21 2018
next sibling parent reply mate <aiueo aiueo.aiueo> writes:
On Friday, 21 September 2018 at 07:37:14 UTC, Walter Bright wrote:

 On 9/21/2018 12:19 AM, mate wrote:

 It depends on the developer not doing anything stupid


 Aye, there's the rub!


;-)

Different sensibilities on where to put restrictions clearly lead 
to different designs. I am not sure myself what is best.

I agree that one would need to realize that compiling a program 
could potentially be harmful, and that could be a significant 
change in one’s habits.
Sep 21 2018
parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Friday, 21 September 2018 at 07:58:16 UTC, mate wrote:

 Different sensibilities on where to put restrictions clearly 
 lead to different designs. I am not sure myself what is best.


The more people you have on your team, the more you appreciate 
the restrictions. If you are working on a personal project alone, 
you are in control and have full knowledge of the entire 
codebase, so restrictions are a hindrance. When you are 
collaborating with someone you know only by name from across the 
globe, being able to reason what their code might or may not do 
is considerably helpful.
Sep 21 2018
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Sep 21, 2018 at 10:53:39AM +0000, Vladimir Panteleev via Digitalmars-d
wrote:

 On Friday, 21 September 2018 at 07:58:16 UTC, mate wrote:

 Different sensibilities on where to put restrictions clearly lead to
 different designs. I am not sure myself what is best.


 
 The more people you have on your team, the more you appreciate the
 restrictions. If you are working on a personal project alone, you are
 in control and have full knowledge of the entire codebase, so
 restrictions are a hindrance. When you are collaborating with someone
 you know only by name from across the globe, being able to reason what
 their code might or may not do is considerably helpful.


+100.  Many things I could get away with in my own personal projects, I
wouldn't do in a team project (which is basically *any* non-trivial
project these days).  Unrestricted freedom to do whatever you want
greatly reduces the ability to reason about the code, which is why these
days structured programming constructs like if/else, while-loops,
functions, etc., are preferred over unrestricted goto's, even though
they are technically "more restrictive".

The challenge is in finding the balance between restriction and not
hampering the programmer's ability to express what he wants without
jumping through hoops (Java's verbosity comes to mind... although, to be
fair, given your typical "enterprise" development environment, this is
not necessarily a bad thing, since it forces even bad code to conform to
a certain predictable structure, which makes it easier to rewrite said
bad code :-P when one of your coworkers turns out to be a cowboy
programmer).  Not an easy balance to strike, which is why designing a
successful programming language is so hard.


T

-- 
Unix is my IDE. -- Justin Whear
Sep 21 2018
prev sibling parent bachmeier <no spam.net> writes:
On Friday, 21 September 2018 at 07:37:14 UTC, Walter Bright wrote:

 On 9/21/2018 12:19 AM, mate wrote:

 It depends on the developer not doing anything stupid


 Aye, there's the rub!


The evolution of programming language discussions from 
"sufficiently smart compiler" to "sufficiently smart programmer 
using a sufficiently smart compiler".
Sep 21 2018
prev sibling parent reply mate <aiueo aiueo.aiueo> writes:
On Friday, 21 September 2018 at 07:19:41 UTC, mate wrote:


 Reproducible builds are out too, as the produced object file 
 is no longer purely a function of the source code and compiler 
 version.


 It depends on the developer not doing anything stupid in the 
 build instructions, be it compiler-executed or not. Doesn’t it?


I realize that with build instructions written in unrestricted 
language it is easier to create a dependency on something else 
than the compiler, such as the OS. Maybe they plan to solve this 
problem with appropriate facilities and discipline.

With standard build systems, the produced object file can depend 
on some specific state of the OS too (I think there were Windows 
updates influencing how VisualStudio was producing object files).
Sep 21 2018
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Sep 21, 2018 at 07:58:56AM +0000, mate via Digitalmars-d wrote:
[...]

 I realize that with build instructions written in unrestricted
 language it is easier to create a dependency on something else than
 the compiler, such as the OS. Maybe they plan to solve this problem
 with appropriate facilities and discipline.


[...]

Relying on discipline, or rather, assuming discipline on the part of
your coworker, never works, as shown by the past 20 years of failures in
software.  All it takes is for *one* person in a team of arbitrary size
to do something stupid, and the entire tower of cards comes crashing
down.  You need actual, hard restrictions guaranteed by the compiler,
not mere "programming by convention". 


T

-- 
It's bad luck to be superstitious. -- YHL
Sep 21 2018
prev sibling parent reply FromAnotherPlanet <fromanotherplanet fromanotherplanet.com> writes:
On Friday, 21 September 2018 at 06:34:47 UTC, Vladimir Panteleev 
wrote:

 On Friday, 21 September 2018 at 06:30:40 UTC, Walter Bright 
 wrote:

 On 9/20/2018 10:11 PM, mate wrote:

 Note that the build can be done at compile time because the 
 metaprogramming capabilities of the language are not limited 
 in terms of system calls.


 Back in the naive olden days, Microsoft released ActiveX, 
 where a web page could load executable objects (!) from the 
 internet and run them in the browser.

 It quickly became apparent that this was a disaster, as lots 
 of people on the internet aren't to be trusted.

 CTFE on D doesn't allow making any system calls. This is on 
 purpose.


 The usual argument against this is that source code 
 distributions already usually include some sort of build or 
 installation script (be it in the form of "configure", or a 
 makefile, or a Visual Studio project), which can already 
 execute arbitrary commands.

 The problem with putting it in the compiler is that it 
 invalidates many contracts (and, thus, use cases) about what 
 invoking the compiler can do. This means you can't bisect or 
 reduce (as with Dustmite) the source code reliably. 
 Reproducible builds are out too, as the produced object file is 
 no longer purely a function of the source code and compiler 
 version.


The counter argument is that the compiler is generally not a 
vector for a computer virus. If I notice I have malware on my 
computer, I'll think of the websites and the installer scripts 
I've run recently. I would never expect that some arbitrary code 
I included in a project would have created a virus and installed 
it while the compiler built. Sounds like a potential disaster to 
me.

I frequently copy and paste code I don't yet understand all of 
the time in hopes of understanding it by running it through a 
compiler. I will personally not ever use Jai.
Sep 23 2018
parent FromAnotherPlanet <fromanotherplanet fromanotherplanet.com> writes:
On Sunday, 23 September 2018 at 21:15:17 UTC, FromAnotherPlanet 
wrote:

 On Friday, 21 September 2018 at 06:34:47 UTC, Vladimir 
 Panteleev wrote:

 On Friday, 21 September 2018 at 06:30:40 UTC, Walter Bright 
 wrote:

 On 9/20/2018 10:11 PM, mate wrote:

 Note that the build can be done at compile time because the 
 metaprogramming capabilities of the language are not limited 
 in terms of system calls.


 Back in the naive olden days, Microsoft released ActiveX, 
 where a web page could load executable objects (!) from the 
 internet and run them in the browser.

 It quickly became apparent that this was a disaster, as lots 
 of people on the internet aren't to be trusted.

 CTFE on D doesn't allow making any system calls. This is on 
 purpose.


 The usual argument against this is that source code 
 distributions already usually include some sort of build or 
 installation script (be it in the form of "configure", or a 
 makefile, or a Visual Studio project), which can already 
 execute arbitrary commands.

 The problem with putting it in the compiler is that it 
 invalidates many contracts (and, thus, use cases) about what 
 invoking the compiler can do. This means you can't bisect or 
 reduce (as with Dustmite) the source code reliably. 
 Reproducible builds are out too, as the produced object file 
 is no longer purely a function of the source code and compiler 
 version.


 The counter argument is that the compiler is generally not a 
 vector for a computer virus. If I notice I have malware on my 
 computer, I'll think of the websites and the installer scripts 
 I've run recently. I would never expect that some arbitrary 
 code I included in a project would have created a virus and 
 installed it while the compiler built. Sounds like a potential 
 disaster to me.

 I frequently copy and paste code I don't yet understand all of 
 the time in hopes of understanding it by running it through a 
 compiler. I will personally not ever use Jai.


Just to add to this previous comment...just imagine my worries 
but having to worry about an entire corporation worth of 
developers potentially doing something stupid and spreading it to 
everyone else via source control and the Jai compiler...
Sep 23 2018
prev sibling next sibling parent Guillaume Piolat <spam smam.org> writes:
On Thursday, 20 September 2018 at 23:13:38 UTC, aliak wrote:

 you can create your build recipe inside the program


But this is not a particularly good idea and is even against the 
times.

Everyone is moving from powerful languages like makefiles to 
_less_ powerful languages (like dub.json) to describe how 
programs are built. Why is that so?

The reasons we have dpldocs.info, dub test, dub build etc. are 
entirely because we use a _restricted_ DSL to build D programs. 
If we were all doing makefiles, it's easy to see there would be 
no common structure hence no automated doc generation, testing 
etc. Like a C project!

There is a shift from imperative to declarative for build recipes 
in all other modern languages and Jai has made (yet another) 
wrong choice.
Sep 21 2018
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 20 September 2018 at 23:13:38 UTC, aliak wrote:

 Alo!


I have been watching Jonathan Blow's Jai for a while myself. 
There are many interesting ideas there, and many of them are what 
made me like D so much in the first place. It's very important to 
note that the speed claims he has been making are all a matter of 
developer discipline. You can have an infinite loop executed at 
compile-time in both D and Jai. There's nothing magical Jai can 
do about that - the infinite loop is not going to finish faster 
;) You can optimize the speed of compile-time computation just 
like you can optimize for run-time speed.

What your observing with D is that right now many libraries 
including Phobos have tried to see how much they can push the 
language (to make for more expressive code or faster run-time) 
and not as much time has been spent on optimizing compile-time. 
If you take a code-base written in Java-like subset of the 
language, I can grantee you that DMD is going to very competitive 

considering that there are many places that could be optimized 
internally in DMD. But overall most of the time spent compiling D 
programs is: a) crazy template / CTFE meta-programming and b) 
inefficient build process (no parallel compilation for 
non-separate compilation, no wide-spread use of incremental 
compilation, etc.). AFAIR, there were several projects for a 
caching D compiler and that can go a long way to improve things.

With а build system like reggae[0] written in the same language 
as the one being compiled, the line between compile-time vs 
run-time becomes quite blurred. If the build system part of your 
project compiles fast enough, ultimately it doesn't matter if it 
runs at compile-time vs run-time. The only important part is 
whether the build system is pleasant to work with - e.g. having a 
concise declarative syntax that covers 80% of the cases while 
also exposing a procedural interface for the difficult parts that 
don't fit in the nice model. And all nice declarative 
abstractions have a procedural implementations that one needs 
write first.

On the other hand, there are things that are much better done at 
compile-time, rather than run-time like traditional 
meta-programming. My biggest gripe with D is that currently you 
only have tools for declaration-level meta-programming (version, 
static if, static foreach, mixin template), but nothing else than 
plain strings for statement-level meta-programming. CTFE is 
great, but why re-implement the compiler in CTFE code, while the 
actual compiler is sitting right there compiling your whole 
program ;)

P.S.

Jai:
loadExecutableIcon(myIcon, exeLocation)

D:
static immutable ubyte[] icon = import("image.png).decodePng;

(In D you have read-only access to the file-system at 
compile-time using the -J flag.)

[0]: https://github.com/atilaneves/reggae
Sep 21 2018
next sibling parent reply aliak <something something.com> writes:
On Friday, 21 September 2018 at 09:21:34 UTC, Petar Kirov 
[ZombineDev] wrote:

 On Thursday, 20 September 2018 at 23:13:38 UTC, aliak wrote:

 Alo!


 I have been watching Jonathan Blow's Jai for a while myself. 
 There are many interesting ideas there, and many of them are 
 what made me like D so much in the first place. It's very 
 important to note that the speed claims he has been making are 
 all a matter of developer discipline. You can have an infinite 
 loop executed at compile-time in both D and Jai. There's 
 nothing magical Jai can do about that - the infinite loop is 
 not going to finish faster ;) You can optimize the speed of 
 compile-time computation just like you can optimize for 
 run-time speed.


Haha well, yes of course, can't argue with that :p I guess it 
makes more sense to compare the "intuitive" coding path of a 
given language. Eg: if I iterate a million objects in a for loop, 
because i want to process them, there is no other non-compile 
time way to do that. If language X takes an hour and language Y 
takes a millisecond, I'm pretty sure language X can't say it 
compiles fast, as that just seems like a pretty common-scenario 
and is not using the language in any way it was not meant to be 
used.


 What your observing with D is that right now many libraries 
 including Phobos have tried to see how much they can push the 
 language (to make for more expressive code or faster run-time) 
 and not as much time has been spent on optimizing compile-time. 
 If you take a code-base written in Java-like subset of the 
 language, I can grantee you that DMD is going to very 

 that's considering that there are many places that could be 
 optimized internally in DMD. But overall most of the time spent 
 compiling D programs is: a) crazy template / CTFE 
 meta-programming and b) inefficient build process (no parallel 
 compilation for non-separate compilation, no wide-spread use of 
 incremental compilation, etc.). AFAIR, there were several 
 projects for a caching D compiler and that can go a long way to 
 improve things.


Ah I see. Ok so there's quite a bit of big wins it seems 
(parallelization e.g.).


 On the other hand, there are things that are much better done 
 at compile-time, rather than run-time like traditional 
 meta-programming. My biggest gripe with D is that currently you 
 only have tools for declaration-level meta-programming 
 (version, static if, static foreach, mixin template), but 
 nothing else than plain strings for statement-level 
 meta-programming. CTFE is great, but why re-implement the 
 compiler in CTFE code, while the actual compiler is sitting 
 right there compiling your whole program ;)


Yeah I've always wondered this. But I just boiled it down to me 
not understanding how compilers work :)


 P.S.

 Jai:
 loadExecutableIcon(myIcon, exeLocation)

 D:
 static immutable ubyte[] icon = import("image.png).decodePng;


Si si, but i believe the loadExecutableIcon actually calls 
windows APIs to set an icon on an executable, and they'd probably 
 system which means I don't think that could be done in D.


 (In D you have read-only access to the file-system at 
 compile-time using the -J flag.)

 [0]: https://github.com/atilaneves/reggae
Sep 21 2018
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 21 September 2018 at 13:37:58 UTC, aliak wrote:

 Si si, but i believe the loadExecutableIcon actually calls 
 windows APIs to set an icon on an executable, and they'd 
 probably  system which means I don't think that could be done 
 in D.


You don't need an API call to do that. You just provide the icon 
in a resource to the linker or a separate resource thing. Some 
C++ environments do it via pragmas, or you can do it 
traditionally in a makefile/build command line pretty easily; no 
need to run fancy code.
Sep 21 2018
parent SashaGreat <s g.com> writes:
On Friday, 21 September 2018 at 18:20:21 UTC, Adam D. Ruppe wrote:

 You don't need an API call to do that. You just provide the 
 icon in a resource to the linker or a separate resource thing. 
 Some C++ environments do it via pragmas, or you can do it 
 traditionally in a makefile/build command line pretty easily; 
 no need to run fancy code.


Jonathan created a built-in function to display an icon to work 
cross-platform without the trouble configuring for each different 
platform or IDE.

SG.
Sep 21 2018
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Friday, 21 September 2018 at 09:21:34 UTC, Petar Kirov 
[ZombineDev] wrote:

 I have been watching Jonathan Blow's Jai for a while myself. 
 There are many interesting ideas there, and many of them are 
 what made me like D so much in the first place. It's very 
 important to note that the speed claims he has been making are 
 all a matter of developer discipline. You can have an infinite 
 loop executed at compile-time in both D and Jai.


You're going to OOM pretty fast in D if you try :)
Sep 21 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/21/18 10:19 AM, Nicholas Wilson wrote:

 On Friday, 21 September 2018 at 09:21:34 UTC, Petar Kirov [ZombineDev] 
 wrote:

 I have been watching Jonathan Blow's Jai for a while myself. There are 
 many interesting ideas there, and many of them are what made me like D 
 so much in the first place. It's very important to note that the speed 
 claims he has been making are all a matter of developer discipline. 
 You can have an infinite loop executed at compile-time in both D and Jai.


 
 You're going to OOM pretty fast in D if you try :)


I can see the marketing now, "D finds infinite loops in compile-time 
code way faster than Jai!".

-Steve
Sep 21 2018
parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/21/2018 7:46 AM, Steven Schveighoffer wrote:

 I can see the marketing now, "D finds infinite loops in compile-time code way 
 faster than Jai!".


We need you over in marketing!
Sep 21 2018
prev sibling parent reply welkam <wwwelkam gmail.com> writes:
On Thursday, 20 September 2018 at 23:13:38 UTC, aliak wrote:

 
 And is there anyway to get even near the performance of Jai 
 when it comes to compilations


I watched the same video today. What a coincidence.

In Jai example 80 000 lines of "code" include comments and empty 
lines. Since we know that that Jai example was written in 
parallel to language we can safely assume that most of that code 
is simple therefore its not surprising that Jai compiled that 
fast. Write C style code and DMD will perform similarly.

Can we improve D compiler speed? Ofcourse but core developers are 
more focused on stability and very needed functionality than 
speed. Thats good because I rather have c++ interop than 10% 
faster compilation speed.

Jai compiler perform parsing and lexing in different thread so 
its kinda multi threaded. Its possible to do the same with D 
front end. We can start here but there are plenty of low hanging 
fruits in compiler you just need to run profiler to find them
Sep 21 2018
parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/21/2018 9:29 AM, welkam wrote:

 Jai compiler perform parsing and lexing in different thread so its kinda multi 
 threaded. Its possible to do the same with D front end. We can start here but 
 there are plenty of low hanging fruits in compiler you just need to run
profiler 
 to find them


D was designed to support mulithreaded compilation, but that was never
implemented.

An earlier DMD would do file I/O and compiling in separate threads. It was
sadly 
removed.
Sep 21 2018