www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.parallelism is accepted into Phobos

reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
The std.parallelism vote is now over, and the results are as follows:

  24 voted yes
   0 voted no

I guess that's what you call a landslide.  It is my pleasure to announce 
that std.parallelism has been accepted for inclusion in Phobos.  
Congratulations, David!

-Lars
Apr 26 2011
next sibling parent Peter Alexander <peter.alexander.au gmail.com> writes:
On 26/04/11 12:21 PM, Lars T. Kyllingstad wrote:
 The std.parallelism vote is now over, and the results are as follows:

    24 voted yes
     0 voted no

 I guess that's what you call a landslide.  It is my pleasure to announce
 that std.parallelism has been accepted for inclusion in Phobos.
 Congratulations, David!

 -Lars

Congrats, David! :-)
Apr 26 2011
prev sibling next sibling parent reply Russel Winder <russel russel.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 2011-04-26 at 11:21 +0000, Lars T. Kyllingstad wrote:
 The std.parallelism vote is now over, and the results are as follows:
=20
   24 voted yes
    0 voted no
=20
 I guess that's what you call a landslide.  It is my pleasure to announce=

 that std.parallelism has been accepted for inclusion in Phobos. =20
 Congratulations, David!

Splendid. Thanks to David for doing all the really hard work. OK so now when do I get a version of Phobos with it in. Tomorrow, Thursday? ;-) More seriously though: I believe Phobos only gets released with a DMD release. Do we know when that is scheduled for? --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Apr 26 2011
parent reply dsimcha <dsimcha yahoo.com> writes:
On 4/26/2011 8:01 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 11:21 +0000, Lars T. Kyllingstad wrote:
 The std.parallelism vote is now over, and the results are as follows:

    24 voted yes
     0 voted no

 I guess that's what you call a landslide.  It is my pleasure to announce
 that std.parallelism has been accepted for inclusion in Phobos.
 Congratulations, David!

Splendid. Thanks to David for doing all the really hard work. OK so now when do I get a version of Phobos with it in. Tomorrow, Thursday? ;-) More seriously though: I believe Phobos only gets released with a DMD release. Do we know when that is scheduled for?

Soon. I'm praying that I can figure out makefiles in that time to check std.parallelism in, since I think they're harder to work with than multithreading. (Ok, I'm exaggerating.) Among the other major improvements in this release: std.net.isemail massive GC optimizations for large allocations (~25% for very small ones to several hundred fold for huge ones) core.cpuid works on 64-bit temporary object destruction massive amounts of CTFE fixes
Apr 26 2011
next sibling parent Don <nospam nospam.com> writes:
Russel Winder wrote:
 On Tue, 2011-04-26 at 08:32 -0400, dsimcha wrote:
 [ . . . ]
 
 Soon.  I'm praying that I can figure out makefiles in that time to check 
 std.parallelism in, since I think they're harder to work with than 
 multithreading.  (Ok, I'm exaggerating.)  Among the other major 
 improvements in this release:

Isn't Make 1970s technology, I'd have thought D would use more up-to-date build technology than that -- even though Go uses it and refuses to look at other options.
 std.net.isemail

 massive GC optimizations for large allocations (~25% for very small ones 
 to several hundred fold for huge ones)

 core.cpuid works on 64-bit

I guess this is still assembler level stuff though, and hence the code is not portable?

Yes. The main functionality it gives is the processor instruction set. For example, whether the processor has SSE2. I don't think it's of much use unless you are writing asm code. (It's currently used by the runtime, and by BigInt).
 
 temporary object destruction

 massive amounts of CTFE fixes


Apr 26 2011
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/26/11 7:50 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 08:32 -0400, dsimcha wrote:
 [ . . . ]

 Soon.  I'm praying that I can figure out makefiles in that time to check
 std.parallelism in, since I think they're harder to work with than
 multithreading.  (Ok, I'm exaggerating.)  Among the other major
 improvements in this release:

Isn't Make 1970s technology, I'd have thought D would use more up-to-date build technology than that -- even though Go uses it and refuses to look at other options.

The debate about make being inadequate is almost as old as make itself :o). Our gnu makefile for Posix isn't in any way difficult or scary, although it did take a few iterations to get it right. It has 312 lines to control a build of 143KLOC, which is a good ratio. The only difficulty David would have to modify that makefile is to find the one place where all modules are enumerated, and insert his module's name there, so I have no idea why he finds that task daunting. (The Windows makefile is crappier and repeats itself a lot of times so that's more annoying to deal with.) The simple fact is that if someone wants to improve our build system they'd have to define it and argue successfully for its superiority. The issues I'm seeing as a build-systems-outsider who doesn't pay much attention are: (a) there are TONS of them; (b) each has issues that prevents it from becoming a new de facto standard; (c) the "best" one depends a lot on who you ask. Andrei
Apr 26 2011
next sibling parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 The debate about make being inadequate is almost as old as make itself
 :o). Our gnu makefile for Posix isn't in any way difficult or scary,
 although it did take a few iterations to get it right. It has 312 lines
 to control a build of 143KLOC, which is a good ratio. The only
 difficulty David would have to modify that makefile is to find the one
 place where all modules are enumerated, and insert his module's name
 there, so I have no idea why he finds that task daunting. (The Windows
 makefile is crappier and repeats itself a lot of times so that's more
 annoying to deal with.)
 The simple fact is that if someone wants to improve our build system
 they'd have to define it and argue successfully for its superiority. The
 issues I'm seeing as a build-systems-outsider who doesn't pay much
 attention are: (a) there are TONS of them; (b) each has issues that
 prevents it from becoming a new de facto standard; (c) the "best" one
 depends a lot on who you ask.
 Andrei

Yeah, my biggest gripe was that, when I tried to integrate std.parallelism into Phobos on Windows the first time by just following what was already there, I got bitten by Bug 4904 (http://d.puremagic.com/issues/show_bug.cgi?id=4904) and thought it might be a latent std.parallelism bug for a while. I was, of course, exaggerating about how bad Make is in my original post, but my general experience (including misc. Linux software I've tried to compile from source) is that it never works and when it doesn't it's hard to figure out why. Ironically, D's makefiles are some of the most well-behaved ones I've dealt with, but I still run into all kinds of weird issues, like that Phobos doesn't build at all on ancient Linux distributions from 2004 that my sysadmin insists on using for some reason.
Apr 26 2011
prev sibling next sibling parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 26/04/2011 16:09, Andrei Alexandrescu wrote:
 On 4/26/11 7:50 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 08:32 -0400, dsimcha wrote:
 [ . . . ]

 Soon. I'm praying that I can figure out makefiles in that time to check
 std.parallelism in, since I think they're harder to work with than
 multithreading. (Ok, I'm exaggerating.) Among the other major
 improvements in this release:

Isn't Make 1970s technology, I'd have thought D would use more up-to-date build technology than that -- even though Go uses it and refuses to look at other options.

The debate about make being inadequate is almost as old as make itself :o). Our gnu makefile for Posix isn't in any way difficult or scary, although it did take a few iterations to get it right. It has 312 lines to control a build of 143KLOC, which is a good ratio. The only difficulty David would have to modify that makefile is to find the one place where all modules are enumerated, and insert his module's name there, so I have no idea why he finds that task daunting. (The Windows makefile is crappier and repeats itself a lot of times so that's more annoying to deal with.) The simple fact is that if someone wants to improve our build system they'd have to define it and argue successfully for its superiority. The issues I'm seeing as a build-systems-outsider who doesn't pay much attention are: (a) there are TONS of them; (b) each has issues that prevents it from becoming a new de facto standard; (c) the "best" one depends a lot on who you ask. Andrei

My current favorite build tool is redo - it's about 200 lines of python and build scripts are about 5 lines for entire projects. It supports full dependencies (a depends on b depends on c, c is changed, a, b, and c recompiled), parallel compilation etc. I've set mine up to need zero modification too when I add new files to my library. Heck, you can even write your build scripts in D! That said, they'll be rather ugly unless you port the python to D, once that is done you could write a beautiful build script using D, it's on my todo list for when I get some free time... (When is that btw? :D). https://github.com/apenwarr/redo more info at http://cr.yp.to/redo.html which is linked from the github page anyway. -- Robert http://octarineparrot.com/
Apr 26 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/26/11 10:51 AM, Robert Clipsham wrote:
 On 26/04/2011 16:09, Andrei Alexandrescu wrote:
 On 4/26/11 7:50 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 08:32 -0400, dsimcha wrote:
 [ . . . ]

 Soon. I'm praying that I can figure out makefiles in that time to check
 std.parallelism in, since I think they're harder to work with than
 multithreading. (Ok, I'm exaggerating.) Among the other major
 improvements in this release:

Isn't Make 1970s technology, I'd have thought D would use more up-to-date build technology than that -- even though Go uses it and refuses to look at other options.

The debate about make being inadequate is almost as old as make itself :o). Our gnu makefile for Posix isn't in any way difficult or scary, although it did take a few iterations to get it right. It has 312 lines to control a build of 143KLOC, which is a good ratio. The only difficulty David would have to modify that makefile is to find the one place where all modules are enumerated, and insert his module's name there, so I have no idea why he finds that task daunting. (The Windows makefile is crappier and repeats itself a lot of times so that's more annoying to deal with.) The simple fact is that if someone wants to improve our build system they'd have to define it and argue successfully for its superiority. The issues I'm seeing as a build-systems-outsider who doesn't pay much attention are: (a) there are TONS of them; (b) each has issues that prevents it from becoming a new de facto standard; (c) the "best" one depends a lot on who you ask. Andrei

My current favorite build tool is redo - it's about 200 lines of python and build scripts are about 5 lines for entire projects. It supports full dependencies (a depends on b depends on c, c is changed, a, b, and c recompiled), parallel compilation etc. I've set mine up to need zero modification too when I add new files to my library. Heck, you can even write your build scripts in D! That said, they'll be rather ugly unless you port the python to D, once that is done you could write a beautiful build script using D, it's on my todo list for when I get some free time... (When is that btw? :D). https://github.com/apenwarr/redo more info at http://cr.yp.to/redo.html which is linked from the github page anyway.

Interesting. It would be great if you tried your hand at expressing the Phobos build (which is rather simple) with redo so we can see the result. Thanks, Andrei
Apr 26 2011
parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 26/04/2011 17:28, Andrei Alexandrescu wrote:
 Interesting. It would be great if you tried your hand at expressing the
 Phobos build (which is rather simple) with redo so we can see the result.

 Thanks,

 Andrei

I have no idea how to work Makefiles, but based on the output I have no idea why you bother with a Makefile at all. All that seems to happen is build C deps, get a relevant list of source files for the platform, pass them to dmd. Surely this can be handled in a <10 line shell script for the unicies/poxises etc and a slightly longer batch script for windows? If you built file by file it might be worth using something more sophisticated, but that isn't the case. If it does more than that then I'd be happy to look into writing the equivalent with redo (maybe even look at porting it to D so the build script can be written in idiomatic D). Of course if I did port it it'd have to be LGPL, writing from scratch could be any license. The minimal version that rebuilds everything each time is public domain. -- Robert http://octarineparrot.com/
Apr 26 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/26/11 12:31 PM, Robert Clipsham wrote:
 On 26/04/2011 17:28, Andrei Alexandrescu wrote:
 Interesting. It would be great if you tried your hand at expressing the
 Phobos build (which is rather simple) with redo so we can see the result.

 Thanks,

 Andrei

I have no idea how to work Makefiles, but based on the output I have no idea why you bother with a Makefile at all. All that seems to happen is build C deps, get a relevant list of source files for the platform, pass them to dmd. Surely this can be handled in a <10 line shell script for the unicies/poxises etc and a slightly longer batch script for windows? If you built file by file it might be worth using something more sophisticated, but that isn't the case.

That's pretty much it. There's also the documentation build, the debug vs. release build. and portability-related stuff. I doubt it could be done in <10 lines of shell scripting because just enumerating the modules would fill those. But if you feel strongly about this definitely do it - if you come with something better we'd be glad to replace things.
 If it does more than that then I'd be happy to look into writing the
 equivalent with redo (maybe even look at porting it to D so the build
 script can be written in idiomatic D). Of course if I did port it it'd
 have to be LGPL, writing from scratch could be any license. The minimal
 version that rebuilds everything each time is public domain.

Probably the basic redo script would give us a good idea of how things would be. Andrei
Apr 26 2011
parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 26/04/2011 18:55, Andrei Alexandrescu wrote:
 That's pretty much it. There's also the documentation build, the debug
 vs. release build. and portability-related stuff.

 I doubt it could be done in <10 lines of shell scripting because just
 enumerating the modules would fill those. But if you feel strongly about
 this definitely do it - if you come with something better we'd be glad
 to replace things.

Why enumerate them? * will do, if there are files you don't want including they shouldn't be in the directory. All the platform specific stuff is separated in its own folders too, so that's not an issue.
 If it does more than that then I'd be happy to look into writing the
 equivalent with redo (maybe even look at porting it to D so the build
 script can be written in idiomatic D). Of course if I did port it it'd
 have to be LGPL, writing from scratch could be any license. The minimal
 version that rebuilds everything each time is public domain.

Probably the basic redo script would give us a good idea of how things would be.

A redo script could be the way to go if phobos was being built incrementally, but it seems like overkill given that most of the build is one command.
 Andrei

-- Robert http://octarineparrot.com/
Apr 26 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/26/11 1:06 PM, Robert Clipsham wrote:
 On 26/04/2011 18:55, Andrei Alexandrescu wrote:
 That's pretty much it. There's also the documentation build, the debug
 vs. release build. and portability-related stuff.

 I doubt it could be done in <10 lines of shell scripting because just
 enumerating the modules would fill those. But if you feel strongly about
 this definitely do it - if you come with something better we'd be glad
 to replace things.

Why enumerate them? * will do, if there are files you don't want including they shouldn't be in the directory. All the platform specific stuff is separated in its own folders too, so that's not an issue.

It's better to have control on the actual names on the files, otherwise what is built depends on what the builder has in that directory. Nevertheless, there's other stuff that takes 10 lines. Consider: # Set CFLAGS ifeq ($(CC),cc) CFLAGS += -m$(MODEL) ifeq ($(BUILD),debug) CFLAGS += -g else CFLAGS += -O3 endif endif That's 10 lines. You may be able to reduce this but not dramatically. Overall I think you're mis-estimating what it would take to do the build in a shell script; in all likelihood the size of the script would be comparable to that of the makefile. I think you'd gain a much better appreciation of what's going on if you set out to write those 10 lines.
 If it does more than that then I'd be happy to look into writing the
 equivalent with redo (maybe even look at porting it to D so the build
 script can be written in idiomatic D). Of course if I did port it it'd
 have to be LGPL, writing from scratch could be any license. The minimal
 version that rebuilds everything each time is public domain.

Probably the basic redo script would give us a good idea of how things would be.

A redo script could be the way to go if phobos was being built incrementally, but it seems like overkill given that most of the build is one command.

Documentation is built incrementally and unittests are executed incrementally (successful ones are not re-run). Also, the C zip library is built incrementally. Andrei
Apr 26 2011
next sibling parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrej Mitrovic (andrej.mitrovich gmail.com)'s article
 If D was available for every platform we could use a simple D script
 to do the building for us. And hey, now we can even do parallel
 builds! (std.parallelism, anyone?) :D
 But then porting to new platforms would be a chicken and the egg problem.

LOL I've actually used D and std.parallelism to do parallel builds. I have a few projects for which I had a good build system set up via CodeBlocks and then found out I needed to be able to build without a GUI and without CodeBlocks installed. I wrote a quick n' dirty D program to parse the CodeBlocks file (Who needs a real XML parser when you have std.string and std.regex?) and build in parallel using std.parallelism. The whole thing is 47 lines of code.
Apr 26 2011
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-04-26 20:52, Andrej Mitrovic wrote:
 If D was available for every platform we could use a simple D script
 to do the building for us. And hey, now we can even do parallel
 builds! (std.parallelism, anyone?) :D

 But then porting to new platforms would be a chicken and the egg problem.

Yeah, Tango had that problem. I couldn't updated to a newer version of Tango since the build script was a pre-built D application and not available on Mac OS X. The source code was written with this new version on Tango so I couldn't compile the build script myself either. I ended up porting it to Ruby instead. -- /Jacob Carlborg
Apr 27 2011
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-04-26 17:09, Andrei Alexandrescu wrote:
 On 4/26/11 7:50 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 08:32 -0400, dsimcha wrote:
 [ . . . ]

 Soon. I'm praying that I can figure out makefiles in that time to check
 std.parallelism in, since I think they're harder to work with than
 multithreading. (Ok, I'm exaggerating.) Among the other major
 improvements in this release:

Isn't Make 1970s technology, I'd have thought D would use more up-to-date build technology than that -- even though Go uses it and refuses to look at other options.

The debate about make being inadequate is almost as old as make itself :o). Our gnu makefile for Posix isn't in any way difficult or scary, although it did take a few iterations to get it right. It has 312 lines to control a build of 143KLOC, which is a good ratio. The only difficulty David would have to modify that makefile is to find the one place where all modules are enumerated, and insert his module's name there, so I have no idea why he finds that task daunting. (The Windows makefile is crappier and repeats itself a lot of times so that's more annoying to deal with.)

I just counted how may times a module name is repeated (in the windows makefile), "isemail" is repeated nine times. I mean, that's insane. I should only have to write it a one place, not nine. One thing I think it's scary about makefiles is that I can add or remove a single space and it all breaks down I have no idea why. It took me some time to figure out that makefiles are very sensitive about spaces.
 The simple fact is that if someone wants to improve our build system
 they'd have to define it and argue successfully for its superiority. The
 issues I'm seeing as a build-systems-outsider who doesn't pay much
 attention are: (a) there are TONS of them; (b) each has issues that
 prevents it from becoming a new de facto standard; (c) the "best" one
 depends a lot on who you ask.


 Andrei

-- /Jacob Carlborg
Apr 26 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/26/11 11:33 AM, Jacob Carlborg wrote:
 On 2011-04-26 17:09, Andrei Alexandrescu wrote:
 On 4/26/11 7:50 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 08:32 -0400, dsimcha wrote:
 [ . . . ]

 Soon. I'm praying that I can figure out makefiles in that time to check
 std.parallelism in, since I think they're harder to work with than
 multithreading. (Ok, I'm exaggerating.) Among the other major
 improvements in this release:

Isn't Make 1970s technology, I'd have thought D would use more up-to-date build technology than that -- even though Go uses it and refuses to look at other options.

The debate about make being inadequate is almost as old as make itself :o). Our gnu makefile for Posix isn't in any way difficult or scary, although it did take a few iterations to get it right. It has 312 lines to control a build of 143KLOC, which is a good ratio. The only difficulty David would have to modify that makefile is to find the one place where all modules are enumerated, and insert his module's name there, so I have no idea why he finds that task daunting. (The Windows makefile is crappier and repeats itself a lot of times so that's more annoying to deal with.)

I just counted how may times a module name is repeated (in the windows makefile), "isemail" is repeated nine times. I mean, that's insane. I should only have to write it a one place, not nine.

I agree. That's why I put together posix.mak. On Windows things have remained that way because Walter doesn't want to depend on cygwin, so the make utility used on Windows is the one he wrote (which doesn't support many features of make). If I had remote access to a Windows machine I'd be able to use posix.mak for the Windows build too. It used to work (on a machine donated by Adam Ruppe) but that machine has gone dark since.
 One thing I think it's scary about makefiles is that I can add or remove
 a single space and it all breaks down I have no idea why. It took me
 some time to figure out that makefiles are very sensitive about spaces.

The one odd thing to remember is that a rule's actions must start with a tab. That's about it. Andrei
Apr 26 2011
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/26/11 12:42 PM, Andrej Mitrovic wrote:
 I don't understand, why not just use gnu make for Windows?
 http://gnuwin32.sourceforge.net/packages/make.htm

 It doesn't need cygwin.

I didn't know of that. I think it's a nice idea! Andrei
Apr 26 2011
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-04-26 19:33, Andrei Alexandrescu wrote:
 On 4/26/11 11:33 AM, Jacob Carlborg wrote:
 On 2011-04-26 17:09, Andrei Alexandrescu wrote:
 On 4/26/11 7:50 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 08:32 -0400, dsimcha wrote:
 [ . . . ]

 Soon. I'm praying that I can figure out makefiles in that time to
 check
 std.parallelism in, since I think they're harder to work with than
 multithreading. (Ok, I'm exaggerating.) Among the other major
 improvements in this release:

Isn't Make 1970s technology, I'd have thought D would use more up-to-date build technology than that -- even though Go uses it and refuses to look at other options.

The debate about make being inadequate is almost as old as make itself :o). Our gnu makefile for Posix isn't in any way difficult or scary, although it did take a few iterations to get it right. It has 312 lines to control a build of 143KLOC, which is a good ratio. The only difficulty David would have to modify that makefile is to find the one place where all modules are enumerated, and insert his module's name there, so I have no idea why he finds that task daunting. (The Windows makefile is crappier and repeats itself a lot of times so that's more annoying to deal with.)

I just counted how may times a module name is repeated (in the windows makefile), "isemail" is repeated nine times. I mean, that's insane. I should only have to write it a one place, not nine.

I agree. That's why I put together posix.mak. On Windows things have remained that way because Walter doesn't want to depend on cygwin, so the make utility used on Windows is the one he wrote (which doesn't support many features of make). If I had remote access to a Windows machine I'd be able to use posix.mak for the Windows build too. It used to work (on a machine donated by Adam Ruppe) but that machine has gone dark since.
 One thing I think it's scary about makefiles is that I can add or remove
 a single space and it all breaks down I have no idea why. It took me
 some time to figure out that makefiles are very sensitive about spaces.

The one odd thing to remember is that a rule's actions must start with a tab. That's about it. Andrei

I'm pretty sure a rule can't start with a space. -- /Jacob Carlborg
Apr 27 2011
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/26/11 11:56 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 10:09 -0500, Andrei Alexandrescu wrote:
 On 4/26/11 7:50 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 08:32 -0400, dsimcha wrote:
 [ . . . ]

 Soon.  I'm praying that I can figure out makefiles in that time to check
 std.parallelism in, since I think they're harder to work with than
 multithreading.  (Ok, I'm exaggerating.)  Among the other major
 improvements in this release:

Isn't Make 1970s technology, I'd have thought D would use more up-to-date build technology than that -- even though Go uses it and refuses to look at other options.

The debate about make being inadequate is almost as old as make itself

Hardly. In 1978 Make was a revelation and a revolution. Well at least on Unix. And that is the problem, Make is all very well if you are on Unix and can rely on /bin/sh and various other tools. As soon as you try to create platform independent builds, Make fails to be sensible except for trivial builds.

This has been a common complaint about make... until gmake came about. Targeting gmake newer than a specific version seldom causes platform-related issues.
 :o). Our gnu makefile for Posix isn't in any way difficult or scary,
 although it did take a few iterations to get it right. It has 312 lines
 to control a build of 143KLOC, which is a good ratio. The only

If the build for DMD and Phobos is a single Makefile of 312 lines then it almost certainly needs refactoring. Ratio of source code lines to Makefile lines is almost certainly a spurious metric.

It is a great metric because they are in a ratio determined by the tools used. 312 lines to build 143K lines seems a good ratio to me - probably some of the best I've seen. The 312 lines make is only for Phobos. If you could reduce and simplify it, we'd all appreciate it.
 difficulty David would have to modify that makefile is to find the one
 place where all modules are enumerated, and insert his module's name
 there, so I have no idea why he finds that task daunting. (The Windows
 makefile is crappier and repeats itself a lot of times so that's more
 annoying to deal with.)

Why is there a separate Windows Makefile? Oh of course you actually have to have separate build systems for Windows, Mac OS X, Linux, Solaris, etc., etc. because Make is not a platform independent system, it relies on /bin/sh and other tools.

Sorry, you are wrong here. Please take a look!
 The simple fact is that if someone wants to improve our build system
 they'd have to define it and argue successfully for its superiority. The
 issues I'm seeing as a build-systems-outsider who doesn't pay much
 attention are: (a) there are TONS of them; (b) each has issues that
 prevents it from becoming a new de facto standard; (c) the "best" one
 depends a lot on who you ask.

Wrong approach I think. If anyone cares enough then they write a Waf, SCons or CMake (or whatever marginal build system people can find) build in parallel to the Make one using a fork of the D project source. They then announce it. Then people use it. Then the Make system dies off.

Exactly!
 Make has issues that stops it being a modern build system, but it hangs
 on because people are scared of change and so do anything and everything
 to stay with using it even though it is an outdated system and there are
 now far superior ones.

The general considerations are most appreciated. I don't think scare of change is an issue for us.
 There are indeed zillions of build frameworks.  Every new programming
 language ends up having someone write a new build system in it -- Go
 appears to have created a record by having about six all written in the
 last year.  It is a knee-jerk reaction.  What is relevant is what are
 the de facto standards as opposed to de jure standards.  Make and
 Autotools are still there because they have been there for 30 years.
 Waf, SCons, CMake and Rake are the only tools with traction off the JVM.
 Gradle, Maven, Gant, Ant, are the market leaders on the JVM with SBT and
 Leiningen have some cult following.  Any other build framework is
 currently marginal -- no matter how good it actually is.

 There are metrics to decide which is best for a given project.  Fashion
 and prejudice do also, sadly, take their place in most projects
 decisions. :-(

What also matters is that stuff that stays is stuff that gets done. You seem to have a good amount of expertise, but at the same time choose to firmly plant your figurative self in the proverbial armchair. If you know how to do it, do it, don't attempt to beat others into doing it. Andrei
Apr 26 2011
next sibling parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
You forgot the new build tool from Microsoft.

Nowadays the official build tool in Windows world, is slowly becoming 
msbuild,
a Microsoft Ant clone, even for C and C++ builds.

--
Paulo


"Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> wrote in message 
news:ip70qm$rfq$1 digitalmars.com...
 On 4/26/11 11:56 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 10:09 -0500, Andrei Alexandrescu wrote:
 On 4/26/11 7:50 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 08:32 -0400, dsimcha wrote:
 [ . . . ]

 Soon.  I'm praying that I can figure out makefiles in that time to 
 check
 std.parallelism in, since I think they're harder to work with than
 multithreading.  (Ok, I'm exaggerating.)  Among the other major
 improvements in this release:

Isn't Make 1970s technology, I'd have thought D would use more up-to-date build technology than that -- even though Go uses it and refuses to look at other options.

The debate about make being inadequate is almost as old as make itself

Hardly. In 1978 Make was a revelation and a revolution. Well at least on Unix. And that is the problem, Make is all very well if you are on Unix and can rely on /bin/sh and various other tools. As soon as you try to create platform independent builds, Make fails to be sensible except for trivial builds.

This has been a common complaint about make... until gmake came about. Targeting gmake newer than a specific version seldom causes platform-related issues.
 :o). Our gnu makefile for Posix isn't in any way difficult or scary,
 although it did take a few iterations to get it right. It has 312 lines
 to control a build of 143KLOC, which is a good ratio. The only

If the build for DMD and Phobos is a single Makefile of 312 lines then it almost certainly needs refactoring. Ratio of source code lines to Makefile lines is almost certainly a spurious metric.

It is a great metric because they are in a ratio determined by the tools used. 312 lines to build 143K lines seems a good ratio to me - probably some of the best I've seen. The 312 lines make is only for Phobos. If you could reduce and simplify it, we'd all appreciate it.
 difficulty David would have to modify that makefile is to find the one
 place where all modules are enumerated, and insert his module's name
 there, so I have no idea why he finds that task daunting. (The Windows
 makefile is crappier and repeats itself a lot of times so that's more
 annoying to deal with.)

Why is there a separate Windows Makefile? Oh of course you actually have to have separate build systems for Windows, Mac OS X, Linux, Solaris, etc., etc. because Make is not a platform independent system, it relies on /bin/sh and other tools.

Sorry, you are wrong here. Please take a look!
 The simple fact is that if someone wants to improve our build system
 they'd have to define it and argue successfully for its superiority. The
 issues I'm seeing as a build-systems-outsider who doesn't pay much
 attention are: (a) there are TONS of them; (b) each has issues that
 prevents it from becoming a new de facto standard; (c) the "best" one
 depends a lot on who you ask.

Wrong approach I think. If anyone cares enough then they write a Waf, SCons or CMake (or whatever marginal build system people can find) build in parallel to the Make one using a fork of the D project source. They then announce it. Then people use it. Then the Make system dies off.

Exactly!
 Make has issues that stops it being a modern build system, but it hangs
 on because people are scared of change and so do anything and everything
 to stay with using it even though it is an outdated system and there are
 now far superior ones.

The general considerations are most appreciated. I don't think scare of change is an issue for us.
 There are indeed zillions of build frameworks.  Every new programming
 language ends up having someone write a new build system in it -- Go
 appears to have created a record by having about six all written in the
 last year.  It is a knee-jerk reaction.  What is relevant is what are
 the de facto standards as opposed to de jure standards.  Make and
 Autotools are still there because they have been there for 30 years.
 Waf, SCons, CMake and Rake are the only tools with traction off the JVM.
 Gradle, Maven, Gant, Ant, are the market leaders on the JVM with SBT and
 Leiningen have some cult following.  Any other build framework is
 currently marginal -- no matter how good it actually is.

 There are metrics to decide which is best for a given project.  Fashion
 and prejudice do also, sadly, take their place in most projects
 decisions. :-(

What also matters is that stuff that stays is stuff that gets done. You seem to have a good amount of expertise, but at the same time choose to firmly plant your figurative self in the proverbial armchair. If you know how to do it, do it, don't attempt to beat others into doing it. Andrei

Apr 26 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/26/11 1:13 PM, Paulo Pinto wrote:
 You forgot the new build tool from Microsoft.

 Nowadays the official build tool in Windows world, is slowly becoming
 msbuild,
 a Microsoft Ant clone, even for C and C++ builds.

 --
 Paulo

Looks like you need to have Visual Studio to have msbuild. Is that correct? Andrei
Apr 26 2011
parent "Paulo Pinto" <pjmlp progtools.org> writes:
No, but you need to have the SDK, which can be easily downloaded from 
Microsoft's web site.
http://msdn.microsoft.com/en-us/windows/bb980924

--
Paulo


"Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> wrote in message 
news:ip73au$vft$2 digitalmars.com...
 On 4/26/11 1:13 PM, Paulo Pinto wrote:
 You forgot the new build tool from Microsoft.

 Nowadays the official build tool in Windows world, is slowly becoming
 msbuild,
 a Microsoft Ant clone, even for C and C++ builds.

 --
 Paulo

Looks like you need to have Visual Studio to have msbuild. Is that correct? Andrei

Apr 26 2011
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/27/11 1:13 AM, Russel Winder wrote:
 What also matters is that stuff that stays is stuff that gets done. You
 seem to have a good amount of expertise, but at the same time choose to
 firmly plant your figurative self in the proverbial armchair. If you
 know how to do it, do it, don't attempt to beat others into doing it.

I think it a bit overly-aggressive to use phrases such as "choose to firmly plant your figurative self in the proverbial armchair" and "beat others into doing it". For people who have full-time jobs and therefore secure income who use some of their work time or spare time to work on projects such as D, there is the safety of having secure income. For those of us who have no full-time job it is a matter of all time spent on things like D is time not spent earning money. Firing off some emails is quick and apart from a few rants, I think most of my emails are constructive. It is therefore a serious barrier to be told I am a back seat driver and to "do something" don't "say something". With people such as Spacen Jasset tentatively volunteering to contribute to some work due to this exchange, to start getting censorious of my efforts to contribute and create an atmosphere of doing something seems somewhat counter-productive, certainly not community building.

I agree. If posting to the newsgroup is significantly faster for you than actually spinning some code, then by all means please do post. I find that posting takes me significant amounts of time. Thanks, Andrei
Apr 27 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-04-27 21:13, Andrei Alexandrescu wrote:
 On 4/27/11 1:13 AM, Russel Winder wrote:
 What also matters is that stuff that stays is stuff that gets done. You
 seem to have a good amount of expertise, but at the same time choose to
 firmly plant your figurative self in the proverbial armchair. If you
 know how to do it, do it, don't attempt to beat others into doing it.

I think it a bit overly-aggressive to use phrases such as "choose to firmly plant your figurative self in the proverbial armchair" and "beat others into doing it". For people who have full-time jobs and therefore secure income who use some of their work time or spare time to work on projects such as D, there is the safety of having secure income. For those of us who have no full-time job it is a matter of all time spent on things like D is time not spent earning money. Firing off some emails is quick and apart from a few rants, I think most of my emails are constructive. It is therefore a serious barrier to be told I am a back seat driver and to "do something" don't "say something". With people such as Spacen Jasset tentatively volunteering to contribute to some work due to this exchange, to start getting censorious of my efforts to contribute and create an atmosphere of doing something seems somewhat counter-productive, certainly not community building.

I agree. If posting to the newsgroup is significantly faster for you than actually spinning some code, then by all means please do post. I find that posting takes me significant amounts of time.

You can't mean that it takes you significantly more time to write the above post of two lines than implementing a build tool.
 Thanks,

 Andrei

-- /Jacob Carlborg
Apr 28 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/28/11 3:24 AM, Jacob Carlborg wrote:
 On 2011-04-27 21:13, Andrei Alexandrescu wrote:
 On 4/27/11 1:13 AM, Russel Winder wrote:
 What also matters is that stuff that stays is stuff that gets done. You
 seem to have a good amount of expertise, but at the same time choose to
 firmly plant your figurative self in the proverbial armchair. If you
 know how to do it, do it, don't attempt to beat others into doing it.

I think it a bit overly-aggressive to use phrases such as "choose to firmly plant your figurative self in the proverbial armchair" and "beat others into doing it". For people who have full-time jobs and therefore secure income who use some of their work time or spare time to work on projects such as D, there is the safety of having secure income. For those of us who have no full-time job it is a matter of all time spent on things like D is time not spent earning money. Firing off some emails is quick and apart from a few rants, I think most of my emails are constructive. It is therefore a serious barrier to be told I am a back seat driver and to "do something" don't "say something". With people such as Spacen Jasset tentatively volunteering to contribute to some work due to this exchange, to start getting censorious of my efforts to contribute and create an atmosphere of doing something seems somewhat counter-productive, certainly not community building.

I agree. If posting to the newsgroup is significantly faster for you than actually spinning some code, then by all means please do post. I find that posting takes me significant amounts of time.

You can't mean that it takes you significantly more time to write the above post of two lines than implementing a build tool.

Of course I can't. I didn't either... Andrei
Apr 28 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-04-28 17:37, Andrei Alexandrescu wrote:
 On 4/28/11 3:24 AM, Jacob Carlborg wrote:
 On 2011-04-27 21:13, Andrei Alexandrescu wrote:
 On 4/27/11 1:13 AM, Russel Winder wrote:
 What also matters is that stuff that stays is stuff that gets done.
 You
 seem to have a good amount of expertise, but at the same time
 choose to
 firmly plant your figurative self in the proverbial armchair. If you
 know how to do it, do it, don't attempt to beat others into doing it.

I think it a bit overly-aggressive to use phrases such as "choose to firmly plant your figurative self in the proverbial armchair" and "beat others into doing it". For people who have full-time jobs and therefore secure income who use some of their work time or spare time to work on projects such as D, there is the safety of having secure income. For those of us who have no full-time job it is a matter of all time spent on things like D is time not spent earning money. Firing off some emails is quick and apart from a few rants, I think most of my emails are constructive. It is therefore a serious barrier to be told I am a back seat driver and to "do something" don't "say something". With people such as Spacen Jasset tentatively volunteering to contribute to some work due to this exchange, to start getting censorious of my efforts to contribute and create an atmosphere of doing something seems somewhat counter-productive, certainly not community building.

I agree. If posting to the newsgroup is significantly faster for you than actually spinning some code, then by all means please do post. I find that posting takes me significant amounts of time.

You can't mean that it takes you significantly more time to write the above post of two lines than implementing a build tool.

Of course I can't. I didn't either... Andrei

You kind of gave that impression. -- /Jacob Carlborg
Apr 29 2011
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 27/04/2011 07:13, Russel Winder wrote:
 For people who have full-time jobs and therefore secure income who use
 some of their work time or spare time to work on projects such as D,
 there is the safety of having secure income.  For those of us who have
 no full-time job it is a matter of all time spent on things like D is
 time not spent earning money.

[OT] Funny, I would have thought it would be mostly the other way around, people without full-time jobs would have little time for D. At least for me it is, if I had a full-time job I would not have the time (or more accurately said, would not be willing to spend the time) to be involved with D, probably it would be hard to just keep up with the newsgroups. It is only because I am working part-time (practically unemployed actually) that I can dedicate this time. Of course, one does need an income, so at some point one does need to work for a living and for income. -- Bruno Medeiros - Software Engineer
May 03 2011
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/04/2011 17:56, Russel Winder wrote:
 Hardly.  In 1978 Make was a revelation and a revolution.  Well at least
 on Unix.  And that is the problem, Make is all very well if you are on
 Unix and can rely on /bin/sh and various other tools.  As soon as you
 try to create platform independent builds, Make fails to be sensible
 except for trivial builds.

I fully agree, not being platform independent is a fundamental problem and (IMO) an unacceptable one for a modern build system (for D or otherwise). I also echo your comments about the build actions/commands: For a build system to be truly platform independent, the core/common set of build actions must be available in the build tool itself, not as external command-line programs (unless the programs are supplied by the build system itself). -- Bruno Medeiros - Software Engineer
Apr 29 2011
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-04-26 18:16, Sean Kelly wrote:
 Bootstrapping issue. Make is almost guaranteed to exist, while the fancier
tools are not.

 Sent from my iPhone

Except for on Windows. -- /Jacob Carlborg
Apr 26 2011
parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
If you have the SDK installed, you get it (nmake), same thing on MacOS X. No 
SDK, no make.

"Jacob Carlborg" <doob me.com> wrote in message 
news:ip6sct$klu$2 digitalmars.com...
 On 2011-04-26 18:16, Sean Kelly wrote:
 Bootstrapping issue. Make is almost guaranteed to exist, while the 
 fancier tools are not.

 Sent from my iPhone

Except for on Windows. -- /Jacob Carlborg

Apr 26 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-04-26 20:14, Paulo Pinto wrote:
 If you have the SDK installed, you get it (nmake), same thing on MacOS X. No
 SDK, no make.

The difference is that on Windows I don't need the SDK to build D applications, on Mac OS X I do. So basically if you're using D on Mac OS X you will have make installed. This is not necessarily true for Windows.
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ip6sct$klu$2 digitalmars.com...
 On 2011-04-26 18:16, Sean Kelly wrote:
 Bootstrapping issue. Make is almost guaranteed to exist, while the
 fancier tools are not.

 Sent from my iPhone

Except for on Windows. -- /Jacob Carlborg


-- /Jacob Carlborg
Apr 27 2011
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/26/11 11:32 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 09:16 -0700, Sean Kelly wrote:
 Bootstrapping issue. Make is almost guaranteed to exist, while the fancier
tools are not.

That may be the case but Makefiles are either incomprehensibly complicated or platform specific -- it is nigh on impossible to create a single Makefile that does its job sensibly on Linux, Solaris, Mac OS X and Windows.

In reality if you target GNU make things are very portable.
 The advantage of tools such as Waf, SCons, CMake is that they manage
 the platform issues for you.

I hope that's not the only advantage. Anyhow, the problem here is that you already mentioned three tools, none of which I know. If you could write the equivalent of posix.mak in your favorite tool, we'd be in a better position to evaluate how it improves our build process. Andrei
Apr 26 2011
parent reply Spacen Jasset <spacenjasset yahoo.co.uk> writes:
On 26/04/2011 18:37, Andrei Alexandrescu wrote:
 On 4/26/11 11:32 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 09:16 -0700, Sean Kelly wrote:
 Bootstrapping issue. Make is almost guaranteed to exist, while the
 fancier tools are not.

That may be the case but Makefiles are either incomprehensibly complicated or platform specific -- it is nigh on impossible to create a single Makefile that does its job sensibly on Linux, Solaris, Mac OS X and Windows.

In reality if you target GNU make things are very portable.
 The advantage of tools such as Waf, SCons, CMake is that they manage
 the platform issues for you.

I hope that's not the only advantage. Anyhow, the problem here is that you already mentioned three tools, none of which I know. If you could write the equivalent of posix.mak in your favorite tool, we'd be in a better position to evaluate how it improves our build process. Andrei

tool I've found to (my) liking. It supports D too. I must say that when dsss and rebuild were maintained it was very handy indeed to be able to it with. (yeah - almost) identical build files on the platforms I built for. Win & linux at the time. I can't but think you haven't written enough makefiles Andrei, for enough cross platform development, but I dare say you have.
Apr 26 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-04-26 22:45, Spacen Jasset wrote:
 On 26/04/2011 18:37, Andrei Alexandrescu wrote:
 On 4/26/11 11:32 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 09:16 -0700, Sean Kelly wrote:
 Bootstrapping issue. Make is almost guaranteed to exist, while the
 fancier tools are not.

That may be the case but Makefiles are either incomprehensibly complicated or platform specific -- it is nigh on impossible to create a single Makefile that does its job sensibly on Linux, Solaris, Mac OS X and Windows.

In reality if you target GNU make things are very portable.
 The advantage of tools such as Waf, SCons, CMake is that they manage
 the platform issues for you.

I hope that's not the only advantage. Anyhow, the problem here is that you already mentioned three tools, none of which I know. If you could write the equivalent of posix.mak in your favorite tool, we'd be in a better position to evaluate how it improves our build process. Andrei

tool I've found to (my) liking. It supports D too. I must say that when dsss and rebuild were maintained it was very handy indeed to be able to it with. (yeah - almost) identical build files on the platforms I built for. Win & linux at the time.

I agree, this was a great tool and still is for D1.
 I can't but think you haven't written enough makefiles Andrei, for
 enough cross platform development, but I dare say you have.

-- /Jacob Carlborg
Apr 27 2011
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Apr 26, 2011, at 10:33 AM, Andrei Alexandrescu wrote:
=20
 I just counted how may times a module name is repeated (in the =


 makefile), "isemail" is repeated nine times. I mean, that's insane. I
 should only have to write it a one place, not nine.

I agree. That's why I put together posix.mak. On Windows things have =

the make utility used on Windows is the one he wrote (which doesn't = support many features of make). There's a standalone version of make in GnuWin32, though I haven't = tested to see if it's as fully functional as the *nix version.=
Apr 26 2011
prev sibling next sibling parent Russel Winder <russel russel.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 2011-04-26 at 08:32 -0400, dsimcha wrote:
[ . . . ]

 Soon.  I'm praying that I can figure out makefiles in that time to check=

 std.parallelism in, since I think they're harder to work with than=20
 multithreading.  (Ok, I'm exaggerating.)  Among the other major=20
 improvements in this release:

Isn't Make 1970s technology, I'd have thought D would use more up-to-date build technology than that -- even though Go uses it and refuses to look at other options.
 std.net.isemail
=20
 massive GC optimizations for large allocations (~25% for very small ones=

 to several hundred fold for huge ones)
=20
 core.cpuid works on 64-bit

I guess this is still assembler level stuff though, and hence the code is not portable?
 temporary object destruction
=20
 massive amounts of CTFE fixes

--=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Apr 26 2011
prev sibling next sibling parent dolive <dolive89 sina.com> writes:
Lars T. Kyllingstad Wrote:

 The std.parallelism vote is now over, and the results are as follows:
 
   24 voted yes
    0 voted no
 
 I guess that's what you call a landslide.  It is my pleasure to announce 
 that std.parallelism has been accepted for inclusion in Phobos.  
 Congratulations, David!
 
 -Lars

Congratulations, David!
Apr 26 2011
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
Bootstrapping issue. Make is almost guaranteed to exist, while the fancier t=
ools are not.=20

Sent from my iPhone

On Apr 26, 2011, at 5:50 AM, Russel Winder <russel russel.org.uk> wrote:

 On Tue, 2011-04-26 at 08:32 -0400, dsimcha wrote:
 [ . . . ]
=20
 Soon.  I'm praying that I can figure out makefiles in that time to check=20=


 std.parallelism in, since I think they're harder to work with than=20
 multithreading.  (Ok, I'm exaggerating.)  Among the other major=20
 improvements in this release:

Isn't Make 1970s technology, I'd have thought D would use more up-to-date build technology than that -- even though Go uses it and refuses to look at other options. =20
 std.net.isemail
=20
 massive GC optimizations for large allocations (~25% for very small ones=20=


 to several hundred fold for huge ones)
=20
 core.cpuid works on 64-bit

I guess this is still assembler level stuff though, and hence the code is not portable? =20
 temporary object destruction
=20
 massive amounts of CTFE fixes

--=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
 Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.=

 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel russel.org.uk
 London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

Apr 26 2011
prev sibling next sibling parent Russel Winder <russel russel.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 2011-04-26 at 09:16 -0700, Sean Kelly wrote:
 Bootstrapping issue. Make is almost guaranteed to exist, while the fancie=

That may be the case but Makefiles are either incomprehensibly complicated or platform specific -- it is nigh on impossible to create a single Makefile that does its job sensibly on Linux, Solaris, Mac OS X and Windows. The advantage of tools such as Waf, SCons, CMake is that they manage the platform issues for you.=20 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Apr 26 2011
prev sibling next sibling parent Russel Winder <russel russel.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 2011-04-26 at 10:09 -0500, Andrei Alexandrescu wrote:
 On 4/26/11 7:50 AM, Russel Winder wrote:
 On Tue, 2011-04-26 at 08:32 -0400, dsimcha wrote:
 [ . . . ]

 Soon.  I'm praying that I can figure out makefiles in that time to che=



 std.parallelism in, since I think they're harder to work with than
 multithreading.  (Ok, I'm exaggerating.)  Among the other major
 improvements in this release:

Isn't Make 1970s technology, I'd have thought D would use more up-to-date build technology than that -- even though Go uses it and refuses to look at other options.

The debate about make being inadequate is almost as old as make itself=

Hardly. In 1978 Make was a revelation and a revolution. Well at least on Unix. And that is the problem, Make is all very well if you are on Unix and can rely on /bin/sh and various other tools. As soon as you try to create platform independent builds, Make fails to be sensible except for trivial builds.
 :o). Our gnu makefile for Posix isn't in any way difficult or scary,=20
 although it did take a few iterations to get it right. It has 312 lines=

 to control a build of 143KLOC, which is a good ratio. The only

If the build for DMD and Phobos is a single Makefile of 312 lines then it almost certainly needs refactoring. Ratio of source code lines to Makefile lines is almost certainly a spurious metric.
 difficulty David would have to modify that makefile is to find the one=

 place where all modules are enumerated, and insert his module's name=20
 there, so I have no idea why he finds that task daunting. (The Windows=

 makefile is crappier and repeats itself a lot of times so that's more=20
 annoying to deal with.)

Why is there a separate Windows Makefile? Oh of course you actually have to have separate build systems for Windows, Mac OS X, Linux, Solaris, etc., etc. because Make is not a platform independent system, it relies on /bin/sh and other tools.=20
 The simple fact is that if someone wants to improve our build system=20
 they'd have to define it and argue successfully for its superiority. The=

 issues I'm seeing as a build-systems-outsider who doesn't pay much=20
 attention are: (a) there are TONS of them; (b) each has issues that=20
 prevents it from becoming a new de facto standard; (c) the "best" one=20
 depends a lot on who you ask.

Wrong approach I think. If anyone cares enough then they write a Waf, SCons or CMake (or whatever marginal build system people can find) build in parallel to the Make one using a fork of the D project source. They then announce it. Then people use it. Then the Make system dies off. Make has issues that stops it being a modern build system, but it hangs on because people are scared of change and so do anything and everything to stay with using it even though it is an outdated system and there are now far superior ones. There are indeed zillions of build frameworks. Every new programming language ends up having someone write a new build system in it -- Go appears to have created a record by having about six all written in the last year. It is a knee-jerk reaction. What is relevant is what are the de facto standards as opposed to de jure standards. Make and Autotools are still there because they have been there for 30 years. Waf, SCons, CMake and Rake are the only tools with traction off the JVM. Gradle, Maven, Gant, Ant, are the market leaders on the JVM with SBT and Leiningen have some cult following. Any other build framework is currently marginal -- no matter how good it actually is. There are metrics to decide which is best for a given project. Fashion and prejudice do also, sadly, take their place in most projects decisions. :-( --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Apr 26 2011
prev sibling next sibling parent Russel Winder <russel russel.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 2011-04-26 at 11:28 -0500, Andrei Alexandrescu wrote:
 On 4/26/11 10:51 AM, Robert Clipsham wrote:

 My current favorite build tool is redo - it's about 200 lines of python
 and build scripts are about 5 lines for entire projects. It supports
 full dependencies (a depends on b depends on c, c is changed, a, b, and
 c recompiled), parallel compilation etc. I've set mine up to need zero
 modification too when I add new files to my library. Heck, you can even
 write your build scripts in D! That said, they'll be rather ugly unless
 you port the python to D, once that is done you could write a beautiful
 build script using D, it's on my todo list for when I get some free
 time... (When is that btw? :D).

 https://github.com/apenwarr/redo more info at http://cr.yp.to/redo.html
 which is linked from the github page anyway.

Interesting. It would be great if you tried your hand at expressing the=

 Phobos build (which is rather simple) with redo so we can see the result.

The blurb indicates it doesn't work on Windows unless you use Cygwin of MSYS, because although it is written in Python it relies on /bin/sh. If the rather grandiose claims of the redo README are even close to reality, then the purveyors really ought to ensure successful builds on all platforms as Waf and SCons do. Then they might gain traction and be in for the running. Currently I would say use Waf or CMake. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Apr 26 2011
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
 On Tue, 2011-04-26 at 11:21 +0000, Lars T. Kyllingstad wrote:
 The std.parallelism vote is now over, and the results are as follows:
 24 voted yes
 
 0 voted no
 
 I guess that's what you call a landslide. It is my pleasure to announce
 that std.parallelism has been accepted for inclusion in Phobos.
 Congratulations, David!

Splendid. Thanks to David for doing all the really hard work. OK so now when do I get a version of Phobos with it in. Tomorrow, Thursday? ;-) More seriously though: I believe Phobos only gets released with a DMD release. Do we know when that is scheduled for?

We should be getting a release relatively soon. I believe that the main issue is that we need to make sure that Don's recent CTFE-related changes to dmd are stable enough to release, and they definitely aren't yet. For instance, I have a pull request for some improvements to std.datetime which compiled fine a few weeks back but gives a CTFE-related compile error now. I'd expect a release shortly after the compiler looks stable enough again and std.parallelism is in. - Jonathan M Davis
Apr 26 2011
parent reply David Nadlinger <see klickverbot.at> writes:
On 4/26/11 7:18 PM, Jonathan M Davis wrote:
 We should be getting a release relatively soon. I believe that the main issue
 is that we need to make sure that Don's recent CTFE-related changes to dmd are
 stable enough to release, and they definitely aren't yet. […]

Seconded – DMD currently can't build the CTFE-heavy parts of QtD (see my latest regression report at dmd-internals). David
Apr 26 2011
parent Don <nospam nospam.com> writes:
David Nadlinger wrote:
 On 4/26/11 7:18 PM, Jonathan M Davis wrote:
 We should be getting a release relatively soon. I believe that the 
 main issue
 is that we need to make sure that Don's recent CTFE-related changes to 
 dmd are
 stable enough to release, and they definitely aren't yet. […]

Seconded – DMD currently can't build the CTFE-heavy parts of QtD (see my latest regression report at dmd-internals). David

I posted pull requests for those. I imagine that Walter's been busy with his ACM talk, so they're not in the trunk yet.
Apr 27 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
I don't understand, why not just use gnu make for Windows?
http://gnuwin32.sourceforge.net/packages/make.htm

It doesn't need cygwin.
Apr 26 2011
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Apr 26, 2011, at 9:32 AM, Russel Winder wrote:

 On Tue, 2011-04-26 at 09:16 -0700, Sean Kelly wrote:
 Bootstrapping issue. Make is almost guaranteed to exist, while the =


=20
 That may be the case but Makefiles are either incomprehensibly
 complicated or platform specific -- it is nigh on impossible to create =

 single Makefile that does its job sensibly on Linux, Solaris, Mac OS X
 and Windows.  The advantage of tools such as Waf, SCons, CMake is that
 they manage the platform issues for you.=20

Windows is the odd man out there, although using make from GnuWin32 may = allow a single makefile to be used everywhere. As it is, we only have = two makefiles: one for Windows and one for Posix.
Apr 26 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
If D was available for every platform we could use a simple D script
to do the building for us. And hey, now we can even do parallel
builds! (std.parallelism, anyone?) :D

But then porting to new platforms would be a chicken and the egg problem.
Apr 26 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
In fact I'm really curious what a D script would look like that builds
Phobos and has all the option as the existing makefile.
Apr 26 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
options*
Apr 26 2011
prev sibling next sibling parent Russel Winder <russel russel.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 2011-04-26 at 21:45 +0100, Spacen Jasset wrote:
[ . . . ]
 Egr... I *might* give it a go at some point in Scons. Which is the only=

 tool I've found to (my) liking. It supports D too.

Is SCons rather than Waf the right tool here? SCons is really a Make replacement whereas Waf is an Autotools replacement. It really depends on what the intended workflow is. For "download the source and build on each of your machines" Waf is probably a better choice that SCons. For most other workflows, it is much more of a coin toss situation. Although SCons has built in D support (as does Waf) the current D tool in SCons isn't actually good enough. I seem to have become the official maintainer of the SCons D support, but rather than work with the SCons core source, I am using a separate tool for development. The Mercurial repository is at https://bitbucket.org/russel/scons_dmd_new Anyone using SCons for working with D source should use this add-in tool, help me develop it, and then I can get the results merged into the SCons core for the next release. If there is to be a SCons build for the D source then I guess the next step is to fork the D repository as a team and add files -- and also keep merging in changes from the mainline repository to ensure the fork is always up-to-date. Can this updating be automated?
 I must say that when dsss and rebuild were maintained it was very handy=

 indeed to be able to it with. (yeah - almost) identical build files on=

 the platforms I built for. Win & linux at the time.

Are these D-specific? --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Apr 26 2011
prev sibling parent Russel Winder <russel russel.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 2011-04-26 at 12:49 -0500, Andrei Alexandrescu wrote:
[ . . . ]
 This has been a common complaint about make... until gmake came about.=

 Targeting gmake newer than a specific version seldom causes=20
 platform-related issues.

Make (of whatever ilk) is fundamentally platform dependent because it is an external DSL. The dependency rules should work correctly on any platform -- though there will be a need for platform specific variation such as dealing with variant suffixes etc. -- but the actions are command lines for the platform in use.=20
 It is a great metric because they are in a ratio determined by the tools=

 used. 312 lines to build 143K lines seems a good ratio to me - probably=

 some of the best I've seen.
=20
 The 312 lines make is only for Phobos. If you could reduce and simplify=

 it, we'd all appreciate it.

Should be easy in either SCons or Waf. [ . . . ]
 What also matters is that stuff that stays is stuff that gets done. You=

 seem to have a good amount of expertise, but at the same time choose to=

 firmly plant your figurative self in the proverbial armchair. If you=20
 know how to do it, do it, don't attempt to beat others into doing it.

I think it a bit overly-aggressive to use phrases such as "choose to firmly plant your figurative self in the proverbial armchair" and "beat others into doing it". For people who have full-time jobs and therefore secure income who use some of their work time or spare time to work on projects such as D, there is the safety of having secure income. For those of us who have no full-time job it is a matter of all time spent on things like D is time not spent earning money. Firing off some emails is quick and apart from a few rants, I think most of my emails are constructive. It is therefore a serious barrier to be told I am a back seat driver and to "do something" don't "say something". With people such as Spacen Jasset tentatively volunteering to contribute to some work due to this exchange, to start getting censorious of my efforts to contribute and create an atmosphere of doing something seems somewhat counter-productive, certainly not community building. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Apr 26 2011