www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D based build system to replace current makefile

reply Timothee Cour <thelastmammoth gmail.com> writes:
--047d7b33d004a3e3bd04ddeabe09
Content-Type: text/plain; charset=ISO-8859-1

We should move to a D-based build system to build
dmd/druntime/phobos/d-programming-language/tools.

Reasons, in case it's not obvious:

1) DRY: makefile is full of repetitions
2) cross platform: different makefiles for different architectures are
needed (even win32 vs win64!)
3) safety: makefile happily ignores that $VAR variables are not set, and
can create havoc; likewise tools/update.sh is a bash script and may not
stop on 1st error.
4) tools/update.sh doesn't work out of the box on OSX last i checked (I
made a pull request for that some time back:
https://github.com/timotheecour/d-programming-language.org/commit/557a2befa74ddfe99ee5c0e12e7c3d028f27d276)

Even a bad D-based build system will be better than a makefile, but it's
not hard to write a reasonable one at least for the particular task of
building dmd/druntime/phobos/d-programming-language/tools. It'll lower the
entry point for people to contribute to dmd/phobos by making rebuilding
trivial.

Here's how it'd work:

1) a mixed D-based/makefile build that'll work 'from scratch' via
bootstrapping:
1a) dmd: first uses a makefile to build dmd+druntime
1b) rdmd built from a D file using dmd
1c) phobos, d-programming-language, tools built from a D file using rdmd

2) a pure D-based build that'll work using an existing dmd binary (eg
distributed binary)
that's the one most will use.

--047d7b33d004a3e3bd04ddeabe09
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>We should move to a D-based build system to build dmd/druntime/phobos/=
d-programming-language/tools.=A0</div><div><br></div><div>Reasons, in case =
it&#39;s not obvious:=A0</div><div><br></div><div>1) DRY: makefile is full =
of repetitions</div>
<div>2) cross platform: different makefiles for different architectures are=
 needed (even win32 vs win64!)</div><div>3) safety: makefile happily ignore=
s that $VAR variables are not set, and can create havoc; likewise tools/upd=
ate.sh is a bash script and may not stop on 1st error.</div>
<div>4) tools/update.sh doesn&#39;t work out of the box on OSX last i check=
ed (I made a pull request for that some time back: <a href=3D"https://githu=
b.com/timotheecour/d-programming-language.org/commit/557a2befa74ddfe99ee5c0=
e12e7c3d028f27d276">https://github.com/timotheecour/d-programming-language.=
org/commit/557a2befa74ddfe99ee5c0e12e7c3d028f27d276</a> )</div>
<div><br></div><div>Even a bad D-based build system will be better than a m=
akefile, but it&#39;s not hard to write a reasonable one at least for the p=
articular task of building dmd/druntime/phobos/d-programming-language/tools=
. It&#39;ll lower the entry point for people to contribute to dmd/phobos by=
 making rebuilding trivial.</div>
<div><br></div><div>Here&#39;s how it&#39;d work:</div><div><br></div><div>=
1) a mixed D-based/makefile build that&#39;ll work &#39;from scratch&#39; v=
ia bootstrapping:</div><div>1a) dmd: first uses a makefile to build dmd+dru=
ntime</div>
<div>1b) rdmd built from a D file using dmd</div><div>1c) phobos,=A0d-progr=
amming-language, tools built from a D file using rdmd</div><div><br></div><=
div>2) a pure D-based build that&#39;ll work using an existing dmd binary (=
eg distributed binary)</div>
<div>that&#39;s the one most will use.<br></div><div><br></div><div><br></d=
iv><div><br></div>

--047d7b33d004a3e3bd04ddeabe09--
May 30 2013
next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 30 May 2013 00:49:57 -0700
Timothee Cour <thelastmammoth gmail.com> wrote:
 
 Here's how it'd work:
 
 1) a mixed D-based/makefile build that'll work 'from scratch' via
 bootstrapping:
 1a) dmd: first uses a makefile to build dmd+druntime
 1b) rdmd built from a D file using dmd
 1c) phobos, d-programming-language, tools built from a D file using
 rdmd
 

Related: https://github.com/D-Programming-Language/installer/pull/18#issuecomment-18662826
May 30 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 30 May 2013 06:13:54 -0400
Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> wrote:

 On Thu, 30 May 2013 00:49:57 -0700
 Timothee Cour <thelastmammoth gmail.com> wrote:
 
 Here's how it'd work:
 
 1) a mixed D-based/makefile build that'll work 'from scratch' via
 bootstrapping:
 1a) dmd: first uses a makefile to build dmd+druntime
 1b) rdmd built from a D file using dmd
 1c) phobos, d-programming-language, tools built from a D file using
 rdmd
 

Related: https://github.com/D-Programming-Language/installer/pull/18#issuecomment-18662826

Erm, I mean: https://github.com/D-Programming-Language/installer/pull/18
May 30 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 30 May 2013 00:49:57 -0700
Timothee Cour <thelastmammoth gmail.com> wrote:

 We should move to a D-based build system to build
 dmd/druntime/phobos/d-programming-language/tools.
 
 Reasons, in case it's not obvious:
 
 1) DRY: makefile is full of repetitions
 2) cross platform: different makefiles for different architectures are
 needed (even win32 vs win64!)
 3) safety: makefile happily ignores that $VAR variables are not set,
 and can create havoc; likewise tools/update.sh is a bash script and
 may not stop on 1st error.
 4) tools/update.sh doesn't work out of the box on OSX last i checked
 (I made a pull request for that some time back:
 https://github.com/timotheecour/d-programming-language.org/commit/557a2befa74ddfe99ee5c0e12e7c3d028f27d276)
 
 Even a bad D-based build system will be better than a makefile, but
 it's not hard to write a reasonable one at least for the particular
 task of building dmd/druntime/phobos/d-programming-language/tools.
 It'll lower the entry point for people to contribute to dmd/phobos by
 making rebuilding trivial.
 
 Here's how it'd work:
 
 1) a mixed D-based/makefile build that'll work 'from scratch' via
 bootstrapping:
 1a) dmd: first uses a makefile to build dmd+druntime
 1b) rdmd built from a D file using dmd
 1c) phobos, d-programming-language, tools built from a D file using
 rdmd
 
 2) a pure D-based build that'll work using an existing dmd binary (eg
 distributed binary)
 that's the one most will use.
 

FWIW, phobos needs to be built before rdmd because rdmd relies on phobos. Also, while I like the idea of replacing makefiles with a D solution, I'm unsure whether it's really a good idea because that may make it more difficult to bootstrap DMD to run on new platforms. However, I do think that wrapping the invocations of all the makefiles is a good idea and would still have the benefit of making it harder to make mistakes such as (like you said) setting up the %VARs wrong. In fact I've started working on such stuff already (And DVM can already compile DMD, too, although it's possible it may still have some room for improvement).
May 30 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, May 30, 2013 16:47:49 Nick Sabalausky wrote:
 Also, while I like the idea of replacing makefiles with a D solution,
 I'm unsure whether it's really a good idea because that may make it
 more difficult to bootstrap DMD to run on new platforms.

I would point out that we're going to have that problem anyway once we move the compiler to D. At some point, already having a fully functional D compiler _will_ be required in order to build dmd, druntime, and Phobos. If we switched to using D for writing the build scripts before switching the compiler to D, we'd simply be introducing that circular dependency earlier. So, that shouldn't ultimately be much of a factor in whether we make the switch. Whether we switch to using D scripts for building dmd, druntime, and Phobos should depend on whether the result would be more maintainable and still be able to reasonably do everything that we need our build system to do. And while many of us really aren't very good at make and find the Makefiles that we have to be rather obtuse, folks that are more proficient at make might not think that making the switch to a D script would make sense. And I really don't know what the tradeoffs would be. I did go with writing my build script in D for a recent project which was generating a document written in ddoc, and it's working out quite well to do that, but I don't know whether it ultimately makes sense to do the same for the dmd toolchain. - Jonathan M Davis
May 30 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 30 May 2013 16:56:48 -0400
"Jonathan M Davis" <jmdavisProg gmx.com> wrote:
 
 And while many of us really aren't very good at make
 and find the Makefiles that we have to be rather obtuse, folks that
 are more proficient at make might not think that making the switch to
 a D script would make sense. And I really don't know what the
 tradeoffs would be.
 

I normally despise make, but FWIW I do find DMD's makefiles to be *vastly* nicer than others, probably because they doesn't really use the DRY-ness features that usually cause make to be such a pain.
May 30 2013
prev sibling parent "Flamaros" <flamaros.xavier gmail.com> writes:
On Thursday, 30 May 2013 at 07:50:11 UTC, Timothee Cour wrote:
 We should move to a D-based build system to build
 dmd/druntime/phobos/d-programming-language/tools.

 Reasons, in case it's not obvious:

 1) DRY: makefile is full of repetitions
 2) cross platform: different makefiles for different 
 architectures are
 needed (even win32 vs win64!)
 3) safety: makefile happily ignores that $VAR variables are not 
 set, and
 can create havoc; likewise tools/update.sh is a bash script and 
 may not
 stop on 1st error.
 4) tools/update.sh doesn't work out of the box on OSX last i 
 checked (I
 made a pull request for that some time back:
 https://github.com/timotheecour/d-programming-language.org/commit/557a2befa74ddfe99ee5c0e12e7c3d028f27d276)

 Even a bad D-based build system will be better than a makefile, 
 but it's
 not hard to write a reasonable one at least for the particular 
 task of
 building dmd/druntime/phobos/d-programming-language/tools. 
 It'll lower the
 entry point for people to contribute to dmd/phobos by making 
 rebuilding
 trivial.

 Here's how it'd work:

 1) a mixed D-based/makefile build that'll work 'from scratch' 
 via
 bootstrapping:
 1a) dmd: first uses a makefile to build dmd+druntime
 1b) rdmd built from a D file using dmd
 1c) phobos, d-programming-language, tools built from a D file 
 using rdmd

 2) a pure D-based build that'll work using an existing dmd 
 binary (eg
 distributed binary)
 that's the one most will use.

You may want take a look at Qbs, it will the next build system of Qt in replacement of QMake. http://qt-project.org/wiki/Qbs-Quick-Reference Using a full featured language is good cause, there is no restriction and build aren't just build, it's also packaging/testing issues. Doing something that just have to call compiler binaries is finaly the simple thing, and provide more control to users.
May 30 2013