www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Prototype buildsystem "Drake"

reply "Nick Sabalausky" <a a.a> writes:
The recent discussions about package managers and buildsystems has prompted 
me to get off my ass (or rather, *on* my ass as per how programming is 
usually performed...) and start on the D-based rake-inspired buildtool I've 
been meaning to make for awhile now. It's not actually usable yet, but 
there's a sample drakefile demonstrating everything, and it does actually 
compile and run (but just doesn't build the dependency tree or actually run 
any of the build steps). *Should* work on Posix, but I only tested on 
Windows, so I may have fucked up the Posix version...

Apologies to Jacob Carlborg for the name being so close to "dake". Didn't 
really know what else to call it ("Duck", maybe?) Like dake, it's inspired 
by Ruby's Rake. But unlike dake, the buildscript is written in D instead of 


Before I go implemeting everything, I'd like to get input on it. Is it 
something that could be a major D tool?

Overview of Drake and links to all the code are here:

https://bitbucket.org/Abscissa256/drake/wiki/Home 
Jul 12 2011
next sibling parent reply Russel Winder <russel russel.org.uk> writes:
On Tue, 2011-07-12 at 21:02 -0400, Nick Sabalausky wrote:
[ . . . ]
 Before I go implemeting everything, I'd like to get input on it. Is it=
=20
 something that could be a major D tool?
[ . . . ] Given the nature of the debate, I will add to the mix that SCons, a pre-existing -- and indeed working :-) -- system, has a D tool and can compile and link D code. What it needs is some love to bring it up to the level of the C and C++ support. SCons has a built-in D tool which needs work, but rather than fork SCons to work on the tool I have created a separate tool that can be used with any SCons installation. See https://bitbucket.org/russel/scons_dmd_new. Prior to any SCons release a patch between this version and the one in the SCons core will be made and applied. I started using SCons (and recently Waf -- which also has a D tool) in preference to Rake because SCons has very superior support for C, C++, Fortran and LaTeX. Using Rake for these is like using assembly language to write a GUI application: with Rake you have to build everything yourself, with SCons most of the work is done for you in the tool infrastructure. There was Rant which tried to replace Rake in the C, C ++, Fortran arena but the project died. Rake appears to have almost no traction outside the Ruby community. Even Buildr (which tried to build on Rake to combat Maven seems to have no headway in the market. Clearly there is always a trend for a language to demand a specialist build tool: Scala has SBT, Clojure has Leiningen, Ruby has Rake, but there is also a section of the universe that thinks the same build tool should be usable across all languages. Ant and Maven were Java specific but now can handle anything targeting the JVM. SCons and Waf come from a C, C++, Fortran, LaTeX, D background but can now handle Java, Scala, etc. Gradle arose as a replacement for Maven on the JVM, but is now (finally) branching out into C, C++, etc. Note also that systems like Gradle and indeed Maven, are not just code compiler and link systems, they also manage code review tools for creating coverage reports, bug reports, documentation generation, even whole websites. Currently, from what I can tell, we have a number of individuals making proposals for ideas. Nothing wrong with that per se. However I think this is the third time this debate has occurred since I have been (peripherally) involved with the core D community. This does strike me as wrong since it means that there is no momentum being created, the energy associated with each debate simply dissipates. I think that in order to create a momentum, a fundamental choice needs to be made: -- Should D have a specialist tool (=C3=A0 la SBT/Scala, Leiningen/Clojure -- at the risk of massive NIH-ism, just as the Scala and Clojure folk appear to have) or go with a generalist tool such as Gradle or SCons. True there has to be debate about the possibilities in each category so as to get the "lay of the land" and a feel for the "pluses" and "minuses", but always the aim should be to answer the above question. Then we can move to debating the possibilities. Then we can create some momentum behind doing something by creating a group of activists. Then D build wins. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jul 12 2011
next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Russel Winder" <russel russel.org.uk> wrote in message 
news:mailman.1585.1310533819.14074.digitalmars-d puremagic.com...
On Tue, 2011-07-12 at 21:02 -0400, Nick Sabalausky wrote:
[ . . . ]
 Before I go implemeting everything, I'd like to get input on it. Is it
 something that could be a major D tool?
[ . . . ] Given the nature of the debate, I will add to the mix that SCons, a pre-existing -- and indeed working :-) -- system, has a D tool and can compile and link D code. What it needs is some love to bring it up to the level of the C and C++ support. SCons has a built-in D tool which needs work, but rather than fork SCons to work on the tool I have created a separate tool that can be used with any SCons installation. See https://bitbucket.org/russel/scons_dmd_new. Prior to any SCons release a patch between this version and the one in the SCons core will be made and applied.
[...Waf, Rake, Ant, Maven, etc...] First of all, just to be clear, Drake isn't D-specific. Like Rake, it's designed to work with anything you can programmatically invoke (and doesn't require any special module to support a particular language or tool). After having worked on projects that, by necessity, involved multiple languages, I find that quality essential in a build system. Regarding SCons, Waf, Rake, etc, the point of Drake is that 1: it doesn't force D projects to be reliant on Python, Ruby, etc. And 2: It allows D users to use D for their buildscript instead of needing to switch to a different (and less preferable IMO) langauge. (And I *hate* language-soup projects - projects that use a totally different language for every little damn thing.) The *make tools (whether they use makefiles directly or indirectly) aren't even worth consideration. Makefiles need to just die, period. Regarding this topic's frequent ressurection, I see that as a direct result of it all being *merely* talk, with very little concrete progress. We don't even *have* a proper package management system (there's DSSS, but it's dead, bitrotted, and I don't think it handles dependencies or multiple versions of the same package). So naturally that topic's going to keep coming up until we have something. And as for build systems, everything either lacks multiple-target/configuration management (rdmd), or creates a dependency on (and a need to write in) some other langauge (SCons, Rake, etc). So naturally there's also going to be people that feel unsatisfied with the build systems until something fills that role. I'm trying to get past "talk", so I've started some actual code to address what I see as the current issues for build systems. Of course, it doesn't help that everyone seems to want something completely different from what everyone else wants. ;)
Jul 13 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday 13 July 2011 03:01:04 Nick Sabalausky wrote:
 Regarding this topic's frequent ressurection, I see that as a direct result
 of it all being *merely* talk, with very little concrete progress. We don't
 even *have* a proper package management system (there's DSSS, but it's dead,
 bitrotted, and I don't think it handles dependencies or multiple versions
 of the same package). So naturally that topic's going to keep coming up
 until we have something.
I belive that Jacob Carlborg is working on a solution. So, there _is_ work being done on it. By no means does that mean that you shouldn't work on a solution, but it's not the case that no one has been working on the problem - though it is true that it's been being brought up for far longer than it's been being worked on. - Jonathan M Davis
Jul 13 2011
next sibling parent "Nick Sabalausky" <a a.a> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1593.1310541508.14074.digitalmars-d puremagic.com...
 On Wednesday 13 July 2011 03:01:04 Nick Sabalausky wrote:
 Regarding this topic's frequent ressurection, I see that as a direct 
 result
 of it all being *merely* talk, with very little concrete progress. We 
 don't
 even *have* a proper package management system (there's DSSS, but it's 
 dead,
 bitrotted, and I don't think it handles dependencies or multiple versions
 of the same package). So naturally that topic's going to keep coming up
 until we have something.
I belive that Jacob Carlborg is working on a solution. So, there _is_ work being done on it.
Right. And I'm definitely keeping an eye on Orb. I just meant that the lack of a D package manager that's already usable (and still living) is the main reason this all keeps getting discussed every so often.
 By no means does that mean that you shouldn't work on a
 solution, but it's not the case that no one has been working on the 
 problem -
 though it is true that it's been being brought up for far longer than it's
 been being worked on.
Yea, I didn't mean that nobody's working on it. Just that, for package management, there isn't really much usable right now, and for build management, there isn't anything that doesn't rely on either makefiles or XML or some non-D dynamic langauge/interpreter, etc.
Jul 13 2011
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-07-13 09:18, Jonathan M Davis wrote:
 On Wednesday 13 July 2011 03:01:04 Nick Sabalausky wrote:
 Regarding this topic's frequent ressurection, I see that as a direct result
 of it all being *merely* talk, with very little concrete progress. We don't
 even *have* a proper package management system (there's DSSS, but it's dead,
 bitrotted, and I don't think it handles dependencies or multiple versions
 of the same package). So naturally that topic's going to keep coming up
 until we have something.
I belive that Jacob Carlborg is working on a solution. So, there _is_ work being done on it. By no means does that mean that you shouldn't work on a solution, but it's not the case that no one has been working on the problem - though it is true that it's been being brought up for far longer than it's been being worked on. - Jonathan M Davis
Yes, I do work on that. And a build tool as well. -- /Jacob Carlborg
Jul 13 2011
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-07-13 07:10, Russel Winder wrote:
 I think that in order to create a momentum, a fundamental choice needs
 to be made:

 --  Should D have a specialist tool (à la SBT/Scala, Leiningen/Clojure
 -- at the risk of massive NIH-ism, just as the Scala and Clojure folk
 appear to have) or go with a generalist tool such as Gradle or SCons.
I think that we should go for a D specific build tool. -- /Jacob Carlborg
Jul 13 2011
prev sibling next sibling parent Russel Winder <russel russel.org.uk> writes:
And, of course, I should have mentioned CMake and CMakeD.

The fact that I forgot, shows my prejudice against Makefile-based
systems and for direct DAG-based systems such as Gradle, SCons and Waf.
This though should not stop CMakeD being a part of this debate.=20

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel russel.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jul 12 2011
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday 13 July 2011 06:12:58 Russel Winder wrote:
 And, of course, I should have mentioned CMake and CMakeD.
 
 The fact that I forgot, shows my prejudice against Makefile-based
 systems and for direct DAG-based systems such as Gradle, SCons and Waf.
 This though should not stop CMakeD being a part of this debate.
From previous discussions, it seems that one of the primary reasons for having 
a D build tool in many people's minds is to also handle package management of D libraries (like Haskell's cabal or rubygems for ruby). And as great as cmaked, scons, gradle, waf, and other such tools may be, they don't do that. - Jonathan M Davis
Jul 12 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-13 07:28, Jonathan M Davis wrote:
 On Wednesday 13 July 2011 06:12:58 Russel Winder wrote:
 And, of course, I should have mentioned CMake and CMakeD.

 The fact that I forgot, shows my prejudice against Makefile-based
 systems and for direct DAG-based systems such as Gradle, SCons and Waf.
 This though should not stop CMakeD being a part of this debate.
 From previous discussions, it seems that one of the primary reasons for having
a D build tool in many people's minds is to also handle package management of D libraries (like Haskell's cabal or rubygems for ruby). And as great as cmaked, scons, gradle, waf, and other such tools may be, they don't do that. - Jonathan M Davis
I don't agree with that. I think a build tool should deal with single files and building. A package manager should deal with packages (of files). In Ruby, RubyGems is the package manager and Rake is the build tool. -- /Jacob Carlborg
Jul 13 2011
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-07-13 08:36, Jacob Carlborg wrote:
 On 2011-07-13 07:28, Jonathan M Davis wrote:
 On Wednesday 13 July 2011 06:12:58 Russel Winder wrote:
 And, of course, I should have mentioned CMake and CMakeD.
 
 The fact that I forgot, shows my prejudice against Makefile-based
 systems and for direct DAG-based systems such as Gradle, SCons and Waf.
 This though should not stop CMakeD being a part of this debate.
 
 From previous discussions, it seems that one of the primary reasons for
 having
a D build tool in many people's minds is to also handle package management of D libraries (like Haskell's cabal or rubygems for ruby). And as great as cmaked, scons, gradle, waf, and other such tools may be, they don't do that. - Jonathan M Davis
I don't agree with that. I think a build tool should deal with single files and building. A package manager should deal with packages (of files). In Ruby, RubyGems is the package manager and Rake is the build tool.
Well, I'm not advocating anything in particular. I was just pointing out that a big part of the discussions on build tools has been package management of libraries, and any build tool solution which doesn't at least integrate with some sort of package management solution is likely to not be what at least some people are looking for. Personally, I don't generally use package management tools for handling libraries even with languages that have such tools, and I don't generally use much in the way of build tools either beyond simple scripts (primarily because I don't generally have projects large enough for it to be an issue). As it stands, if I were to choose a build tool for a larger project, I'd probably choose CmakeD, but I'm not super-familiar with all of the tools out there and haven't generally found much use for them. I was just trying to point out that a fair bit of the discussion for such tools in this list has related to package management, and Nick's solution doesn't address that at all AFAIK. - Jonathan m Davis
Jul 13 2011
next sibling parent "Nick Sabalausky" <a a.a> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1611.1310578056.14074.digitalmars-d puremagic.com...
 On 2011-07-13 08:36, Jacob Carlborg wrote:
 I don't agree with that. I think a build tool should deal with single
 files and building. A package manager should deal with packages (of
 files). In Ruby, RubyGems is the package manager and Rake is the build
 tool.
Well, I'm not advocating anything in particular. I was just pointing out that a big part of the discussions on build tools has been package management of libraries, and any build tool solution which doesn't at least integrate with some sort of package management solution is likely to not be what at least some people are looking for. Personally, I don't generally use package management tools for handling libraries even with languages that have such tools, and I don't generally use much in the way of build tools either beyond simple scripts (primarily because I don't generally have projects large enough for it to be an issue). As it stands, if I were to choose a build tool for a larger project, I'd probably choose CmakeD, but I'm not super-familiar with all of the tools out there and haven't generally found much use for them. I was just trying to point out that a fair bit of the discussion for such tools in this list has related to package management, and Nick's solution doesn't address that at all AFAIK.
Fair enough. And yea, my solution doesn't even try to address package management since it considers that a separate issue. May have been a little misleading for me to bring up those discussions in the first place. Really what happened was just: For a long time I'd been meaning to make a Rake-like tool based on D, Jacob posted his Rake-like tool based on Ruby, and that lit a fire under me to finally get started on mine. Although I have to admit, I've only now started to look at Waf. While I definitely still wouldn't want to use it for D projects (for reasons I've already mentioned), it does have some interesting design elements that I'm going to have to think about.
Jul 13 2011
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-13 19:27, Jonathan M Davis wrote:
 On 2011-07-13 08:36, Jacob Carlborg wrote:
 On 2011-07-13 07:28, Jonathan M Davis wrote:
 On Wednesday 13 July 2011 06:12:58 Russel Winder wrote:
 And, of course, I should have mentioned CMake and CMakeD.

 The fact that I forgot, shows my prejudice against Makefile-based
 systems and for direct DAG-based systems such as Gradle, SCons and Waf.
 This though should not stop CMakeD being a part of this debate.

  From previous discussions, it seems that one of the primary reasons for
 having
a D build tool in many people's minds is to also handle package management of D libraries (like Haskell's cabal or rubygems for ruby). And as great as cmaked, scons, gradle, waf, and other such tools may be, they don't do that. - Jonathan M Davis
I don't agree with that. I think a build tool should deal with single files and building. A package manager should deal with packages (of files). In Ruby, RubyGems is the package manager and Rake is the build tool.
Well, I'm not advocating anything in particular. I was just pointing out that a big part of the discussions on build tools has been package management of libraries, and any build tool solution which doesn't at least integrate with some sort of package management solution is likely to not be what at least some people are looking for.
Ok, I see. I think as well that the build tool and package manager should interact with each other. For example, specifying package dependencies in the build script. But, I think they need to be separate and be usable on their own.
 Personally, I don't generally use package management tools for handling
 libraries even with languages that have such tools, and I don't generally use
 much in the way of build tools either beyond simple scripts (primarily because
 I don't generally have projects large enough for it to be an issue). As it
 stands, if I were to choose a build tool for a larger project, I'd probably
 choose CmakeD, but I'm not super-familiar with all of the tools out there and
 haven't generally found much use for them.

 I was just trying to point out that a fair bit of the discussion for such
 tools in this list has related to package management, and Nick's solution
 doesn't address that at all AFAIK.

 - Jonathan m Davis
Ok, I see. -- /Jacob Carlborg
Jul 13 2011
parent reply James Fisher <jameshfisher gmail.com> writes:
When writing Haskell and compiling with GHC, the compiler automatically
finds the module dependencies and compiles them in the correct order.  If I
touch a few modules and recompile, GHC will only recompile the touched
modules as appropriate, then re-link.  In other words, GHC is the build tool
as well as the compiler.  The advantages of this approach over having a
separate build tool are:

- the compiler already knows how to parse module files, so it's in the
perfect position to know what the dependencies are.  We don't have some
hacky extra tool sniffing out lines that look like "import X;".  This alone
seems error-prone, but more importantly, D modules can have arbitrary
compile-time logic executed before importing something, so the build tool
essentially either has to give up or reimplement the compiler to cover all
cases.  (This is the primary reason I advocate this approach -- I don't see
how a generic build tool can automatically detect dependencies without the
participation of a D compiler.)
- the steps to compile a one-file program are exactly the same as compiling
a complex multi-module program: "ghc Main".  If I split out one file into
several, I don't have to immediately switch to using Make/waf/etc; it will
Just Work.  There's one less tool to remember, and once you've got your D
compiler, you're all set.

On Wed, Jul 13, 2011 at 7:28 PM, Jacob Carlborg <doob me.com> wrote:

 On 2011-07-13 19:27, Jonathan M Davis wrote:

 On 2011-07-13 08:36, Jacob Carlborg wrote:

 On 2011-07-13 07:28, Jonathan M Davis wrote:

 On Wednesday 13 July 2011 06:12:58 Russel Winder wrote:

 And, of course, I should have mentioned CMake and CMakeD.

 The fact that I forgot, shows my prejudice against Makefile-based
 systems and for direct DAG-based systems such as Gradle, SCons and Waf.
 This though should not stop CMakeD being a part of this debate.

  From previous discussions, it seems that one of the primary reasons
 for
 having
a D build tool in many people's minds is to also handle package management of D libraries (like Haskell's cabal or rubygems for ruby). And as great as cmaked, scons, gradle, waf, and other such tools may be, they don't do that. - Jonathan M Davis
I don't agree with that. I think a build tool should deal with single files and building. A package manager should deal with packages (of files). In Ruby, RubyGems is the package manager and Rake is the build tool.
Well, I'm not advocating anything in particular. I was just pointing out that a big part of the discussions on build tools has been package management of libraries, and any build tool solution which doesn't at least integrate with some sort of package management solution is likely to not be what at least some people are looking for.
Ok, I see. I think as well that the build tool and package manager should interact with each other. For example, specifying package dependencies in the build script. But, I think they need to be separate and be usable on their own. Personally, I don't generally use package management tools for handling
 libraries even with languages that have such tools, and I don't generally
 use
 much in the way of build tools either beyond simple scripts (primarily
 because
 I don't generally have projects large enough for it to be an issue). As it
 stands, if I were to choose a build tool for a larger project, I'd
 probably
 choose CmakeD, but I'm not super-familiar with all of the tools out there
 and
 haven't generally found much use for them.

 I was just trying to point out that a fair bit of the discussion for such
 tools in this list has related to package management, and Nick's solution
 doesn't address that at all AFAIK.

 - Jonathan m Davis
Ok, I see. -- /Jacob Carlborg
Jul 18 2011
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-07-18 19:37, James Fisher wrote:
 When writing Haskell and compiling with GHC, the compiler automatically
 finds the module dependencies and compiles them in the correct order.
   If I touch a few modules and recompile, GHC will only recompile the
 touched modules as appropriate, then re-link.  In other words, GHC is
 the build tool as well as the compiler.  The advantages of this approach
 over having a separate build tool are:

 - the compiler already knows how to parse module files, so it's in the
 perfect position to know what the dependencies are.  We don't have some
 hacky extra tool sniffing out lines that look like "import X;".  This
 alone seems error-prone, but more importantly, D modules can have
 arbitrary compile-time logic executed before importing something, so the
 build tool essentially either has to give up or reimplement the compiler
 to cover all cases.  (This is the primary reason I advocate this
 approach -- I don't see how a generic build tool can automatically
 detect dependencies without the participation of a D compiler.)
 - the steps to compile a one-file program are exactly the same as
 compiling a complex multi-module program: "ghc Main".  If I split out
 one file into several, I don't have to immediately switch to using
 Make/waf/etc; it will Just Work.  There's one less tool to remember, and
 once you've got your D compiler, you're all set.
With DMD you can use the -deps flag to get all the dependencies of all specified files. No reason to try to parse import statements. Can GHC compiler other types than executables, i.e. libraries? -- /Jacob Carlborg
Jul 18 2011
prev sibling parent "Nick Sabalausky" <a a.a> writes:
"James Fisher" <jameshfisher gmail.com> wrote in message 
news:mailman.1733.1311010642.14074.digitalmars-d puremagic.com...
 When writing Haskell and compiling with GHC, the compiler automatically
 finds the module dependencies and compiles them in the correct order.  If 
 I
 touch a few modules and recompile, GHC will only recompile the touched
 modules as appropriate, then re-link.  In other words, GHC is the build 
 tool
 as well as the compiler.  The advantages of this approach over having a
 separate build tool are:
In the context of the rest of this thread, "build tool" refers to something that can handle multiple targets, multiple configurations and custom build steps. So that doesn't sound like quite the same thing. D does already have RDMD which is equivalent to what you're talking about (and it uses the -deps flag Jacob mentioned to get the deps directly from the actual compiler).
Jul 18 2011
prev sibling next sibling parent reply Russel Winder <russel russel.org.uk> writes:
On Tue, 2011-07-12 at 22:28 -0700, Jonathan M Davis wrote:
[ . . . ]
 From previous discussions, it seems that one of the primary reasons for h=
aving=20
 a D build tool in many people's minds is to also handle package managemen=
t of=20
 D libraries (like Haskell's cabal or rubygems for ruby). And as great as=
=20
 cmaked, scons, gradle, waf, and other such tools may be, they don't do th=
at. Go is currently using Make for this -- they have a structured Makefile hierarchy that handles most compilation and linking in the context of a rigidly enforced filestore structure, and goinstall for bringing in packages from outside into the filestore hierarchy. It is a bit primitive at the minute, but is being worked on and rapidly improved. Go actually has a plethora of build tools, include a couple of SCons-based ones, most of which are falling by the wayside. I think the effort expended doing this has not been a waste as information is being generated that is adding to the pool. I think they will replace the Makefile system shortly with one of the tools, possibly GoBuild, but I can't remember exactly. Haskell and Ruby both have inward looking approaches -- to put it bluntly (at the risk of causing offense to some), Haskell build only cares about Haskell source and Ruby build only cares about Ruby source. Almost by definition D has to live in a mixed C, C++, Fortran, D universe, so this has to be an issue from the outset. I agree with the premise that package management must be a core part of the build management, but I disagree that Gradle, SCons, and Waf cannot handle this. They cannot handle this today true, but in the same way that there is currently no system that does it properly as yet. So I would suggest that package management is currently a "green field" problem that can be handled by a D-specific tool or one of Gradle, SCons or Waf. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jul 12 2011
parent reply "Nick Sabalausky" <a a.a> writes:
From: "Russel Winder" <russel russel.org.uk>
I agree with the premise that package management must be a core part of
the build management,
If I understand what you're saying correctly, then I strongly disagree. A package management tool, by necessity, must be a common agreed-upon standard (otherwise it won't serve it's purpose). But if that package management is tied into build management then we're *also* forcing a single standard build system on everyone. So then instead of being able to choose your build system, everyone now has to come to an agreement on the same build system,
Jul 12 2011
parent reply Kagamin <spam here.lot> writes:
Nick Sabalausky Wrote:

 From: "Russel Winder" <russel russel.org.uk>
I agree with the premise that package management must be a core part of
the build management,
If I understand what you're saying correctly, then I strongly disagree. A package management tool, by necessity, must be a common agreed-upon standard (otherwise it won't serve it's purpose). But if that package management is tied into build management then we're *also* forcing a single standard build system on everyone. So then instead of being able to choose your build system, everyone now has to come to an agreement on the same build system,
A build tool should be able to invoke a package manager and package manager should be able to invoke a build tool, this doesn't mean a tie, it means a working out of the box solution. This solution is meant to just work. If people don't want solution that just works, they can go with something else of course. For example I prefer makefiles.
Jul 13 2011
parent "Nick Sabalausky" <a a.a> writes:
"Kagamin" <spam here.lot> wrote in message 
news:ivjlkm$1amf$1 digitalmars.com...
 Nick Sabalausky Wrote:

 From: "Russel Winder" <russel russel.org.uk>
I agree with the premise that package management must be a core part of
the build management,
If I understand what you're saying correctly, then I strongly disagree. A package management tool, by necessity, must be a common agreed-upon standard (otherwise it won't serve it's purpose). But if that package management is tied into build management then we're *also* forcing a single standard build system on everyone. So then instead of being able to choose your build system, everyone now has to come to an agreement on the same build system,
A build tool should be able to invoke a package manager and package manager should be able to invoke a build tool, this doesn't mean a tie, it means a working out of the box solution.
Ok, that does make sense.
This solution is meant to just work. If people don't want solution that 
just works, they can go with something else of course. For example I prefer 
makefiles. 
Jul 13 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday 13 July 2011 06:53:28 Russel Winder wrote:
 On Tue, 2011-07-12 at 22:28 -0700, Jonathan M Davis wrote:
 [ . . . ]
 
 From previous discussions, it seems that one of the primary reasons for
 having a D build tool in many people's minds is to also handle package
 management of D libraries (like Haskell's cabal or rubygems for ruby).
 And as great as cmaked, scons, gradle, waf, and other such tools may
 be, they don't do that.
Go is currently using Make for this -- they have a structured Makefile hierarchy that handles most compilation and linking in the context of a rigidly enforced filestore structure, and goinstall for bringing in packages from outside into the filestore hierarchy. It is a bit primitive at the minute, but is being worked on and rapidly improved. Go actually has a plethora of build tools, include a couple of SCons-based ones, most of which are falling by the wayside. I think the effort expended doing this has not been a waste as information is being generated that is adding to the pool. I think they will replace the Makefile system shortly with one of the tools, possibly GoBuild, but I can't remember exactly. Haskell and Ruby both have inward looking approaches -- to put it bluntly (at the risk of causing offense to some), Haskell build only cares about Haskell source and Ruby build only cares about Ruby source. Almost by definition D has to live in a mixed C, C++, Fortran, D universe, so this has to be an issue from the outset. I agree with the premise that package management must be a core part of the build management, but I disagree that Gradle, SCons, and Waf cannot handle this. They cannot handle this today true, but in the same way that there is currently no system that does it properly as yet. So I would suggest that package management is currently a "green field" problem that can be handled by a D-specific tool or one of Gradle, SCons or Waf.
Feel free to propose whatever you'd like for a D build system (be it something entirely new or an improvement of an existing build tool). I'm just pointing out that one of the primary reasons that a number of the people on this list want a build tool is to handle package management of D libraries, so whatever solution you propose should incorporate that or a lot of people aren't going to be all that interested in it or will at least see it as inferior to another solution which does incorporate package management. - Jonathan M Davis
Jul 12 2011
prev sibling next sibling parent reply Chris Molozian <chris cmoz.me> writes:
I asked about build tools for D on the mailing list a while ago. I 
needed a solution that allowed me to mix C++ and D builds in a 
cross-platform way with minimum fuss. You can find the discussion about 
it here 
<http://www.digitalmars.com/d/archives/digitalmars/D/Best_build_tool_for_D_p
ojects_136103.html> 
(you were also a part of it IIRC). My biggest requirement at the time was:

  * Keeping platform checks e.g. IF (MAC) {} ELSE IF(LINUX) {} ... etc.
    to an absolute minimum. What's the point in a cross-platform
    language if when you build projects in it you need to write a short
    essay for the build system...
  * Pre-built binaries available to all platforms (or as many as possible).

In the end I settled for Jam (ftjam) 
<http://www.freetype.org/jam/index.html>, it works tremendously well. 
It's easy to get binaries for almost any operating system (especially 
with projects like homebrew <http://mxcl.github.com/homebrew/> for Mac). 
I have very few gripes with it:

  * The build description language is very simple (once you fully grok
    it), but could have done with a lot more example-based documentation.
  * It's missing a few features like recursive-directory scanning (for
    source files), and the ability to pipe the output from a program
    into a variable in the build script (e.g. C++FLAGS = `llvm-config
    --cxxflags`).
  * A convenient way to force all generated object and library files
    into a dedicated build folder. It's possible but not very easy to do.
  * I'm a big fan of convention over configuration, I know it's a very
    subjective topic but I like the way Maven3 and Gradle assume a
    project structure (that you can deviate from if you need to). This
    usually requires very good supporting documentation.

The build script I put together for my project looked like this, 
http://mysticpaste.com/private/TCcTE6KGxn .

Hope this helps,

Chris


On 07/13/11 02:02, Nick Sabalausky wrote:
 The recent discussions about package managers and buildsystems has prompted
 me to get off my ass (or rather, *on* my ass as per how programming is
 usually performed...) and start on the D-based rake-inspired buildtool I've
 been meaning to make for awhile now. It's not actually usable yet, but
 there's a sample drakefile demonstrating everything, and it does actually
 compile and run (but just doesn't build the dependency tree or actually run
 any of the build steps). *Should* work on Posix, but I only tested on
 Windows, so I may have fucked up the Posix version...

 Apologies to Jacob Carlborg for the name being so close to "dake". Didn't
 really know what else to call it ("Duck", maybe?) Like dake, it's inspired
 by Ruby's Rake. But unlike dake, the buildscript is written in D instead of


 Before I go implemeting everything, I'd like to get input on it. Is it
 something that could be a major D tool?

 Overview of Drake and links to all the code are here:

 https://bitbucket.org/Abscissa256/drake/wiki/Home
Jul 13 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Chris Molozian" <chris cmoz.me> wrote in message 
news:mailman.1595.1310554903.14074.digitalmars-d puremagic.com...
I asked about build tools for D on the mailing list a while ago. I
 needed a solution that allowed me to mix C++ and D builds in a
 cross-platform way with minimum fuss. You can find the discussion about
 it here
 <http://www.digitalmars.com/d/archives/digitalmars/D/Best_build_tool_for_D_projects_136103.html>
 (you were also a part of it IIRC). My biggest requirement at the time was:

  * Keeping platform checks e.g. IF (MAC) {} ELSE IF(LINUX) {} ... etc.
    to an absolute minimum. What's the point in a cross-platform
    language if when you build projects in it you need to write a short
    essay for the build system...
FWIW, my Drake system takes that as a high priority, too. For example, if you have project "foo", then you can get the cross-platform binary filename, object filename, shared lib filename, static lib filename, etc like this: "foo".exe // "foo.exe" or "foo" "foo".obj // "foo.obj" or "foo.o" "foo".lib // "foo.lib" or "foo.a" "foo".slib // "foo.dll" or "foo.so" "foo".bat // "foo.bat" or "foo" "foo".sh // "foo.bat" or "foo.sh" And new ones are easy to define: I also intend to make sure that invoking "./blah" works on Windows.
  * Pre-built binaries available to all platforms (or as many as possible).
With Drake, the only pre-built binaries that are needed are DMD. The drake command itself is a trivial one-line shell/batch script that invokes RDMD to build+run the drakefile (which imports the rest of drake). Of course, something still needs to be done to set up the DRAKE env var to point to the Drake directory (or at least do something to get the path to Drake into the -I arg sent to RDMD...) Not entirely sure how I want to do that, although I think I have one idea.
 In the end I settled for Jam (ftjam)
 <http://www.freetype.org/jam/index.html>, it works tremendously well.
 It's easy to get binaries for almost any operating system (especially
 with projects like homebrew <http://mxcl.github.com/homebrew/> for Mac).
 I have very few gripes with it:

  * The build description language is very simple (once you fully grok
    it), but could have done with a lot more example-based documentation.
  * It's missing a few features like recursive-directory scanning (for
    source files), and the ability to pipe the output from a program
    into a variable in the build script (e.g. C++FLAGS = `llvm-config
    --cxxflags`).
  * A convenient way to force all generated object and library files
    into a dedicated build folder. It's possible but not very easy to do.
  * I'm a big fan of convention over configuration, I know it's a very
    subjective topic but I like the way Maven3 and Gradle assume a
    project structure (that you can deviate from if you need to). This
    usually requires very good supporting documentation.

 The build script I put together for my project looked like this,
 http://mysticpaste.com/private/TCcTE6KGxn .
Interesting points to keep in mind, thanks. After recently looking at Waf, I do agree with you that it'd be nice to, as you say, "assume a project structure (that you can deviate from if you need to)". Definitely some benefits to be gained from that.
Jul 13 2011
next sibling parent "Nick Sabalausky" <a a.a> writes:
"Nick Sabalausky" <a a.a> wrote in message 
news:ivkoms$7d9$1 digitalmars.com...
 "foo".exe  // "foo.exe" or "foo"
 "foo".obj  // "foo.obj" or "foo.o"
 "foo".lib  // "foo.lib" or "foo.a"
 "foo".slib  // "foo.dll" or "foo.so"
 "foo".bat  // "foo.bat" or "foo"
 "foo".sh   // "foo.bat" or "foo.sh"

 And new ones are easy to define:
[Forgot to finish that thought:] alias crossPlatformSuffix!(".obj", ".o") obj;
Jul 13 2011
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-13 20:31, Nick Sabalausky wrote:
 "Chris Molozian"<chris cmoz.me>  wrote in message
 news:mailman.1595.1310554903.14074.digitalmars-d puremagic.com...
 I asked about build tools for D on the mailing list a while ago. I
 needed a solution that allowed me to mix C++ and D builds in a
 cross-platform way with minimum fuss. You can find the discussion about
 it here
 <http://www.digitalmars.com/d/archives/digitalmars/D/Best_build_tool_for_D_projects_136103.html>
 (you were also a part of it IIRC). My biggest requirement at the time was:

   * Keeping platform checks e.g. IF (MAC) {} ELSE IF(LINUX) {} ... etc.
     to an absolute minimum. What's the point in a cross-platform
     language if when you build projects in it you need to write a short
     essay for the build system...
FWIW, my Drake system takes that as a high priority, too. For example, if you have project "foo", then you can get the cross-platform binary filename, object filename, shared lib filename, static lib filename, etc like this: "foo".exe // "foo.exe" or "foo" "foo".obj // "foo.obj" or "foo.o" "foo".lib // "foo.lib" or "foo.a" "foo".slib // "foo.dll" or "foo.so" "foo".bat // "foo.bat" or "foo" "foo".sh // "foo.bat" or "foo.sh"
Don't forget "foo.dylib" on Mac OS X. -- /Jacob Carlborg
Jul 13 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:ivkpdp$8dj$1 digitalmars.com...
 FWIW, my Drake system takes that as a high priority, too. For example, if
 you have project "foo", then you can get the cross-platform binary 
 filename,
 object filename, shared lib filename, static lib filename, etc like this:

 "foo".exe  // "foo.exe" or "foo"
 "foo".obj  // "foo.obj" or "foo.o"
 "foo".lib  // "foo.lib" or "foo.a"
 "foo".slib  // "foo.dll" or "foo.so"
 "foo".bat  // "foo.bat" or "foo"
 "foo".sh   // "foo.bat" or "foo.sh"
Don't forget "foo.dylib" on Mac OS X.
Thanks, I didn't know about that. Does OSX use .dylib instead of .so, or is it another thing that it has in addition to .so? Are there any other OSX-specific (or BSD-specific, for that matter) extensions to be aware of? Until now, I thought it just used all the same extensions as Linux. Oh also, to Posix-people: Could "slib" be easily confused as being to "lib" what "sbin" is to "bin"?
Jul 13 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-07-14 00:29, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ivkpdp$8dj$1 digitalmars.com...
 FWIW, my Drake system takes that as a high priority, too. For example, if
 you have project "foo", then you can get the cross-platform binary
 filename,
 object filename, shared lib filename, static lib filename, etc like this:

 "foo".exe  // "foo.exe" or "foo"
 "foo".obj  // "foo.obj" or "foo.o"
 "foo".lib  // "foo.lib" or "foo.a"
 "foo".slib  // "foo.dll" or "foo.so"
 "foo".bat  // "foo.bat" or "foo"
 "foo".sh   // "foo.bat" or "foo.sh"
Don't forget "foo.dylib" on Mac OS X.
Thanks, I didn't know about that. Does OSX use .dylib instead of .so, or is it another thing that it has in addition to .so?
Yes it uses .dylib instead of .so. It also has frameworks and other bundles which probably is a little overkill for the tool to support. A framework is just a folder name <name>.framework that contains a specific file and directory layout. It contains a dylib, header files, resources like images and sounds and possibly other dynamic libraries. Basically a package with everything the library needs to run. BTW, DMD can build dynamic libraries on Mac OS X using the "-dylib" switch.
 Are there any other OSX-specific (or BSD-specific, for that matter)
 extensions to be aware of? Until now, I thought it just used all the same
 extensions as Linux.
It depends on what the build tool should be able to do and know about. For example, GUI applications on Mac OS X is expected to be in an application bundle (<name>.app), similar to a framework but containing an executable instead of a dynamic library.
 Oh also, to Posix-people: Could "slib" be easily confused as being to "lib"
 what "sbin" is to "bin"?
-- /Jacob Carlborg
Jul 14 2011
prev sibling parent Chris Molozian <chris cmoz.me> writes:
I'm glad the feedback helped, this may be of use to help in abstracting 
away platform specific configuration: 
http://public.perforce.com/public/jam/src/Jambase

Cheers,

Chris


On 07/13/11 19:31, Nick Sabalausky wrote:
 "Chris Molozian"<chris cmoz.me>  wrote in message
 news:mailman.1595.1310554903.14074.digitalmars-d puremagic.com...
 I asked about build tools for D on the mailing list a while ago. I
 needed a solution that allowed me to mix C++ and D builds in a
 cross-platform way with minimum fuss. You can find the discussion about
 it here
 <http://www.digitalmars.com/d/archives/digitalmars/D/Best_build_tool_for_D_projects_136103.html>
 (you were also a part of it IIRC). My biggest requirement at the time was:

   * Keeping platform checks e.g. IF (MAC) {} ELSE IF(LINUX) {} ... etc.
     to an absolute minimum. What's the point in a cross-platform
     language if when you build projects in it you need to write a short
     essay for the build system...
FWIW, my Drake system takes that as a high priority, too. For example, if you have project "foo", then you can get the cross-platform binary filename, object filename, shared lib filename, static lib filename, etc like this: "foo".exe // "foo.exe" or "foo" "foo".obj // "foo.obj" or "foo.o" "foo".lib // "foo.lib" or "foo.a" "foo".slib // "foo.dll" or "foo.so" "foo".bat // "foo.bat" or "foo" "foo".sh // "foo.bat" or "foo.sh" And new ones are easy to define: I also intend to make sure that invoking "./blah" works on Windows.
   * Pre-built binaries available to all platforms (or as many as possible).
With Drake, the only pre-built binaries that are needed are DMD. The drake command itself is a trivial one-line shell/batch script that invokes RDMD to build+run the drakefile (which imports the rest of drake). Of course, something still needs to be done to set up the DRAKE env var to point to the Drake directory (or at least do something to get the path to Drake into the -I arg sent to RDMD...) Not entirely sure how I want to do that, although I think I have one idea.
 In the end I settled for Jam (ftjam)
 <http://www.freetype.org/jam/index.html>, it works tremendously well.
 It's easy to get binaries for almost any operating system (especially
 with projects like homebrew<http://mxcl.github.com/homebrew/>  for Mac).
 I have very few gripes with it:

   * The build description language is very simple (once you fully grok
     it), but could have done with a lot more example-based documentation.
   * It's missing a few features like recursive-directory scanning (for
     source files), and the ability to pipe the output from a program
     into a variable in the build script (e.g. C++FLAGS = `llvm-config
     --cxxflags`).
   * A convenient way to force all generated object and library files
     into a dedicated build folder. It's possible but not very easy to do.
   * I'm a big fan of convention over configuration, I know it's a very
     subjective topic but I like the way Maven3 and Gradle assume a
     project structure (that you can deviate from if you need to). This
     usually requires very good supporting documentation.

 The build script I put together for my project looked like this,
 http://mysticpaste.com/private/TCcTE6KGxn .
Interesting points to keep in mind, thanks. After recently looking at Waf, I do agree with you that it'd be nice to, as you say, "assume a project structure (that you can deviate from if you need to)". Definitely some benefits to be gained from that.
Jul 13 2011
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-13 03:02, Nick Sabalausky wrote:
 The recent discussions about package managers and buildsystems has prompted
 me to get off my ass (or rather, *on* my ass as per how programming is
 usually performed...) and start on the D-based rake-inspired buildtool I've
 been meaning to make for awhile now. It's not actually usable yet, but
 there's a sample drakefile demonstrating everything, and it does actually
 compile and run (but just doesn't build the dependency tree or actually run
 any of the build steps). *Should* work on Posix, but I only tested on
 Windows, so I may have fucked up the Posix version...

 Apologies to Jacob Carlborg for the name being so close to "dake". Didn't
 really know what else to call it ("Duck", maybe?) Like dake, it's inspired
 by Ruby's Rake. But unlike dake, the buildscript is written in D instead of


 Before I go implemeting everything, I'd like to get input on it. Is it
 something that could be a major D tool?

 Overview of Drake and links to all the code are here:

 https://bitbucket.org/Abscissa256/drake/wiki/Home
First I have to say that I know you are doing this because you want to use D as the language for the build scripts. The reason I did choose Ruby because I think D will be too verbose and when I'm looking at drakefile.d I do think it's too verbose. But instead of starting a debate between Ruby and D I'm going to give you a few suggestions and comments instead. The minimal drakefile doesn't look that minimal. This is the idea I have for a minimal build script file: target("main.d"); // builds "main.d" as an executable Or: target("foobar"); // builds the directory "foobar" as a library DSSS did this very good. Now looking in the drakefile.d. You have targets and tasks all over the place, almost every target has "Task" as a template parameter. Why is this necessary? Have one function for targets and one for tasks. Example from the drakefile.d: target!Task("upload", "all", (Target t) { system("ftp ..."); } ); Seems this could look like this: task("upload", "all", { system("ftp..."); }); // if D just could get a better looking delegate syntax "getDCompiler" seems unnecessary, both ldc and gdc have a dmd compatible wrapper, ldmd and gdmd. These wrappers takes the same flags as dmd and translate them to the correct "native" flags. The "drakfile" function seems unnecessary. I would use a string import (or what it's called), importing the whole build script into a method in a class. All code in the "drakefile" function would instead be at top level. Most of the functions called in the build script should be instance methods, this will allow to use threading, possible invoke multiple targets/tasks simultaneously and running several build scripts simultaneously. I see installation related code in the build script. I think a package manager should be responsible for that. A build tool should deal with single files and a package manager should deal with packages (of files). -- /Jacob Carlborg
Jul 13 2011
next sibling parent reply Russel Winder <russel russel.org.uk> writes:
On Wed, 2011-07-13 at 17:33 +0200, Jacob Carlborg wrote:

 target("main.d"); // builds "main.d" as an executable
The SCons equivalent is: Program ( 'main.d' )
 Or:
=20
 target("foobar"); // builds the directory "foobar" as a library
SCons (and I think Waf as well) does not support this idea (at least currently). In SCons you have to say: SharedLibrary ( 'libraryRootName' , Glob ( 'foobar/*.d' ) ) StaticLibrary ( 'libraryRootName' , Glob ( 'foobar/*.d' ) ) D support in Waf is currently far more sophisticated than that of SCons, but is still based on being a bit more explicit that your proposal above. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jul 13 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-13 19:18, Russel Winder wrote:
 On Wed, 2011-07-13 at 17:33 +0200, Jacob Carlborg wrote:

 target("main.d"); // builds "main.d" as an executable
The SCons equivalent is: Program ( 'main.d' )
 Or:

 target("foobar"); // builds the directory "foobar" as a library
SCons (and I think Waf as well) does not support this idea (at least currently). In SCons you have to say: SharedLibrary ( 'libraryRootName' , Glob ( 'foobar/*.d' ) ) StaticLibrary ( 'libraryRootName' , Glob ( 'foobar/*.d' ) ) D support in Waf is currently far more sophisticated than that of SCons, but is still based on being a bit more explicit that your proposal above.
I think that one of the problem with these language independent build tools is that they don't make it as easy as it could, because they usually don't know enough about a given language. DSSS works just the same, you specify a D file or a directory and it will compile it as an executable or a library. I don't think that a new build tool should do any worse than, for example, DSSS. -- /Jacob Carlborg
Jul 13 2011
parent reply Russel Winder <russel russel.org.uk> writes:
On Wed, 2011-07-13 at 20:32 +0200, Jacob Carlborg wrote:
[ . . . ]
 I think that one of the problem with these language independent build=20
 tools is that they don't make it as easy as it could, because they=20
 usually don't know enough about a given language.
This is generally due to incomplete support in the tools, this is not a problem with general build frameworks per se. In systems such as Waf and SCons, the framework provides facilities for creating tools and for those tools to build DAGs that then get "resolved" in the second phase of activity. The language specific tool does things as a first phase of activity -- i.e. building the DAG. The tools should realize all the language specific things that that languages users needs and/or the language expects. Anything that is not handled is a problem that should be fixed by the tool maintainer. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jul 13 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-14 08:48, Russel Winder wrote:
 On Wed, 2011-07-13 at 20:32 +0200, Jacob Carlborg wrote:
 [ . . . ]
 I think that one of the problem with these language independent build
 tools is that they don't make it as easy as it could, because they
 usually don't know enough about a given language.
This is generally due to incomplete support in the tools, this is not a problem with general build frameworks per se. In systems such as Waf and SCons, the framework provides facilities for creating tools and for those tools to build DAGs that then get "resolved" in the second phase of activity. The language specific tool does things as a first phase of activity -- i.e. building the DAG. The tools should realize all the language specific things that that languages users needs and/or the language expects. Anything that is not handled is a problem that should be fixed by the tool maintainer.
Ok, I see, but it sounds complicated. And I basically give all my hope to the tool maintainer to be willing to merge/implement the necessary functionality/changes that the tool need to support D. -- /Jacob Carlborg
Jul 14 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:ivniga$2lt$1 digitalmars.com...
 On 2011-07-14 08:48, Russel Winder wrote:
 On Wed, 2011-07-13 at 20:32 +0200, Jacob Carlborg wrote:
 [ . . . ]
 I think that one of the problem with these language independent build
 tools is that they don't make it as easy as it could, because they
 usually don't know enough about a given language.
This is generally due to incomplete support in the tools, this is not a problem with general build frameworks per se. In systems such as Waf and SCons, the framework provides facilities for creating tools and for those tools to build DAGs that then get "resolved" in the second phase of activity. The language specific tool does things as a first phase of activity -- i.e. building the DAG. The tools should realize all the language specific things that that languages users needs and/or the language expects. Anything that is not handled is a problem that should be fixed by the tool maintainer.
Ok, I see, but it sounds complicated. And I basically give all my hope to the tool maintainer to be willing to merge/implement the necessary functionality/changes that the tool need to support D.
Personally, I normally prefer just giving the commandline myself. I never feel comfortable trusting some generalized thing to do what I want or to be flexible enough to let me reliably make all adjustments I might need to make.
Jul 14 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-07-14 23:16, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ivniga$2lt$1 digitalmars.com...
 On 2011-07-14 08:48, Russel Winder wrote:
 On Wed, 2011-07-13 at 20:32 +0200, Jacob Carlborg wrote:
 [ . . . ]
 I think that one of the problem with these language independent build
 tools is that they don't make it as easy as it could, because they
 usually don't know enough about a given language.
This is generally due to incomplete support in the tools, this is not a problem with general build frameworks per se. In systems such as Waf and SCons, the framework provides facilities for creating tools and for those tools to build DAGs that then get "resolved" in the second phase of activity. The language specific tool does things as a first phase of activity -- i.e. building the DAG. The tools should realize all the language specific things that that languages users needs and/or the language expects. Anything that is not handled is a problem that should be fixed by the tool maintainer.
Ok, I see, but it sounds complicated. And I basically give all my hope to the tool maintainer to be willing to merge/implement the necessary functionality/changes that the tool need to support D.
Personally, I normally prefer just giving the commandline myself. I never feel comfortable trusting some generalized thing to do what I want or to be flexible enough to let me reliably make all adjustments I might need to make.
Exactly. -- /Jacob Carlborg
Jul 15 2011
prev sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message
news:ivke5k$2m78$1 digitalmars.com...
 First I have to say that I know you are doing this because you want to use 
 D as the language for the build scripts. The reason I did choose Ruby 
 because I think D will be too verbose and when I'm looking at drakefile.d 
 I do think it's too verbose.
Yea, D is likely to be a little more verbose than what could be done in Ruby (or Python). Personally, I think that's well worth it, though. I don't know how many others would agree or not.
 But instead of starting a debate between Ruby and D I'm going to give you 
 a few suggestions and comments instead.

 The minimal drakefile doesn't look that minimal.
I think I may be able to make the "immutable modes = [];" optional by checking for it with some compile-time reflecion. The other parts I'll address further below...
 This is the idea I have for a minimal build script file:

 target("main.d"); // builds "main.d" as an executable

 Or:

 target("foobar"); // builds the directory "foobar" as a library

 DSSS did this very good.
I've never been very comfortable with build systems magically understanding and inferring all those details about a language and filetypes. It just feels too "magic", it's much more difficult to expand to new languages/tools, and every time I use a system like that I feel completely lost every time I need to include even just one custom buildstep. However, Drake's design is expandable. Add-on modules can be created that define new target types, so you could do: module drake_dlang; class DLang : Target (or File) { //...etc } target!DLang("main.d"); And that would essentially wrap a File target, adding in any D-specific things such as "The target is 'main.exe'". And if you really wanted you could take it one more step and do: void targetD(string source) { target!DLang(source); } targetD("main.d"); Good add-ons could even be folded into the standard Drake.
 Now looking in the drakefile.d. You have targets and tasks all over the 
 place, almost every target has "Task" as a template parameter. Why is this 
 necessary? Have one function for targets and one for tasks. Example from 
 the drakefile.d:

         target!Task("upload", "all",
                 (Target t)
                 {
                         system("ftp ...");
                 }
         );

 Seems this could look like this:

 task("upload", "all", {
     system("ftp...");
 }); // if D just could get a better looking delegate syntax
Some Drake terminology first: Every "node" in the dependency tree is a "Target". There are three pre-defined types of targets: "Task", "File" and "Dir". Other new target types can also be defined. So the template param is due to that. However, I suppose it may be a good idea for every target type to come with a matching shortcut: alias target!Task task; alias target!File file; alias target!Dir dir; Then your "task("upload", ...);" example should work. I agree that D's delegate syntax could be better. I think something like your example would be possible, but I ran into what seemed like some type inference limitations with all the overloading and templates, etc.
 "getDCompiler" seems unnecessary, both ldc and gdc have a dmd compatible 
 wrapper, ldmd and gdmd. These wrappers takes the same flags as dmd and 
 translate them to the correct "native" flags.
That's great, I didn't know that. But, you still have to say "ldmd" or "gdmd" instead of "dmd". And since the idea of "getDCompiler" is to allow the user to select which compiler to use (if the drakefile's author wants to support that), something roughly like "getDCompiler" may still be needed, albeit in a greatly simplified form. In any case, it was just an example of using the "modes" feature.
 The "drakfile" function seems unnecessary. I would use a string import (or 
 what it's called), importing the whole build script into a method in a 
 class. All code in the "drakefile" function would instead be at top level.
I'm not entirely opposed to that idea, but here are the (perhaps minor?) reasons I did it this way: - Using "import" inside functions is brand-new and likely still buggy. And unless I'm mistaken, I think there's other problems that still exist with using the same code inside a function as outside (such as some order-of-declaration limitations, IIRC). I wanted to get moving on something that would work reliably right away without being too sensitive to bleeding edge-cases. I figured if all that gets sorted out later on, then maybe a Drake v2 could go that route. - I wasn't sure if a D file that isn't strictly a proper D file would be too weird, too confusing, too magical, or would confuse the fuck out of advanced IDE's. If those are really just totally bullshit reasons, then I'm certainly open to the idea of doing it as you suggest.
 Most of the functions called in the build script should be instance 
 methods, this will allow to use threading, possible invoke multiple 
 targets/tasks simultaneously and running several build scripts 
 simultaneously.
My understanding is that with thread-local being the default, they *don't* need to be instance methods to be thread-safe. Also, dependency-checking and task-invocation (once they're implemented) won't occur at all until *after* the drakefile() function completes. The target!...(...) function merely defines the targets, adding them to an internal list - nothing more. So I'm not sure that's not really a thread-appropriate matter. Not really sure what you mean about running several build scripts simultaneously. The buildscript gets compiled into an exe, so the buildscripts all run in separate processes anyway.
 I see installation related code in the build script. I think a package 
 manager should be responsible for that. A build tool should deal with 
 single files and a package manager should deal with packages (of files).
Those are just there as examples. You can do it however you want. ------------------------------- Not sent from an iPhone.
Jul 13 2011
next sibling parent reply Ulrik Mikaelsson <ulrik.mikaelsson gmail.com> writes:
2011/7/13 Nick Sabalausky <a a.a>:
 "Jacob Carlborg" <doob me.com> wrote in message
 news:ivke5k$2m78$1 digitalmars.com...
 First I have to say that I know you are doing this because you want to use
 D as the language for the build scripts. The reason I did choose Ruby
 because I think D will be too verbose and when I'm looking at drakefile.d
 I do think it's too verbose.
Yea, D is likely to be a little more verbose than what could be done in Ruby (or Python). Personally, I think that's well worth it, though. I don't know how many others would agree or not.
Not trying to be argumentative, but what exactly do you see as the gains in having a D-buildtool built in D (or D-specific build-tool in any language, for that matter)? Seriously, I'm really asking, since I'm having a hard time seeing it? Personally, I can only see the drawbacks; * Building druntime and phobos might be difficult with a d-based buildtool * The build-tool itself will need bootstrapping. A user that wants to test some D-project, will first have to aquire (build) and install some D-compiler with custom tools. Then install druntime with another custom build-system. Then Phobos. Then drake. And THEN, the application/library he/she was interested in. Shortening this path is IMHO REALLY important to see more D adoption. From my personal experience, convincing developers and testers to fight through this path is HARD. * Cross-language builds (project with bindings), and builds with external targets might be more difficult than need be, if the "2nd" language is not supported by drake. * Verbose build-script is IMHO a _really_ undesireable trait. * How soon will I as a developer be able to "just build" a D-binding to a C++-app (with needed C-glue-code) in Drake? Will a user of, say Gentoo, be able to count on this working in his/her envrironment too? * Non-compilation actions will have to be reimplemented; document generation, test execution, install-tasks following OS-specific install procedures (XDG-guidelines etc.), ... IMHO, it sounds like a case of the NIH-syndrome, but I might be missing something obvious?
Jul 13 2011
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/13/11 5:32 PM, Ulrik Mikaelsson wrote:
 Not trying to be argumentative, but what exactly do you see as the
 gains in having a D-buildtool built in D (or D-specific build-tool in
 any language, for that matter)?
I think it's a matter of positioning D and eating one's dogfood. If D is inconceivable for the kind of tasks that Python or Ruby are adept at, then sure, we could and should use either. On the other hand, if we advocate D as a good tool for short scripts, using the competition would hurt its brand. Personally I believe D is plenty adequate for short scripts, of which I've written a ton of. So the path of least resistance for a package manager or for a build tool is D, not any other language. I'd question much harder the decision of using another language (D is, however, not a competitor for the likes of bash or make).
 Seriously, I'm really asking, since
 I'm having a hard time seeing it? Personally, I can only see the
 drawbacks;

   * Building druntime and phobos might be difficult with a d-based buildtool
Druntime and phobos are distributed, and only built by a small number of users. We should not worry about building them. Also, a tool based on other language would require that language's environment to be already available.
   * The build-tool itself will need bootstrapping. A user that wants to
 test some D-project, will first have to aquire (build) and install
 some D-compiler with custom tools. Then install druntime with another
 custom build-system. Then Phobos. Then drake. And THEN, the
 application/library he/she was interested in. Shortening this path is
 IMHO REALLY important to see more D adoption. From my personal
 experience, convincing developers and testers to fight through this
 path is HARD.
The installation provides the adequate tools. Installing and building from scratch is unnecessary. A user wanting to test some project needs to (a) install dmd using OS-specific installation tools (which we already have) and (b) use the provided tools to build whatever project.
   * Cross-language builds (project with bindings), and builds with
 external targets might be more difficult than need be, if the "2nd"
 language is not supported by drake.
This is a good advantage of generic build tools.
   * Verbose build-script is IMHO a _really_ undesireable trait.
I don't understand this.
   * How soon will I as a developer be able to "just build" a D-binding
 to a C++-app (with needed C-glue-code) in Drake? Will a user of, say
 Gentoo, be able to count on this working in his/her envrironment too?
This pretty much depends on the capabilities of the tool, which would have to be fairly D-specific.
   * Non-compilation actions will have to be reimplemented; document
 generation, test execution, install-tasks following OS-specific
 install procedures (XDG-guidelines etc.), ...
Here you're conflating the capabilities of the tool with the capabilities of the _langauge_ used to implement the tool.
 IMHO, it sounds like a case of the NIH-syndrome, but I might be
 missing something obvious?
I see no NIH here. D is an ample language scaling up to large programs and down to scripts. If the question is to build a tool from scratch, D is the obvious choice and any other choice is just odd. It's like some tools I've seen (none successful) that required the competition's product to be installed in order to work. Andrei
Jul 13 2011
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Why build a tool anyway? Why not make a build library that we can use
in our build scripts?

Then we just import the library in our `build.d` script, write the
build script with the help of those library functions, and let the
user run `rdmd build.d`. No need to install any tools, just run rdmd.

I'm already using this approach for DWinProj., the only issue is that
the build script is a little large (~300 lines), but a lot of its code
could be put into a library (e.g. it has file traversing, parallel
builds, checking if all tools are present, checking if all libraries
are present, converting header files, etc..).

This would reduce the script to a few dozen lines. And that's probably
what I'll do to make my other projects easier to build.
Jul 13 2011
next sibling parent Alix Pexton <alix.DOT.pexton gmail.DOT.com> writes:
On 14/07/2011 03:53, Andrej Mitrovic wrote:
 Why build a tool anyway? Why not make a build library that we can use
 in our build scripts?

 Then we just import the library in our `build.d` script, write the
 build script with the help of those library functions, and let the
 user run `rdmd build.d`. No need to install any tools, just run rdmd.

 I'm already using this approach for DWinProj., the only issue is that
 the build script is a little large (~300 lines), but a lot of its code
 could be put into a library (e.g. it has file traversing, parallel
 builds, checking if all tools are present, checking if all libraries
 are present, converting header files, etc..).

 This would reduce the script to a few dozen lines. And that's probably
 what I'll do to make my other projects easier to build.
Oooh, I like this build lib idea! A...
Jul 14 2011
prev sibling next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Andrej Mitrovic" <andrej.mitrovich gmail.com> wrote in message 
news:mailman.1624.1310612000.14074.digitalmars-d puremagic.com...
 Why build a tool anyway? Why not make a build library that we can use
 in our build scripts?

 Then we just import the library in our `build.d` script, write the
 build script with the help of those library functions, and let the
 user run `rdmd build.d`. No need to install any tools, just run rdmd.

 I'm already using this approach for DWinProj., the only issue is that
 the build script is a little large (~300 lines), but a lot of its code
 could be put into a library (e.g. it has file traversing, parallel
 builds, checking if all tools are present, checking if all libraries
 are present, converting header files, etc..).

 This would reduce the script to a few dozen lines. And that's probably
 what I'll do to make my other projects easier to build.
Yea, I've given that some thought. Heck, I wrote the D-based buildscript for the recent versions of DDMD, so I know D's good for scripts like that :) It's definitely not a bad idea. Ultimately, I think going with a rake/scons-like build-system approach just allows for a little bit more boilerplate to be removed, which is good for something like a buildscript. For example, if done as a library, once you've defined all your targets, you'd have to call some function to actually do the build. As a build system, defining the targets is all you need to do, and the build gets invoked automatically. It's kinda like how dmain() calls the user's main() instead of the work inside dmain() being a library functions that the main() author needs to call. Plus, as a build-system, the "void main(string[] args) {" part can eventually be removed (once sticking normally-module-level things inside a function is mature enough, if it isn't already). That said, making Drake's functionality at least available in library form does sound like a good idea. And the buildscript-wrapping system (for people who want it) can, of course, be built on top of that.
Jul 14 2011
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
Nick Sabalausky wrote:
 Ultimately, I think going with a rake/scons-like build-system approach just
 allows for a little bit more boilerplate to be removed, which is good for
 something like a buildscript. For example, if done as a library, once you've
 defined all your targets, you'd have to call some function to actually do
 the build. As a build system, defining the targets is all you need to do,
 and the build gets invoked automatically. It's kinda like how dmain() calls
 the user's main() instead of the work inside dmain() being a library
 functions that the main() author needs to call. Plus, as a build-system, the
 "void main(string[] args) {" part can eventually be removed (once sticking
 normally-module-level things inside a function is mature enough, if it isn't
 already).
It is not mature at all. Module-level things inside functions do not currently work well: Eg: - No forward references. - No forward references, reloaded: void main(){ int foo(); int bar(){return foo();} // ok int foo(){return 100;} // nope, *!* this is called a "redefinition" *!* } - No (template) overloading: void main(){ int foo(int x){return x;} double foo(double x){return x;} // nope int bar(int x)(){return x;} double bar(double x)(){return x;} // nope } Those limitations are very itchy (at least for the kind of stuff I'd like to be doing), and should be removed. Cheers, -Timon
Jul 14 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
Timon Gehr wrote:
 Nick Sabalausky wrote:
 Ultimately, I think going with a rake/scons-like build-system approach just
 allows for a little bit more boilerplate to be removed, which is good for
 something like a buildscript. For example, if done as a library, once you've
 defined all your targets, you'd have to call some function to actually do
 the build. As a build system, defining the targets is all you need to do,
 and the build gets invoked automatically. It's kinda like how dmain() calls
 the user's main() instead of the work inside dmain() being a library
 functions that the main() author needs to call. Plus, as a build-system, >>the
 "void main(string[] args) {" part can eventually be removed (once sticking
 normally-module-level things inside a function is mature enough, if it >isn't
 already).
It is not mature at all. Module-level things inside functions do not currently work well: Eg: - No forward references. - No forward references, reloaded: void main(){ int foo(); int bar(){return foo();} // ok int foo(){return 100;} // nope, *!* this is called a "redefinition" *!* } - No (template) overloading: void main(){ int foo(int x){return x;} double foo(double x){return x;} // nope int bar(int x)(){return x;} double bar(double x)(){return x;} // nope } Those limitations are very itchy (at least for the kind of stuff I'd like to be doing), and should be removed. Cheers, -Timon
Oh, I forgot: template constraints seem to be lexically invalid on local templates. Cheers, -Timon
Jul 14 2011
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-07-14 21:34, Nick Sabalausky wrote:
 That said, making Drake's functionality at least available in library form
 does sound like a good idea. And the buildscript-wrapping system (for people
 who want it) can, of course, be built on top of that.
The functionality of the build tool should absolutly be built as a library, then creating a thin wrapper around the library which will be the actual tool the user invokes. This will allow to have the build tool built into an IDE or similar. Have a look at Orbit, the install command for example: https://github.com/jacob-carlborg/orbit/blob/master/orbit/orb/commands/Install.d Just two lines to do the invoke the installer (which could be one line). -- /Jacob Carlborg
Jul 15 2011
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-14 04:53, Andrej Mitrovic wrote:
 Why build a tool anyway? Why not make a build library that we can use
 in our build scripts?

 Then we just import the library in our `build.d` script, write the
 build script with the help of those library functions, and let the
 user run `rdmd build.d`. No need to install any tools, just run rdmd.

 I'm already using this approach for DWinProj., the only issue is that
 the build script is a little large (~300 lines), but a lot of its code
 could be put into a library (e.g. it has file traversing, parallel
 builds, checking if all tools are present, checking if all libraries
 are present, converting header files, etc..).

 This would reduce the script to a few dozen lines. And that's probably
 what I'll do to make my other projects easier to build.
I think a few dozen lines is way to much for a build script. If I choose to use all default configurations I want to only need to write one line: target("main.d"); -- /Jacob Carlborg
Jul 14 2011
parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 7/14/11, Jacob Carlborg <doob me.com> wrote:
 I think a few dozen lines is way to much for a build script. If I choose
 to use all default configurations I want to only need to write one line:

 target("main.d");
I'm not talking about simple projects. I'm talking about libraries which might use other C libraries, where you need to download dependencies before building (due to licensing issues that prevent distribution for example) or invoke code generators, and check that the user has all the tools before building. Have you seen the QtD CMakefile? It's almost 1000 lines.
Jul 14 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-07-14 22:25, Andrej Mitrovic wrote:
 On 7/14/11, Jacob Carlborg<doob me.com>  wrote:
 I think a few dozen lines is way to much for a build script. If I choose
 to use all default configurations I want to only need to write one line:

 target("main.d");
I'm not talking about simple projects. I'm talking about libraries which might use other C libraries, where you need to download dependencies before building (due to licensing issues that prevent distribution for example) or invoke code generators, and check that the user has all the tools before building. Have you seen the QtD CMakefile? It's almost 1000 lines.
Ok. I rather not look at a 1000 line cmakefile :) -- /Jacob Carlborg
Jul 15 2011
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-07-14 04:53, Andrej Mitrovic wrote:
 Why build a tool anyway? Why not make a build library that we can use
 in our build scripts?

 Then we just import the library in our `build.d` script, write the
 build script with the help of those library functions, and let the
 user run `rdmd build.d`. No need to install any tools, just run rdmd.

 I'm already using this approach for DWinProj., the only issue is that
 the build script is a little large (~300 lines), but a lot of its code
 could be put into a library (e.g. it has file traversing, parallel
 builds, checking if all tools are present, checking if all libraries
 are present, converting header files, etc..).

 This would reduce the script to a few dozen lines. And that's probably
 what I'll do to make my other projects easier to build.
If you want to run arbitrary tasks with the build tool? You would need some handling of command line arguments. It just seems more complicated than having a dedicated build tool -- /Jacob Carlborg
Jul 15 2011
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-14 04:37, Andrei Alexandrescu wrote:
 On 7/13/11 5:32 PM, Ulrik Mikaelsson wrote:
 Not trying to be argumentative, but what exactly do you see as the
 gains in having a D-buildtool built in D (or D-specific build-tool in
 any language, for that matter)?
I think it's a matter of positioning D and eating one's dogfood. If D is inconceivable for the kind of tasks that Python or Ruby are adept at, then sure, we could and should use either. On the other hand, if we advocate D as a good tool for short scripts, using the competition would hurt its brand. Personally I believe D is plenty adequate for short scripts, of which I've written a ton of. So the path of least resistance for a package manager or for a build tool is D, not any other language. I'd question much harder the decision of using another language (D is, however, not a competitor for the likes of bash or make).
Isn't it a competitor for make in this case?
 I see no NIH here. D is an ample language scaling up to large programs
 and down to scripts. If the question is to build a tool from scratch, D
 is the obvious choice and any other choice is just odd. It's like some
 tools I've seen (none successful) that required the competition's
 product to be installed in order to work.


 Andrei
I though we were talking about what language the build scripts should use, not the language the actual build tool should be written in. I think these are two separate issues and all tools mentioned (Drake, Orbit, Dake) in this thread are implemented in D. My tools, using Ruby as the scripting language, have Ruby embedded in the tool and requires no installation of Ruby. -- /Jacob Carlborg
Jul 14 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/14/11 3:00 PM, Jacob Carlborg wrote:
 On 2011-07-14 04:37, Andrei Alexandrescu wrote:
 On 7/13/11 5:32 PM, Ulrik Mikaelsson wrote:
 Not trying to be argumentative, but what exactly do you see as the
 gains in having a D-buildtool built in D (or D-specific build-tool in
 any language, for that matter)?
I think it's a matter of positioning D and eating one's dogfood. If D is inconceivable for the kind of tasks that Python or Ruby are adept at, then sure, we could and should use either. On the other hand, if we advocate D as a good tool for short scripts, using the competition would hurt its brand. Personally I believe D is plenty adequate for short scripts, of which I've written a ton of. So the path of least resistance for a package manager or for a build tool is D, not any other language. I'd question much harder the decision of using another language (D is, however, not a competitor for the likes of bash or make).
Isn't it a competitor for make in this case?
What I meant here is that it's reasonable for a build tool for D to require make without impacting its brand.
 I see no NIH here. D is an ample language scaling up to large programs
 and down to scripts. If the question is to build a tool from scratch, D
 is the obvious choice and any other choice is just odd. It's like some
 tools I've seen (none successful) that required the competition's
 product to be installed in order to work.


 Andrei
I though we were talking about what language the build scripts should use, not the language the actual build tool should be written in. I think these are two separate issues and all tools mentioned (Drake, Orbit, Dake) in this thread are implemented in D. My tools, using Ruby as the scripting language, have Ruby embedded in the tool and requires no installation of Ruby.
I believe use of Ruby for scripting is a strategic mistake for a small benefit. The users your tools should not need to learn any amount of Ruby (or Scala for that matter) in order to build D programs. What we need is, if necessary, to improve D to better address the needs for scripting a build tool. That way your work would be pushing D's state of the art in addition to adding value on its own. I suggest you reconsider. Thanks, Andrei
Jul 15 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-07-15 18:13, Andrei Alexandrescu wrote:
 I believe use of Ruby for scripting is a strategic mistake for a small
 benefit. The users your tools should not need to learn any amount of
 Ruby (or Scala for that matter) in order to build D programs. What we
 need is, if necessary, to improve D to better address the needs for
 scripting a build tool. That way your work would be pushing D's state of
 the art in addition to adding value on its own. I suggest you reconsider.


 Thanks,

 Andrei
I have very difficult in believing that features from Ruby, Scala or some other language that I think make it possible to create good looking DSL will end up in D. I've listed (some of) these features in another reply to you. On the other hand if it is possible that some of these features end up in D then that's great, and will be a very big step in conniving me to switch to D for the build scripts. -- /Jacob Carlborg
Jul 15 2011
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-07-14 00:32, Ulrik Mikaelsson wrote
 Not trying to be argumentative, but what exactly do you see as the
 gains in having a D-buildtool built in D (or D-specific build-tool in
 any language, for that matter)? Seriously, I'm really asking, since
 I'm having a hard time seeing it? Personally, I can only see the
 drawbacks;

   * Building druntime and phobos might be difficult with a d-based buildtool
   * The build-tool itself will need bootstrapping. A user that wants to
 test some D-project, will first have to aquire (build) and install
 some D-compiler with custom tools. Then install druntime with another
 custom build-system. Then Phobos. Then drake. And THEN, the
 application/library he/she was interested in. Shortening this path is
 IMHO REALLY important to see more D adoption. From my personal
 experience, convincing developers and testers to fight through this
 path is HARD.
The build tool should provide pre-compiled binaries on all supported platforms. Then the build tool should provide a build script for it self, so the build tool can build it self.
   * Cross-language builds (project with bindings), and builds with
 external targets might be more difficult than need be, if the "2nd"
 language is not supported by drake.
You can offer to run system commands.
   * Verbose build-script is IMHO a _really_ undesireable trait.
I agree with this.
   * How soon will I as a developer be able to "just build" a D-binding
 to a C++-app (with needed C-glue-code) in Drake? Will a user of, say
 Gentoo, be able to count on this working in his/her envrironment too?
   * Non-compilation actions will have to be reimplemented; document
 generation, test execution, install-tasks following OS-specific
 install procedures (XDG-guidelines etc.), ...
Yes, this is preferable to have.
 IMHO, it sounds like a case of the NIH-syndrome, but I might be
 missing something obvious?
In my experience other build tools aren't good enough in at least: 1. Making it as easy as it could be to build simple executables and libraries 2. Syntax is not as good/clean as it could be 1. Is because they don't know about D and D compilers. 2. I don't know about this one. This is an example of a Rubygems file: spec = Gem::Specification.new do |s| s.name = 'example' s.version = '1.0' s.summary = 'Example gem specification' end Even this tool, which uses Ruby, has unnecessary noise in the syntax. This could instead be written as: gem do name 'example' version '1.0' summary 'Example gem specification' end In this case the Ruby syntax doesn't allow to use the equal sign, like this: "name = 'example'". -- /Jacob Carlborg
Jul 14 2011
prev sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Ulrik Mikaelsson" <ulrik.mikaelsson gmail.com> wrote in message 
news:mailman.1620.1310596361.14074.digitalmars-d puremagic.com...
 2011/7/13 Nick Sabalausky <a a.a>:
 "Jacob Carlborg" <doob me.com> wrote in message
 news:ivke5k$2m78$1 digitalmars.com...
 First I have to say that I know you are doing this because you want to 
 use
 D as the language for the build scripts. The reason I did choose Ruby
 because I think D will be too verbose and when I'm looking at 
 drakefile.d
 I do think it's too verbose.
Yea, D is likely to be a little more verbose than what could be done in Ruby (or Python). Personally, I think that's well worth it, though. I don't know how many others would agree or not.
Not trying to be argumentative, but what exactly do you see as the gains in having a D-buildtool built in D (or D-specific build-tool in any language, for that matter)? Seriously, I'm really asking, since I'm having a hard time seeing it?
Perfectly understandable question. First of all, it isn't really a D-specific tool, it's intended to be usable for anything (including D, of course). It's just that it's written in D and it uses buildscripts that are written in D. As for why a new one based on D instead of just using Rake/SCons/Waf/etc., like Andrei said, there's the dogfooding and brand image issues. Also: - D programmers and people working on D projects don't have to learn or switch to another language for their buildscripts. - D projects don't have to be dependent on other languages or VMs. Although I have to say I was impressed that Jacob's Ruby-buildscript-based Dake avoids this problem. - Utility functions can easily be shared between a D program and it's build script (for whatever that's worth, but I can imagine it could come in handy now and then). - Also, I just like D a lot more than I like Ruby/Python and I think it's safe to assume there are probably other D users who feel the same (since they're D users after all).
 Personally, I can only see the
 drawbacks;

 * Building druntime and phobos might be difficult with a d-based buildtool
Those already use make, and I'm not sure that's really going to change.
 * The build-tool itself will need bootstrapping. A user that wants to
 test some D-project, will first have to aquire (build) and install
 some D-compiler with custom tools. Then install druntime with another
 custom build-system. Then Phobos. Then drake. And THEN, the
 application/library he/she was interested in. Shortening this path is
 IMHO REALLY important to see more D adoption. From my personal
 experience, convincing developers and testers to fight through this
 path is HARD.
It's not as complicated as you suggest. Suppose someone wants to try building FooApp: Step 1: D compilers already come prebuilt, with Phobos and Druntime already set up. So just install DMD. Done. Or users can install DVM and then install DMD with "dvm install {version} && dvm use -d (version)". Step 2: Install drake. Or, FooApp could even just include Drake. Or, FooApp could be installed with a package manager that sees FooApp depends on Drake and so auto-installs Drake first. Step 3: Run "drake all". That invokes a trivial one-line shell/batch script that will automatically compile drake and the buildscript (if they're not already built), and then run the buildscript. Or, when we get a good package manager, we could just offer pre-built versions of the package manager and then: Step 1: Install the package manager (which might even install DMD automatically, possibly via DVM). Step 2: Run "d-package-manager install FooApp". The package manager determines FooApp depends on DMD and Drake (and maybe DVM), and automatically installs all of that, then installs FooApp and invokes FooApp's post-install script which is "drake all", and thus builds FooApp. Step 3: Enjoy a nice latte, because you're done.
 * Cross-language builds (project with bindings), and builds with
 external targets might be more difficult than need be, if the "2nd"
 language is not supported by drake.
One of my primary goals for Drake is that anything you can do on the command line can easily be done with Drake. So that amounts to pretty much any language. Drake doesn't require special bindings to use any particular tool (although bindings to abstract away the command line can be made, if desired)
 * Verbose build-script is IMHO a _really_ undesireable trait.
Fair enough point. It's possible I might not be able to get it quite as terse as some systems, but I am trying my best to minimize verbosity. I think the cross-platform extensions like "foo".exe are a good example of that. It uses the trickery of module-level "properties" combined with UFCS to achieve what I think may be the simplest possible D syntax for cross-platform file extensions (at least without getting too crazy).
 * How soon will I as a developer be able to "just build" a D-binding
 to a C++-app (with needed C-glue-code) in Drake? Will a user of, say
 Gentoo, be able to count on this working in his/her envrironment too?
TBH, I'm not exactly well-versed in linking C++ and D. Actually, I've never even looked into it, let along done it... Basically, if you know how to do it on the command line, then you can "just do it" in Drake. If you're looking for something more magical, then I'd say "Whenever someone writes an abstraction for it".
 * Non-compilation actions will have to be reimplemented; document
 generation, test execution, install-tasks following OS-specific
 install procedures (XDG-guidelines etc.), ...
Not really sure what you mean here.
 IMHO, it sounds like a case of the NIH-syndrome, but I might be
 missing something obvious?
I suppose someone could argue that it's NIH, technically speaking. (Personally, I feel that anti-NIH sentiment is overrated.) But I think it's worthwhile just to be able to write buildscripts in D. After all, nobody really *has* to use D for anything. They can just write their apps in C, C++, Java, Ruby, Python, etc. But a growing number of people use D because they like it and feel it provides certain benefits. So I guess I'd say: Why shouldn't we be able to write buildscripts in D and make use of a helpful framework/library to do it?
Jul 14 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-07-15 05:31, Nick Sabalausky wrote:
 "Ulrik Mikaelsson"<ulrik.mikaelsson gmail.com>  wrote in message
 * The build-tool itself will need bootstrapping. A user that wants to
 test some D-project, will first have to aquire (build) and install
 some D-compiler with custom tools. Then install druntime with another
 custom build-system. Then Phobos. Then drake. And THEN, the
 application/library he/she was interested in. Shortening this path is
 IMHO REALLY important to see more D adoption. From my personal
 experience, convincing developers and testers to fight through this
 path is HARD.
It's not as complicated as you suggest. Suppose someone wants to try building FooApp: Step 1: D compilers already come prebuilt, with Phobos and Druntime already set up. So just install DMD. Done. Or users can install DVM and then install DMD with "dvm install {version}&& dvm use -d (version)". Step 2: Install drake. Or, FooApp could even just include Drake. Or, FooApp could be installed with a package manager that sees FooApp depends on Drake and so auto-installs Drake first. Step 3: Run "drake all". That invokes a trivial one-line shell/batch script that will automatically compile drake and the buildscript (if they're not already built), and then run the buildscript. Or, when we get a good package manager, we could just offer pre-built versions of the package manager and then: Step 1: Install the package manager (which might even install DMD automatically, possibly via DVM). Step 2: Run "d-package-manager install FooApp". The package manager determines FooApp depends on DMD and Drake (and maybe DVM), and automatically installs all of that, then installs FooApp and invokes FooApp's post-install script which is "drake all", and thus builds FooApp. Step 3: Enjoy a nice latte, because you're done.
When a standard build tool and package manager become available I will let DVM install it. -- /Jacob Carlborg
Jul 15 2011
prev sibling next sibling parent Chris Molozian <chris cmoz.me> writes:
I completely agree. I definitely believe that work on a new build tool 
is worthwhile especially if it has built in support for D. It's a shame 
that so many build tools at the moment are being developed using 
interpreted languages, it's obviously for the benefits of creating DSLs 
that extend the base language itself. They tend to suffer from slowness 
though (my subjective opinion ;-) ).

I don't think building the tool in D is a good idea either for reasons 
mentioned by Ulrik. It's worth considering using Lua 
<http://www.lua.org/> or Io <http://iolanguage.com/> for the build 
description language but writing the tool in C (or other).

Cheers,

Chris


On 07/13/11 23:32, Ulrik Mikaelsson wrote:
 2011/7/13 Nick Sabalausky<a a.a>:
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ivke5k$2m78$1 digitalmars.com...
 First I have to say that I know you are doing this because you want to use
 D as the language for the build scripts. The reason I did choose Ruby
 because I think D will be too verbose and when I'm looking at drakefile.d
 I do think it's too verbose.
Yea, D is likely to be a little more verbose than what could be done in Ruby (or Python). Personally, I think that's well worth it, though. I don't know how many others would agree or not.
Not trying to be argumentative, but what exactly do you see as the gains in having a D-buildtool built in D (or D-specific build-tool in any language, for that matter)? Seriously, I'm really asking, since I'm having a hard time seeing it? Personally, I can only see the drawbacks; * Building druntime and phobos might be difficult with a d-based buildtool * The build-tool itself will need bootstrapping. A user that wants to test some D-project, will first have to aquire (build) and install some D-compiler with custom tools. Then install druntime with another custom build-system. Then Phobos. Then drake. And THEN, the application/library he/she was interested in. Shortening this path is IMHO REALLY important to see more D adoption. From my personal experience, convincing developers and testers to fight through this path is HARD. * Cross-language builds (project with bindings), and builds with external targets might be more difficult than need be, if the "2nd" language is not supported by drake. * Verbose build-script is IMHO a _really_ undesireable trait. * How soon will I as a developer be able to "just build" a D-binding to a C++-app (with needed C-glue-code) in Drake? Will a user of, say Gentoo, be able to count on this working in his/her envrironment too? * Non-compilation actions will have to be reimplemented; document generation, test execution, install-tasks following OS-specific install procedures (XDG-guidelines etc.), ... IMHO, it sounds like a case of the NIH-syndrome, but I might be missing something obvious?
Jul 13 2011
prev sibling next sibling parent reply David Nadlinger <see klickverbot.at> writes:
On 7/13/11 11:41 PM, Nick Sabalausky wrote:
 "getDCompiler" seems unnecessary, both ldc and gdc have a dmd compatible
 wrapper, ldmd and gdmd. These wrappers takes the same flags as dmd and
 translate them to the correct "native" flags.
That's great, I didn't know that. […]
Be aware though that neither of them supports the -lib switch. By the way, does anyone have a wrapper emulating it for LDC lying around? That way, the Phobos build system wouldn't have to be duplicated for it, and e.g. the test suite could be built easily. David
Jul 13 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-14 00:54, David Nadlinger wrote:
 On 7/13/11 11:41 PM, Nick Sabalausky wrote:
 "getDCompiler" seems unnecessary, both ldc and gdc have a dmd compatible
 wrapper, ldmd and gdmd. These wrappers takes the same flags as dmd and
 translate them to the correct "native" flags.
That's great, I didn't know that. […]
Be aware though that neither of them supports the -lib switch. By the way, does anyone have a wrapper emulating it for LDC lying around? That way, the Phobos build system wouldn't have to be duplicated for it, and e.g. the test suite could be built easily. David
http://pastebin.com/u2mBJDjA -- /Jacob Carlborg
Jul 14 2011
parent reply David Nadlinger <see klickverbot.at> writes:
On 7/14/11 9:53 PM, Jacob Carlborg wrote:
 On 2011-07-14 00:54, David Nadlinger wrote:
 Be aware though that neither of them supports the -lib switch. By the
 way, does anyone have a wrapper emulating it for LDC lying around? That
 way, the Phobos build system wouldn't have to be duplicated for it, and
 e.g. the test suite could be built easily.

 David
http://pastebin.com/u2mBJDjA
Thanks, but my question was specifically about the »-lib« switch, which is not supported by the normal ldmd. David
Jul 14 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-07-14 22:13, David Nadlinger wrote:
 On 7/14/11 9:53 PM, Jacob Carlborg wrote:
 On 2011-07-14 00:54, David Nadlinger wrote:
 Be aware though that neither of them supports the -lib switch. By the
 way, does anyone have a wrapper emulating it for LDC lying around? That
 way, the Phobos build system wouldn't have to be duplicated for it, and
 e.g. the test suite could be built easily.

 David
http://pastebin.com/u2mBJDjA
Thanks, but my question was specifically about the »-lib« switch, which is not supported by the normal ldmd. David
Oh, I see. -- /Jacob Carlborg
Jul 15 2011
prev sibling next sibling parent reply Russel Winder <russel russel.org.uk> writes:
Nick,

On Wed, 2011-07-13 at 17:41 -0400, Nick Sabalausky wrote:
[ . . . ]
 Yea, D is likely to be a little more verbose than what could be done in R=
uby=20
 (or Python). Personally, I think that's well worth it, though. I don't kn=
ow=20
 how many others would agree or not.
[ . . . ] You might want to take a quick look at SBT -- the standard Scala build framework. It's advocates started it because Ant and Maven are XML hell and are generally problematic, and Gradle is build on Java and Groovy, and (as you might expect) Scala folk abhor all things dynamic (i.e. Groovy) and insist on static type checking. I have some doubts about the huge downloads they dump into each project hierarchy, but there is no doubt that they have made excellent use of a statically type language to create a DSL for building Scala things. If D can be used to go down this sort of road, and if it can support the build and install facilities of SCons and Waf, then it could be a winner.=20 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jul 13 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-14 08:56, Russel Winder wrote:
 Nick,

 On Wed, 2011-07-13 at 17:41 -0400, Nick Sabalausky wrote:
 [ . . . ]
 Yea, D is likely to be a little more verbose than what could be done in Ruby
 (or Python). Personally, I think that's well worth it, though. I don't know
 how many others would agree or not.
[ . . . ] You might want to take a quick look at SBT -- the standard Scala build framework. It's advocates started it because Ant and Maven are XML hell and are generally problematic, and Gradle is build on Java and Groovy, and (as you might expect) Scala folk abhor all things dynamic (i.e. Groovy) and insist on static type checking. I have some doubts about the huge downloads they dump into each project hierarchy, but there is no doubt that they have made excellent use of a statically type language to create a DSL for building Scala things. If D can be used to go down this sort of road, and if it can support the build and install facilities of SCons and Waf, then it could be a winner.
Scala have a lot of features making a DSL looking a lot better than one written in D. -- /Jacob Carlborg
Jul 14 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/14/11 3:09 PM, Jacob Carlborg wrote:
 On 2011-07-14 08:56, Russel Winder wrote:
 Nick,

 On Wed, 2011-07-13 at 17:41 -0400, Nick Sabalausky wrote:
 [ . . . ]
 Yea, D is likely to be a little more verbose than what could be done
 in Ruby
 (or Python). Personally, I think that's well worth it, though. I
 don't know
 how many others would agree or not.
[ . . . ] You might want to take a quick look at SBT -- the standard Scala build framework. It's advocates started it because Ant and Maven are XML hell and are generally problematic, and Gradle is build on Java and Groovy, and (as you might expect) Scala folk abhor all things dynamic (i.e. Groovy) and insist on static type checking. I have some doubts about the huge downloads they dump into each project hierarchy, but there is no doubt that they have made excellent use of a statically type language to create a DSL for building Scala things. If D can be used to go down this sort of road, and if it can support the build and install facilities of SCons and Waf, then it could be a winner.
Scala have a lot of features making a DSL looking a lot better than one written in D.
Does it have something akin to string mixins? Andrei
Jul 14 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-15 00:06, Andrei Alexandrescu wrote:
 On 7/14/11 3:09 PM, Jacob Carlborg wrote:
 On 2011-07-14 08:56, Russel Winder wrote:
 Nick,

 On Wed, 2011-07-13 at 17:41 -0400, Nick Sabalausky wrote:
 [ . . . ]
 Yea, D is likely to be a little more verbose than what could be done
 in Ruby
 (or Python). Personally, I think that's well worth it, though. I
 don't know
 how many others would agree or not.
[ . . . ] You might want to take a quick look at SBT -- the standard Scala build framework. It's advocates started it because Ant and Maven are XML hell and are generally problematic, and Gradle is build on Java and Groovy, and (as you might expect) Scala folk abhor all things dynamic (i.e. Groovy) and insist on static type checking. I have some doubts about the huge downloads they dump into each project hierarchy, but there is no doubt that they have made excellent use of a statically type language to create a DSL for building Scala things. If D can be used to go down this sort of road, and if it can support the build and install facilities of SCons and Waf, then it could be a winner.
Scala have a lot of features making a DSL looking a lot better than one written in D.
Does it have something akin to string mixins? Andrei
I'm not very familiar with Scala but I found this: http://blog.darevay.com/2009/01/remedial-scala-interpreting-scala-from-scala/ -- /Jacob Carlborg
Jul 15 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/15/11 7:58 AM, Jacob Carlborg wrote:
 On 2011-07-15 00:06, Andrei Alexandrescu wrote:
 On 7/14/11 3:09 PM, Jacob Carlborg wrote:
 On 2011-07-14 08:56, Russel Winder wrote:
 Nick,

 On Wed, 2011-07-13 at 17:41 -0400, Nick Sabalausky wrote:
 [ . . . ]
 Yea, D is likely to be a little more verbose than what could be done
 in Ruby
 (or Python). Personally, I think that's well worth it, though. I
 don't know
 how many others would agree or not.
[ . . . ] You might want to take a quick look at SBT -- the standard Scala build framework. It's advocates started it because Ant and Maven are XML hell and are generally problematic, and Gradle is build on Java and Groovy, and (as you might expect) Scala folk abhor all things dynamic (i.e. Groovy) and insist on static type checking. I have some doubts about the huge downloads they dump into each project hierarchy, but there is no doubt that they have made excellent use of a statically type language to create a DSL for building Scala things. If D can be used to go down this sort of road, and if it can support the build and install facilities of SCons and Waf, then it could be a winner.
Scala have a lot of features making a DSL looking a lot better than one written in D.
Does it have something akin to string mixins? Andrei
I'm not very familiar with Scala but I found this: http://blog.darevay.com/2009/01/remedial-scala-interpreting-scala-from-scala/
Interesting. What are the features that do make a DSL better looking in Scala than in D? Thanks, Andrei
Jul 15 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-15 17:16, Andrei Alexandrescu wrote:
 On 7/15/11 7:58 AM, Jacob Carlborg wrote:
 I'm not very familiar with Scala but I found this:
 http://blog.darevay.com/2009/01/remedial-scala-interpreting-scala-from-scala/
Interesting. What are the features that do make a DSL better looking in Scala than in D? Thanks, Andrei
This is just what I think (I've thrown in Ruby and CoffeeScript as well) Scala: * Delegate syntax - Scala allows passing a delegate to a syntax like this: loop { // code } This allows to create, what look likes, new statements. I don't recall how the "loop" function should be implemented but it was quite complex and had something do with partial functions. Scala also has this nice delegate syntax: foo(a => a + a) * Infix operators - Allows to call any method that takes one argument without parentheses and without the dot: object.func("asd") Can be called like this: object func "asd" * Naming methods - In Scala you can name a method "+", "-", "*" and similar. You are not limited to A-Za-z_0-9. Together with infix operators this is how Scala implements operator overloading. This also allows to add new operators. * No semicolons * Is in general very good at inferring types, including return types * Almost everything is an expression * The constructor is built-in the class declaration: class Foo { // this is the constructor def foo = "foo" } Ruby: * Methods can be called without parentheses * Class bodies can execute code * Not limited to top level declarations * No semicolons * Delegate/block syntax - In Ruby you passes blocks to methods after the regular arguments: foo(3, 4) do |a| end Or the one-line syntax: foo(3, 4) { |a| } * Simplified hash literal syntax - The standard hash syntax in Ruby is: a = { key => value } But if you call a method with hash literal you can most of the times omit the braces: foo(key => value) * Almost everything is an expression - This is possible b = if a == 3 4 else 5 end * Trailing if statements foo(3) if a == 4 CoffeeScript: * Methods can be called without parentheses (if they take at least on argument) * Almost everything is an expression * Trailing if statements * No semicolons * Class bodies can execute code * Not limited to top level declarations * Keywords for "or", "and", "yes", "no" and "not". * Delegate/functoin syntax - Example: foo (a, b) -> a + b * Hash/object syntax - Example: hash = foo: 1 bar: 2 hash = foo: 1, bar: 2 CoffeeScript: http://jashkenas.github.com/coffee-script/ -- /Jacob Carlborg
Jul 15 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
Jacob Carlborg wrote:
 On 2011-07-15 17:16, Andrei Alexandrescu wrote:
 On 7/15/11 7:58 AM, Jacob Carlborg wrote:
 I'm not very familiar with Scala but I found this:
 http://blog.darevay.com/2009/01/remedial-scala-interpreting-scala-from-scala/
Interesting. What are the features that do make a DSL better looking in Scala than in D? Thanks, Andrei
This is just what I think (I've thrown in Ruby and CoffeeScript as well) Scala: * Delegate syntax - Scala allows passing a delegate to a syntax like this: loop { // code } This allows to create, what look likes, new statements. I don't recall how the "loop" function should be implemented but it was quite complex and had something do with partial functions. Scala also has this nice delegate syntax: foo(a => a + a) * Infix operators - Allows to call any method that takes one argument without parentheses and without the dot: object.func("asd") Can be called like this: object func "asd" * Naming methods - In Scala you can name a method "+", "-", "*" and similar. You are not limited to A-Za-z_0-9. Together with infix operators this is how Scala implements operator overloading. This also allows to add new operators. * No semicolons * Is in general very good at inferring types, including return types * Almost everything is an expression * The constructor is built-in the class declaration: class Foo { // this is the constructor def foo = "foo" }
One of the more important features you didn't mention are the so-called "implicits". Implicits are basically scoped implicit conversions, but strictly more powerful: They can deduce the conversion function based on an extended context, like the signature of a member function the resulting type should provide. Eg: class IntExtensions(i: Int){ def times(j: Int) = i*j } implicit def intExtensions(i: Int) = new IntExtensions(i) // implicit conversion of Int to IntExtensions implicit def strToInt(s: String) = s match { // implicit conversion from String to Int case "one" => 1 case "two" => 2 case "three" => 3 //... case _ => throw new Exception("Unknown number " + s) } implicit def intExtensions(s: String) = new IntExtensions(strToInt(s)) // implicit conversion from String to IntExtensions (implicits are not automagically transitive, probably for compiler efficiency) // now you can write stuff like (this also depends on infix operators): val x="three" times "two" times "one" val y=3 times "two" val z="three" times 2 assert(x==6 && y==6 && z==6); If D had UFCS and infix operators, it would have a lot more power to build DSELs. I don't know exactly how infix operators would play with Ds somewhat ambiguous declaration syntax. Of course, there are always string mixins, but then you are departing from valid D syntax with your DSL. mixin(myDSLtoD(q{not D code})); or myDSL!q{not D code}; Cheers, -Timon
Jul 16 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-07-16 15:24, Timon Gehr wrote:
 Jacob Carlborg wrote:
 On 2011-07-15 17:16, Andrei Alexandrescu wrote:
 On 7/15/11 7:58 AM, Jacob Carlborg wrote:
 I'm not very familiar with Scala but I found this:
 http://blog.darevay.com/2009/01/remedial-scala-interpreting-scala-from-scala/
Interesting. What are the features that do make a DSL better looking in Scala than in D? Thanks, Andrei
This is just what I think (I've thrown in Ruby and CoffeeScript as well) Scala: * Delegate syntax - Scala allows passing a delegate to a syntax like this: loop { // code } This allows to create, what look likes, new statements. I don't recall how the "loop" function should be implemented but it was quite complex and had something do with partial functions. Scala also has this nice delegate syntax: foo(a => a + a) * Infix operators - Allows to call any method that takes one argument without parentheses and without the dot: object.func("asd") Can be called like this: object func "asd" * Naming methods - In Scala you can name a method "+", "-", "*" and similar. You are not limited to A-Za-z_0-9. Together with infix operators this is how Scala implements operator overloading. This also allows to add new operators. * No semicolons * Is in general very good at inferring types, including return types * Almost everything is an expression * The constructor is built-in the class declaration: class Foo { // this is the constructor def foo = "foo" }
One of the more important features you didn't mention are the so-called "implicits". Implicits are basically scoped implicit conversions, but strictly more powerful: They can deduce the conversion function based on an extended context, like the signature of a member function the resulting type should provide. Eg: class IntExtensions(i: Int){ def times(j: Int) = i*j } implicit def intExtensions(i: Int) = new IntExtensions(i) // implicit conversion of Int to IntExtensions implicit def strToInt(s: String) = s match { // implicit conversion from String to Int case "one" => 1 case "two" => 2 case "three" => 3 //... case _ => throw new Exception("Unknown number " + s) } implicit def intExtensions(s: String) = new IntExtensions(strToInt(s)) // implicit conversion from String to IntExtensions (implicits are not automagically transitive, probably for compiler efficiency) // now you can write stuff like (this also depends on infix operators): val x="three" times "two" times "one" val y=3 times "two" val z="three" times 2 assert(x==6&& y==6&& z==6);
Yeah, implicits are nice as well.
 If D had UFCS and infix operators, it would have a lot more power to build
DSELs.
 I don't know exactly how infix operators would play with Ds somewhat ambiguous
 declaration syntax.
 Of course, there are always string mixins, but then you are departing from
valid D
 syntax with your DSL.

 mixin(myDSLtoD(q{not D code}));
 or
 myDSL!q{not D code};


 Cheers,
 -Timon
String mixins are not a solution to DSLs in my opinion. -- /Jacob Carlborg
Jul 17 2011
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-13 23:41, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ivke5k$2m78$1 digitalmars.com...
 First I have to say that I know you are doing this because you want to use
 D as the language for the build scripts. The reason I did choose Ruby
 because I think D will be too verbose and when I'm looking at drakefile.d
 I do think it's too verbose.
Yea, D is likely to be a little more verbose than what could be done in Ruby (or Python). Personally, I think that's well worth it, though. I don't know how many others would agree or not.
 But instead of starting a debate between Ruby and D I'm going to give you
 a few suggestions and comments instead.

 The minimal drakefile doesn't look that minimal.
I think I may be able to make the "immutable modes = [];" optional by checking for it with some compile-time reflecion. The other parts I'll address further below...
 This is the idea I have for a minimal build script file:

 target("main.d"); // builds "main.d" as an executable

 Or:

 target("foobar"); // builds the directory "foobar" as a library

 DSSS did this very good.
I've never been very comfortable with build systems magically understanding and inferring all those details about a language and filetypes. It just feels too "magic", it's much more difficult to expand to new languages/tools, and every time I use a system like that I feel completely lost every time I need to include even just one custom buildstep. However, Drake's design is expandable. Add-on modules can be created that define new target types, so you could do: module drake_dlang; class DLang : Target (or File) { //...etc } target!DLang("main.d"); And that would essentially wrap a File target, adding in any D-specific things such as "The target is 'main.exe'". And if you really wanted you could take it one more step and do: void targetD(string source) { target!DLang(source); } targetD("main.d");
If that's possible, then great, I was hoping this could be default. I see no reason why it should be more complicated or requires more syntax to such a simple thing as building a standard executable with default configurations. If it's not this easy by default, then I think the build tool have failed, just my opinion.
 Good add-ons could even be folded into the standard Drake.


 Now looking in the drakefile.d. You have targets and tasks all over the
 place, almost every target has "Task" as a template parameter. Why is this
 necessary? Have one function for targets and one for tasks. Example from
 the drakefile.d:

          target!Task("upload", "all",
                  (Target t)
                  {
                          system("ftp ...");
                  }
          );

 Seems this could look like this:

 task("upload", "all", {
      system("ftp...");
 }); // if D just could get a better looking delegate syntax
Some Drake terminology first: Every "node" in the dependency tree is a "Target". There are three pre-defined types of targets: "Task", "File" and "Dir". Other new target types can also be defined. So the template param is due to that. However, I suppose it may be a good idea for every target type to come with a matching shortcut: alias target!Task task; alias target!File file; alias target!Dir dir; Then your "task("upload", ...);" example should work.
Ok, now I see. I think this is quite important the you first show the most simple forms and then show what they actually are.
 I agree that D's delegate syntax could be better. I think something like
 your example would be possible, but I ran into what seemed like some type
 inference limitations with all the overloading and templates, etc.
Ok.
 "getDCompiler" seems unnecessary, both ldc and gdc have a dmd compatible
 wrapper, ldmd and gdmd. These wrappers takes the same flags as dmd and
 translate them to the correct "native" flags.
That's great, I didn't know that. But, you still have to say "ldmd" or "gdmd" instead of "dmd". And since the idea of "getDCompiler" is to allow the user to select which compiler to use (if the drakefile's author wants to support that), something roughly like "getDCompiler" may still be needed, albeit in a greatly simplified form. In any case, it was just an example of using the "modes" feature.
Ok, I see. I would prefer if the tool had built-in support for this. Just calling a function named "compiler" and it will use that particular compiler.
 The "drakfile" function seems unnecessary. I would use a string import (or
 what it's called), importing the whole build script into a method in a
 class. All code in the "drakefile" function would instead be at top level.
I'm not entirely opposed to that idea, but here are the (perhaps minor?) reasons I did it this way: - Using "import" inside functions is brand-new and likely still buggy. And unless I'm mistaken, I think there's other problems that still exist with using the same code inside a function as outside (such as some order-of-declaration limitations, IIRC). I wanted to get moving on something that would work reliably right away without being too sensitive to bleeding edge-cases. I figured if all that gets sorted out later on, then maybe a Drake v2 could go that route.
I was referring to this: void main () { writeln(import("main.d")); } This has been working as long as I've been using D. It works in D1 as well.
 - I wasn't sure if a D file that isn't strictly a proper D file would be too
 weird, too confusing, too magical, or would confuse the fuck out of advanced
 IDE's.
I have no idea how an IDE would behave with an incomplete file like that. I pretty sure this only matters if the IDE does some semantic processing, i.e. something more than just syntax highlighting. Textmate, which basically only does syntax highlighting, has no problem with an incomplete D file.
 If those are really just totally bullshit reasons, then I'm certainly open
 to the idea of doing it as you suggest.


 Most of the functions called in the build script should be instance
 methods, this will allow to use threading, possible invoke multiple
 targets/tasks simultaneously and running several build scripts
 simultaneously.
My understanding is that with thread-local being the default, they *don't* need to be instance methods to be thread-safe. Also, dependency-checking and task-invocation (once they're implemented) won't occur at all until *after* the drakefile() function completes. The target!...(...) function merely defines the targets, adding them to an internal list - nothing more. So I'm not sure that's not really a thread-appropriate matter. Not really sure what you mean about running several build scripts simultaneously. The buildscript gets compiled into an exe, so the buildscripts all run in separate processes anyway.
Hm, it may actually not be a problem. I'll have to give this some more thought, if there is any problem or not. It's probably me doesn't think straight.
 I see installation related code in the build script. I think a package
 manager should be responsible for that. A build tool should deal with
 single files and a package manager should deal with packages (of files).
Those are just there as examples. You can do it however you want. ------------------------------- Not sent from an iPhone.
Ok, I see. -- /Jacob Carlborg
Jul 14 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:ivm9lq$2q1d$1 digitalmars.com...
 On 2011-07-13 23:41, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ivke5k$2m78$1 digitalmars.com...
 The "drakfile" function seems unnecessary. I would use a string import 
 (or
 what it's called), importing the whole build script into a method in a
 class. All code in the "drakefile" function would instead be at top 
 level.
I'm not entirely opposed to that idea, but here are the (perhaps minor?) reasons I did it this way: - Using "import" inside functions is brand-new and likely still buggy. And unless I'm mistaken, I think there's other problems that still exist with using the same code inside a function as outside (such as some order-of-declaration limitations, IIRC). I wanted to get moving on something that would work reliably right away without being too sensitive to bleeding edge-cases. I figured if all that gets sorted out later on, then maybe a Drake v2 could go that route.
I was referring to this: void main () { writeln(import("main.d")); } This has been working as long as I've been using D. It works in D1 as well.
I assume you meant "mixin" instead of "writeln". What I meant is that, suppose wr have this: // internal_to_drake.d: void main() { mixin(import("main.d")); } // main.d: import blah; // rest of buildscript... Then that becomes: void main() { import blah; // rest of buildscript... } And *that's* a brand-new feature. And I think there may be some other things, too, that might still be a little buggy if they're stuck inside a function like that.
 - I wasn't sure if a D file that isn't strictly a proper D file would be 
 too
 weird, too confusing, too magical, or would confuse the fuck out of 
 advanced
 IDE's.
I have no idea how an IDE would behave with an incomplete file like that. I pretty sure this only matters if the IDE does some semantic processing, i.e. something more than just syntax highlighting. Textmate, which basically only does syntax highlighting, has no problem with an incomplete D file.
Right, but a lot of people use fancy IDE's like Eclipse and Visual Studio.
 If those are really just totally bullshit reasons, then I'm certainly 
 open
 to the idea of doing it as you suggest.


 Most of the functions called in the build script should be instance
 methods, this will allow to use threading, possible invoke multiple
 targets/tasks simultaneously and running several build scripts
 simultaneously.
My understanding is that with thread-local being the default, they *don't* need to be instance methods to be thread-safe. Also, dependency-checking and task-invocation (once they're implemented) won't occur at all until *after* the drakefile() function completes. The target!...(...) function merely defines the targets, adding them to an internal list - nothing more. So I'm not sure that's not really a thread-appropriate matter. Not really sure what you mean about running several build scripts simultaneously. The buildscript gets compiled into an exe, so the buildscripts all run in separate processes anyway.
Hm, it may actually not be a problem. I'll have to give this some more thought, if there is any problem or not. It's probably me doesn't think straight.
Maybe you're thinking D1? I *think* the thread-local-by-default is D2-only (though I'm not certain).
Jul 14 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-14 21:15, Nick Sabalausky wrote:
 I assume you meant "mixin" instead of "writeln".
Doesn't matter. It's the import statement I'm referring to.
 What I meant is that, suppose wr have this:

      // internal_to_drake.d:
      void main()
      {
          mixin(import("main.d"));
      }

      // main.d:
      import blah;
      // rest of buildscript...

 Then that becomes:

      void main()
      {
          import blah;
          // rest of buildscript...
      }

 And *that's* a brand-new feature. And I think there may be some other
 things, too, that might still be a little buggy if they're stuck inside a
 function like that.
Ah, now I see. I'm hoping the user never needs to write import statements in the build script. It's just these little things that I don't like about using D for the build scripts. Imports and no top level code.
 - I wasn't sure if a D file that isn't strictly a proper D file would be
 too
 weird, too confusing, too magical, or would confuse the fuck out of
 advanced
 IDE's.
I have no idea how an IDE would behave with an incomplete file like that. I pretty sure this only matters if the IDE does some semantic processing, i.e. something more than just syntax highlighting. Textmate, which basically only does syntax highlighting, has no problem with an incomplete D file.
Right, but a lot of people use fancy IDE's like Eclipse and Visual Studio.
Exactly.
 Maybe you're thinking D1? I *think* the thread-local-by-default is D2-only
 (though I'm not certain).
I was thinking in Ruby, but I've already answered this in another post. -- /Jacob Carlborg
Jul 15 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:ivp0oa$3lq$1 digitalmars.com...
 On 2011-07-14 21:15, Nick Sabalausky wrote:

 Ah, now I see. I'm hoping the user never needs to write import statements 
 in the build script. It's just these little things that I don't like about 
 using D for the build scripts. Imports and no top level code.
I agree that the default imports should be sufficient for most typical uses. But it should be possible to create and share a Drake add-on in the form of a simple module to be imported. An import would be needed for that. Plus, even without Drake add-ons, it's reasonable to think that some people may have reason to import something from Phobos, or something from their own project, that wouldn't normally be used in most other buildscripts.
Jul 15 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-07-16 00:07, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ivp0oa$3lq$1 digitalmars.com...
 On 2011-07-14 21:15, Nick Sabalausky wrote:

 Ah, now I see. I'm hoping the user never needs to write import statements
 in the build script. It's just these little things that I don't like about
 using D for the build scripts. Imports and no top level code.
I agree that the default imports should be sufficient for most typical uses. But it should be possible to create and share a Drake add-on in the form of a simple module to be imported. An import would be needed for that. Plus, even without Drake add-ons, it's reasonable to think that some people may have reason to import something from Phobos, or something from their own project, that wouldn't normally be used in most other buildscripts.
Absolutely, imports will be needed. -- /Jacob Carlborg
Jul 16 2011
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-13 23:41, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 Most of the functions called in the build script should be instance
 methods, this will allow to use threading, possible invoke multiple
 targets/tasks simultaneously and running several build scripts
 simultaneously.
My understanding is that with thread-local being the default, they *don't* need to be instance methods to be thread-safe. Also, dependency-checking and task-invocation (once they're implemented) won't occur at all until *after* the drakefile() function completes. The target!...(...) function merely defines the targets, adding them to an internal list - nothing more. So I'm not sure that's not really a thread-appropriate matter. Not really sure what you mean about running several build scripts simultaneously. The buildscript gets compiled into an exe, so the buildscripts all run in separate processes anyway.
Now I remember how I was thinking, I was thinking in Ruby :) This will most likely not work in D but let me explain anyway. Say that you have a build script with two targets, two executables: target("foo.d", (Target t) { t.buildflags ~= "-L-ldwt"; }); target("main.d" (Target t) { t.buildflags ~= "-release" }); In Ruby this would work the same: target "foo.d" do |t| t.buildflags << "-L-ldwt" end target "main.d" do |t| t.buildflags << "-release" end In Ruby you could improve this syntax a little, like this: target "foo.d" do buildflags << "-L-ldwt" end target "main.d" do buildflags << "-release" end In Ruby you can take advantage of the instance_eval method, allowing to evaluate a block/delegate in the context of an instance. In the above example "buildflags" would be a an instance method in the class that the block is evaluated in. Then you don't need to pass the a Target instance to the block. In D, with this syntax: target("foo.d", { buildflags ~= "-L-ldwt"; }); target("main.d" { buildflags ~= "-release" }); "buildflags" would probably be a global function or an instance method. If this should work "buildflags" needs to keep some data structure with buildflags for each target. This seems quite complicated, making sure the correct build flags are used with the correct target.
Jul 14 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:ivme1u$31i8$1 digitalmars.com...

[...snip...]
 In D, with this syntax:

 target("foo.d", {
     buildflags ~=  "-L-ldwt";
 });

 target("main.d" {
     buildflags ~=  "-release"
 });

 "buildflags" would probably be a global function or an instance method. If 
 this should work "buildflags" needs to keep some data structure with 
 buildflags for each target. This seems quite complicated, making sure the 
 correct build flags are used with the correct target.
That's an interestng idea. I think it could be done fairly easily by having Drake invoke the delegates indirectly like this: Target t; // global void invokeBuildStep(Target currTarget, {type} dg) { t = currTarget; dg(); t = null; } Only thing though, is if there's any special members added to the Target subclass being used, the user would have to cast 't' to access them. Then again, that's an issue in the current design, too. I'll have to think about this...
Jul 14 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-14 21:06, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ivme1u$31i8$1 digitalmars.com...

 [...snip...]
 In D, with this syntax:

 target("foo.d", {
      buildflags ~=  "-L-ldwt";
 });

 target("main.d" {
      buildflags ~=  "-release"
 });

 "buildflags" would probably be a global function or an instance method. If
 this should work "buildflags" needs to keep some data structure with
 buildflags for each target. This seems quite complicated, making sure the
 correct build flags are used with the correct target.
That's an interestng idea. I think it could be done fairly easily by having Drake invoke the delegates indirectly like this: Target t; // global void invokeBuildStep(Target currTarget, {type} dg) { t = currTarget; dg(); t = null; }
The functions invoked in the delegate till use the global target? Seems this would work.
 Only thing though, is if there's any special members added to the Target
 subclass being used, the user would have to cast 't' to access them. Then
 again, that's an issue in the current design, too. I'll have to think about
 this...
Maybe in that case it's better to pass the target to the delegate. Just provide several overloads of the function taking the delegate. -- /Jacob Carlborg
Jul 15 2011
parent "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:ivpfoa$uu7$1 digitalmars.com...
 On 2011-07-14 21:06, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ivme1u$31i8$1 digitalmars.com...

 [...snip...]
 In D, with this syntax:

 target("foo.d", {
      buildflags ~=  "-L-ldwt";
 });

 target("main.d" {
      buildflags ~=  "-release"
 });

 "buildflags" would probably be a global function or an instance method. 
 If
 this should work "buildflags" needs to keep some data structure with
 buildflags for each target. This seems quite complicated, making sure 
 the
 correct build flags are used with the correct target.
That's an interestng idea. I think it could be done fairly easily by having Drake invoke the delegates indirectly like this: Target t; // global void invokeBuildStep(Target currTarget, {type} dg) { t = currTarget; dg(); t = null; }
The functions invoked in the delegate till use the global target? Seems this would work.
That's the idea, yes.
 Only thing though, is if there's any special members added to the Target
 subclass being used, the user would have to cast 't' to access them. Then
 again, that's an issue in the current design, too. I'll have to think 
 about
 this...
Maybe in that case it's better to pass the target to the delegate. Just provide several overloads of the function taking the delegate.
Perhaps. Something to think about anyway. Although, IIRC, I think already I ran into some trouble with IFTI, type inference, overloading, etc. when trying to do something like that... But anyway, all stuff to think about...
Jul 15 2011
prev sibling parent reply jdrewsen <jdrewsen nospam.com> writes:
Den 13-07-2011 03:02, Nick Sabalausky skrev:
 The recent discussions about package managers and buildsystems has prompted
 me to get off my ass (or rather, *on* my ass as per how programming is
 usually performed...) and start on the D-based rake-inspired buildtool I've
 been meaning to make for awhile now. It's not actually usable yet, but
 there's a sample drakefile demonstrating everything, and it does actually
 compile and run (but just doesn't build the dependency tree or actually run
 any of the build steps). *Should* work on Posix, but I only tested on
 Windows, so I may have fucked up the Posix version...

 Apologies to Jacob Carlborg for the name being so close to "dake". Didn't
 really know what else to call it ("Duck", maybe?) Like dake, it's inspired
 by Ruby's Rake. But unlike dake, the buildscript is written in D instead of


 Before I go implemeting everything, I'd like to get input on it. Is it
 something that could be a major D tool?

 Overview of Drake and links to all the code are here:

 https://bitbucket.org/Abscissa256/drake/wiki/Home
A good start I think. The first thing that I think should be fixed is the naming. Drake should be called dbuild Orb should be called dpack You read the name and you have an idea of what it does - nothing fancy. People coming from ruby may recognize the drake name, but I guess many c++ peoply have no idea. Orb is not derived from anything I guess. Regarding drake itself: I really like how general it is but at the same time miss some simplicity. "Make the common task easy and the uncommon possible" Some steps that you could take in that direction is to predefine commonly used modes like "build", "clean" etc. And the define buildMode(), cleanMode() as builtins. That would get rid of the "immutable modes = ..." lines in simple projects. Maybe let the wrapper that loads the drakefile insert the "import drake.all" and also wrap the "void drakefile() { ... }" around the file. Maybe that is what your have already listed on the todo? As suggested elsewhere you could get rid of target!Task or target!File and just defined the target() and file() functions. Also a convenience could be to just include all .d files found for an exe target if no .d files are explicitly listed. I guess this would make it possible to have a drakefile as simple as: task("appA".exe); It seems to lack the configure part of the build process e.g. locate the libfoobar.a file for linking or locate a D compiler. Thanks, /Jonas
Jul 13 2011
next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"jdrewsen" <jdrewsen nospam.com> wrote in message 
news:ivl1gp$o1v$1 digitalmars.com...
 A good start I think. The first thing that I think should be fixed is the 
 naming.

 Drake should be called dbuild
 Orb should be called dpack

 You read the name and you have an idea of what it does - nothing fancy.
I think that's an excellent point. Only thing: Is "dbuild" already taken? Also I'd hate to claim that name if nobody ended up liking it ;)
 People coming from ruby may recognize the drake name, but I guess many c++ 
 peoply have no idea.
 Orb is not derived from anything I guess.

 Regarding drake itself:

 I really like how general it is but at the same time miss some simplicity. 
 "Make the common task easy and the uncommon possible"
Yea, that's a fair point. But I figure I should start with general and then build the simple/structured on top of that, rather than trying to go the other way around.
 Some steps that you could take in that direction is to predefine commonly 
 used modes like "build", "clean" etc. And the define buildMode(), 
 cleanMode() as builtins. That would get rid of the "immutable modes = ..." 
 lines in simple projects.

 Maybe let the wrapper that loads the drakefile insert the "import 
 drake.all" and also wrap the "void drakefile() { ... }" around the file. 
 Maybe that is what your have already listed on the todo?

 As suggested elsewhere you could get rid of target!Task or target!File and 
 just defined the target() and file() functions.

 Also a convenience could be to just include all .d files found for an exe 
 target if no .d files are explicitly listed.

 I guess this would make it possible to have a drakefile as simple as:

 task("appA".exe);
Thanks, definitely things to think about.
 It seems to lack the configure part of the build process e.g. locate the 
 libfoobar.a file for linking or locate a D compiler.
Hmm, yea, configure didn't even occur to me. I think it could acually be done as just another File target that everything else depends on. Only thing is then you'd have to save/load the configuration file manually, so maybe there should be some convenience mechnism for that (Maybe utilizing Jacob's Orange serialization library once that rewrite is done? Or maybe that would be overkill? Is there some INI-file util, or am I just imagining that?)
Jul 13 2011
next sibling parent "Nick Sabalausky" <a a.a> writes:
"Nick Sabalausky" <a a.a> wrote in message 
news:ivl4f8$t29$1 digitalmars.com...
 "jdrewsen" <jdrewsen nospam.com> wrote in message 
 news:ivl1gp$o1v$1 digitalmars.com...
 A good start I think. The first thing that I think should be fixed is the 
 naming.

 Drake should be called dbuild
 Orb should be called dpack

 You read the name and you have an idea of what it does - nothing fancy.
I think that's an excellent point. Only thing: Is "dbuild" already taken? Also I'd hate to claim that name if nobody ended up liking it ;)
I mean, if nobody ended up liking the tool. A tool with an official-sounding name like "dbuild" should be something with a strong following.
Jul 13 2011
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-13 23:52, Nick Sabalausky wrote:
 Hmm, yea, configure didn't even occur to me. I think it could acually be
 done as just another File target that everything else depends on. Only thing
 is then you'd have to save/load the configuration file manually, so maybe
 there should be some convenience mechnism for that (Maybe utilizing Jacob's
 Orange serialization library once that rewrite is done? Or maybe that would
 be overkill? Is there some INI-file util, or am I just imagining that?)
What about doing something like DSSS/rebuild does. There are config files for dmd, ldc and gdc on various platforms. You can choose to invoke different compilers and it will use the config file for that particular compiler on the current platform. I think a complete serialization library might be a little overkill (but I don't know). The rewrite of Orange will take me longer time then I expected to finish, it's (de)serializing pointers that give me a headache, almost everything else is finished. For now Orbit has higher priority. Tango has a INI-file module. Otherwise JSON might be a good idea. For linking with libraries I think integration with a package manager would be the best idea. This is how I'm planning with Orbit and the build tool (Dake) I'm working on. If Drake becomes the standard build tool for D and Orbit the package manager I'm more than happy to help integrate Drake with Orbit. -- /Jacob Carlborg
Jul 14 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:ivng77$305d$1 digitalmars.com...
 On 2011-07-13 23:52, Nick Sabalausky wrote:
 Hmm, yea, configure didn't even occur to me. I think it could acually be
 done as just another File target that everything else depends on. Only 
 thing
 is then you'd have to save/load the configuration file manually, so maybe
 there should be some convenience mechnism for that (Maybe utilizing 
 Jacob's
 Orange serialization library once that rewrite is done? Or maybe that 
 would
 be overkill? Is there some INI-file util, or am I just imagining that?)
What about doing something like DSSS/rebuild does. There are config files for dmd, ldc and gdc on various platforms. You can choose to invoke different compilers and it will use the config file for that particular compiler on the current platform.
One of my main goals for Drake is that using any old random tool (besides just building D code) doesn't feel like an afterthought. That DSSS/rebuild-style approach doesn't seem like it would scale particularly well in that direction. I think whatever solution is used needs to work just as well for the user's own custom tools as it does for DMD/LDC/GDC. Then, any D-specific simplifications/improvements that are made should be built on top of that. For example, we could provide a function the user calls from within their "configure" step that automatically handles D-specific configuring (And yea, maybe we could even automatically provide a default "configure" that already calls that function.)
Jul 14 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-14 21:50, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ivng77$305d$1 digitalmars.com...
 On 2011-07-13 23:52, Nick Sabalausky wrote:
 Hmm, yea, configure didn't even occur to me. I think it could acually be
 done as just another File target that everything else depends on. Only
 thing
 is then you'd have to save/load the configuration file manually, so maybe
 there should be some convenience mechnism for that (Maybe utilizing
 Jacob's
 Orange serialization library once that rewrite is done? Or maybe that
 would
 be overkill? Is there some INI-file util, or am I just imagining that?)
What about doing something like DSSS/rebuild does. There are config files for dmd, ldc and gdc on various platforms. You can choose to invoke different compilers and it will use the config file for that particular compiler on the current platform.
One of my main goals for Drake is that using any old random tool (besides just building D code) doesn't feel like an afterthought. That DSSS/rebuild-style approach doesn't seem like it would scale particularly well in that direction. I think whatever solution is used needs to work just as well for the user's own custom tools as it does for DMD/LDC/GDC. Then, any D-specific simplifications/improvements that are made should be built on top of that. For example, we could provide a function the user calls from within their "configure" step that automatically handles D-specific configuring (And yea, maybe we could even automatically provide a default "configure" that already calls that function.)
Ok, have you thought about how this will look (in code) and behave? -- /Jacob Carlborg
Jul 15 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:ivp1gd$500$2 digitalmars.com...
 On 2011-07-14 21:50, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ivng77$305d$1 digitalmars.com...
 On 2011-07-13 23:52, Nick Sabalausky wrote:
 Hmm, yea, configure didn't even occur to me. I think it could acually 
 be
 done as just another File target that everything else depends on. Only
 thing
 is then you'd have to save/load the configuration file manually, so 
 maybe
 there should be some convenience mechnism for that (Maybe utilizing
 Jacob's
 Orange serialization library once that rewrite is done? Or maybe that
 would
 be overkill? Is there some INI-file util, or am I just imagining that?)
What about doing something like DSSS/rebuild does. There are config files for dmd, ldc and gdc on various platforms. You can choose to invoke different compilers and it will use the config file for that particular compiler on the current platform.
One of my main goals for Drake is that using any old random tool (besides just building D code) doesn't feel like an afterthought. That DSSS/rebuild-style approach doesn't seem like it would scale particularly well in that direction. I think whatever solution is used needs to work just as well for the user's own custom tools as it does for DMD/LDC/GDC. Then, any D-specific simplifications/improvements that are made should be built on top of that. For example, we could provide a function the user calls from within their "configure" step that automatically handles D-specific configuring (And yea, maybe we could even automatically provide a default "configure" that already calls that function.)
Ok, have you thought about how this will look (in code) and behave?
Not extensively, but here's what I have in mind so far. In an early initial version of Drake, it would be like this: task("configure", "drake.conf"); file("drake.conf", (Target t) { // Do all configuring here and save to drake.conf } ); Then, Drake can add built-in tools for configuring DMD and saving/loading a configuation file: task("configure", "drake.conf"); file("drake.conf", (Target t) { auto configure = configureDMD(); // Do all custom configuring (if any) here, // adding data to be saved into 'configure'. configure.save(t.name); } ); And finally, all that boilerplate can be hidden away and handled automatically by Drake: // Completely optional: file("drake.conf", (Target t) { // Do all custom configuring here (if any is even desired), // adding data to be saved into global 'configure'. } ); At that point, the automatic configuring could optionally be disabled like this: defaultConfigure = false; // Completely optional, completely manual: task("configure", "drake.conf"); file("drake.conf", (Target t) { // Do all configuring here and save to drake.conf. // The 'configureDMD' and 'configure' can still be // used manually. } );
Jul 17 2011
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-17 23:01, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 Ok, have you thought about how this will look (in code) and behave?
Not extensively, but here's what I have in mind so far. In an early initial version of Drake, it would be like this: task("configure", "drake.conf"); file("drake.conf", (Target t) { // Do all configuring here and save to drake.conf } );
Ok. Will the build script change the configuration file? Seems kind of strange. Usually the config file is used to control the build script, not the other way around. -- /Jacob Carlborg
Jul 17 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:j00krh$1i3q$1 digitalmars.com...
 On 2011-07-17 23:01, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 Ok, have you thought about how this will look (in code) and behave?
Not extensively, but here's what I have in mind so far. In an early initial version of Drake, it would be like this: task("configure", "drake.conf"); file("drake.conf", (Target t) { // Do all configuring here and save to drake.conf } );
Ok. Will the build script change the configuration file?
Yes.
 Seems kind of strange. Usually the config file is used to control the 
 build script, not the other way around.
It's not really "the other way around" since the buildscript would still load the configuration file and use the settings in it (I guess I neglected to mention that part). It's just that creating it isn't done separately from the buildscript like with the clunky autotools. From a quick browse through the docs, Waf seems to have the buildscript direct the creation of the configuration file, too.
Jul 18 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-19 04:51, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 Ok. Will the build script change the configuration file?
Yes.
 Seems kind of strange. Usually the config file is used to control the
 build script, not the other way around.
It's not really "the other way around" since the buildscript would still load the configuration file and use the settings in it (I guess I neglected to mention that part). It's just that creating it isn't done separately from the buildscript like with the clunky autotools. From a quick browse through the docs, Waf seems to have the buildscript direct the creation of the configuration file, too.
I start to think that we mean different things when we say "config file". What would you have in the config file? -- /Jacob Carlborg
Jul 19 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:j03fe0$101m$1 digitalmars.com...
 On 2011-07-19 04:51, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 Ok. Will the build script change the configuration file?
Yes.
 Seems kind of strange. Usually the config file is used to control the
 build script, not the other way around.
It's not really "the other way around" since the buildscript would still load the configuration file and use the settings in it (I guess I neglected to mention that part). It's just that creating it isn't done separately from the buildscript like with the clunky autotools. From a quick browse through the docs, Waf seems to have the buildscript direct the creation of the configuration file, too.
I start to think that we mean different things when we say "config file". What would you have in the config file?
Granted, I've never used this sort of feature from any buildsystem, but I thought we were talking about something like the "configure" command in Waf. My understanding is that it's like the "./configure" from autotool's "./configure && make && make install", (except hopefully without needing to check quite so much stuff or needing the user to manually run the configure command). Apperently some projects need to have their buildsystem check for the existance of, locations of, and details about certain things on the local system before building. So...that stuff.
Jul 19 2011
next sibling parent Russel Winder <russel russel.org.uk> writes:
On Tue, 2011-07-19 at 05:28 -0400, Nick Sabalausky wrote:
[ . . . ]
 Granted, I've never used this sort of feature from any buildsystem, but I=
=20
 thought we were talking about something like the "configure" command in W=
af.=20
 My understanding is that it's like the "./configure" from autotool's=20
 "./configure && make && make install", (except hopefully without needing =
to=20
 check quite so much stuff or needing the user to manually run the configu=
re=20
 command).
Waf does indeed have a configure phase and a build phase. As long as the configuration remains unchanged after a "waf configure" and there hasn't been a "waf distclean" then you only ever need to build. SCons on the other hand runs all the configuration checks in the SConscript file on every run. I am never sure which I prefer, they both have positives and negatives.
 Apperently some projects need to have their buildsystem check for the=20
 existance of, locations of, and details about certain things on the local=
=20
 system before building. So...that stuff.
I would have thought this would be all projects that are distributed as source? --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jul 19 2011
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-19 11:28, Nick Sabalausky wrote:
 Granted, I've never used this sort of feature from any buildsystem, but I
 thought we were talking about something like the "configure" command in Waf.
 My understanding is that it's like the "./configure" from autotool's
 "./configure&&  make&&  make install", (except hopefully without needing to
 check quite so much stuff or needing the user to manually run the configure
 command).
I would hope D never had to use something like "./configure". It checks for C language features and similar (probably other things as well). I don't think that that belongs in a build system for D.
 Apperently some projects need to have their buildsystem check for the
 existance of, locations of, and details about certain things on the local
 system before building. So...that stuff.
Isn't that to check what libraries, and so on, are present? That is the whole point of a package manager. A package specifies what dependencies it has, then the package manager makes sure the dependencies are met, or else it can't be installed. Don't know what locations it would check for. The package as no saying in where it should be installed, the package manger decides that. Oh, now I mixing package manager and build tool and assume the build tool will be used with the package manager. When I was talking about "config file" I was thinking something more like Rebuild's config files, example: http://pastebin.com/rYc47wXQ -- /Jacob Carlborg
Jul 19 2011
next sibling parent reply Ulrik Mikaelsson <ulrik.mikaelsson gmail.com> writes:
2011/7/19 Jacob Carlborg <doob me.com>:
 Apperently some projects need to have their buildsystem check for the
 existance of, locations of, and details about certain things on the local
 system before building. So...that stuff.
Isn't that to check what libraries, and so on, are present? That is the whole point of a package manager. A package specifies what dependencies it has, then the package manager makes sure the dependencies are met, or else it can't be installed.
All "dependencies" aren't always mandatory. It's not uncommon for some software to adapt itself to the environment, say enabling certain features IF certain other packages can be found, otherwise just disable the functionality. Also it might adapt itself on other conditions, say auto-detecting checking what OS-kernel we are building for, and pass along D versions-keywords accordingly, including allowing the user to override, to do cross-platform or cross-version builds. Also, most build-systems offers options for whomever is building, such as --use-test-codeX, much like Tango now allows you to build with the regular, or the less tested concurrent GC.
 Don't know what locations it would check for. The package as no saying in
 where it should be installed, the package manger decides that.
Same here. You may have multiple builds of the same package, and want to cross test a dependent package against two different installs to do regression-testing. (Or the classic i386/x86_64 debacle)
 Oh, now I mixing package manager and build tool and assume the build tool
 will be used with the package manager.
I think the build-tool and package manager must be independent, but well integrated. Otherwise, you risk making it difficult for D-software to come by default in Linux-distros, and you make it difficult for the D-package-manager to carry a package with a different build-system (even a D based package with different build-system).
Jul 19 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-07-19 15:48, Ulrik Mikaelsson wrote:
 2011/7/19 Jacob Carlborg<doob me.com>:
 Apperently some projects need to have their buildsystem check for the
 existance of, locations of, and details about certain things on the local
 system before building. So...that stuff.
Isn't that to check what libraries, and so on, are present? That is the whole point of a package manager. A package specifies what dependencies it has, then the package manager makes sure the dependencies are met, or else it can't be installed.
All "dependencies" aren't always mandatory. It's not uncommon for some software to adapt itself to the environment, say enabling certain features IF certain other packages can be found, otherwise just disable the functionality. Also it might adapt itself on other conditions, say auto-detecting checking what OS-kernel we are building for, and pass along D versions-keywords accordingly, including allowing the user to override, to do cross-platform or cross-version builds. Also, most build-systems offers options for whomever is building, such as --use-test-codeX, much like Tango now allows you to build with the regular, or the less tested concurrent GC.
Ok, I see.
 Don't know what locations it would check for. The package as no saying in
 where it should be installed, the package manger decides that.
Same here. You may have multiple builds of the same package, and want to cross test a dependent package against two different installs to do regression-testing. (Or the classic i386/x86_64 debacle)
 Oh, now I mixing package manager and build tool and assume the build tool
 will be used with the package manager.
I think the build-tool and package manager must be independent, but well integrated. Otherwise, you risk making it difficult for D-software to come by default in Linux-distros, and you make it difficult for the D-package-manager to carry a package with a different build-system (even a D based package with different build-system).
The package manager and build system will be independent. I just got confuse and it was a mess in my head :). Was unsure if what I wrote would make any sense. -- /Jacob Carlborg
Jul 19 2011
prev sibling parent reply Russel Winder <russel russel.org.uk> writes:
On Tue, 2011-07-19 at 14:55 +0200, Jacob Carlborg wrote:
 On 2011-07-19 11:28, Nick Sabalausky wrote:
 Granted, I've never used this sort of feature from any buildsystem, but=
I
 thought we were talking about something like the "configure" command in=
Waf.
 My understanding is that it's like the "./configure" from autotool's
 "./configure&&  make&&  make install", (except hopefully without needin=
g to
 check quite so much stuff or needing the user to manually run the confi=
gure
 command).
=20 I would hope D never had to use something like "./configure". It checks=
=20
 for C language features and similar (probably other things as well). I=
=20
 don't think that that belongs in a build system for D.
Well Autoconf, Waf and SCons check for a whole lot more than C features, there are C++, Fortran, Python, . . . but yes one part of configuration is (not unreasonably) which version of which languages are available in order to build from source.
 Apperently some projects need to have their buildsystem check for the
 existance of, locations of, and details about certain things on the loc=
al
 system before building. So...that stuff.
=20 Isn't that to check what libraries, and so on, are present? That is the=
=20
 whole point of a package manager. A package specifies what dependencies=
=20
 it has, then the package manager makes sure the dependencies are met, or=
=20
 else it can't be installed.
There is clearly a string coupling between configuration and package management. Autoconf, Waf and SCons have to be portable across package management since they are not dedicated to one scheme -- springing up as they did from before package management as standard.
 Don't know what locations it would check for. The package as no saying=
=20
 in where it should be installed, the package manger decides that.
=20
 Oh, now I mixing package manager and build tool and assume the build=20
 tool will be used with the package manager.
There are some potentially tricky issues here. Ruby gets round it by having a language specific package and build system which therefore causes Debian, Fedora, FreeBSD, Macports, Fink, etc. packagers massive headaches. Haskell/Cabal, Python/Eggs, etc. The conflict between language packaging and platform packaging is central. Having a D language packaging system that was in some way harmonious with Apt/Deb, Yum/RPM, Ports, etc. would make a lot of people very happy. Indirectly it would make traction a whole lot easier. As evidence I present Java, Ruby, Python, Haskell, etc.
 When I was talking about "config file" I was thinking something more=20
 like Rebuild's config files, example: http://pastebin.com/rYc47wXQ
=20
--=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jul 19 2011
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-19 16:46, Russel Winder wrote:
 There is clearly a string coupling between configuration and package
 management.   Autoconf, Waf and SCons have to be portable across package
 management since they are not dedicated to one scheme -- springing up as
 they did from before package management as standard.
Ok.
 There are some potentially tricky issues here.  Ruby gets round it by
 having a language specific package and build system which therefore
 causes Debian, Fedora, FreeBSD, Macports, Fink, etc. packagers massive
 headaches.  Haskell/Cabal, Python/Eggs, etc.  The conflict between
 language packaging and platform packaging is central.  Having a D
 language packaging system that was in some way harmonious with Apt/Deb,
 Yum/RPM, Ports, etc. would make a lot of people very happy.  Indirectly
 it would make traction a whole lot easier.  As evidence I present Java,
 Ruby, Python, Haskell, etc.
The whole point of having a D package manager is so I don't have to create packages for all these system specific package managers. -- /Jacob Carlborg
Jul 19 2011
parent reply Ulrik Mikaelsson <ulrik.mikaelsson gmail.com> writes:
2011/7/19 Jacob Carlborg <doob me.com>:
 On 2011-07-19 16:46, Russel Winder wrote:
 There is clearly a string coupling between configuration and package
 management. =C2=A0 Autoconf, Waf and SCons have to be portable across pa=
ckage
 management since they are not dedicated to one scheme -- springing up as
 they did from before package management as standard.
Ok.
 There are some potentially tricky issues here. =C2=A0Ruby gets round it =
by
 having a language specific package and build system which therefore
 causes Debian, Fedora, FreeBSD, Macports, Fink, etc. packagers massive
 headaches. =C2=A0Haskell/Cabal, Python/Eggs, etc. =C2=A0The conflict bet=
ween
 language packaging and platform packaging is central. =C2=A0Having a D
 language packaging system that was in some way harmonious with Apt/Deb,
 Yum/RPM, Ports, etc. would make a lot of people very happy. =C2=A0Indire=
ctly
 it would make traction a whole lot easier. =C2=A0As evidence I present J=
ava,
 Ruby, Python, Haskell, etc.
The whole point of having a D package manager is so I don't have to creat=
e
 packages for all these system specific package managers.
In my experience with gem and easy_install, it's not really practical for the end-user. After several attempts, I always ran into problems. Especially library-bindings in the language-specific package-manager tends to conflict with what library is installed in the system, causing problems. In all the cases I ended up biting the bullet and create a OS-package for the Python/Ruby packages we're using in production. I've later learned the experience is shared with others, I know one company who even went back to .NET due to too much hassle with Ruby On Rails deployments (gem), and I suppose distribution-developers wouldn't put as much effort into packaging all the Python and Ruby-packages they do if not others had seen problems with per-language packagers. Especially, if D is to gain traction and attract developers, getting proof-of-concept applications out in the respective "app store" of the target OS:es is quite important, which requires working with the native packaging format.
Jul 20 2011
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-20 10:36, Ulrik Mikaelsson wrote:
 In my experience with gem and easy_install, it's not really practical
 for the end-user. After several attempts, I always ran into problems.
 Especially library-bindings in the language-specific package-manager
 tends to conflict with what library is installed in the system,
 causing problems. In all the cases I ended up biting the bullet and
 create a OS-package for the Python/Ruby packages we're using in
 production. I've later learned the experience is shared with others, I
 know one company who even went back to .NET due to too much hassle
 with Ruby On Rails deployments (gem), and I suppose
 distribution-developers wouldn't put as much effort into packaging all
 the Python and Ruby-packages they do if not others had seen problems
 with per-language packagers.
I never had this problem with Ruby or Rails. I find it very hard to belive that the only reason they switch back to .NET was problems with the package manager. I mean Ruby on Rails is the easiest platform I've used, just list the packages you want in a file and run "bundle" and everything is installed. So what are you suggesting, that we don't have a package manager for D?
 Especially, if D is to gain traction and attract developers, getting
 proof-of-concept applications out in the respective "app store" of the
 target OS:es is quite important, which requires working with the
 native packaging format.
Linux and FreeBSD is the only platforms that have a native package manager. Mac OS X has macports and homebrew, none of which are installed by default. Windows doesn't have any package manager, as far as I know. So you suggest that I should create, something like, six packages for one library just to support all platforms and package mangers. And then it won't even support Windows. That would probably also be in 6 different "languages" for the package scripts. -- /Jacob Carlborg
Jul 20 2011
parent reply Ulrik Mikaelsson <ulrik.mikaelsson gmail.com> writes:
 So what are you suggesting, that we don't have a package manager for D?
I'm suggesting, I'm not likely to use it, and developing a D-specific package manager should not accidentally exclude stand-alone building of packages. As I said before, the build-system and packaging system must be independent, but well integrated. Quoting again from Russel Winder:
 Having a D language packaging system that was in some way harmonious
 with Apt/Deb, Yum/RPM, Ports, etc. would make a lot of people very happy.
 Indirectly it would make traction a whole lot easier.
Jul 20 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-07-20 14:01, Ulrik Mikaelsson wrote:
 So what are you suggesting, that we don't have a package manager for D?
I'm suggesting, I'm not likely to use it, and developing a D-specific package manager should not accidentally exclude stand-alone building of packages. As I said before, the build-system and packaging system must be independent, but well integrated.
Of course, they should be independent. If I gave the impression of anything else, then that's a misunderstanding.
 Quoting again from Russel Winder:
 Having a D language packaging system that was in some way harmonious
 with Apt/Deb, Yum/RPM, Ports, etc. would make a lot of people very happy.
 Indirectly it would make traction a whole lot easier.
-- /Jacob Carlborg
Jul 20 2011
prev sibling parent reply David Nadlinger <see klickverbot.at> writes:
On 7/20/11 10:36 AM, Ulrik Mikaelsson wrote:
 […]I've later learned the experience is shared with others, I
 know one company who even went back to .NET due to too much hassle
 with Ruby On Rails deployments (gem), […]
Do you happen to known what issues they experienced in more detail? I'm curious because I personally found Rails deployment to be quite convenient, extending the Gemfile during development as needed, and then using Bundler (http://gembundler.com/) to ensure that the exact same versions of the dependencies are used when staging and deploying… David
Jul 20 2011
next sibling parent reply Ulrik Mikaelsson <ulrik.mikaelsson gmail.com> writes:
2011/7/20 David Nadlinger <see klickverbot.at>:
 Do you happen to known what issues they experienced in more detail? I'm
 curious because I personally found Rails deployment to be quite convenien=
t,
 extending the Gemfile during development as needed, and then using Bundle=
r
 (http://gembundler.com/) to ensure that the exact same versions of the
 dependencies are used when staging and deploying=E2=80=A6
I do not know exactly what their problems was. For me, it was mainly conflicts between OS package manager and language-specific. I recall curl-bindings, and MySQL binding causing a lot of problems, also I recall some graphics library depending on GTK breaking spectacularly. Perhaps it's more polished now, or perhaps I just had bad luck.
Jul 20 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-20 14:05, Ulrik Mikaelsson wrote:
 2011/7/20 David Nadlinger<see klickverbot.at>:
 Do you happen to known what issues they experienced in more detail? I'm
 curious because I personally found Rails deployment to be quite convenient,
 extending the Gemfile during development as needed, and then using Bundler
 (http://gembundler.com/) to ensure that the exact same versions of the
 dependencies are used when staging and deploying…
I do not know exactly what their problems was. For me, it was mainly conflicts between OS package manager and language-specific. I recall curl-bindings, and MySQL binding causing a lot of problems, also I recall some graphics library depending on GTK breaking spectacularly. Perhaps it's more polished now, or perhaps I just had bad luck.
What do you need curl for? Ruby has built-in support for bascially all curl's functionality. I haven't had any problems with using Ruby on Rails together with MySQL, on Linux. -- /Jacob Carlborg
Jul 20 2011
parent Ulrik Mikaelsson <ulrik.mikaelsson gmail.com> writes:
2011/7/20 Jacob Carlborg <doob me.com>:
 What do you need curl for?
Saturating gigabit file transfers with wide support of varying protocols, encryption and authentication schemes. In any case, there's always going to be needs for bindings, either because a package in other language (usually C/C++) offers more functionality, or (not likely a problem for D) higher performance.
Jul 20 2011
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-07-20 12:50, David Nadlinger wrote:
 On 7/20/11 10:36 AM, Ulrik Mikaelsson wrote:
 […]I've later learned the experience is shared with others, I
 know one company who even went back to .NET due to too much hassle
 with Ruby On Rails deployments (gem), […]
Do you happen to known what issues they experienced in more detail? I'm curious because I personally found Rails deployment to be quite convenient, extending the Gemfile during development as needed, and then using Bundler (http://gembundler.com/) to ensure that the exact same versions of the dependencies are used when staging and deploying… David
Exactly. -- /Jacob Carlborg
Jul 20 2011
prev sibling parent "Nick Sabalausky" <a a.a> writes:
"Russel Winder" <russel russel.org.uk> wrote in message 
news:mailman.1770.1311086829.14074.digitalmars-d puremagic.com...
There are some potentially tricky issues here.  Ruby gets round it by
having a language specific package and build system which therefore
causes Debian, Fedora, FreeBSD, Macports, Fink, etc. packagers massive
headaches.  Haskell/Cabal, Python/Eggs, etc.  The conflict between
language packaging and platform packaging is central.  Having a D
language packaging system that was in some way harmonious with Apt/Deb,
Yum/RPM, Ports, etc. would make a lot of people very happy.  Indirectly
it would make traction a whole lot easier.  As evidence I present Java,
Ruby, Python, Haskell, etc.
What would be needed for a D package manager to be "harmonious with" such systems? I don't know anything about the issues OS packagers face regarding language-oriented package systems.
Jul 19 2011
prev sibling parent "Nick Sabalausky" <a a.a> writes:
"Nick Sabalausky" <a a.a> wrote in message 
news:ivviur$2ks8$1 digitalmars.com...
 "Jacob Carlborg" <doob me.com> wrote in message 
 news:ivp1gd$500$2 digitalmars.com...
 Ok, have you thought about how this will look (in code) and behave?
Not extensively, but here's what I have in mind so far. In an early initial version of Drake, it would be like this: task("configure", "drake.conf"); file("drake.conf", (Target t) { // Do all configuring here and save to drake.conf } );
Actually, I think that might have to be more like: // This part added: if(DrakeArgs.target == "configure") remove("drake.conf"); task("configure", "drake.conf"); file("drake.conf", (Target t) { // Do all configuring here and save to drake.conf } ); Otherwise "drake configure" wouldn't re-generate the configuration unless drake.conf was manually deleted first. Or, maybe there should just be a "--force" option (also accesible from code? on a per-task basis?) to skip the dependency checks and forcefully rebuild. Oh, and maybe the configuration fle should just be named "configure": // Use --force to re-configure file("configure", (Target t) { // Do all configuring here and save to 'configure' } ); Pardon the public brainstorming...
Jul 19 2011
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-13 23:03, jdrewsen wrote:
 Den 13-07-2011 03:02, Nick Sabalausky skrev:
 The recent discussions about package managers and buildsystems has
 prompted
 me to get off my ass (or rather, *on* my ass as per how programming is
 usually performed...) and start on the D-based rake-inspired buildtool
 I've
 been meaning to make for awhile now. It's not actually usable yet, but
 there's a sample drakefile demonstrating everything, and it does actually
 compile and run (but just doesn't build the dependency tree or
 actually run
 any of the build steps). *Should* work on Posix, but I only tested on
 Windows, so I may have fucked up the Posix version...

 Apologies to Jacob Carlborg for the name being so close to "dake". Didn't
 really know what else to call it ("Duck", maybe?) Like dake, it's
 inspired
 by Ruby's Rake. But unlike dake, the buildscript is written in D
 instead of


 Before I go implemeting everything, I'd like to get input on it. Is it
 something that could be a major D tool?

 Overview of Drake and links to all the code are here:

 https://bitbucket.org/Abscissa256/drake/wiki/Home
A good start I think. The first thing that I think should be fixed is the naming. Drake should be called dbuild Orb should be called dpack You read the name and you have an idea of what it does - nothing fancy.
"Drake" was probably chosen because it's basically "rake for D" and possibly because "dake" was already taken. "rake" was most likely chosen because it's "make" but the makefiles are written in Ruby instead. I chose "Orbit" as the name of the package manager and named the tool "Orb". Why? No particular reason, but after that someone mentioned it has the same theme as (Digital)Mars and Phobos.
 People coming from ruby may recognize the drake name, but I guess many
 c++ peoply have no idea.
"Drake" is derived from Rake which is derived from Make.
 Orb is not derived from anything I guess.

 Regarding drake itself:

 I really like how general it is but at the same time miss some
 simplicity. "Make the common task easy and the uncommon possible"
So true, so true.
 Some steps that you could take in that direction is to predefine
 commonly used modes like "build", "clean" etc. And the define
 buildMode(), cleanMode() as builtins. That would get rid of the
 "immutable modes = ..." lines in simple projects.

 Maybe let the wrapper that loads the drakefile insert the "import
 drake.all" and also wrap the "void drakefile() { ... }" around the file.
 Maybe that is what your have already listed on the todo?

 As suggested elsewhere you could get rid of target!Task or target!File
 and just defined the target() and file() functions.

 Also a convenience could be to just include all .d files found for an
 exe target if no .d files are explicitly listed.

 I guess this would make it possible to have a drakefile as simple as:

 task("appA".exe);


 It seems to lack the configure part of the build process e.g. locate the
 libfoobar.a file for linking or locate a D compiler.

 Thanks,
 /Jonas
I agree with most of this. -- /Jacob Carlborg
Jul 14 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:ivnfk8$2v3p$1 digitalmars.com...
 On 2011-07-13 23:03, jdrewsen wrote:
 A good start I think. The first thing that I think should be fixed is
 the naming.

 Drake should be called dbuild
 Orb should be called dpack

 You read the name and you have an idea of what it does - nothing fancy.
"Drake" was probably chosen because it's basically "rake for D" and possibly because "dake" was already taken. "rake" was most likely chosen because it's "make" but the makefiles are written in Ruby instead. I chose "Orbit" as the name of the package manager and named the tool "Orb". Why? No particular reason, but after that someone mentioned it has the same theme as (Digital)Mars and Phobos.
Between "Orb" and "Orange" I was thinking you just liked using names based on round things :)
Jul 14 2011
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday 14 July 2011 15:18:46 Nick Sabalausky wrote:
 "Jacob Carlborg" <doob me.com> wrote in message
 news:ivnfk8$2v3p$1 digitalmars.com...
 
 On 2011-07-13 23:03, jdrewsen wrote:
 A good start I think. The first thing that I think should be fixed is
 the naming.
 
 Drake should be called dbuild
 Orb should be called dpack
 
 You read the name and you have an idea of what it does - nothing
 fancy.
"Drake" was probably chosen because it's basically "rake for D" and possibly because "dake" was already taken. "rake" was most likely chosen because it's "make" but the makefiles are written in Ruby instead. I chose "Orbit" as the name of the package manager and named the tool "Orb". Why? No particular reason, but after that someone mentioned it has the same theme as (Digital)Mars and Phobos.
Between "Orb" and "Orange" I was thinking you just liked using names based on round things :)
That and names starting with "Or". - Jonathan M Davis
Jul 14 2011
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-14 21:18, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ivnfk8$2v3p$1 digitalmars.com...
 On 2011-07-13 23:03, jdrewsen wrote:
 A good start I think. The first thing that I think should be fixed is
 the naming.

 Drake should be called dbuild
 Orb should be called dpack

 You read the name and you have an idea of what it does - nothing fancy.
"Drake" was probably chosen because it's basically "rake for D" and possibly because "dake" was already taken. "rake" was most likely chosen because it's "make" but the makefiles are written in Ruby instead. I chose "Orbit" as the name of the package manager and named the tool "Orb". Why? No particular reason, but after that someone mentioned it has the same theme as (Digital)Mars and Phobos.
Between "Orb" and "Orange" I was thinking you just liked using names based on round things :)
Hehe. I actually started using names of fruits for my projects (I have an unreleased library named "lime"), don't really know why I chose Orbit. -- /Jacob Carlborg
Jul 15 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:ivp0rm$3lq$2 digitalmars.com...
 On 2011-07-14 21:18, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 news:ivnfk8$2v3p$1 digitalmars.com...
 On 2011-07-13 23:03, jdrewsen wrote:
 A good start I think. The first thing that I think should be fixed is
 the naming.

 Drake should be called dbuild
 Orb should be called dpack

 You read the name and you have an idea of what it does - nothing fancy.
"Drake" was probably chosen because it's basically "rake for D" and possibly because "dake" was already taken. "rake" was most likely chosen because it's "make" but the makefiles are written in Ruby instead. I chose "Orbit" as the name of the package manager and named the tool "Orb". Why? No particular reason, but after that someone mentioned it has the same theme as (Digital)Mars and Phobos.
Between "Orb" and "Orange" I was thinking you just liked using names based on round things :)
Hehe. I actually started using names of fruits for my projects (I have an unreleased library named "lime"), don't really know why I chose Orbit.
Fan of the Mango project? ;)
Jul 15 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-07-16 00:11, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 Hehe. I actually started using names of fruits for my projects (I have an
 unreleased library named "lime"), don't really know why I chose Orbit.
Fan of the Mango project? ;)
Hehe, I've actually never used it. But probably there I got the initial idea. It can be so hard to name a library/tool, so why not just take a simple word that doesn't say anything about the actual tool and be done with it. -- /Jacob Carlborg
Jul 16 2011
parent "Nick Sabalausky" <a a.a> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:ivrm1t$1f20$2 digitalmars.com...
 On 2011-07-16 00:11, Nick Sabalausky wrote:
 "Jacob Carlborg"<doob me.com>  wrote in message
 Hehe. I actually started using names of fruits for my projects (I have 
 an
 unreleased library named "lime"), don't really know why I chose Orbit.
Fan of the Mango project? ;)
Hehe, I've actually never used it. But probably there I got the initial idea. It can be so hard to name a library/tool, so why not just take a simple word that doesn't say anything about the actual tool and be done with it.
That's a good point. I agonize over project names far too much.
Jul 16 2011