www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Build using D

reply NaN <divide by.zero> writes:
Why not actually use D as build system for D projects? I assume 
DUB is written in D, but why not just have DUB features in a 
library and use a D build script?

rdmd build.d

Is it a dumb idea?
Apr 16
next sibling parent reply Cym13 <cpicard openmailbox.org> writes:
On Thursday, 16 April 2020 at 17:11:39 UTC, NaN wrote:
 Why not actually use D as build system for D projects? I assume 
 DUB is written in D, but why not just have DUB features in a 
 library and use a D build script?

 rdmd build.d

 Is it a dumb idea?
You might be interested in https://github.com/atilaneves/reggae
Apr 16
parent ted <spam spam.spam> writes:
On 17/4/20 2:47 am, Cym13 wrote:
 On Thursday, 16 April 2020 at 17:11:39 UTC, NaN wrote:
 Why not actually use D as build system for D projects? I assume DUB is 
 written in D, but why not just have DUB features in a library and use 
 a D build script?

 rdmd build.d

 Is it a dumb idea?
You might be interested in https://github.com/atilaneves/reggae
Also have a look at https://github.com/GrahamStJack/bottom-up-build (used to build/develop mission-critical software)
Apr 16
prev sibling next sibling parent Cogitri <oss cogitri.dev> writes:
On Thursday, 16 April 2020 at 17:11:39 UTC, NaN wrote:
 Why not actually use D as build system for D projects? I assume 
 DUB is written in D, but why not just have DUB features in a 
 library and use a D build script?

 rdmd build.d

 Is it a dumb idea?
Using a fully featured programming language for the build system usually leads to everyone doing things in a slightly different (and usually very brittle) way. See e.g. build.rs from the Rust world - you need different environment variables for each package to use system packages instead of using some random version the -sys package distributes, half of the packages don't even use pkg-config to detect native libs and just assume you have everything in the same spot as the distro the developer uses and generally everything is prone to breaking if you have a slightly different setup than the developers. Really, I think it the less custom and "clever" a buildsystem is, the better. Personally, I think it's best to just use dub or meson.
Apr 16
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/16/20 1:11 PM, NaN wrote:
 Why not actually use D as build system for D projects? I assume DUB is 
 written in D, but why not just have DUB features in a library and use a 
 D build script?
 
 rdmd build.d
 
 Is it a dumb idea?
 
 
DMD doesn't think so: https://github.com/dlang/dmd/blob/master/src/build.d And it's fast too. I really like it better than the makefile bs before. -Steve
Apr 16
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Apr 16, 2020 at 01:45:51PM -0400, Steven Schveighoffer via
Digitalmars-d wrote:
 On 4/16/20 1:11 PM, NaN wrote:
 Why not actually use D as build system for D projects? I assume DUB
 is written in D, but why not just have DUB features in a library and
 use a D build script?
 
 rdmd build.d
 
 Is it a dumb idea?
DMD doesn't think so: https://github.com/dlang/dmd/blob/master/src/build.d And it's fast too. I really like it better than the makefile bs before.
[...] Just about *anything* is better than the makefile bs. :-P T -- Prosperity breeds contempt, and poverty breeds consent. -- Suck.com
Apr 16
prev sibling parent NaN <divide by.zero> writes:
On Thursday, 16 April 2020 at 17:45:51 UTC, Steven Schveighoffer 
wrote:
 On 4/16/20 1:11 PM, NaN wrote:
 Why not actually use D as build system for D projects? I 
 assume DUB is written in D, but why not just have DUB features 
 in a library and use a D build script?
 
 rdmd build.d
 
 Is it a dumb idea?
 
 
DMD doesn't think so: https://github.com/dlang/dmd/blob/master/src/build.d And it's fast too. I really like it better than the makefile bs before.
How much of that build.d could be factored out into a separate build utilities module? I was thinking not that people would roll their own completely custom build script, more like you'd have a bunch of utilities and be able to have a concise build script that was very flexible and easy to add extra features. So it's quick and easy if people want it to be, but can be extended easily, plus its D, so it's not another language / bunch of rules/quirks to learn.
Apr 16
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Thursday, 16 April 2020 at 17:11:39 UTC, NaN wrote:
 Why not actually use D as build system for D projects? I assume 
 DUB is written in D, but why not just have DUB features in a 
 library and use a D build script?

 rdmd build.d

 Is it a dumb idea?
It's a bad idea. A declarative build description trades being power for being conventional and deriving rules that works on every project (through "lack of power"). For example, dub test, dub dustmite etc... are impossible with powerful immediate build (as opposed to declarative). Your "unique" build combination has to be justified. If a standard build system cannot express your program, _perhaps_ it's up to you to simplify your build, rather than the build system to adapt to unique complexity. That said, there is always a large class of programs that cannot easily be built, that domain is cursed.
Apr 17
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Apr 17, 2020 at 08:24:06AM +0000, Guillaume Piolat via Digitalmars-d
wrote:
[...]
 A declarative build description trades being power for being
 conventional and deriving rules that works on every project (through
 "lack of power").
IMNSHO, this is a flawed argument. The basics of a build system is a directed acyclic graph, with build sources, any intermediate targets, and targets represented by the nodes, and derivation rules represented by the edges. This model works for every project, but does not trade for any "lack of power". The problem with most modern build systems is that instead of tackling this essential problem structure they try to "simplify" it by tacking on various assumptions and conventions, which impose artificial limitations on the system. Just like the old version control systems like RCS/CVS, Subversion, etc., they assume a particular workflow / revision history structure, and therefore when you encounter a case that wasn't considered in the original design, it's a pain to work around (CVS has incredibly long times just for creating a branch, for example; SVN is much better in that it's just a copy, but still requires server roundtrips, and merges can sometimes be nightmarish). Modern systems like Git directly address the underlying structure of a revision graph -- it's a directed acyclic graph, you see -- and therefore allows extremely efficient branching/merging, and other graph manipulations unimaginable in other revision control systems. It's about time the same thing happened in the build system world. Something that tackles the underlying DAG directly, and provides modern guarantees like 100% build reproducibility, ability to handle any task that fits into the DAG structure,[1] not just some canned operations like "derive executable from C++ sources", and build times proportional to the size of change rather than the size of the entire workspace. I have a website project, for example, where sources include mathematical model files which get transformed into POVRay input files that are fed to povray to produce images, then post-processed by imagemagick to produce .png's which are then installed onto a webserver -- nowhere is a "compiler" involved but the structure of the task remains the same: input -> transform -> output. Another project takes 3D model data as input, and transforms them into runtime data and accessor code in the form of D source code, that then gets cross-compiled to ARM and packaged into an APK for Android. In all of these cases, the details of the input -> output transformation differs widely, but they nonetheless all fit into the framework of dependency resolution over a DAG. This is not even a tough problem like satisfying versioned dependencies, an NP-complete problem. Current technology is well able to handle it with ease. The only real problem that stands in the way is, IMNSHO, inertia of developers too used to their aging tools to try something new, and the lack of a well-known project to propel the prospective build system from obscurity into the limelight (e.g., without Linux, I honestly doubt most people would've even heard of git -- Torvalds probably wouldn't have bothered to write it in the first place).
 For example, dub test, dub dustmite etc... are impossible with
 powerful immediate build (as opposed to declarative).
I don't agree with this. A "powerful" build system can still be fully declarative, and amenable to auto-testing, bisecting, etc.. Nothing about the DAG model requires that your build must be expressed in an imperative language. You just have to think outside the box a bit to arrive at a workable solution.
 Your "unique" build combination has to be justified.  If a standard
 build system cannot express your program, _perhaps_ it's up to you to
 simplify your build, rather than the build system to adapt to unique
 complexity.
I can't agree with this. Why should I change the structure of my build just because some build system stuck with a dated paradigm can't handle its "complexity"? Nothing can be more elementary that the concept of transforming a bunch of inputs into some set of outputs via some specified transformation. I'm not even asking for something so complex as solving an NP-complete problem in order to satisfy versioned dependencies. If a build system can't handle something as simple as that, _perhaps_ that build system ought to be replaced with something more competent! ;-) (And I note, ironically, that dub *does* include a solver for the NP-complete problem of satisfying versioned dependencies, *yet* it is unable to handle something so simple as compiling a helper program from some subset of input files, running said program to generate some more source files, and compiling said source files into the final executable. Note that each of these steps is again a simple input -> transform -> output step. Something capable of solving an NP-complete problem ought to find such a thing beyond trivial! Yet it cannot because of the artificial restrictions arbitrarily placed upon it -- *needless* arbitrary restrictions IMNSHO.)
 That said, there is always a large class of programs that cannot
 easily be built, that domain is cursed.
It's only cursed if one refuses to drop one's outmoded way of thinking and embrace a better paradigm. ;-) T -- What's an anagram of "BANACH-TARSKI"? BANACH-TARSKI BANACH-TARSKI.
Apr 29
parent reply ted <spam spam.spam> writes:
On 30/4/20 2:39 am, H. S. Teoh wrote:
 On Fri, Apr 17, 2020 at 08:24:06AM +0000, Guillaume Piolat via Digitalmars-d
wrote:
 [...]
 A declarative build description trades being power for being
 conventional and deriving rules that works on every project (through
 "lack of power").
IMNSHO, this is a flawed argument. The basics of a build system is a directed acyclic graph, with build sources, any intermediate targets, and targets represented by the nodes, and derivation rules represented by the edges. This model works for every project, but does not trade for any "lack of power".
<snip>
 
 It's about time the same thing happened in the build system world.
 Something that tackles the underlying DAG directly, and provides modern
 guarantees like 100% build reproducibility, ability to handle any task
 that fits into the DAG structure,[1] not just some canned operations
 like "derive executable from C++ sources", and build times proportional
 to the size of change rather than the size of the entire workspace.
 
<snip> I agree 100% - which is the exact approach taken by https://github.com/GrahamStJack/bottom-up-build --ted
Apr 29
parent Panke <tobias pankrath.net> writes:
On Thursday, 30 April 2020 at 05:20:28 UTC, ted wrote:
 I agree 100% - which is the exact approach taken by 
 https://github.com/GrahamStJack/bottom-up-build

 --ted
It's also what bazel does, which has a little bit more support behind it. I've started to investigate how it could interact with the dub registry, but have not gotten back to it yet.
Apr 30
prev sibling parent guai <guai fake.mail> writes:
On Thursday, 16 April 2020 at 17:11:39 UTC, NaN wrote:
 Is it a dumb idea?
This is what gradle does for java. It's made around java API and groovy/kotlin DSL on top of it. In most cases users do not go beyond some declarative dsl describing project name, applying plugins, adding simple tasks and their dependencies from standard ones; where files lay, where to pull libs from, credentials and so on. But when this isn't enough one can write their own hooks in groovy or kotlin, or even write a plugin. And I find it better since in fully declarative maven once you hit some nonstandard need (which large projects eventually do) you fallback to painful programming in xml. With gradle you just write and debug another program, which you already know how to do, just using their API, and not digging some alien declarative stuff, and how it actually being converted to code.
Apr 30