www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - dsss / scons

reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Anyone given any thought to building a dsss-like system on top of SCONS? 
  Maybe even one compatible with the real dsss?  Maybe even mr. Gregor 
Richards has considered this?

Just a thought that popped in my head after watching my app's resources 
get needlessly compiled for the umpteenth time because of the lack of 
dependency tracking for anything but D code in DSSS.

--bb
Jan 01 2008
next sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
Bill Baxter wrote:

 Anyone given any thought to building a dsss-like system on top of SCONS?
   Maybe even one compatible with the real dsss?  Maybe even mr. Gregor
 Richards has considered this?
 
 Just a thought that popped in my head after watching my app's resources
 get needlessly compiled for the umpteenth time because of the lack of
 dependency tracking for anything but D code in DSSS.
 
 --bb

A few years back (before build) someone added D support to SCONS, and I added it to A-A-P. Neither gained much momentum as the python dependency for the build system seemed to put most people off. -- Lars Ivar Igesund blog at http://larsivi.net DSource, #d.tango & #D: larsivi Dancing the Tango
Jan 01 2008
prev sibling parent reply =?ISO-8859-1?Q?=22J=E9r=F4me_M=2E_Berger=22?= <jeberger free.fr> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bill Baxter wrote:
 Anyone given any thought to building a dsss-like system on top of SCONS?
  Maybe even one compatible with the real dsss?  Maybe even mr. Gregor
 Richards has considered this?
 
 Just a thought that popped in my head after watching my app's resources
 get needlessly compiled for the umpteenth time because of the lack of
 dependency tracking for anything but D code in DSSS.
 

much for it. What I have however is a system built on top of SCons which you can point at a folder and say (among other things): "pick every D source file in here and build a program / a lib with them". I can make it available if you'd like. Jerome - -- +------------------------- Jerome M. BERGER ---------------------+ | mailto:jeberger free.fr | ICQ: 238062172 | | http://jeberger.free.fr/ | Jabber: jeberger jabber.fr | +---------------------------------+------------------------------+ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) iD8DBQFHejXUd0kWM4JG3k8RAjRwAKCIOfkgD62xGME1iMEBzv/m5tcYvACfdLzx ODaA23OZluOWuTHhCogJqZ8= =cxmH -----END PGP SIGNATURE-----
Jan 01 2008
parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Jérôme M. Berger wrote:
 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1
 
 Bill Baxter wrote:
 Anyone given any thought to building a dsss-like system on top of SCONS?
  Maybe even one compatible with the real dsss?  Maybe even mr. Gregor
 Richards has considered this?

 Just a thought that popped in my head after watching my app's resources
 get needlessly compiled for the umpteenth time because of the lack of
 dependency tracking for anything but D code in DSSS.

much for it. What I have however is a system built on top of SCons which you can point at a folder and say (among other things): "pick every D source file in here and build a program / a lib with them". I can make it available if you'd like.

How is that different from the built-in D builder in SCons? Honest question. I've never tried to build D code with SCons before, so I really don't know what the current state of the art is. If your code is better, maybe it should go upstream to be included with SCons itself. --bb
Jan 01 2008
parent reply =?ISO-8859-1?Q?=22J=E9r=F4me_M=2E_Berger=22?= <jeberger free.fr> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bill Baxter wrote:
 Jérôme M. Berger wrote:
 Bill Baxter wrote:
 Anyone given any thought to building a dsss-like system on top of SCONS?
  Maybe even one compatible with the real dsss?  Maybe even mr. Gregor
 Richards has considered this?

 Just a thought that popped in my head after watching my app's resources
 get needlessly compiled for the umpteenth time because of the lack of
 dependency tracking for anything but D code in DSSS.

much for it. What I have however is a system built on top of SCons which you can point at a folder and say (among other things): "pick every D source file in here and build a program / a lib with them". I can make it available if you'd like.

How is that different from the built-in D builder in SCons? Honest question. I've never tried to build D code with SCons before, so I really don't know what the current state of the art is. If your code is better, maybe it should go upstream to be included with SCons itself.

three things: - A 10-lines python function that scans a folder and returns the list of files matching a given regexp. This list can then be given to the standard SCons builder to generate a lib or a program; - An improved dependency scanner which recognizes the '.di' extension and multiple imports (eg "import a, b;"). That one was included upstream a couple of weeks ago and should be available in the next development release of SCons; - Support for gdc (since dmd doesn't work on 64bits linux). Unfortunately, that one is not properly integrated with the standard SCons and cannot be sent upstream as is. What I've thought about doing is a function that would take a program (or lib) name and a list of source files (possibly including only a single file) and automatically compile all the dependencies. Here's an example. Let's assume you want to build a program named "foo" and you have the following source hierarchy: src `- main.d `- module1.d `- module2.d And main.d imports both module1 and module2. With standard SCons, you need to do: env.Program ("foo", Split ("main.d module1.d module2.d")) Which can become pretty cumbersome if you have lots of source files. With my current system, you can do: env.Program ("foo", scanFolder ("src", re.compile (r".*\.d$"))) Which means that each program or lib must have its own folder independent of the others. With what I've thought to do when I get the time, you would only have to say: env.DProgram ("foo", "main.d") Jerome - -- +------------------------- Jerome M. BERGER ---------------------+ | mailto:jeberger free.fr | ICQ: 238062172 | | http://jeberger.free.fr/ | Jabber: jeberger jabber.fr | +---------------------------------+------------------------------+ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) iD8DBQFHekHWd0kWM4JG3k8RApAEAJ9VTBiFrYzLxwBVp5Qa56UHTFAh6ACfU1uo P1JUE82H/VALE7PxbV3OUXY= =P1oP -----END PGP SIGNATURE-----
Jan 01 2008
next sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Jérôme M. Berger wrote:
 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1
 
 Bill Baxter wrote:
 Jérôme M. Berger wrote:
 Bill Baxter wrote:
 Anyone given any thought to building a dsss-like system on top of SCONS?
  Maybe even one compatible with the real dsss?  Maybe even mr. Gregor
 Richards has considered this?

 Just a thought that popped in my head after watching my app's resources
 get needlessly compiled for the umpteenth time because of the lack of
 dependency tracking for anything but D code in DSSS.

much for it. What I have however is a system built on top of SCons which you can point at a folder and say (among other things): "pick every D source file in here and build a program / a lib with them". I can make it available if you'd like.

question. I've never tried to build D code with SCons before, so I really don't know what the current state of the art is. If your code is better, maybe it should go upstream to be included with SCons itself.

three things: - A 10-lines python function that scans a folder and returns the list of files matching a given regexp. This list can then be given to the standard SCons builder to generate a lib or a program; - An improved dependency scanner which recognizes the '.di' extension and multiple imports (eg "import a, b;"). That one was included upstream a couple of weeks ago and should be available in the next development release of SCons; - Support for gdc (since dmd doesn't work on 64bits linux). Unfortunately, that one is not properly integrated with the standard SCons and cannot be sent upstream as is. What I've thought about doing is a function that would take a program (or lib) name and a list of source files (possibly including only a single file) and automatically compile all the dependencies. Here's an example. Let's assume you want to build a program named "foo" and you have the following source hierarchy: src `- main.d `- module1.d `- module2.d And main.d imports both module1 and module2. With standard SCons, you need to do: env.Program ("foo", Split ("main.d module1.d module2.d")) Which can become pretty cumbersome if you have lots of source files. With my current system, you can do: env.Program ("foo", scanFolder ("src", re.compile (r".*\.d$"))) Which means that each program or lib must have its own folder independent of the others. With what I've thought to do when I get the time, you would only have to say: env.DProgram ("foo", "main.d")

I see. Thanks for the explanation. So the "what you've thought to do" part is to basically make a bud/rebuild-like builder? I think the compiler's "-v -o-" flags can help get a simple prototype working pretty quickly, but I guess there must be more to it that that or else rebuild wouldn't be lugging around the entire front end source code just to find dependencies. And maybe those flags only work for dmd? --bb
Jan 01 2008
next sibling parent =?ISO-8859-1?Q?=22J=E9r=F4me_M=2E_Berger=22?= <jeberger free.fr> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bill Baxter wrote:
 Jérôme M. Berger wrote:
 Bill Baxter wrote:
 Jérôme M. Berger wrote:
 Bill Baxter wrote:
 Anyone given any thought to building a dsss-like system on top of
 SCONS?
  Maybe even one compatible with the real dsss?  Maybe even mr. Gregor
 Richards has considered this?

 Just a thought that popped in my head after watching my app's
 resources
 get needlessly compiled for the umpteenth time because of the lack of
 dependency tracking for anything but D code in DSSS.

much for it. What I have however is a system built on top of SCons which you can point at a folder and say (among other things): "pick every D source file in here and build a program / a lib with them". I can make it available if you'd like.

question. I've never tried to build D code with SCons before, so I really don't know what the current state of the art is. If your code is better, maybe it should go upstream to be included with SCons itself.

three things: - A 10-lines python function that scans a folder and returns the list of files matching a given regexp. This list can then be given to the standard SCons builder to generate a lib or a program; - An improved dependency scanner which recognizes the '.di' extension and multiple imports (eg "import a, b;"). That one was included upstream a couple of weeks ago and should be available in the next development release of SCons; - Support for gdc (since dmd doesn't work on 64bits linux). Unfortunately, that one is not properly integrated with the standard SCons and cannot be sent upstream as is. What I've thought about doing is a function that would take a program (or lib) name and a list of source files (possibly including only a single file) and automatically compile all the dependencies. Here's an example. Let's assume you want to build a program named "foo" and you have the following source hierarchy: src `- main.d `- module1.d `- module2.d And main.d imports both module1 and module2. With standard SCons, you need to do: env.Program ("foo", Split ("main.d module1.d module2.d")) Which can become pretty cumbersome if you have lots of source files. With my current system, you can do: env.Program ("foo", scanFolder ("src", re.compile (r".*\.d$"))) Which means that each program or lib must have its own folder independent of the others. With what I've thought to do when I get the time, you would only have to say: env.DProgram ("foo", "main.d")

I see. Thanks for the explanation. So the "what you've thought to do" part is to basically make a bud/rebuild-like builder?

 I think the compiler's "-v -o-" flags can help get a simple prototype
 working pretty quickly, but I guess there must be more to it that that
 or else rebuild wouldn't be lugging around the entire front end source
 code just to find dependencies. 

I'd guess that the reason they need the entire front-end is because imports may be emitted from string mixins with the strings generated through CTFE and in order to handle conditional compilation. The system I had in mind would be simpler than that: it would only handle basic imports (but that should cover 95+% of user needs). Moreover, I don't know how much support there would be for conditional compilation (no "static if" and I don't know how much "version").
 And maybe those flags only work for dmd?
 

This might indeed be a way to look. Jerome - -- +------------------------- Jerome M. BERGER ---------------------+ | mailto:jeberger free.fr | ICQ: 238062172 | | http://jeberger.free.fr/ | Jabber: jeberger jabber.fr | +---------------------------------+------------------------------+ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) iD8DBQFHek22d0kWM4JG3k8RAs11AJ9Z31SDTB8jjzYa7cA4Pefu39wb1gCfQ1So in1Jdkm3+Wcr5Ls5ivbANi4= =o1N3 -----END PGP SIGNATURE-----
Jan 01 2008
prev sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
Bill Baxter wrote:
 
 I think the compiler's "-v -o-" flags can help get a simple prototype
 working pretty quickly, but I guess there must be more to it that that
 or else rebuild wouldn't be lugging around the entire front end source
 code just to find dependencies.  And maybe those flags only work for dmd?

It is fairly easy detecting imports correctly (except possibly for the string mixin stuff) without the frontend, but the frontend _do_ give rebuild a whole lot of potential power, for instance the ddoc generation is enabled there. Partially I guess it is mostly done because Gregor have several dmd frontend projects, and using it for rebuild was convenient. -- Lars Ivar Igesund blog at http://larsivi.net DSource, #d.tango & #D: larsivi Dancing the Tango
Jan 01 2008
prev sibling parent Jay Norwood <jayn io.com> writes:
Jérôme M. Berger Wrote:

 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1
 
 Bill Baxter wrote:
 Jérôme M. Berger wrote:
 Bill Baxter wrote:
 Anyone given any thought to building a dsss-like system on top of SCONS?
  Maybe even one compatible with the real dsss?  Maybe even mr. Gregor
 Richards has considered this?

 Just a thought that popped in my head after watching my app's resources
 get needlessly compiled for the umpteenth time because of the lack of
 dependency tracking for anything but D code in DSSS.

much for it. What I have however is a system built on top of SCons which you can point at a folder and say (among other things): "pick every D source file in here and build a program / a lib with them". I can make it available if you'd like.

How is that different from the built-in D builder in SCons? Honest question. I've never tried to build D code with SCons before, so I really don't know what the current state of the art is. If your code is better, maybe it should go upstream to be included with SCons itself.

three things: - A 10-lines python function that scans a folder and returns the list of files matching a given regexp. This list can then be given to the standard SCons builder to generate a lib or a program;

scons has something similar now with glob(). Was that your contribution? http://www.scons.org/doc/0.97.0d20071212/HTML/scons-api/index.html glob(self, pathname, ondisk=True, source=False, strings=False) Returns a list of Nodes (or strings) matching a specified pathname pattern. Pathname patterns follow UNIX shell semantics: * matches any-length strings of any characters, ? matches any character, and [] can enclose lists or ranges of characters. Matches do not span directory separators. The matches take into account Repositories, returning local Nodes if a corresponding entry exists in a Repository (either an in-memory Node or something on disk). By defafult, the glob() function matches entries that exist on-disk, in addition to in-memory Nodes. Setting the "ondisk" argument to False (or some other non-true value) causes the glob() function to only match in-memory Nodes. The default behavior is to return both the on-disk and in-memory Nodes. The "source" argument, when true, specifies that corresponding source Nodes must be returned if you're globbing in a build directory (initialized with BuildDir()). The default behavior is to return Nodes local to the BuildDir(). The "strings" argument, when true, returns the matches as strings, not Nodes. The strings are path names relative to this directory. The underlying algorithm is adapted from the glob.glob() function in the Python library (but heavily modified), and uses fnmatch() under the covers.
Jan 06 2008