www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Why CTFE I/O would be awesome: No-Build-Tool Custom Build Steps

reply "Nick Sabalausky" <a a.a> writes:
"Nick Sabalausky" <a a.a> wrote in message 
news:jkdho8$1p3a$1 digitalmars.com...
 "F i L" <witte2008 gmail.com> wrote in message
 news:sofbqiiyvragxjaxqbix forum.dlang.org...
 Kapps wrote:
 or even automatically creating bindings for a C header file.

I've thought about that before. It would be great to have fully automatic linking to C libs. You could even have a cache system that stores .di files and only regenerates them if the .h file was modified.

That hits on the one irritating limitation of CTFE: No I/O. So that cache system would be a no-go right now.

(I'm branching this off from "Three Unlikely Successful Features of D") At least one way I think CTFE I/O could be a really big thing: A library could include any custom build steps it needed (or at least pre-build steps) *in* the library itself without the library's user needing to deal with buildscripts/buildtools. You can *kinda* do it now, but it requires all the processing be done within the naturally-slow and memory-hungry CTFE interpreter (compared to just running already-compiled native code), and *everything* will be recomputed on every compile whether it needs to be or not. So it'd be more of a curiosity right now than a real professional approach. But with CTFE I/O, that can change, and I think that could be a very big thing: For example (and I apologize if this sounds like shameless promotion of my project - but it's a good example of the scenarios I have in mind), the typical expected workflow in most cases for Goldie Parsing System is like this: 1. Run 'goldie-grmc' to compile your grammar definition from the BNF-like GRM file to a CGT file ("compiled grammar tables" - ie, precomputed LALR and DFA tables). This can take some real non-trivial time for non-trivial grammars. 'Couse, I think I can still optimize it more, but the nature of LALR still prevents this step from scaling well even when natively compiled, let alone if it were to be run inside CTFE. 2. Run 'goldie-staticlang' on the CGT file to generate a D library custom-tailered for your grammar. 3. Compile your program (which imports the generated D files from 'goldie-static''). Obviously there's improvements I can still make, such allowing 'goldie-staticlang' to optionally do the job of 'goldie-grmc' (thus merging steps 1 and 2). Or, it should be possible to make 'goldie-staticlang' (minus any 'goldie-grmc' features) a CTFE-based library instead of an external tool, once some DMD bugs are fixed. But there's *still* a custom build step required, which you have to manually add into your buildscript, or even run manually. And while that build step could theoretically be made CTFE-able, it would be much less efficient, and would *considerably* slow down compile times for *every* compile. But: If CTFE had I/O, I'm pretty sure I could set things up so that *all* you'd need to do is step 3: import Goldie and compile your program. Then, inside Goldie, there would be some CTFE that checks if the CGT file is missing or out-of-date, and if so (and *only* if so), invokes 'goldie-grmc' on it automatically - which will run at *full* natively-compiled efficiency. Then it would do the same for 'goldie-staticlang' (although it should be reasonable to make 'goldie-staticlang' an CTFE-based library instead of an external tool, just with some existing DMD bugs fixed). And finally it would import your either-already-existing-or-freshly-rebuilt custom-tailered D library. Hell, it could even rebuild 'goldie-grmc' itself if it needed to! All CTFE needs is system(), file I/O, and date/time stuff, and then need for buildsystems could be heavily reduced and make external tools/libs easier to use. I can see that being a big selling point for D. That might even make it possible (though I haven't really thought it through) to implement the dependency-searching feature of RDMD as a library.
Mar 22 2012
next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 3/23/12, Nick Sabalausky <a a.a> wrote:
 A library could include any custom build steps it needed (or at least
 pre-build steps) *in* the library itself without the library's user needing
 to deal with buildscripts/buildtools.

Ahh, yes, I can already envision it. It's 2014, and a newbie just posted in D.learn: "Hey guys whats a great, flexible, and easy to use build tool??" And one of the oldtimers replies: "Why, just import std.build and write this: (inserts 2 lines of code)" :D
Mar 22 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-03-23 02:12, Nick Sabalausky wrote:
 (I'm branching this off from "Three Unlikely Successful Features of D")

 At least one way I think CTFE I/O could be a really big thing:

 A library could include any custom build steps it needed (or at least
 pre-build steps) *in* the library itself without the library's user needing
 to deal with buildscripts/buildtools. You can *kinda* do it now, but it
 requires all the processing be done within the naturally-slow and
 memory-hungry CTFE interpreter (compared to just running already-compiled
 native code), and *everything* will be recomputed on every compile whether
 it needs to be or not. So it'd be more of a curiosity right now than a real
 professional approach. But with CTFE I/O, that can change, and I think that
 could be a very big thing:

 For example (and I apologize if this sounds like shameless promotion of my
 project - but it's a good example of the scenarios I have in mind), the
 typical expected workflow in most cases for Goldie Parsing System is like
 this:

 1. Run 'goldie-grmc' to compile your grammar definition from the BNF-like
 GRM file to a CGT file ("compiled grammar tables" - ie, precomputed LALR and
 DFA tables). This can take some real non-trivial time for non-trivial
 grammars. 'Couse, I think I can still optimize it more, but the nature of
 LALR still prevents this step from scaling well even when natively compiled,
 let alone if it were to be run inside CTFE.

 2. Run 'goldie-staticlang' on the CGT file to generate a D library
 custom-tailered for your grammar.

 3. Compile your program (which imports the generated D files from
 'goldie-static'').

 Obviously there's improvements I can still make, such allowing
 'goldie-staticlang' to optionally do the job of 'goldie-grmc' (thus merging
 steps 1 and 2). Or, it should be possible to make 'goldie-staticlang' (minus
 any 'goldie-grmc' features) a CTFE-based library instead of an external
 tool, once some DMD bugs are fixed. But there's *still* a custom build step
 required, which you have to manually add into your buildscript, or even run
 manually. And while that build step could theoretically be made CTFE-able,
 it would be much less efficient, and would *considerably* slow down compile
 times for *every* compile.

 But:

 If CTFE had I/O, I'm pretty sure I could set things up so that *all* you'd
 need to do is step 3: import Goldie and compile your program. Then, inside
 Goldie, there would be some CTFE that checks if the CGT file is missing or
 out-of-date, and if so (and *only* if so), invokes 'goldie-grmc' on it
 automatically - which will run at *full* natively-compiled efficiency.  Then
 it would do the same for 'goldie-staticlang' (although it should be
 reasonable to make 'goldie-staticlang' an CTFE-based library instead of an
 external tool, just with some existing DMD bugs fixed). And finally it would
 import your either-already-existing-or-freshly-rebuilt custom-tailered D
 library. Hell, it could even rebuild 'goldie-grmc' itself if it needed to!

 All CTFE needs is system(), file I/O, and date/time stuff, and then need for
 buildsystems could be heavily reduced and make external tools/libs easier to
 use. I can see that being a big selling point for D.

 That might even make it possible (though I haven't really thought it
 through) to implement the dependency-searching feature of RDMD as a library.

Wouldn't you need to make sure DMD sees the correct D file before seeing the other files? -- /Jacob Carlborg
Mar 25 2012