www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Compile time executable calling?

reply "Tofu Ninja" <emmons0 purdue.edu> writes:
So I had an idea recently, wouldn't it be cool to have the 
ability to call an executable at compile time and capture its 
output. Something like the string imports but instead of opening 
and reading a text file, it run an executable, waits for it to 
finish, and grabs its output.

It would get really cool if you could pass this executable some 
args and then mix in its out put into your own code. It could be 
used similarly to how CTFE are used but with out the overhead of 
trying to compile that function and what not and with out the 
limitations on what it can do.

I could imagine all sorts of things that would be possible with 
this that is currently not.

Not sure if this is something that could be implemented easily, 
but seems like something that could be done and something that 
would be really cool.
Jul 12 2013
next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--047d7b676104bcbab204e15765fd
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Jul 12, 2013 at 1:42 PM, Tofu Ninja <emmons0 purdue.edu> wrote:

 So I had an idea recently, wouldn't it be cool to have the ability to call
 an executable at compile time and capture its output. Something like the
 string imports but instead of opening and reading a text file, it run an
 executable, waits for it to finish, and grabs its output.

 It would get really cool if you could pass this executable some args and
 then mix in its out put into your own code. It could be used similarly to
 how CTFE are used but with out the overhead of trying to compile that
 function and what not and with out the limitations on what it can do.

 I could imagine all sorts of things that would be possible with this that
 is currently not.

 Not sure if this is something that could be implemented easily, but seems
 like something that could be done and something that would be really cool.

I would definitely love that, and I don't see why this should be hard. It should be enabled by a compiler switch (dmd -enable_exec) for obvious safety reasons though (just as string import requires -J). This enables or simplifies a number of things: * ctfeWriteln becomes trivial (this feature has been asked a while ago) * CTFE in situations where current CTFE fails * do things that would temprarily require malloc * easily customize compilation process (user would be able to add logging / profile info without touching the dmd compiler) * accessing environment variables during compilation seems to me all that is needed is optionally redirecting stdin/stdout of a forked process inside dmd compilation, with the exec C function family. --047d7b676104bcbab204e15765fd Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Fri, Jul 12, 2013 at 1:42 PM, Tofu Ninja <span dir=3D"ltr">&lt;<a href= =3D"mailto:emmons0 purdue.edu" target=3D"_blank">emmons0 purdue.edu</a>&gt;= </span> wrote:<br><div class=3D"gmail_quote"><blockquote class=3D"gmail_quo= te" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"=

all an executable at compile time and capture its output. Something like th= e string imports but instead of opening and reading a text file, it run an = executable, waits for it to finish, and grabs its output.<br> <br> It would get really cool if you could pass this executable some args and th= en mix in its out put into your own code. It could be used similarly to how= CTFE are used but with out the overhead of trying to compile that function= and what not and with out the limitations on what it can do.<br> <br> I could imagine all sorts of things that would be possible with this that i= s currently not.<br> <br> Not sure if this is something that could be implemented easily, but seems l= ike something that could be done and something that would be really cool.<b= r> </blockquote></div><br><div>I would definitely love that, and I don&#39;t s= ee why this should be hard.</div><div>It should be enabled by a compiler sw= itch (dmd -enable_exec) for obvious safety reasons though (just as string i= mport requires -J).</div> <div><br></div><div>This enables or simplifies a number of things:</div><di= v><br></div><div>* ctfeWriteln becomes trivial (this feature has been asked= a while ago)</div><div>* CTFE in situations where current CTFE fails</div> <div>* do things that would temprarily require malloc</div><div>* easily cu= stomize compilation process (user would be able to add logging / profile in= fo without touching the dmd compiler)</div><div>* accessing environment var= iables during compilation</div> <div><br></div><div>seems to me all that is needed is optionally redirectin= g stdin/stdout of a forked process inside dmd compilation, with the exec C = function family.</div><div><br></div> --047d7b676104bcbab204e15765fd--
Jul 12 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/12/2013 1:42 PM, Tofu Ninja wrote:
 So I had an idea recently, wouldn't it be cool to have the ability to call an
 executable at compile time and capture its output. Something like the string
 imports but instead of opening and reading a text file, it run an executable,
 waits for it to finish, and grabs its output.

 It would get really cool if you could pass this executable some args and then
 mix in its out put into your own code. It could be used similarly to how CTFE
 are used but with out the overhead of trying to compile that function and what
 not and with out the limitations on what it can do.

 I could imagine all sorts of things that would be possible with this that is
 currently not.

 Not sure if this is something that could be implemented easily, but seems like
 something that could be done and something that would be really cool.

This is actually done in the makefile that builds DMD. A program (optabgen) is compiled and then run. Optabgen's output is several .c files which are then compiled into DMD's binary. The programs impcnvgen and idgen do similar things. It is a very powerful technique.
Jul 12 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/12/2013 3:12 PM, Timothee Cour wrote:
 I think the OP was refering to something different:
 ability to call an arbitrary executable / shell command during compile time of
a
 D function, whereas optabgen is during compiling dmd itself:

It's still the same idea - using external programs to generate source code.
 what we want to have is this:

I do understand that. I'm just saying that this can currently (but awkwardly) be done in the makefile.
Jul 12 2013
parent Paulo Pinto <pjmlp progtools.org> writes:
Am 13.07.2013 01:43, schrieb H. S. Teoh:
 On Fri, Jul 12, 2013 at 03:35:30PM -0700, Walter Bright wrote:
 On 7/12/2013 3:12 PM, Timothee Cour wrote:
 I think the OP was refering to something different: ability to call
 an arbitrary executable / shell command during compile time of a D
 function, whereas optabgen is during compiling dmd itself:

It's still the same idea - using external programs to generate source code.

This idea isn't new. lex/yacc (or their modern incarnations flex/bison) come to mind. The usage is a bit clunky, but the essence is the same.
 what we want to have is this:

I do understand that. I'm just saying that this can currently (but awkwardly) be done in the makefile.

At what point does the balance shift from having such an ability built-in, vs. just using OS-level facilities for combining different programs? For example, one could pipe source through a program that performs arbitrary transformations on it, then pipe the result through the compiler. Or one can write a program that generates arbitrary source code and pipe that into the compiler. Presently, such things are easily handled by a modern build system (of which makefiles are a rather clunky implementation thereof). T

In most modern build systems you can easily achieve that via plugins. Does DUB accept plugins? -- Paulo
Jul 13 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--089e013a100a9c61e904e157cef9
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Jul 12, 2013 at 2:54 PM, Walter Bright
<newshound2 digitalmars.com>wrote:

 On 7/12/2013 1:42 PM, Tofu Ninja wrote:

 So I had an idea recently, wouldn't it be cool to have the ability to
 call an
 executable at compile time and capture its output. Something like the
 string
 imports but instead of opening and reading a text file, it run an
 executable,
 waits for it to finish, and grabs its output.

 It would get really cool if you could pass this executable some args and
 then
 mix in its out put into your own code. It could be used similarly to how
 CTFE
 are used but with out the overhead of trying to compile that function and
 what
 not and with out the limitations on what it can do.

 I could imagine all sorts of things that would be possible with this that
 is
 currently not.

 Not sure if this is something that could be implemented easily, but seems
 like
 something that could be done and something that would be really cool.

This is actually done in the makefile that builds DMD. A program (optabgen) is compiled and then run. Optabgen's output is several .c files which are then compiled into DMD's binary. The programs impcnvgen and idgen do similar things. It is a very powerful technique.

I think the OP was refering to something different: ability to call an arbitrary executable / shell command during compile time of a D function, whereas optabgen is during compiling dmd itself: what we want to have is this: ---- import("some_source_file.txt"); //currently this is our only interaction with outside world during compile time import std.process; //or std.ctfeprocess ? string getStringAtCompileTime(string command){ if(!__ctfe) assert(0); //below is currently impossible, but would be great to have Tuple!(int,string,string) result=systemCaptureStdinStdout(command); //likewise without redirecting stdin/stdout assert(!result[0]); return result[1]; } void main(){ enum host = getStringAtCompileTime("env | grep HOST"); } ---- --089e013a100a9c61e904e157cef9 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Fri, Jul 12, 2013 at 2:54 PM, Walter Bright <span dir=3D"ltr">&lt;<a hre= f=3D"mailto:newshound2 digitalmars.com" target=3D"_blank">newshound2 digita= lmars.com</a>&gt;</span> wrote:<br><div class=3D"gmail_quote"><blockquote c= lass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;= padding-left:1ex"> <div class=3D"HOEnZb"><div class=3D"h5">On 7/12/2013 1:42 PM, Tofu Ninja wr= ote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> So I had an idea recently, wouldn&#39;t it be cool to have the ability to c= all an<br> executable at compile time and capture its output. Something like the strin= g<br> imports but instead of opening and reading a text file, it run an executabl= e,<br> waits for it to finish, and grabs its output.<br> <br> It would get really cool if you could pass this executable some args and th= en<br> mix in its out put into your own code. It could be used similarly to how CT= FE<br> are used but with out the overhead of trying to compile that function and w= hat<br> not and with out the limitations on what it can do.<br> <br> I could imagine all sorts of things that would be possible with this that i= s<br> currently not.<br> <br> Not sure if this is something that could be implemented easily, but seems l= ike<br> something that could be done and something that would be really cool.<br> </blockquote> <br></div></div> This is actually done in the makefile that builds DMD. A program (optabgen)= is compiled and then run. Optabgen&#39;s output is several .c files which = are then compiled into DMD&#39;s binary. The programs impcnvgen and idgen d= o similar things.<br> <br> It is a very powerful technique.<br> </blockquote></div><br><div>I think the OP was refering to something differ= ent:</div><div>ability to call an arbitrary executable / shell command duri= ng compile time of a D function, whereas optabgen is during compiling dmd i= tself:</div> <div><br></div><div>what we want to have is this:</div><div><br></div><div>= ----</div><div>import(&quot;some_source_file.txt&quot;); //currently this i= s our only interaction with outside world during compile time</div><div> import std.process; //or std.ctfeprocess ?=A0</div><div><br></div><div>stri= ng getStringAtCompileTime(string command){</div><div>=A0 if(!__ctfe) assert= (0);</div><div>=A0 //below is currently impossible, but would be great to h= ave</div> <div>=A0 Tuple!(int,string,string) result=3DsystemCaptureStdinStdout(comman= d); //likewise without redirecting stdin/stdout</div><div>=A0 assert(!resul= t[0]);</div><div>=A0 return result[1];</div><div>}</div><div><br></div><div=
void main(){</div>

</div><div>}</div><div><div>----</div></div><div><br></div><div><br></div><= div><br></div><div><br></div><div><br></div> --089e013a100a9c61e904e157cef9--
Jul 12 2013
prev sibling next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
 So I had an idea recently, wouldn't it be cool to have the 
 ability to call an executable at compile time and capture its 
 output.

How long until D compilers are able to read mail? :-) There's many obvious applications of this proposed feature, but I say such things should be delegated to the build process. Just run those executables using a makefile, or whatever build system you use. If you need compile time inputs from your D code then just run a separate build to extract them. Yes it's more work and less convenient, but I think we need to be careful with how much is added to the D compilers. We don't want to turn them into operating systems.
Jul 12 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/12/2013 5:00 PM, Timothee Cour wrote:
 Let's put it another way: if I or someone else made a pull request for CTFE
 "exec", would it have a chance of being accepted?

A big problem with it would be the equivalent of the "SQL Injection Exploit". Since the compiler can now execute arbitrary code, someone passing around malicious source code could do anything to your system.
Jul 12 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/12/2013 11:52 PM, BLM768 wrote:
 On Saturday, 13 July 2013 at 04:23:56 UTC, Walter Bright wrote:
 A big problem with it would be the equivalent of the "SQL Injection Exploit".
 Since the compiler can now execute arbitrary code, someone passing around
 malicious source code could do anything to your system.

Assuming that the user is compiling the code in order to run it (which does seem to be the most common case, at least in my experience), the user is already running arbitrary code. I don't really see how this would create a greater security risk than what already exists.

People can be endlessly creative at this kind of thing. I'm not at all sure you and I have thought of every possible exploit.
Jul 13 2013
next sibling parent Piotr Szturmaj <bncrbme jadamspam.pl> writes:
W dniu 13.07.2013 09:13, Walter Bright pisze:
 On 7/12/2013 11:52 PM, BLM768 wrote:
 On Saturday, 13 July 2013 at 04:23:56 UTC, Walter Bright wrote:
 A big problem with it would be the equivalent of the "SQL Injection
 Exploit".
 Since the compiler can now execute arbitrary code, someone passing
 around
 malicious source code could do anything to your system.

Assuming that the user is compiling the code in order to run it (which does seem to be the most common case, at least in my experience), the user is already running arbitrary code. I don't really see how this would create a greater security risk than what already exists.

People can be endlessly creative at this kind of thing. I'm not at all sure you and I have thought of every possible exploit.

Use sandboxing. On Linux it's easy: http://en.wikipedia.org/wiki/Seccomp. But, it could be difficult to create cross-platform solution.
Jul 13 2013
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/13/2013 1:34 AM, Tofu Ninja wrote:
 Any idea how difficult such a thing would be to implement? Any one willing to
 work on something like this? If something like this was made, would it be
included?

It would be easy to implement. I don't know if it should be included or not, though.
Jul 13 2013
prev sibling next sibling parent Justin Whear <justin economicmodeling.com> writes:
On Sat, 13 Jul 2013 00:36:21 +0200, Peter Alexander wrote:

 On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
 So I had an idea recently, wouldn't it be cool to have the ability to
 call an executable at compile time and capture its output.

How long until D compilers are able to read mail? :-) There's many obvious applications of this proposed feature, but I say such things should be delegated to the build process. Just run those executables using a makefile, or whatever build system you use. If you need compile time inputs from your D code then just run a separate build to extract them. Yes it's more work and less convenient, but I think we need to be careful with how much is added to the D compilers. We don't want to turn them into operating systems.

Other compilers allow symbols/macros to be defined on the command line, e.g. gcc's `-D` flag; it'd be nice for DMD to define a similar mechanism for defining enums, e.g. ---D code--- void main() { writeln(VERSION_NUMBER); } ------------ dmd main.d -DVERSION_NUMBER=1.2 Currently I have to use my makefiles to output a whole file which defines the various build constants.
Jul 12 2013
prev sibling next sibling parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Friday, 12 July 2013 at 22:36:22 UTC, Peter Alexander wrote:
 On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
 So I had an idea recently, wouldn't it be cool to have the 
 ability to call an executable at compile time and capture its 
 output.

How long until D compilers are able to read mail? :-) There's many obvious applications of this proposed feature, but I say such things should be delegated to the build process. Just run those executables using a makefile, or whatever build system you use. If you need compile time inputs from your D code then just run a separate build to extract them. Yes it's more work and less convenient, but I think we need to be careful with how much is added to the D compilers. We don't want to turn them into operating systems.

I think this is in line with what you want though, it allows the things that really should be separate to be separate in another executable, the only thing that it is really bring into the language is that you wont have to rely on a makefile as much, which in my opinion is a good thing.
Jul 12 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--089e01184c8c82df3204e158f16d
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Jul 12, 2013 at 3:36 PM, Peter Alexander <
peter.alexander.au gmail.com> wrote:

 On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:

 So I had an idea recently, wouldn't it be cool to have the ability to
 call an executable at compile time and capture its output.

How long until D compilers are able to read mail? :-) There's many obvious applications of this proposed feature, but I say such things should be delegated to the build process. Just run those executables using a makefile, or whatever build system you use. If you need compile time inputs from your D code then just run a separate build to extract them. Yes it's more work and less convenient, but I think we need to be careful with how much is added to the D compilers. We don't want to turn them into operating systems.

regarding added complexity: the only thing this adds is 1 function (calling an executable, with option to redirect stdin/out/err). And yes, that could read mail as you joked if the user called such a program inside his D function, but that would require ZERO change in compiler, apart from that ONE function to support calling external programs. we need this for the same reason we need CTFE: try using makefiles to achieve what CTFE does. Using a separate build is way less convenient and doesn't allow complex interactions, as it requires the process to be sequential: do other stuff THEN compile with dmd. Whereas integrating it inside compilation would allow interdependent computations offloaded to an external process. With makefile-like build, you'd have to have potentially many layers of interaction between compiling and calling external programs. Not only would that be much slower (dmd has overhead), but would also require parsing files each time. --089e01184c8c82df3204e158f16d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Fri, Jul 12, 2013 at 3:36 PM, Peter Alexander <span dir=3D"ltr">&lt;<a h= ref=3D"mailto:peter.alexander.au gmail.com" target=3D"_blank">peter.alexand= er.au gmail.com</a>&gt;</span> wrote:<br><div class=3D"gmail_quote"><blockq= uote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc = solid;padding-left:1ex"> <div class=3D"im">On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote= :<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> So I had an idea recently, wouldn&#39;t it be cool to have the ability to c= all an executable at compile time and capture its output.<br> </blockquote> <br></div> How long until D compilers are able to read mail? :-)<br> <br> There&#39;s many obvious applications of this proposed feature, but I say s= uch things should be delegated to the build process. Just run those executa= bles using a makefile, or whatever build system you use.<br> <br> If you need compile time inputs from your D code then just run a separate b= uild to extract them. Yes it&#39;s more work and less convenient, but I thi= nk we need to be careful with how much is added to the D compilers. We don&= #39;t want to turn them into operating systems.<br> </blockquote></div><div><br></div>regarding added complexity: the only thin= g this adds is 1 function (calling an executable, with option to redirect s= tdin/out/err). And yes, that could read mail as you joked if the user calle= d such a program inside his D function, but that would require ZERO change = in compiler, apart from that ONE function to support calling external progr= ams.=A0<div> <br></div><div>we need this for the same reason we need CTFE: try using mak= efiles to achieve what CTFE does. Using a separate build is way less conven= ient and doesn&#39;t allow complex interactions, as it requires the process= to be sequential: do other stuff THEN compile with dmd. Whereas integratin= g it inside compilation would allow interdependent computations offloaded t= o an external process.</div> <div><br></div><div>With makefile-like build, you&#39;d have to have potent= ially many layers of interaction between compiling and calling external pro= grams. Not only would that be much slower (dmd has overhead), but would als= o require parsing files each time.</div> <div><br></div><div><br></div> --089e01184c8c82df3204e158f16d--
Jul 12 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jul 12, 2013 at 03:35:30PM -0700, Walter Bright wrote:
 On 7/12/2013 3:12 PM, Timothee Cour wrote:
I think the OP was refering to something different: ability to call
an arbitrary executable / shell command during compile time of a D
function, whereas optabgen is during compiling dmd itself:

It's still the same idea - using external programs to generate source code.

This idea isn't new. lex/yacc (or their modern incarnations flex/bison) come to mind. The usage is a bit clunky, but the essence is the same.
what we want to have is this:

I do understand that. I'm just saying that this can currently (but awkwardly) be done in the makefile.

At what point does the balance shift from having such an ability built-in, vs. just using OS-level facilities for combining different programs? For example, one could pipe source through a program that performs arbitrary transformations on it, then pipe the result through the compiler. Or one can write a program that generates arbitrary source code and pipe that into the compiler. Presently, such things are easily handled by a modern build system (of which makefiles are a rather clunky implementation thereof). T -- This is not a sentence.
Jul 12 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Jul 13, 2013 at 12:51:03AM +0200, Tofu Ninja wrote:
 On Friday, 12 July 2013 at 22:36:22 UTC, Peter Alexander wrote:
On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
So I had an idea recently, wouldn't it be cool to have the
ability to call an executable at compile time and capture its
output.

How long until D compilers are able to read mail? :-)


Zawinski's law of software envelopment: Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can. :-)
There's many obvious applications of this proposed feature, but I
say such things should be delegated to the build process. Just run
those executables using a makefile, or whatever build system you
use.

If you need compile time inputs from your D code then just run a
separate build to extract them. Yes it's more work and less
convenient, but I think we need to be careful with how much is
added to the D compilers. We don't want to turn them into
operating systems.

I think this is in line with what you want though, it allows the things that really should be separate to be separate in another executable, the only thing that it is really bring into the language is that you wont have to rely on a makefile as much, which in my opinion is a good thing.

Makefiles are really a poor example of build systems. A proper build system can do such things in a much better, consistent way. Makefiles, for example, can't correctly recompile your program if you change compiler flags. And complex interdependencies, like you said, require painful manual maintenance, otherwise they tend to screw up and produce inconsistent results. For an example of a modern build system, see http://gittup.org/tup/. T -- Ph.D. = Permanent head Damage
Jul 12 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--089e01184c8cbc090c04e159501e
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Jul 12, 2013 at 4:51 PM, H. S. Teoh <hsteoh quickfur.ath.cx> wrote:

 On Sat, Jul 13, 2013 at 12:51:03AM +0200, Tofu Ninja wrote:
 On Friday, 12 July 2013 at 22:36:22 UTC, Peter Alexander wrote:
On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
So I had an idea recently, wouldn't it be cool to have the
ability to call an executable at compile time and capture its
output.

How long until D compilers are able to read mail? :-)


Zawinski's law of software envelopment: Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.

forget about reading mail, we currently don't even have hello world! (we have pragma(msg) but no ctfewriteln, which pragma(msg) can't do)
 :-)


There's many obvious applications of this proposed feature, but I
say such things should be delegated to the build process. Just run
those executables using a makefile, or whatever build system you
use.

If you need compile time inputs from your D code then just run a
separate build to extract them. Yes it's more work and less
convenient, but I think we need to be careful with how much is
added to the D compilers. We don't want to turn them into
operating systems.

I think this is in line with what you want though, it allows the things that really should be separate to be separate in another executable, the only thing that it is really bring into the language is that you wont have to rely on a makefile as much, which in my opinion is a good thing.

Makefiles are really a poor example of build systems. A proper build system can do such things in a much better, consistent way. Makefiles, for example, can't correctly recompile your program if you change compiler flags. And complex interdependencies, like you said, require painful manual maintenance, otherwise they tend to screw up and produce inconsistent results. For an example of a modern build system, see http://gittup.org/tup/.

dmd source code, which requires posix.ma / win32.mak / win64.mak which is not DRY and hard to maintain), and suffer from the points I mentioned in my previous post. We don't need yet another build system if we have "exec" available at CT. Let's put it another way: if I or someone else made a pull request for CTFE "exec", would it have a chance of being accepted? --089e01184c8cbc090c04e159501e Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <br><br><div class=3D"gmail_quote">On Fri, Jul 12, 2013 at 4:51 PM, H. S. T= eoh <span dir=3D"ltr">&lt;<a href=3D"mailto:hsteoh quickfur.ath.cx" target= =3D"_blank">hsteoh quickfur.ath.cx</a>&gt;</span> wrote:<br><blockquote cla= ss=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pa= dding-left:1ex"> <div class=3D"im">On Sat, Jul 13, 2013 at 12:51:03AM +0200, Tofu Ninja wrot= e:<br> &gt; On Friday, 12 July 2013 at 22:36:22 UTC, Peter Alexander wrote:<br> &gt; &gt;On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:<br> &gt; &gt;&gt;So I had an idea recently, wouldn&#39;t it be cool to have the= <br> &gt; &gt;&gt;ability to call an executable at compile time and capture its<= br> &gt; &gt;&gt;output.<br> &gt; &gt;<br> &gt; &gt;How long until D compilers are able to read mail? :-)<br> <br> </div>Zawinski&#39;s law of software envelopment:<br> <br> =A0 =A0 =A0 =A0 Every program attempts to expand until it can read mail. Th= ose<br> =A0 =A0 =A0 =A0 programs which cannot so expand are replaced by ones which = can.</blockquote><div><br></div><div>forget about reading mail, we currentl= y don&#39;t even have hello world! (we have pragma(msg) but no ctfewriteln,= which pragma(msg) can&#39;t do)=A0</div> <div><br></div><div>=A0</div><blockquote class=3D"gmail_quote" style=3D"mar= gin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class=3D"i= m">:-)<br> <br> <br> &gt; &gt;There&#39;s many obvious applications of this proposed feature, bu= t I<br> &gt; &gt;say such things should be delegated to the build process. Just run= <br> &gt; &gt;those executables using a makefile, or whatever build system you<b= r> &gt; &gt;use.<br> &gt; &gt;<br> &gt; &gt;If you need compile time inputs from your D code then just run a<b= r> &gt; &gt;separate build to extract them. Yes it&#39;s more work and less<br=

&gt; &gt;added to the D compilers. We don&#39;t want to turn them into<br> &gt; &gt;operating systems.<br> &gt;<br> &gt; I think this is in line with what you want though, it allows the<br> &gt; things that really should be separate to be separate in another<br> &gt; executable, the only thing that it is really bring into the language<b= r> &gt; is that you wont have to rely on a makefile as much, which in my<br> &gt; opinion is a good thing.<br> <br> </div>Makefiles are really a poor example of build systems. A proper build<= br> system can do such things in a much better, consistent way.<br> <br> Makefiles, for example, can&#39;t correctly recompile your program if you<b= r> change compiler flags. And complex interdependencies, like you said,<br> require painful manual maintenance, otherwise they tend to screw up and<br> produce inconsistent results.<br> <br> For an example of a modern build system, see <a href=3D"http://gittup.org/t= up/" target=3D"_blank">http://gittup.org/tup/</a>.<br><span class=3D"HOEnZb= "><font color=3D"#888888"><br></font></span></blockquote><div><br></div><di= v>Using a build system would likely be non-portable (see current mess even = in dmd source code, which requires <a href=3D"http://posix.ma">posix.ma</a>= / win32.mak / win64.mak which is not DRY and hard to maintain), and suffer= from the points I mentioned in my previous post.</div> <div><br></div><div><div>We don&#39;t need yet another build system if we h= ave &quot;exec&quot; available at CT.</div></div><div>=A0</div><div>Let&#39= ;s put it another way: if I or someone else made a pull request for CTFE &q= uot;exec&quot;, would it have a chance of being accepted?</div> <div><br></div></div><br> --089e01184c8cbc090c04e159501e--
Jul 12 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jul 12, 2013 at 05:00:29PM -0700, Timothee Cour wrote:
 On Fri, Jul 12, 2013 at 4:51 PM, H. S. Teoh <hsteoh quickfur.ath.cx> wrote:
 
 On Sat, Jul 13, 2013 at 12:51:03AM +0200, Tofu Ninja wrote:
 On Friday, 12 July 2013 at 22:36:22 UTC, Peter Alexander wrote:
On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
So I had an idea recently, wouldn't it be cool to have the
ability to call an executable at compile time and capture its
output.

How long until D compilers are able to read mail? :-)


Zawinski's law of software envelopment: Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.

forget about reading mail, we currently don't even have hello world! (we have pragma(msg) but no ctfewriteln, which pragma(msg) can't do)

Why should the compiler turn into a hello world program? [...]
 Makefiles, for example, can't correctly recompile your program if you
 change compiler flags. And complex interdependencies, like you said,
 require painful manual maintenance, otherwise they tend to screw up and
 produce inconsistent results.

 For an example of a modern build system, see http://gittup.org/tup/.

even in dmd source code, which requires posix.ma / win32.mak / win64.mak which is not DRY and hard to maintain), and suffer from the points I mentioned in my previous post.

Again, makefiles are a poor example of build systems. There *are* build systems that allow you to write portable build scripts, such as SCons. They're not perfect, but makefiles are a far cry from what they can do.
 We don't need yet another build system if we have "exec" available at
 CT.

That doesn't replace a proper build system. What if your external program itself is a D program that needs to be compiled first? What if you need to run an external program like povray in order to generate images (say, animation frames) that are string-imported into your program? Should DMD have the capability of running arbitrary programs that produce arbitrary output and be able to sort out which programs must run first and which output should be imported into the D program? So we're no longer talking about the D compiler, but about the D operating system. The real "DOS". :-P Which, mind you, would be a good thing IMO, but I think it's a bit of a stretch from where we are currently.
 Let's put it another way: if I or someone else made a pull request for
 CTFE "exec", would it have a chance of being accepted?

I'm not one to decide, you have to convince Walter. And I'm not saying this idea is good or bad, I'm just saying that you can already do such things without compiler support. At this point, I'd rather we focus on stabilizing the language and improving its implementation, than to continue adding new features that reinvent the wheel. T -- For every argument for something, there is always an equal and opposite argument against it. Debates don't give answers, only wounded or inflated egos.
Jul 12 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--089e01184c8c28aab404e159dd95
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Jul 12, 2013 at 5:10 PM, H. S. Teoh <hsteoh quickfur.ath.cx> wrote:

 On Fri, Jul 12, 2013 at 05:00:29PM -0700, Timothee Cour wrote:
 On Fri, Jul 12, 2013 at 4:51 PM, H. S. Teoh <hsteoh quickfur.ath.cx>

 On Sat, Jul 13, 2013 at 12:51:03AM +0200, Tofu Ninja wrote:
 On Friday, 12 July 2013 at 22:36:22 UTC, Peter Alexander wrote:
On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
So I had an idea recently, wouldn't it be cool to have the
ability to call an executable at compile time and capture its
output.

How long until D compilers are able to read mail? :-)


Zawinski's law of software envelopment: Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.

forget about reading mail, we currently don't even have hello world! (we have pragma(msg) but no ctfewriteln, which pragma(msg) can't do)

Why should the compiler turn into a hello world program?

For logging / debugging purposes, we often need to be able to print at compile time some variables, but ctfeWriteln doesn't even work. And again, pragma(msg) can't replace it.
 [...]
 Makefiles, for example, can't correctly recompile your program if you
 change compiler flags. And complex interdependencies, like you said,
 require painful manual maintenance, otherwise they tend to screw up and
 produce inconsistent results.

 For an example of a modern build system, see http://gittup.org/tup/.

even in dmd source code, which requires posix.ma / win32.mak / win64.mak which is not DRY and hard to maintain), and suffer from the points I mentioned in my previous post.

Again, makefiles are a poor example of build systems. There *are* build systems that allow you to write portable build scripts, such as SCons. They're not perfect, but makefiles are a far cry from what they can do.
 We don't need yet another build system if we have "exec" available at
 CT.

That doesn't replace a proper build system.

I wrote a D based makefile-like tool in D (which uses direct acyclic graph to find compilation order). It's much cleaner in D for obvious reasons.
 What if your external program itself is a D program that needs to be
 compiled first?

call exec("dmd mydeps.d"); (or exec("make -all") or the D based makefile)
 What if you need to run an external program like povray in order to
 generate
 images (say, animation frames) that are string-imported into your
 program?

I don't see how that's different: void main(){ enum s=exec("program generating images"); import(s); //use the string import //or even simply: use s directly, depending on conditions }
 Should DMD have the capability of running arbitrary programs
 that produce arbitrary output and be able to sort out which programs
 must run first and which output should be imported into the D program?

The order would be simply program order, as in the example above: "enum s=exec(...)" is run during CTFE before import(s), as it comes first.
 So we're no longer talking about the D compiler, but about the D
 operating system. The real "DOS". :-P Which, mind you, would be a good
 thing IMO, but I think it's a bit of a stretch from where we are
 currently.

No, all it requires in terms of dmd modification is CTFE exec. The user can use it to do complex stuff if he so wishes. CTFE already runs very complex program (involving std.algorithm and whatnot), so there's already a justification for running complex programs during CTFE. Allowing that one simple function will enable many more things.
 Let's put it another way: if I or someone else made a pull request for
 CTFE "exec", would it have a chance of being accepted?

I'm not one to decide, you have to convince Walter. And I'm not saying this idea is good or bad, I'm just saying that you can already do such things without compiler support.

we can't even print variables during CTFE. Using a build system to do that is way overkill. --089e01184c8c28aab404e159dd95 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Fri, Jul 12, 2013 at 5:10 PM, H. S. Teoh <span dir=3D"ltr">&lt;<a href= =3D"mailto:hsteoh quickfur.ath.cx" target=3D"_blank">hsteoh quickfur.ath.cx= </a>&gt;</span> wrote:<br><div class=3D"gmail_quote"><blockquote class=3D"g= mail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-l= eft:1ex"> <div class=3D"im">On Fri, Jul 12, 2013 at 05:00:29PM -0700, Timothee Cour w= rote:<br> &gt; On Fri, Jul 12, 2013 at 4:51 PM, H. S. Teoh &lt;<a href=3D"mailto:hste= oh quickfur.ath.cx">hsteoh quickfur.ath.cx</a>&gt; wrote:<br> &gt;<br> &gt; &gt; On Sat, Jul 13, 2013 at 12:51:03AM +0200, Tofu Ninja wrote:<br> &gt; &gt; &gt; On Friday, 12 July 2013 at 22:36:22 UTC, Peter Alexander wro= te:<br> &gt; &gt; &gt; &gt;On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrot= e:<br> &gt; &gt; &gt; &gt;&gt;So I had an idea recently, wouldn&#39;t it be cool t= o have the<br> &gt; &gt; &gt; &gt;&gt;ability to call an executable at compile time and ca= pture its<br> &gt; &gt; &gt; &gt;&gt;output.<br> &gt; &gt; &gt; &gt;<br> &gt; &gt; &gt; &gt;How long until D compilers are able to read mail? :-)<br=

&gt; &gt; Zawinski&#39;s law of software envelopment:<br> &gt; &gt;<br> &gt; &gt; =A0 =A0 =A0 =A0 Every program attempts to expand until it can rea= d mail. Those<br> &gt; &gt; =A0 =A0 =A0 =A0 programs which cannot so expand are replaced by o= nes which can.<br> &gt;<br> &gt;<br> &gt; forget about reading mail, we currently don&#39;t even have hello worl= d! (we<br> &gt; have pragma(msg) but no ctfewriteln, which pragma(msg) can&#39;t do)<b= r> <br> </div>Why should the compiler turn into a hello world program?<br></blockqu= ote><div><br></div><div><br></div><div>For logging / debugging purposes, we= often need to be able to print at compile time some variables, but ctfeWri= teln doesn&#39;t even work. And again, pragma(msg) can&#39;t replace it.</d= iv> <div><br></div><div>=A0</div><blockquote class=3D"gmail_quote" style=3D"mar= gin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> [...]<br> <div class=3D"im">&gt; &gt; Makefiles, for example, can&#39;t correctly rec= ompile your program if you<br> &gt; &gt; change compiler flags. And complex interdependencies, like you sa= id,<br> &gt; &gt; require painful manual maintenance, otherwise they tend to screw = up and<br> &gt; &gt; produce inconsistent results.<br> &gt; &gt;<br> &gt; &gt; For an example of a modern build system, see <a href=3D"http://gi= ttup.org/tup/" target=3D"_blank">http://gittup.org/tup/</a>.<br> &gt; &gt;<br> &gt; &gt;<br> &gt; Using a build system would likely be non-portable (see current mess<br=

rget=3D"_blank">posix.ma</a> / win32.mak /<br> &gt; win64.mak which is not DRY and hard to maintain), and suffer from the<= br> &gt; points I mentioned in my previous post.<br> <br> </div>Again, makefiles are a poor example of build systems. There *are* bui= ld<br> systems that allow you to write portable build scripts, such as SCons.<br> They&#39;re not perfect, but makefiles are a far cry from what they can do.= <br> <div class=3D"im"><br> <br> &gt; We don&#39;t need yet another build system if we have &quot;exec&quot;= available at<br> &gt; CT.<br> <br> </div>That doesn&#39;t replace a proper build system.</blockquote><div>=A0<= /div><div>I wrote a D based makefile-like tool in D (which uses direct acyc= lic graph to find compilation order). It&#39;s much cleaner in D for obviou= s reasons.</div> <div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;= border-left:1px #ccc solid;padding-left:1ex">What if your external=A0progra= m itself is a D program that needs to be compiled first?</blockquote><div><= br></div> <div>call exec(&quot;dmd mydeps.d&quot;); (or exec(&quot;make -all&quot;) o= r the D based makefile)</div><div>=A0</div><blockquote class=3D"gmail_quote= " style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> = What if=A0you need to run an external program like povray in order to gener= ate<br> images (say, animation frames) that are string-imported into your<br> program?</blockquote><div><br></div><div>I don&#39;t see how that&#39;s dif= ferent:</div><div>void main(){</div><div>enum s=3Dexec(&quot;program genera= ting images&quot;);</div><div>import(s);</div><div>//use the string import<= /div> <div>//or even simply: use s directly, depending on conditions</div><div>}<= /div><div><br></div><div>=A0</div><blockquote class=3D"gmail_quote" style= =3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Should = DMD have the capability of running arbitrary programs<br> that produce arbitrary output and be able to sort out which programs<br> must run first and which output should be imported into the D program?<br><= /blockquote><div><br></div><div>The order would be simply program order, as= in the example above: &quot;enum s=3Dexec(...)&quot; is run during CTFE be= fore import(s), as it comes first.</div> <div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;= border-left:1px #ccc solid;padding-left:1ex"> So we&#39;re no longer talking about the D compiler, but about the D<br> operating system. The real &quot;DOS&quot;. :-P Which, mind you, would be a= good<br> thing IMO, but I think it&#39;s a bit of a stretch from where we are<br> currently.<br></blockquote><div><br></div><div>No, all it requires in terms= of dmd modification is CTFE exec.=A0</div><div>The user can use it to do c= omplex stuff if he so wishes.=A0</div><div><br></div><div>CTFE already runs= very complex program (involving std.algorithm and whatnot), so there&#39;s= already a justification for running complex programs during CTFE. Allowing= that one simple function will enable many more things.=A0</div> <div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;= border-left:1px #ccc solid;padding-left:1ex"><div class=3D"im"> &gt; Let&#39;s put it another way: if I or someone else made a pull request= for<br> &gt; CTFE &quot;exec&quot;, would it have a chance of being accepted?<br> <br> </div>I&#39;m not one to decide, you have to convince Walter.<br> <br> And I&#39;m not saying this idea is good or bad, I&#39;m just saying that y= ou<br> can already do such things without compiler support. </blockquote><div><br>= </div><div>we can&#39;t even print variables during CTFE. Using a build sys= tem to do that is way overkill.</div><div><br></div><div>=A0</div></div> --089e01184c8c28aab404e159dd95--
Jul 12 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jul 12, 2013 at 05:39:45PM -0700, Timothee Cour wrote:
 On Fri, Jul 12, 2013 at 5:10 PM, H. S. Teoh <hsteoh quickfur.ath.cx> wrote:
 
 On Fri, Jul 12, 2013 at 05:00:29PM -0700, Timothee Cour wrote:
 On Fri, Jul 12, 2013 at 4:51 PM, H. S. Teoh <hsteoh quickfur.ath.cx>

 On Sat, Jul 13, 2013 at 12:51:03AM +0200, Tofu Ninja wrote:
 On Friday, 12 July 2013 at 22:36:22 UTC, Peter Alexander wrote:
On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
So I had an idea recently, wouldn't it be cool to have the
ability to call an executable at compile time and capture its
output.

How long until D compilers are able to read mail? :-)


Zawinski's law of software envelopment: Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.

forget about reading mail, we currently don't even have hello world! (we have pragma(msg) but no ctfewriteln, which pragma(msg) can't do)

Why should the compiler turn into a hello world program?

For logging / debugging purposes, we often need to be able to print at compile time some variables, but ctfeWriteln doesn't even work. And again, pragma(msg) can't replace it.

This is a bug / limitation in CTFE, not a justification for invoking arbitrary programs from the compiler. [...]
 We don't need yet another build system if we have "exec" available
 at CT.

That doesn't replace a proper build system.

I wrote a D based makefile-like tool in D (which uses direct acyclic graph to find compilation order). It's much cleaner in D for obvious reasons.

I'm interested. Is the code on github?
 What if your external program itself is a D program that needs to be
 compiled first?

call exec("dmd mydeps.d"); (or exec("make -all") or the D based makefile)

So basically you're saying DMD should be expanded into a D shell? [...]
 So we're no longer talking about the D compiler, but about the D
 operating system. The real "DOS". :-P Which, mind you, would be a good
 thing IMO, but I think it's a bit of a stretch from where we are
 currently.

No, all it requires in terms of dmd modification is CTFE exec. The user can use it to do complex stuff if he so wishes. CTFE already runs very complex program (involving std.algorithm and whatnot), so there's already a justification for running complex programs during CTFE. Allowing that one simple function will enable many more things.

In the old days, somebody wrote a 1-line universal C program that could do literally *anything*. All you have to do is to specify, in C syntax, what you want the program to do. The program's source code is: #include "/dev/tty" :-) (Sadly, this no longer works on modern systems.) Seriously, though, I think adding an exec function is total overkill for working around CTFE limitations. If that's what we're trying to solve, we should be looking at how to improve CTFE, not adding a nuclear warhead to dmd.
 Let's put it another way: if I or someone else made a pull request
 for CTFE "exec", would it have a chance of being accepted?

I'm not one to decide, you have to convince Walter. And I'm not saying this idea is good or bad, I'm just saying that you can already do such things without compiler support.

we can't even print variables during CTFE. Using a build system to do that is way overkill.

If that's the root problem, then we should fix/improve CTFE. I'm not convinced that this should be grounds for adding an exec function to dmd. It seems to be complete overkill just for working around CTFE limitations. But maybe Walter has a different opinion about this. :) T -- Let's call it an accidental feature. -- Larry Wall
Jul 12 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--e89a8fb1f522fb6bdc04e15af600
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Jul 12, 2013 at 6:14 PM, H. S. Teoh <hsteoh quickfur.ath.cx> wrote:

 On Fri, Jul 12, 2013 at 05:39:45PM -0700, Timothee Cour wrote:
 On Fri, Jul 12, 2013 at 5:10 PM, H. S. Teoh <hsteoh quickfur.ath.cx>

 On Fri, Jul 12, 2013 at 05:00:29PM -0700, Timothee Cour wrote:
 On Fri, Jul 12, 2013 at 4:51 PM, H. S. Teoh <hsteoh quickfur.ath.cx>

 On Sat, Jul 13, 2013 at 12:51:03AM +0200, Tofu Ninja wrote:
 On Friday, 12 July 2013 at 22:36:22 UTC, Peter Alexander wrote:
On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
So I had an idea recently, wouldn't it be cool to have the
ability to call an executable at compile time and capture its
output.

How long until D compilers are able to read mail? :-)


Zawinski's law of software envelopment: Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.

forget about reading mail, we currently don't even have hello world!



 have pragma(msg) but no ctfewriteln, which pragma(msg) can't do)

Why should the compiler turn into a hello world program?

For logging / debugging purposes, we often need to be able to print at compile time some variables, but ctfeWriteln doesn't even work. And again, pragma(msg) can't replace it.

This is a bug / limitation in CTFE, not a justification for invoking arbitrary programs from the compiler.

there would be so many other things to fix (what if I want to access environment variables? or know whether a program / library is available?). Whereas simply allowing CTFE exec unleashes arbitrarily complex use cases. The choice would be up to the user to make use of it or not. [...]
 We don't need yet another build system if we have "exec" available
 at CT.

That doesn't replace a proper build system.

I wrote a D based makefile-like tool in D (which uses direct acyclic graph to find compilation order). It's much cleaner in D for obvious reasons.

I'm interested. Is the code on github?

Not yet, code isn't very presentable at the moment but API is easy to use; user can mix both pure D and a makefile-like DSL syntax, so it makes simple things simple and complex things possible. It's DRY (nothing has to be repeated twice, which makes maintenance easy). I can push it to github if there's interest, but my point is that doing such things in D is easier than one might think.
 What if your external program itself is a D program that needs to be
 compiled first?

call exec("dmd mydeps.d"); (or exec("make -all") or the D based makefile)

So basically you're saying DMD should be expanded into a D shell?

No I never intended that. the call to exec("dmd...") would be a regular statement in a regular D file to be compiled. All I'm suggesting is making at least some of the functions in std.process CTFE-able.
 [...]
 So we're no longer talking about the D compiler, but about the D
 operating system. The real "DOS". :-P Which, mind you, would be a good
 thing IMO, but I think it's a bit of a stretch from where we are
 currently.

No, all it requires in terms of dmd modification is CTFE exec. The user can use it to do complex stuff if he so wishes. CTFE already runs very complex program (involving std.algorithm and whatnot), so there's already a justification for running complex programs during CTFE. Allowing that one simple function will enable many more things.

In the old days, somebody wrote a 1-line universal C program that could do literally *anything*. All you have to do is to specify, in C syntax, what you want the program to do. The program's source code is: #include "/dev/tty" :-) (Sadly, this no longer works on modern systems.) Seriously, though, I think adding an exec function is total overkill for working around CTFE limitations. If that's what we're trying to solve, we should be looking at how to improve CTFE, not adding a nuclear warhead to dmd.

I disagree. From the compiler implementation perspective, it's very easy (only requires making some of std.process CTFE-able), yet unleashes lots of potential use cases. Versus what you're suggesting which is to provide implementation for each individual CTFE limitation one at a time. Being lazy is good :-) A similar argument could be made for runtime: if we didn't have std.process, we'd have to reimplement or wrap "in D" all missing features. But thanks to std.process I can easily add many missing D functions, eg for readLink: ---- string realPath(string file){ import std.process; return runPython(`import os; a=os.path.realpath(`~file.escapeShellFileName~`); print(a);`); } string runPython(string code){ import std.process; string command=`python -c `~code.escapeShellFileName; return command.executeShell.outputThrows; } ---- bam, that was easy. I see no point in arbitrarily impose restrictions on future use cases.
 Let's put it another way: if I or someone else made a pull request
 for CTFE "exec", would it have a chance of being accepted?

I'm not one to decide, you have to convince Walter. And I'm not saying this idea is good or bad, I'm just saying that you can already do such things without compiler support.

we can't even print variables during CTFE. Using a build system to do that is way overkill.

If that's the root problem, then we should fix/improve CTFE. I'm not convinced that this should be grounds for adding an exec function to dmd. It seems to be complete overkill just for working around CTFE limitations. But maybe Walter has a different opinion about this. :)

ping, Walter ? --e89a8fb1f522fb6bdc04e15af600 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Fri, Jul 12, 2013 at 6:14 PM, H. S. Teoh <span dir=3D"ltr">&lt;<a href= =3D"mailto:hsteoh quickfur.ath.cx" target=3D"_blank">hsteoh quickfur.ath.cx= </a>&gt;</span> wrote:<br><div class=3D"gmail_quote"><blockquote class=3D"g= mail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-l= eft:1ex"> <div class=3D"im">On Fri, Jul 12, 2013 at 05:39:45PM -0700, Timothee Cour w= rote:<br> &gt; On Fri, Jul 12, 2013 at 5:10 PM, H. S. Teoh &lt;<a href=3D"mailto:hste= oh quickfur.ath.cx">hsteoh quickfur.ath.cx</a>&gt; wrote:<br> &gt;<br> &gt; &gt; On Fri, Jul 12, 2013 at 05:00:29PM -0700, Timothee Cour wrote:<br=

ailto:hsteoh quickfur.ath.cx">hsteoh quickfur.ath.cx</a>&gt;<br> &gt; &gt; wrote:<br> &gt; &gt; &gt;<br> &gt; &gt; &gt; &gt; On Sat, Jul 13, 2013 at 12:51:03AM +0200, Tofu Ninja wr= ote:<br> &gt; &gt; &gt; &gt; &gt; On Friday, 12 July 2013 at 22:36:22 UTC, Peter Ale= xander wrote:<br> &gt; &gt; &gt; &gt; &gt; &gt;On Friday, 12 July 2013 at 20:42:50 UTC, Tofu = Ninja wrote:<br> &gt; &gt; &gt; &gt; &gt; &gt;&gt;So I had an idea recently, wouldn&#39;t it= be cool to have the<br> &gt; &gt; &gt; &gt; &gt; &gt;&gt;ability to call an executable at compile t= ime and capture its<br> &gt; &gt; &gt; &gt; &gt; &gt;&gt;output.<br> &gt; &gt; &gt; &gt; &gt; &gt;<br> &gt; &gt; &gt; &gt; &gt; &gt;How long until D compilers are able to read ma= il? :-)<br> &gt; &gt; &gt; &gt;<br> &gt; &gt; &gt; &gt; Zawinski&#39;s law of software envelopment:<br> &gt; &gt; &gt; &gt;<br> &gt; &gt; &gt; &gt; =A0 =A0 =A0 =A0 Every program attempts to expand until = it can read mail.<br> &gt; &gt; &gt; &gt; =A0 =A0 =A0 =A0 Those programs which cannot so expand a= re replaced by<br> &gt; &gt; &gt; &gt; =A0 =A0 =A0 =A0 ones which can.<br> &gt; &gt; &gt;<br> &gt; &gt; &gt;<br> &gt; &gt; &gt; forget about reading mail, we currently don&#39;t even have = hello world! (we<br> &gt; &gt; &gt; have pragma(msg) but no ctfewriteln, which pragma(msg) can&#= 39;t do)<br> &gt; &gt;<br> &gt; &gt; Why should the compiler turn into a hello world program?<br> &gt; &gt;<br> &gt;<br> &gt;<br> &gt; For logging / debugging purposes, we often need to be able to print at= <br> &gt; compile time some variables, but ctfeWriteln doesn&#39;t even work. An= d<br> &gt; again, pragma(msg) can&#39;t replace it.<br> <br> </div>This is a bug / limitation in CTFE, not a justification for invoking<= br> arbitrary programs from the compiler.<br></blockquote><div><br></div><div>t= here would be so many other things to fix (what if I want to access environ= ment variables? or know whether a program / library is available?).=A0</div=

ses. The choice would be up to the user to make use of it or not.</div><div=
<br></div><div><br></div><blockquote class=3D"gmail_quote" style=3D"margin=

[...]<br> <div class=3D"im">&gt; &gt; &gt; We don&#39;t need yet another build system= if we have &quot;exec&quot; available<br> &gt; &gt; &gt; at CT.<br> &gt; &gt;<br> &gt; &gt; That doesn&#39;t replace a proper build system.<br> &gt;<br> &gt;<br> &gt; I wrote a D based makefile-like tool in D (which uses direct acyclic<b= r> &gt; graph to find compilation order). It&#39;s much cleaner in D for obvio= us<br> &gt; reasons.<br> <br> </div>I&#39;m interested. Is the code on github?</blockquote><div><br></div=
<div>Not yet, code isn&#39;t very presentable at the moment but API is eas=

kes simple things simple and complex things possible. It&#39;s DRY (nothing= has to be repeated twice, which makes maintenance easy). I can push it to = github if there&#39;s interest, but my point is that doing such things in D= is easier than one might think.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"><div class=3D"im"> &gt; &gt; What if your external program itself is a D program that needs to= be<br> &gt; &gt; compiled first?<br> &gt;<br> &gt;<br> &gt; call exec(&quot;dmd mydeps.d&quot;); (or exec(&quot;make -all&quot;) o= r the D based makefile)<br> <br> </div>So basically you&#39;re saying DMD should be expanded into a D shell?= <br></blockquote><div><br></div><div>No I never intended that. the call to = exec(&quot;dmd...&quot;) would be a regular statement in a regular D file t= o be compiled. All I&#39;m suggesting is making at least some of the functi= ons in std.process CTFE-able.</div> <div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;= border-left:1px #ccc solid;padding-left:1ex"> [...]<br> <div class=3D"im">&gt; &gt; So we&#39;re no longer talking about the D comp= iler, but about the D<br> &gt; &gt; operating system. The real &quot;DOS&quot;. :-P Which, mind you, = would be a good<br> &gt; &gt; thing IMO, but I think it&#39;s a bit of a stretch from where we = are<br> &gt; &gt; currently.<br> &gt; &gt;<br> &gt;<br> &gt; No, all it requires in terms of dmd modification is CTFE exec.<br> &gt; The user can use it to do complex stuff if he so wishes.<br> &gt;<br> &gt; CTFE already runs very complex program (involving std.algorithm and<br=

r> &gt; programs during CTFE. Allowing that one simple function will enable<br=

<br> </div>In the old days, somebody wrote a 1-line universal C program that cou= ld<br> do literally *anything*. All you have to do is to specify, in C syntax,<br> what you want the program to do. The program&#39;s source code is:<br> <br> =A0 =A0 =A0 =A0 #include &quot;/dev/tty&quot;<br> <br> :-) =A0(Sadly, this no longer works on modern systems.)<br> <br> Seriously, though, I think adding an exec function is total overkill for<br=

ve,<br> we should be looking at how to improve CTFE, not adding a nuclear<br> warhead to dmd.<br></blockquote><div><br></div><div>I disagree. From the co= mpiler implementation perspective, it&#39;s very easy (only requires making= some of std.process CTFE-able), yet unleashes lots of potential use cases.= Versus what you&#39;re suggesting which is to provide implementation for e= ach individual CTFE limitation one at a time.</div> <div><br></div><div>Being lazy is good :-)</div><div><br></div><div>A simil= ar argument could be made for runtime: if we didn&#39;t have std.process, w= e&#39;d have to reimplement or wrap &quot;in D&quot; all missing features.<= /div> <div>But thanks to std.process I can easily add many missing D functions, e= g for readLink:</div><div>----</div><div><div>string realPath(string file){= </div><div><span class=3D"Apple-tab-span" style=3D"white-space:pre"> </span=
import std.process;</div>

n runPython(`import os; a=3Dos.path.realpath(`~file.escapeShellFileName~`);= print(a);`);</div><div>}</div></div><div><div>string runPython(string code= ){</div> <div><span class=3D"Apple-tab-span" style=3D"white-space:pre"> </span>impor= t std.process;</div><div><span class=3D"Apple-tab-span" style=3D"white-spac= e:pre"> </span>string command=3D`python -c `~code.escapeShellFileName;</div=
<div>

mand.executeShell.outputThrows;</div><div>}</div></div><div><div>----</div>= <div><br></div><div>bam, that was easy.=A0</div><div><br></div><div>I see n= o point in arbitrarily impose restrictions on future use cases.</div> <div><br></div><div></div></div><div>=A0</div><blockquote class=3D"gmail_qu= ote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex= "><div class=3D"im"> &gt; &gt; &gt; Let&#39;s put it another way: if I or someone else made a pu= ll request<br> &gt; &gt; &gt; for CTFE &quot;exec&quot;, would it have a chance of being a= ccepted?<br> &gt; &gt;<br> &gt; &gt; I&#39;m not one to decide, you have to convince Walter.<br> &gt; &gt;<br> &gt; &gt; And I&#39;m not saying this idea is good or bad, I&#39;m just say= ing that<br> &gt; &gt; you can already do such things without compiler support.<br> &gt;<br> &gt;<br> &gt; we can&#39;t even print variables during CTFE. Using a build system to= do<br> &gt; that is way overkill.<br> <br> </div>If that&#39;s the root problem, then we should fix/improve CTFE. I&#3= 9;m not<br> convinced that this should be grounds for adding an exec function to<br> dmd. It seems to be complete overkill just for working around CTFE<br> limitations.<br> <br> But maybe Walter has a different opinion about this. :)<br></blockquote><di= v><br></div><div>ping, Walter ?</div><div><br></div><div>=A0</div></div> --e89a8fb1f522fb6bdc04e15af600--
Jul 12 2013
prev sibling next sibling parent "BLM768" <blm768 gmail.com> writes:
On Saturday, 13 July 2013 at 04:23:56 UTC, Walter Bright wrote:
 A big problem with it would be the equivalent of the "SQL 
 Injection Exploit". Since the compiler can now execute 
 arbitrary code, someone passing around malicious source code 
 could do anything to your system.

Assuming that the user is compiling the code in order to run it (which does seem to be the most common case, at least in my experience), the user is already running arbitrary code. I don't really see how this would create a greater security risk than what already exists. That said, I'm not completely sold on this idea, either, at least not while there are more important issues to solve. It could be nice at times, but I'm having a hard time coming up with a usage case where this would really be much more convenient than just using the build system.
Jul 12 2013
prev sibling next sibling parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Saturday, 13 July 2013 at 07:13:37 UTC, Walter Bright wrote:
 On 7/12/2013 11:52 PM, BLM768 wrote:
 On Saturday, 13 July 2013 at 04:23:56 UTC, Walter Bright wrote:
 A big problem with it would be the equivalent of the "SQL 
 Injection Exploit".
 Since the compiler can now execute arbitrary code, someone 
 passing around
 malicious source code could do anything to your system.

Assuming that the user is compiling the code in order to run it (which does seem to be the most common case, at least in my experience), the user is already running arbitrary code. I don't really see how this would create a greater security risk than what already exists.

People can be endlessly creative at this kind of thing. I'm not at all sure you and I have thought of every possible exploit.

Any idea how difficult such a thing would be to implement? Any one willing to work on something like this? If something like this was made, would it be included?
Jul 13 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Friday, 12 July 2013 at 23:34:05 UTC, Timothee Cour wrote:
 regarding added complexity: the only thing this adds is 1 
 function (calling
 an executable, with option to redirect stdin/out/err). And yes, 
 that could
 read mail as you joked if the user called such a program inside 
 his D
 function, but that would require ZERO change in compiler, apart 
 from that
 ONE function to support calling external programs.

It's one function now. Like CTFE, it won't be long before everyone is using it and we see its limitations, or we see more opportunities for further enhancement.
 we need this for the same reason we need CTFE: try using 
 makefiles to
 achieve what CTFE does. Using a separate build is way less 
 convenient and
 doesn't allow complex interactions, as it requires the process 
 to be
 sequential: do other stuff THEN compile with dmd. Whereas 
 integrating it
 inside compilation would allow interdependent computations 
 offloaded to an
 external process.

Let's be clear here: we don't "need" this we "want" this. There's an endless number of features we want, but we have to draw the line somewhere. This is classic feature creep. As Andrei said at DConf, we need to be more professional in the development of D. Endlessly adding features is not a professional approach to software development. In my opinion, the only features that should be added at this point are to fix language problems. For example, some form of copy constructors will be necessary to fix the const postblit problem. Adding features for convenience should be postponed for D3, or at the very least postponed until all major language issues are resolved.
Jul 13 2013
prev sibling next sibling parent "Daniel Kozak" <kozzi11 gmail.com> writes:
On Friday, 12 July 2013 at 22:46:39 UTC, Justin Whear wrote:
 On Sat, 13 Jul 2013 00:36:21 +0200, Peter Alexander wrote:

 On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
 So I had an idea recently, wouldn't it be cool to have the 
 ability to
 call an executable at compile time and capture its output.

How long until D compilers are able to read mail? :-) There's many obvious applications of this proposed feature, but I say such things should be delegated to the build process. Just run those executables using a makefile, or whatever build system you use. If you need compile time inputs from your D code then just run a separate build to extract them. Yes it's more work and less convenient, but I think we need to be careful with how much is added to the D compilers. We don't want to turn them into operating systems.

Other compilers allow symbols/macros to be defined on the command line, e.g. gcc's `-D` flag; it'd be nice for DMD to define a similar mechanism for defining enums, e.g. ---D code--- void main() { writeln(VERSION_NUMBER); } ------------ dmd main.d -DVERSION_NUMBER=1.2 Currently I have to use my makefiles to output a whole file which defines the various build constants.

A few minutes ago, I have a same issue. I need add some enum values as a compile option. Now I have something like this: version (Parser) { enum Application = "parser"; } version (Report) { enum Application = "report"; } But would be nice to have posibility to add same value as a compile parametr, so I could have code like this: enum Application = __traits(getParametrValue, "MY_APPLICATION_NAME"); and compile code with some args for eg.: -VMY_APPLICATION_NAME=parser
Jul 13 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 13 July 2013 at 11:19:49 UTC, Daniel Kozak wrote:
 A few minutes ago, I have a same issue. I need add some enum 
 values as a compile option.

 Now I have something like this:
 version (Parser) {
 	enum Application = "parser";
 }
 version (Report) {
 	enum Application = "report";
 }

 But would be nice to have posibility to add same value as a 
 compile parametr, so I could have code like this:

 enum Application = __traits(getParametrValue, 
 "MY_APPLICATION_NAME");

 and compile code with some args for eg.: 
 -VMY_APPLICATION_NAME=parser

You can solve this using import expressions. ---parser/config.include--- enum Application = "parser"; ---report/config.include--- enum Application = "report"; ---main.d--- mixin(import("config.include")); ---Makefile--- parser: dmd main.d -Jparser report: dmd main.d -Jreport
Jul 13 2013
prev sibling next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 7/12/13 5:42 PM, Tofu Ninja wrote:
 So I had an idea recently, wouldn't it be cool to have the ability to
 call an executable at compile time and capture its output. Something
 like the string imports but instead of opening and reading a text file,
 it run an executable, waits for it to finish, and grabs its output.

 It would get really cool if you could pass this executable some args and
 then mix in its out put into your own code. It could be used similarly
 to how CTFE are used but with out the overhead of trying to compile that
 function and what not and with out the limitations on what it can do.

 I could imagine all sorts of things that would be possible with this
 that is currently not.

 Not sure if this is something that could be implemented easily, but
 seems like something that could be done and something that would be
 really cool.

What would actually be cool is to run any kind of code at compile time (within a sandbox, if you want). For example reading a mysql database and generating classes for the tables at compile time. No need to run a separate executable and remember to run it before compiling your program. But this has been discussed many times in this list, and it won't happen, because compile time functions are not jitted and run, they are just interpreted. You ask for executing a program at compile time. Somebody will ask reading a file at compile time. Another one will ask reading a schema from a database. Yet another will want to do a web scraper at compile time to generate efficient code from that (not sure that's useful :-P)
Jul 13 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 07/13/2013 09:05 AM, Ary Borenszweig wrote:

 What would actually be cool is to run any kind of code at compile time
 (within a sandbox, if you want). For example reading a mysql database
 and generating classes for the tables at compile time. No need to run a
 separate executable and remember to run it before compiling your program.

Humans make mistakes all the time, especially with repetitive tasks. That's why such tasks are handled by tools like 'make'. Ali
Jul 15 2013
parent Ary Borenszweig <ary esperanto.org.ar> writes:
On 7/15/13 2:03 PM, Ali Çehreli wrote:
 On 07/13/2013 09:05 AM, Ary Borenszweig wrote:

  > What would actually be cool is to run any kind of code at compile time
  > (within a sandbox, if you want). For example reading a mysql database
  > and generating classes for the tables at compile time. No need to run a
  > separate executable and remember to run it before compiling your
 program.

 Humans make mistakes all the time, especially with repetitive tasks.
 That's why such tasks are handled by tools like 'make'.

 Ali

Why have CTFE at all if you can produce the desired code with an external program, output it to some file and then use it with import("file")? Just use 'make'. I believe a programming language should be as self-contained as possible for building an executable out of it. The more you can do within the language itself, minimizing external dependencies, the better. Because users need only to understand the language. They don't need to understand make, build systems and whatnot. But that's just my opinion...
Jul 15 2013
prev sibling next sibling parent "Daniel Kozak" <kozzi11 gmail.com> writes:
On Saturday, 13 July 2013 at 11:25:05 UTC, Peter Alexander wrote:
 On Saturday, 13 July 2013 at 11:19:49 UTC, Daniel Kozak wrote:
 A few minutes ago, I have a same issue. I need add some enum 
 values as a compile option.

 Now I have something like this:
 version (Parser) {
 	enum Application = "parser";
 }
 version (Report) {
 	enum Application = "report";
 }

 But would be nice to have posibility to add same value as a 
 compile parametr, so I could have code like this:

 enum Application = __traits(getParametrValue, 
 "MY_APPLICATION_NAME");

 and compile code with some args for eg.: 
 -VMY_APPLICATION_NAME=parser

You can solve this using import expressions. ---parser/config.include--- enum Application = "parser"; ---report/config.include--- enum Application = "report"; ---main.d--- mixin(import("config.include")); ---Makefile--- parser: dmd main.d -Jparser report: dmd main.d -Jreport

Jul 13 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--001a1133050ca6571d04e182bf7d
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Jul 12, 2013 at 9:23 PM, Walter Bright
<newshound2 digitalmars.com>wrote:

 On 7/12/2013 5:00 PM, Timothee Cour wrote:

 Let's put it another way: if I or someone else made a pull request for
 CTFE
 "exec", would it have a chance of being accepted?

A big problem with it would be the equivalent of the "SQL Injection Exploit". Since the compiler can now execute arbitrary code, someone passing around malicious source code could do anything to your system.

by a compiler switch (eg: dmd -enable_exec) for safety reasons (just as string import requires -J). At the very least it is useful for debugging personal projects. --001a1133050ca6571d04e182bf7d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Fri, Jul 12, 2013 at 9:23 PM, Walter Bright <span dir=3D"ltr">&lt;<a hre= f=3D"mailto:newshound2 digitalmars.com" target=3D"_blank">newshound2 digita= lmars.com</a>&gt;</span> wrote:<br><div class=3D"gmail_quote"><blockquote c= lass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;= padding-left:1ex"> <div class=3D"im">On 7/12/2013 5:00 PM, Timothee Cour wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Let&#39;s put it another way: if I or someone else made a pull request for = CTFE<br> &quot;exec&quot;, would it have a chance of being accepted?<br> </blockquote> <br></div> A big problem with it would be the equivalent of the &quot;SQL Injection Ex= ploit&quot;. Since the compiler can now execute arbitrary code, someone pas= sing around malicious source code could do anything to your system.<br> <br> </blockquote></div><div><br></div>Which is why I suggested in my first post= above that i<span style=3D"background-color:rgb(255,255,255);color:rgb(34,= 34,34);font-family:arial,sans-serif;font-size:13px">t should be enabled by = a compiler switch (eg: dmd -enable_exec) for safety reasons (just as string= import requires -J).</span><div> At the very least it is useful for debugging personal projects.</div><div><= br></div> --001a1133050ca6571d04e182bf7d--
Jul 14 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--089e01184c8cedcb5f04e182c487
Content-Type: text/plain; charset=ISO-8859-1

On Sat, Jul 13, 2013 at 9:05 AM, Ary Borenszweig <ary esperanto.org.ar>wrote:

 On 7/12/13 5:42 PM, Tofu Ninja wrote:

 So I had an idea recently, wouldn't it be cool to have the ability to
 call an executable at compile time and capture its output. Something
 like the string imports but instead of opening and reading a text file,
 it run an executable, waits for it to finish, and grabs its output.

 It would get really cool if you could pass this executable some args and
 then mix in its out put into your own code. It could be used similarly
 to how CTFE are used but with out the overhead of trying to compile that
 function and what not and with out the limitations on what it can do.

 I could imagine all sorts of things that would be possible with this
 that is currently not.

 Not sure if this is something that could be implemented easily, but
 seems like something that could be done and something that would be
 really cool.

What would actually be cool is to run any kind of code at compile time (within a sandbox, if you want). For example reading a mysql database and generating classes for the tables at compile time. No need to run a separate executable and remember to run it before compiling your program. But this has been discussed many times in this list, and it won't happen, because compile time functions are not jitted and run, they are just interpreted. You ask for executing a program at compile time. Somebody will ask reading a file at compile time. Another one will ask reading a schema from a database. Yet another will want to do a web scraper at compile time to generate efficient code from that (not sure that's useful :-P)

reading a file at compile time is already possible with string import. --089e01184c8cedcb5f04e182c487 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <br><br><div class=3D"gmail_quote">On Sat, Jul 13, 2013 at 9:05 AM, Ary Bor= enszweig <span dir=3D"ltr">&lt;<a href=3D"mailto:ary esperanto.org.ar" targ= et=3D"_blank">ary esperanto.org.ar</a>&gt;</span> wrote:<br><blockquote cla= ss=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pa= dding-left:1ex"> <div class=3D"HOEnZb"><div class=3D"h5">On 7/12/13 5:42 PM, Tofu Ninja wrot= e:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> So I had an idea recently, wouldn&#39;t it be cool to have the ability to<b= r> call an executable at compile time and capture its output. Something<br> like the string imports but instead of opening and reading a text file,<br> it run an executable, waits for it to finish, and grabs its output.<br> <br> It would get really cool if you could pass this executable some args and<br=

to how CTFE are used but with out the overhead of trying to compile that<br=

<br> I could imagine all sorts of things that would be possible with this<br> that is currently not.<br> <br> Not sure if this is something that could be implemented easily, but<br> seems like something that could be done and something that would be<br> really cool.<br> </blockquote> <br></div></div> What would actually be cool is to run any kind of code at compile time (wit= hin a sandbox, if you want). For example reading a mysql database and gener= ating classes for the tables at compile time. No need to run a separate exe= cutable and remember to run it before compiling your program.<br> <br> But this has been discussed many times in this list, and it won&#39;t happe= n, because compile time functions are not jitted and run, they are just int= erpreted.<br> <br> You ask for executing a program at compile time. Somebody will ask reading = a file at compile time. Another one will ask reading a schema from a databa= se. Yet another will want to do a web scraper at compile time to generate e= fficient code from that (not sure that&#39;s useful :-P)<br> </blockquote><div><br></div><div>reading a file at compile time is already = possible with string import.</div><div>=A0</div></div><br> --089e01184c8cedcb5f04e182c487--
Jul 14 2013
prev sibling next sibling parent reply "Don" <prosthetictelevisions teletubby.medical.com> writes:
On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
 So I had an idea recently, wouldn't it be cool to have the 
 ability to call an executable at compile time and capture its 
 output. Something like the string imports but instead of 
 opening and reading a text file, it run an executable, waits 
 for it to finish, and grabs its output.

 It would get really cool if you could pass this executable some 
 args and then mix in its out put into your own code. It could 
 be used similarly to how CTFE are used but with out the 
 overhead of trying to compile that function and what not and 
 with out the limitations on what it can do.

 I could imagine all sorts of things that would be possible with 
 this that is currently not.

 Not sure if this is something that could be implemented easily, 
 but seems like something that could be done and something that 
 would be really cool.

I personally think it's a *horrible* idea. It's one of those things which looks good in small cases but doesn't scale. Don't underestimate the fact that the compiler is a deterministic program at present. When compiled with the same flags, on the same set of source files, the results should always be the same. It's a classic filter program. input -> compiler -> output Losing that purity is a HUGE thing to give up.
Jul 15 2013
next sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 07/15/2013 08:05 AM, Tofu Ninja wrote:

 We should be as flexible as possible and leave the decision up to the
 programmer. We shouldn't restrict a feature based on a fictional idea of
 purity or vague worry of feature bloat(as others have mentioned)

My worry is not vague at all. What stops one from adding emailing ability to the compiler then? That would be immensely useful too becaues the compiler could email me. (!) Where is the line? A compiler is a tool that compiles source code in a preset environment. Simple is better.
 especially in a case like this where the implementation is fairly simple
 and the use cases are immense.

The use cases are not immense compared to the current situation. There are lots of free tools that can set the environment for compilation and the compiler compiles. Ali
Jul 15 2013
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/15/13 6:26 AM, Don wrote:
 On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
 So I had an idea recently, wouldn't it be cool to have the ability to
 call an executable at compile time and capture its output. Something
 like the string imports but instead of opening and reading a text
 file, it run an executable, waits for it to finish, and grabs its output.

 It would get really cool if you could pass this executable some args
 and then mix in its out put into your own code. It could be used
 similarly to how CTFE are used but with out the overhead of trying to
 compile that function and what not and with out the limitations on
 what it can do.

 I could imagine all sorts of things that would be possible with this
 that is currently not.

 Not sure if this is something that could be implemented easily, but
 seems like something that could be done and something that would be
 really cool.

I personally think it's a *horrible* idea. It's one of those things which looks good in small cases but doesn't scale.

I think the right direction here is to factor compiler parts as a library. Then user code may insert external artifacts (notably REPL) on its own. Andrei
Jul 15 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-07-16 06:10, Andrei Alexandrescu wrote:

 I think the right direction here is to factor compiler parts as a
 library.

Yes, I agree. I've been wanting this for a long time. -- /Jacob Carlborg
Jul 17 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-07-17 16:55, H. S. Teoh wrote:

 We've been talking about this for a while. Have we made any progress in
 this direction since?

There's been a bunch of pull requests merged related to adopting the DMD source code to make it work with the tool, I think it was, Daniel Murphy has created. This tool will be used to automatically port DMD to D. It's designed specifically for the DMD code base. Look for pull request tagged with [DDMD] in the title. Then I think the idea is refactor DMD to make it usable as a library. -- /Jacob Carlborg
Jul 17 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-07-18 00:15, H. S. Teoh wrote:

 Wow. That sounds like it's still a long ways off. But at least we're
 making progress.

Oh, forgot to mention. There are several other people here that are working on implementing a D front end. These are mostly lexers. There's one in the review queue, std.d.lexer, by Brian Schott: http://wiki.dlang.org/Review_Queue -- /Jacob Carlborg
Jul 17 2013
prev sibling next sibling parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Monday, 15 July 2013 at 13:26:13 UTC, Don wrote:
 On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
 So I had an idea recently, wouldn't it be cool to have the 
 ability to call an executable at compile time and capture its 
 output. Something like the string imports but instead of 
 opening and reading a text file, it run an executable, waits 
 for it to finish, and grabs its output.

 It would get really cool if you could pass this executable 
 some args and then mix in its out put into your own code. It 
 could be used similarly to how CTFE are used but with out the 
 overhead of trying to compile that function and what not and 
 with out the limitations on what it can do.

 I could imagine all sorts of things that would be possible 
 with this that is currently not.

 Not sure if this is something that could be implemented 
 easily, but seems like something that could be done and 
 something that would be really cool.

I personally think it's a *horrible* idea. It's one of those things which looks good in small cases but doesn't scale. Don't underestimate the fact that the compiler is a deterministic program at present. When compiled with the same flags, on the same set of source files, the results should always be the same. It's a classic filter program. input -> compiler -> output Losing that purity is a HUGE thing to give up.

Ehh... to say its deterministic is kinda a stretch, for one, compiling on different systems could result in different output with the ability to version different sections of code. Also as has been mentioned before, string imports already make the same kind of break in purity. If the programmer wants to use it then they also should be well aware of the consequences of such a decision, which can be considered on a case by case basis. We should be as flexible as possible and leave the decision up to the programmer. We shouldn't restrict a feature based on a fictional idea of purity or vague worry of feature bloat(as others have mentioned) especially in a case like this where the implementation is fairly simple and the use cases are immense.
Jul 15 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Monday, 15 July 2013 at 15:05:39 UTC, Tofu Ninja wrote:
 On Monday, 15 July 2013 at 13:26:13 UTC, Don wrote:
 Don't underestimate the fact that the compiler is a 
 deterministic program at present. When compiled with the same 
 flags, on the same set of source files, the results should 
 always be the same. It's a classic filter program.

 input -> compiler -> output

 Losing that purity is a HUGE thing to give up.

Ehh... to say its deterministic is kinda a stretch, for one, compiling on different systems could result in different output with the ability to version different sections of code. Also as has been mentioned before, string imports already make the same kind of break in purity.

If you want to be technical then yes, the purity takes the operating system as an input. String imports don't break purity, they just add an input. By allowing arbitrary code execution, you essentially require the whole state of the host machine as an input.
 If the programmer wants to use it then they also should be well 
 aware of the consequences of such a decision, which can be 
 considered on a case by case basis.

 We should be as flexible as possible and leave the decision up 
 to the programmer. We shouldn't restrict a feature based on a 
 fictional idea of purity or vague worry of feature bloat(as 
 others have mentioned) especially in a case like this where the 
 implementation is fairly simple and the use cases are immense.

No, this is a bad attitude to have. If you add a feature that is easy to misuse then it WILL be misused. Just looks at macros in C. Any program that uses compile time executable calling will almost certainly become non-portable. This is not a good thing.
Jul 15 2013
prev sibling next sibling parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Monday, 15 July 2013 at 17:08:48 UTC, Ali Çehreli wrote:
 On 07/15/2013 08:05 AM, Tofu Ninja wrote:

 We should be as flexible as possible and leave the decision

 programmer. We shouldn't restrict a feature based on a

 purity or vague worry of feature bloat(as others have

My worry is not vague at all. What stops one from adding emailing ability to the compiler then? That would be immensely useful too becaues the compiler could email me. (!) Where is the line? A compiler is a tool that compiles source code in a preset environment. Simple is better.
 especially in a case like this where the implementation is

 and the use cases are immense.

The use cases are not immense compared to the current situation. There are lots of free tools that can set the environment for compilation and the compiler compiles. Ali

The use cases I am more interested in are not possible with make. Having the ability to pass the arguments from within the language itself allows you to define your use cases inline instead of having to separately define the use case outside. Something where you would have many different points in the code that needed different input from what ever executable you were calling. With out ctec you would have to maintain a list of every time you needed something generated from the outside and add that to your make which is error prone. I think it is in these cases where the ctec would be the most useful as it would remove a lot of the burden of keeping track of all the pregenerated code.
Jul 15 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Jul 15, 2013 at 02:06:47PM -0300, Ary Borenszweig wrote:
 On 7/15/13 2:03 PM, Ali ehreli wrote:
On 07/13/2013 09:05 AM, Ary Borenszweig wrote:

 What would actually be cool is to run any kind of code at compile
 time (within a sandbox, if you want). For example reading a mysql
 database and generating classes for the tables at compile time. No
 need to run a separate executable and remember to run it before
 compiling your program.

Humans make mistakes all the time, especially with repetitive tasks. That's why such tasks are handled by tools like 'make'. Ali

Why have CTFE at all if you can produce the desired code with an external program, output it to some file and then use it with import("file")? Just use 'make'.

That's an invalid argument. CTFE allows compile-time introspection of types and declarations within the language. To move this outside the compiler would require essentially re-implementing the compiler so that you can parse your code to extract types, declarations, etc., in order to perform CTFE. This means it's better for it to be in the compiler. OTOH, executing arbitrary programs has nothing to do with the compiling of D source code. That belongs to the OS level of operations, not inside the compiler.
 I believe a programming language should be as self-contained as
 possible for building an executable out of it. The more you can do
 within the language itself, minimizing external dependencies, the
 better. Because users need only to understand the language. They don't
 need to understand make, build systems and whatnot.

I don't understand this reluctance to learn how to use build systems. Why insist on building a house with only a hammer, and refuse to learn how to use a screwdriver and wrench? Sure, you *can* build a house with only a hammer if you try hard enough, but why not use tools that already exist? (Of course, the problem is exacerbated by make being a really poor example of a build system, but also the one most people know about. Perhaps that explains part of the bad rep of build systems.) T -- What are you when you run out of Monet? Baroque.
Jul 15 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Jul 15, 2013 at 07:35:44PM +0200, Tofu Ninja wrote:
[...]
 The use cases I am more interested in are not possible with make.
 Having the ability to pass the arguments from within the language
 itself allows you to define your use cases inline instead of having
 to separately define the use case outside. Something where you would
 have many different points in the code that needed different input
 from what ever executable you were calling. With out ctec you would
 have to maintain a list of every time you needed something generated
 from the outside and add that to your make which is error prone. I
 think it is in these cases where the ctec would be the most useful
 as it would remove a lot of the burden of keeping track of all the
 pregenerated code.

You mean, you want to do something like this? static inputValues = [ "data1", "data2", "data3" ]; static foreach (val; inputValues) { import exec("prog", val); } I can see why that could be useful. T -- By understanding a machine-oriented language, the programmer will tend to use a much more efficient method; it is much closer to reality. -- D. Knuth
Jul 15 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--047d7b4725327b091304e1907edc
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Mon, Jul 15, 2013 at 10:35 AM, Tofu Ninja <emmons0 purdue.edu> wrote:

 On Monday, 15 July 2013 at 17:08:48 UTC, Ali =C7ehreli wrote:

 On 07/15/2013 08:05 AM, Tofu Ninja wrote:

 We should be as flexible as possible and leave the decision

 programmer. We shouldn't restrict a feature based on a

 purity or vague worry of feature bloat(as others have

My worry is not vague at all. What stops one from adding emailing abilit=


 to the compiler then? That would be immensely useful too becaues the
 compiler could email me. (!) Where is the line?

 A compiler is a tool that compiles source code in a preset environment.
 Simple is better.

 especially in a case like this where the implementation is

 and the use cases are immense.

The use cases are not immense compared to the current situation. There are lots of free tools that can set the environment for compilation and =


 compiler compiles.

 Ali

The use cases I am more interested in are not possible with make. Having the ability to pass the arguments from within the language itself allows you to define your use cases inline instead of having to separately defin=

 the use case outside. Something where you would have many different point=

 in the code that needed different input from what ever executable you wer=

 calling. With out ctec you would have to maintain a list of every time yo=

 needed something generated from the outside and add that to your make whi=

 is error prone. I think it is in these cases where the ctec would be the
 most useful as it would remove a lot of the burden of keeping track of al=

 the pregenerated code.

for it. People can test it individually and see whether it creates havoc or enables interesting scenarios. Then we can decide after some time. --047d7b4725327b091304e1907edc Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Mon, Jul 15, 2013 at 10:35 AM, Tofu Ninja <span dir=3D"ltr">&lt;<a href= =3D"mailto:emmons0 purdue.edu" target=3D"_blank">emmons0 purdue.edu</a>&gt;= </span> wrote:<br><div class=3D"gmail_quote"><blockquote class=3D"gmail_quo= te" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"=

UTC, Ali =C7ehreli wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On 07/15/2013 08:05 AM, Tofu Ninja wrote:<br> <br> &gt; We should be as flexible as possible and leave the decision<br> up to the<br> &gt; programmer. We shouldn&#39;t restrict a feature based on a<br> fictional idea of<br> &gt; purity or vague worry of feature bloat(as others have<br> mentioned)<br> <br> My worry is not vague at all. What stops one from adding emailing ability t= o the compiler then? That would be immensely useful too becaues the compile= r could email me. (!) Where is the line?<br> <br> A compiler is a tool that compiles source code in a preset environment. Sim= ple is better.<br> <br> &gt; especially in a case like this where the implementation is<br> fairly simple<br> &gt; and the use cases are immense.<br> <br> The use cases are not immense compared to the current situation. There are = lots of free tools that can set the environment for compilation and the com= piler compiles.<br> <br> Ali<br> </blockquote> <br></div></div> The use cases I am more interested in are not possible with make. Having th= e ability to pass the arguments from within the language itself allows you = to define your use cases inline instead of having to separately define the = use case outside. Something where you would have many different points in t= he code that needed different input from what ever executable you were call= ing. With out ctec you would have to maintain a list of every time you need= ed something generated from the outside and add that to your make which is = error prone. I think it is in these cases where the ctec would be the most = useful as it would remove a lot of the burden of keeping track of all the p= regenerated code.<br> <br> </blockquote></div><br><div>Instead of debating forever we should implement= it and make a pull request for it. People can test it individually and see= whether it creates havoc or enables interesting scenarios. Then we can dec= ide after some time.</div> <div>=A0</div> --047d7b4725327b091304e1907edc--
Jul 15 2013
prev sibling next sibling parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Monday, 15 July 2013 at 17:49:04 UTC, H. S. Teoh wrote:
 On Mon, Jul 15, 2013 at 07:35:44PM +0200, Tofu Ninja wrote:
 [...]
 The use cases I am more interested in are not possible with 
 make.
 Having the ability to pass the arguments from within the 
 language
 itself allows you to define your use cases inline instead of 
 having
 to separately define the use case outside. Something where you 
 would
 have many different points in the code that needed different 
 input
 from what ever executable you were calling. With out ctec you 
 would
 have to maintain a list of every time you needed something 
 generated
 from the outside and add that to your make which is error 
 prone. I
 think it is in these cases where the ctec would be the most 
 useful
 as it would remove a lot of the burden of keeping track of all 
 the
 pregenerated code.

You mean, you want to do something like this? static inputValues = [ "data1", "data2", "data3" ]; static foreach (val; inputValues) { import exec("prog", val); } I can see why that could be useful. T

based on the output of an executable, so in that case the template args would be passed into the executable call.
Jul 15 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Jul 15, 2013 at 07:54:56PM +0200, Tofu Ninja wrote:
 On Monday, 15 July 2013 at 17:49:04 UTC, H. S. Teoh wrote:
On Mon, Jul 15, 2013 at 07:35:44PM +0200, Tofu Ninja wrote:
[...]
The use cases I am more interested in are not possible with make.
Having the ability to pass the arguments from within the language
itself allows you to define your use cases inline instead of having
to separately define the use case outside. Something where you would
have many different points in the code that needed different input
from what ever executable you were calling. With out ctec you would
have to maintain a list of every time you needed something generated
from the outside and add that to your make which is error prone.  I
think it is in these cases where the ctec would be the most useful
as it would remove a lot of the burden of keeping track of all the
pregenerated code.

You mean, you want to do something like this? static inputValues = [ "data1", "data2", "data3" ]; static foreach (val; inputValues) { import exec("prog", val); } I can see why that could be useful. T

on the output of an executable, so in that case the template args would be passed into the executable call.

Hmm. Let's see: module dmd; static grammar = import exec("wget", "http://dlang.org/spec"); static c_code = import exec("yacc", grammar); void main() { import exec("d-ify", c_code); } Hey look, we just solved the problem of rewriting DMD in D, *and* the problem of the spec being out-of-sync with the compiler! :-P On a more serious note, though, if I ever needed a template of the sort you're describing, I'd prefer to use CTFE to generate it (and fix/improve CTFE if for whatever reason it can't yet do what I want it to do). The problem with invoking an external program to produce template code is that you have to somehow serialize the template arguments so that you can pass them to the program, and the program has to somehow deserialize them and reconstruct what the compiler has already constructed. Whereas if you did everything in CTFE, you have access to whatever the compiler has internally built to describe the types and values. Serializing / deserializing these structures generally are a big hiding place for bugs; I'd imagine that CTFE, being part of the compiler, is far more reliable at interpreting these structures correctly. Besides, I'd hate for my D code to have opaque blocks of import exec() where I have to examine another program to figure out exactly what's being compiled, instead of just reading the source code. So I'm still kinda on the fence about this whole thing (ignoring, of course, the major security concerns involving having import exec() in the first place). What I really want to see, is for CTFE to improve to the point where you can essentially write an arbitrary program that runs in CTFE to produce whatever code you wish to produce for the executable, thus making import exec() redundant. T -- MASM = Mana Ada Sistem, Man!
Jul 15 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Friday, 12 July 2013 at 21:54:02 UTC, Walter Bright wrote:
 On 7/12/2013 1:42 PM, Tofu Ninja wrote:
 So I had an idea recently, wouldn't it be cool to have the 
 ability to call an
 executable at compile time and capture its output. Something 
 like the string
 imports but instead of opening and reading a text file, it run 
 an executable,
 waits for it to finish, and grabs its output.

 It would get really cool if you could pass this executable 
 some args and then
 mix in its out put into your own code. It could be used 
 similarly to how CTFE
 are used but with out the overhead of trying to compile that 
 function and what
 not and with out the limitations on what it can do.

 I could imagine all sorts of things that would be possible 
 with this that is
 currently not.

 Not sure if this is something that could be implemented 
 easily, but seems like
 something that could be done and something that would be 
 really cool.

This is actually done in the makefile that builds DMD. A program (optabgen) is compiled and then run. Optabgen's output is several .c files which are then compiled into DMD's binary. The programs impcnvgen and idgen do similar things. It is a very powerful technique.

I did similar stuff for a mixed D/C++ program. The build contained some intermediary utilities used to spit out infos about the C++ code (typically sizeof and other characteristics like that) that could then be used to generate part of the binding in D.
Jul 15 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 16 July 2013 at 04:10:35 UTC, Andrei Alexandrescu 
wrote:
 I think the right direction here is to factor compiler parts as 
 a library. Then user code may insert external artifacts 
 (notably REPL) on its own.

 Andrei

SDC is already based on that principle, but still far from release state.
Jul 15 2013
prev sibling next sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
--047d7b5d301a6f6b9e04e1addf8a
Content-Type: text/plain; charset=ISO-8859-1

On Mon, Jul 15, 2013 at 9:10 PM, Andrei Alexandrescu <
SeeWebsiteForEmail erdani.org> wrote:

 On 7/15/13 6:26 AM, Don wrote:

 On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:

 So I had an idea recently, wouldn't it be cool to have the ability to
 call an executable at compile time and capture its output. Something
 like the string imports but instead of opening and reading a text
 file, it run an executable, waits for it to finish, and grabs its output.

 It would get really cool if you could pass this executable some args
 and then mix in its out put into your own code. It could be used
 similarly to how CTFE are used but with out the overhead of trying to
 compile that function and what not and with out the limitations on
 what it can do.

 I could imagine all sorts of things that would be possible with this
 that is currently not.

 Not sure if this is something that could be implemented easily, but
 seems like something that could be done and something that would be
 really cool.

I personally think it's a *horrible* idea. It's one of those things which looks good in small cases but doesn't scale.

I think the right direction here is to factor compiler parts as a library. Then user code may insert external artifacts (notably REPL) on its own.

of concept: automatically import C header files"), but having a compiler-as-a-library would make implementation easier and more customizable, eg by redirecting certain symbols such as 'CTFE exec' to a given delegate. myfile.d: import std.process; void main(){enum HOME=exec("echo $HOME", Redirect.all);} driver.d: import std.compiler; void exec_dg(){...} void main(){"myfile.d".readText.compile(&exec_dg);} --047d7b5d301a6f6b9e04e1addf8a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Mon, Jul 15, 2013 at 9:10 PM, Andrei Alexandrescu <span dir=3D"ltr">&lt;= <a href=3D"mailto:SeeWebsiteForEmail erdani.org" target=3D"_blank">SeeWebsi= teForEmail erdani.org</a>&gt;</span> wrote:<br><div class=3D"gmail_quote"><= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex"> <div class=3D"im">On 7/15/13 6:26 AM, Don wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> So I had an idea recently, wouldn&#39;t it be cool to have the ability to<b= r> call an executable at compile time and capture its output. Something<br> like the string imports but instead of opening and reading a text<br> file, it run an executable, waits for it to finish, and grabs its output.<b= r> <br> It would get really cool if you could pass this executable some args<br> and then mix in its out put into your own code. It could be used<br> similarly to how CTFE are used but with out the overhead of trying to<br> compile that function and what not and with out the limitations on<br> what it can do.<br> <br> I could imagine all sorts of things that would be possible with this<br> that is currently not.<br> <br> Not sure if this is something that could be implemented easily, but<br> seems like something that could be done and something that would be<br> really cool.<br> </blockquote> <br> I personally think it&#39;s a *horrible* idea. It&#39;s one of those things= <br> which looks good in small cases but doesn&#39;t scale.<br> </blockquote> <br></div> I think the right direction here is to factor compiler parts as a library. = Then user code may insert external artifacts (notably REPL) on its own.<spa= n class=3D"HOEnZb"><font color=3D"#888888"><br> <br></font></span></blockquote><div>=A0</div><div>CTFE exec would still be = needed (see yet another use case in EMAIL:&quot;Proof of concept: automatic= ally import C header files&quot;), but having a compiler-as-a-library would= make implementation easier and more customizable, eg by redirecting certai= n symbols =A0such as &#39;CTFE exec&#39; to a given delegate.</div> <div><br></div><div>myfile.d:</div><div>import std.process;</div><div>void = main(){enum HOME=3Dexec(&quot;echo $HOME&quot;, Redirect.all);}=A0</div><di= v><br></div><div>driver.d:</div><div>import std.compiler;</div><div>void ex= ec_dg(){...}</div> <div>void main(){&quot;myfile.d&quot;.readText.compile(&amp;exec_dg);}</div=
<div><br></div><div><br></div></div>

--047d7b5d301a6f6b9e04e1addf8a--
Jul 16 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jul 17, 2013 at 10:34:32AM +0200, Jacob Carlborg wrote:
 On 2013-07-16 06:10, Andrei Alexandrescu wrote:
 
I think the right direction here is to factor compiler parts as a
library.

Yes, I agree. I've been wanting this for a long time.

We've been talking about this for a while. Have we made any progress in this direction since? T -- What do you get if you drop a piano down a mineshaft? A flat minor.
Jul 17 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jul 17, 2013 at 05:40:03PM +0200, Jacob Carlborg wrote:
 On 2013-07-17 16:55, H. S. Teoh wrote:
 
We've been talking about this for a while. Have we made any progress
in this direction since?

There's been a bunch of pull requests merged related to adopting the DMD source code to make it work with the tool, I think it was, Daniel Murphy has created. This tool will be used to automatically port DMD to D. It's designed specifically for the DMD code base. Look for pull request tagged with [DDMD] in the title. Then I think the idea is refactor DMD to make it usable as a library.

Wow. That sounds like it's still a long ways off. But at least we're making progress. T -- The volume of a pizza of thickness a and radius z can be described by the following formula: pi zz a. -- Wouter Verhelst
Jul 17 2013
prev sibling parent "Tofu Ninja" <emmons0 purdue.edu> writes:
So if some one were to implement this, any ideas on the preferred 
syntax for such a feature?
Jul 17 2013