www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D meta language ?

reply Georg Wrede <georg.wrede nospam.org> writes:
I've been thinkin hard on a meta language.

I've talked with folks, amongst them prof. Jaakko Järvi, who actually 
met Walter at the C++ meeting last fall.

Bad news: I seem to have spilled the beans, big time. After asking his 
opinion about some of the core concepts I'd been cooking, it occurred to 
me that -- this guy's office is down the hall from Bjarne Stroustrup!

If these guys stop dwelling _within_ the C++ template jungle, and start 
looking at it from above, then it'll be just 18 months before one of 
them holds a ground breaking lecture.   :-(   :-(


D meta language is an important thing. But now it is also urgent.

Is there a mailing list for it, or should I continue on my own track?
Mar 09 2005
next sibling parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Georg Wrede wrote:

 I've been thinkin hard on a meta language.

 D meta language is an important thing. But now it is also urgent.

What would the "D meta language" be ? Something like YACC, but for D ? (maybe related to DGrammar http://www.dsource.org/projects/dgrammar/ ?) Or something completely different ? By definition it's "a language that can be used to describe languages" Just wondering what your little project was, and why it was urgent... --anders
Mar 09 2005
next sibling parent reply Georg Wrede <georg.wrede nospam.org> writes:
Anders F Björklund wrote:
 Georg Wrede wrote:
 
 I've been thinkin hard on a meta language.

[...]
 D meta language is an important thing. But now it is also urgent.

What would the "D meta language" be ? Something like YACC, but for D ? (maybe related to DGrammar http://www.dsource.org/projects/dgrammar/ ?)

BTW, where do I get the D grammar itself? (Other than copy/pasting from the docs, piece by piece. ;-) )
 Or something completely different ?
 
 By definition it's "a language that can be used to describe languages"
 Just wondering what your little project was, and why it was urgent...

Well, the C++ preprocessor has grown into a language. It's even Turing complete, so nobody can dispute it being a language on its own. The template system now is really a meta language. One could say that the STL is a language (of sorts), too, written with the syntax of the preprocessor/template grammar. What I've been thinking of, is something like a preprocessor that is aware of the language. (At least, for the time being it would be a (more or less) preprocessor.) The goal is, however, to have something that is a genuine part of the compiler, and that is intimately "aware" of the "underlying" language. And could conduct two-way communication with "the rest of" the compiler. "Preprocessor" here being merely a temporary technical solution. The way I see the state of language development today, is that real productivity, real power of expression, real versatility, and adequate abstraction -- all should be possible using a single medium. Now, there have been numeros ambitious enterprises in this arena. Quite a few passing fads have been hyped as "the ultimate tool for _everything_". Lisp, for example, is capable of expressing anything at all, with relative ease too (at least for the lisp gurus). But we don't see Lisp running everything. With C you can do "everything" too. But nobody uses it for abstract or high level stuff. A single language cannot fit everything any more. The fields of application and levels of abstraction are way too disparate for a single syntax to manage, if we want it to still be usable for as much as possible. (And if we don't, the competition will!) The time has come for languages that are genuine compounds of two separate levels of abstraction. Having two separate syntaxes "for one language" is no big deal today. C++ does it all the time. D has it (so far) in a smaller scale, if we think of D templates combined with mixins. The rules of this are different from "plain" D. Many machine code assemblers have had a two-level syntax for ages. The one I've played with, was Borland's TASM. Now, with TASM you could even do OO! The syntax and even support was there. And this was like 15 years ago. Having two syntaxes lets each of these be tuned and optimised for its own half of the cake. Also, having two syntaxes helps keep a kind of separation within the applications written. This actually resembles what many programmers do right now: do the nitty gritty in C, make it a DLL, then do the UI or program logic in, say Python, or even Euphoria. The D philosophy fits nicely here. D has always tried to be a language where you don't need a preprocessor. Somehow though, we seem to have a long way to go. For example, here's a snippet from the DMD distribution (written in C++, but anyway): Expression *Parser::parseAssignExp() { Expression *e; Expression *e2; Loc loc; e = parseCondExp(); while (1) { loc = this->loc; switch (token.value) { #define X(tok,ector) case tok: nextToken(); \ e2 = parseAssignExp(); e = new ector(loc,e,e2); continue; X(TOKassign, AssignExp); X(TOKaddass, AddAssignExp); X(TOKminass, MinAssignExp); X(TOKmulass, MulAssignExp); X(TOKdivass, DivAssignExp); X(TOKmodass, ModAssignExp); X(TOKandass, AndAssignExp); X(TOKorass, OrAssignExp); X(TOKxorass, XorAssignExp); X(TOKshlass, ShlAssignExp); X(TOKshrass, ShrAssignExp); X(TOKushrass, UshrAssignExp); X(TOKcatass, CatAssignExp); #undef X default: break; } break; } return e; } Doing this (what Walter's done here with the preprocessor) is just one example of what I want to be easy in D, too, without a separate preprocessor. Another example would be a mathematician. He defines a notation to use in source code, and then is able to use it. Of course, one might criticize that for illegibility to the normal programmer. But more important is that it is legible to the professionals in their own field. Say someone really needs to have (the infamous?) unquoted regular expression syntax, which was discussed here a couple of weeks ago. He could write a definition, and then just use any regular expressions within D code. And I can't even begin to imagine what people like prof. Järvi could conjure up, given a tool like this. So, whether easy or hairy stuff is needed, this would give more expressive power to D on so many new levels. Our "metalanguage" should be aware of its surroundings. __LINE__ etc. are only the beginning. It should be able to get any information a D program can get -- about files, the machine, whatever. And all at compile time. It should make the (revered, adminred, feared, sworn at) C++ template system look like an enormously hard to use toy. Done right, it would be easy to grasp, easy to use, and therefore less error prone.
Mar 09 2005
next sibling parent reply xs0 <xs0 xs0.com> writes:
But what would it look like? I really like the idea of a meta-language, 
I just can't come up with something that fits in D and at the same time 
makes it extensible..

As I see it, two issues are involved:
- being able to specify new syntax, both locally (as the X example 
below) and globally, where the main problem is - what is the syntax for 
syntax, and how do you make it self-extensible; if it's not 
self-extensible, it's just another layer of templates, I guess..

- being able to modify existing code, because just defining syntax 
expansions (like X below) is not enough. For example, if you wanted to 
add aspects, you'd need to define the syntax, have the compiler parse 
the source, then do something like "for each aspect that is defined, go 
through all the method and class declarations, find those that match, 
and insert the code from the aspect there and there if that and that". 
Now that's basically a program in itself, so it's a good question 
whether it is not better to simply open the compiler to plugins, than to 
try to fit this in the language and have the compiler be practically a 
virtual machine..

Even after you're able to do that, there are issues of collisions (what 
if two meta-things want to modify the same, say, class?), infinite loops 
(code A produces B, code B produces Aa, which becomes Ba, which becomes 
Aaa, ...), optimization, etc.

Especially the issue of optimization seems a big problem to me... I 
mean, "A x B" where x is some nice unicode symbol is not that much 
better than mul(A,B) that people would swallow like 2x slower running 
time over it, in the case the compiler won't be able to optimize it. And 
AFAIK, the whole optimization thing is more or less quite specific cases 
of what gets changed to what (I may be wrong on this, I'm no expert).

OTOH, perhaps it's not that big a deal - there could still be major 
gains in productivity, which can often be far more important than speed..

For a final thought, I guess a real victory would be if the 
meta-language was so powerful, that the entire D we have now could be 
expressed in it (except perhaps some really really basic constructs -- 
you do need something to build with in the first place).


xs0


Georg Wrede wrote:
 Anders F Björklund wrote:
 
 Georg Wrede wrote:

 I've been thinkin hard on a meta language.

[...]
 D meta language is an important thing. But now it is also urgent.

What would the "D meta language" be ? Something like YACC, but for D ? (maybe related to DGrammar http://www.dsource.org/projects/dgrammar/ ?)

BTW, where do I get the D grammar itself? (Other than copy/pasting from the docs, piece by piece. ;-) )
 Or something completely different ?

 By definition it's "a language that can be used to describe languages"
 Just wondering what your little project was, and why it was urgent...

Well, the C++ preprocessor has grown into a language. It's even Turing complete, so nobody can dispute it being a language on its own. The template system now is really a meta language. One could say that the STL is a language (of sorts), too, written with the syntax of the preprocessor/template grammar. What I've been thinking of, is something like a preprocessor that is aware of the language. (At least, for the time being it would be a (more or less) preprocessor.) The goal is, however, to have something that is a genuine part of the compiler, and that is intimately "aware" of the "underlying" language. And could conduct two-way communication with "the rest of" the compiler. "Preprocessor" here being merely a temporary technical solution. The way I see the state of language development today, is that real productivity, real power of expression, real versatility, and adequate abstraction -- all should be possible using a single medium. Now, there have been numeros ambitious enterprises in this arena. Quite a few passing fads have been hyped as "the ultimate tool for _everything_". Lisp, for example, is capable of expressing anything at all, with relative ease too (at least for the lisp gurus). But we don't see Lisp running everything. With C you can do "everything" too. But nobody uses it for abstract or high level stuff. A single language cannot fit everything any more. The fields of application and levels of abstraction are way too disparate for a single syntax to manage, if we want it to still be usable for as much as possible. (And if we don't, the competition will!) The time has come for languages that are genuine compounds of two separate levels of abstraction. Having two separate syntaxes "for one language" is no big deal today. C++ does it all the time. D has it (so far) in a smaller scale, if we think of D templates combined with mixins. The rules of this are different from "plain" D. Many machine code assemblers have had a two-level syntax for ages. The one I've played with, was Borland's TASM. Now, with TASM you could even do OO! The syntax and even support was there. And this was like 15 years ago. Having two syntaxes lets each of these be tuned and optimised for its own half of the cake. Also, having two syntaxes helps keep a kind of separation within the applications written. This actually resembles what many programmers do right now: do the nitty gritty in C, make it a DLL, then do the UI or program logic in, say Python, or even Euphoria. The D philosophy fits nicely here. D has always tried to be a language where you don't need a preprocessor. Somehow though, we seem to have a long way to go. For example, here's a snippet from the DMD distribution (written in C++, but anyway): Expression *Parser::parseAssignExp() { Expression *e; Expression *e2; Loc loc; e = parseCondExp(); while (1) { loc = this->loc; switch (token.value) { #define X(tok,ector) case tok: nextToken(); \ e2 = parseAssignExp(); e = new ector(loc,e,e2); continue; X(TOKassign, AssignExp); X(TOKaddass, AddAssignExp); X(TOKminass, MinAssignExp); X(TOKmulass, MulAssignExp); X(TOKdivass, DivAssignExp); X(TOKmodass, ModAssignExp); X(TOKandass, AndAssignExp); X(TOKorass, OrAssignExp); X(TOKxorass, XorAssignExp); X(TOKshlass, ShlAssignExp); X(TOKshrass, ShrAssignExp); X(TOKushrass, UshrAssignExp); X(TOKcatass, CatAssignExp); #undef X default: break; } break; } return e; } Doing this (what Walter's done here with the preprocessor) is just one example of what I want to be easy in D, too, without a separate preprocessor. Another example would be a mathematician. He defines a notation to use in source code, and then is able to use it. Of course, one might criticize that for illegibility to the normal programmer. But more important is that it is legible to the professionals in their own field. Say someone really needs to have (the infamous?) unquoted regular expression syntax, which was discussed here a couple of weeks ago. He could write a definition, and then just use any regular expressions within D code. And I can't even begin to imagine what people like prof. Järvi could conjure up, given a tool like this. So, whether easy or hairy stuff is needed, this would give more expressive power to D on so many new levels. Our "metalanguage" should be aware of its surroundings. __LINE__ etc. are only the beginning. It should be able to get any information a D program can get -- about files, the machine, whatever. And all at compile time. It should make the (revered, adminred, feared, sworn at) C++ template system look like an enormously hard to use toy. Done right, it would be easy to grasp, easy to use, and therefore less error prone.

Mar 10 2005
next sibling parent reply Georg Wrede <georg.wrede nospam.org> writes:
xs0 wrote:
 But what would it look like? I really like the idea of a meta-language, 
 I just can't come up with something that fits in D and at the same time 
 makes it extensible..

Well, at this point I think the best thing to do is to carefully think about what we _want_ it to be able to do. IMHO, if we try to come up with, say a syntax, or a way of "blending into D", then we unnecessarily create restraints. Both syntaxwise, but more important, we fence us into our newly made box, and that's the end of "out of the box" thinking. Right now is the time for the latter, and the longer we keep at it, the better we understand _what_ has to be done. Once that is clear, only then can we allow ourselves to think about _how_.
 As I see it, two issues are involved:
 - being able to specify new syntax, both locally (as the X example 
 below) and globally, where the main problem is - what is the syntax for 
 syntax, and how do you make it self-extensible; if it's not 
 self-extensible, it's just another layer of templates, I guess..

We shouldn't care! Just assume whatever we need to, we'll come up with in due time. First we need a good list of what we _want_. Suggestions and help, even (possibly hard) criticism is welcome too! The C crowd _started_ with a syntax. Then kept appending to it, and we all see where that leads. They've never had a clue of where they are ultimately aiming. Well, we'gre gonna do it the other way around.
 - being able to modify existing code, because just defining syntax 
 expansions (like X below) is not enough. 

Right!
 For example, if you wanted to 
 add aspects, you'd need to define the syntax, have the compiler parse 
 the source, then do something like "for each aspect that is defined, go 
 through all the method and class declarations, find those that match, 
 and insert the code from the aspect there and there if that and that". 

The meta language idea should be able to accommodate that. But not like we specifically thought of aspects in particular, rather like if it is what we wanted it to be, and only after that somebody invents aspects, the meta language should handle that too. Or, more to the point, the programmer who wants to implement aspects. And what he has to write in meta to make it so should not be incomprehensible to others.
 Now that's basically a program in itself, so it's a good question 

Ecactly. And that is what I've meant each time I've mentioned Turing! I see no way around it. -- The C++ preprocessor language had reached Turing level without any of the C++ honchos ever even noticing. Several _years_ later somebody just happened to discover the fact. My view is that cpp being TC is not a coincidence. Rather, it is a requisite. And therefore _we_ take that as a starting point.
 whether it is not better to simply open the compiler to plugins, than to 
 try to fit this in the language and have the compiler be practically a 
 virtual machine..

I would like to know more of this plugin idea.
 Even after you're able to do that, there are issues of collisions (what 
 if two meta-things want to modify the same, say, class?), infinite loops 
 (code A produces B, code B produces Aa, which becomes Ba, which becomes 
 Aaa, ...), optimization, etc.

We'll take care of them at the time. Had Walter got scared of circular imports too early -- but he didn't. Same with meta.
 Especially the issue of optimization seems a big problem to me... I 
 mean, "A x B" where x is some nice unicode symbol is not that much 
 better than mul(A,B) that people would swallow like 2x slower running 
 time over it, in the case the compiler won't be able to optimize it. And 
 AFAIK, the whole optimization thing is more or less quite specific cases 
 of what gets changed to what (I may be wrong on this, I'm no expert).
 
 OTOH, perhaps it's not that big a deal - there could still be major 
 gains in productivity, which can often be far more important than speed..

Optimization I hope happens after the meta phase. Of course, that does not give us the right to make meta inefficient. :-)
 For a final thought, I guess a real victory would be if the 
 meta-language was so powerful, that the entire D we have now could be 
 expressed in it (except perhaps some really really basic constructs -- 
 you do need something to build with in the first place).

Hmm. I wouldn't bet against it. OTOH, I'm not ready to have it as a goal either. Not at least right now.
 Georg Wrede wrote:
 
 Anders F Björklund wrote:

 Georg Wrede wrote:

 I've been thinkin hard on a meta language.

[...]
 D meta language is an important thing. But now it is also urgent.

What would the "D meta language" be ? Something like YACC, but for D ? (maybe related to DGrammar http://www.dsource.org/projects/dgrammar/ ?)

BTW, where do I get the D grammar itself? (Other than copy/pasting from the docs, piece by piece. ;-) )
 Or something completely different ?

 By definition it's "a language that can be used to describe languages"
 Just wondering what your little project was, and why it was urgent...

Well, the C++ preprocessor has grown into a language. It's even Turing complete, so nobody can dispute it being a language on its own. The template system now is really a meta language. One could say that the STL is a language (of sorts), too, written with the syntax of the preprocessor/template grammar. What I've been thinking of, is something like a preprocessor that is aware of the language. (At least, for the time being it would be a (more or less) preprocessor.) The goal is, however, to have something that is a genuine part of the compiler, and that is intimately "aware" of the "underlying" language. And could conduct two-way communication with "the rest of" the compiler. "Preprocessor" here being merely a temporary technical solution. The way I see the state of language development today, is that real productivity, real power of expression, real versatility, and adequate abstraction -- all should be possible using a single medium. Now, there have been numeros ambitious enterprises in this arena. Quite a few passing fads have been hyped as "the ultimate tool for _everything_". Lisp, for example, is capable of expressing anything at all, with relative ease too (at least for the lisp gurus). But we don't see Lisp running everything. With C you can do "everything" too. But nobody uses it for abstract or high level stuff. A single language cannot fit everything any more. The fields of application and levels of abstraction are way too disparate for a single syntax to manage, if we want it to still be usable for as much as possible. (And if we don't, the competition will!) The time has come for languages that are genuine compounds of two separate levels of abstraction. Having two separate syntaxes "for one language" is no big deal today. C++ does it all the time. D has it (so far) in a smaller scale, if we think of D templates combined with mixins. The rules of this are different from "plain" D. Many machine code assemblers have had a two-level syntax for ages. The one I've played with, was Borland's TASM. Now, with TASM you could even do OO! The syntax and even support was there. And this was like 15 years ago. Having two syntaxes lets each of these be tuned and optimised for its own half of the cake. Also, having two syntaxes helps keep a kind of separation within the applications written. This actually resembles what many programmers do right now: do the nitty gritty in C, make it a DLL, then do the UI or program logic in, say Python, or even Euphoria. The D philosophy fits nicely here. D has always tried to be a language where you don't need a preprocessor. Somehow though, we seem to have a long way to go. For example, here's a snippet from the DMD distribution (written in C++, but anyway): Expression *Parser::parseAssignExp() { Expression *e; Expression *e2; Loc loc; e = parseCondExp(); while (1) { loc = this->loc; switch (token.value) { #define X(tok,ector) case tok: nextToken(); \ e2 = parseAssignExp(); e = new ector(loc,e,e2); continue; X(TOKassign, AssignExp); X(TOKaddass, AddAssignExp); X(TOKminass, MinAssignExp); X(TOKmulass, MulAssignExp); X(TOKdivass, DivAssignExp); X(TOKmodass, ModAssignExp); X(TOKandass, AndAssignExp); X(TOKorass, OrAssignExp); X(TOKxorass, XorAssignExp); X(TOKshlass, ShlAssignExp); X(TOKshrass, ShrAssignExp); X(TOKushrass, UshrAssignExp); X(TOKcatass, CatAssignExp); #undef X default: break; } break; } return e; } Doing this (what Walter's done here with the preprocessor) is just one example of what I want to be easy in D, too, without a separate preprocessor. Another example would be a mathematician. He defines a notation to use in source code, and then is able to use it. Of course, one might criticize that for illegibility to the normal programmer. But more important is that it is legible to the professionals in their own field. Say someone really needs to have (the infamous?) unquoted regular expression syntax, which was discussed here a couple of weeks ago. He could write a definition, and then just use any regular expressions within D code. And I can't even begin to imagine what people like prof. Järvi could conjure up, given a tool like this. So, whether easy or hairy stuff is needed, this would give more expressive power to D on so many new levels. Our "metalanguage" should be aware of its surroundings. __LINE__ etc. are only the beginning. It should be able to get any information a D program can get -- about files, the machine, whatever. And all at compile time. It should make the (revered, adminred, feared, sworn at) C++ template system look like an enormously hard to use toy. Done right, it would be easy to grasp, easy to use, and therefore less error prone.


Mar 10 2005
parent reply xs0 <xs0 xs0.com> writes:
Georg Wrede wrote:
 xs0 wrote:
 
 But what would it look like? I really like the idea of a 
 meta-language, I just can't come up with something that fits in D and 
 at the same time makes it extensible..

Well, at this point I think the best thing to do is to carefully think about what we _want_ it to be able to do. IMHO, if we try to come up with, say a syntax, or a way of "blending into D", then we unnecessarily create restraints. Both syntaxwise, but more important, we fence us into our newly made box, and that's the end of "out of the box" thinking. Right now is the time for the latter, and the longer we keep at it, the better we understand _what_ has to be done. Once that is clear, only then can we allow ourselves to think about _how_.

Well, the first thing I want is macros somewhat like C's, except not on textual level, so you can't define something like #define BLAH(a,b) a + b + I still want to be able to generate new identifiers: #define SUM(T,U) sum_#{T}_#{U}(T a, U b) { return a+b; } and SUM(int,double) becomes sum_int_double(int a, double b) {...} Third, I wan't this: func!(__myBasicTypes__).doit(); to get expanded to this: func!(int).doit(); func!(double).doit(); func!(MyClass).doit(); for any "func" and "doit" i.e. whenever __myBasicTypes__ appears as a type, I want the expression to be expanded to three expressions with appropriate types. And not just in expressions, why not have template specializations for multiple types at the same time? Or function definitions: func(__anyCharType[]) { // do something } becomes func(char[]) func(wchar[]) func(dchar[]) Fourth, I want to be able to insert code from "outside" the method/func/class/whatever. Like: #match C:class(*) { #prepend_or_create_method (C, static this()) { globals.all_classes~=#get(C.fullName); } } If we manage that, we rock (and Walter doesn't have to implement reflection :)
 As I see it, two issues are involved:
 - being able to specify new syntax, both locally (as the X example 
 below) and globally, where the main problem is - what is the syntax 
 for syntax, and how do you make it self-extensible; if it's not 
 self-extensible, it's just another layer of templates, I guess..

We shouldn't care! Just assume whatever we need to, we'll come up with in due time. First we need a good list of what we _want_. Suggestions and help, even (possibly hard) criticism is welcome too!

Well, I think that just wildly brainstorming isn't going to be very productive. We should instead start by thinking of a few examples of what specifically we'd want to do, and conjure some way of encoding that. After the initial "spec" is done, we can see what else we'd like to do, and check whether it's already possible. If it is, great, if not, we amend and revise the "spec" to include both the old and new functionality. We can repeat this as long as we want, and in the end we'll have a solid syntax+semantics for the meta-D, and all that is left is implementing it, which will hopefully be relatively straightforward. OTOH, if we just dream about something, we'll only have dreams in the end.
 The C crowd _started_ with a syntax. Then kept appending to it, and we 
 all see where that leads. They've never had a clue of where they are 
 ultimately aiming. Well, we'gre gonna do it the other way around.

I'd say C started just as you're proposing - they thought of what they wanted to do and did it. The problem was that they later wanted to do additional stuff, and had to append to what was there already. I know you're trying to say that we should have something that is infinitely self-extensible, so that we'll never have to "append" something to it, but even if we manage to do that, we'll eventually run into the same problems. I mean, there's only so many ways you can overload, say, '(' and ')', before it becomes ambiguous what exactly you're trying to do. After that happens, you need new symbols/syntax for your new feature, but eventually you still run out of them. In other words, even if you have an infinitely extensible language, you can't infinitely extend it (at least realistically - noone will want to use 250 different operators, even if they're defined in Unicode).
 - being able to modify existing code, because just defining syntax 
 expansions (like X below) is not enough. 

Right!

Hmm, after thinking about it, I don't think that's possible at all. Say you did manage to define aspects. You can't use them in the aspect defining code, so that's limitation number one (because it can't process itself - if an aspect is needed for aspects to work, they can't work, because they don't work before they work, or something :). Second, if some other extension during processing sees an aspect, it will skip it because it doesn't know what to do with it. Later, when the aspect gets applied, the first extension won't see the code again, so won't work there. If you do run it again, it is possible that it will break what the aspect extension did, which is possibly even worse.
 For example, if you wanted to add aspects, you'd need to define the 
 syntax, have the compiler parse the source, then do something like 
 "for each aspect that is defined, go through all the method and class 
 declarations, find those that match, and insert the code from the 
 aspect there and there if that and that". 

The meta language idea should be able to accommodate that. But not like we specifically thought of aspects in particular, rather like if it is what we wanted it to be, and only after that somebody invents aspects, the meta language should handle that too. Or, more to the point, the programmer who wants to implement aspects. And what he has to write in meta to make it so should not be incomprehensible to others.

Well, we can't make a language for things we can't foresee (I don't mean stuff like aspects specifically, but more like "something that modifies other code by pattern matching on identifiers")
 Now that's basically a program in itself, so it's a good question 

Exactly. And that is what I've meant each time I've mentioned Turing! I see no way around it. -- The C++ preprocessor language had reached Turing level without any of the C++ honchos ever even noticing. Several _years_ later somebody just happened to discover the fact.

Hmm, when you say C++ preprocessor, you do mean its template system, right?
 whether it is not better to simply open the compiler to plugins, than 
 to try to fit this in the language and have the compiler be 
 practically a virtual machine..

I would like to know more of this plugin idea.

Well, if the compiler worked in some really well-defined way and had an open interface to extensions (as in .DLLs or similar), it could let the plugins work on its internal structures (like definitions for tokens and grammar before parsing and AST after parsing), which would basically mean "anything" is possible. But its not as simple as it sounds, especially the grammar part, grammars are very delicate...
 Even after you're able to do that, there are issues of collisions 
 (what if two meta-things want to modify the same, say, class?), 
 infinite loops (code A produces B, code B produces Aa, which becomes 
 Ba, which becomes Aaa, ...), optimization, etc.

We'll take care of them at the time. Had Walter got scared of circular imports too early -- but he didn't. Same with meta.

Well, it's not quite the same :) And those are issues that need to be considered straight from the start (esp. collisions), if you want something successful. xs0
Mar 11 2005
parent Georg Wrede <georg.wrede nospam.org> writes:
xs0 wrote:
 Georg Wrede wrote:
 xs0 wrote:
 Well, at this point I think the best thing to do is to carefully think 
 about what we _want_ it to be able to do. IMHO, if we try to come up 
 with, say a syntax, or a way of "blending into D", then we 
 unnecessarily create restraints. Both syntaxwise, but more important, 
 we fence us into our newly made box, and that's the end of "out of the 
 box" thinking. Right now is the time for the latter, and the longer we 
 keep at it, the better we understand _what_ has to be done. Once that 
 is clear, only then can we allow ourselves to think about _how_.


Good list below!
 Well, the first thing I want is macros somewhat like C's, except not on 
 textual level, so you can't define something like
 
 #define BLAH(a,b) a + b +

That ought to be easy. Definitely local scope is what should be used. Both textual level, and global scope in general "are for the other guys"!
 I still want to be able to generate new identifiers:
 
 #define SUM(T,U) sum_#{T}_#{U}(T a, U b) { return a+b; }
 
 and SUM(int,double) becomes sum_int_double(int a, double b) {...}

Ha! This revealed to me that the meta system could have a library of its own. Some of the usual things (like the one above) could be there all along. Then folks wouldn't have to actually write them, just look them up in the manual, and then activate.
 Third, I wan't this:
 
 func!(__myBasicTypes__).doit();
 
 to get expanded to this:
 
 func!(int).doit();
 func!(double).doit();
 func!(MyClass).doit();

So you are creating a method for a bunch of classes from a template? Resembles multiple inheritance, and aspects? This would definitely be doable in C++. But with D having method definitions inside the classes, I can't say off hand. Probably it would still be doable, but you'd possibly need more work than with inheritance. (Here I'm deliberately overlooking the difference between int and double, vs classes.) You may want to be more specific on this one.
 for any "func" and "doit" i.e. whenever __myBasicTypes__ appears as a 
 type, I want the expression to be expanded to three expressions with 
 appropriate types. And not just in expressions, why not have template 
 specializations for multiple types at the same time? Or function 
 definitions:
 
 func(__anyCharType[]) {
   // do something
 }
 
 becomes
 
 func(char[])
 func(wchar[])
 func(dchar[])

That depends on the "// do something". That is, if it can handle all of the types. Then it should be doable.
 Fourth, I want to be able to insert code from "outside" the 
 method/func/class/whatever. Like:
 
 #match C:class(*) {
     #prepend_or_create_method (C, static this()) {
         globals.all_classes~=#get(C.fullName);
     }
 }

Gee, a question I have no answer for! You'd have to ask Walter about this.
 If we manage that, we rock (and Walter doesn't have to implement 
 reflection :)

I wouldn't be surprised if making this work would be the same amount of work for Walter.
 As I see it, two issues are involved:
 - being able to specify new syntax, both locally (as the X example 
 below) and globally, where the main problem is - what is the syntax 
 for syntax, and how do you make it self-extensible; if it's not 
 self-extensible, it's just another layer of templates, I guess..



My answer here missed your main point. :-(
 We shouldn't care! Just assume whatever we need to, we'll come up with 
 in due time. First we need a good list of what we _want_. Suggestions 
 and help, even (possibly hard) criticism is welcome too!

Well, I think that just wildly brainstorming isn't going to be very productive. We should instead start by thinking of a few examples of what specifically we'd want to do, and conjure some way of encoding that.

True. But just like with programming in general, the longer you abstain from the keyboard, the faster the overall app gets ready, and with better quality.
 After the initial "spec" is done, we can see what else we'd like 
 to do, and check whether it's already possible. If it is, great, if not, 
 we amend and revise the "spec" to include both the old and new 
 functionality. We can repeat this as long as we want, and in the end 
 we'll have a solid syntax+semantics for the meta-D, and all that is left 
 is implementing it, which will hopefully be relatively straightforward.

Yes. And this is a deep issue, so we ought to get folks tinking hard, before we know what we want in the spec.
 OTOH, if we just dream about something, we'll only have dreams in the end.

:-)
 The C crowd _started_ with a syntax. Then kept appending to it, and we 
 all see where that leads. They've never had a clue of where they are 
 ultimately aiming. Well, we'gre gonna do it the other way around.

I'd say C started just as you're proposing - they thought of what they wanted to do and did it. The problem was that they later wanted to do additional stuff, and had to append to what was there already. I know you're trying to say that we should have something that is infinitely self-extensible, so that we'll never have to "append" something to it,

Yes.
 but even if we manage to do that, we'll eventually run 
 into the same problems. I mean, there's only so many ways you can 
 overload, say, '(' and ')', before it becomes ambiguous what exactly 
 you're trying to do. After that happens, you need new symbols/syntax for 
 your new feature, but eventually you still run out of them. In other 
 words, even if you have an infinitely extensible language, you can't 
 infinitely extend it (at least realistically - noone will want to use 
 250 different operators, even if they're defined in Unicode).

That's where locality of scope comes to rescue. With this system statisticians, mathematicians, physicists, even astrologists, could reuse the same symbols -- just like in paper books today.
 - being able to modify existing code, because just defining syntax 
 expansions (like X below) is not enough. 

Right!

Hmm, after thinking about it, I don't think that's possible at all. Say you did manage to define aspects. You can't use them in the aspect defining code, so that's limitation number one (because it can't process itself - if an aspect is needed for aspects to work, they can't work, because they don't work before they work, or something :). Second, if some other extension during processing sees an aspect, it will skip it because it doesn't know what to do with it. Later, when the aspect gets applied, the first extension won't see the code again, so won't work there. If you do run it again, it is possible that it will break what the aspect extension did, which is possibly even worse.
 For example, if you wanted to add aspects, you'd need to define the 
 syntax, have the compiler parse the source, then do something like 
 "for each aspect that is defined, go through all the method and class 
 declarations, find those that match, and insert the code from the 
 aspect there and there if that and that". 

The meta language idea should be able to accommodate that. But not like we specifically thought of aspects in particular, rather like if it is what we wanted it to be, and only after that somebody invents aspects, the meta language should handle that too. Or, more to the point, the programmer who wants to implement aspects. And what he has to write in meta to make it so should not be incomprehensible to others.

Well, we can't make a language for things we can't foresee (I don't mean stuff like aspects specifically, but more like "something that modifies other code by pattern matching on identifiers")

About aspects, I really need more knowledge on them. So, on second thought I won't comment on them for the time being. "Something that modifies other code by pattern matching on identifiers" is more or less what I'm thinking about in general with this meta thing.
 Now that's basically a program in itself, so it's a good question 

Exactly. And that is what I've meant each time I've mentioned Turing! I see no way around it. -- The C++ preprocessor language had reached Turing level without any of the C++ honchos ever even noticing. Several _years_ later somebody just happened to discover the fact.

Hmm, when you say C++ preprocessor, you do mean its template system, right?

Yes. (Sorry for the ambiguous expression.)
 whether it is not better to simply open the compiler to plugins, than 
 to try to fit this in the language and have the compiler be 
 practically a virtual machine..

I would like to know more of this plugin idea.

Well, if the compiler worked in some really well-defined way and had an open interface to extensions (as in .DLLs or similar), it could let the plugins work on its internal structures (like definitions for tokens and grammar before parsing and AST after parsing), which would basically mean "anything" is possible. But its not as simple as it sounds, especially the grammar part, grammars are very delicate...

Currently I imagine the meta thing and the compiler would work in cooperation. The meta would not change the compiler's structures, only ask it to add or change things. Hopefully not even that -- if I'm at all on the right track here.
 Even after you're able to do that, there are issues of collisions 
 (what if two meta-things want to modify the same, say, class?), 
 infinite loops (code A produces B, code B produces Aa, which becomes 
 Ba, which becomes Aaa, ...), optimization, etc.

We'll take care of them at the time. Had Walter got scared of circular imports too early -- but he didn't. Same with meta.

Well, it's not quite the same :) And those are issues that need to be considered straight from the start (esp. collisions), if you want something successful.

Yes. Circularities I just cavalierly assume Walter could handle. Or at least detect, and error on them. So it's more like a "syntax issue". I mean, how to aid the programmer in not unnecessarily creating them and stumbling over compiler errors because of them. Collisions, well, to begin with, we'd need to have a defined order of things. That's something even the application writer needs to be clear on, right? Meta or no meta. If there is a clear order to what is done, then collisions should become a very much smaller issue.
Mar 11 2005
prev sibling parent Georg Wrede <georg.wrede nospam.org> writes:
xs0 wrote:
 As I see it, two issues are involved:
 - being able to specify new syntax, both locally (as the X example 
 below) and globally, where the main problem is - what is the syntax for 
 syntax, and how do you make it self-extensible; if it's not 
 self-extensible, it's just another layer of templates, I guess..

Forgot to answer this. Of course it has to be sef-extensible. I see no other option. The entire point of a Decent Meta Language is, to be able to cope with the unforeseen. It would have to be self-extensible, and self-modifiable. -- Without imposing either on the down-to-earth regular user of it. It's like, D allows you to do OO or do without OO, or mix them. The Meta shoud be the same. ------------- We could view (with some effort and will power) D itself as an asm metalanguage. "D is another asm compiler, but it wraps many of the most-used things in a convenient meta language, and has an extensive (in this context anyway) library (phobos), to relieve the programmer of writing the most usual things over and over. It handles high-level I/O, the usual data structures, and conversions for you." But because it is not self-extensible and not self-modifiable, it can only (in a broad sense) be used with the kind of problems its creators had in mind. (( A derogatory sounding way of saying `it's for stuff you'd use C, C++, Java etc. for`.))
Mar 11 2005
prev sibling parent reply "Charlie Patterson" <charliep1 excite.com> writes:
I'm not sure where you want to go with this meta-language but I'm here to 
burst your bubble.  (-:  I always wanted a meta-langauge... until I saw one. 
There is one for C++ and it is indecipherable.  It seems the movement in 
langagues is towards cleaning up and pairing down, not making them more 
"tricky."

Lots of people still claim C++ is too hard precisely because you don't know 
what it will do under the covers for a simple assignment statement.  And 
keep in mind C++ and even Java and D have some basic "meta programming" with 
operator= (opAssign) type options.

If you really, really want to add whole new calculus to the language, well 
that is what I thought I wanted until I saw an example.  Most people just 
get disturbed by creating it and even just being forced to use the results.

Finally, I've been on a tear for the last few years that people over-use 
languages anyway.  I think I got the idea from a Myer's C++ book (and also 
that Andrescu template book).  I was following along with all the insanely 
detailed examples of using private inheritance with a customized new 
operator, etc, etc, when they started adjusting the C++ vtable in order to 
create some lisp-like capabilities (non-portable obviously).  That was the 
last straw for me.  If you have to *beat* the language, then you need to 
simplify your app... or suffer the ugly consequences of maintaing it.  Or 
writing a new language!  But at some level of power, people just drop off.

My two cents.

"Georg Wrede" <georg.wrede nospam.org> wrote in message 
news:422F7E8C.7040509 nospam.org...
 Anders F Björklund wrote:
 Georg Wrede wrote:

 I've been thinkin hard on a meta language.

[...]
 D meta language is an important thing. But now it is also urgent.

What would the "D meta language" be ? Something like YACC, but for D ? (maybe related to DGrammar http://www.dsource.org/projects/dgrammar/ ?)

BTW, where do I get the D grammar itself? (Other than copy/pasting from the docs, piece by piece. ;-) )
 Or something completely different ?

 By definition it's "a language that can be used to describe languages"
 Just wondering what your little project was, and why it was urgent...

Well, the C++ preprocessor has grown into a language. It's even Turing complete, so nobody can dispute it being a language on its own. The template system now is really a meta language. One could say that the STL is a language (of sorts), too, written with the syntax of the preprocessor/template grammar. What I've been thinking of, is something like a preprocessor that is aware of the language. (At least, for the time being it would be a (more or less) preprocessor.) The goal is, however, to have something that is a genuine part of the compiler, and that is intimately "aware" of the "underlying" language. And could conduct two-way communication with "the rest of" the compiler. "Preprocessor" here being merely a temporary technical solution. The way I see the state of language development today, is that real productivity, real power of expression, real versatility, and adequate abstraction -- all should be possible using a single medium. Now, there have been numeros ambitious enterprises in this arena. Quite a few passing fads have been hyped as "the ultimate tool for _everything_". Lisp, for example, is capable of expressing anything at all, with relative ease too (at least for the lisp gurus). But we don't see Lisp running everything. With C you can do "everything" too. But nobody uses it for abstract or high level stuff. A single language cannot fit everything any more. The fields of application and levels of abstraction are way too disparate for a single syntax to manage, if we want it to still be usable for as much as possible. (And if we don't, the competition will!) The time has come for languages that are genuine compounds of two separate levels of abstraction. Having two separate syntaxes "for one language" is no big deal today. C++ does it all the time. D has it (so far) in a smaller scale, if we think of D templates combined with mixins. The rules of this are different from "plain" D. Many machine code assemblers have had a two-level syntax for ages. The one I've played with, was Borland's TASM. Now, with TASM you could even do OO! The syntax and even support was there. And this was like 15 years ago. Having two syntaxes lets each of these be tuned and optimised for its own half of the cake. Also, having two syntaxes helps keep a kind of separation within the applications written. This actually resembles what many programmers do right now: do the nitty gritty in C, make it a DLL, then do the UI or program logic in, say Python, or even Euphoria. The D philosophy fits nicely here. D has always tried to be a language where you don't need a preprocessor. Somehow though, we seem to have a long way to go. For example, here's a snippet from the DMD distribution (written in C++, but anyway): Expression *Parser::parseAssignExp() { Expression *e; Expression *e2; Loc loc; e = parseCondExp(); while (1) { loc = this->loc; switch (token.value) { #define X(tok,ector) case tok: nextToken(); \ e2 = parseAssignExp(); e = new ector(loc,e,e2); continue; X(TOKassign, AssignExp); X(TOKaddass, AddAssignExp); X(TOKminass, MinAssignExp); X(TOKmulass, MulAssignExp); X(TOKdivass, DivAssignExp); X(TOKmodass, ModAssignExp); X(TOKandass, AndAssignExp); X(TOKorass, OrAssignExp); X(TOKxorass, XorAssignExp); X(TOKshlass, ShlAssignExp); X(TOKshrass, ShrAssignExp); X(TOKushrass, UshrAssignExp); X(TOKcatass, CatAssignExp); #undef X default: break; } break; } return e; } Doing this (what Walter's done here with the preprocessor) is just one example of what I want to be easy in D, too, without a separate preprocessor. Another example would be a mathematician. He defines a notation to use in source code, and then is able to use it. Of course, one might criticize that for illegibility to the normal programmer. But more important is that it is legible to the professionals in their own field. Say someone really needs to have (the infamous?) unquoted regular expression syntax, which was discussed here a couple of weeks ago. He could write a definition, and then just use any regular expressions within D code. And I can't even begin to imagine what people like prof. Järvi could conjure up, given a tool like this. So, whether easy or hairy stuff is needed, this would give more expressive power to D on so many new levels. Our "metalanguage" should be aware of its surroundings. __LINE__ etc. are only the beginning. It should be able to get any information a D program can get -- about files, the machine, whatever. And all at compile time. It should make the (revered, adminred, feared, sworn at) C++ template system look like an enormously hard to use toy. Done right, it would be easy to grasp, easy to use, and therefore less error prone.

Mar 10 2005
parent reply Georg Wrede <georg.wrede nospam.org> writes:
Charlie Patterson wrote:
 I'm not sure where you want to go with this meta-language but I'm here to 
 burst your bubble.  (-:  

Thanks! It NEEDS to be stabbed at, wrought, wrinkled, etc. It's the only way we'll prune the chaff off. Or maybe prune the whole deal if it doesn't deserve to live.
 I always wanted a meta-langauge... until I saw one. 
 There is one for C++ and it is indecipherable.  It seems the movement in 
 langagues is towards cleaning up and pairing down, not making them more 
 "tricky."

True. And the C++ one is so gross, it makes everybody give up.
 Lots of people still claim C++ is too hard precisely because you don't know 
 what it will do under the covers for a simple assignment statement.  And 
 keep in mind C++ and even Java and D have some basic "meta programming" with 
 operator= (opAssign) type options.

Right. Now, if Java and D already have some of this, then I think it just proves we can't live without one.
 If you really, really want to add whole new calculus to the language, well 
 that is what I thought I wanted until I saw an example.  Most people just 
 get disturbed by creating it and even just being forced to use the results.

Naaw, a new calculus is not the target. What I want, is a way to, at one end express repetitions and patterns in source with something smarter than just plain writing it all out (c.f. the DMD #define example), and at the other, a way to increase the expressivenes of the, er, "combined" language. And it ABSOLUTELY has to be easy to write and understand. For example, simple #defines in cpp are pretty easy to understand, even for those who've never written one. But C++ templates get just unreadable pretty fast. And I blame their specific template language for it. I don't believe it _has_ to be so difficult. The way to express a solution should never have to be more complicated than the solution itself.
 Finally, I've been on a tear for the last few years that people over-use 
 languages anyway.  I think I got the idea from a Myer's C++ book (and also 
 that Andrescu template book).  I was following along with all the insanely 
 detailed examples of using private inheritance with a customized new 
 operator, etc, etc, when they started adjusting the C++ vtable in order to 
 create some lisp-like capabilities (non-portable obviously).  That was the 
 last straw for me.  If you have to *beat* the language, then you need to 
 simplify your app... or suffer the ugly consequences of maintaing it.  Or 
 writing a new language!  But at some level of power, people just drop off.

True. D is a "practical language for practical programmers". We need a practical meta language, that is practical for practical purposes. And if we can give it expressive power that does not run out of stream right after the first block, without making either simple or hard things harder than they _inherently_ are, then we've succeeded.
Mar 10 2005
parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Georg Wrede" <georg.wrede nospam.org> wrote in message 
news:4230C679.40509 nospam.org...
 Charlie Patterson wrote:
 I'm not sure where you want to go with this meta-language but I'm 
 here to burst your bubble.  (-:

Thanks! It NEEDS to be stabbed at, wrought, wrinkled, etc. It's the only way we'll prune the chaff off. Or maybe prune the whole deal if it doesn't deserve to live.

Being the dullard that I am, I have a real hard time imagining what you're talking about absent examples. Can you cook up a couple - just small one's be ok - and that'd help me get into it? Cheers Matthew
Mar 10 2005
next sibling parent Georg Wrede <georg.wrede nospam.org> writes:
Matthew wrote:
 "Georg Wrede" <georg.wrede nospam.org> wrote:
Charlie Patterson wrote:

I'm not sure where you want to go with this meta-language but I'm 
here to burst your bubble.  (-:

Thanks! It NEEDS to be stabbed at, wrought, wrinkled, etc. It's the only way we'll prune the chaff off. Or maybe prune the whole deal if it doesn't deserve to live.

Being the dullard that I am, I have a real hard time imagining what you're talking about absent examples. Can you cook up a couple - just small one's be ok - and that'd help me get into it?

OK. But I will not do examples on the language, only how it could be used. We need to have that covered, before we even start getting into the meta syntax. Say you need to do a long case statement, like the C++ one: --------------------- switch (token.value) { #define X(tok,ector) case tok: nextToken(); \ e2 = parseAssignExp(); e = new ector(loc,e,e2); continue; X(TOKassign, AssignExp); X(TOKaddass, AddAssignExp); //etc...... X(TOKushrass, UshrAssignExp); X(TOKcatass, CatAssignExp); #undef X default: break; } --------------------- One way to write it could be: --------------------- switch (token.value) { lkjsdflkjlklasldkjlasd // <-- the meta def TOKassign AssignExp TOKaddass AddAssignExp //etc...... TOKushrass UshrAssignExp TOKcatass CatAssignExp lksdjflksjdlf // <-- undo meta def default: break; } --------------------- Or, if we wanted to make it more readable to the C++ people, we'd do the meta def so that we could write the TOKassign lines just like they'd write: --------------------- switch (token.value) { lkjsdflkjlklasldkjlasd // <-- another meta def X(TOKassign, AssignExp); X(TOKaddass, AddAssignExp); //etc...... X(TOKushrass, UshrAssignExp); X(TOKcatass, CatAssignExp); lksdjflksjdlf // <-- undo meta def default: break; } --------------------- At this point everyting is open. For example: do we want the meta syntax to be "invisible"? (That means, an unconspicuous part of D.) Or do we want it to stand out like heck? Unconspicuous: --------------------- char[] foo = "lksdf"; int number = 8; meta(create, kjdslfkjlskdjlsk); // <-- a meta def here tree appletree trunk mainBranch strong internetConnected hangar branch filesystemAware auxBranch temporary stdErrOutput roots mainRoot suction localNetConnected hardwareBranch tapesystem rw hwType HP9245 hwType MaxstoreUnidriver unmeta(last); // <-- undo meta def int errlevel = 0; --------------------- Conspicuous: --------------------- char[] foo = "lksdf"; int number = 8; ####!!!!! META-LANGUAGE-DEFINITION-SECTION-BEGIN ##!!! METADEF (CREATE, lksjdflksjdlks) <-- a meta def here tree appletree trunk mainBranch strong internetConnected hangar branch filesystemAware auxBranch temporary stdErrOutput roots mainRoot suction localNetConnected hardwareBranch tapesystem rw hwType HP9245 hwType MaxstoreUnidriver ##!!! DESTROY-METADEF (lksjdflksjdlks) <-- undo meta def ####!!!!! META-LANGUAGE-DEFINITION-SECTION-END int errlevel = 0; --------------------- The time for meta syntax is not yet. Issues like the above have to be thought out before that. In the mean time: We need more use cases!! Tell me a few stumbling blocks you run into when writing STL for D! Or other things that are currently hard in D and not in C++. Another example: --------------------- // excerpt from the Aaron Spelling far-out math library // some formulas from the text book, compiled here // these take and return floats lksjdflksajdlfksadjlf // the metadef doubleSin(x) := sin(x) + sin(x); antiSin(x) := sin(x + pi); lksjdf // undo metadef --------------------- Sorry, that example looks disgustingly trivial. Totally worthless. Let's write something worse, this time actually from a math book. :-( [[ Two hours later: Crap, I'll do this example later. I will, honest. In the meantime, let's see something else: ]] Say, we need to do a lot of matrix literals. This might be part of some library code: --------------------- LKAJSDLFKAJSLDKFJLSADKJF // the metadef eigen_schmeigen = 2 3 2 0 1 0 -3 0 0 0 1 -3 ; //... snip, a lot of them here wamon_schmamon = 0 -1 0 -1 1 0 0 1 1 1 1 1 1 -1 -1 0 0 1 ; LKAJSDF // end metadef --------------------- So what's the use, you might ask. Point is, for a lot of things there are established ways of expressing them. Often, these are clearer than doing the same in regular programming language syntax. So, i bet some corporations, universities, oss, and others will contribute to our meta language library, and later the programmer only has to write: --------------------- import uni_groeningen.matrix; alias uni_groeningen.matrix.free_mat_notation.begin ug.begin; alias uni_groeningen.matrix.free_mat_notation.end ug.end; // ... ug.begin; eigen_schmeigen = 2 3 2 0 1 0 -3 0 0 0 1 -3 ; wamon_schmanon = 0 -1 0 -1 1 0 0 1 1 1 1 1 1 -1 -1 0 0 1 ; ug.end; --------------------- Sure, this opens the door for obfuscation C++ has never even seen. But hey, even if I have a kitchen knife in my kitchen, I don't stab everyone who comes there. (I'll leave NRA out of this. :-( ) ######### Somehow, though, I feel that examples from template programming (D, C++) might be needed here. What's something you currently wanted to do but can't?
Mar 11 2005
prev sibling parent Georg Wrede <georg.wrede nospam.org> writes:
Matthew wrote:
 Being the dullard that I am, I have a real hard time imagining what 
 you're talking about absent examples.

I'm not talking about absent examples. You are. SORRY, couldn'resist! :-) ############################################## Disclaimer: The stuff below is idiotic. Pleas try to look at the language part of it, not the stupidity of the examples themselves... :-/ ############################################## This guy works in a company where most of the core competence is in using the Intel math coprocessor in creative ways. They do contracting for NASA, NSA, SHELL, and some even for CERN. They have reams of code libraries to ease the use of the math coprocessor. However, the languages they've used so far (Forth, C, asm, C++, Lisp) have only been able to cater for some aspect of application development. Currently they have separate staff groups for each language, and (the applications made being huge and complicated) most projects use the work from several of these groups. They have tried to create a C++ template library, and a Lisp (actually Scheme) library, that'd let the programmer express his wishes both at low level as well as on the very high level this company needs. But nothing seems to work seamlessly, or efficiently. Then they found D. An excerpt from the low level programming: ---- library code module foo.bar; lkajsdlkjaldskfjlsadk; /* That was a meta definition that essentially creates the function wrapper for D function definitions that need low-level math coprosessor routines. Inside this kind of function definitions you can mix [Our Proprietary, Secret Notation] with regular D code. */ ---- production code import foo.bar; alias foo.bar b; float mLiff; b.floatFunc float aCoolLength(p1, p2, p3, p4) { // all params float, unless other specified float m1, m2, m3; if ( p1 < p2 ) { p1 p3 push sin p4 push log exp p2 inv mul m1 pop } if (m1 < p4) m2 = m1 else m2 = m1 + p1; return m2; } void main() { mLiff = aCoolLength(2.3, 4.5, 6.7, 8.9); writefln("The meaning of Life is: %f.2", mLiff); } Now, this was an example from the low level. I'd like to do one with high level too, but that'll take some more time. ################################################# Another example: This firm manufactures and sells sugar cube size mass produced brains. But model life cycle is short, and customers keep inventing unforeseen uses for them. Essentially, it is a cpu, some rom, some ram, and pins for 16 TTL connections programmable to be in or out, and an infrared interface for field programming. It also has 3 analog inputs and 3 outputs. Their customers include toy manufacturers, mining robotics, fire departments, the military, and some unnamed government agencies. Often a project is to create the needed software for a new gadget -- and (with some unnamed customers) it has to be ready and bug free in a matter of hours. (They still remember the hulabaloo with a project during the Gulf War.) They could make a huge library, have a language of their own, the works. But specs change, processors change, etc. So they need to do C like code, but it has to be convenient to do the behavioral primitives. All the gadgets, once out in the field, are programmable with a small, interpreted language. End-users do field programming with this language. This firm designs and assembles these sugar-cube brains, and ships them with firmware customized for the retail gadget (or hand-held nuclear missile guidance system). They also sell the field programming software, and of course there are different UI versions for the toy makers and the marines. They have found out that "We really want a C, but with faster development times, and vastly less bugs in the binaries." Since most programs are just a single file, the low level gets defined in the same file as where it is used to build behavioral primitives. Few languages have been of help here. Then they found D. Some excerpts, not necessarily from same project: void step() { if ( ! obstacle(ahead) ) { unlock_brake forward.slow while (!obstacle(ahead) && distance < STEPDISTANCE ) {} stop } } void deployPayload() { unlockArm extendArm wristRight extendFinger1 jawOpen smallExtendElbow jawClose } Well, this turned out to be intermediate level code. :-( What they really wanted was to change having to write this (pure D code) void function foo(){ bar(); barf(); bar(); barf(); boo(); doo(); dee(); dii(); lee(); loo(); luu(); moo(); soo(); } to this: void function foo(){ bar barf bar barf boo doo dee dii lee loo luu moo soo } Most of their programming is just invoking tiny functions that neither take or return anything. Slapping lots of them in your code day in and day out gets tedious with all the "();" all over the place. Also, they wanted the code to look cleaner, since they discovered that it reduces bugs, and helps find errors, too. So, their standard library merely defines that inside void fname() you don't have to use either semicolons or parentheses. They want, however to remain flexible. That's why they were so interested in D, with its superior meta code facilities. Now they are considering automatic creation of function sets. The programmer would create all of the functions like: int fname(int a = 1, int b = -1) and the meta system would automatically create a set of these. This would be used where functions need to return, say, a success code, or length traversed. Most of the time that is not needed (well, in this example company, anyway), but sometimes it is needed. For example the programmer creates: // walk on water int wow (int distance = 1, int careful = -1) { // let's say careful == -1 means keep last level // The code here would use the params, and return // a value, in this case the distance traversed. // But most often it is not used. } Then the meta would create the following versions: void wow (int distance = 1, int careful = -1) { // take none, return none // just walk one unit with default or // last used carefulness } int Wow (int distance = 1, int careful = -1) { // return distance } int woW (int distance = 1, int careful = -1) { // walk till obstacle } int WOW (int distance = 1, int careful = -1) { // panic walk (careful at min, dist till obstacle) } And so on. (Since I haven't worked there, I don't know the standard permutations! But you get the drift, right?) Those permutations would contain upper case letters in the name. The standard for which letter means what, has become plain obvious at the company, and it has been used for years, very successfully. Problem was, that it was a constant source of bugs. People capped the wrong letter, etc. Now they see the end of those bugs. And since there is only one "copy" of actual function code, it stays in sync with itself. Now they use this tac for most of the primitives. Walk, extendArm, drill, turn, sleep, all of them are now encoded with the three-first-letter-caps. Programmers are happy, and roll-out times smaller. This is also the default for any new kind of action primitives. ######################################## The way I see the meta language is, that it would not bee too hard to have it check and enforce the "self-made" syntaxes. Not automatically, but with writing meta code for the checking and enforcing -- and still be understandable to a normal person!
Mar 11 2005
prev sibling next sibling parent Georg Wrede <georg.wrede nospam.org> writes:
Anders F Björklund wrote:
 Georg Wrede wrote:
 
 I've been thinkin hard on a meta language.

[...]
 D meta language is an important thing. But now it is also urgent.

What would the "D meta language" be ? Something like YACC, but for D ? (maybe related to DGrammar http://www.dsource.org/projects/dgrammar/ ?) Or something completely different ? By definition it's "a language that can be used to describe languages" Just wondering what your little project was, and why it was urgent... --anders

Copy/pasting from another thread here. But read my other answer to your post first. ############################################### begin include Why not push to get the enum-names exposed via reflection instead? I wouldn't wish to have to re-type them all over again, in a different manner. Doesn't make sense. - Kris In article <d0onfb$1afp$1 digitaldaemon.com>, Andrew Fedoniouk says...
Thanks a lot,

What about this:

enum E {
  ZERO,
  ONE,
  TWO
}

char[] toString(E e)
{
  static char[][] map =
  [
    E.ZERO  :"primordial",
    E.ONE   :"first",
    E.TWO   :"second"
  ];
  return map[e];
}

for simple continuous enums?


################################################ end include Things like the above would be handled with the meta language real neatly: enum E { some_meta_language_def_here; ZERO primordial ONE first TWO second undefine_the_meta_def_here; } With a good meta language, even this would be _at_least_ as easy as with the C preprocessor. The "some_meta_language_def_here" has to be easy to write, easy to understand, even if the language is capable of Awesome Deep Stuff too. You want printable enums? "Do this." You want reversed enums? "Do that." You want a recursive compile time generated static tree structure to hold the Universal Pointers with a Riemann-Lembowski hyperbolic naturalised reverse transform in 9 dimensions? "Well, do this, and then type the definition in from your textbook." You want enums where more than one enum-word has the same integer value? "Write it as below:" enum E { some_meta_language_def_here; FOO BAR BARF NOO NAR NARF NIG NAG NUG BOOBOO AUX_BOOBOO NOTHER_BOOBOO undefine_the_meta_def_here; } And you'll get enums where FOO == BARF, NIG != NAG. BTW, a lot of what Matthew, others (and even I) have suggested lately could be implemented with this meta language. Carrying the thought a little further, once this language exists, then it might be worthwhile to refactor things between D and meta. For example, the current templates, mixins, and possibly other stuff, could be moved from D itself to "defined in terms of the metalanguage". This may make the compiler core more simple and faster, for all I know. New D features could _all_ be created in the metalanguage, at first. If it later turns out that they fit better in D itself, then so be it. The user should not have to need to know.
Mar 10 2005
prev sibling parent Georg Wrede <georg.wrede nospam.org> writes:
Anders F Björklund wrote:
 Georg Wrede wrote:
 D meta language is an important thing. But now it is also urgent.


 What would the "D meta language" be ? 

The way I see it now, having this meta language in D would not even slow down DMD at all, as long as you don't use meta constructs! All that would be needed is a few more reserved words in the language. Unless they are encountered, then DMD speed should stay exactly the same. (I.e. industry shattering!) Of this I'm sure of. If a local meta definition is encountered, then compiling would be slower for the part of source code where the definition is in scope. With a lot of global scope definitions, compiling would be noticeably slower. But still a lot faster than C++. Speaking of which, there is no reason at all to have definitions with a global effect. Locality, code hiding, etc. should be obeyed here too.
Mar 11 2005
prev sibling next sibling parent reply Norbert Nemec <Norbert Nemec-online.de> writes:
Georg Wrede schrieb:
 I've been thinkin hard on a meta language.
=20
 I've talked with folks, amongst them prof. Jaakko J=E4rvi, who actually=

 met Walter at the C++ meeting last fall.
=20
 Bad news: I seem to have spilled the beans, big time. After asking his =

 opinion about some of the core concepts I'd been cooking, it occurred t=

 me that -- this guy's office is down the hall from Bjarne Stroustrup!
=20
 If these guys stop dwelling _within_ the C++ template jungle, and start=

 looking at it from above, then it'll be just 18 months before one of=20
 them holds a ground breaking lecture.   :-(   :-(

Don't worry. The basic idea has been around for some time already.=20 Within C++, it will be hard to go far beyond the current state without=20 seriously breaking compatibility. It can only be new languages that can=20 take the next step.
Mar 09 2005
parent reply Georg Wrede <georg.wrede nospam.org> writes:
Norbert Nemec wrote:
 Georg Wrede schrieb:
 
 I've been thinkin hard on a meta language.

 I've talked with folks, amongst them prof. Jaakko Järvi, who actually 
 met Walter at the C++ meeting last fall.

 Bad news: I seem to have spilled the beans, big time. After asking his 
 opinion about some of the core concepts I'd been cooking, it occurred 
 to me that -- this guy's office is down the hall from Bjarne Stroustrup!

 If these guys stop dwelling _within_ the C++ template jungle, and 
 start looking at it from above, then it'll be just 18 months before 
 one of them holds a ground breaking lecture.   :-(   :-(

Don't worry. The basic idea has been around for some time already. Within C++, it will be hard to go far beyond the current state without seriously breaking compatibility. It can only be new languages that can take the next step.

Of course I worry. Guys of this caliber do things "right". And once they've figured out a replacement to their current pp/tl system, the new one will take care of the old pp/tl syntax just with a snap of fingers. The goal, after all, once they start thinking "outside the box" is to create a universal meta language. For such a language it is a piece of cake to handle all of the current crap. So, they'll get their Backward Compatibility, but this time as a _mere_ special case -- just one small thing among the tousands of grand things it'll handle. One bad (for us, that is) thing that follows from this, is that once they've done this, then, given time, folks will rewrite the current STL, BOOST, etc. with the new syntax (possibly even skipping the bla<foo<bar>> notation), and this will inevitably result in a vast speed-up of compilation!!!! So, I'm worried as hell. For D. But for Human Kind this is of course a Good Thing(TM). ----------- If we have a smarter language to begin with (like D), then right now there is a window of opportunity for fame and reputation for D. But the window won't stay open for good. After that our meta language will be "just another thing somebody does `like what's done in C++`". Farewell, publicity, fame, hype, exponential sales. Back to our niche. My real goof was in waking up this guy, from "dwelling withing the template jungle" to looking at it "from above", "outside the box". Men of this kind don't need more than that.
Mar 10 2005
next sibling parent reply Andy Friesen <andy ikagames.com> writes:
Georg Wrede wrote:
 My real goof was in waking up this guy, from "dwelling withing the 
 template jungle" to looking at it "from above", "outside the box". Men 
 of this kind don't need more than that.

Don't worry, you haven't pointed out anything he didn't know already. Lisp has been doing this for years now. The majority of the syntax of the Nemerle language <http://nemerle.org> is implemented as macros. (even stuff like 'if' statements) Also, Daveed Vandevoorde had a quasi-working implementation of such an extension for C++ as far back as 2003: <http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2003/n1471.pdf> -- andy
Mar 10 2005
parent Georg Wrede <georg.wrede nospam.org> writes:
Andy Friesen wrote:
 Georg Wrede wrote:
 
 My real goof was in waking up this guy, from "dwelling withing the 
 template jungle" to looking at it "from above", "outside the box". Men 
 of this kind don't need more than that.

Don't worry, you haven't pointed out anything he didn't know already. Lisp has been doing this for years now. The majority of the syntax of the Nemerle language <http://nemerle.org> is implemented as macros. (even stuff like 'if' statements) Also, Daveed Vandevoorde had a quasi-working implementation of such an extension for C++ as far back as 2003: <http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2003/n1471.pdf>

Thanks! Already peeked at 'em. Gonna take a closer look.
Mar 11 2005
prev sibling parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Georg Wrede" <georg.wrede nospam.org> wrote in message 
news:42300F90.40600 nospam.org...
 Norbert Nemec wrote:
 Georg Wrede schrieb:

 I've been thinkin hard on a meta language.

 I've talked with folks, amongst them prof. Jaakko Järvi, who 
 actually met Walter at the C++ meeting last fall.

 Bad news: I seem to have spilled the beans, big time. After asking 
 his opinion about some of the core concepts I'd been cooking, it 
 occurred to me that -- this guy's office is down the hall from 
 Bjarne Stroustrup!

 If these guys stop dwelling _within_ the C++ template jungle, and 
 start looking at it from above, then it'll be just 18 months before 
 one of them holds a ground breaking lecture.   :-(   :-(

Don't worry. The basic idea has been around for some time already. Within C++, it will be hard to go far beyond the current state without seriously breaking compatibility. It can only be new languages that can take the next step.

Of course I worry. Guys of this caliber do things "right". And once they've figured out a replacement to their current pp/tl system, the new one will take care of the old pp/tl syntax just with a snap of fingers. The goal, after all, once they start thinking "outside the box" is to create a universal meta language. For such a language it is a piece of cake to handle all of the current crap. So, they'll get their Backward Compatibility, but this time as a _mere_ special case -- just one small thing among the tousands of grand things it'll handle. One bad (for us, that is) thing that follows from this, is that once they've done this, then, given time, folks will rewrite the current STL, BOOST, etc. with the new syntax (possibly even skipping the bla<foo<bar>> notation), and this will inevitably result in a vast speed-up of compilation!!!! So, I'm worried as hell. For D. But for Human Kind this is of course a Good Thing(TM). -----------

I'd chill out. I have not the slightest doubt that Walter's smart enough to do this. And I'm confident it'll be out before any C++ implementations. But maybe I'm a starry-eyed optimist. :-)
Mar 10 2005
next sibling parent clayasaurus <clayasaurus gmail.com> writes:
Matthew wrote:
 "Georg Wrede" <georg.wrede nospam.org> wrote in message 
 news:42300F90.40600 nospam.org...
 
Norbert Nemec wrote:

Georg Wrede schrieb:


I've been thinkin hard on a meta language.

I've talked with folks, amongst them prof. Jaakko Järvi, who 
actually met Walter at the C++ meeting last fall.

Bad news: I seem to have spilled the beans, big time. After asking 
his opinion about some of the core concepts I'd been cooking, it 
occurred to me that -- this guy's office is down the hall from 
Bjarne Stroustrup!

If these guys stop dwelling _within_ the C++ template jungle, and 
start looking at it from above, then it'll be just 18 months before 
one of them holds a ground breaking lecture.   :-(   :-(

Don't worry. The basic idea has been around for some time already. Within C++, it will be hard to go far beyond the current state without seriously breaking compatibility. It can only be new languages that can take the next step.

Of course I worry. Guys of this caliber do things "right". And once they've figured out a replacement to their current pp/tl system, the new one will take care of the old pp/tl syntax just with a snap of fingers. The goal, after all, once they start thinking "outside the box" is to create a universal meta language. For such a language it is a piece of cake to handle all of the current crap. So, they'll get their Backward Compatibility, but this time as a _mere_ special case -- just one small thing among the tousands of grand things it'll handle. One bad (for us, that is) thing that follows from this, is that once they've done this, then, given time, folks will rewrite the current STL, BOOST, etc. with the new syntax (possibly even skipping the bla<foo<bar>> notation), and this will inevitably result in a vast speed-up of compilation!!!! So, I'm worried as hell. For D. But for Human Kind this is of course a Good Thing(TM). -----------

I'd chill out. I have not the slightest doubt that Walter's smart enough to do this. And I'm confident it'll be out before any C++ implementations. But maybe I'm a starry-eyed optimist. :-)

I remember Walter saying something about using '$' for something big, like meta language. Besides, what would be so bad if D sat back, watched it happened, then found a better way to add it when it is ready?
Mar 10 2005
prev sibling parent reply Georg Wrede <georg.wrede nospam.org> writes:
Matthew wrote:
 "Georg Wrede" <georg.wrede nospam.org> wrote:

The goal, after all, once they start thinking "outside the box" is to 
create a universal meta language. For such a language it is a piece of 
cake to handle all of the current crap.

So, they'll get their Backward Compatibility, but this time as a 
_mere_ special case -- just one small thing among the tousands of 
grand things it'll handle.


So, I'm worried as hell.

I'd chill out. I have not the slightest doubt that Walter's smart enough to do this. And I'm confident it'll be out before any C++ implementations.

Course he is smart enough! But isn't this newsgroup here to let ideas cook, folks find the early worm, ask the Right Questions, etc.? (Not forgetting helping newbies, too.)
 But maybe I'm a starry-eyed optimist. :-)

Somehow I get the feeling there _is_ a meta language project, already well rolling forward -- behind the scenes. For some reason it's been awful quiet around here about its specifics. Could be a lot of reasons. Maybe we'd crowd it with all these ideas, maybe he wants some peace with it, maybe Walter wants to save it for a Grand Opening, maybe it is "secret" to avoid embarrasment in case it folds? Or, he's talking with superhumans in the academia and other Einstein/MahreshYogi look-alike hermits, the only ones who've Received the Knowledge from Above. If this is the case, then I'd better shut up. I could always try to cook something up, and if it tastes good, then maybe try to earn some money with it? The it'd of course be a pure preprocessor.
Mar 11 2005
parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
 Matthew wrote:
 "Georg Wrede" <georg.wrede nospam.org> wrote:

The goal, after all, once they start thinking "outside the box" is to 
create a universal meta language. For such a language it is a piece 
of cake to handle all of the current crap.

So, they'll get their Backward Compatibility, but this time as a 
_mere_ special case -- just one small thing among the tousands of 
grand things it'll handle.


So, I'm worried as hell.

I'd chill out. I have not the slightest doubt that Walter's smart enough to do this. And I'm confident it'll be out before any C++ implementations.

Course he is smart enough! But isn't this newsgroup here to let ideas cook, folks find the early worm, ask the Right Questions, etc.? (Not forgetting helping newbies, too.)

It is. My point was that you seem to be over-reacting/stating. I mean, let's be honest, D's not going to kick off C++ or Java/.NET in a single stroke, no matter how killer a feature it has. Any migration from them to it would take time, and be based on multipart user judgements. Even if D, right now, had the most amazing meta language ever seen, it wouldn't bring zillions of people over because (i) the language still has non-trivial flaws, (ii) the runtime library is still woefully inadequate, and (iii) MP is not all that important to most people (and I'd hazard a guess to say it never will be). None of that is to say it's not desirable, and won't be a big component of the overall eventual picture of attractiveness. I just don't think it's anything to panic about.
 But maybe I'm a starry-eyed optimist. :-)

Somehow I get the feeling there _is_ a meta language project, already well rolling forward -- behind the scenes. For some reason it's been awful quiet around here about its specifics.

I think there might. Walter's a thinker as well as a language porker. (Nomenclature: v. pork. To mess with, to hack. Can imply a degree of recklessness. Usage "pork away [on that library]". Derivation: my friends Leigh and Scott Perry, who were unfailingly bemused at my desire to get in and fix libraries which were flawed but for which they had adequate workarounds. <g>)
 Or, he's talking with superhumans in the academia and other 
 Einstein/MahreshYogi look-alike hermits, the only ones who've Received 
 the Knowledge from Above.

Well, I wasn't aware that there were any superhumans in academia. Maybe you're referring to people who use terminology with which you're unfamiliar? But that doesn't make them superhuman. (In fact, the more I learn about the rarefied heights of the programming world, the less impressed I am with anyone's omnipotence. Most acute, naturally, being my own. <g>)
 If this is the case, then I'd better shut up. I could always try to 
 cook something up, and if it tastes good, then maybe try to earn some 
 money with it? The it'd of course be a pure preprocessor.

Please don't shut up. You've intrigued me enormously about possibilities. It's just that I am so inexperienced in such matters - at least from a generic (not C++ TMP) perspective - that I am not able to grasp your intent without suitably simple examples. <g>
Mar 11 2005
parent reply Georg Wrede <georg.wrede nospam.org> writes:
Matthew wrote:
Matthew wrote:
"Georg Wrede" <georg.wrede nospam.org> wrote:
So, they'll get their Backward Compatibility, but this time as a 
_mere_ special case -- just one small thing among the tousands of 
grand things it'll handle.
So, I'm worried as hell.

I'd chill out. I have not the slightest doubt that Walter's smart enough to do this.

cook, folks find the early worm, ask the Right Questions, etc.? (Not forgetting helping newbies, too.)


A (delusional) sense of urgency, importance, and paranoia. That has advanced the mankind, on several (actually most) occasions. Got us to the Moon, gave us (non-americans anyway) serious encryption, nuclear technology, good computers, submarines, GPS, teflon, solar panels. And Ada. Complacency, that's what kills a language! (Quote of the day(TM))
 I mean, let's be honest, D's not going to kick off C++ or Java/.NET in a 
 single stroke, no matter how killer a feature it has.

I was thinkin more like the "extra" fame and hype D would get for doing something right, before the others. More press exposure won't hurt. This "right" is, of course (so far) only my personal belief. Hopefully others will agree, in time. (Ha, and if I'm on a completely wrong track here, I wish to Bob somebody would kick down the stack of cartons I'm trying to lecture from. Before this becomes too public!! ) ?Ha?s/on a completely wrong/not on precisely the right/ :-) Right: Turing complete. Two-level. Infinitely extensible (honest!). Not only descriptive but procedural too. In practice: No must-use. Comprehensible to the casual, or the newbie. Practical. Practical. Practical. Without self-promotional esotheric bells and whistles -- just get the job done. Any job. I ADMIT THIS PLATE IS A BIT FULL. (No shit?!) But I have to admit, by now this is more than wishful thinking. I have (eh, like Walter "may be") some actual ideas on how to implement this. Why else would I sound so confident, even to a few specific questions by folks.
 Any migration from 
 them to it would take time, and be based on multipart user judgements. 
 Even if D, right now, had the most amazing meta language ever seen, it 
 wouldn't bring zillions of people over 

Actually, we live in the 21st century now. Times change, my friend. Obstacles are only as hard as you think. (Took me almost 50 years :-( to figure out.) Today we have something there's never been: we can turn 400 million heads overnight. If 0.001% of these start actually thinking, we'll be well off. Say there's a big article in PC-Magazine, that essentially says "we really think this is the language the industry will use in two years." Well, nothing will happen for a while. Six months from then, Walter's download server will crash. Not enough, though. But have editorials saying essentially the same in DDJ and CUJ, hopefully one month after the PC-Magazine one, and we fly.
 because 
 (i) the language still has non-trivial flaws,
 (ii) the runtime library is still woefully inadequate, and 
 (iii) MP is not all that important to most people 
 (and I'd hazard a guess to say it never will be).

(i) We need to keep PC-Magazine ignorant of us till that's fixed. :-) (ii) Now, that's something that will not proceed fast enough the way things currently are. We need to CHANGE THE SETUP around it. Do something DRASTIC. (Put it on dsource, or sourceforge, whatever! But something really has to be done to this, now! Projects on SourceForge have to have an "owner", so Walter wouldn't eve lose control!) Lots of talent just seeking a worthwhile, interesting project, with a long time future. (iii) I fully and completely agree. MP is not for the average user. While easy things will be easy (like the X example in this thread), the regular (even professional) programmer will never use it. MP (at least the way I see our concoction becoming used) is for corporate use, research, libraries, D development(!) -- and should be used invisibly to most things. It will find its use in a massive reduction of the size of app-field specific libraries, reducing complicity when doing "hairy stuff", and for programming language research. (Issues like overloading on return type, unquoted regexps as anonymous functions, a serious string type or class -- or just regular D features investiagetd for inclusion or exlcusion.) Most regular programmers will never know they've been using it! In time some things regularly done with MP will be incorporated directly in the D "core" (for speed of compilation, others just because the MP usage has shown them to be practical and useful (a new keyword, a new operator, new syntax for something)).
 None of that is to say it's not desirable, and won't be a big component 
 of the overall eventual picture of attractiveness. I just don't think 
 it's anything to panic about.

There's psychology here too. I used to sell SLR cameras at a major department store. We sold Canon, Nikon, Minolta, Olympus, and a half dozen now extinct brands. Customers dithering between two cameras ended up choosing the one with which "you could do this" that you couldn't do with the other. Most of them never noticed this themselves!! Usually it was a feature (both they and I knew, but did't say) they'd never get around to using. But ultimately the choice was formulated as something "non-childish", like durability, long-term availability of accessories, quality, etc. And if the chosen one just happened to cost less than the other, then Economy was the Thing. (Wife'll be less angry!) (Oh, and if economy was the real reason, then the pretense was of course whatever else.) I've participated in a few language shoot-outs. (University IT department curriculum planning meeting, customer board room, regular consulting with customers, mens' room, the local pub.) Any non-obvious choices end up like the SLR sales: ultimately the "cool thing" the other one doesn't have -- whether that is relevant to the choice at hand or not. And believe it or not, the people who (by profession, or by position) definitely should not succumb to this, they're actually the worst. We need to use this to our advantage. It's there for the taking! Even the moron who'll _never_ be able to grasp even recursion, has to brag to his equal friend that he's Chosen the "infinitely extensible D" programming language. Ditto the Pointy Haired boss over beer at the Country Club.
But maybe I'm a starry-eyed optimist. :-)

Somehow I get the feeling there _is_ a meta language project, already well rolling forward -- behind the scenes. For some reason it's been awful quiet around here about its specifics.

I think there might. Walter's a thinker as well as a language porker.

Then I might be happy. And not alone in the world.
Or, he's talking with superhumans in the academia and other 
Einstein/MahreshYogi look-alike hermits, the only ones who've Received 
the Knowledge from Above.

Well, I wasn't aware that there were any superhumans in academia.

Alfred Aho, Edsger Dijkstra, Donald Knuth, Linus Torvalds, Roger Penrose, Alan Turing....... Not granting Superhuman Status to those who deserve it -- is arrogant. :-) :-P OTOH, you might visit several hundred universities, and not find a single one. And that's a disgrace!
 Maybe 
 you're referring to people who use terminology with which you're 
 unfamiliar? 

LOL! I know what you mean! I had the good fortune to know my mother's cousin, who compulsively tried to pull that one off. At age 11 I noticed! The best thing was, mom had a lot of respect for him, and bought most of this crap -- so I couldn't shoot him down. I learned to "not notice". Man, the lot of fun I've had through the years with these guys. And gals. Both in and out of academia. And none of them know!
 But that doesn't make them superhuman. (In fact, the more I 
 learn about the rarefied heights of the programming world, the less 
 impressed I am with anyone's omnipotence. Most acute, naturally, being 
 my own. <g>)

I was thinking of a few names I expressly did not list above. There's a particular example, a guy who is undeniably accomplished, who's known to everyone in the non-profit programming world, with a huge reputation. We sat a few hours drinking beer after his lecture at an academic convention. I was shattered to discover that he actually is a superhuman-wannabe, a man of lust and avarice, envy, and a petty soul. (Maybe I should have spelled it like "superhuman wannabe"!) <humor_intent quality="lame"> Omnipotence... not for us, but for D !! </humor_intent>
If this is the case, then I'd better shut up. I could always try to 
cook something up, and if it tastes good, then maybe try to earn some 
money with it? The it'd of course be a pure preprocessor.

Please don't shut up. You've intrigued me enormously about possibilities. It's just that I am so inexperienced in such matters - at least from a generic (not C++ TMP) perspective - that I am not able to grasp your intent without suitably simple examples. <g>

Humble! :-) Thanks! I'm relieved to hear this intrigues. I wish, if other people read this thread, that they'd at least be entertained, stimulated, and possibly motivated. ------------ Corollary: I had a career revelation. The day FADPL (the Foundation for the Advancement of the D Programming Language, a US non-profit with tax-exempt status, later backed by Lawrence Livermore, GM, DoE, and others, hopefully to be started by somebody, some day), is looking for paid D evangelists, I'll submit my resume. They could do a lot worse than hiring me. :-)
Mar 12 2005
parent reply xs0 <xs0 xs0.com> writes:
 But I have to admit, by 
 now this is more than wishful thinking. I have (eh, like Walter "may 
 be") some actual ideas on how to implement this. Why else would I sound 
 so confident, even to a few specific questions by folks.

Well, for Bob's sake, what exactly do you want to implement and how do you propose it is done? :) xs0
Mar 13 2005
parent reply Georg Wrede <georg.wrede nospam.org> writes:
xs0 wrote:
 But I have to admit, by now this is more than wishful thinking. I have 
 (eh, like Walter "may be") some actual ideas on how to implement this. 
 Why else would I sound so confident, even to a few specific questions 
 by folks.

Well, for Bob's sake, what exactly do you want to implement and how do you propose it is done? :)

Right now I'm trying to find a D grammar somewhere. I'd like to add a couple of keywords to D. They'd be for defining meta things, and undoing the definitions. (At least in the beginning there'll be a define and an undefine. Later, with more experience and confidence, lexical scoping would work "automatically". Even more later global things could be defined. And after that, maybe local disabling of global things. Then I wonder about this-file-only defines, how this should work with importing files, and other, eh, more challenging obstacles.) (Oh, I forgot, there are no Obstacles, only Interesting Things Encountered on the way! :-O ) I'm trying to find a language to use in this pre-prototyping (in reality I'll do 1-screenful short programs that get D source code, and output D source code (just like any preprocessor). While I'm toying around (mostly to see what problems I've totally missed), with awk, sed, grep, possibly ed, maybe D, Euphoria, and whatever other seems to be nice for the particular mini thing, I am trying to figure out how to do it for real. (Heck, I may even end up searching around for my Perl book.) Do what? Well, a pre-alpha version of a proof-of-concept kind of prototype. Or, if it looks like too much, then I'll try to make a couple different protorypes, each of which would demonstrate some part of the whole. Something for #define X(a,b) foo(b,a);bar(a);blabla(a,b,a,b) kind of trivial C preprocessor things. Something else for having a snippet of self-made language inside a procedure instead of writing that too with D (see the sugar-cube robotics example in this thread). And then maybe one or a couple of different try-outs for the hairy stuff. With that I dream you could do Lisp like stuff, that's really for "the superhuman". That's a bit hard, since I'm not. :-( ----- For these pathetic first attempts I'd either make small one-trick-preprocessors, or maybe later get a GDC version that I can compile, and then break the parser and lexer "trying to play Walter"! ;-) -- With neither his skills, experience, nor abilities. If any of these one-trick things does anything useful at all, then I'll post them for others to laugh at. Then at least folks will get a glimpse of some parts of the whole. With any luck there'll be discussion and feedback, to keep us going on! If we haven't given up by this time, then we'll try to do the same with some self-made-language inserters. Probably the really first ones will have some arbitrary but fixed language as the "inserted language", later the testers (you gyus) get versions actually capable of doing minilanguages defined in some input file by each of you -- for using and abusing them. If we're still at it at this point, then it's time for the hairy stuff. I don't know how we'll test that, without asking some heavyweight hairy-stuffers. I mean, even if a particular preprocessor happened to actually work, where do we find the kind of guys who are actually capable of trying any hairy stuff on them? Or giving meaningful comments?? Phew! If we're /still/ here, then it'd be time to make a proposal for Walter.
Mar 13 2005
parent xs0 <xs0 xs0.com> writes:
After some thinking, I determined this has to be a pre/postprocessor, 
unless the compiler is completely redone to support dynamic grammar. 
Because, you obviously want to be able to define new tokens (like 
"aspect" or "-=>") and new syntax, but by the time the compiler is aware 
of that, parsing has already happened (and most probably failed). As for 
the scope, it would be nice to simply have two constructs, like #define 
for globals and #local ... #{    #}  for locals.. #define should perhaps 
be renamed to make an obvious distinction from C's..

Hmm, after some more thought, it may make sense to move new syntax 
definitions completely out of D source, where they'll have no 
constraints/conflicts with the existing syntax, and it can be used with 
something like

#extension(aspect)

in the source. Local stuff will be somewhat limited then, but I guess it 
would be mostly used for stuff like the X macro, which shouldn't be too 
hard, and perhaps for mini operators, like -=>. OTOH, maybe these could 
be skipped altogether? You can write the extension normally and use 
something like

#with(graphOperator) {{

#}}


xs0

Georg Wrede wrote:
 xs0 wrote:
 
 But I have to admit, by now this is more than wishful thinking. I 
 have (eh, like Walter "may be") some actual ideas on how to implement 
 this. Why else would I sound so confident, even to a few specific 
 questions by folks.

Well, for Bob's sake, what exactly do you want to implement and how do you propose it is done? :)

Right now I'm trying to find a D grammar somewhere. I'd like to add a couple of keywords to D. They'd be for defining meta things, and undoing the definitions. (At least in the beginning there'll be a define and an undefine. Later, with more experience and confidence, lexical scoping would work "automatically". Even more later global things could be defined. And after that, maybe local disabling of global things. Then I wonder about this-file-only defines, how this should work with importing files, and other, eh, more challenging obstacles.) (Oh, I forgot, there are no Obstacles, only Interesting Things Encountered on the way! :-O ) I'm trying to find a language to use in this pre-prototyping (in reality I'll do 1-screenful short programs that get D source code, and output D source code (just like any preprocessor). While I'm toying around (mostly to see what problems I've totally missed), with awk, sed, grep, possibly ed, maybe D, Euphoria, and whatever other seems to be nice for the particular mini thing, I am trying to figure out how to do it for real. (Heck, I may even end up searching around for my Perl book.) Do what? Well, a pre-alpha version of a proof-of-concept kind of prototype. Or, if it looks like too much, then I'll try to make a couple different protorypes, each of which would demonstrate some part of the whole. Something for #define X(a,b) foo(b,a);bar(a);blabla(a,b,a,b) kind of trivial C preprocessor things. Something else for having a snippet of self-made language inside a procedure instead of writing that too with D (see the sugar-cube robotics example in this thread). And then maybe one or a couple of different try-outs for the hairy stuff. With that I dream you could do Lisp like stuff, that's really for "the superhuman". That's a bit hard, since I'm not. :-( ----- For these pathetic first attempts I'd either make small one-trick-preprocessors, or maybe later get a GDC version that I can compile, and then break the parser and lexer "trying to play Walter"! ;-) -- With neither his skills, experience, nor abilities. If any of these one-trick things does anything useful at all, then I'll post them for others to laugh at. Then at least folks will get a glimpse of some parts of the whole. With any luck there'll be discussion and feedback, to keep us going on! If we haven't given up by this time, then we'll try to do the same with some self-made-language inserters. Probably the really first ones will have some arbitrary but fixed language as the "inserted language", later the testers (you gyus) get versions actually capable of doing minilanguages defined in some input file by each of you -- for using and abusing them. If we're still at it at this point, then it's time for the hairy stuff. I don't know how we'll test that, without asking some heavyweight hairy-stuffers. I mean, even if a particular preprocessor happened to actually work, where do we find the kind of guys who are actually capable of trying any hairy stuff on them? Or giving meaningful comments?? Phew! If we're /still/ here, then it'd be time to make a proposal for Walter.

Mar 13 2005
prev sibling next sibling parent reply "Lionello Lunesu" <lio lunesu.removethis.com> writes:
Just for my understanding: would a metalanguage allow you to define new 
operators on the fly?

This way, the current D language would in fact be an implementation using 
(its own) metalanguage for defining the existing operators and their 
precedence?

Lionello. 
Mar 11 2005
parent reply Georg Wrede <georg.wrede nospam.org> writes:
Lionello Lunesu wrote:
 Just for my understanding: would a metalanguage allow you to define new 
 operators on the fly?

Yes. Suppose you are into Graph Theory, and you need to create a lot of Directed Graphs. You could do the meta definition of the operator right where it is needed, or you could have it in a library for regular use. Let's take the local case. Class Vertex { Vertex[] toVertices, fromVertices; public addTo(Vertex v) { toVertices ~= v } public addFrom (Vertex v) { fromVertices ~= v } } void main () { Vertex a = new Vertex; ... and b,c,d,e.... Now, we need to create the connections. For this we need an operator, let's call it "-=>" and define it. metadefinition ( lkjdsallkjakjsf ); //Nobody knows the syntax Then just go on with the graph: a -=> b; c -=> d; d -=> a; c -=> b; } // end main. Our connecting operator is in scope only in main itself, so we won't pollute all of the application source code. The meta definitions should obey normal D scoping rules. ///////////////////////////////// Now, let's take the module case. module foo.bar; meta myGraphOperator (lkjsdfljsadl); // metadefinition, that // defines the operator -=> // still unknown syntax. :-) Production code would be: import foo.bar; Class Vertex { Vertex[] toVertices, fromVertices; public addTo(Vertex v) { toVertices ~= v } public addFrom (Vertex v) { fromVertices ~= v } } void main () { Vertex a = new Vertex; ... and b,c,d,e.... // Now, we need to create the connections. // For this we need the operator. foo.bar.myGraphOperator; // get it into this scope // Then just go on with the graph: a -=> b; c -=> d; d -=> a; c -=> b; } // end main.
 This way, the current D language would in fact be an implementation using 
 (its own) metalanguage for defining the existing operators and their 
 precedence?

I think it could. Of course, compiling would then be slower!
Mar 11 2005
parent "Lionello Lunesu" <lio lunesu.removethis.com> writes:
Thanks a lot for your example! I finally know what all the (totally 
justified) fuzz is about!

Thanks again..

Lionello. 
Mar 13 2005
prev sibling parent reply Norbert Nemec <Norbert Nemec-online.de> writes:
Georg Wrede schrieb:
 I've been thinkin hard on a meta language.

Word from the wise: Don't try to reinvent the wheel but learn from those to tried before (even if they failed) There are loads of programming languages or add-ons do existing languages out there that allow meta-programming in some way. I don't have all the references at hand that I have read over the past years. There have been thorough scientific investigations of the matter. I doubt it is possible to outstrip those projects by starting from scratch. The point of D is not to give it the perfect features for meta-programming but designing features in such a way that they are practical, fit nicely into the general language and don't get in the way with regular programming. A preprocessor is not such a great idea, since it will generally split the language into two levels causing all kinds of problems. Walter had good reason to drop the concept of a preprocessor. In general, the big pieces of the meta-language already exist in D: templates. These already contain a complete functional programming language that can be used for meta-programming. Several pieces are missing for full comfort and usability, but these are details that have to be fixed. The meta-language formed by templates certainly is not very handy when trying to do regular calculations at compile time, but then, it has to be clear that this is not the point of a meta-language. Meta-programming has the very specific purpose of doing code transformations and for this, templates are excellently designed. It is not a general functional language, but a highly specialized language. In principle, you can do anything, but going beyond the special purpose of the meta-language will be both awkward and inefficient. Now: if you can identify single weak points of the existing meta-language and propose clear solutions, these may have a chance. For doing a complete redesign of templates, the language is a bit too mature, though. Ciao, Norbert
Mar 14 2005
parent reply Georg Wrede <georg.wrede nospam.org> writes:
Norbert Nemec wrote:
 Georg Wrede schrieb:
 
 I've been thinkin hard on a meta language.

Word from the wise: Don't try to reinvent the wheel but learn from those to tried before (even if they failed) There are loads of programming languages or add-ons do existing languages out there that allow meta-programming in some way. I don't have all the references at hand that I have read over the past years. There have been thorough scientific investigations of the matter. I doubt it is possible to outstrip those projects by starting from scratch.

Thanks. I welcome this and other criticisms. Really! The harder, the better. (If anybody can actually convince me that this is either impossible or useless, I'll be thankful. The sooner, the more thankful. Then I could stop wasting time and bandwidth, and maybe find something else where I feel I can contribute to D. :-) ) If you happen to find some (especially in your own opinion) good references, I'd appreciate. Both of successful things, and those explaining what or why something should not be done, or is downright proved impossible.
 The point of D is not to give it the perfect features for 
 meta-programming but designing features in such a way that they are 
 practical, fit nicely into the general language and don't get in the way 
 with regular programming.

I fully agree with all of that. If somebody wants to do ultra heavy duty meta programming, they probably would do it in Lisp anyway. (Or something.) However, what I'm _specifically_ out to get, is a syntax that is not derived from currently existing template language solutions. (D, C++, and the like.) I can't help feeling that path is a dead end. While easier for trivial (or at least not too complicated) template transforms, I somehow feel (lack of Ph.D. in this area confines me to a feelings-only basis here, appologies to all), that such an approach turns counter productive already in moderately advanced contexts. (Already the Recursive Templates example (http://www.digitalmars.com/d/template.html) seems unnecessarily complicated to me. You need to use two separate templates, and an enum. And all for something that I wish could be expressed as simply as the non-template recursive function. Not to mention, that I (hope) I'm not the only one who has to look hard at what's being done there, even if recursion itself is very familiar to me. Every language is like a person. It appears one way at the surface, and once you really get to know them, then you (most often) realise that some of the things you thought one could do or couldn't, actually are the other way around. (I wrote about that at length in my Master's Thesis.) The old amongst us remember Mbasic. (That was before MS-DOS and the PC. Mbasic was by Microsoft, for the CP/M operating system.) My personal _feeling_ was that programming effort grew exponentially with program size. When I found Turbo Pascal, the effort seemed to grow logarithmically! (I know this is not so, but I'm talking about the feeling. Oh, the joy!) With templates, I again have a feeling. And that is, effort grows too fast. (Unscientific, I admit!) Doing easy stuff is easy, even next-to-easy is next-to-easy. But already moderate is hard. NOTE: I think meta and the existing template system, are not mutually exclusive! Even if you can do some things in both.
 A preprocessor is not such a great idea, since it will generally split 
 the language into two levels causing all kinds of problems. Walter had 
 good reason to drop the concept of a preprocessor.

True. For the time being, however, when we toy around with what may be done with meta language, it seems like less work to have it as a preprocessor. I fully agree that (if done at all) this should be done respecting Walter's ideas: - context free grammar - no preprocessor - straightforward and practical
 In general, the big pieces of the meta-language already exist in D: 
 templates. These already contain a complete functional programming 
 language that can be used for meta-programming. Several pieces are 
 missing for full comfort and usability, but these are details that have 
 to be fixed.

Yes. And my quest here should in no way slow down that work!
 The meta-language formed by templates certainly is not very handy when 
 trying to do regular calculations at compile time, but then, it has to 
 be clear that this is not the point of a meta-language.

Strictly speaking then, should we call this, er, "quest", something else than meta? I'd have no problem with that. (I assume this thread has given a pretty good idea to everybody of what I'm aiming at.)
 Meta-programming 
 has the very specific purpose of doing code transformations and for 
 this, templates are excellently designed. It is not a general functional 
 language, but a highly specialized language. 

Code transformations, yes. But I want one to be able to do arbitrary code transformations. (Yes, yes, I know, this is reaching for the stars, a wild goose chase. And even if we pull this off, the documentation has to explicitly state that this is not _supposed_ to be used for everything. This _should_ be used sparingly, if at all. And only when templates, modules, OO, etc. fail -- and even then "don't try this at home". But at this point, know that your job _will_ get done with this.) Of course, already one could download (Windows people, Others have these already) awk, sed, perl, scheme, and do code transformations limited only by their own imagination. But I want more. To reach this "more", this needs to be inside a language. (Just like Walter already did with the templates!) Only then can we get some serious value-add. (( Not to mention, if this was entirely a preprocessor thing, then the Other Guys could use it on their out-dated language! :-) )) It's a matter of principle. This thing has to be without arbitrary limits. Just like a word processor with "unlimited line length, unlimited file size". Everybody knows that this doesn't work: $ wget -O - -r -l 9999 "ftp://archive.loc.gov/texts" | $EDIT - no matter how "unlimited" the editor is in theory, or what a monster mainframe you're currently using. Still it matters for many practical purposes that you have "unlimited ll/fs". Same with what I'm pursuing. Nobody wants to have the compiler invent the entire source code of the Google engine. But we don't want arbitrary limits here either.
 In principle, you can do anything, 

I assume the example applications of meta in this thread would not be included in this "anything"?
 but going beyond the special purpose of the meta-language will 
 be both awkward and inefficient.

I wish I can prove different. (I'm not sure at all, but I'm making a hell of a try at it.)
 Now: if you can identify single weak points of the existing 
 meta-language and propose clear solutions, these may have a chance.

I will, if I find any.
 For doing a complete redesign of templates, the language is 
 a bit too mature, though.

Yes, let's not tear down what we already have.
Mar 15 2005
next sibling parent reply Matthias Becker <Matthias_member pathlink.com> writes:
(Already the  Recursive Templates example 
(http://www.digitalmars.com/d/template.html) seems unnecessarily 
complicated to me. You need to use two separate templates, and an enum. 
And all for something that I wish could be expressed as simply as the 
non-template recursive function.

I just looked at the example and it is wrong! The factorial of 0 is 1. So it should be: # template factorial(int n : 0) // not 1 # { # enum { factorial = 1 } # } # template factorial(int n) # { # // Note . used to find global template rather than enum # enum { factorial = n* .factorial!(n-1) } # } Well after doing some meta-programming in C++ this simple example isn't hard to read. But how to do this in a simpler way? # metacode metaInt factorial (metaInt n) # { # if (n == 0) # return 1; # else # return n * factorial(n-1); # } Might be cool. But that is a totaly different kind of meta-programming than I thought would be discussed in this thread. I thought it would be something like "compiler-plugins" that work on the AST (abstract synthax tree), so we can define our own language constructs. I never thought about any kind of compiletime-calculation. -- Matthias Becker
Mar 15 2005
parent Georg Wrede <georg.wrede nospam.org> writes:
Matthias Becker wrote:
(Already the  Recursive Templates example 
(http://www.digitalmars.com/d/template.html) seems unnecessarily 
complicated to me. You need to use two separate templates, and an enum. 
And all for something that I wish could be expressed as simply as the 
non-template recursive function.


 Well after doing some meta-programming in C++ this simple example 
 isn't hard to read.
 
 But how to do this in a simpler way?
 
 # metacode metaInt factorial (metaInt n)
 # {
 #    if (n == 0)
 #       return 1;
 #    else
 #       return n * factorial(n-1);
 # }

Yes.
 Might be cool. But that is a totaly different kind of meta-programming than I
 thought would be discussed in this thread. I thought it would be something like
 "compiler-plugins" that work on the AST (abstract synthax tree), so we can
 define our own language constructs. 

Ultimately, I hope, this will be the case. But I have to admit, if I had created D (that is, if I were Walter :-) ) it would take an army to force me to open up the AST at compile time. I'd fear this leads to intractable error messages, false errors in plainly correct code, people believing DMD is broken, and all just because of bugs in people's own meta code. If this meta thing advances well, then that might become reality. But I wouldn't bet it'd be before D 3.0. Even with luck, 2.0 seems remote. What I do, however, think is possible, is that a kind of non-AST-messing version might be incorporated in D 2.0. This would of course be more limited than the ultimate version, but it's ok. We really do need some time to get used to this sort of thing, see what it might be applied to, see if folks use it to do something unexpectedly cool, etc. In doing that, we'll have good time to figure out a smooth and practical syntax for the 3.0 meta thing. (Of course not forgetting that none of this might ever even become reality! Creativity, perseverance, hard work, but most of all, a good dose of plain good luck is needed.)
 I never thought about any kind of compiletime-calculation.

Compile-time calculation opens a new field for us. It gives possibilities that are hard to reach otherwise.
Mar 16 2005
prev sibling parent Norbert Nemec <Norbert Nemec-online.de> writes:
Sorry, that I don't go into the details of your mail. The points you 
bring up contains much that I've considered before as well. The topic 
meta-programming is extremely deep. The more you think about it, the 
less you know what way you should go. That's probably why I never really 
stuck with one concept long enough to really elaborate it.

In any case, what you write sounds somewhat like what is tried with the 
OpenC++ project:

	http://opencxx.sourceforge.net/

I never managed go beyond scratching the surface of this, and it seems 
like it is just far too powerful to make it into mainstream development.

There also is an OpenJava project, that takes the ideas and ports them 
to Java. Obviously, the concept does not depend too much on the details 
of the language. Maybe OpenD would be just the project that you were 
thinking of?

I think the best place to start reading about OpenC++ would be the 
programmers guide by Shigeru Chiba:

	http://www.csg.is.titech.ac.jp/~chiba/pub/SPL96-024.ps.gz

I never found the time to dig into it, but maybe you are interested?
Mar 16 2005