www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP78 - macros without syntax extensions

reply "Kagamin" <spam here.lot> writes:
http://wiki.dlang.org/DIP78 - Proposal for a macro system without 
syntactical extensions to the language. Hence it doesn't allow 
arbitrary syntax.
May 26 2015
next sibling parent reply "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Tuesday, 26 May 2015 at 20:23:11 UTC, Kagamin wrote:
 http://wiki.dlang.org/DIP78 - Proposal for a macro system 
 without syntactical extensions to the language. Hence it 
 doesn't allow arbitrary syntax.
If this proposal is considered, it is required to propose to look at the implementation of macros in Nemerle. Many believe that it is in Nemerle macros implemented the most successful compared to other modern languages. Of course, the most successful macros are implemented in Lisp, but the syntax of the language is poor :) Some information here: http://www.nemerle.org/About Great article about macros Nemerle, but in Russian: http://rsdn.ru/article/nemerle/NemerleStingFormating.xml Of course, it is necessary to look at Rust and Nim, but macros Nemerle implemented better. I also think that the D needed macros, but I sure would not want to see the lite version of the macros. If implemented, then at least to the end and complete! Or not to implement macros in general... Although if you create something in between lite and full version of the macro, then I think it would be a big plus for D, so entirely without macros somehow uncomfortable. Some mixins little. Sometimes I think that D do not need these macros, and sometimes it seems to me that they are severely lacking.Although the competition is also necessary to consider the other languages that have at least some macros, such as, Nim/Rust. In general, D needs some macros.
May 26 2015
parent reply "Kagamin" <spam here.lot> writes:
On Tuesday, 26 May 2015 at 23:47:41 UTC, Dennis Ritchie wrote:
 If this proposal is considered, it is required to propose to 
 look
 at the implementation of macros in Nemerle. Many believe that it
 is in Nemerle macros implemented the most successful compared to
 other modern languages. Of course, the most successful macros 
 are
 implemented in Lisp, but the syntax of the language is poor :)
The problem with declarative macro system is that you would need to learn yet another language. Possibly turing-complete. And a declarative turing-complete language is an overkill both for usage and implementation. Imperative macros get it done in an intuitive way in the existing language.
May 27 2015
next sibling parent reply "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 08:14:36 UTC, Kagamin wrote:
 The problem with declarative macro system is that you would 
 need to learn yet another language. Possibly turing-complete. 
 And a declarative turing-complete language is an overkill both 
 for usage and implementation. Imperative macros get it done in 
 an intuitive way in the existing language.
Yes, declarative macros can destroy an imperative language, because they can change the syntax of the language beyond recognition. Ie macrosystem will be more powerful than the language itself. Unfortunately, Nemerle, Scala and turned :) But, in my opinion, there should be options to combine declarative macros with imperative languages. You just need to properly limit these macros to a certain moment. Is not it possible to combine in the desired degree of declarative macros with imperative languages?
May 27 2015
parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 27 May 2015 at 09:31:33 UTC, Dennis Ritchie wrote:
 Is not it possible to combine in the desired degree of 
 declarative macros with imperative languages?
The point is to streamline ast creation. I think, helper methods can be provided by phobos in, say, std.macros to simplify ast creation for common constructs. Well, in a way libraries are a form of DSL...
May 27 2015
parent "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 09:40:35 UTC, Kagamin wrote:
 The point is to streamline ast creation. I think, helper 
 methods can be provided by phobos in, say, std.macros to 
 simplify ast creation for common constructs. Well, in a way 
 libraries are a form of DSL...
std.macros it may, this is the right way! At least, it would suit me :)
May 27 2015
prev sibling parent reply "Idan Arye" <GenericNPC gmail.com> writes:
On Wednesday, 27 May 2015 at 08:14:36 UTC, Kagamin wrote:
 On Tuesday, 26 May 2015 at 23:47:41 UTC, Dennis Ritchie wrote:
 If this proposal is considered, it is required to propose to 
 look
 at the implementation of macros in Nemerle. Many believe that 
 it
 is in Nemerle macros implemented the most successful compared 
 to
 other modern languages. Of course, the most successful macros 
 are
 implemented in Lisp, but the syntax of the language is poor :)
The problem with declarative macro system is that you would need to learn yet another language. Possibly turing-complete. And a declarative turing-complete language is an overkill both for usage and implementation. Imperative macros get it done in an intuitive way in the existing language.
But D already has such a declarative language - the one used in template metaprogramming. I think a macro system that plays well with that template metaprogramming sub-language will be really nice. For example, CTFE that works like a macro and returns types/aliases: Auto deduceType(Auto args) { // some complex imperative code to deduce the type from the args return DeducedType; } struct Foo(T...) { deduceType(T) value; }
May 27 2015
parent reply Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 05/27/15 17:47, Idan Arye via Digitalmars-d wrote:
 On Wednesday, 27 May 2015 at 08:14:36 UTC, Kagamin wrote:
 On Tuesday, 26 May 2015 at 23:47:41 UTC, Dennis Ritchie wrote:
 If this proposal is considered, it is required to propose to look
 at the implementation of macros in Nemerle. Many believe that it
 is in Nemerle macros implemented the most successful compared to
 other modern languages. Of course, the most successful macros are
 implemented in Lisp, but the syntax of the language is poor :)
The problem with declarative macro system is that you would need to learn yet another language. Possibly turing-complete. And a declarative turing-complete language is an overkill both for usage and implementation. Imperative macros get it done in an intuitive way in the existing language.
But D already has such a declarative language - the one used in template metaprogramming. I think a macro system that plays well with that template metaprogramming sub-language will be really nice. For example, CTFE that works like a macro and returns types/aliases: Auto deduceType(Auto args) { // some complex imperative code to deduce the type from the args return DeducedType; } struct Foo(T...) { deduceType(T) value; }
That already works. Eg: alias deduceType(Args...) = typeof({ // some complex imperative code to deduce the type from the args import std.range; return mixin(iota(Args.length).map!q{`Args[`~text(a)~']'}().join("+")); }()); struct Foo(T...) { deduceType!(T) value; } static assert(is(typeof(Foo!(short, ubyte, bool).value)==int)); What all these proposals seem to be about is: a) better introspection (ie exposing a (preferably simplified and std) AST) b) AST injection c) "better" syntax d) better "optimizations", meaning skipping the emission of code and data that is never used at runtime. artur
May 27 2015
parent "Kagamin" <spam here.lot> writes:
On Wednesday, 27 May 2015 at 17:03:30 UTC, Artur Skawina wrote:
 That already works. Eg:

    alias deduceType(Args...) = typeof({
       // some complex imperative code to deduce the type from 
 the args
       import std.range;
       return 
 mixin(iota(Args.length).map!q{`Args[`~text(a)~']'}().join("+"));
    }());

    struct Foo(T...) {
       deduceType!(T) value;
    }

    static assert(is(typeof(Foo!(short, ubyte, 
 bool).value)==int));


 What all these proposals seem to be about is:

 a) better introspection (ie exposing a (preferably simplified 
 and std) AST)
 b) AST injection
 c) "better" syntax
 d) better "optimizations", meaning skipping the emission of 
 code and data
      that is never used at runtime.

 artur
Maybe extending capabilities of templates would be a better direction. They already accept aliases which are essentially simple hygienic identifier expressions.
May 28 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-05-26 22:23, Kagamin wrote:
 http://wiki.dlang.org/DIP78 - Proposal for a macro system without
 syntactical extensions to the language. Hence it doesn't allow arbitrary
 syntax.
I'm not sure how this is different for DIP50. I can see the obvious differences in the syntax how to declare a macro and so on. But otherwise they look very similar. -- /Jacob Carlborg
May 26 2015
parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 27 May 2015 at 06:54:56 UTC, Jacob Carlborg wrote:
 I'm not sure how this is different for DIP50. I can see the 
 obvious differences in the syntax how to declare a macro and so 
 on. But otherwise they look very similar.
I believe 78 allows a simpler implementation and requires less changes to the language, though I'm not sure if frontend can get it done on a purely semantical level: similar techniques like mixins and static if rely on syntax. Maybe a macro should be called with a special syntax, e.g. myAssert!(a==b);
May 27 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-05-27 10:06, Kagamin wrote:

 I believe 78 allows a simpler implementation and requires less changes
 to the language, though I'm not sure if frontend can get it done on a
 purely semantical level: similar techniques like mixins and static if
 rely on syntax.
DIP50 would require a minimal amount of syntax change, the only thing is prefixing a function with the "macro" keyword. DIP78 on the other hand, it's not so easy to see that a function declaration is actual a macro declaration. In that case I would prefer the "macro" keyword. It's already a reserved, for exactly this purpose, so it will be backwards compatible. I would say that neither of the DIP's contain a lot of detail. So it's hard to know which one is more complex. In DIP50 there's quite a lot of optional/bonus features. To me, implementing only the minimal requirements look very similar to DIP78. One thing that would be more complex in DIP50 would be the "Context" class. Although I'm not sure if that's needed in DIP78 as well.
 Maybe a macro should be called with a special syntax,
 e.g. myAssert!(a==b);
I have thought of that too. But I haven't been able to come up with a syntax that looks good and doesn't conflict with any existing syntax/symbol. The above syntax is already used for template instantiation. -- /Jacob Carlborg
May 27 2015
next sibling parent reply "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 11:53:00 UTC, Jacob Carlborg wrote:
 I have thought of that too. But I haven't been able to come up 
 with a syntax that looks good and doesn't conflict with any 
 existing syntax/symbol. The above syntax is already used for 
 template instantiation.
What if use the symbol '#' ?
May 27 2015
parent reply Daniel =?UTF-8?B?S296w6Fr?= via Digitalmars-d writes:
On Wed, 27 May 2015 12:07:09 +0000
Dennis Ritchie via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On Wednesday, 27 May 2015 at 11:53:00 UTC, Jacob Carlborg wrote:
 I have thought of that too. But I haven't been able to come up 
 with a syntax that looks good and doesn't conflict with any 
 existing syntax/symbol. The above syntax is already used for 
 template instantiation.
What if use the symbol '#' ?
Yep, I like this symbol for macro too.
May 27 2015
next sibling parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 27 May 2015 at 12:34:49 UTC, Daniel Kozák wrote:
 What if use the symbol '#' ?
Yep, I like this symbol for macro too.
https://issues.dlang.org/show_bug.cgi?id=2660
May 27 2015
parent reply "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 12:37:51 UTC, Kagamin wrote:
 On Wednesday, 27 May 2015 at 12:34:49 UTC, Daniel Kozák wrote:
 What if use the symbol '#' ?
Yep, I like this symbol for macro too.
https://issues.dlang.org/show_bug.cgi?id=2660
OK. This symbol is already used in a wrong place :) http://dlang.org/lex.html#special-token-sequence You can still use `:`. The colon is used even to refer to some Lisp macros, for example: (loop :for key :in keys :collect (cons key 0)) Still, as an option, you can use the `'` . This symbol is also used in Lisp, in some places, for example: (key-weight 'sweet) Or apostrophes, too, somewhere involved in the D?
May 27 2015
next sibling parent "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 12:55:05 UTC, Dennis Ritchie wrote:
 Or apostrophes, too, somewhere involved in the D?
Similarly, this option is no longer, as is used as: char c = 'c'; Though...
May 27 2015
prev sibling parent reply "Meta" <jared771 gmail.com> writes:
On Wednesday, 27 May 2015 at 12:55:05 UTC, Dennis Ritchie wrote:
 On Wednesday, 27 May 2015 at 12:37:51 UTC, Kagamin wrote:
 On Wednesday, 27 May 2015 at 12:34:49 UTC, Daniel Kozák wrote:
 What if use the symbol '#' ?
Yep, I like this symbol for macro too.
https://issues.dlang.org/show_bug.cgi?id=2660
OK. This symbol is already used in a wrong place :) http://dlang.org/lex.html#special-token-sequence You can still use `:`. The colon is used even to refer to some Lisp macros, for example: (loop :for key :in keys :collect (cons key 0)) Still, as an option, you can use the `'` . This symbol is also used in Lisp, in some places, for example: (key-weight 'sweet) Or apostrophes, too, somewhere involved in the D?
As far as I know, the : and ' symbols in Lisp don't have anything to do with macros. : is for keyword arguments and ' is for creating AST literals. It makes sense that these would be heavily used with macros, of course, but they are not part of Lisp's macro system.
May 27 2015
parent "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 13:12:05 UTC, Meta wrote:
 As far as I know, the : and ' symbols in Lisp don't have 
 anything to do with macros. : is for keyword arguments and ' is 
 for creating AST literals. It makes sense that these would be 
 heavily used with macros, of course, but they are not part of 
 Lisp's macro system.
Yes, but IMO, these symbols can be activated in the future macrosystem D.
May 27 2015
prev sibling parent reply "ixid" <adamsibson hotmail.com> writes:
On Wednesday, 27 May 2015 at 12:34:49 UTC, Daniel Kozák wrote:
 On Wed, 27 May 2015 12:07:09 +0000
 Dennis Ritchie via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:

 On Wednesday, 27 May 2015 at 11:53:00 UTC, Jacob Carlborg 
 wrote:
 I have thought of that too. But I haven't been able to come 
 up with a syntax that looks good and doesn't conflict with 
 any existing syntax/symbol. The above syntax is already used 
 for template instantiation.
What if use the symbol '#' ?
Yep, I like this symbol for macro too.
What's wrong with using the word macro? We don't want symbol soup.
May 27 2015
parent reply "weaselcat" <weaselcat gmail.com> writes:
On Wednesday, 27 May 2015 at 12:39:52 UTC, ixid wrote:
 On Wednesday, 27 May 2015 at 12:34:49 UTC, Daniel Kozák wrote:
 On Wed, 27 May 2015 12:07:09 +0000
 Dennis Ritchie via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:

 On Wednesday, 27 May 2015 at 11:53:00 UTC, Jacob Carlborg 
 wrote:
 I have thought of that too. But I haven't been able to come 
 up with a syntax that looks good and doesn't conflict with 
 any existing syntax/symbol. The above syntax is already 
 used for template instantiation.
What if use the symbol '#' ?
Yep, I like this symbol for macro too.
What's wrong with using the word macro? We don't want symbol soup.
this, there's a reason people don't use lisp.
May 27 2015
parent "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 15:20:38 UTC, weaselcat wrote:
 On Wednesday, 27 May 2015 at 12:39:52 UTC, ixid wrote:
 On Wednesday, 27 May 2015 at 12:34:49 UTC, Daniel Kozák wrote:
 On Wed, 27 May 2015 12:07:09 +0000
 Dennis Ritchie via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Wednesday, 27 May 2015 at 11:53:00 UTC, Jacob Carlborg 
 wrote:
 I have thought of that too. But I haven't been able to 
 come up with a syntax that looks good and doesn't conflict 
 with any existing syntax/symbol. The above syntax is 
 already used for template instantiation.
What if use the symbol '#' ?
Yep, I like this symbol for macro too.
What's wrong with using the word macro? We don't want symbol soup.
this, there's a reason people don't use lisp.
Lisp is not used due to other reasons, symbol soup typical for Perl :) $? ? s:;s:s;;$?: : s;;=]=>%-{<-|}<&|`{; ; y; -/:- [-`{-};`-{/" -; ; s;;$_;see
May 27 2015
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 27 May 2015 at 11:53:00 UTC, Jacob Carlborg wrote:
 DIP50 would require a minimal amount of syntax change, the only 
 thing is prefixing a function with the "macro" keyword. DIP78 
 on the other hand, it's not so easy to see that a function 
 declaration is actual a macro declaration.
Well, that's the point: the function is a normal function, only some of its parameters require specially prepared arguments, this can't be missed as soon as arguments are passed to the respective parameters.
 In that case I would prefer the "macro" keyword. It's already a 
 reserved, for exactly this purpose, so it will be backwards 
 compatible.
Well, maybe, I just didn't need the keyword.
 One thing that would be more complex in DIP50 would be the 
 "Context" class. Although I'm not sure if that's needed in 
 DIP78 as well.
No, passing of Context is not proposed.
 Maybe a macro should be called with a special syntax,
 e.g. myAssert!(a==b);
I have thought of that too. But I haven't been able to come up with a syntax that looks good and doesn't conflict with any existing syntax/symbol. The above syntax is already used for template instantiation.
I mean, the template instantiation syntax can inform the compiler that the expression is evaluated at compile time with possible code generation, so that the compiler is prepared to what macro will do. This resembles similarity between macros and templates. If macros can use existing syntax of a function call, I see no problem if they use another existing syntax.
May 27 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-05-27 14:17, Kagamin wrote:

 Well, that's the point: the function is a normal function, only some of
 its parameters require specially prepared arguments, this can't be
 missed as soon as arguments are passed to the respective parameters.
I prefer to be more explicit in this case, especially since the keyword is already available.
 In that case I would prefer the "macro" keyword. It's already a
 reserved, for exactly this purpose, so it will be backwards compatible.
Well, maybe, I just didn't need the keyword.
I don't need it either, it's just what I preferred.
 No, passing of Context is not proposed.
The feeling I have it that it's hard to know if it's needed or not without implementing/using the macro system.
 I mean, the template instantiation syntax can inform the compiler that
 the expression is evaluated at compile time with possible code
 generation, so that the compiler is prepared to what macro will do. This
 resembles similarity between macros and templates. If macros can use
 existing syntax of a function call, I see no problem if they use another
 existing syntax.
True, but how would that the syntax look like for template macro, if possible? -- /Jacob Carlborg
May 27 2015
parent reply "Idan Arye" <GenericNPC gmail.com> writes:
On Wednesday, 27 May 2015 at 19:15:34 UTC, Jacob Carlborg wrote:
 On 2015-05-27 14:17, Kagamin wrote:

 Well, that's the point: the function is a normal function, 
 only some of
 its parameters require specially prepared arguments, this 
 can't be
 missed as soon as arguments are passed to the respective 
 parameters.
I prefer to be more explicit in this case, especially since the keyword is already available.
I think it's more important to be explicit in the macro invocation than in the macro declaration. You can tell from the macro declaration that it's a macro you are looking at, even without the `macro` keyword, but the proposed syntaxes offer no no way to tell between a regular function call and a macro invocation by looking at the callsite alone.
May 27 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-05-27 21:41, Idan Arye wrote:

 I think it's more important to be explicit in the macro invocation than
 in the macro declaration. You can tell from the macro declaration that
 it's a macro you are looking at, even without the `macro` keyword,
It depends. As far as I can see, just looking at the macro declaration it's just a regular function declaration. You need to look at the types. First, you need to know that "Auto" is a special type. Second, you need to know that it's only a special type if the fully qualified name is "core.macros.Auto". It's not enough to look at the imports in the current module, since D supports public imports: module foo; public import core.macros; module bar; import foo; Auto myAssert(Auto condition, Auto message) { return message; } Not so easy to tell that "myAssert" is a macro declaration just by looking in the module "bar". -- /Jacob Carlborg
May 27 2015
parent reply "Kagamin" <spam here.lot> writes:
On Thursday, 28 May 2015 at 06:39:11 UTC, Jacob Carlborg wrote:
 Auto myAssert(Auto condition, Auto message)
 {
     return message;
 }

 Not so easy to tell that "myAssert" is a macro declaration just 
 by looking in the module "bar".
Well, that's the point: it's a normal function.
May 28 2015
parent Jacob Carlborg <doob me.com> writes:
On 2015-05-28 10:24, Kagamin wrote:

 Well, that's the point: it's a normal function.
But it's not. -- /Jacob Carlborg
May 28 2015
prev sibling next sibling parent "ZombineDev" <valid_email he.re> writes:
On Tuesday, 26 May 2015 at 20:23:11 UTC, Kagamin wrote:
 http://wiki.dlang.org/DIP78 - Proposal for a macro system 
 without syntactical extensions to the language. Hence it 
 doesn't allow arbitrary syntax.
I really like how powerful and easy to use are AST macros in nim. IMO, if D gets such, it would become the language with the best meta-programming abilities D -> :D Here's a nice presentation about them: http://www.infoq.com/presentations/nimrod
May 27 2015
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/26/15 2:23 PM, Kagamin wrote:
 http://wiki.dlang.org/DIP78 - Proposal for a macro system without
 syntactical extensions to the language. Hence it doesn't allow arbitrary
 syntax.
Quote from dconf W&A ask me anything: Q: "Will we get a macro system" Both Walter and Andrei: "no" Doesn't seem like much of a point for this proposal then. There didn't seem to be any wiggle room. -Steve
May 28 2015
parent "Kagamin" <spam here.lot> writes:
On Thursday, 28 May 2015 at 13:34:24 UTC, Steven Schveighoffer 
wrote:
 Quote from dconf W&A ask me anything:

 Q: "Will we get a macro system"

 Both Walter and Andrei: "no"
It's safe to close the corresponding DIPs?
May 29 2015
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 26 May 2015 at 20:23:11 UTC, Kagamin wrote:
 http://wiki.dlang.org/DIP78 - Proposal for a macro system 
 without syntactical extensions to the language. Hence it 
 doesn't allow arbitrary syntax.
This is not even remotely enough to make a DIP. You seem to be using some API to build AST nodes, but this API is completely undefined. Generally, you don't want an API to create AST. Both the programmer and the compiler already know one: the language syntax itself.
May 28 2015