Digital Mars - digitalmars.D - Template-based Preprocessing

↑ ↓ ← → Garett Bass <garettbass studiotekne.com> writes:

D's template mechanism is very promising, but I'm frequently frustrated by the
lack of C-preprocessor-equivalent power.  D templates currently provide no
means for insertion of arbitrarily parameterized blocks of code, but I think
they potentially could.

One feature I'm sorely missing that the C preprocessor happily provides is the
ability to insert blocks of code parameterized by an arbitrary identifier, e.g.:

// C/C++
#define SERIALIZABLE(type, identifier) \
    type identifier;                   \
                                       \
    string serialize_##identifier() {  \
        return toString(identifier);   \
    }

class Foo {
    SERIALIZABLE(int, i);
};

Though this can't be reproduced by D templates, I'd like to suggest the
following:

// D (suggestion)
template Serializable(T, token I) {
    T I;

    string serialize_`I`() {
        return toString(I);
    }
}

class Foo {
    mixin Serializable!(int, i);
}

Here I've used backticks to expand the identifier into code akin to the C
preprocessor's string-pasting operator (##).  I've included a keyword "token"
to mean something different from "alias", since it need not specify an existing
global identifier or alias.

I'm not really satisfied with this suggestion, maybe C-preprocessor-style
operators like # (stringizing) and ## (string-pasting) would be more
appropriate, and perhaps a different keyword than "token".  I'm open to other
ideas.

I've not looked into the D front-end codebase yet, but I'm taking a compilers
class this semester, so I was hoping I might be able implement this eventually.
 Any ideas whether this is doable?  Any objections?

Regards,
Garett

Sep 03 2006

↑ ↓ ← → Garett Bass <garettbass studiotekne.com> writes:

Walter Bright wrote:
 defmac is often trotted out as a big productivity gainer in Lisp,
 because with it one can define one's own syntax. D's lazy evaluation
 does the equivalent. 


Can someone explain to me how lazy is equivalent to defmac?  After reading Paul
Graham's "Hackers & Painters", I thought defmac was more like #define macros.

Sep 03 2006

↑ ↓ ← → Don Clugston <dac nospam.com.au> writes:

Garett Bass wrote:
 D's template mechanism is very promising, but I'm frequently frustrated 
 by the lack of C-preprocessor-equivalent power.  D templates currently 
 provide no means for insertion of arbitrarily parameterized blocks of 
 code, but I think they potentially could.
 
 One feature I'm sorely missing that the C preprocessor happily provides 
 is the ability to insert blocks of code parameterized by an arbitrary 
 identifier, e.g.:
 
 // C/C++
 #define SERIALIZABLE(type, identifier) \
    type identifier;                   \
                                       \
    string serialize_##identifier() {  \
        return toString(identifier);   \
    }
 
 class Foo {
    SERIALIZABLE(int, i);
 };
 
 Though this can't be reproduced by D templates, I'd like to suggest the 
 following:
 
 // D (suggestion)
 template Serializable(T, token I) {
    T I;
 
    string serialize_`I`() {
        return toString(I);
    }
 }
 
 class Foo {
    mixin Serializable!(int, i);
 }
 
 Here I've used backticks to expand the identifier into code akin to the 
 C preprocessor's string-pasting operator (##).  I've included a keyword 
 "token" to mean something different from "alias", since it need not 
 specify an existing global identifier or alias.
 
 I'm not really satisfied with this suggestion, maybe 
 C-preprocessor-style operators like # (stringizing) and ## 
 (string-pasting) would be more appropriate, and perhaps a different 
 keyword than "token".  I'm open to other ideas.


I've previously suggested the 'identifier' keyword for string pasting.
Stringizing is already possible, you can find the code for 
symbolnameof!() in dsource/ddl/meta, together with qualifiednameof!() 
which is significantly more powerful than anything in C++.
With identifier, you'd write:

string identifier("serialize_" ~ symbolnameof!(I))() {
   return toString(I);
}

 
 I've not looked into the D front-end codebase yet, but I'm taking a 
 compilers class this semester, so I was hoping I might be able implement 
 this eventually.  Any ideas whether this is doable?  Any objections?
 
 Regards,
 Garett

Sep 03 2006

↑ ↓ ← → Walter Bright <newshound digitalmars.com> writes:

Don Clugston wrote:
 I've previously suggested the 'identifier' keyword for string pasting.
 Stringizing is already possible, you can find the code for 
 symbolnameof!() in dsource/ddl/meta, together with qualifiednameof!() 
 which is significantly more powerful than anything in C++.
 With identifier, you'd write:
 
 string identifier("serialize_" ~ symbolnameof!(I))() {
   return toString(I);
 }


Not a bad idea, but it would require a lot of internal rewriting of the 
compiler to implement.

Sep 04 2006

↑ ↓ ← → Garett Bass <garettbass studiotekne.com> writes:

Walter Bright wrote:
 Not a bad idea, but it would require a lot of internal rewriting of the 
 compiler to implement.


Rewriting the front-end only or the back-end too?  I'm disappointed to hear
that it is so much work; this would really wrap up the boilerplate for my
serialization library:

   template Serializable(T, char[] I) {
       T identifier(I);
          string identifier("serialize_" ~ I)() {
           return toString(identifier(I));
       }
   }

   class Foo {
       mixin Serializable!(int, "i");
   }

Foo should expand to the equivalent of:

   class Foo {
       int i;

       string serialize_i() {
           return toString(i);
       }
   }

Of course there would also be a static registrar that would register the
dynamically generated serialization and deserialization delegates at the class
level at static initialization time, and associate them by string.  With a
couple other components, this allows serializable classes enough reflection to
serialize and deserialize their members and properties.

Sep 04 2006

↑ ↓ ← → Reiner Pope <reiner.pope REMOVE.THIS.gmail.com> writes:

Walter Bright wrote:
 Don Clugston wrote:
 I've previously suggested the 'identifier' keyword for string pasting.
 Stringizing is already possible, you can find the code for 
 symbolnameof!() in dsource/ddl/meta, together with qualifiednameof!() 
 which is significantly more powerful than anything in C++.
 With identifier, you'd write:

 string identifier("serialize_" ~ symbolnameof!(I))() {
   return toString(I);
 }


 Not a bad idea, but it would require a lot of internal rewriting of the 
 compiler to implement.


the following:

template foo(char[] name)
{
    bool bar;
    static if (name == "a") alias bar a;
    static if (name == "b") alias bar b;
    static if (name == "c") alias bar c;
    ...
}

Since we can already emulate the 'identifier' keyword by doing this, 
what's the difficulty in making this supported in the compiler?

Cheers,

Reiner

Sep 20 2006

↑ ↓ ← → Georg Wrede <georg.wrede nospam.org> writes:

Reiner Pope wrote:
 Walter Bright wrote:
 
 Don Clugston wrote:

 I've previously suggested the 'identifier' keyword for string pasting.
 Stringizing is already possible, you can find the code for 
 symbolnameof!() in dsource/ddl/meta, together with qualifiednameof!() 
 which is significantly more powerful than anything in C++.
 With identifier, you'd write:

 string identifier("serialize_" ~ symbolnameof!(I))() {
   return toString(I);
 }



 Not a bad idea, but it would require a lot of internal rewriting of 
 the compiler to implement.


 Why is implementing 'identifier' different from/harder than supporting 
 the following:
 
 template foo(char[] name)
 {
    bool bar;
    static if (name == "a") alias bar a;
    static if (name == "b") alias bar b;
    static if (name == "c") alias bar c;
    ...
 }
 
 Since we can already emulate the 'identifier' keyword by doing this, 
 what's the difficulty in making this supported in the compiler?


Because in the above code the identifiers a,b,c do exist right from the 
start, whereas with the identifier suggestion they'd be "born" at a 
later stage of processing, and this means grafting them late into the 
compiler data structures.

Sep 23 2006

↑ ↓ ← → Garett Bass <garettbass studiotekne.com> writes:

Don Clugston wrote:
 I've previously suggested the 'identifier' keyword for string pasting.
 Stringizing is already possible, you can find the code for 
 symbolnameof!() in dsource/ddl/meta, together with qualifiednameof!() 
 which is significantly more powerful than anything in C++.


Thanks, Don!  I now recall your suggestion from a thread I prompted about a
year ago.  I've updated my example to use your suggested 'identifier' keyword,
as this obviates the need for my suggested 'token' keyword:

    template Serializable(T, char[] I) {
        T identifier(I);
    
        string identifier("serialize_" ~ I)() {
            return toString(identifier(I));
        }
    }

    class Foo {
        mixin Serializable!(int, "i");
    }

Foo should expand to the equivalent of:

    class Foo {
        int i;

        string serialize_i() {
            return toString(i);
        }
    }

Do you have any idea what's involved in implementing the 'identifier' keyword
as described here?

Regards,
Garett

Sep 04 2006

↑ ↓ ← → Don Clugston <dac nospam.com.au> writes:

Garett Bass wrote:
 Don Clugston wrote:
 I've previously suggested the 'identifier' keyword for string pasting.
 Stringizing is already possible, you can find the code for 
 symbolnameof!() in dsource/ddl/meta, together with qualifiednameof!() 
 which is significantly more powerful than anything in C++.


 Thanks, Don!  I now recall your suggestion from a thread I prompted 
 about a year ago.  I've updated my example to use your suggested 
 'identifier' keyword, as this obviates the need for my suggested 'token' 
 keyword:
 
    template Serializable(T, char[] I) {
        T identifier(I);
           string identifier("serialize_" ~ I)() {
            return toString(identifier(I));
        }
    }
 
    class Foo {
        mixin Serializable!(int, "i");
    }
 
 Foo should expand to the equivalent of:
 
    class Foo {
        int i;
 
        string serialize_i() {
            return toString(i);
        }
    }
 
 Do you have any idea what's involved in implementing the 'identifier' 
 keyword as described here?


I don't know. It's clearly easy to get through the syntactic pass. But 
does it cause problems during name lookup?

More importantly, would it encourage bad coding styles? IMHO, it needs 
some convincing use cases -- or, if Walter gets more experimental 
post-1.0, this would be an extremely cool thing to try out. (It would 
kill one of the last metaprogramming things C++ still has compared to D).

Sep 04 2006

↑ ↓ ← → Kirk McDonald <kirklin.mcdonald gmail.com> writes:

Don Clugston wrote:
 Garett Bass wrote:
 
 Don Clugston wrote:

 I've previously suggested the 'identifier' keyword for string pasting.
 Stringizing is already possible, you can find the code for 
 symbolnameof!() in dsource/ddl/meta, together with qualifiednameof!() 
 which is significantly more powerful than anything in C++.



 Thanks, Don!  I now recall your suggestion from a thread I prompted 
 about a year ago.  I've updated my example to use your suggested 
 'identifier' keyword, as this obviates the need for my suggested 
 'token' keyword:

    template Serializable(T, char[] I) {
        T identifier(I);
           string identifier("serialize_" ~ I)() {
            return toString(identifier(I));
        }
    }

    class Foo {
        mixin Serializable!(int, "i");
    }

 Foo should expand to the equivalent of:

    class Foo {
        int i;

        string serialize_i() {
            return toString(i);
        }
    }

 Do you have any idea what's involved in implementing the 'identifier' 
 keyword as described here?


 
 I don't know. It's clearly easy to get through the syntactic pass. But 
 does it cause problems during name lookup?
 
 More importantly, would it encourage bad coding styles? IMHO, it needs 
 some convincing use cases -- or, if Walter gets more experimental 
 post-1.0, this would be an extremely cool thing to try out. (It would 
 kill one of the last metaprogramming things C++ still has compared to D).


I have a use-case. In the Python/C API, a module must have an init 
function named "init" ~ module_name. This function is called by Python 
when the module is imported. The init function must furthermore contain 
a call to the Py_InitModule function (or one of its variants), which 
takes the name of the module as a string argument. If we have a module 
named "foo", we might say:

extern(C)
export void initfoo() {
     Py_InitModule("foo", null);
}

Boost.Python wraps this business with some preprocessor tricks. The 
following is more or less equivalent to the above code:

BOOST_PYTHON_MODULE(foo) {
}

I cannot do this in Pyd. The only difference between Pyd code and the 
first example above is that the user must call Pyd's module_init 
function rather than Python's Py_InitModule. With the proposed 
"identifier" keyword, I could get rid of the burdensome "init" function 
definition, and the redundancy of specifying "foo" twice (both in the 
function name and in the call to the init function).

More generally, this syntax would help D interface with other C 
libraries that expect special function names like this.

-- 
Kirk McDonald
Pyd: Wrapping Python with D
http://pyd.dsource.org

Sep 04 2006

↑ ↓ ← → Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:

Don Clugston wrote:
 Garett Bass wrote:
 Don Clugston wrote:
 I've previously suggested the 'identifier' keyword for string pasting.
 Stringizing is already possible, you can find the code for 
 symbolnameof!() in dsource/ddl/meta, together with qualifiednameof!() 
 which is significantly more powerful than anything in C++.


 Thanks, Don!  I now recall your suggestion from a thread I prompted 
 about a year ago.  I've updated my example to use your suggested 
 'identifier' keyword, as this obviates the need for my suggested 
 'token' keyword:

    template Serializable(T, char[] I) {
        T identifier(I);
           string identifier("serialize_" ~ I)() {
            return toString(identifier(I));
        }
    }

    class Foo {
        mixin Serializable!(int, "i");
    }

 Foo should expand to the equivalent of:

    class Foo {
        int i;

        string serialize_i() {
            return toString(i);
        }
    }

 Do you have any idea what's involved in implementing the 'identifier' 
 keyword as described here?


 I don't know. It's clearly easy to get through the syntactic pass. But 
 does it cause problems during name lookup?
 
 More importantly, would it encourage bad coding styles? IMHO, it needs 
 some convincing use cases -- or, if Walter gets more experimental 
 post-1.0, this would be an extremely cool thing to try out. (It would 
 kill one of the last metaprogramming things C++ still has compared to D).


Indeed I was about to ask that too, what would be the use of such a 
feature? I have to say I'm a bit skeptical, it feels way hackish.

I've read Kirk's reply about Python/C API, but it is only required there 
  because Python interfacing requires manipulation of object code 
implementation-level semantics (the function names), as could require 
any other interfacing with a C library .
But what about any actual "conceptual"/"design" use?

-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D

Sep 05 2006