www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Design deficiency of C++ and D templates

reply Norbert Nemec <Norbert.Nemec gmx.de> writes:
Hi there,

after having worked with C++ templates quite intensively (writing a small
expression template linear algebra package that really took the language to
its limits) I found the one fundamental deficiency of templates not their
syntax (which has been nicely solved in D) but the lack of a clean and
tight interface. Now, with D, this problem has not been addressed at all,
and with the newest draft of mixins, I feel that this problem will become
even more pressing:

If you look at a library based on functions, it has a clean interface. Every
function has typed parameters. If you try to call a function with
parameters of the wrong type, the compiler will immediately complain, and
you will easily spot the error. A class-based interface of a library does
not change this at all.

Now, imagine a library based on templates: Every template parameter has some
special meaning. Anyhow, there is no way to specify any restrictions on the
template parameters. If you use the template library, you have to be very
careful to know how to use the individual templates. If you use a template
with the wrong kind of parameters, the compiler will probably complain.
Anyhow, since there was no interface defined, the compiler will not tell
you that the parameter type was inadequate (like it would, if you call a
function with the wrong type of arguments) but it will point somewhere deep
into the template library to some place where that parameter is used.

Actually, without a cleanly specified interface, you may even have problems
to decide whether the bug is in your program or in the library.

Trying to see the big picture: D (and already C++) is actually a combination
of two programming languages: the first one interpreted at compile time,
the second one compiled and later executed at runtime. The second one is
strongly typed and there are powerful tools for debugging. The first one is
not typed at all and nearly impossible to debug if you have anything
moderately complex.

Comparing mixins with C-preprocessor-macros: It is generally percived that
mixins are safer than C-macros. I believe it is the other way around: the
C-preprocessor is mostly trivial. It has no loops, nesting, recursion or
anything like that. After all, it is not a complete programming language. I
have never seen code where the preprocessor commands would have been to
complex to understand. Debugging preprocessor code is mostly trivial. Now,
imagine debugging a program that does not compile in D and uses templates
from a library.

I believe: before D is made even more powerful, it should be considered how
to make it managable. In C++, it was only recently discovered at all that
templates offered a turing complete language which is interpreted at
compile-time. An absolutely ugly language, but still a language which is
even used in recent libraries.

In D, this compile-time language is now being harnessed, and it is
forseeable that people will use it far more than in C++. Anyhow, it still
is an untyped language worse than Perl...

Ciao,
Nobbi
May 17 2004
next sibling parent reply "Walter" <newshound digitalmars.com> writes:
"Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
news:c8al55$od$1 digitaldaemon.com...
 In D, this compile-time language is now being harnessed, and it is
 forseeable that people will use it far more than in C++. Anyhow, it still
 is an untyped language worse than Perl...

I understand your point. There are various proposals floating around for putting user defined type constraints onto template parameters. I think this will address the problem.
May 17 2004
parent reply Norbert Nemec <Norbert.Nemec gmx.de> writes:
Walter wrote:

 
 "Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
 news:c8al55$od$1 digitaldaemon.com...
 In D, this compile-time language is now being harnessed, and it is
 forseeable that people will use it far more than in C++. Anyhow, it still
 is an untyped language worse than Perl...

I understand your point. There are various proposals floating around for putting user defined type constraints onto template parameters. I think this will address the problem.

I guess this will be quite difficult to get right. It is probably most important to become aware that this compile-time language really is a full, interpreted language and then look at other well-designed interpreted languages like Python to see what they do to get the code working. Instead of looking at D from the C++ perspective where the template-mechanism "accidentally" evolved into a turing complete language, we should really look at it from the perspective of a good interpreted language to learn what is needed to handle complex projects.
May 17 2004
parent reply "Walter" <newshound digitalmars.com> writes:
"Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
news:c8au3i$ep1$1 digitaldaemon.com...
 Instead of looking at D from the C++ perspective where the
 template-mechanism "accidentally" evolved into a turing complete language,
 we should really look at it from the perspective of a good interpreted
 language to learn what is needed to handle complex projects.

This is a good idea, and has been proposed before. Unfortunately, the proposed syntax for them turned out to be fundamentally unworkable for D. I'm open to some new ideas here.
May 17 2004
parent Norbert Nemec <Norbert.Nemec gmx.de> writes:
Walter wrote:

 
 "Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
 news:c8au3i$ep1$1 digitaldaemon.com...
 Instead of looking at D from the C++ perspective where the
 template-mechanism "accidentally" evolved into a turing complete
 language, we should really look at it from the perspective of a good
 interpreted language to learn what is needed to handle complex projects.

This is a good idea, and has been proposed before. Unfortunately, the proposed syntax for them turned out to be fundamentally unworkable for D. I'm open to some new ideas here.

Some ideas are already beginning to form in my head. Seems to become just beautiful, even without much of a change to the existing language. Anyhow: at the moment, arrays have higher priority...
May 18 2004
prev sibling next sibling parent reply Norbert Nemec <Norbert.Nemec gmx.de> writes:
Actually, thinking about it all a little more, I realized, that we should
not really look at Python or some other scripting language, but at
*functional* languages!

Took me some time to realize, that the compile-time language of D (just like
the C++-metaprogramming-language) is a *pure functional language*! Constant
symbols can only be assigned once, loops are expressed as recursions, etc.
I'm really looking forward to examine that language closer...



Norbert Nemec wrote:

 Hi there,
 
 after having worked with C++ templates quite intensively (writing a small
 expression template linear algebra package that really took the language
 to its limits) I found the one fundamental deficiency of templates not
 their syntax (which has been nicely solved in D) but the lack of a clean
 and tight interface. Now, with D, this problem has not been addressed at
 all, and with the newest draft of mixins, I feel that this problem will
 become even more pressing:
 
 If you look at a library based on functions, it has a clean interface.
 Every function has typed parameters. If you try to call a function with
 parameters of the wrong type, the compiler will immediately complain, and
 you will easily spot the error. A class-based interface of a library does
 not change this at all.
 
 Now, imagine a library based on templates: Every template parameter has
 some special meaning. Anyhow, there is no way to specify any restrictions
 on the template parameters. If you use the template library, you have to
 be very careful to know how to use the individual templates. If you use a
 template with the wrong kind of parameters, the compiler will probably
 complain. Anyhow, since there was no interface defined, the compiler will
 not tell you that the parameter type was inadequate (like it would, if you
 call a function with the wrong type of arguments) but it will point
 somewhere deep into the template library to some place where that
 parameter is used.
 
 Actually, without a cleanly specified interface, you may even have
 problems to decide whether the bug is in your program or in the library.
 
 Trying to see the big picture: D (and already C++) is actually a
 combination of two programming languages: the first one interpreted at
 compile time, the second one compiled and later executed at runtime. The
 second one is strongly typed and there are powerful tools for debugging.
 The first one is not typed at all and nearly impossible to debug if you
 have anything moderately complex.
 
 Comparing mixins with C-preprocessor-macros: It is generally percived that
 mixins are safer than C-macros. I believe it is the other way around: the
 C-preprocessor is mostly trivial. It has no loops, nesting, recursion or
 anything like that. After all, it is not a complete programming language.
 I have never seen code where the preprocessor commands would have been to
 complex to understand. Debugging preprocessor code is mostly trivial. Now,
 imagine debugging a program that does not compile in D and uses templates
 from a library.
 
 I believe: before D is made even more powerful, it should be considered
 how to make it managable. In C++, it was only recently discovered at all
 that templates offered a turing complete language which is interpreted at
 compile-time. An absolutely ugly language, but still a language which is
 even used in recent libraries.
 
 In D, this compile-time language is now being harnessed, and it is
 forseeable that people will use it far more than in C++. Anyhow, it still
 is an untyped language worse than Perl...
 
 Ciao,
 Nobbi

May 18 2004
next sibling parent "Alex A. B." <freshmind fromru.com> writes:
 Actually, thinking about it all a little more, I realized, that we should
 not really look at Python or some other scripting language, but at
 *functional* languages!

 Took me some time to realize, that the compile-time language of D (just

 the C++-metaprogramming-language) is a *pure functional language*!

 symbols can only be assigned once, loops are expressed as recursions, etc.
 I'm really looking forward to examine that language closer...

could be implemented without much hassle is support for real compile-time lists(as template parameters) and some form of list pattern matching. Another good idea would be support for strings. As for type-checking some form of template predicates could be implemented. It could probably look like this: // this declaration will fail if C is POD-type or doesn't contain 'someMember' class Some(C) : C ? !(Traits::IsPOD(C)) && C::someMember { // ... };
May 18 2004
prev sibling parent reply "Bent Rasmussen" <exo bent-rasmussen.info> writes:
 Actually, thinking about it all a little more, I realized, that we should
 not really look at Python or some other scripting language, but at
 *functional* languages!

That is definitely an interesting topic. My first instinct to translate my simple surface functions from ML was to use templates fun elliptictorus (a,b,c) = let fun f 1 (u,v) = (a+b*cos(v))*cos(u) | f 2 (u,v) = (a+b*cos(v))*sin(u) | f 3 (u,v) = c*sin(v) | f _ (u,v) = 0.0 in f end So here's one possible* translation into D template elliptictorus(float a, float b, float c) { float x(float u, float v) {...} float y(float u, float v) {...} float z(float u, float v) {...} } * Except templates only accept integral-typed value-parameters. Probably not the best of examples though. :-) Its nice have have a multiparadigm language like D that makes several programming styles possible and doesn't force you into using classes for everything. It has a very clean feel. Also I note that a major goal of D is: a.. Support multi-paradigm programming, i.e. at a minimum support imperative, structured, object oriented, and generic programming paradigms.
 Took me some time to realize, that the compile-time language of D (just

 the C++-metaprogramming-language) is a *pure functional language*!

 symbols can only be assigned once, loops are expressed as recursions, etc.
 I'm really looking forward to examine that language closer...



 Norbert Nemec wrote:

 Hi there,

 after having worked with C++ templates quite intensively (writing a


 expression template linear algebra package that really took the language
 to its limits) I found the one fundamental deficiency of templates not
 their syntax (which has been nicely solved in D) but the lack of a clean
 and tight interface. Now, with D, this problem has not been addressed at
 all, and with the newest draft of mixins, I feel that this problem will
 become even more pressing:

 If you look at a library based on functions, it has a clean interface.
 Every function has typed parameters. If you try to call a function with
 parameters of the wrong type, the compiler will immediately complain,


 you will easily spot the error. A class-based interface of a library


 not change this at all.

 Now, imagine a library based on templates: Every template parameter has
 some special meaning. Anyhow, there is no way to specify any


 on the template parameters. If you use the template library, you have to
 be very careful to know how to use the individual templates. If you use


 template with the wrong kind of parameters, the compiler will probably
 complain. Anyhow, since there was no interface defined, the compiler


 not tell you that the parameter type was inadequate (like it would, if


 call a function with the wrong type of arguments) but it will point
 somewhere deep into the template library to some place where that
 parameter is used.

 Actually, without a cleanly specified interface, you may even have
 problems to decide whether the bug is in your program or in the library.

 Trying to see the big picture: D (and already C++) is actually a
 combination of two programming languages: the first one interpreted at
 compile time, the second one compiled and later executed at runtime. The
 second one is strongly typed and there are powerful tools for debugging.
 The first one is not typed at all and nearly impossible to debug if you
 have anything moderately complex.

 Comparing mixins with C-preprocessor-macros: It is generally percived


 mixins are safer than C-macros. I believe it is the other way around:


 C-preprocessor is mostly trivial. It has no loops, nesting, recursion or
 anything like that. After all, it is not a complete programming


 I have never seen code where the preprocessor commands would have been


 complex to understand. Debugging preprocessor code is mostly trivial.


 imagine debugging a program that does not compile in D and uses


 from a library.

 I believe: before D is made even more powerful, it should be considered
 how to make it managable. In C++, it was only recently discovered at all
 that templates offered a turing complete language which is interpreted


 compile-time. An absolutely ugly language, but still a language which is
 even used in recent libraries.

 In D, this compile-time language is now being harnessed, and it is
 forseeable that people will use it far more than in C++. Anyhow, it


 is an untyped language worse than Perl...

 Ciao,
 Nobbi


May 18 2004
parent Norbert Nemec <Norbert.Nemec gmx.de> writes:
Bent Rasmussen wrote:

 Actually, thinking about it all a little more, I realized, that we should
 not really look at Python or some other scripting language, but at
 *functional* languages!

Its nice have have a multiparadigm language like D that makes several programming styles possible and doesn't force you into using classes for everything. It has a very clean feel.

I think "multiparadigm language" is a bit misleading in this context. What I am talking about is, that D contains two independant languages: one that is interpreted at compile time and one, that is compiled to be executed lateron. The first one is pure functional and not object oriented, the latter one is mostly imperative (and of course object oriented) with a few features to support other paradigms to some extent. The execution of the first language at compile time could probably be compared to a preprocessor run. Of course, it is too closely knit into the language to really split this preprocessor out from the real compilation, but as a concept this really helps. Consider templates, instantiations and mixins as parts of a powerful functional macro language that outputs some code that is similar to D with all templates resolved, which is then compiled.
May 18 2004
prev sibling parent reply "Achilleas Margaritis" <axilmar b-online.gr> writes:
"Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
news:c8al55$od$1 digitaldaemon.com...
 Hi there,

 after having worked with C++ templates quite intensively (writing a small
 expression template linear algebra package that really took the language

 its limits) I found the one fundamental deficiency of templates not their
 syntax (which has been nicely solved in D) but the lack of a clean and
 tight interface. Now, with D, this problem has not been addressed at all,
 and with the newest draft of mixins, I feel that this problem will become
 even more pressing:

 If you look at a library based on functions, it has a clean interface.

 function has typed parameters. If you try to call a function with
 parameters of the wrong type, the compiler will immediately complain, and
 you will easily spot the error. A class-based interface of a library does
 not change this at all.

 Now, imagine a library based on templates: Every template parameter has

 special meaning. Anyhow, there is no way to specify any restrictions on

 template parameters. If you use the template library, you have to be very
 careful to know how to use the individual templates. If you use a template
 with the wrong kind of parameters, the compiler will probably complain.
 Anyhow, since there was no interface defined, the compiler will not tell
 you that the parameter type was inadequate (like it would, if you call a
 function with the wrong type of arguments) but it will point somewhere

 into the template library to some place where that parameter is used.

 Actually, without a cleanly specified interface, you may even have

 to decide whether the bug is in your program or in the library.

 Trying to see the big picture: D (and already C++) is actually a

 of two programming languages: the first one interpreted at compile time,
 the second one compiled and later executed at runtime. The second one is
 strongly typed and there are powerful tools for debugging. The first one

 not typed at all and nearly impossible to debug if you have anything
 moderately complex.

 Comparing mixins with C-preprocessor-macros: It is generally percived that
 mixins are safer than C-macros. I believe it is the other way around: the
 C-preprocessor is mostly trivial. It has no loops, nesting, recursion or
 anything like that. After all, it is not a complete programming language.

 have never seen code where the preprocessor commands would have been to
 complex to understand. Debugging preprocessor code is mostly trivial. Now,
 imagine debugging a program that does not compile in D and uses templates
 from a library.

 I believe: before D is made even more powerful, it should be considered

 to make it managable. In C++, it was only recently discovered at all that
 templates offered a turing complete language which is interpreted at
 compile-time. An absolutely ugly language, but still a language which is
 even used in recent libraries.

 In D, this compile-time language is now being harnessed, and it is
 forseeable that people will use it far more than in C++. Anyhow, it still
 is an untyped language worse than Perl...

 Ciao,
 Nobbi

Doesn't template specialization cover this topic ? I thought that D was already capable of defining constraints for template types.
May 18 2004
parent reply Norbert Nemec <Norbert.Nemec gmx.de> writes:
Achilleas Margaritis wrote:
 Doesn't template specialization cover this topic ? I thought that D was
 already capable of defining constraints for template types.

That is just some very basic pattern matching that does not have the necessary power. What I was talking about are common cases like: ------------------ template X(T) { void x(T a) { a.something(); } } void myfunc(mytype b) { X!(mytype).x(b); } ------------------ Now, if mytype does not offer a .something() member function, the compiler will, of course, complain. But is it an error in the template code or is it an error of the instantiation with an illegal type parameter? The problem is, that there is no way to specify in the definition of a template what capabilities a parameter should have. If you have code that nests over several layers of templates, this really gives messy compiler errors. On the other hand, I realize now: looking at the compile-time-language as full functional programming language, template instantiations would actually correspond to function calls. If an error occurs several levels deep, the compiler error would of course have to include a "stack trace" (which is exactly, what the "instantiated at ..., instantiated at ..." lists in C++ are!)
May 18 2004
next sibling parent Kevin Bealer <Kevin_member pathlink.com> writes:
In article <c8dukv$2als$1 digitaldaemon.com>, Norbert Nemec says...
.
Now, if mytype does not offer a .something() member function, the compiler
will, of course, complain. But is it an error in the template code or is it
an error of the instantiation with an illegal type parameter? The problem
is, that there is no way to specify in the definition of a template what
capabilities a parameter should have.

If you have code that nests over several layers of templates, this really
gives messy compiler errors.

On the other hand, I realize now: looking at the compile-time-language as
full functional programming language, template instantiations would
actually correspond to function calls. If an error occurs several levels
deep, the compiler error would of course have to include a "stack
trace" (which is exactly, what the "instantiated at ..., instantiated
at ..." lists in C++ are!)

I was thinking about this today (i.e. about templates as functions). Imagine variables that can hold: 1. types. 2. fragments of code. 3. lists of names, values or statements. And the ability to: Loop over these (arbitrary statement bodies or variable names), Pass them into and return them from "functions", i.e. templates or other code generation mechanisms. The functions of this language would produce lists or arrrays of language elements. Such a list could be: a. Passed into a (code gen) function b. Filtered or processed in some way (not sure...) c. "Promoted" into actual code (i.e. inserted into the compile stream). Given a list of names, I would like to (for example) generate 10 classes using those names or some combination thereof as: 1. variable names. 2. string literals. 3. concatenated with a prefix or postfix string and then 1 or 2. 4. Used to construct member variables or methods, using the name as a typename for a template parameter. 5. or even as a string substitution. The last sounds a little like "macro", so maybe a more type safe way (3) should be available first. With C, its not so much that the language is bad as it now stands (though it has some shortcomings that are a doozy), as that the bad techniques were available first; maybe it always happens in that order though. Kevin
May 18 2004
prev sibling parent reply "Achilleas Margaritis" <axilmar b-online.gr> writes:
"Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
news:c8dukv$2als$1 digitaldaemon.com...
 Achilleas Margaritis wrote:
 Doesn't template specialization cover this topic ? I thought that D was
 already capable of defining constraints for template types.

That is just some very basic pattern matching that does not have the necessary power. What I was talking about are common cases like: ------------------ template X(T) { void x(T a) { a.something(); } } void myfunc(mytype b) { X!(mytype).x(b); } ------------------ Now, if mytype does not offer a .something() member function, the compiler will, of course, complain. But is it an error in the template code or is

 an error of the instantiation with an illegal type parameter? The problem
 is, that there is no way to specify in the definition of a template what
 capabilities a parameter should have.

 If you have code that nests over several layers of templates, this really
 gives messy compiler errors.

 On the other hand, I realize now: looking at the compile-time-language as
 full functional programming language, template instantiations would
 actually correspond to function calls. If an error occurs several levels
 deep, the compiler error would of course have to include a "stack
 trace" (which is exactly, what the "instantiated at ..., instantiated
 at ..." lists in C++ are!)

The problem is the messy compiler reporting, not the language. The compiler could easily report the error without the template parameters, which is where the problem actually is (the template parameters make reading the error report difficult).
May 22 2004
parent Norbert Nemec <Norbert.Nemec gmx.de> writes:
Achilleas Margaritis wrote:

 On the other hand, I realize now: looking at the compile-time-language as
 full functional programming language, template instantiations would
 actually correspond to function calls. If an error occurs several levels
 deep, the compiler error would of course have to include a "stack
 trace" (which is exactly, what the "instantiated at ..., instantiated
 at ..." lists in C++ are!)

The problem is the messy compiler reporting, not the language. The compiler could easily report the error without the template parameters, which is where the problem actually is (the template parameters make reading the error report difficult).

There should probably be options to tell the compiler how to give error reports. Nested templates are really similar to nested function calls in a program. If an error happens, you might need a detailed stack-trace to find the problem, but such a stack trace really has to be formatted in a readable way to be helpful. Today's C++ compilers are just not very helpful there...
May 22 2004