www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Avoiding some template bloat?

reply "bearophile" <bearophileHUGS lycos.com> writes:
Maybe someone someday will want to implement two Phobos printing 
functions usable like this:

writecfln!"%d %s"(10, "hello");
writecf!"%d %s"(10, "hello");

They accept a format string as template argument (format strings 
are often compile-time constants and sometimes they are 
computable on the fly at compile-time), verifies at compile time 
(calling another CTFE function in its template constraint) that 
the format string matches with the type tuple of the arguments, 
and then just call writefln()/writef() with the format string and 
arguments.

This variable-length-templated function avoids format string 
errors at run-time. This in my opinion means taking advantage of 
the static typing of D to avoid some bugs. In my opinion it's 
silly to use a language with static typing, unlike Python, and 
then _not_ use such static knowledge where it's usable and useful.

Using "%s" everywhere in writefln/writef is an option, but it's 
not always usable, like when you want to use the advanced new 
formatting syntax for arrays.

writecfln() and writecf() just call writefln/writef, so they are 
very light, in the binary they just call another function. But if 
there are many different format strings you get many similar 
useless little functions in the binary (maybe inlined).

Is it possible for the D compiler to avoid the (small amount of?) 
template bloat caused by writecfln/writecf? Is it useful to 
introduce some kind of annotation to tell the compiler to 
avoid/remove any bloat for similar functions turning them 
essentially into just compile-time tests for writefln/writef? Or 
is it enough to rely on the inliner of the compiler?

Beside writecfln()/writecf() I have other use cases for such 
ideas. When I write certain kind of matrix code that use 
fixed-sized 2D matrices, they enforce the correct sizes at 
compile-time, but then they just call functions that use run-time 
sized arrays, to avoid template bloat (if necessary slicing the 
fixed sized matrix into a dynamic array of dynamic arrays). In 
this case too I'd like to minimize or remove template bloat and 
just have compile-time tests on data that is then managed with 
run-time sizes (so it's not actually templated on the size of the 
matrix).

Here you see a little example, regarding matrix multiplication 
(here it doesn't call another function with run-time sizes, so 
this is really templated):


template TMMul(M1, M2) { // helper
     alias Unqual!(typeof(M1[0][0]))[M2[0].length][M1.length] 
TMMul;
}

void matrixMul(T, T2, size_t k, size_t m, size_t n)
               (in ref T[m][k] A, in ref T[n][m] B,
                ref T2[n][k] result) pure nothrow
if (is(T2 == Unqual!T)) {
     T2[m] aux;
     foreach (j; 0 .. n) {
         foreach (k, row; B)
             aux[k] = row[j];
         foreach (i, ai; A)
             result[i][j] = dotProduct(ai, aux);
     }
}


A third use case for such need is a poor's man "homemade" 
management of integer-indexed dependent types. In such cases you 
use the compile-time known sizes and values to enforce 
compile-time sanity constraints, but then you use normal run-time 
not-templated functions to avoid useless template bloat. This 
means the compile-time values are meant to be used just by the 
type system to verify things, but the functions are not actually 
templated. Similar ghost types and other types that vanish are 
commonly used in functional languages, like OCaML. Avoiding any 
template bloat when you do such things is kind of needed, unless 
you want to produce large binaries that do little at run time.

Bye,
bearophile
Oct 08 2012
next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
 (if necessary slicing the fixed sized matrix into a dynamic 
 array of dynamic arrays).

It's not hard to convert a fixed sized matrix into this run-time value that requires no templating on the sizes and requires no slicing and no heap allocations inside the template function that performs just the compile-time tests: struct Mat2D(T) { T* ptr; size_t nrows, ncols; // + indexing methods } Using something like that the matrix multiplication function I've shown becomes just a shell that calls a function that accepts two Mat2D and is templated only on T. Bye, bearophile
Oct 08 2012
prev sibling parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Tue, Oct 9, 2012 at 12:13 AM, bearophile <bearophileHUGS lycos.com> wrote:
 Maybe someone someday will want to implement two Phobos printing functions
 usable like this:

 writecfln!"%d %s"(10, "hello");
 writecf!"%d %s"(10, "hello");

 They accept a format string as template argument (format strings are often
 compile-time constants and sometimes they are computable on the fly at
 compile-time), verifies at compile time (calling another CTFE function in
 its template constraint) that the format string matches with the type tuple
 of the arguments, and then just call writefln()/writef() with the format
 string and arguments.

IIRC, I did that as an example in my template tutorial: https://github.com/PhilippeSigaud/D-templates-tutorial/blob/master/dtemplates.pdf?raw=true (section 32, on page 167) Code in https://github.com/PhilippeSigaud/D-templates-tutorial/blob/master/templates_examples.tex#L2000 and below.
Oct 08 2012