www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Help with Template Code

reply John Demme <me teqdruid.com> writes:
Hey all!

There's a particular problem I'm trying to solve using templates.  I don't
see a reason that the compiler couldn't do this, but I'm not certain I can
do it with templates yet.

Here's a slightly simplified pseudo-code-ish version of what I want to do:

T inst(T : struct)(T.tupleof t);

Yes- this makes no sense, so let me describe.  I want to create a templated
function wherein the template argument is a struct... OK, that's easy. 
Next, I want the parameters of the function to be the types in the struct. 
For example, if I have the following struct:
struct Foo {
        int a;
        float b;
}

then the following call:
Foo f = inst!(Foo)(5, 8.26)
would pass the Tuple!(int,float)(5, 8.26) into the inst function.  No, it's
not OK to add stuff so the calling code, but the inst function can be as
ugly as necessary.

I feel like this should be possible, but I don't know how... Any ideas?

Thanks

-- 
~John Demme
me teqdruid.com
http://www.teqdruid.com/
Mar 30 2007
parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"John Demme" <me teqdruid.com> wrote in message 
news:eukg7o$m31$1 digitalmars.com...
 Hey all!

 There's a particular problem I'm trying to solve using templates.  I don't
 see a reason that the compiler couldn't do this, but I'm not certain I can
 do it with templates yet.

 Here's a slightly simplified pseudo-code-ish version of what I want to do:

 T inst(T : struct)(T.tupleof t);

 Yes- this makes no sense, so let me describe.  I want to create a 
 templated
 function wherein the template argument is a struct... OK, that's easy.
 Next, I want the parameters of the function to be the types in the struct.
 For example, if I have the following struct:
 struct Foo {
        int a;
        float b;
 }

 then the following call:
 Foo f = inst!(Foo)(5, 8.26)
 would pass the Tuple!(int,float)(5, 8.26) into the inst function.  No, 
 it's
 not OK to add stuff so the calling code, but the inst function can be as
 ugly as necessary.

 I feel like this should be possible, but I don't know how... Any ideas?

 Thanks

 -- 
 ~John Demme
 me teqdruid.com
 http://www.teqdruid.com/

Wow! struct S { int x; float y; char[] z; static S opCall(typeof(S.tupleof) args) { S s; foreach(i, arg; args) s.tupleof[i] = arg; return s; } } void main() { S s = S(1, 2.3, "hi"); writefln(s.x); writefln(s.y); writefln(s.z); } I really didn't think I would be able to write that.
Mar 30 2007
parent reply John Demme <me teqdruid.com> writes:
Jarrett Billingsley wrote:

 "John Demme" <me teqdruid.com> wrote in message
 news:eukg7o$m31$1 digitalmars.com...
 Hey all!

 There's a particular problem I'm trying to solve using templates.  I
 don't see a reason that the compiler couldn't do this, but I'm not
 certain I can do it with templates yet.

 Here's a slightly simplified pseudo-code-ish version of what I want to
 do:

 T inst(T : struct)(T.tupleof t);

 Yes- this makes no sense, so let me describe.  I want to create a
 templated
 function wherein the template argument is a struct... OK, that's easy.
 Next, I want the parameters of the function to be the types in the
 struct. For example, if I have the following struct:
 struct Foo {
        int a;
        float b;
 }

 then the following call:
 Foo f = inst!(Foo)(5, 8.26)
 would pass the Tuple!(int,float)(5, 8.26) into the inst function.  No,
 it's
 not OK to add stuff so the calling code, but the inst function can be as
 ugly as necessary.

 I feel like this should be possible, but I don't know how... Any ideas?

 Thanks

 --
 ~John Demme
 me teqdruid.com
 http://www.teqdruid.com/

Wow! struct S { int x; float y; char[] z; static S opCall(typeof(S.tupleof) args) { S s; foreach(i, arg; args) s.tupleof[i] = arg; return s; } } void main() { S s = S(1, 2.3, "hi"); writefln(s.x); writefln(s.y); writefln(s.z); } I really didn't think I would be able to write that.

Ahh!!! typeof! That does it for me... much thanks. BTW, with your example above, you could probably turn that opCall into a mixin... It'd be a nice little mixin to have in Tango and/or Phobos. I've got one more template problem, but I'm pretty sure I can't do this. I now want to access the names of the struct's fields so that I could, for example, make a templated function that accepts a struct and prints name:value pairs for all the fields. Is there any way I can do this? (If so, I'm gonna be really, really impressed.) Thanks again -- ~John Demme me teqdruid.com http://www.teqdruid.com/
Mar 30 2007
next sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
John Demme wrote:
 Jarrett Billingsley wrote:
 
 "John Demme" <me teqdruid.com> wrote in message
 news:eukg7o$m31$1 digitalmars.com...
 Hey all!

 There's a particular problem I'm trying to solve using templates.  I
 don't see a reason that the compiler couldn't do this, but I'm not
 certain I can do it with templates yet.

 Here's a slightly simplified pseudo-code-ish version of what I want to
 do:

 T inst(T : struct)(T.tupleof t);

 Yes- this makes no sense, so let me describe.  I want to create a
 templated
 function wherein the template argument is a struct... OK, that's easy.
 Next, I want the parameters of the function to be the types in the
 struct. For example, if I have the following struct:
 struct Foo {
        int a;
        float b;
 }

 then the following call:
 Foo f = inst!(Foo)(5, 8.26)
 would pass the Tuple!(int,float)(5, 8.26) into the inst function.  No,
 it's
 not OK to add stuff so the calling code, but the inst function can be as
 ugly as necessary.

 I feel like this should be possible, but I don't know how... Any ideas?

 Thanks

 --
 ~John Demme
 me teqdruid.com
 http://www.teqdruid.com/

struct S { int x; float y; char[] z; static S opCall(typeof(S.tupleof) args) { S s; foreach(i, arg; args) s.tupleof[i] = arg; return s; } } void main() { S s = S(1, 2.3, "hi"); writefln(s.x); writefln(s.y); writefln(s.z); } I really didn't think I would be able to write that.

Ahh!!! typeof! That does it for me... much thanks. BTW, with your example above, you could probably turn that opCall into a mixin... It'd be a nice little mixin to have in Tango and/or Phobos. I've got one more template problem, but I'm pretty sure I can't do this. I now want to access the names of the struct's fields so that I could, for example, make a templated function that accepts a struct and prints name:value pairs for all the fields. Is there any way I can do this? (If so, I'm gonna be really, really impressed.) Thanks again

Not directly. The way I'm going to solve this problem is to have a convention that any struct whose I want to be able to access by name should have a 'fieldsof' property. This will be a tuple of strings that name the fields in the order they appear in the struct. So, for your example: struct Foo { alias Tuple!("a","b") fieldsof; int a; float b; } In that case, Foo.fieldsof[i] is the name of the field Foo.tupleof[i]. Would be nice to have this built-in, but it's not a big drama. -- Daniel -- int getRandomNumber() { return 4; // chosen by fair dice roll. // guaranteed to be random. } http://xkcd.com/ v2sw5+8Yhw5ln4+5pr6OFPma8u6+7Lw4Tm6+7l6+7D i28a2Xs3MSr2e4/6+7t4TNSMb6HTOp5en5g6RAHCP http://hackerkey.com/
Mar 30 2007
parent John Demme <me teqdruid.com> writes:
Daniel Keep wrote:

 
 
 John Demme wrote:
 Jarrett Billingsley wrote:
 
 "John Demme" <me teqdruid.com> wrote in message
 news:eukg7o$m31$1 digitalmars.com...
 Hey all!

 There's a particular problem I'm trying to solve using templates.  I
 don't see a reason that the compiler couldn't do this, but I'm not
 certain I can do it with templates yet.

 Here's a slightly simplified pseudo-code-ish version of what I want to
 do:

 T inst(T : struct)(T.tupleof t);

 Yes- this makes no sense, so let me describe.  I want to create a
 templated
 function wherein the template argument is a struct... OK, that's easy.
 Next, I want the parameters of the function to be the types in the
 struct. For example, if I have the following struct:
 struct Foo {
        int a;
        float b;
 }

 then the following call:
 Foo f = inst!(Foo)(5, 8.26)
 would pass the Tuple!(int,float)(5, 8.26) into the inst function.  No,
 it's
 not OK to add stuff so the calling code, but the inst function can be
 as ugly as necessary.

 I feel like this should be possible, but I don't know how... Any ideas?

 Thanks

 --
 ~John Demme
 me teqdruid.com
 http://www.teqdruid.com/

struct S { int x; float y; char[] z; static S opCall(typeof(S.tupleof) args) { S s; foreach(i, arg; args) s.tupleof[i] = arg; return s; } } void main() { S s = S(1, 2.3, "hi"); writefln(s.x); writefln(s.y); writefln(s.z); } I really didn't think I would be able to write that.

Ahh!!! typeof! That does it for me... much thanks. BTW, with your example above, you could probably turn that opCall into a mixin... It'd be a nice little mixin to have in Tango and/or Phobos. I've got one more template problem, but I'm pretty sure I can't do this. I now want to access the names of the struct's fields so that I could, for example, make a templated function that accepts a struct and prints name:value pairs for all the fields. Is there any way I can do this? (If so, I'm gonna be really, really impressed.) Thanks again

Not directly. The way I'm going to solve this problem is to have a convention that any struct whose I want to be able to access by name should have a 'fieldsof' property. This will be a tuple of strings that name the fields in the order they appear in the struct. So, for your example: struct Foo { alias Tuple!("a","b") fieldsof; int a; float b; } In that case, Foo.fieldsof[i] is the name of the field Foo.tupleof[i]. Would be nice to have this built-in, but it's not a big drama. -- Daniel

Yeah... I'm doing something similar now. I guess I'll stick with that. Thanks -- ~John Demme me teqdruid.com http://www.teqdruid.com/
Mar 31 2007
prev sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"John Demme" <me teqdruid.com> wrote in message 
news:eukjpj$opg$1 digitalmars.com...
 Wow!

 struct S
 {
     int x;
     float y;
     char[] z;

     static S opCall(typeof(S.tupleof) args)
     {
         S s;

         foreach(i, arg; args)
             s.tupleof[i] = arg;

         return s;
     }
 }

 void main()
 {
     S s = S(1, 2.3, "hi");
     writefln(s.x);
     writefln(s.y);
     writefln(s.z);
 }

 I really didn't think I would be able to write that.

Ahh!!! typeof! That does it for me... much thanks. BTW, with your example above, you could probably turn that opCall into a mixin... It'd be a nice little mixin to have in Tango and/or Phobos.

template StructCtor(T) { static T opCall(typeof(T.tupleof) args) { T t; foreach(i, arg; args) t.tupleof[i] = arg; return t; } } struct s { int x; float y; char[] z; mixin StructCtor!(S); }
Mar 31 2007
parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Jarrett Billingsley wrote:
 "John Demme" <me teqdruid.com> wrote in message 
 news:eukjpj$opg$1 digitalmars.com...
 above, you could probably turn that opCall into a mixin... It'd be a nice
 little mixin to have in Tango and/or Phobos.

template StructCtor(T) { static T opCall(typeof(T.tupleof) args) { T t; foreach(i, arg; args) t.tupleof[i] = arg; return t; } } struct s { int x; float y; char[] z; mixin StructCtor!(S); }

Come on, we can do a bit better than that! --- template StructCtor() { static typeof(*this) opCall(typeof(typeof(*this).tupleof) args) { typeof(*this) t; foreach(i, arg; args) t.tupleof[i] = arg; return t; } } struct S { int x; float y; char[] z; mixin StructCtor; } --- There. Now you don't need to specify the type, and you get rid of the "!(S)" at point-of-use entirely. I didn't know you could omit !() from a template mixin statement until I tried it just now, by the way. Cool.
Mar 31 2007
parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Frits van Bommel" <fvbommel REMwOVExCAPSs.nl> wrote in message 
news:eum0c9$2b75$1 digitalmars.com...
 Come on, we can do a bit better than that!
 ---
 template StructCtor()
 {
     static typeof(*this) opCall(typeof(typeof(*this).tupleof) args)
     {
         typeof(*this) t;

         foreach(i, arg; args)
             t.tupleof[i] = arg;

         return t;
     }
 }

 struct S
 {
     int x;
     float y;
     char[] z;

     mixin StructCtor;
 }

Ahh, I was hoping that was possible!
Mar 31 2007
parent reply Max Samukha <samukha voliacable.com> writes:
On Sat, 31 Mar 2007 20:55:54 -0400, "Jarrett Billingsley"
<kb3ctd2 yahoo.com> wrote:

"Frits van Bommel" <fvbommel REMwOVExCAPSs.nl> wrote in message 
news:eum0c9$2b75$1 digitalmars.com...
 Come on, we can do a bit better than that!
 ---
 template StructCtor()
 {
     static typeof(*this) opCall(typeof(typeof(*this).tupleof) args)
     {
         typeof(*this) t;

         foreach(i, arg; args)
             t.tupleof[i] = arg;

         return t;
     }
 }

 struct S
 {
     int x;
     float y;
     char[] z;

     mixin StructCtor;
 }

Ahh, I was hoping that was possible!

the loop. To improve performance, you could skip the struct initialization: template StructCtor() { static typeof(*this) opCall(typeof(typeof(*this).tupleof) args) { typeof(*this) t = void; foreach(i, arg; args) t.tupleof[i] = arg; return t; } }
Apr 01 2007
parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Max Samukha" <samukha voliacable.com> wrote in message 
news:dopu039g3u5dtb0ubovm8c7ach304bdbb7 4ax.com...
 Note that the mixin ctor is slower then manually coded one because of
 the loop. To improve performance, you could skip the struct
 initialization:

That loop is unrolled at compile time since it's iterating over a tuple. It'll be just as fast as writing the initialization out line-by-line. But using =void is another nice optimization :)
Apr 01 2007
parent reply Max Samukha <samukha voliacable.com> writes:
On Sun, 1 Apr 2007 11:43:58 -0400, "Jarrett Billingsley"
<kb3ctd2 yahoo.com> wrote:

"Max Samukha" <samukha voliacable.com> wrote in message 
news:dopu039g3u5dtb0ubovm8c7ach304bdbb7 4ax.com...
 Note that the mixin ctor is slower then manually coded one because of
 the loop. To improve performance, you could skip the struct
 initialization:

That loop is unrolled at compile time since it's iterating over a tuple.

I thought it should, too. But when tested on Windows with dmd 1.010, the tuple version is significantly slower. I'm still not sure why.
It'll be just as fast as writing the initialization out line-by-line.  But 
using =void is another nice optimization :) 

Apr 01 2007
parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Max Samukha" <samukha voliacable.com> wrote in message 
news:nmmv03h5g5mtbkn6hetbnu33ei6nnnhd67 4ax.com...
 I thought it should, too. But when tested on Windows with dmd 1.010,
 the tuple version is significantly slower. I'm still not sure why.

Ahh, looking at the disassembly it makes sense now. What happens is that when you write: foreach(i, arg; args) t.tupleof[i] = arg; It gets turned into something like _this_: typeof(args[0]) arg0 = args[0]; t.tupleof[0] = arg0; typeof(args[1]) arg1 = args[1]; t.tupleof[1] = arg1; typeof(args[2]) arg2 = args[2]; t.tupleof[2] = arg2; Notice it copies the argument value into a temp variable, then that temp variable into the struct. Very inefficient. Unfortunately I don't know of any way to get around this.. I was hoping t.tupleof[] = args[]; would work, but it's illegal. A static for loop would be nice.
Apr 01 2007
parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Jarrett Billingsley wrote:
 "Max Samukha" <samukha voliacable.com> wrote in message 
 news:nmmv03h5g5mtbkn6hetbnu33ei6nnnhd67 4ax.com...
 I thought it should, too. But when tested on Windows with dmd 1.010,
 the tuple version is significantly slower. I'm still not sure why.

Ahh, looking at the disassembly it makes sense now. What happens is that when you write: foreach(i, arg; args) t.tupleof[i] = arg; It gets turned into something like _this_: typeof(args[0]) arg0 = args[0]; t.tupleof[0] = arg0; typeof(args[1]) arg1 = args[1]; t.tupleof[1] = arg1; typeof(args[2]) arg2 = args[2]; t.tupleof[2] = arg2; Notice it copies the argument value into a temp variable, then that temp variable into the struct. Very inefficient. Unfortunately I don't know of any way to get around this..

Yes, DMD does that, *unless you turn on optimizations* ;). Measuring performance without optimization switches is pretty much useless. With optimizations it just moves mem->reg, reg->mem. It generates code bit-for-bit identical to: --- static S opCall(int x_, float y_, char[] z_) { S s = void; s.x = x_; s.y = y_; s.z = z_; return s; } --- for the version Max posted (with =void) (The only difference is the mangled name; the mixin name is in there for the mixed-in version)
Apr 01 2007
parent reply Max Samukha <samukha voliacable.com> writes:
On Sun, 01 Apr 2007 22:19:03 +0200, Frits van Bommel
<fvbommel REMwOVExCAPSs.nl> wrote:

Jarrett Billingsley wrote:
 "Max Samukha" <samukha voliacable.com> wrote in message 
 news:nmmv03h5g5mtbkn6hetbnu33ei6nnnhd67 4ax.com...
 I thought it should, too. But when tested on Windows with dmd 1.010,
 the tuple version is significantly slower. I'm still not sure why.

Ahh, looking at the disassembly it makes sense now. What happens is that when you write: foreach(i, arg; args) t.tupleof[i] = arg; It gets turned into something like _this_: typeof(args[0]) arg0 = args[0]; t.tupleof[0] = arg0; typeof(args[1]) arg1 = args[1]; t.tupleof[1] = arg1; typeof(args[2]) arg2 = args[2]; t.tupleof[2] = arg2; Notice it copies the argument value into a temp variable, then that temp variable into the struct. Very inefficient. Unfortunately I don't know of any way to get around this..

Yes, DMD does that, *unless you turn on optimizations* ;). Measuring performance without optimization switches is pretty much useless. With optimizations it just moves mem->reg, reg->mem. It generates code bit-for-bit identical to: --- static S opCall(int x_, float y_, char[] z_) { S s = void; s.x = x_; s.y = y_; s.z = z_; return s; } --- for the version Max posted (with =void) (The only difference is the mangled name; the mixin name is in there for the mixed-in version)

time difference is more than 40%. The source is this: import std.stdio; import std.c.time; template StructCtor() { static typeof(*this) opCall(typeof(typeof(*this).tupleof) args) { typeof(*this) t = void; foreach(i, arg; args) t.tupleof[i] = arg; return t; } } struct Bar { int x; int y; int z; //mixin StructCtor; static Bar opCall(int x, int y, int z) { Bar result = void; result.x = x; result.y = y; result.z = z; return result; } } void main() { auto c = clock(); for (int i = 0; i < 100000000; i++) { auto test = Bar(i, i, i); } writefln(clock() - c); } What am I doing wrong?
Apr 01 2007
parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Max Samukha wrote:
 On Sun, 01 Apr 2007 22:19:03 +0200, Frits van Bommel
 <fvbommel REMwOVExCAPSs.nl> wrote:

 With optimizations it just moves mem->reg, reg->mem. It generates code 
 bit-for-bit identical to:
 ---
     static S opCall(int x_, float y_, char[] z_) {
 	    S s = void;
 	    s.x = x_;
 	    s.y = y_;
 	    s.z = z_;
 	    return s;
     }
 ---
 for the version Max posted (with =void)

 (The only difference is the mangled name; the mixin name is in there for 
 the mixed-in version)

time difference is more than 40%. The source is this:

 
 What am I doing wrong?

For one thing, clock() isn't exactly accurate. Also, you didn't mention how many times you ran the test (the first time will likely take longer because the program has to be loaded into cache first, it's best to run it a couple of times before looking at the results) However, those don't seem to be the issue here. Looking at the generated assembly I see that while the functions compile to the exact same thing (as I mentioned in my previous post), DMD doesn't seem to inline the mixed-in version :(... (You can also see this without inspecting the generated code: if you leave off -inline from the command line the two versions take the same amount of time, at least on my computer) So it would seem there's no way to get the mixed-in version to equal speed simply because it won't be inlined by DMD... Note: GDC (with -O3 -finline) doesn't seem to have this problem. In fact, I had to add some code so it doesn't optimize out the entire loop :P. Even then, the code seems to be identical and (unsurprisingly) runs just as fast. P.S. I performed these tests on Linux (amd64). Another fun fact: the GDC-compiled version ran about twice as fast as the fastest DMD-compiled one. I think that my GDC being set up to generate 64-bit code may have had something to do with this though, so it's not really a fair comparison of the optimizers in the compilers. (Unless you count generating 64-bit code for 64-bit processors as an optimization ;) )
Apr 01 2007
parent Max Samukha <samukha voliacable.com> writes:
On Mon, 02 Apr 2007 01:24:03 +0200, Frits van Bommel
<fvbommel REMwOVExCAPSs.nl> wrote:

Max Samukha wrote:
 On Sun, 01 Apr 2007 22:19:03 +0200, Frits van Bommel
 <fvbommel REMwOVExCAPSs.nl> wrote:

 With optimizations it just moves mem->reg, reg->mem. It generates code 
 bit-for-bit identical to:
 ---
     static S opCall(int x_, float y_, char[] z_) {
 	    S s = void;
 	    s.x = x_;
 	    s.y = y_;
 	    s.z = z_;
 	    return s;
     }
 ---
 for the version Max posted (with =void)

 (The only difference is the mangled name; the mixin name is in there for 
 the mixed-in version)

time difference is more than 40%. The source is this:

 
 What am I doing wrong?

For one thing, clock() isn't exactly accurate. Also, you didn't mention how many times you ran the test (the first time will likely take longer because the program has to be loaded into cache first, it's best to run it a couple of times before looking at the results)

I was running it over and over again. You are right, of course. I shouldn't have run those silly speed tests at all. The disassembly is a D programmer's friend:).
However, those don't seem to be the issue here.
Looking at the generated assembly I see that while the functions compile 
to the exact same thing (as I mentioned in my previous post), DMD 
doesn't seem to inline the mixed-in version :(...
(You can also see this without inspecting the generated code: if you 
leave off -inline from the command line the two versions take the same 
amount of time, at least on my computer)

So it would seem there's no way to get the mixed-in version to equal 
speed simply because it won't be inlined by DMD...

Note: GDC (with -O3 -finline) doesn't seem to have this problem. In 
fact, I had to add some code so it doesn't optimize out the entire loop 
:P. Even then, the code seems to be identical and (unsurprisingly) runs 
just as fast.


P.S. I performed these tests on Linux (amd64). Another fun fact: the 
GDC-compiled version ran about twice as fast as the fastest DMD-compiled 
one. I think that my GDC being set up to generate 64-bit code may have 
had something to do with this though, so it's not really a fair 
comparison of the optimizers in the compilers. (Unless you count 
generating 64-bit code for 64-bit processors as an optimization ;) )

It seems like dmd is not going to support 64 bit processors in the foreseeable future (stdio super-performance seems to be the priority)
Apr 02 2007