www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Map one tuple to another Tuple of different type

reply Daniel Gibson <metalcaedes gmail.com> writes:
Hi,
I have a variadic templated function and want to call a C varargs 
function. I want to be able to pass static arrays, which D2 passes by 
value and C by reference, so I'd like to automagically translate those 
arguments.

My idea was something like this:

   extern (C) origFun(int x, ...);

   T transTupleElem(T)(T arg) { return arg; }

   float* transTupleElem(T : float[3])(T arg) {
     return arg.ptr;
   }

   void fun(T...)(int x, T argTuple) {
     // create a new tuple type that replaces all static float[3]
     // arrays with float* to emulate C call-by-reference behavior
     alias ReplaceAll!(float[3], float*, T) ModifiedTuple;
     ModifiedTuple modTuple;

     foreach(size_t i ; 0 .. T.length)
       modTuple[i] = transTupleElem(argTuple[i]); // BOOM!

     origFun(modTuple); // or is it modTuple.expand ?
   }

However, this doesn't work (dmd 2.065 linux64), because:
"Error: variable i cannot be read at compile time"

In C++11 (where almost everything else about variadic templates is 
pretty painful), this could probably be done like:
   template<typename... T>
   void fun(T... args)
   {
     origFun( transTupleElem(args)... );
   }
so I guess it shouldn't be too hard to do something equivalent in D?

I looked for some kind of staticMap() function that could map values 
from one tuple to another (of the same type), but only found 
std.typetuple.staticMap() which only seems to modify types in 
TypeTuples, not values in tuple instances.

Cheers,
Daniel
Jul 20 2014
next sibling parent reply "Vlad Levenfeld" <vlevenfeld gmail.com> writes:
Thats real weird that it would reject your "i" variable, given 
that T.length is known at compile time. I think this is a bug. I 
can get your code to compile if I change your foreach loop to 
this:

foreach(i, U; T)
    modTuple[i] = transTupleElem(argTuple[i]); // ok
Jul 20 2014
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 21.07.2014 03:05, schrieb Vlad Levenfeld:

 Thats real weird that it would reject your "i" variable, given that
 T.length is known at compile time. I think this is a bug. I can get your
 code to compile if I change your foreach loop to this:

 foreach(i, U; T)
     modTuple[i] = transTupleElem(argTuple[i]); // ok


That works indeeed.

I also tried "foreach(int i, x; argTuple)" which also with the same 
error as "foreach(i ; 0 .. T.length)".

As a workaround I created a TupleIndices template, that would return a 
tuple with 0 .. len and did "foreach(i; TupleIndices!(T.length)" but 
that was kinda messy and reminded me of the loops I had to jump through 
in C++ to do anything useful with variadic templates..

I agree that this is a bug, but at least your workaround is much nicer, 
thanks a lot! :-)

Cheers,
Daniel

( Vlad: Originally I answered you directly because the Thunderbird 
developers thought it was a clever idea to put an "answer" button that 
answers to the author instead of to the newsgroup prominently into the GUI)
Jul 20 2014
parent reply "Vlad Levenfeld" <vlevenfeld gmail.com> writes:
On Monday, 21 July 2014 at 01:29:40 UTC, Daniel Gibson wrote:

 Am 21.07.2014 03:05, schrieb Vlad Levenfeld:

 Thats real weird that it would reject your "i" variable, given 
 that
 T.length is known at compile time. I think this is a bug. I 
 can get your
 code to compile if I change your foreach loop to this:

 foreach(i, U; T)
    modTuple[i] = transTupleElem(argTuple[i]); // ok


 That works indeeed.

 I also tried "foreach(int i, x; argTuple)" which also with the 
 same error as "foreach(i ; 0 .. T.length)".

 As a workaround I created a TupleIndices template, that would 
 return a tuple with 0 .. len and did "foreach(i; 
 TupleIndices!(T.length)" but that was kinda messy and reminded 
 me of the loops I had to jump through in C++ to do anything 
 useful with variadic templates..

 I agree that this is a bug, but at least your workaround is 
 much nicer, thanks a lot! :-)

 Cheers,
 Daniel

 ( Vlad: Originally I answered you directly because the 
 Thunderbird developers thought it was a clever idea to put an 
 "answer" button that answers to the author instead of to the 
 newsgroup prominently into the GUI)


You're very welcome. The reason foreach(int i, x; argTuple) 
failed is because argTuple is a value (of type T), and so known 
only at run-time. To get a foreach to run at compile-time, you 
have to give it something whose value is known to the compiler 
(so, T and typeof(argTuple) would suffice, and 0..T.length really 
should as well).

A nice way to test is "pragma(msg, Foo)" where Foo is your 
argument... if its knowable at compile-time, then your compiler 
should output "Foo" (or the name of whatever its aliasing) to the 
console.
Jul 20 2014
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 21.07.2014 03:34, schrieb Vlad Levenfeld:

 You're very welcome. The reason foreach(int i, x; argTuple) failed is
 because argTuple is a value (of type T), and so known only at run-time.


Hmm but the only thing the compiler would need to know at compile-time 
is still i, which only depends on argTuple.length which is known at 
compile-time.

But ok, I can kinda understand that this doesn't work, probably foreach 
either operates completely on compile-time stuff (and does so 
statically) or completely on runtime-stuff, done dynamically.


 To get a foreach to run at compile-time, you have to give it something
 whose value is known to the compiler (so, T and typeof(argTuple) would
 suffice, and 0..T.length really should as well).


Yup


 A nice way to test is "pragma(msg, Foo)" where Foo is your argument...
 if its knowable at compile-time, then your compiler should output "Foo"
 (or the name of whatever its aliasing) to the console.


Thanks for the advice!

Cheers,
Daniel
Jul 20 2014
parent reply "TheFlyingFiddle" <kurtyan student.chalmers.se> writes:
On Monday, 21 July 2014 at 01:42:58 UTC, Daniel Gibson wrote:

 Am 21.07.2014 03:34, schrieb Vlad Levenfeld:

 To get a foreach to run at compile-time, you have to give it 
 something
 whose value is known to the compiler (so, T and 
 typeof(argTuple) would
 suffice, and 0..T.length really should as well).


 Yup


I use this when i want a compile time foreach(from a constant 
number). It's slightly longer but has worked great for me thus 
far.

template staticIota(size_t s, size_t e, size_t step = 1)
{
     import std.typetuple : TypeTuple;
     static if(s < e)
	alias staticIota = TypeTuple!(s, staticIota!(s + step, e));
     else
	alias staticIota = TypeTuple!();
}


usage:

unittest
{
     foreach(i; staticIota!(0, 10))
     {
         pragma(msg, i);
     }
}

//Outputs 1 to 10 at compile-time.
Jul 21 2014
next sibling parent "TheFlyingFiddle" <kurtyan student.chalmers.se> writes:
On Monday, 21 July 2014 at 15:04:14 UTC, TheFlyingFiddle wrote:

//Outputs 1 to 10 at compile-time.




Edit: 0 to 9
Jul 21 2014
prev sibling next sibling parent "TheFlyingFiddle" <kurtyan student.chalmers.se> writes:
On Monday, 21 July 2014 at 15:04:14 UTC, TheFlyingFiddle wrote:

 template staticIota(size_t s, size_t e, size_t step = 1)
 {
     import std.typetuple : TypeTuple;
     static if(s < e)
 	alias staticIota = TypeTuple!(s, staticIota!(s + step, e, 
 step));
     else
 	alias staticIota = TypeTuple!();
 }


Edit: Missed the second step.
Jul 21 2014
prev sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 21.07.2014 17:04, schrieb TheFlyingFiddle:

 On Monday, 21 July 2014 at 01:42:58 UTC, Daniel Gibson wrote:

 Am 21.07.2014 03:34, schrieb Vlad Levenfeld:

 To get a foreach to run at compile-time, you have to give it something
 whose value is known to the compiler (so, T and typeof(argTuple) would
 suffice, and 0..T.length really should as well).


 Yup


 I use this when i want a compile time foreach(from a constant number).
 It's slightly longer but has worked great for me thus far.

 template staticIota(size_t s, size_t e, size_t step = 1)
 {
      import std.typetuple : TypeTuple;
      static if(s < e)
      alias staticIota = TypeTuple!(s, staticIota!(s + step, e));
      else
      alias staticIota = TypeTuple!();
 }


Yeah, I had a similar workaround:

template TupleIndicesImpl(alias len, I...)
{
   static if(len == I.length) // also handles len == 0
     alias TupleIndicesImpl = I;
   else static if(I.length == 0)
     alias TupleIndicesImpl = TupleIndicesImpl!(len, 0);
   else // I contains 0 ... I.length - 1, so add I.length
     alias TupleIndicesImpl = TupleIndicesImpl!(len, I, I.length);
}

template TupleIndices(alias len)
{
   alias TupleIndices = TupleIndicesImpl!(len);
}

foreach(i; TupleIndices!(myTuple.length) { ... }

At least for iterating over a tuple Vlad's way suggestion
("foreach(i, U; TupleType)") is nicer and more concise.

However, having something like staticIota in the stdlib would probably 
make sense.

Cheers,
Daniel
Jul 21 2014
parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Mon, Jul 21, 2014 at 06:36:04PM +0200, Daniel Gibson via Digitalmars-d-learn
wrote:
[...]

 However, having something like staticIota in the stdlib would probably
 make sense.


[...]

It's already in std.typecons.

(Admittedly, that's not exactly the most obvious place to look for it...)


T

-- 
There are two ways to write error-free programs; only the third one works.
Jul 21 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
H. S. Teoh:


 It's already in std.typecons.


But it is not online yet?

Bye,
bearophile
Jul 21 2014
parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Mon, Jul 21, 2014 at 06:48:45PM +0000, bearophile via Digitalmars-d-learn
wrote:

 H. S. Teoh:
 

It's already in std.typecons.


 
 But it is not online yet?


[...]

Hmph. Apparently it is undocumented. :-/  It has been in Phobos since
last April, but was private until November when it became 'package'.

Hold on a sec... so how is it that my code compiles with it?! Apparently
some package protection bug? Sigh...

Recently there was a pull request that implements static foreach /
static while, but I can't seem to find it anymore. It's clear that this
functionality is desirable. Maybe we should rouse a racket on the main D
forum to either make staticIota public, or implement static foreach. ;-)


T

-- 
People tell me I'm stubborn, but I refuse to accept it!
Jul 21 2014
parent reply "Vlad Levenfeld" <vlevenfeld gmail.com> writes:
On Monday, 21 July 2014 at 19:02:59 UTC, H. S. Teoh via 
Digitalmars-d-learn wrote:

 functionality is desirable. Maybe we should rouse a racket on 
 the main D
 forum to either make staticIota public, or implement static 
 foreach. ;-)


static switch would be so sick. I frequently find myself doing 
some compile-time branching with more than 2 branches or with an 
enum (like for policies/strategies/whatever). Compile-time case 
labels would clean that code up, and final switch would be a 
maintenance improvement as well.

static while sounds cool, but how would it work? (as in use case, 
not implementation). The condition would have to be immutable, 
wouldn't it?
Jul 21 2014
parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Tue, Jul 22, 2014 at 02:47:51AM +0000, Vlad Levenfeld via
Digitalmars-d-learn wrote:

 On Monday, 21 July 2014 at 19:02:59 UTC, H. S. Teoh via Digitalmars-d-learn
 wrote:

functionality is desirable. Maybe we should rouse a racket on the
main D forum to either make staticIota public, or implement static
foreach. ;-)


 
 static switch would be so sick. I frequently find myself doing some
 compile-time branching with more than 2 branches or with an enum (like
 for policies/strategies/whatever). Compile-time case labels would
 clean that code up, and final switch would be a maintenance
 improvement as well.
 
 static while sounds cool, but how would it work? (as in use case, not
 implementation). The condition would have to be immutable, wouldn't
 it?


I don't know about use cases in general, but one place where it comes in
handy is in iterating over template argument lists ("type tuples").
Currently, I have to resort to:

	Tuple!(int,int,int,int,int) fields;
	foreach (i; staticIota!(0, n))
	{
		fields[i]++; // for example
	}

Which is not bad for simple operations, but would be cleaner if we had
static while / static foreach.


T

-- 
We've all heard that a million monkeys banging on a million typewriters will
eventually reproduce the entire works of Shakespeare.  Now, thanks to the
Internet, we know this is not true. -- Robert Wilensk
Jul 21 2014
parent reply "Vlad Levenfeld" <vlevenfeld gmail.com> writes:
I'm just confused about how static while is supposed to work 
because static foreach, to my understanding, would have to work 
by making a new type for each iteration. I say this because, 1) 
runtime foreach works like that (with type => range), and 2) 
without ctfe foreach, the only way I know of to iterate a 
typelist is to make a new type with one less element, so I 
imagine static foreach lowers to that.

I suppose its possible to make a struct with static immutable 
start and end iterators, and make new types out of advancing the 
start iterator until it was equal to the end. Seems like a step 
backward though.

Anyway my actual question is: if all values are constant at 
compile time, how would a static while loop terminate?
Jul 22 2014
parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Tue, Jul 22, 2014 at 03:52:14PM +0000, Vlad Levenfeld via
Digitalmars-d-learn wrote:

 I'm just confused about how static while is supposed to work because
 static foreach, to my understanding, would have to work by making a
 new type for each iteration. I say this because, 1) runtime foreach
 works like that (with type => range), and 2) without ctfe foreach, the
 only way I know of to iterate a typelist is to make a new type with
 one less element, so I imagine static foreach lowers to that.
 
 I suppose its possible to make a struct with static immutable start
 and end iterators, and make new types out of advancing the start
 iterator until it was equal to the end. Seems like a step backward
 though.
 
 Anyway my actual question is: if all values are constant at compile
 time, how would a static while loop terminate?


Basically, think of it as custom loop unrolling:

	TypeTuple!(
		int, "x",
		float, "y",
		uint, "z"
	) t;

	// This loop:
	foreach (i; staticIota(0, 3)) {
		t[i]++;
	}

	// Is equivalent to:
	t[0]++;
	t[1]++;
	t[2]++;

	// Which is equivalent to:
	t.x++;
	t.y++;
	t.z++;

The loop body is basically expanded for each iteration, with the loop
variable suitably substituted with each element of the typelist.


T

-- 
Microsoft is to operating systems & security ... what McDonalds is to gourmet
cooking.
Jul 22 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 22 July 2014 at 16:42:14 UTC, H. S. Teoh via 
Digitalmars-d-learn wrote:

 On Tue, Jul 22, 2014 at 03:52:14PM +0000, Vlad Levenfeld via 
 Digitalmars-d-learn wrote:

 Anyway my actual question is: if all values are constant at 
 compile
 time, how would a static while loop terminate?


 Basically, think of it as custom loop unrolling:

 	TypeTuple!(
 		int, "x",
 		float, "y",
 		uint, "z"
 	) t;

 	// This loop:
 	foreach (i; staticIota(0, 3)) {
 		t[i]++;
 	}

 	// Is equivalent to:
 	t[0]++;
 	t[1]++;
 	t[2]++;

 	// Which is equivalent to:
 	t.x++;
 	t.y++;
 	t.z++;

 The loop body is basically expanded for each iteration, with 
 the loop
 variable suitably substituted with each element of the typelist.


You're misunderstanding him. Your example is a static foreach, 
but Vlad asked about static while. I too don't see how a static 
while is supposed to work.
Jul 22 2014
parent "Vlad Levenfeld" <vlevenfeld gmail.com> writes:
Yes, though the loop unrolling is news to me. I'll have to keep
that in mind next time I'm trying to squeeze some extra
performance out of a loop.

btw, found a static switch enhancement request here:
https://issues.dlang.org/show_bug.cgi?id=6921
Jul 22 2014
prev sibling parent Daniel Gibson <metalcaedes gmail.com> writes:
Am 21.07.2014 20:09, schrieb H. S. Teoh via Digitalmars-d-learn:

 On Mon, Jul 21, 2014 at 06:36:04PM +0200, Daniel Gibson via
Digitalmars-d-learn wrote:
 [...]

 However, having something like staticIota in the stdlib would probably
 make sense.


 [...]

 It's already in std.typecons.

 (Admittedly, that's not exactly the most obvious place to look for it...)


 T


static.typecons is actually where I would have expected it, as it 
constructs a tuple.. but it isn't mentioned on 
http://dlang.org/library/std/typecons.html or 
http://dlang.org/phobos/std_typecons.html
and at least in my /usr/include/dmd/phobos/std/typecons.d (2.065) it's 
private:
private template staticIota(int beg, int end)
{ ... }
And it seems like I can't use it.
Anyway, good to know that it exists and hopefully future versions of D2 
make that function public, so thanks for showing up another alternative 
to solve my problem :-)

BTW: The name "Iota" is horrible.. it doesn't describe at all what the 
function does.
And "But C++11 STL has a function of the same name that does the same 
thing" or "some obscure programming language from the 60ies (APL) used 
the Greek iota letter to do this" is no excuse, one shouldn't expect 
potential D users to know about that (even after using C++ for years I 
never encountered std::iota..)
Maybe "Numerate" or something like that would be more descriptive..

Cheers,
Daniel
Jul 21 2014
prev sibling parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Mon, Jul 21, 2014 at 12:55:34AM +0200, Daniel Gibson via Digitalmars-d-learn
wrote:

 Hi,
 I have a variadic templated function and want to call a C varargs function.
 I want to be able to pass static arrays, which D2 passes by value and C by
 reference, so I'd like to automagically translate those arguments.
 
 My idea was something like this:
 
   extern (C) origFun(int x, ...);
 
   T transTupleElem(T)(T arg) { return arg; }
 
   float* transTupleElem(T : float[3])(T arg) {
     return arg.ptr;
   }
 
   void fun(T...)(int x, T argTuple) {
     // create a new tuple type that replaces all static float[3]
     // arrays with float* to emulate C call-by-reference behavior
     alias ReplaceAll!(float[3], float*, T) ModifiedTuple;
     ModifiedTuple modTuple;
 
     foreach(size_t i ; 0 .. T.length)
       modTuple[i] = transTupleElem(argTuple[i]); // BOOM!
 
     origFun(modTuple); // or is it modTuple.expand ?
   }
 
 However, this doesn't work (dmd 2.065 linux64), because:
 "Error: variable i cannot be read at compile time"


[...]

Try this:

	import std.typecons : staticIota;
	foreach (i; staticIota!(0, T.length))
		modTuple[i] = transTupleElem(argTuple[i]);


T

-- 
The volume of a pizza of thickness a and radius z can be described by the
following formula: pi zz a. -- Wouter Verhelst
Jul 21 2014
parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 21 July 2014 at 18:10:14 UTC, H. S. Teoh via 
Digitalmars-d-learn wrote:

 On Mon, Jul 21, 2014 at 12:55:34AM +0200, Daniel Gibson via 
 Digitalmars-d-learn wrote:

 Hi,
 I have a variadic templated function and want to call a C 
 varargs function.
 I want to be able to pass static arrays, which D2 passes by 
 value and C by
 reference, so I'd like to automagically translate those 
 arguments.
 
 My idea was something like this:
 
   extern (C) origFun(int x, ...);
 
   T transTupleElem(T)(T arg) { return arg; }
 
   float* transTupleElem(T : float[3])(T arg) {
     return arg.ptr;
   }
 
   void fun(T...)(int x, T argTuple) {
     // create a new tuple type that replaces all static 
 float[3]
     // arrays with float* to emulate C call-by-reference 
 behavior
     alias ReplaceAll!(float[3], float*, T) ModifiedTuple;
     ModifiedTuple modTuple;
 
     foreach(size_t i ; 0 .. T.length)
       modTuple[i] = transTupleElem(argTuple[i]); // BOOM!
 
     origFun(modTuple); // or is it modTuple.expand ?
   }
 
 However, this doesn't work (dmd 2.065 linux64), because:
 "Error: variable i cannot be read at compile time"


 [...]

 Try this:

 	import std.typecons : staticIota;
 	foreach (i; staticIota!(0, T.length))
 		modTuple[i] = transTupleElem(argTuple[i]);


 T


staticIota is marked package in std.typecons
Jul 21 2014