www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - AliasTuples, rather than Records, to return multiple values

reply "Dario Schiavon" <dario.schiavon gmail.com> writes:
Hi everybody!

I've been lurking this forum for quite some time, attracted by 
the power and elegance of D. Although I'm not a programmer by 
profession, I often happen to develop programs for scientific 
data evaluation at work and I've always being intrigued with 
programming languages since I was a teenager. A lot of time has 
passed since the last time I looked at D, and now I'm really 
impressed with the amount of progress that has been done.

However, I don't understand why some feature that would be so 
desirable are still not implemented and, although there are 
already a lot of posts about them, little to no progress is done 
at implementing them. One of these is using tuples to return 
multiple values from functions. I'd like to share with you my 
thoughts and opinions about how tuples might work and be useful 
in D. I hope you find this contribution useful, and you will let 
me understand where the problems are otherwise. I also apologize 
for the length of my post if it doesn't really contain anything 
new.

At the time being we have two kinds of tuples in D and a lot of 
confusion about them. The first one is the Tuple object in Phobos 
(std.typecons.Tuple), which I'm going to call "Record" for the 
rest of the post to avoid confusion. They are pretty similar to 
Python's tuples, except that they are not immutable and have 
named items (admittedly a useful addition).

By introducing Records, you were probably trying to achieve the 
following two points:

1) Provide a way to group values together, as if they were 
anonymous struct's. Unfortunately, Records are not compatible 
with ordinary struct's despite all their similarities. And I'm 
also not totally convinced they are that useful, except when 
dealing with point 2. They just confuse novices about whether 
they should be using Records or struct's, just like they can't 
choose between tuples and lists in Python.

2) Provide a mechanism for functions to return multiple values. 
It may be noted that functions in D can already return multiple 
values through out/ref arguments, but most people agree that 
returning multiple values would be a neater solution (I also do). 
This mechanism doesn't work yet because of the lack of compiler 
support. I don't understand, however, why Records should be a 
better candidate to implement this feature than TypeTuples (more 
about that later).

Before going on, let me open a parenthesis about how returning 
multiple values works in Python. Suppose that the function "func" 
returns two values. The following saves the two values in the 
variables a and b.

(a, b) = func()

The parentheses around the tuple are not necessary, but I include 
them anyway for clarity. I'd like you to notice that this syntax 
is treated specially in Python. Python's tuples are immutable so 
you can't assign values to them.

c = (0, 1)


different tuple

d = (a, b)



Ok, enough for Python, let's go on with D.

The second kind of tuple is the in-built construct used in 
templates to group template arguments (std.typetuple.TypeTuple). 
Let's call it AliasTuple for the rest of the post, since 
TypeTuple is really a misnomer (as others before me have already 
pointed out: they can contain more than just types).

It must be noted that AliasTuples are not containers. They may be 
considered a kind of compile-time container, but definitely not a 
run-time container. With this, I mean that they don't copy their 
content in a structured form in a determined region of memory at 
runtime, like arrays, linked-lists and Records do. This implies, 
for example, that we can't take their address or make an array of 
them. AliasTuples are just collections of aliases, they don't 
contain actual data. So they are not "just another struct-like 
container in the language", like Records are.

We may debate about how many defects AliasTuples have but, I 
guess, we all agree that they are an extremely useful construct 
for D's templates. Without them, templates in D would certainly 
be much more difficult to use and they would lose much of their 
power. Therefore, I hope nobody really intends to scrap them. If 
they have deficiencies (for instance, they can't actually be used 
to return multiple values from functions), I think we should 
improve them so that they cover all the useful use cases.

It is my opinion that AliasTuples are much more appropriate to 
manage multiple return values than Records. However, for that to 
be possible, we must solve some of their weaknesses. One of them 
is that there isn't a concise literal expression yet. Let's 
suppose that we can create a tuple like this:

 (1, 2, float)  // equivalent to TypeTuple!(1, 2, float)

Of course it would be preferable to have just the parentheses 
without the  . Unfortunately, it would clash with the normal 
parentheses and the comma expression. In Python it works that 
way, but single-valued tuples are awkwardly defined as (1,). 
Bearophile suggested for syntax (|1, 2|) (although that was for 
Records), which I happen to like even less than  (1, 2). Maybe 
someone will come out with a better syntax later on.

Since an AliasTuple contains just aliases, assigning a value to 
an item of the AliasTuple is actually equivalent to assign the 
value to the "thing" the alias points to. The following would be 
valid D code (it already works if you replace  () with 
TypeTuple!()).

int a;
alias  (1, 2, float, a) b;  // defines b to be the given tuple
b[3] = 3;  // assigns 3 to the variable a

The following snipped, instead, doesn't work yet but I think it 
should be made to work:

int a=1, b=2;
 (a, b) =  (b, a);  // should swap values between a and b
writeln(a, b);  // outputs "22" but should output "21"

Let's suppose it is possible to define a function that returns an 
AliasTuple. The following might be a possible syntax. If the 
return type of a function is an AliasTuple of only types, then 
the function should returns an AliasTuple of values with those 
types.

 (int, int) func() {
   return  (4, 5);
}
int x, y;
 (x, y) = func();

Note that the last line is not a special case as it is in Python. 
It naturally works that way because an AliasTuple is a collection 
of aliases and not a container. There should not be any 
implementation problem in this feature, because it would be just 
syntax sugar for:

int func(out int z) {
   z = 5;
   return 4;
}
int x, y;
x = func(y);

What do you think of it?
May 17 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 17/05/2012 16:20, Dario Schiavon a écrit :

 Hi everybody!

 I've been lurking this forum for quite some time, attracted by the power
 and elegance of D. Although I'm not a programmer by profession, I often
 happen to develop programs for scientific data evaluation at work and
 I've always being intrigued with programming languages since I was a
 teenager. A lot of time has passed since the last time I looked at D,
 and now I'm really impressed with the amount of progress that has been
 done.

 However, I don't understand why some feature that would be so desirable
 are still not implemented and, although there are already a lot of posts
 about them, little to no progress is done at implementing them. One of
 these is using tuples to return multiple values from functions. I'd like
 to share with you my thoughts and opinions about how tuples might work
 and be useful in D. I hope you find this contribution useful, and you
 will let me understand where the problems are otherwise. I also
 apologize for the length of my post if it doesn't really contain
 anything new.

 At the time being we have two kinds of tuples in D and a lot of
 confusion about them. The first one is the Tuple object in Phobos
 (std.typecons.Tuple), which I'm going to call "Record" for the rest of
 the post to avoid confusion. They are pretty similar to Python's tuples,
 except that they are not immutable and have named items (admittedly a
 useful addition).

 By introducing Records, you were probably trying to achieve the
 following two points:

 1) Provide a way to group values together, as if they were anonymous
 struct's. Unfortunately, Records are not compatible with ordinary
 struct's despite all their similarities. And I'm also not totally
 convinced they are that useful, except when dealing with point 2. They
 just confuse novices about whether they should be using Records or
 struct's, just like they can't choose between tuples and lists in Python.

 2) Provide a mechanism for functions to return multiple values. It may
 be noted that functions in D can already return multiple values through
 out/ref arguments, but most people agree that returning multiple values
 would be a neater solution (I also do). This mechanism doesn't work yet
 because of the lack of compiler support. I don't understand, however,
 why Records should be a better candidate to implement this feature than
 TypeTuples (more about that later).

 Before going on, let me open a parenthesis about how returning multiple
 values works in Python. Suppose that the function "func" returns two
 values. The following saves the two values in the variables a and b.

 (a, b) = func()

 The parentheses around the tuple are not necessary, but I include them
 anyway for clarity. I'd like you to notice that this syntax is treated
 specially in Python. Python's tuples are immutable so you can't assign
 values to them.

 c = (0, 1)


 different tuple

 d = (a, b)



 Ok, enough for Python, let's go on with D.

 The second kind of tuple is the in-built construct used in templates to
 group template arguments (std.typetuple.TypeTuple). Let's call it
 AliasTuple for the rest of the post, since TypeTuple is really a
 misnomer (as others before me have already pointed out: they can contain
 more than just types).

 It must be noted that AliasTuples are not containers. They may be
 considered a kind of compile-time container, but definitely not a
 run-time container. With this, I mean that they don't copy their content
 in a structured form in a determined region of memory at runtime, like
 arrays, linked-lists and Records do. This implies, for example, that we
 can't take their address or make an array of them. AliasTuples are just
 collections of aliases, they don't contain actual data. So they are not
 "just another struct-like container in the language", like Records are.

 We may debate about how many defects AliasTuples have but, I guess, we
 all agree that they are an extremely useful construct for D's templates.
 Without them, templates in D would certainly be much more difficult to
 use and they would lose much of their power. Therefore, I hope nobody
 really intends to scrap them. If they have deficiencies (for instance,
 they can't actually be used to return multiple values from functions), I
 think we should improve them so that they cover all the useful use cases.

 It is my opinion that AliasTuples are much more appropriate to manage
 multiple return values than Records. However, for that to be possible,
 we must solve some of their weaknesses. One of them is that there isn't
 a concise literal expression yet. Let's suppose that we can create a
 tuple like this:

  (1, 2, float) // equivalent to TypeTuple!(1, 2, float)

 Of course it would be preferable to have just the parentheses without
 the  . Unfortunately, it would clash with the normal parentheses and the
 comma expression. In Python it works that way, but single-valued tuples
 are awkwardly defined as (1,). Bearophile suggested for syntax (|1, 2|)
 (although that was for Records), which I happen to like even less than
  (1, 2). Maybe someone will come out with a better syntax later on.

 Since an AliasTuple contains just aliases, assigning a value to an item
 of the AliasTuple is actually equivalent to assign the value to the
 "thing" the alias points to. The following would be valid D code (it
 already works if you replace  () with TypeTuple!()).

 int a;
 alias  (1, 2, float, a) b; // defines b to be the given tuple
 b[3] = 3; // assigns 3 to the variable a

 The following snipped, instead, doesn't work yet but I think it should
 be made to work:

 int a=1, b=2;
  (a, b) =  (b, a); // should swap values between a and b
 writeln(a, b); // outputs "22" but should output "21"

 Let's suppose it is possible to define a function that returns an
 AliasTuple. The following might be a possible syntax. If the return type
 of a function is an AliasTuple of only types, then the function should
 returns an AliasTuple of values with those types.

  (int, int) func() {
 return  (4, 5);
 }
 int x, y;
  (x, y) = func();

 Note that the last line is not a special case as it is in Python. It
 naturally works that way because an AliasTuple is a collection of
 aliases and not a container. There should not be any implementation
 problem in this feature, because it would be just syntax sugar for:

 int func(out int z) {
 z = 5;
 return 4;
 }
 int x, y;
 x = func(y);

 What do you think of it?


I think you show a real need here, but I don't really like your 
proposal. I'd advocate for recycling the comma operator for tuple building.

This would be very similar to your proposal as a result, but no need to 
introduce a new syntax.
May 17 2012
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, May 17, 2012 at 07:20:38PM +0200, deadalnix wrote:
[...]

 I think you show a real need here, but I don't really like your
 proposal. I'd advocate for recycling the comma operator for tuple
 building.


+1.

I know the topic of comma operator has been beaten to death several
times over, but I'm curious, how much is it _actually_ being used
outside of for loops? Would introducing (x,y,z) tuple syntax _really_
break a lot of code? Would it even break _any_ code? -- since tuple
syntax would tend to be used where you normally don't use the comma
operator.


T

-- 
Never ascribe to malice that which is adequately explained by incompetence. --
Napoleon Bonaparte
May 17 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 17/05/2012 19:27, H. S. Teoh a écrit :

 On Thu, May 17, 2012 at 07:20:38PM +0200, deadalnix wrote:
 [...]

 I think you show a real need here, but I don't really like your
 proposal. I'd advocate for recycling the comma operator for tuple
 building.


 +1.

 I know the topic of comma operator has been beaten to death several
 times over, but I'm curious, how much is it _actually_ being used
 outside of for loops? Would introducing (x,y,z) tuple syntax _really_
 break a lot of code? Would it even break _any_ code? -- since tuple
 syntax would tend to be used where you normally don't use the comma
 operator.


 T


I'd advocate for the following behavior :
1/ void member of tuple are computed, but not stored in the tuple.
2/ A tuple with one member can unpack automagically.

With both, I'm pretty the code broken is close to none.

To go further, I advocate for declaration to be expression. It would 
allow (int a, int b) = foo();

Which rox, as Andrei said :D
May 17 2012
parent "Dario Schiavon" <dario.schiavon gmail.com> writes:
On Thursday, 17 May 2012 at 23:11:55 UTC, deadalnix wrote:

 Le 17/05/2012 19:27, H. S. Teoh a écrit :

 On Thu, May 17, 2012 at 07:20:38PM +0200, deadalnix wrote:
 [...]

 I think you show a real need here, but I don't really like 
 your
 proposal. I'd advocate for recycling the comma operator for 
 tuple
 building.


 +1.

 I know the topic of comma operator has been beaten to death 
 several
 times over, but I'm curious, how much is it _actually_ being 
 used
 outside of for loops? Would introducing (x,y,z) tuple syntax 
 _really_
 break a lot of code? Would it even break _any_ code? -- since 
 tuple
 syntax would tend to be used where you normally don't use the 
 comma
 operator.


 T


 I'd advocate for the following behavior :
 1/ void member of tuple are computed, but not stored in the 
 tuple.
 2/ A tuple with one member can unpack automagically.

 With both, I'm pretty the code broken is close to none.

 To go further, I advocate for declaration to be expression. It 
 would allow (int a, int b) = foo();

 Which rox, as Andrei said :D


Can you please argument the reasons behind the need of those two 
rules? Do you propose void members to allow for empty and 
single-item tuples, as in "void" and "a,void"? And why should a 
single-item tuple automagically unpack instead of being unpacked 
with tuple[0]?

Declarations as expressions are cool! :D
May 18 2012
prev sibling parent reply "Dario Schiavon" <dario.schiavon gmail.com> writes:
On Thursday, 17 May 2012 at 17:13:50 UTC, deadalnix wrote:

 Le 17/05/2012 16:20, Dario Schiavon a écrit :

 Hi everybody!

 I've been lurking this forum for quite some time, attracted by 
 the power
 and elegance of D. Although I'm not a programmer by 
 profession, I often
 happen to develop programs for scientific data evaluation at 
 work and
 I've always being intrigued with programming languages since I 
 was a
 teenager. A lot of time has passed since the last time I 
 looked at D,
 and now I'm really impressed with the amount of progress that 
 has been
 done.

 However, I don't understand why some feature that would be so 
 desirable
 are still not implemented and, although there are already a 
 lot of posts
 about them, little to no progress is done at implementing 
 them. One of
 these is using tuples to return multiple values from 
 functions. I'd like
 to share with you my thoughts and opinions about how tuples 
 might work
 and be useful in D. I hope you find this contribution useful, 
 and you
 will let me understand where the problems are otherwise. I also
 apologize for the length of my post if it doesn't really 
 contain
 anything new.

 At the time being we have two kinds of tuples in D and a lot of
 confusion about them. The first one is the Tuple object in 
 Phobos
 (std.typecons.Tuple), which I'm going to call "Record" for the 
 rest of
 the post to avoid confusion. They are pretty similar to 
 Python's tuples,
 except that they are not immutable and have named items 
 (admittedly a
 useful addition).

 By introducing Records, you were probably trying to achieve the
 following two points:

 1) Provide a way to group values together, as if they were 
 anonymous
 struct's. Unfortunately, Records are not compatible with 
 ordinary
 struct's despite all their similarities. And I'm also not 
 totally
 convinced they are that useful, except when dealing with point 
 2. They
 just confuse novices about whether they should be using 
 Records or
 struct's, just like they can't choose between tuples and lists 
 in Python.

 2) Provide a mechanism for functions to return multiple 
 values. It may
 be noted that functions in D can already return multiple 
 values through
 out/ref arguments, but most people agree that returning 
 multiple values
 would be a neater solution (I also do). This mechanism doesn't 
 work yet
 because of the lack of compiler support. I don't understand, 
 however,
 why Records should be a better candidate to implement this 
 feature than
 TypeTuples (more about that later).

 Before going on, let me open a parenthesis about how returning 
 multiple
 values works in Python. Suppose that the function "func" 
 returns two
 values. The following saves the two values in the variables a 
 and b.

 (a, b) = func()

 The parentheses around the tuple are not necessary, but I 
 include them
 anyway for clarity. I'd like you to notice that this syntax is 
 treated
 specially in Python. Python's tuples are immutable so you 
 can't assign
 values to them.

 c = (0, 1)


 a
 different tuple

 d = (a, b)



 Ok, enough for Python, let's go on with D.

 The second kind of tuple is the in-built construct used in 
 templates to
 group template arguments (std.typetuple.TypeTuple). Let's call 
 it
 AliasTuple for the rest of the post, since TypeTuple is really 
 a
 misnomer (as others before me have already pointed out: they 
 can contain
 more than just types).

 It must be noted that AliasTuples are not containers. They may 
 be
 considered a kind of compile-time container, but definitely 
 not a
 run-time container. With this, I mean that they don't copy 
 their content
 in a structured form in a determined region of memory at 
 runtime, like
 arrays, linked-lists and Records do. This implies, for 
 example, that we
 can't take their address or make an array of them. AliasTuples 
 are just
 collections of aliases, they don't contain actual data. So 
 they are not
 "just another struct-like container in the language", like 
 Records are.

 We may debate about how many defects AliasTuples have but, I 
 guess, we
 all agree that they are an extremely useful construct for D's 
 templates.
 Without them, templates in D would certainly be much more 
 difficult to
 use and they would lose much of their power. Therefore, I hope 
 nobody
 really intends to scrap them. If they have deficiencies (for 
 instance,
 they can't actually be used to return multiple values from 
 functions), I
 think we should improve them so that they cover all the useful 
 use cases.

 It is my opinion that AliasTuples are much more appropriate to 
 manage
 multiple return values than Records. However, for that to be 
 possible,
 we must solve some of their weaknesses. One of them is that 
 there isn't
 a concise literal expression yet. Let's suppose that we can 
 create a
 tuple like this:

  (1, 2, float) // equivalent to TypeTuple!(1, 2, float)

 Of course it would be preferable to have just the parentheses 
 without
 the  . Unfortunately, it would clash with the normal 
 parentheses and the
 comma expression. In Python it works that way, but 
 single-valued tuples
 are awkwardly defined as (1,). Bearophile suggested for syntax 
 (|1, 2|)
 (although that was for Records), which I happen to like even 
 less than
  (1, 2). Maybe someone will come out with a better syntax 
 later on.

 Since an AliasTuple contains just aliases, assigning a value 
 to an item
 of the AliasTuple is actually equivalent to assign the value 
 to the
 "thing" the alias points to. The following would be valid D 
 code (it
 already works if you replace  () with TypeTuple!()).

 int a;
 alias  (1, 2, float, a) b; // defines b to be the given tuple
 b[3] = 3; // assigns 3 to the variable a

 The following snipped, instead, doesn't work yet but I think 
 it should
 be made to work:

 int a=1, b=2;
  (a, b) =  (b, a); // should swap values between a and b
 writeln(a, b); // outputs "22" but should output "21"

 Let's suppose it is possible to define a function that returns 
 an
 AliasTuple. The following might be a possible syntax. If the 
 return type
 of a function is an AliasTuple of only types, then the 
 function should
 returns an AliasTuple of values with those types.

  (int, int) func() {
 return  (4, 5);
 }
 int x, y;
  (x, y) = func();

 Note that the last line is not a special case as it is in 
 Python. It
 naturally works that way because an AliasTuple is a collection 
 of
 aliases and not a container. There should not be any 
 implementation
 problem in this feature, because it would be just syntax sugar 
 for:

 int func(out int z) {
 z = 5;
 return 4;
 }
 int x, y;
 x = func(y);

 What do you think of it?


 I think you show a real need here, but I don't really like your 
 proposal. I'd advocate for recycling the comma operator for 
 tuple building.

 This would be very similar to your proposal as a result, but no 
 need to introduce a new syntax.


As I see it, creating single-item tuples would still be difficult 
with the comma syntax, except by introducing syntaxes like (a,) 
or (a,void), which don't look very good. Sure, single-item tuples 
don't appear that useful at first, but let's assume you want to 
append a new item to the tuple: can you write anything better 
than tuple~(a,)? Python's solution is not very elegant in these 
cases.

However, another point of my post was whether we really need 
Records (std.typecons.Tuple's), which take approximately the same 
role as traditional struct's after all, to allow returning 
multiple values from functions. Wouldn't AliasTuples take on the 
role as well?
May 18 2012
parent reply "Dario Schiavon" <dario.schiavon gmail.com> writes:
On Friday, 18 May 2012 at 10:47:11 UTC, Dario Schiavon wrote:

 On Thursday, 17 May 2012 at 17:13:50 UTC, deadalnix wrote:

 [...]

 I think you show a real need here, but I don't really like 
 your proposal. I'd advocate for recycling the comma operator 
 for tuple building.

 This would be very similar to your proposal as a result, but 
 no need to introduce a new syntax.


 As I see it, creating single-item tuples would still be 
 difficult with the comma syntax, except by introducing syntaxes 
 like (a,) or (a,void), which don't look very good. Sure, 
 single-item tuples don't appear that useful at first, but let's 
 assume you want to append a new item to the tuple: can you 
 write anything better than tuple~(a,)? Python's solution is not 
 very elegant in these cases.


Although, it just came to my mind, appending new items to tuples 
is quite easy with the actual rules for AliasTuples. Since 
AliasTuples can't have a hierarchy, they always unpack 
automatically. So the expression "tuple,a" would actually append 
the item "a" to the AliasTuple "tuple". The following works 
already in D:

import std.stdio;
import std.typetuple;

void main() {
   alias TypeTuple!(1, 2) a;
   alias TypeTuple!(a, 3) b;  // appends 3 to a
   writeln(b);  // prints "123"
}

Appending items to tuples actually covers 99% of my needs of 
single-item tuples in Python. Can anyone find other needs for 
single-item tuples? Or for empty tuples?


 However, another point of my post was whether we really need 
 Records (std.typecons.Tuple's), which take approximately the 
 same role as traditional struct's after all, to allow returning 
 multiple values from functions. Wouldn't AliasTuples take on 
 the role as well?
May 18 2012
parent travert phare.normalesup.org (Christophe Travert) writes:
"Dario Schiavon" , dans le message (digitalmars.D:167822), a écrit :

 void main() {
    alias TypeTuple!(1, 2) a;
    alias TypeTuple!(a, 3) b;  // appends 3 to a
    writeln(b);  // prints "123"
 }
 
 Appending items to tuples actually covers 99% of my needs of 
 single-item tuples in Python. Can anyone find other needs for 
 single-item tuples? Or for empty tuples?
 

 However, another point of my post was whether we really need 
 Records (std.typecons.Tuple's), which take approximately the 
 same role as traditional struct's after all, to allow returning 
 multiple values from functions. Wouldn't AliasTuples take on 
 the role as well?





Personally, I don't like Tuple unpacking automagically. It prevents 
creating a single element Typle, or a Tuple of Tuple, which may be 
harmful if Tuple gain a greater role than they have now. Creating a 
Tuple of Tuple is impossible or require heavy workarrounds with 
automagically unpacking Tuples, whereas with not automagically unpacking 
Tuples, it is very easy to introduce an unpacking syntax when necessary 
with a random unary operator.

-- 
Christophe
May 22 2012