www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP for multiple auto ref values

reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
Hello,

The DIP is to allow multiple auto ref return values.

After discussion with Ilya in this [1] thread, I'm looking 
forward to taking
a DIP as a GSoC project. This a summary of my discussion with 
Ilya.

So, right now, we can return a ref value:

ref int foo1(int[] a) {
         return a[0];
}

void main() {
         int[] a = [1,2,3];
         foo1(a) = 4;  // a[0] is now 4
}

and also an auto ref value, which is the usual: Automatically
infer if you should return an lvalue (aka ref) or a value.

To have multiple return values, the possible workarounds are:

-- a) Multiple out/ref parameters

So, let's say I want to return 1 int and 1 double (values, 
nothing related
to refs):
void foo2(ref int x, ref double y) {
         x = 1;
         y = 1.2;
}

void main() {
         int a;
         double b;
         foo2(a, b);  // a == 1, b == 1.2
}

-- b) Use std.type.Tuple

import std.typecons;
Tuple!(int, double) foo3() {
     return tuple(1, 1.2);
}

void main() {
         int a;
         double b;
         auto res = foo3();
         a = res[0];
         b = res[1];
}

Ilya:


 Both of them don't allow to return multiple values by 
 reference: for
 example, references on array/container elements.


To return multiple ref values we could:

-- c. Add additoinal arguments as ref/out _pointer_ parameters

-- d. Use mir.functional.RefTuple.

Ilya:

 Both of them don't well fit to modern D, Mir, and future generic
 containers. For example, Mir's Series consist as pair of two 
 arrays
 (keys and values), keys are sorted. It would be awesome to use 
 D Range
 syntax (front, popFront, empty) to iterate Series by reference. 
 Also,
 it would be very good for perfomance, for D GC-free 
 reference-counted
 libraries.

 The possible (but may not be the best) syntax is:

 auto ref fun(int[] a)
 {
     return (a[0], a[1] * 3.4); // returns ref int and double
 }
 
 void handle(ref int a, double b)
 {
     a = cast(int) (b * b);
 }
 
 int twice(int a) { a * 2; }
 
 void main()
 {
     int[] ar = [1, 2];
     handle(fun(ar));
     auto (i : &$0, d, e : $0 + $1, f : $1.twice) = fun(ar);
 
     // i is a pointer to ar[0], type of int*
     // d stores a values of a[1] * 3.4, type of double
     // e stores value ar[0] + a[1] * 3.4, type of double
     // f stores value ar[0] * 2, type of int
 }


So, then I proposed that we could internalize RefTuple as part of 
the syntax.
That is, when one writes:


 auto ref fun(int[] a)

 {

     return (a[0], a[1] * 3.4); // returns ref int and double

 }


they should expect to get a RefTuple.

Then, Ilya pointed that:

 D has two kinds of struct tuples, expanded and not expanded.
 
 1. AliasSeq!(1, a, b,) - is expanded tuple.
 2. tuple(1, a, b) - is not expanded tuple (just a struct)
 3. tuple(1, a, b).expand is an expanded tuple where 'expand' is 
 an alias sequence of structs members.
 
 For not expanded return value the approach you have proposed is 
 a solution.
 For already expanded return value the approach isn't required.


The problem that he pointed is that:


 The RefTuple/Tuple is a structure. So, a compiler should 
 generate opAssign, and may also generate copy constructor and 
 destructor.
 So returning expanded tuple should be implemented in compiler 
 using a way that does not require to create a return type at 
 all.
 I think expanded tuple is more preferable, more elegant, and 
 does not require to incorporate a (Ref)Tuple struct 
 implementation into the language.


---- FINALLY ----

Here are the two variants for a solution from my point of view:
1) Incorporate the expanded tuple into the language, so that no 
type is created.
2) Incorporate RefTuple into the language, so that when one 
returns multiple
ref values, he gets a RefTuple.

Waiting for opinions.



[1] 
https://forum.dlang.org/thread/pvhacqomqgnhmqienfpi forum.dlang.org?page=3
Apr 09 2019
next sibling parent reply Andre Pany <andre s-e-a-p.de> writes:
On Tuesday, 9 April 2019 at 14:37:17 UTC, Stefanos Baziotis wrote:

 Hello,

 The DIP is to allow multiple auto ref return values.

 [...]


Just to make sure, you now there is also another dip touching 
tuples:

https://github.com/tgehr/DIPs/blob/tuple-syntax/DIPs/DIP1xxx-tg.md

Kind regards
Andre
Apr 09 2019
parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Tuesday, 9 April 2019 at 15:05:49 UTC, Andre Pany wrote:

 Just to make sure, you now there is also another dip touching 
 tuples:

 https://github.com/tgehr/DIPs/blob/tuple-syntax/DIPs/DIP1xxx-tg.md


No, I didn't, thanks a lot Andre. From what I can understand, 
having tuples
built into the language does not solve the problem of returning 
multiple
auto ref values (or just multiple ref values), correct?

- Stefanos
Apr 09 2019
next sibling parent reply Andre Pany <andre s-e-a-p.de> writes:
On Tuesday, 9 April 2019 at 15:12:10 UTC, Stefanos Baziotis wrote:

 On Tuesday, 9 April 2019 at 15:05:49 UTC, Andre Pany wrote:

 Just to make sure, you now there is also another dip touching 
 tuples:

 https://github.com/tgehr/DIPs/blob/tuple-syntax/DIPs/DIP1xxx-tg.md


 No, I didn't, thanks a lot Andre. From what I can understand, 
 having tuples
 built into the language does not solve the problem of returning 
 multiple
 auto ref values (or just multiple ref values), correct?

 - Stefanos


It doesn't solve the issue but maybe there is some info which you 
can reuse for your dip or maybe you need some part from the other 
dip.

Kind regards
Andre
Apr 09 2019
parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Tuesday, 9 April 2019 at 15:20:58 UTC, Andre Pany wrote:

 It doesn't solve the issue but maybe there is some info which 
 you can reuse for your dip or maybe you need some part from the 
 other dip.


Definitely. I have read it about 4 times by now, and surely I 
don't in any way pretend to know how to write DIPs. :P

Best,
Stefanos
Apr 09 2019
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2019-04-09 17:12, Stefanos Baziotis wrote:


 No, I didn't, thanks a lot Andre. From what I can understand, having tuples
 built into the language does not solve the problem of returning multiple
 auto ref values (or just multiple ref values), correct?


Technically returning a tuple would be returning a single value. But the 
value is a collection of values, so it contains multiple values. With 
some syntax sugar to auto expand the tuple it's very similar, if not, 
exactly the same as returning multiple values.

-- 
/Jacob Carlborg
Apr 09 2019
parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Tuesday, 9 April 2019 at 18:15:30 UTC, Jacob Carlborg wrote:

 Technically returning a tuple would be returning a single 
 value. But the value is a collection of values, so it contains 
 multiple values. With some syntax sugar to auto expand the 
 tuple it's very similar, if not, exactly the same as returning 
 multiple values.


I might be wrong but:
The problem is not returning multiple values. The problem is 
returning multiple
ref values. Even if the tuple is built into the language, a tuple 
can't have ref
values because ref is not a type, a problem for which RefTuple 
was created.
RefTuple has difficult usage in an of itself for the users. As 
you can see, my
original proposal was to incorporate RefTuple to the language but 
RefTuple
is a struct and that comes with an overhead. So, the other idea 
is to do something
similar to what you say, i.e. auto-expand the tuple at 
compile-time but with the
support of ref values.
Apr 09 2019
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09.04.19 16:37, Stefanos Baziotis wrote:

 
 
 Here are the two variants for a solution from my point of view:
 1) Incorporate the expanded tuple into the language, so that no type is 
 created.


Expanded tuples already exist, but you can't return them from functions:

---
auto ref seq(T...)(return auto ref T values){
     return values;
}
---
Error: functions cannot return a tuple
---

I think the reason for this restriction is that

1) Walter originally wanted multiple return values to follow calling 
conventions similar to passing multiple parameters.
2) It would need special name mangling.

The DIP could try to lift this restriction while also allowing multiple 
`ref` return values.


 2) Incorporate RefTuple into the language, so that when one returns 
 multiple ref values, he gets a RefTuple.
 
 Waiting for opinions.


I would presume that RefTuple does not work properly in  safe code? Your 
DIP probably would have to support ` safe` and `return` annotations. One 
question here will again be name mangling. (Which the tuple DIP itself 
sidesteps, but it seems a bit more complicated here)


In any case, this kind of thing should be an extension of the 
work-in-progress tuple DIP:
https://github.com/tgehr/DIPs/blob/tuple-syntax/DIPs/DIP1xxx-tg.md


Basically, on top of the tuple DIP, just handle this kind of thing:

---
void foo((ref int a, int b), int c){
     a=b+c;
}
(ref int, int) bar(return ref int x,int y){
     return (x,y);
}
void main(){
     auto x=(1,2), y=3;
     foo(x,y);
     assert(x==(5,2));
     x[1]+=1;
     foo(bar(x),y);
     assert(x==(6,3));
}
---

---
auto ref foo(return ref int x){
     return (x,1); // auto ref applies to individual fields if return 
value is a tuple literal
}

void main(){
     int x=2;
     foo(x)[0]+=1;
     assert(x==3);
     assert(foo(x)[1]==1);
}
---

---
import std.range, std.algorithm; // (need to be changed)
void main(){
     auto a=[1,2,3,4,5];
     foreach((i,ref j);enumerate(a)){
         a[i]=i;
     }
     assert(a==[0,1,2,3,4]);
}
---

Are you interested in DIP writing only or also compiler implementation?
My partial progress on a tuple DIP implementation is here:
https://github.com/dlang/dmd/compare/master...tgehr:tuple-syntax

It supports parts of the first three proposals from the DIP: unpacking 
variable declarations, calling functions with multiple parameters with a 
single tuple argument, and tuple literals.
Apr 09 2019
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09.04.19 23:05, Timon Gehr wrote:

 ---
 import std.range, std.algorithm; // (need to be changed)
 void main(){
      auto a=[1,2,3,4,5];
      foreach((i,ref j);enumerate(a)){
          a[i]=i;
      }
      assert(a==[0,1,2,3,4]);
 }
 ---


Obviously, this should have been:

---
import std.range, std.algorithm; // (need to be changed)
void main(){
      auto a=[1,2,3,4,5];
      foreach((i,ref j);enumerate(a)){
          j=i;
      }
      assert(a==[0,1,2,3,4]);
}
---
Apr 09 2019
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09.04.19 23:07, Timon Gehr wrote:

 ...
 
 Obviously, this should have been:
 
 ---
 import std.range, std.algorithm; // (need to be changed)
 void main(){
       auto a=[1,2,3,4,5];
       foreach((i,ref j);enumerate(a)){
           j=i;
       }
       assert(a==[0,1,2,3,4]);
 }
 ---


And if you want it to compile on 64 bits:

---
import std.range, std.algorithm; // (need to be changed)
void main(){
       size_t[] a=[1,2,3,4,5];
       foreach((i,ref j);enumerate(a)){
           j=i;
       }
       assert(a==[0,1,2,3,4]);
}
---

(I have sent this message before, but it does not seem to have made it 
to the server. I'm sorry if this is a double-post.)
Apr 09 2019
prev sibling parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Tuesday, 9 April 2019 at 21:05:25 UTC, Timon Gehr wrote:

 Expanded tuples already exist, but you can't return them from 
 functions:


True, with Ilya we meant incorporate it so that one can return 
them.


 2) It would need special name mangling.


That is an issue, I haven't thought of that, thanks!


 The DIP could try to lift this restriction while also allowing 
 multiple `ref` return values.


Yes, pretty much that was one of the proposals in -- FINALLY -- on
the original post


 I would presume that RefTuple does not work properly in  safe 
 code? Your DIP probably would have to support ` safe` and 
 `return` annotations. One question here will again be name 
 mangling. (Which the tuple DIP itself sidesteps, but it seems a 
 bit more complicated here)


I don't know how to tackle those yet but thanks for mentioning 
cause
I haven't thought of them.


 In any case, this kind of thing should be an extension of the 
 work-in-progress tuple DIP:
 https://github.com/tgehr/DIPs/blob/tuple-syntax/DIPs/DIP1xxx-tg.md


Actually, just know I realized that you're the one with the tuple 
DIP
mentioned previously.
I didn't know about that DIP, this indeed seems a good enhancement
to that DIP.


 ---
 void foo((ref int a, int b), int c){
     a=b+c;
 }
 (ref int, int) bar(return ref int x,int y){
     return (x,y);
 }


Yes, that is a possible syntax (that I have mentioned in the 
discussion
with Ilya, I might have forgot to include it in the summary 
post). I should
just mention again though that this thing can't be done by simply
incorporating tuples because they don't support refs
because ref is not a type. One of the important points of the DIP
that I try to think is how to support similar syntax (and the 
proposals
were the first thoughts of Ilya and me).


 ---

 ---
 auto ref foo(return ref int x){
     return (x,1); // auto ref applies to individual fields if 
 return value is a tuple literal
 }

 void main(){
     int x=2;
     foo(x)[0]+=1;
     assert(x==3);
     assert(foo(x)[1]==1);
 }


That is another possible syntax. This example actually looks
somewhat like Ilya's.


 ---

 import std.range, std.algorithm; // (need to be changed)
 void main(){
     auto a=[1,2,3,4,5];
     foreach((i,ref j);enumerate(a)){
         j=i;
     }
      assert(a==[0,1,2,3,4]);
 }


This might be a DIP in and of itself because I believe what
you're writing here is an improvement on foreach that will
actually require the ability to return multiple ref values.
I might be wrong though, look the 3rd point of Ilya in this [1]
post.


 Are you interested in DIP writing only or also compiler 
 implementation?


Actually, I think I will be way better in compiler implementation
because (as it is probably evident), I'm quite bad
in DIP writing (I should mention that I learned about D before 3 
weeks).
Of course I will try to do a good job on both but I might need 
some
help in the DIP writing.


 My partial progress on a tuple DIP implementation is here:
 https://github.com/dlang/dmd/compare/master...tgehr:tuple-syntax

 It supports parts of the first three proposals from the DIP: 
 unpacking variable declarations, calling functions with 
 multiple parameters with a single tuple argument, and tuple 
 literals.


That's great!

[1] 
https://forum.dlang.org/post/shqiuqfegpbrnjtzyvwf forum.dlang.org
Apr 09 2019