www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - inout and opApply

reply Shachar Shemesh <shachar weka.io> writes:
Please consider the following program:

struct V(T) {
     int opApply(scope int delegate(ref T value) dg) {
         return 0;
     }
     int opApply(scope int delegate(ref const T value) dg) const {
         return 0;
     }
}

struct V1(T) {
     int opApply(scope int delegate(ref inout T value) dg) inout {
         return 0;
     }
}

void main() {
     const V!int mytype1;
     V!int mytype2;
     V1!int mytype3;

     foreach( v; mytype1 ) {
     }
     foreach( v; mytype2 ) {
     }
     foreach( v; mytype3 ) { // <- This line doesn't compile: cannot 
uniquely infer foreach argument types
     }
}

So the only way to get inout to work is... not to use it?

Shachar
Jun 20 2016
parent reply Dicebot <public dicebot.lv> writes:
On 06/20/2016 04:47 PM, Shachar Shemesh wrote:

 Please consider the following program:
 
 struct V(T) {
     int opApply(scope int delegate(ref T value) dg) {
         return 0;
     }
     int opApply(scope int delegate(ref const T value) dg) const {
         return 0;
     }
 }
 
 struct V1(T) {
     int opApply(scope int delegate(ref inout T value) dg) inout {
         return 0;
     }
 }
 
 void main() {
     const V!int mytype1;
     V!int mytype2;
     V1!int mytype3;
 
     foreach( v; mytype1 ) {
     }
     foreach( v; mytype2 ) {
     }
     foreach( v; mytype3 ) { // <- This line doesn't compile: cannot
 uniquely infer foreach argument types
     }
 }
 
 So the only way to get inout to work is... not to use it?
 
 Shachar


I think Steven has mentioned this issue during his DConf inout talk.
Jun 20 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/20/16 9:55 AM, Dicebot wrote:

 On 06/20/2016 04:47 PM, Shachar Shemesh wrote:

 Please consider the following program:

 struct V(T) {
     int opApply(scope int delegate(ref T value) dg) {
         return 0;
     }
     int opApply(scope int delegate(ref const T value) dg) const {
         return 0;
     }
 }

 struct V1(T) {
     int opApply(scope int delegate(ref inout T value) dg) inout {
         return 0;
     }
 }

 void main() {
     const V!int mytype1;
     V!int mytype2;
     V1!int mytype3;

     foreach( v; mytype1 ) {
     }
     foreach( v; mytype2 ) {
     }
     foreach( v; mytype3 ) { // <- This line doesn't compile: cannot
 uniquely infer foreach argument types
     }
 }

 So the only way to get inout to work is... not to use it?


 I think Steven has mentioned this issue during his DConf inout talk.


Yes, it's something I'm going to post about soon.

In essence, the delegate which takes inout parameter is currently its 
own wrapping of inout. Meaning, the implicit delegate the compiler 
creates does not participate in the inout wrapping for the opApply call.

Consider how this works for the compiler. I'll give you an example:

V1!int mutableX;

foreach(ref int a; mutableX)
{
    a = 5;
}

Here, I've removed the type inference to set that issue aside for now.

What the compiler does is convert your foreach body into a local 
delegate. This local delegate looks like this:

int __foreachBody(ref int a)
{
    a = 5;

    // this is added by the compiler for flow control
    return 0;
}

and then calls:

switch(mutableX.opApply(&__foreachBody))
{
    // process specialized returns from delegate
}

So, ignoring the special opApply machinery, we can see the fundamental 
issue here is that we are passing a function pointer of type int (ref 
int a) into the opApply. However, currently, the compiler treats a 
function pointer with inout parameters to be simply a pointer to an 
inout function. The constructed delegate cannot be this, since a is ref 
int, not ref inout(int). So it cannot compile. We could change the body 
to not modify a, and possibly we could create a solution, but it's not 
ideal.

What I would like the compiler to do (and I went over this in my talk), 
is to allow the compiler to inout-wrap a delegate along with the other 
inout prameters to the function. That is, for:

int opApply(scope int delegate(ref inout T value) dg) inout

The inout inside the delegate is wrapped just like the inout of the 
'this' parameter. effectively, this becomes equivalent to several 
function signatures:

int opApply(scope int delegate(ref T value) dg)
int opApply(scope int delegate(ref const T value) dg) const
int opApply(scope int delegate(ref immutable T value) dg) immutable
int opApply(scope int delegate(ref inout T value) dg) inout

And interestingly enough, the rules are kind of backwards for delegates 
-- while inout doesn't cast to anything, delegates with inout parameters 
can cast to any type of mutability modifier (I believe this is called 
contravariance). This is because the actual function is inout, so it 
cannot harm the mutability. So something like this:

foreach(inout a; anyV1)
{
}

should work for any flavor of V1.

I think this should work with existing code, and would simply open up 
things like opApply to inout support.

The only issue is that the compiler has to assume that while calling the 
delegate, the inout parameters passed COULD change value, because it 
doesn't know whether the passed delegate is truly inout, or just matches 
the constancy of the type, which could potentially be mutable. This 
differs from current expectations of inout delegate or function pointers.

It's a lot of complex explanation, but the end result is that you could 
simply tag your opApply's with inout and have one version for all modifiers.

-Steve
Jun 20 2016
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 20.06.2016 17:09, Steven Schveighoffer wrote:

 What I would like the compiler to do (and I went over this in my talk),
 is to allow the compiler to inout-wrap a delegate along with the other
 inout prameters to the function. That is, for:

 int opApply(scope int delegate(ref inout T value) dg) inout

 The inout inside the delegate is wrapped just like the inout of the
 'this' parameter. effectively, this becomes equivalent to several
 function signatures:

 int opApply(scope int delegate(ref T value) dg)
 int opApply(scope int delegate(ref const T value) dg) const
 int opApply(scope int delegate(ref immutable T value) dg) immutable
 int opApply(scope int delegate(ref inout T value) dg) inout

 And interestingly enough, the rules are kind of backwards for delegates
 -- while inout doesn't cast to anything, delegates with inout parameters
 can cast to any type of mutability modifier (I believe this is called
 contravariance). This is because the actual function is inout, so it
 cannot harm the mutability. So something like this:

 foreach(inout a; anyV1)
 {
 }

 should work for any flavor of V1.

 I think this should work with existing code, and would simply open up
 things like opApply to inout support.

 The only issue is that the compiler has to assume that while calling the
 delegate, the inout parameters passed COULD change value, because it
 doesn't know whether the passed delegate is truly inout, or just matches
 the constancy of the type, which could potentially be mutable. This
 differs from current expectations of inout delegate or function pointers.

 It's a lot of complex explanation, but the end result is that you could
 simply tag your opApply's with inout and have one version for all modifiers.


The problem here is that both variants make sense depending on context 
and there is no syntax to distinguish between them. This proposal 
interacts in a weird way with IFTI.
Jun 21 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/21/16 5:19 PM, Timon Gehr wrote:


 The problem here is that both variants make sense depending on context
 and there is no syntax to distinguish between them. This proposal
 interacts in a weird way with IFTI.


I know you are probably right, but can you explain maybe via an example 
what you mean? I'm not understanding your point.

-Steve
Jun 23 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 23.06.2016 14:58, Steven Schveighoffer wrote:

 On 6/21/16 5:19 PM, Timon Gehr wrote:


 The problem here is that both variants make sense depending on context
 and there is no syntax to distinguish between them. This proposal
 interacts in a weird way with IFTI.


 I know you are probably right, but can you explain maybe via an example
 what you mean? I'm not understanding your point.

 -Steve


One might legitimately want to pass a inout delegate to some inout 
function and then use the delegate on data with arbitrary mutability 
qualifiers, not just inout.


The IFTI comment was about for example something like this:

struct S{
     void bar(inout int){}
     void foo(T)(T arg)inout{
         // this looks like it can be called with any argument
         arg(&bar);
     }
}

void main(){
     immutable(S) s;
     void delegate(void delegate(inout(int))) dg = x=>x(2);
     s.foo(dg); // today, this works
     // with the proposed semantics, IFTI deduces
     // T = void delegate(void delegate(inout(int)))
     // but then cannot convert dg to a T after
     // inout resolution
}

But I don't feel strongly about which of the two variants it should be. 
It's weird either way, as long as there can be only one inout in scope.
Jun 23 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/23/16 3:20 PM, Timon Gehr wrote:

 On 23.06.2016 14:58, Steven Schveighoffer wrote:

 On 6/21/16 5:19 PM, Timon Gehr wrote:


 The problem here is that both variants make sense depending on context
 and there is no syntax to distinguish between them. This proposal
 interacts in a weird way with IFTI.


 I know you are probably right, but can you explain maybe via an example
 what you mean? I'm not understanding your point.


 One might legitimately want to pass a inout delegate to some inout
 function and then use the delegate on data with arbitrary mutability
 qualifiers, not just inout.


As you say, we don't have a lot of options. We only have one designation 
for inout, and we must treat it the same no matter where it is.

A possible workaround is to store the inout function pointer/delegate as 
a global variable. I don't like that idea at all, but it would work.

Perhaps we could find a way to mark a function parameter as not being 
part of the inout wrapping, or perhaps mark it as *being* part of the 
inout wrapping (to be backwards compatible). But this is a special thing 
for delegates and function pointers only, since it makes no sense for 
plain parameters.

The more I think about it, the more I think we need to have a clear 
demarcation of when you want the inout-containing delegate to 
participate in the wrapping or not. We can't eliminate existing 
behavior, and without special marking, the rules are going to be ugly 
and unintuitive.


 The IFTI comment was about for example something like this:

 struct S{
     void bar(inout int){}
     void foo(T)(T arg)inout{
         // this looks like it can be called with any argument
         arg(&bar);
     }
 }

 void main(){
     immutable(S) s;
     void delegate(void delegate(inout(int))) dg = x=>x(2);
     s.foo(dg); // today, this works
     // with the proposed semantics, IFTI deduces
     // T = void delegate(void delegate(inout(int)))
     // but then cannot convert dg to a T after
     // inout resolution
 }


Considering the possible use cases for something like this, I think we 
are better off having this limitation than not being able to use opApply 
with inout.

-Steve
Jun 23 2016
parent reply Shachar Shemesh <shachar weka.io> writes:
On 24/06/16 01:22, Steven Schveighoffer wrote:


 Considering the possible use cases for something like this, I think we
 are better off having this limitation than not being able to use opApply
 with inout.

 -Steve


I am not sure I followed the discussion correctly. If I did, however, it 
seems to me the following would also become impossible:

size_t numElements(T)(const ref T val) {
   size_t num;

   foreach(i; val) {
     num++;
   }

   return num;
}

If "val" being const means that the delegate passed to opApply must also 
be const, the meaning is that that delegate may not change "num", which 
seems, to me, unworkable.

All assuming I correctly understood your suggestion.

Shachar
Jun 23 2016
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/24/16 2:08 AM, Shachar Shemesh wrote:

 On 24/06/16 01:22, Steven Schveighoffer wrote:


 Considering the possible use cases for something like this, I think we
 are better off having this limitation than not being able to use opApply
 with inout.


 I am not sure I followed the discussion correctly. If I did, however, it
 seems to me the following would also become impossible:

 size_t numElements(T)(const ref T val) {
   size_t num;

   foreach(i; val) {
     num++;
   }

   return num;
 }


No, this will work fine. Examine the opApply function again:

int opApply(scope int delegate(ref inout T) dg) inout

Note that the delegate itself is NOT marked inout. Just the parameter it 
takes is (and the aggregate). The compiler is going to enforce that 
while inside opApply, the structure itself is inout, and not mutable. 
However, your delegate will not be inout (or const), and can modify 
whatever it likes in terms of your function attributes. The compiler 
simply allows the implicit wrapping using inout from `int delegate(ref 
const T)` to `int delegate(ref inout T)`.

-Steve
Jun 24 2016
prev sibling parent reply Kagamin <spam here.lot> writes:
On Monday, 20 June 2016 at 15:09:22 UTC, Steven Schveighoffer 
wrote:

 int opApply(scope int delegate(ref inout T value) dg) inout

 The inout inside the delegate is wrapped just like the inout of 
 the 'this' parameter. effectively, this becomes equivalent to 
 several function signatures:

 int opApply(scope int delegate(ref T value) dg)
 int opApply(scope int delegate(ref const T value) dg) const
 int opApply(scope int delegate(ref immutable T value) dg) 
 immutable
 int opApply(scope int delegate(ref inout T value) dg) inout


AFAIK, there was a bugzilla issue for it. Wasn't it implemented?
Jun 22 2016
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/22/16 6:28 AM, Kagamin wrote:

 On Monday, 20 June 2016 at 15:09:22 UTC, Steven Schveighoffer wrote:

 int opApply(scope int delegate(ref inout T value) dg) inout

 The inout inside the delegate is wrapped just like the inout of the
 'this' parameter. effectively, this becomes equivalent to several
 function signatures:

 int opApply(scope int delegate(ref T value) dg)
 int opApply(scope int delegate(ref const T value) dg) const
 int opApply(scope int delegate(ref immutable T value) dg) immutable
 int opApply(scope int delegate(ref inout T value) dg) inout


 AFAIK, there was a bugzilla issue for it. Wasn't it implemented?


No, this hasn't happened yet. There may be a bugzilla issue for it, I'm 
not sure.

-Steve
Jun 23 2016