www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - inout and function/delegate parameters

reply Stewart Gordon <smjg_1998 yahoo.com> writes:
At the moment, if a function has an inout parameter, it must have an inout
return type.

But this prevents doing stuff like

     void test(ref inout(int)[] x, inout(int)[] y) {
         x = y;
     }

or passing the constancy through to a delegate instead of a return value.

A typical use case of the latter is to define an opApply that works regardless
of the 
constancy of this and allows the delegate to modify the iterated-through
objects _if_ this 
is mutable.

     int opApply(int delegate(ref inout(T)) dg) inout;

But then I realised a potential ambiguity:
(a) the constancy is passed through to the delegate
(b) the delegate has an inout parameter in its own right

If we go by interpretation (b), then each signature contains only one inout, so
even if we 
relaxed the rules to allow this it would just be equivalent to

     int opApply(int delegate(ref const(T)) dg) const;

however, this won't always be true in the general case.

The essence of functions with inout parameters is that they have a hidden
constancy 
parameter.  This is essentially a template parameter, except that only one
instance of the 
function is generated, rather like Java generics.  If we made this parameter
explicit in 
the code, we could distinguish the two meanings:

(a) int opApply(constancy K)(int delegate(ref K(T)) dg) K;
(b) int opApply(constancy K)(int delegate(constancy L)(ref L(T)) dg) K;

Moreover, in case (a), opApply would accept for dg:
- an int delegate(ref T) only if this is mutable
- an int delegate(ref immutable(T)) only if this is immutable
- an int delegate(ref const(T)), or a delegate that is itself
constancy-templated, 
regardless of the constancy of this

Perhaps the simplest example where meaning (b) is actually useful is

     inout(char)[] process(inout(char)[] delegate(inout(char)[]) dg,
inout(char[]) text) {
         return text;
     }

but still, somebody might want meaning (a).  Anyway, under DMD 2.058 (Win32)
this gives

inout_delegate.d(1): Error: inout must be all or none on top level for 
inout(char)[](inout(char)[] function(inout(char)[]) fn, inout(char[]) text)

but why?  At least it seems that DMD acknowledges the ambiguity, even if the
error message 
doesn't make sense.


The question really is: When inout is applied both to a parameter in a
function's 
signature and to something in the signature of a function/delegate parameter
therewithin, 
how should it be interpreted?  The spec doesn't seem to address the issue at
all.  Indeed, 
we can ask two things:
- what actually does the compiler make of it at the moment?
- what would be the ideal way for it to work?

Possibilities I can see:

- always (a)
- always (b)
- (a) if at either level inout only occurs once, otherwise (b) (probably
undesirable 
because of fragility)
- just reject such signatures as ambiguous

And moreover, should we support some syntax (similar to what I've used here or
otherwise) 
to state explicitly whether we want to pass the constancy through to the
delegate 
signature or not?

Stewart.
Feb 19 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/19/2012 03:27 PM, Stewart Gordon wrote:
 At the moment, if a function has an inout parameter, it must have an
 inout return type.

 But this prevents doing stuff like

 void test(ref inout(int)[] x, inout(int)[] y) {
 x = y;
 }

 or passing the constancy through to a delegate instead of a return value.

 A typical use case of the latter is to define an opApply that works
 regardless of the constancy of this and allows the delegate to modify
 the iterated-through objects _if_ this is mutable.

 int opApply(int delegate(ref inout(T)) dg) inout;

 But then I realised a potential ambiguity:
 (a) the constancy is passed through to the delegate
 (b) the delegate has an inout parameter in its own right

 If we go by interpretation (b), then each signature contains only one
 inout, so even if we relaxed the rules to allow this it would just be
 equivalent to

 int opApply(int delegate(ref const(T)) dg) const;

 however, this won't always be true in the general case.

 The essence of functions with inout parameters is that they have a
 hidden constancy parameter. This is essentially a template parameter,
 except that only one instance of the function is generated, rather like
 Java generics. If we made this parameter explicit in the code, we could
 distinguish the two meanings:

 (a) int opApply(constancy K)(int delegate(ref K(T)) dg) K;
 (b) int opApply(constancy K)(int delegate(constancy L)(ref L(T)) dg) K;

 Moreover, in case (a), opApply would accept for dg:
 - an int delegate(ref T) only if this is mutable
 - an int delegate(ref immutable(T)) only if this is immutable
 - an int delegate(ref const(T)), or a delegate that is itself
 constancy-templated, regardless of the constancy of this

 Perhaps the simplest example where meaning (b) is actually useful is

 inout(char)[] process(inout(char)[] delegate(inout(char)[]) dg,
 inout(char[]) text) {
 return text;
 }

 but still, somebody might want meaning (a). Anyway, under DMD 2.058
 (Win32) this gives

 inout_delegate.d(1): Error: inout must be all or none on top level for
 inout(char)[](inout(char)[] function(inout(char)[]) fn, inout(char[]) text)

 but why? At least it seems that DMD acknowledges the ambiguity, even if
 the error message doesn't make sense.


 The question really is: When inout is applied both to a parameter in a
 function's signature and to something in the signature of a
 function/delegate parameter therewithin, how should it be interpreted?
 The spec doesn't seem to address the issue at all. Indeed, we can ask
 two things:
 - what actually does the compiler make of it at the moment?
 - what would be the ideal way for it to work?

 Possibilities I can see:

 - always (a)
 - always (b)
 - (a) if at either level inout only occurs once, otherwise (b) (probably
 undesirable because of fragility)
 - just reject such signatures as ambiguous

 And moreover, should we support some syntax (similar to what I've used
 here or otherwise) to state explicitly whether we want to pass the
 constancy through to the delegate signature or not?

 Stewart.

I think some kind of disambiguation is essential. I see multiple solutions: 1. (b) would be the default. We could use the some parameter storage class to opt-in a): (a) int opApply( somequalifier int delegate(ref inout(T)) dg) inout; (b) int opApply(int delegate(ref inout(T)) dg) inout; This would work well. Maybe this is a little bit hackish, but I propose to use somequalifier:=inout a) int opApply(inout int delegate(ref inout(T)) dg) inout; b) int opApply(constancy K)(int delegate(constancy L)(ref L(T)) dg) K; Actually affecting the constancy of the delegate itself is not an useful operation anyway, but it could still be achieved: a) int opApply(inout inout(int delegate(ref inout(T)))) dg) inout; // every inout means the same thing b) int opApply(inout(int delegate(ref inout(T)))) dg) inout; // the inout on the delegate parameter is distinct from the other two 2. We introduce an infinite number of wildcard storage classes: inout, inout', inout'', inout''', inout'''', ... a) int opApply(int delegate(ref inout(T)) dg) inout; b) int opApply(int delegate(ref inout'(T)) dg) inout; This would have the additional boon that it would allow to type check a larger set of const-correct functions [1], like the following: void swapPairs(ref inout(int)[] x1, ref inout'(int)[] y1, ref inout(int)[] x2, ref inout'(int)[] y2){ swap(x1,x2); swap(y1,y2); } [1] furthermore, we'd finally get identifier' identifiers ;)
Feb 19 2012
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 19/02/2012 15:31, Timon Gehr wrote:
<snip excessive quote>
<snip>
 a) int opApply(inout int delegate(ref inout(T)) dg) inout;
 b) int opApply(constancy K)(int delegate(constancy L)(ref L(T)) dg) K;

???
 Actually affecting the constancy of the delegate itself is not an useful
operation anyway,
 but it could still be achieved:

 a) int opApply(inout inout(int delegate(ref inout(T)))) dg) inout; // every
inout means
 the same thing

So "inout inout" would be a special token sequence to link the outer and inner inouts together.
 b) int opApply(inout(int delegate(ref inout(T)))) dg) inout; // the inout on
the delegate
 parameter is distinct from the other two

I don't see what the inout applied to the type of dg is actually doing here....
 2.

 We introduce an infinite number of wildcard storage classes:
 inout, inout', inout'', inout''', inout'''', ...

 [1] furthermore, we'd finally get identifier' identifiers ;)

The lexer would need to be tweaked to support these. And it might get confusing to overload ' like this, given that some other keywords can be immediately followed by a character literal. Maybe inout$ with some kind of identifying token.... (a) int opApply(int delegate(ref inout$0(T)) dg) inout$0; (b) int opApply(int delegate(ref inout$1(T)) dg) inout$0; Hmm.... Stewart.
Feb 19 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02/19/2012 06:09 PM, Stewart Gordon wrote:
 On 19/02/2012 15:31, Timon Gehr wrote:
 <snip excessive quote>
 <snip>
 a) int opApply(inout int delegate(ref inout(T)) dg) inout;
 b) int opApply(constancy K)(int delegate(constancy L)(ref L(T)) dg) K;

???

oops... a) int opApply(inout int delegate(ref inout(T)) dg) inout; b) int opApply(int delegate(ref inout(T)) dg) inout;
 Actually affecting the constancy of the delegate itself is not an
 useful operation anyway,
 but it could still be achieved:

 a) int opApply(inout inout(int delegate(ref inout(T)))) dg) inout; //
 every inout means
 the same thing

So "inout inout" would be a special token sequence to link the outer and inner inouts together.

No, but probably it is more clear with the fixed b) case above. The inout parameter storage class would be used for linking.
 b) int opApply(inout(int delegate(ref inout(T)))) dg) inout; // the
 inout on the delegate
 parameter is distinct from the other two

I don't see what the inout applied to the type of dg is actually doing here....
 2.

 We introduce an infinite number of wildcard storage classes:
 inout, inout', inout'', inout''', inout'''', ...

 [1] furthermore, we'd finally get identifier' identifiers ;)

The lexer would need to be tweaked to support these. And it might get confusing to overload ' like this, given that some other keywords can be immediately followed by a character literal.

I am not aware of any except 'case'. But you are right. It would certainly confuse most existing D editors.
 Maybe inout$ with some kind of identifying token....

 (a) int opApply(int delegate(ref inout$0(T)) dg) inout$0;
 (b) int opApply(int delegate(ref inout$1(T)) dg) inout$0;

 Hmm....

 Stewart.

That would be a possibility.
Feb 19 2012
prev sibling next sibling parent reply kenji hara <k.hara.pg gmail.com> writes:
I think the 'scope' keyword may resolve issue.

Attributes of lazy parameter in template function belong to caller side.

int foo()(lazy int value)  safe pure /*nothrow*/ { return value(); }
void main() {
    int n =3D foo(10);
    // evaluating lazy parameter never break safety and purity of foo.
    // (violating nowthrow-ness might be a bug.)
    // because the violation belongs to caller side - it is main function.
}

Similarly, scope delegate body always in caller side, so calling it
means temporary exiting to caller side.

void foo(ref inout int x, scope void delegate(ref inout(int)) dg) {
    dg(x);
}
void main() {
    int a;
    foo(a, (ref int x){ x =3D 10; });  // don't break foo's inout-ness,
so should be allowed
    assert(a =3D=3D 10);
}

How about?

Kenji Hara

2012/2/19 Stewart Gordon <smjg_1998 yahoo.com>:
 At the moment, if a function has an inout parameter, it must have an inou=

 return type.

 But this prevents doing stuff like

 =A0 =A0void test(ref inout(int)[] x, inout(int)[] y) {
 =A0 =A0 =A0 =A0x =3D y;
 =A0 =A0}

 or passing the constancy through to a delegate instead of a return value.

 A typical use case of the latter is to define an opApply that works
 regardless of the constancy of this and allows the delegate to modify the
 iterated-through objects _if_ this is mutable.

 =A0 =A0int opApply(int delegate(ref inout(T)) dg) inout;

 But then I realised a potential ambiguity:
 (a) the constancy is passed through to the delegate
 (b) the delegate has an inout parameter in its own right

 If we go by interpretation (b), then each signature contains only one ino=

 so even if we relaxed the rules to allow this it would just be equivalent=

 =A0 =A0int opApply(int delegate(ref const(T)) dg) const;

 however, this won't always be true in the general case.

 The essence of functions with inout parameters is that they have a hidden
 constancy parameter. =A0This is essentially a template parameter, except =

 only one instance of the function is generated, rather like Java generics=

 =A0If we made this parameter explicit in the code, we could distinguish t=

 two meanings:

 (a) int opApply(constancy K)(int delegate(ref K(T)) dg) K;
 (b) int opApply(constancy K)(int delegate(constancy L)(ref L(T)) dg) K;

 Moreover, in case (a), opApply would accept for dg:
 - an int delegate(ref T) only if this is mutable
 - an int delegate(ref immutable(T)) only if this is immutable
 - an int delegate(ref const(T)), or a delegate that is itself
 constancy-templated, regardless of the constancy of this

 Perhaps the simplest example where meaning (b) is actually useful is

 =A0 =A0inout(char)[] process(inout(char)[] delegate(inout(char)[]) dg,
 inout(char[]) text) {
 =A0 =A0 =A0 =A0return text;
 =A0 =A0}

 but still, somebody might want meaning (a). =A0Anyway, under DMD 2.058 (W=

 this gives

 inout_delegate.d(1): Error: inout must be all or none on top level for
 inout(char)[](inout(char)[] function(inout(char)[]) fn, inout(char[]) tex=

 but why? =A0At least it seems that DMD acknowledges the ambiguity, even i=

 error message doesn't make sense.


 The question really is: When inout is applied both to a parameter in a
 function's signature and to something in the signature of a
 function/delegate parameter therewithin, how should it be interpreted? =

 spec doesn't seem to address the issue at all. =A0Indeed, we can ask two
 things:
 - what actually does the compiler make of it at the moment?
 - what would be the ideal way for it to work?

 Possibilities I can see:

 - always (a)
 - always (b)
 - (a) if at either level inout only occurs once, otherwise (b) (probably
 undesirable because of fragility)
 - just reject such signatures as ambiguous

 And moreover, should we support some syntax (similar to what I've used he=

 or otherwise) to state explicitly whether we want to pass the constancy
 through to the delegate signature or not?

 Stewart.

Feb 19 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02/19/2012 11:27 PM, kenji hara wrote:
 I think the 'scope' keyword may resolve issue.

 Attributes of lazy parameter in template function belong to caller side.

 int foo()(lazy int value)  safe pure /*nothrow*/ { return value(); }
 void main() {
      int n = foo(10);
      // evaluating lazy parameter never break safety and purity of foo.
      // (violating nowthrow-ness might be a bug.)
      // because the violation belongs to caller side - it is main function.
 }

 Similarly, scope delegate body always in caller side, so calling it
 means temporary exiting to caller side.

 void foo(ref inout int x, scope void delegate(ref inout(int)) dg) {
      dg(x);
 }
 void main() {
      int a;
      foo(a, (ref int x){ x = 10; });  // don't break foo's inout-ness,
 so should be allowed
      assert(a == 10);
 }

 How about?

 Kenji Hara

If I get you right, then this cannot work in general. immutable int y=0; void foo(ref inout int x, scope void delegate(ref inout(int)) dg) { dg(y); // boom }
Feb 19 2012
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 19 Feb 2012 09:27:42 -0500, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:

 At the moment, if a function has an inout parameter, it must have an  
 inout return type.

 But this prevents doing stuff like

      void test(ref inout(int)[] x, inout(int)[] y) {
          x = y;
      }

This is a legitimate request, I think there is an effort underway by myself Timon and Kenji to make this work. (well mostly Kenji and Timon)
 or passing the constancy through to a delegate instead of a return value.

 A typical use case of the latter is to define an opApply that works  
 regardless of the constancy of this and allows the delegate to modify  
 the iterated-through objects _if_ this is mutable.

      int opApply(int delegate(ref inout(T)) dg) inout;

What you ask isn't possible given the current design of inout. During inout function execution, inout is a special form of const, even if the object on which opApply is being called is mutable.
 But then I realised a potential ambiguity:
 (a) the constancy is passed through to the delegate
 (b) the delegate has an inout parameter in its own right

 If we go by interpretation (b), then each signature contains only one  
 inout, so even if we relaxed the rules to allow this it would just be  
 equivalent to

      int opApply(int delegate(ref const(T)) dg) const;

Yes, this is what I think it should be equivalent to. As I said, inside opApply, inout is like const, and is transitive. So you cannot "temporarily" make it mutable.
 however, this won't always be true in the general case.

 The essence of functions with inout parameters is that they have a  
 hidden constancy parameter.  This is essentially a template parameter,  
 except that only one instance of the function is generated, rather like  
 Java generics.  If we made this parameter explicit in the code, we could  
 distinguish the two meanings:

No the constancy is not a parameter (not even a hidden one). The magic of inout happens at the call, not inside the function. Inside, it's just another type of const. However, there is an entire part of delegates that is yet untapped -- implicit conversion of delegates. For example, int delegate(ref const int x) could be implicitly converted to int delegate(ref int x). Maybe there is something there that can be used to solve this problem. Maybe there is a rule we could apply when calling an inout-enabled function that allows implicit conversion of a delegate to an inout -flavored version of the delegate (as long as the normal delegate matches the decided inout constancy factor). But that violates transitivity. I'm not sure Walter would go for this. -Steve
Feb 24 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/24/2012 05:26 PM, Steven Schveighoffer wrote:
 On Sun, 19 Feb 2012 09:27:42 -0500, Stewart Gordon <smjg_1998 yahoo.com>
 wrote:

 At the moment, if a function has an inout parameter, it must have an
 inout return type.

 But this prevents doing stuff like

 void test(ref inout(int)[] x, inout(int)[] y) {
 x = y;
 }

This is a legitimate request, I think there is an effort underway by myself Timon and Kenji to make this work. (well mostly Kenji and Timon)
 or passing the constancy through to a delegate instead of a return value.

 A typical use case of the latter is to define an opApply that works
 regardless of the constancy of this and allows the delegate to modify
 the iterated-through objects _if_ this is mutable.

 int opApply(int delegate(ref inout(T)) dg) inout;

What you ask isn't possible given the current design of inout. During inout function execution, inout is a special form of const, even if the object on which opApply is being called is mutable.

The call site has enough information to type check the call given that there is some syntax to tie the two inout qualifiers together. inout shouldn't imply const, we already have const for that.
 But then I realised a potential ambiguity:
 (a) the constancy is passed through to the delegate
 (b) the delegate has an inout parameter in its own right

 If we go by interpretation (b), then each signature contains only one
 inout, so even if we relaxed the rules to allow this it would just be
 equivalent to

 int opApply(int delegate(ref const(T)) dg) const;

Yes, this is what I think it should be equivalent to. As I said, inside opApply, inout is like const, and is transitive. So you cannot "temporarily" make it mutable.

inout means 'some qualifier but we don't know which one'. The call site on the other hand knows which qualifier it is. If the call is made to work there is no 'making it mutable', because it is mutable all the time. The inout function cannot change the data because it cannot know what the constancy is.
 however, this won't always be true in the general case.

 The essence of functions with inout parameters is that they have a
 hidden constancy parameter. This is essentially a template parameter,
 except that only one instance of the function is generated, rather
 like Java generics. If we made this parameter explicit in the code, we
 could distinguish the two meanings:

No the constancy is not a parameter (not even a hidden one). The magic of inout happens at the call, not inside the function.

The same is (mostly) true for Java generics.
 Inside, it's just another type of const.

 However, there is an entire part of delegates that is yet untapped --
 implicit conversion of delegates. For example, int delegate(ref const
 int x) could be implicitly converted to int delegate(ref int x). Maybe
 there is something there that can be used to solve this problem. Maybe
 there is a rule we could apply when calling an inout-enabled function
 that allows implicit conversion of a delegate to an inout -flavored
 version of the delegate (as long as the normal delegate matches the
 decided inout constancy factor). But that violates transitivity. I'm not
 sure Walter would go for this.

Unfortunately this violates type safety. Also see: http://d.puremagic.com/issues/show_bug.cgi?id=7542
Feb 25 2012
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 05/03/2012 13:49, Steven Schveighoffer wrote:
 On Sat, 25 Feb 2012 09:02:47 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 inout means 'some qualifier but we don't know which one'. The call site on the
other
 hand knows which qualifier it is. If the call is made to work there is no
'making it
 mutable', because it is mutable all the time. The inout function cannot change
the data
 because it cannot know what the constancy is.

What you would propose is that a delegate which takes a const/mutable/immutable implicitly translates to one which takes an inout. I agree it can be made to work, but it does not fit into the current definition of inout.

I'm not sure if you understand correctly. Is isn't a matter of implicit conversion of delegates from one type to another, but a matter of (in this case) opApply accepting a delegate with a parameter whose constancy matches that of this. You can think of a function signature with an inout parameter as being really a union of three function signatures. inout(T)[] func(inout(T)[] param) is a union of T[] func(T[] param) immutable(T)[] func(immutable(T)[] param) const(T)[] func(const(T)[] param) In the same way, the essence of what Timon and I are proposing is that we'd be able to make opApply's signature a union of void opApply(int delegate(ref T) dg) void opApply(int delegate(ref immutable(T)) dg) immutable void opApply(int delegate(ref const(T)) dg) const or, to generalise a bit, declare a function whose signature is a union of void func(ref T param, int delegate(ref T) dg) void func(ref immutable(T) param, int delegate(ref immutable(T)) dg) void func(ref const(T) param, int delegate(ref const(T)) dg) or similarly with pointer or array parameters.
 It would be ideal for inout to solve this, but it's not chartered in such a
way to do so.

If I'm not mistaken, inout isn't chartered to do anything particular when it occurs in a function signature nested within another. The spec just leaves it ambiguous.
 It's currently transitive, and this would break transitivity. If we want to
look at
 fundamentally redefining inout so that it can break transitivity, then we can
look at
 that. But I don't think this is a simple "add-on" to the current functionality.

Can you give an example of how it breaks transitivity? Stewart.
Mar 05 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 03/05/2012 11:31 PM, Steven Schveighoffer wrote:
 On Mon, 05 Mar 2012 11:17:41 -0500, Stewart Gordon <smjg_1998 yahoo.com>
 wrote:

 On 05/03/2012 13:49, Steven Schveighoffer wrote:
 On Sat, 25 Feb 2012 09:02:47 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 inout means 'some qualifier but we don't know which one'. The call
 site on the other
 hand knows which qualifier it is. If the call is made to work there
 is no 'making it
 mutable', because it is mutable all the time. The inout function
 cannot change the data
 because it cannot know what the constancy is.

What you would propose is that a delegate which takes a const/mutable/immutable implicitly translates to one which takes an inout. I agree it can be made to work, but it does not fit into the current definition of inout.

I'm not sure if you understand correctly. Is isn't a matter of implicit conversion of delegates from one type to another, but a matter of (in this case) opApply accepting a delegate with a parameter whose constancy matches that of this.

I understand the problem and the proposed solution quite well.
 It would be ideal for inout to solve this, but it's not chartered in
 such a way to do so.

If I'm not mistaken, inout isn't chartered to do anything particular when it occurs in a function signature nested within another. The spec just leaves it ambiguous.

No, inout is not modifiable during the function execution. Given the transitivity of const and immutable, inout itself must also be transitive. Think about it this way: inout(int)* foo(inout(int)* x) { // begin here, at this point x is not modifiable // *x = 5; // not allowed! return x; // end here, at this point inout reverts back to it's original constancy (including return value). }

I don't think this is the best way to think about inout. Try to think about it as if the constancy was determined, but unknown. It does not actually change, the function just does not know what it is because it has to work in a polymorphic way.
 The original definition I used for inout was "scoped const", meaning
 it's const within the scope and reverts back after the scope. I did not
 plan for having the constancy "temporarily" revert back to the original
 constancy. What you wish for is this to take a delegate which matches
 the constancy of the x parameter:

 inout(int)* foo(inout(int) *x, void delegate(inout(int)* p) dg)
 {
 dg(x); // valid, since the type matches
 return x; // oops, now x could have changed! Even through it's an inout
 reference!
 }

Well, that can happen even for const references. What is your point?
 It's currently transitive, and this would break transitivity. If we
 want to look at
 fundamentally redefining inout so that it can break transitivity,
 then we can look at
 that. But I don't think this is a simple "add-on" to the current
 functionality.

Can you give an example of how it breaks transitivity?

Using the above foo function: void main() { void bar(int *x) {*x = 5;} int i = 2; foo(&i, &bar); // this sets i to 5 via the inout reference we pass into it. } In other words, inout is transitively not constant during the function call.

Because it has been matched as *mutable*. There is no issue.
 I'm not saying we cannot bend the rules to allow for this, because
 clearly it doesn't violate the "true" constancy type of the data passed
 in, but I don't think it's a straightforward change conceptually. I'm
 not a compiler writer, so I don't know how this works in their minds,
 I'd like to have their input.

Implementation of what you propose should be quite simple. The issue is with the design, i.e. allowing both tying the inout in the delegate parameter signature to the inout in the enclosing signature and having an independent inout delegate parameter needs workable syntax.
Mar 05 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 03/06/2012 12:27 AM, Steven Schveighoffer wrote:
...
 There are two parts to inout, one is that it can be one function called
 3 different ways, the other is that you know it's constant during
 function execution. Some people like that second part, even if it
 doesn't fully guarantee everything. I.e. there's a reason people use
 const in C++ besides it being trendy.

By passing a delegate that changes an inout-matched argument it is made explicit that inout data may change. Technically, none of the existing guarantees are lost, we just add some expressiveness to the type system.
 I'm not saying we cannot bend the rules to allow for this, because
 clearly it doesn't violate the "true" constancy type of the data passed
 in, but I don't think it's a straightforward change conceptually. I'm
 not a compiler writer, so I don't know how this works in their minds,
 I'd like to have their input.

Implementation of what you propose should be quite simple. The issue is with the design, i.e. allowing both tying the inout in the delegate parameter signature to the inout in the enclosing signature and having an independent inout delegate parameter needs workable syntax.

I think it can be done: 1. determine inout match based on existing rules, excluding delegate parameters. 2. change delegate parameter inouts to matched value. 3. Use implicit delegate conversion (using contravariance) to allow passing delegates of proper type. For example: void foo(inout(int)* x, void delegate(inout(int)* y) dg); void main() { int mx; immutable int ix; const int cx; void bar1(int *mp) {} void bar2(immutable(int) *ip) {} void bar3(const(int) *cp) {} void bar4(inout(int) *iop) {} // inout matched as mutable due to mx, signature becomes void foo(int *x, void delegate(int *y) dg); foo(&mx, &bar1); // fine, direct match of both parameters foo(&mx, &bar2); // not fine, immutable delegate does not implicitly convert to mutable foo(&mx, &bar3); // fine, const delegate can implicitly convert to mutable foo(&mx, &bar4); // fine, inout delegate can implicitly convert to mutable // signature becomes void foo(immutable(int) *x, void delegate(immutable(int) *y) dg); foo(&ix, &bar1); // error foo(&ix, &bar2); // ok foo(&ix, &bar3); // fine, const delegate can implicitly convert to immutable foo(&ix, &bar4); // fine, inout delegate can implicitly convert to immutable // signature becomes void foo(const(int) *x, void delegate(const(int) *y) dg); foo(&cx, &bar1); // error foo(&cx, &bar2); // error foo(&cx, &bar3); // ok foo(&cx, &bar4); // ok // etc... }

I understand, but how would you support this use case?: inout(int)[] foo(inout(int)[] delegate(inout(int)[] dg), inout(int)[] arr){ int[] x = dg(new int[16]); immutable(int)[] y = dg(new immutable(int)[16]); // ... inout(int)[] z = dg(arr); return foo(z,y,z); } In essence, it should be possible to pass an inout delegate to an inout function, such that the function can use the delegate as an inout delegate.
 Note that Walter has explicitly rejected contravariance conversion for
 delegates.

That is unfortunate.
 You have good persuasive skills, maybe you can help :) See
 bug http://d.puremagic.com/issues/show_bug.cgi?id=3180 and
 http://d.puremagic.com/issues/show_bug.cgi?id=3075

IIRC Kenji Hara has already started attempts to loosen the restrictions regarding delegate implicit conversions. Hopefully Walter will reconsider.
Mar 06 2012
next sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 07/03/2012 15:41, Steven Schveighoffer wrote:
<snip>
 In fact, I think this is valid D code:

 int i;
 const int *pi = &i;
 int *p = cast()pi;
 *p = 5; // legal because I know this points at i, and i is really mutable

cast() is an abomination. I'm not sure OTTOMH whether it's a bug that it works.
 But from an API point of view, I look at at inout as guaranteeing anything the
parameter
 points at won't change while inside the function *using that parameter*. Even
though it's
 legal, it's disingenuous (at least as long as we define inout that way).

That's what const is for. The whole point of inout is that the caller has the choice of whether to pass in and get out again mutable, const or immutable data. Allowing the caller to pass in a delegate with a mutable, const or immutable parameter would fit right in with this. <snip>
 As usual, you find and poke holes in all my arguments :) I'm thinking that in
order to
 make this work we need a way to specify an inout modifier really represents
the matched
 type rather than a full-fledged inout parameter.

Yes, and ways of specifying this have been suggested early in this thread. Stewart.
Mar 07 2012
next sibling parent Stewart Gordon <smjg_1998 yahoo.com> writes:
On 07/03/2012 22:48, Steven Schveighoffer wrote:
 On Wed, 07 Mar 2012 17:37:53 -0500, Stewart Gordon <smjg_1998 yahoo.com> wrote:

 cast() is an abomination. I'm not sure OTTOMH whether it's a bug that it works.

Sorry, it's just easier than typing cast(int*).

Which is another abomination. The means of casting away constancy should be explicit. <snip>
 But from an API point of view, I look at at inout as guaranteeing anything the
parameter
 points at won't change while inside the function *using that parameter*. Even
though it's
 legal, it's disingenuous (at least as long as we define inout that way).

That's what const is for.

And inout. Sorry, it was meant that way, even if you don't agree.

Maybe _you_ meant it that way, but did anyone else? Stewart.
Mar 07 2012
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 03/07/2012 11:37 PM, Stewart Gordon wrote:
 On 07/03/2012 15:41, Steven Schveighoffer wrote:
 <snip>
 In fact, I think this is valid D code:

 int i;
 const int *pi = &i;
 int *p = cast()pi;
 *p = 5; // legal because I know this points at i, and i is really mutable

cast() is an abomination. I'm not sure OTTOMH whether it's a bug that it works.

It is not legal code. I did not point it out because it was clear what was meant. cast() only casts away the top level of modifiers. It is perfectly safe except for class objects.
Mar 07 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 03/08/2012 12:37 PM, Steven Schveighoffer wrote:
 On Wed, 07 Mar 2012 19:06:14 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 03/07/2012 11:37 PM, Stewart Gordon wrote:
 On 07/03/2012 15:41, Steven Schveighoffer wrote:
 <snip>
 In fact, I think this is valid D code:

 int i;
 const int *pi = &i;
 int *p = cast()pi;
 *p = 5; // legal because I know this points at i, and i is really
 mutable

cast() is an abomination. I'm not sure OTTOMH whether it's a bug that it works.

It is not legal code. I did not point it out because it was clear what was meant. cast() only casts away the top level of modifiers. It is perfectly safe except for class objects.

If it's not legal code, then how is implicitly casting away inout during function execution legal code? Isn't this the same thing? -Steve

It is not legal code because the assignment const(int)* to int* does not succeed. The other part is up to debate. The specification does not define the semantics of casting away const and changing the data. It is also not the same as with the proposed change to inout. inout would not be 'removed' in the function body, it would be 'removed' upon inout-matching the parameters. Inout should be able to replace overloads on const, therefore I think that is the way it should work on the conceptual level.
Mar 08 2012
parent Stewart Gordon <smjg_1998 yahoo.com> writes:
On 08/03/2012 19:38, Steven Schveighoffer wrote:
<snip>
 Yes, I couldn't really find that. It does specifically say casting away const
and then
 modifying is invalid, but it does not say anything about "if you know the
underlying data
 is mutable". But again, this is the point I was trying to make, we are casting
away a
 const-like attribute and modifying the data.

No more than we are already doing with the inout return. Stewart.
Mar 08 2012
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 03/07/2012 04:41 PM, Steven Schveighoffer wrote:
 On Tue, 06 Mar 2012 05:11:34 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 03/06/2012 12:27 AM, Steven Schveighoffer wrote:
 ...
 There are two parts to inout, one is that it can be one function called
 3 different ways, the other is that you know it's constant during
 function execution. Some people like that second part, even if it
 doesn't fully guarantee everything. I.e. there's a reason people use
 const in C++ besides it being trendy.

By passing a delegate that changes an inout-matched argument it is made explicit that inout data may change. Technically, none of the existing guarantees are lost, we just add some expressiveness to the type system.

Yes, I understand that it works. I know that it doesn't violate technically any const guarantees.

Yes, that is not what I was after: void foo(inout(int)*x, void delegate(inout(int)*) dg) // both inout's denote the same wildcard { dg(x); } void main(){ int x; foo(&x, (p){*p=2;}); // this currently cannot be valid code } The function call is explicit about that may change the inout parameter. There are no guarantees lost. It is the call site who changes the parameter. Code that currently has the guarantee that the parameter won't change will keep it, because there is no valid code that could silently change semantics under the change and does not use some fancy introspection.
Mar 07 2012
prev sibling parent Stewart Gordon <smjg_1998 yahoo.com> writes:
On 05/03/2012 22:31, Steven Schveighoffer wrote:
 On Mon, 05 Mar 2012 11:17:41 -0500, Stewart Gordon <smjg_1998 yahoo.com> wrote:

 On 05/03/2012 13:49, Steven Schveighoffer wrote:


 It's currently transitive, and this would break transitivity. If we want to
look at
 fundamentally redefining inout so that it can break transitivity, then we can
look at
 that. But I don't think this is a simple "add-on" to the current functionality.

Can you give an example of how it breaks transitivity?

Using the above foo function:

Which has only one level of indirection. So I can't see how transitivity comes into play.
 void main()
 {
 void bar(int *x) {*x = 5;}
 int i = 2;
 foo(&i, &bar); // this sets i to 5 via the inout reference we pass into it.
 }

The foo function doesn't set i. The bar function does. The design of inout is that the function with inout parameters doesn't know or care about the constancy of its arguments, _therefore_ cannot modify the data. The function that is passed in as a delegate, OTOH, _does_ know the constancy of its argument, and so _can_ modify it if the constancy permits. This is necessary to support foreach in the way we're talking about. Stewart.
Mar 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 25 Feb 2012 09:02:47 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 02/24/2012 05:26 PM, Steven Schveighoffer wrote:
 On Sun, 19 Feb 2012 09:27:42 -0500, Stewart Gordon <smjg_1998 yahoo.com>
 wrote:

 At the moment, if a function has an inout parameter, it must have an
 inout return type.

 But this prevents doing stuff like

 void test(ref inout(int)[] x, inout(int)[] y) {
 x = y;
 }

This is a legitimate request, I think there is an effort underway by myself Timon and Kenji to make this work. (well mostly Kenji and Timon)
 or passing the constancy through to a delegate instead of a return  
 value.

 A typical use case of the latter is to define an opApply that works
 regardless of the constancy of this and allows the delegate to modify
 the iterated-through objects _if_ this is mutable.

 int opApply(int delegate(ref inout(T)) dg) inout;

What you ask isn't possible given the current design of inout. During inout function execution, inout is a special form of const, even if the object on which opApply is being called is mutable.

The call site has enough information to type check the call given that there is some syntax to tie the two inout qualifiers together. inout shouldn't imply const, we already have const for that.

inout is defined as "during this function execution the given parameter will be treated as const". This is necessary to have a single implementation for all flavors of const. Remember, inside the function the compiler has no idea that the actual data is mutable, const or immutable. One of the cornerstones of inout is that it must generate one function.
 But then I realised a potential ambiguity:
 (a) the constancy is passed through to the delegate
 (b) the delegate has an inout parameter in its own right

 If we go by interpretation (b), then each signature contains only one
 inout, so even if we relaxed the rules to allow this it would just be
 equivalent to

 int opApply(int delegate(ref const(T)) dg) const;

Yes, this is what I think it should be equivalent to. As I said, inside opApply, inout is like const, and is transitive. So you cannot "temporarily" make it mutable.

inout means 'some qualifier but we don't know which one'. The call site on the other hand knows which qualifier it is. If the call is made to work there is no 'making it mutable', because it is mutable all the time. The inout function cannot change the data because it cannot know what the constancy is.

What you would propose is that a delegate which takes a const/mutable/immutable implicitly translates to one which takes an inout. I agree it can be made to work, but it does not fit into the current definition of inout. It would be ideal for inout to solve this, but it's not chartered in such a way to do so. It's currently transitive, and this would break transitivity. If we want to look at fundamentally redefining inout so that it can break transitivity, then we can look at that. But I don't think this is a simple "add-on" to the current functionality.
 however, this won't always be true in the general case.

 The essence of functions with inout parameters is that they have a
 hidden constancy parameter. This is essentially a template parameter,
 except that only one instance of the function is generated, rather
 like Java generics. If we made this parameter explicit in the code, we
 could distinguish the two meanings:

No the constancy is not a parameter (not even a hidden one). The magic of inout happens at the call, not inside the function.

The same is (mostly) true for Java generics.
 Inside, it's just another type of const.

 However, there is an entire part of delegates that is yet untapped --
 implicit conversion of delegates. For example, int delegate(ref const
 int x) could be implicitly converted to int delegate(ref int x). Maybe
 there is something there that can be used to solve this problem. Maybe
 there is a rule we could apply when calling an inout-enabled function
 that allows implicit conversion of a delegate to an inout -flavored
 version of the delegate (as long as the normal delegate matches the
 decided inout constancy factor). But that violates transitivity. I'm not
 sure Walter would go for this.

Unfortunately this violates type safety. Also see: http://d.puremagic.com/issues/show_bug.cgi?id=7542

That bug does not exactly capture what I said. The inout translation *must* happen at the same point where inout is resolved at the call site. For example, you can't do this: int delegate(inout(int)*) iod; int delegate(int *) md; iod = md; // error, we don't know what inout will resolve to. But this could be made to work: void foo(inout(int)* x, int delegate(inout(int)*) d) { d(x);} int x; foo(&x, md); // translation is valid, because we know what inout resolves to. foo(&x, iod); // also valid, because iod must have been defined to actually take an inout parameter. Again, if you look at inout as a form of const, this breaks transitivity, it's kind of a cousin of logical const. Walter needs to be involved to have his say. I think it could be made to work. The more I think about it, the more I think this is worth changing. inout provides a unique way to make something that is normally a template that generates multiple copies of the same code into one that generates one copy. The large benefit of it is you can write one function instead of 3, and still have it not be a template. -Steve
Mar 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 05 Mar 2012 11:17:41 -0500, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:

 On 05/03/2012 13:49, Steven Schveighoffer wrote:
 On Sat, 25 Feb 2012 09:02:47 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 inout means 'some qualifier but we don't know which one'. The call  
 site on the other
 hand knows which qualifier it is. If the call is made to work there is  
 no 'making it
 mutable', because it is mutable all the time. The inout function  
 cannot change the data
 because it cannot know what the constancy is.

What you would propose is that a delegate which takes a const/mutable/immutable implicitly translates to one which takes an inout. I agree it can be made to work, but it does not fit into the current definition of inout.

I'm not sure if you understand correctly. Is isn't a matter of implicit conversion of delegates from one type to another, but a matter of (in this case) opApply accepting a delegate with a parameter whose constancy matches that of this.

I understand the problem and the proposed solution quite well.
 It would be ideal for inout to solve this, but it's not chartered in  
 such a way to do so.

If I'm not mistaken, inout isn't chartered to do anything particular when it occurs in a function signature nested within another. The spec just leaves it ambiguous.

No, inout is not modifiable during the function execution. Given the transitivity of const and immutable, inout itself must also be transitive. Think about it this way: inout(int)* foo(inout(int)* x) { // begin here, at this point x is not modifiable // *x = 5; // not allowed! return x; // end here, at this point inout reverts back to it's original constancy (including return value). } The original definition I used for inout was "scoped const", meaning it's const within the scope and reverts back after the scope. I did not plan for having the constancy "temporarily" revert back to the original constancy. What you wish for is this to take a delegate which matches the constancy of the x parameter: inout(int)* foo(inout(int) *x, void delegate(inout(int)* p) dg) { dg(x); // valid, since the type matches return x; // oops, now x could have changed! Even through it's an inout reference! }
 It's currently transitive, and this would break transitivity. If we  
 want to look at
 fundamentally redefining inout so that it can break transitivity, then  
 we can look at
 that. But I don't think this is a simple "add-on" to the current  
 functionality.

Can you give an example of how it breaks transitivity?

Using the above foo function: void main() { void bar(int *x) {*x = 5;} int i = 2; foo(&i, &bar); // this sets i to 5 via the inout reference we pass into it. } In other words, inout is transitively not constant during the function call. I'm not saying we cannot bend the rules to allow for this, because clearly it doesn't violate the "true" constancy type of the data passed in, but I don't think it's a straightforward change conceptually. I'm not a compiler writer, so I don't know how this works in their minds, I'd like to have their input. -Steve
Mar 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 05 Mar 2012 18:01:34 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 03/05/2012 11:31 PM, Steven Schveighoffer wrote:
 On Mon, 05 Mar 2012 11:17:41 -0500, Stewart Gordon <smjg_1998 yahoo.com>
 wrote:

 On 05/03/2012 13:49, Steven Schveighoffer wrote:
 On Sat, 25 Feb 2012 09:02:47 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 inout means 'some qualifier but we don't know which one'. The call
 site on the other
 hand knows which qualifier it is. If the call is made to work there
 is no 'making it
 mutable', because it is mutable all the time. The inout function
 cannot change the data
 because it cannot know what the constancy is.

What you would propose is that a delegate which takes a const/mutable/immutable implicitly translates to one which takes an inout. I agree it can be made to work, but it does not fit into the current definition of inout.

I'm not sure if you understand correctly. Is isn't a matter of implicit conversion of delegates from one type to another, but a matter of (in this case) opApply accepting a delegate with a parameter whose constancy matches that of this.

I understand the problem and the proposed solution quite well.
 It would be ideal for inout to solve this, but it's not chartered in
 such a way to do so.

If I'm not mistaken, inout isn't chartered to do anything particular when it occurs in a function signature nested within another. The spec just leaves it ambiguous.

No, inout is not modifiable during the function execution. Given the transitivity of const and immutable, inout itself must also be transitive. Think about it this way: inout(int)* foo(inout(int)* x) { // begin here, at this point x is not modifiable // *x = 5; // not allowed! return x; // end here, at this point inout reverts back to it's original constancy (including return value). }

I don't think this is the best way to think about inout. Try to think about it as if the constancy was determined, but unknown. It does not actually change, the function just does not know what it is because it has to work in a polymorphic way.

Best way or not, it was designed and accepted using these assumptions. Again, I am not against the idea, I just want to hear what Walter and company think about it.
 The original definition I used for inout was "scoped const", meaning
 it's const within the scope and reverts back after the scope. I did not
 plan for having the constancy "temporarily" revert back to the original
 constancy. What you wish for is this to take a delegate which matches
 the constancy of the x parameter:

 inout(int)* foo(inout(int) *x, void delegate(inout(int)* p) dg)
 {
 dg(x); // valid, since the type matches
 return x; // oops, now x could have changed! Even through it's an inout
 reference!
 }

Well, that can happen even for const references. What is your point?

Not via the const parameter. For example, mark the delegate and the function as pure, it cannot happen with const.
 It's currently transitive, and this would break transitivity. If we
 want to look at
 fundamentally redefining inout so that it can break transitivity,
 then we can look at
 that. But I don't think this is a simple "add-on" to the current
 functionality.

Can you give an example of how it breaks transitivity?

Using the above foo function: void main() { void bar(int *x) {*x = 5;} int i = 2; foo(&i, &bar); // this sets i to 5 via the inout reference we pass into it. } In other words, inout is transitively not constant during the function call.

Because it has been matched as *mutable*. There is no issue.

There are two parts to inout, one is that it can be one function called 3 different ways, the other is that you know it's constant during function execution. Some people like that second part, even if it doesn't fully guarantee everything. I.e. there's a reason people use const in C++ besides it being trendy.
 I'm not saying we cannot bend the rules to allow for this, because
 clearly it doesn't violate the "true" constancy type of the data passed
 in, but I don't think it's a straightforward change conceptually. I'm
 not a compiler writer, so I don't know how this works in their minds,
 I'd like to have their input.

Implementation of what you propose should be quite simple. The issue is with the design, i.e. allowing both tying the inout in the delegate parameter signature to the inout in the enclosing signature and having an independent inout delegate parameter needs workable syntax.

I think it can be done: 1. determine inout match based on existing rules, excluding delegate parameters. 2. change delegate parameter inouts to matched value. 3. Use implicit delegate conversion (using contravariance) to allow passing delegates of proper type. For example: void foo(inout(int)* x, void delegate(inout(int)* y) dg); void main() { int mx; immutable int ix; const int cx; void bar1(int *mp) {} void bar2(immutable(int) *ip) {} void bar3(const(int) *cp) {} void bar4(inout(int) *iop) {} // inout matched as mutable due to mx, signature becomes void foo(int *x, void delegate(int *y) dg); foo(&mx, &bar1); // fine, direct match of both parameters foo(&mx, &bar2); // not fine, immutable delegate does not implicitly convert to mutable foo(&mx, &bar3); // fine, const delegate can implicitly convert to mutable foo(&mx, &bar4); // fine, inout delegate can implicitly convert to mutable // signature becomes void foo(immutable(int) *x, void delegate(immutable(int) *y) dg); foo(&ix, &bar1); // error foo(&ix, &bar2); // ok foo(&ix, &bar3); // fine, const delegate can implicitly convert to immutable foo(&ix, &bar4); // fine, inout delegate can implicitly convert to immutable // signature becomes void foo(const(int) *x, void delegate(const(int) *y) dg); foo(&cx, &bar1); // error foo(&cx, &bar2); // error foo(&cx, &bar3); // ok foo(&cx, &bar4); // ok // etc... } Note that Walter has explicitly rejected contravariance conversion for delegates. You have good persuasive skills, maybe you can help :) See bug http://d.puremagic.com/issues/show_bug.cgi?id=3180 and http://d.puremagic.com/issues/show_bug.cgi?id=3075 -Steve
Mar 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 05 Mar 2012 20:34:09 -0500, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:

 On 05/03/2012 22:31, Steven Schveighoffer wrote:
 On Mon, 05 Mar 2012 11:17:41 -0500, Stewart Gordon  
 <smjg_1998 yahoo.com> wrote:

 On 05/03/2012 13:49, Steven Schveighoffer wrote:


 It's currently transitive, and this would break transitivity. If we  
 want to look at
 fundamentally redefining inout so that it can break transitivity,  
 then we can look at
 that. But I don't think this is a simple "add-on" to the current  
 functionality.

Can you give an example of how it breaks transitivity?

Using the above foo function:

Which has only one level of indirection. So I can't see how transitivity comes into play.
 void main()
 {
 void bar(int *x) {*x = 5;}
 int i = 2;
 foo(&i, &bar); // this sets i to 5 via the inout reference we pass into  
 it.
 }

The foo function doesn't set i. The bar function does.

The issue is that the bar function modifies i via the inout parameter to foo. That is, bar does not modify i directly, but modifies foo's parameter.
 The design of inout is that the function with inout parameters doesn't  
 know or care about the constancy of its arguments, _therefore_ cannot  
 modify the data.  The function that is passed in as a delegate, OTOH,  
 _does_ know the constancy of its argument, and so _can_ modify it if the  
 constancy permits.  This is necessary to support foreach in the way  
 we're talking about.

I think maybe transitivity is not the right term. It's just a straight const violation. The thing I'm getting at is, there are two pieces to inout -- one is, it passes the constancy of the parameters back through the return value. The second is it acts as a surrogate for const in terms of API guarantees. That means, if I pass in a parameter as an inout parameter, it will not be modified during the function execution *through that parameter*. Again, I want to stress that I think the idea is sound and could be implemented. But I think it fundamentally changes the way inout is conceptually designed. -Steve
Mar 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 06 Mar 2012 05:11:34 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 03/06/2012 12:27 AM, Steven Schveighoffer wrote:
 ...
 There are two parts to inout, one is that it can be one function called
 3 different ways, the other is that you know it's constant during
 function execution. Some people like that second part, even if it
 doesn't fully guarantee everything. I.e. there's a reason people use
 const in C++ besides it being trendy.

By passing a delegate that changes an inout-matched argument it is made explicit that inout data may change. Technically, none of the existing guarantees are lost, we just add some expressiveness to the type system.

Yes, I understand that it works. I know that it doesn't violate technically any const guarantees. In fact, I think this is valid D code: int i; const int *pi = &i; int *p = cast()pi; *p = 5; // legal because I know this points at i, and i is really mutable But from an API point of view, I look at at inout as guaranteeing anything the parameter points at won't change while inside the function *using that parameter*. Even though it's legal, it's disingenuous (at least as long as we define inout that way). An interesting side note is that inout (if designed in this way) can allow the above code snippit to be modularized.
 I'm not saying we cannot bend the rules to allow for this, because
 clearly it doesn't violate the "true" constancy type of the data  
 passed
 in, but I don't think it's a straightforward change conceptually. I'm
 not a compiler writer, so I don't know how this works in their minds,
 I'd like to have their input.

Implementation of what you propose should be quite simple. The issue is with the design, i.e. allowing both tying the inout in the delegate parameter signature to the inout in the enclosing signature and having an independent inout delegate parameter needs workable syntax.

I think it can be done: 1. determine inout match based on existing rules, excluding delegate parameters. 2. change delegate parameter inouts to matched value. 3. Use implicit delegate conversion (using contravariance) to allow passing delegates of proper type. For example: void foo(inout(int)* x, void delegate(inout(int)* y) dg); void main() { int mx; immutable int ix; const int cx; void bar1(int *mp) {} void bar2(immutable(int) *ip) {} void bar3(const(int) *cp) {} void bar4(inout(int) *iop) {} // inout matched as mutable due to mx, signature becomes void foo(int *x, void delegate(int *y) dg); foo(&mx, &bar1); // fine, direct match of both parameters foo(&mx, &bar2); // not fine, immutable delegate does not implicitly convert to mutable foo(&mx, &bar3); // fine, const delegate can implicitly convert to mutable foo(&mx, &bar4); // fine, inout delegate can implicitly convert to mutable // signature becomes void foo(immutable(int) *x, void delegate(immutable(int) *y) dg); foo(&ix, &bar1); // error foo(&ix, &bar2); // ok foo(&ix, &bar3); // fine, const delegate can implicitly convert to immutable foo(&ix, &bar4); // fine, inout delegate can implicitly convert to immutable // signature becomes void foo(const(int) *x, void delegate(const(int) *y) dg); foo(&cx, &bar1); // error foo(&cx, &bar2); // error foo(&cx, &bar3); // ok foo(&cx, &bar4); // ok // etc... }

I understand, but how would you support this use case?: inout(int)[] foo(inout(int)[] delegate(inout(int)[] dg), inout(int)[] arr){ int[] x = dg(new int[16]); immutable(int)[] y = dg(new immutable(int)[16]); // ... inout(int)[] z = dg(arr); return foo(z,y,z); }

As usual, you find and poke holes in all my arguments :) I'm thinking that in order to make this work we need a way to specify an inout modifier really represents the matched type rather than a full-fledged inout parameter. This is not a good thing -- inout is already pretty weird and hard to understand. Need to do some more thinking, maybe there's an easy way.
 Note that Walter has explicitly rejected contravariance conversion for
 delegates.

That is unfortunate.
 You have good persuasive skills, maybe you can help :) See
 bug http://d.puremagic.com/issues/show_bug.cgi?id=3180 and
 http://d.puremagic.com/issues/show_bug.cgi?id=3075

IIRC Kenji Hara has already started attempts to loosen the restrictions regarding delegate implicit conversions. Hopefully Walter will reconsider.

I hope so. This is a vastly awesome untapped low-hanging fruit. -Steve
Mar 07 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 07 Mar 2012 17:37:53 -0500, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:

 On 07/03/2012 15:41, Steven Schveighoffer wrote:
 <snip>
 In fact, I think this is valid D code:

 int i;
 const int *pi = &i;
 int *p = cast()pi;
 *p = 5; // legal because I know this points at i, and i is really  
 mutable

cast() is an abomination. I'm not sure OTTOMH whether it's a bug that it works.

Sorry, it's just easier than typing cast(int*). And I believe it's legal, as long as you *know* that the underlying data is mutable. The above code snippit is legal, because it is shown as long as the first two lines are included that the actual data is mutable.
 But from an API point of view, I look at at inout as guaranteeing  
 anything the parameter
 points at won't change while inside the function *using that  
 parameter*. Even though it's
 legal, it's disingenuous (at least as long as we define inout that way).

That's what const is for.

And inout. Sorry, it was meant that way, even if you don't agree. -Steve
Mar 07 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 07 Mar 2012 18:01:10 -0500, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:

 On 07/03/2012 22:48, Steven Schveighoffer wrote:
 On Wed, 07 Mar 2012 17:37:53 -0500, Stewart Gordon  
 <smjg_1998 yahoo.com> wrote:

 cast() is an abomination. I'm not sure OTTOMH whether it's a bug that  
 it works.

Sorry, it's just easier than typing cast(int*).

Which is another abomination. The means of casting away constancy should be explicit.

I agree, but it doesn't make it illegal. It was just a means to show what I meant.
 <snip>
 But from an API point of view, I look at at inout as guaranteeing  
 anything the parameter
 points at won't change while inside the function *using that  
 parameter*. Even though it's
 legal, it's disingenuous (at least as long as we define inout that  
 way).

That's what const is for.

And inout. Sorry, it was meant that way, even if you don't agree.

Maybe _you_ meant it that way, but did anyone else?

I actually designed it... http://d.puremagic.com/issues/show_bug.cgi?id=1961 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP2 -Steve
Mar 07 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 07 Mar 2012 19:06:14 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 03/07/2012 11:37 PM, Stewart Gordon wrote:
 On 07/03/2012 15:41, Steven Schveighoffer wrote:
 <snip>
 In fact, I think this is valid D code:

 int i;
 const int *pi = &i;
 int *p = cast()pi;
 *p = 5; // legal because I know this points at i, and i is really  
 mutable

cast() is an abomination. I'm not sure OTTOMH whether it's a bug that it works.

It is not legal code. I did not point it out because it was clear what was meant. cast() only casts away the top level of modifiers. It is perfectly safe except for class objects.

If it's not legal code, then how is implicitly casting away inout during function execution legal code? Isn't this the same thing? -Steve
Mar 08 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 08 Mar 2012 13:17:15 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 03/08/2012 12:37 PM, Steven Schveighoffer wrote:
 On Wed, 07 Mar 2012 19:06:14 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 On 03/07/2012 11:37 PM, Stewart Gordon wrote:
 On 07/03/2012 15:41, Steven Schveighoffer wrote:
 <snip>
 In fact, I think this is valid D code:

 int i;
 const int *pi = &i;
 int *p = cast()pi;
 *p = 5; // legal because I know this points at i, and i is really
 mutable

cast() is an abomination. I'm not sure OTTOMH whether it's a bug that it works.

It is not legal code. I did not point it out because it was clear what was meant. cast() only casts away the top level of modifiers. It is perfectly safe except for class objects.

If it's not legal code, then how is implicitly casting away inout during function execution legal code? Isn't this the same thing? -Steve

It is not legal code because the assignment const(int)* to int* does not succeed.

Oh right, I forgot that casting using cast() just goes to the tail-const version. grr... We really need const_cast...
 The other part is up to debate. The specification does not define the  
 semantics of casting away const and changing the data.

Yes, I couldn't really find that. It does specifically say casting away const and then modifying is invalid, but it does not say anything about "if you know the underlying data is mutable". But again, this is the point I was trying to make, we are casting away a const-like attribute and modifying the data.
 It is also not the same as with the proposed change to inout. inout  
 would not be 'removed' in the function body, it would be 'removed' upon  
 inout-matching the parameters. Inout should be able to replace overloads  
 on const, therefore I think that is the way it should work on the  
 conceptual level.

It is essentially the same as this: void bar(const(int) * i, void delegate(const(int)* i) dg) {dg(i);} void main() { void foo(int *i) {*i = 5;} bar(&i, cast(delegate(const(int)*)) &foo); } Which I don't know if it's valid. Given that compiler enforcement of inout being *sure* that the data is actually mutable, it's much safer than what I wrote above, but it's certainly no different. It's just compiler-checked vs. manually checked. -Steve
Mar 08 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 08 Mar 2012 17:43:34 -0500, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:

 On 08/03/2012 19:38, Steven Schveighoffer wrote:
 <snip>
 Yes, I couldn't really find that. It does specifically say casting away  
 const and then
 modifying is invalid, but it does not say anything about "if you know  
 the underlying data
 is mutable". But again, this is the point I was trying to make, we are  
 casting away a
 const-like attribute and modifying the data.

No more than we are already doing with the inout return.

Except that technique has been approved by Walter and included in the language. -Steve
Mar 09 2012
prev sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 19/02/2012 14:27, Stewart Gordon wrote:
<snip>
 int opApply(int delegate(ref inout(T)) dg) inout;

 But then I realised a potential ambiguity:
 (a) the constancy is passed through to the delegate
 (b) the delegate has an inout parameter in its own right

Thinking about it, if we go with (a), then (b) can be achieved by defining an alias of the delegate type. Just one problem I can see: since a signature that uses (b) can't be represented in code without an alias, how would compiler messages, .stringof and TypeInfo notate the type? But if we go with (b), then there doesn't seem to be a way to achieve (a) without inventing new syntax. Stewart.
Mar 09 2012
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 09 Mar 2012 19:44:20 -0500, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:

 On 19/02/2012 14:27, Stewart Gordon wrote:
 <snip>
 int opApply(int delegate(ref inout(T)) dg) inout;

 But then I realised a potential ambiguity:
 (a) the constancy is passed through to the delegate
 (b) the delegate has an inout parameter in its own right

Thinking about it, if we go with (a), then (b) can be achieved by defining an alias of the delegate type. Just one problem I can see: since a signature that uses (b) can't be represented in code without an alias, how would compiler messages, .stringof and TypeInfo notate the type? But if we go with (b), then there doesn't seem to be a way to achieve (a) without inventing new syntax.

As I recently mentioned in the bug report, (b) must be the case without any additional syntax, because otherwise you have two types with identical syntax, but different underlying types. In order to do (a), we need new syntax. That's not a good thing, but not the end of the world. -Steve
Mar 09 2012