www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - It turns out it's quite hard to have safe pure nothrow functions. Oh,

reply "Atila Neves" <atila.neves gmail.com> writes:
This happens to me all the time. I write a function, stick the 
aforementioned attributes on as a default then let the compiler 
tell me when I can't.

That happens a lot more often than I thought it would. Pretty 
much anytime I call a Phobos function I have to remove at least 
one of them but usually all three.

Is it similar for everyone else? Is it considered a problem?

The other thing is I frequently have to "unconstify" my variables 
to get them accepted by Phobos functions as well.

I should've kept examples but I didn't, this is all from what I 
remember happening in the last week or so.

Atila
Sep 12 2014
next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Atila Neves:


 Is it similar for everyone else?


Yes, it's common.



 Is it considered a problem?


It's a moderately small problem, and it's being worked on since 
lot of time.

Originally D didn't have those attributes, so Phobos was not 
designed for them. So adding them to Phobos (and D) is a work in 
progress, and there are also some blocking problems (Like: 
http://forum.dlang.org/thread/stlslhjndgugecvmbowd forum.dlang.org 
). In some cases some functions can't just have all attributes, 
because they need to throw, or they are not designed to be pure, 
etc.

Bye,
bearophile
Sep 12 2014
prev sibling next sibling parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Friday, 12 September 2014 at 09:53:45 UTC, Atila Neves wrote:

 This happens to me all the time. I write a function, stick the 
 aforementioned attributes on as a default then let the compiler 
 tell me when I can't.

 That happens a lot more often than I thought it would. Pretty 
 much anytime I call a Phobos function I have to remove at least 
 one of them but usually all three.

 Is it similar for everyone else? Is it considered a problem?


Phobos still hasn't been fully annotated since these attributes 
were introduced, but we are making progress. For one, I believe 
we got  safe std.stdio recently, which should be a big boost for 
 safe adoption in general.

It is slowly getting better. Pull requests are welcome.


 The other thing is I frequently have to "unconstify" my 
 variables to get them accepted by Phobos functions as well.


D's const is very different from C++'s const. It's tempting to 
use in the same situations because of superficial similarities, 
but D's const should only be used when immutable is in the 
picture. D simply doesn't have the equivalent of C++'s const 
(which is intentional), despite their similar names.

That said, there are fundamental issues with const and immutable 
that have yet to be resolved - for example, given an immutable 
container or a const reference to a container, it's not possible 
to get a head-mutable range over it for iteration. This is 
different from in-built slices which are conveniently convertible 
from const(T[]) to const(T)[], something that is not expressible 
with user-defined types at the moment.

Further, `inout` does not support considering callback parameters 
to be "out parameters":

struct S
{
     int* p;

     inout(int)* foo() inout { return p; } // OK

     void bar(void delegate(inout int*) dg) inout { // Not 
supported
         dg(p);
     }
}

Both of these issues have been discussed before and IIRC, 
consensus seemed to be that we do want to do something about them.
Sep 12 2014
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
I know about the differences between C++ const and D const. I'm 
not talking about head-const, tail-const or logical const.
If I'm not mistaken, the last thing that happened to me was 
storing the captures from a regex into a const variable, then I 
couldn't index it.
I didn't look at the implementation, but it's very weird to me 
that getting an element can't be done on a const object. And yes, 
I'll look into doing a PR for it.

I'm using stdx.data.json a bit now as well, and will have to give 
feedback there. I can't make anything const, it seems.

I wasn't aware of  safe stdio, it always annoyed me that  safe 
functions can't call writeln, that never made any sense to me.

Atila

On Friday, 12 September 2014 at 10:19:27 UTC, Jakob Ovrum wrote:

 On Friday, 12 September 2014 at 09:53:45 UTC, Atila Neves wrote:

 This happens to me all the time. I write a function, stick the 
 aforementioned attributes on as a default then let the 
 compiler tell me when I can't.

 That happens a lot more often than I thought it would. Pretty 
 much anytime I call a Phobos function I have to remove at 
 least one of them but usually all three.

 Is it similar for everyone else? Is it considered a problem?


 Phobos still hasn't been fully annotated since these attributes 
 were introduced, but we are making progress. For one, I believe 
 we got  safe std.stdio recently, which should be a big boost 
 for  safe adoption in general.

 It is slowly getting better. Pull requests are welcome.


 The other thing is I frequently have to "unconstify" my 
 variables to get them accepted by Phobos functions as well.


 D's const is very different from C++'s const. It's tempting to 
 use in the same situations because of superficial similarities, 
 but D's const should only be used when immutable is in the 
 picture. D simply doesn't have the equivalent of C++'s const 
 (which is intentional), despite their similar names.

 That said, there are fundamental issues with const and 
 immutable that have yet to be resolved - for example, given an 
 immutable container or a const reference to a container, it's 
 not possible to get a head-mutable range over it for iteration. 
 This is different from in-built slices which are conveniently 
 convertible from const(T[]) to const(T)[], something that is 
 not expressible with user-defined types at the moment.

 Further, `inout` does not support considering callback 
 parameters to be "out parameters":

 struct S
 {
     int* p;

     inout(int)* foo() inout { return p; } // OK

     void bar(void delegate(inout int*) dg) inout { // Not 
 supported
         dg(p);
     }
 }

 Both of these issues have been discussed before and IIRC, 
 consensus seemed to be that we do want to do something about 
 them.
Sep 12 2014
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
12-Sep-2014 16:43, Atila Neves пишет:

 I know about the differences between C++ const and D const. I'm not
 talking about head-const, tail-const or logical const.
 If I'm not mistaken, the last thing that happened to me was storing the
 captures from a regex into a const variable, then I couldn't index it.
 I didn't look at the implementation, but it's very weird to me that
 getting an element can't be done on a const object. And yes, I'll look
 into doing a PR for it.


My problem with const is that people try to use it as often as logical 
const in C++ (everywhere), it DOESN'T HAVE to be as used as often.

Being binary-wise immutable is harsh requirement on implementation, and 
forbids many techniques (like COW and ref-counting). That's one reason 
we might never have const Captures in std.regex.


 I'm using stdx.data.json a bit now as well, and will have to give
 feedback there. I can't make anything const, it seems.

 I wasn't aware of  safe stdio, it always annoyed me that  safe functions
 can't call writeln, that never made any sense to me.

 Atila

 On Friday, 12 September 2014 at 10:19:27 UTC, Jakob Ovrum wrote:

 On Friday, 12 September 2014 at 09:53:45 UTC, Atila Neves wrote:

 This happens to me all the time. I write a function, stick the
 aforementioned attributes on as a default then let the compiler tell
 me when I can't.

 That happens a lot more often than I thought it would. Pretty much
 anytime I call a Phobos function I have to remove at least one of
 them but usually all three.

 Is it similar for everyone else? Is it considered a problem?


 Phobos still hasn't been fully annotated since these attributes were
 introduced, but we are making progress. For one, I believe we got
  safe std.stdio recently, which should be a big boost for  safe
 adoption in general.

 It is slowly getting better. Pull requests are welcome.


 The other thing is I frequently have to "unconstify" my variables to
 get them accepted by Phobos functions as well.


 D's const is very different from C++'s const. It's tempting to use in
 the same situations because of superficial similarities, but D's const
 should only be used when immutable is in the picture. D simply doesn't
 have the equivalent of C++'s const (which is intentional), despite
 their similar names.

 That said, there are fundamental issues with const and immutable that
 have yet to be resolved - for example, given an immutable container or
 a const reference to a container, it's not possible to get a
 head-mutable range over it for iteration. This is different from
 in-built slices which are conveniently convertible from const(T[]) to
 const(T)[], something that is not expressible with user-defined types
 at the moment.

 Further, `inout` does not support considering callback parameters to
 be "out parameters":

 struct S
 {
     int* p;

     inout(int)* foo() inout { return p; } // OK

     void bar(void delegate(inout int*) dg) inout { // Not supported
         dg(p);
     }
 }

 Both of these issues have been discussed before and IIRC, consensus
 seemed to be that we do want to do something about them.





-- 
Dmitry Olshansky
Sep 12 2014
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Friday, 12 September 2014 at 10:19:27 UTC, Jakob Ovrum wrote:

 Further, `inout` does not support considering callback 
 parameters to be "out parameters":

 struct S
 {
     int* p;

     inout(int)* foo() inout { return p; } // OK

     void bar(void delegate(inout int*) dg) inout { // Not 
 supported
         dg(p);
     }
 }


Looks like it works: http://dpaste.dzfl.pl/04a33be05658
Sep 12 2014
parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Friday, 12 September 2014 at 13:11:19 UTC, Kagamin wrote:

 On Friday, 12 September 2014 at 10:19:27 UTC, Jakob Ovrum wrote:

 Further, `inout` does not support considering callback 
 parameters to be "out parameters":

 struct S
 {
    int* p;

    inout(int)* foo() inout { return p; } // OK

    void bar(void delegate(inout int*) dg) inout { // Not 
 supported
        dg(p);
    }
 }


 Looks like it works: http://dpaste.dzfl.pl/04a33be05658


Now try calling it: http://dpaste.dzfl.pl/bbd02a4d61df
Sep 12 2014
parent reply "Kagamin" <spam here.lot> writes:
On Saturday, 13 September 2014 at 01:37:59 UTC, Jakob Ovrum wrote:

 Now try calling it: http://dpaste.dzfl.pl/bbd02a4d61df


That way it shouldn't work as it would break the type system, 
that code is equivalent to:

struct S
{
     int* p;

     inout(int)* foo() inout { return p; } // OK

     void bar(void delegate(const int*) dg) const {
         dg(p);
     }
}

void main()
{
	import std.stdio;
	S s;
	writeln(s.foo());
	
	immutable(S) s2;
	s2.bar((const int* p) => writeln(p));
	s.bar((const int* p) => writeln(p));
}

Is there a reason, why you would need inout there?
Sep 13 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/13/2014 06:44 PM, Kagamin wrote:

 Is there a reason, why you would need inout there?


s.bar((int* p){ ++*p; });
Sep 13 2014
parent reply "Kagamin" <spam here.lot> writes:
On Saturday, 13 September 2014 at 16:51:09 UTC, Timon Gehr wrote:

 s.bar((int* p){ ++*p; });


Huh? inout is for functions, which don't modify their arguments.
Sep 13 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/13/2014 07:48 PM, Kagamin wrote:

 On Saturday, 13 September 2014 at 16:51:09 UTC, Timon Gehr wrote:

 s.bar((int* p){ ++*p; });


 Huh? inout is for functions, which don't modify their arguments.


With Jakob's code working, this would not be warranted. (But the 
situation needs to change in any case. The inout delegate semantics DMD 
2.066.0 implements cause type unsoundness as explained here: 
https://issues.dlang.org/show_bug.cgi?id=10850 .)

If inout should still guarantee non-modification in the future, then 
nested inout functions will need to introduce their own inout context 
(i.e. the inout qualifier of the nested function should be incompatible 
with the inout qualifier of the enclosing function.) I assume this is a 
major PITA to implement in DMD, and it also leads to growing pains that 
are hard to counter without enhancing the syntax, hence the current 
semantics.

In any case, why is it important that inout functions are guaranteed not 
to change their arguments, even if they are mutable and the mutation is 
from within a delegate that was passed to such functions for this purpose?
Sep 13 2014
next sibling parent reply "Kagamin" <spam here.lot> writes:
On Saturday, 13 September 2014 at 18:36:53 UTC, Timon Gehr wrote:

 On 09/13/2014 07:48 PM, Kagamin wrote:

 On Saturday, 13 September 2014 at 16:51:09 UTC, Timon Gehr 
 wrote:

 s.bar((int* p){ ++*p; });


 Huh? inout is for functions, which don't modify their 
 arguments.


 With Jakob's code working, this would not be warranted.


Huh? See rationale in 
https://issues.dlang.org/show_bug.cgi?id=1961


 If inout should still guarantee non-modification in the future, 
 then nested inout functions will need to introduce their own 
 inout context (i.e. the inout qualifier of the nested function 
 should be incompatible with the inout qualifier of the 
 enclosing function.) I assume this is a major PITA to implement 
 in DMD, and it also leads to growing pains that are hard to 
 counter without enhancing the syntax, hence the current 
 semantics.


I think, it can be more or less simple without syntax 
enhancements, see the description of how it can be implemented.


 In any case, why is it important that inout functions are 
 guaranteed not to change their arguments, even if they are 
 mutable and the mutation is from within a delegate that was 
 passed to such functions for this purpose?


The same way const is important: it makes code easier to reason 
about.
Sep 13 2014
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/13/2014 09:58 PM, Kagamin wrote:

 On Saturday, 13 September 2014 at 18:36:53 UTC, Timon Gehr wrote:

 On 09/13/2014 07:48 PM, Kagamin wrote:

 On Saturday, 13 September 2014 at 16:51:09 UTC, Timon Gehr wrote:

 s.bar((int* p){ ++*p; });


 Huh? inout is for functions, which don't modify their arguments.


 With Jakob's code working, this would not be warranted.


 Huh? See rationale in https://issues.dlang.org/show_bug.cgi?id=1961
 ...


All the examples there keep their non-modification guarantees.
Modifications to inout parameters are visible at the call site.


 If inout should still guarantee non-modification in the future, then
 nested inout functions will need to introduce their own inout context
 (i.e. the inout qualifier of the nested function should be
 incompatible with the inout qualifier of the enclosing function.) I
 assume this is a major PITA to implement in DMD, and it also leads to
 growing pains that are hard to counter without enhancing the syntax,
 hence the current semantics.


 I think, it can be more or less simple without syntax enhancements, see
 the description of how it can be implemented.


Even without going into details, it is easy to appreciate the fact that 
erasing type information while enforcing type safety invariably tends to 
lead to pain some way down the road. E.g. the inout qualifier of the 
enclosing function should still be fully compatible with _itself_ even 
when data is accessed and modified from the nested function.
Sep 13 2014
parent reply "Kagamin" <spam here.lot> writes:
On Saturday, 13 September 2014 at 23:41:55 UTC, Timon Gehr wrote:

 All the examples there keep their non-modification guarantees.
 Modifications to inout parameters are visible at the call site.


Modifications are visible if you know the inout argument will be 
passed to the delegate, i.e. if you know the callee's 
implementation, and can't reason about behavior of inout argument 
just by seeing it being inout.


 E.g. the inout qualifier of the enclosing function should still 
 be fully compatible with _itself_ even when data is accessed 
 and modified from the nested function.


Yeah, I think, it's logical if the nested function is just a 
fragment of enclosing function; that way nested functions can be 
created without hindering access to variables of enclosing 
function.
Sep 14 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/14/2014 10:47 AM, Kagamin wrote:

 On Saturday, 13 September 2014 at 23:41:55 UTC, Timon Gehr wrote:

 All the examples there keep their non-modification guarantees.
 Modifications to inout parameters are visible at the call site.


 Modifications are visible if you know the inout argument will be passed
 to the delegate, i.e. if you know the callee's implementation,


The circumstances under which a callback will be invoked are hardly 
implementation details.


 and can't reason about behavior of inout argument just by seeing it being
inout.
 ...


Most inout functions don't also take an inout callback.
Sep 14 2014
prev sibling parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Saturday, 13 September 2014 at 19:58:16 UTC, Kagamin wrote:

 On Saturday, 13 September 2014 at 18:36:53 UTC, Timon Gehr 
 wrote:

 On 09/13/2014 07:48 PM, Kagamin wrote:

 On Saturday, 13 September 2014 at 16:51:09 UTC, Timon Gehr 
 wrote:

 s.bar((int* p){ ++*p; });


 Huh? inout is for functions, which don't modify their 
 arguments.


 With Jakob's code working, this would not be warranted.


 Huh? See rationale in 
 https://issues.dlang.org/show_bug.cgi?id=1961


The function doesn't modify `p` - it's modified by a callback 
that was accepted because it's known at the call-site that `p` is 
modifiable.

This is necessary for `inout` to work with callback functions, 
such as with internal iteration (i.e. `opApply`). It can be 
worked around exactly the same way you would work around it with 
functions that return a value - by duplicating the function. It 
is essentially the same problem and thus `inout` could easily be 
used to fix it.
Sep 13 2014
parent reply "Kagamin" <spam here.lot> writes:
On Sunday, 14 September 2014 at 01:38:33 UTC, Jakob Ovrum wrote:

 This is necessary for `inout` to work with callback functions, 
 such as with internal iteration (i.e. `opApply`).


Can you write a pass and xfail test for it? This scenario looks 
non-trivial.


 It can be worked around exactly the same way you would work 
 around it with functions that return a value - by duplicating 
 the function. It is essentially the same problem and thus 
 `inout` could easily be used to fix it.


Can you elaborate? I don't quite understand, what you mean.
Sep 14 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/14/2014 10:50 AM, Kagamin wrote:

 On Sunday, 14 September 2014 at 01:38:33 UTC, Jakob Ovrum wrote:

 This is necessary for `inout` to work with callback functions, such as
 with internal iteration (i.e. `opApply`).


 Can you write a pass and xfail test for it? This scenario looks
 non-trivial.
 ...


struct S{
     int[] a;
     int opApply(scope int delegate(ref inout int) dg)inout{
         foreach(ref x;a) if(auto r=dg(x)) return r;
         return 0;
     }
}

void main(){
     auto s=S([1,2,3]);
     foreach(ref x;s) x++; // ok
     const(S) s2=s;
     foreach(ref x;s2){
         import std.stdio;
         writeln(x);
         // x++; error
     }
}



 It can be worked around exactly the same way you would work around it
 with functions that return a value - by duplicating the function. It
 is essentially the same problem and thus `inout` could easily be used
 to fix it.


 Can you elaborate? I don't quite understand, what you mean.


inout's primary goal is to eliminate patterns similar to:

struct S{
     int[] a;
     int opApply(scope int delegate(ref int) dg){
         foreach(ref x;a) if(auto r=dg(x)) return r;
         return 0;
     }
     int opApply(scope int delegate(ref const int) dg)const{
         foreach(ref x;a) if(auto r=dg(x)) return r;
         return 0;
     }
     int opApply(scope int delegate(ref immutable int) dg)immutable{
         foreach(ref x;a) if(auto r=dg(x)) return r;
         return 0;
     }
}
Sep 14 2014
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Saturday, 13 September 2014 at 18:36:53 UTC, Timon Gehr wrote:

 (But the situation needs to change in any case. The inout 
 delegate semantics DMD 2.066.0 implements cause type 
 unsoundness as explained here: 
 https://issues.dlang.org/show_bug.cgi?id=10850 .)


BTW, how the change manifests? Your examples show the qualifier 
is not really bound to the outermost function.
Sep 13 2014
parent "Kagamin" <spam here.lot> writes:
On Saturday, 13 September 2014 at 20:01:49 UTC, Kagamin wrote:

 On Saturday, 13 September 2014 at 18:36:53 UTC, Timon Gehr 
 wrote:

 (But the situation needs to change in any case. The inout 
 delegate semantics DMD 2.066.0 implements cause type 
 unsoundness as explained here: 
 https://issues.dlang.org/show_bug.cgi?id=10850 .)


 BTW, how the change manifests? Your examples show the qualifier 
 is not really bound to the outermost function.


Ah, I see, but the error message is strange.
Sep 13 2014
prev sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Sep 12, 2014 at 09:53:44AM +0000, Atila Neves via Digitalmars-d wrote:

 This happens to me all the time. I write a function, stick the
 aforementioned attributes on as a default then let the compiler tell
 me when I can't.
 
 That happens a lot more often than I thought it would. Pretty much
 anytime I call a Phobos function I have to remove at least one of them
 but usually all three.


Yeah, I run into that a lot. Unfortunately, it just makes me give up and
not use those attributes.



 Is it similar for everyone else? Is it considered a problem?


Well, we've been getting a steady stream of Phobos PR's to annotate
Phobos functions so that they have maximal attributes. It's slow work,
but it's progressing. Not long ago, for example, std.format was made
pure and  safe, so now it is CTFE-able, which is a big boon for
compile-time metaprogramming. Additional help in this area would be
greatly appreciated!



 The other thing is I frequently have to "unconstify" my variables to
 get them accepted by Phobos functions as well.
 
 I should've kept examples but I didn't, this is all from what I
 remember happening in the last week or so.


[...]

Keep in mind that D's const is much stronger than C++'s const, it
imposes *binary* const-ness, not just logical const-ness like in C++.
This means you can't do logically const operations like lazy class
object loading, caching, updating reference counts, etc.. Which in turn
means that there are less places where you can use const in D, than in
C++.

There are some rough corners, though, like iterating over a const
container. If the container is a linked-list, say, and you pass in a
const container, then all pointers in the container will be const as
well, by transitivity. However, that means you can't iterate over the
pointers, because, being const, they cannot be modified, so it's illegal
to update your iteration pointer! There are ways to work around this
without breaking the type system, but it does require extra work.


T

-- 
Public parking: euphemism for paid parking. -- Flora
Sep 12 2014