www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - trusted purity?

reply "monarch_dodra" <monarchdodra gmail.com> writes:
Is there *any* way to make a call to a non-pure function in a 
pure context, if you know you won't violate your own purity?

This is something you can do with  safe ( trusted), but what 
about pure?

For example, "free" is not pure, because you can't call it twice 
on the same pointer. But if you manage the pointer yourself 
inside a struct, you can guarantee the purity of your own 
functions. But the language won't allow you to do that.

Related question:
Can a function that "sometimes throws" be considered as pure?
Apr 29 2013
next sibling parent reply "Henning Pohl" <henning still-hidden.de> writes:
I've been working on a pull request and came up with something 
like this:

private void initialize(A...)(auto ref A args)
{
     auto m = cast(void* function(size_t size) pure)&malloc;
     _store = cast(Impl*) enforce(m(Impl.sizeof));
     auto r = cast(void function(in void* p, size_t sz) nothrow 
pure)&GC.addRange;
     static if (hasIndirections!T)
         r(&_store._payload, T.sizeof);
     emplace(&_store._payload, args);
     _store._count = 1;
}

The purity of "emplace" depends on the purity of the ctor called. 
I'm not sure how to fix that.
Apr 29 2013
parent reply "Henning Pohl" <henning still-hidden.de> writes:
By the way, my post is related to the impurity of RefCounted: 
http://d.puremagic.com/issues/show_bug.cgi?id=9998
Apr 29 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 29 April 2013 at 11:15:20 UTC, Henning Pohl wrote:

 I've been working on a pull request and came up with something 
 like this:

 private void initialize(A...)(auto ref A args)
 {
     auto m = cast(void* function(size_t size) pure)&malloc;
     _store = cast(Impl*) enforce(m(Impl.sizeof));
     auto r = cast(void function(in void* p, size_t sz) nothrow 
 pure)&GC.addRange;
     static if (hasIndirections!T)
         r(&_store._payload, T.sizeof);
     emplace(&_store._payload, args);
     _store._count = 1;
 }


I always forget you can cast the type of a function...


 The purity of "emplace" depends on the purity of the ctor 
 called. I'm not sure how to fix that.


I'm not sure there's anything to fix there: If the CTor is not 
pure, then how could emplace be pure?

I did some work on emplace that is awaiting to be pulled, which 
should improve its purity.

On Monday, 29 April 2013 at 11:19:33 UTC, Henning Pohl wrote:

 By the way, my post is related to the impurity of RefCounted: 
 http://d.puremagic.com/issues/show_bug.cgi?id=9998


Yes, that is also what I am investigating. The cast is would 
indeed be a fix for malloc/free. RefCounted's 
isInitialized/refCount still need to be marked as pure though.

I'm still worried about what it means for a pure function to 
throw... (I'm thinking about the  "enforce(malloc)" scheme)
Apr 29 2013
parent reply "Henning Pohl" <henning still-hidden.de> writes:
On Monday, 29 April 2013 at 11:32:33 UTC, monarch_dodra wrote:

 I'm still worried about what it means for a pure function to 
 throw... (I'm thinking about the  "enforce(malloc)" scheme)


If malloc returns null, we are out of memory. In D this is not an 
exception, it is an error. So I guess we just need to check the 
pointer returned by malloc and throw an OutOfMemoryError on 
failure. Thus if the ctor called is nothrow, it can be marked as 
nothrow, too.

So in this case, there should be no problem making it pure.
Apr 29 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 29 April 2013 at 11:50:11 UTC, Henning Pohl wrote:

 On Monday, 29 April 2013 at 11:32:33 UTC, monarch_dodra wrote:

 I'm still worried about what it means for a pure function to 
 throw... (I'm thinking about the  "enforce(malloc)" scheme)


 If malloc returns null, we are out of memory. In D this is not 
 an exception, it is an error. So I guess we just need to check 
 the pointer returned by malloc and throw an OutOfMemoryError on 
 failure. Thus if the ctor called is nothrow, it can be marked 
 as nothrow, too.

 So in this case, there should be no problem making it pure.


I've hit this issue before: In D, if the *managed* memory runs 
out, then it is an error (since then *everything* crumbles: 
arrays, GC. etc). The reason it is an error is that since the 
memory is managed by the language, there is nothing the user can 
do anyway, so throwing is pointless.

for unmanaged memory, on the otherhand, the user *can* do 
something about it, so throwing is better.

I myself am not sure I 100% agree with this, but that was the 
conclusion last time I tried to transform an malloc=>Exception 
into a malloc=>Error+Nothrow
Apr 29 2013
next sibling parent reply "Henning Pohl" <henning still-hidden.de> writes:
On Monday, 29 April 2013 at 12:08:58 UTC, monarch_dodra wrote:

 I've hit this issue before: In D, if the *managed* memory runs 
 out, then it is an error (since then *everything* crumbles: 
 arrays, GC. etc). The reason it is an error is that since the 
 memory is managed by the language, there is nothing the user 
 can do anyway, so throwing is pointless.

 for unmanaged memory, on the otherhand, the user *can* do 
 something about it, so throwing is better.

 I myself am not sure I 100% agree with this, but that was the 
 conclusion last time I tried to transform an malloc=>Exception 
 into a malloc=>Error+Nothrow


What about using allocators the user can specify? The default one 
would be malloc + Error + nothrow. All the signatures of 
RefCounted have to change depending on the allocator's ones, 
though. This is where attribute inference is rather needed.
Apr 29 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
I'm getting strange behavior trying to cast to pure. This is my
test program:

//--------
import std.stdio;
import core.stdc.stdlib;

void main()
{
      auto p1 = &core.stdc.stdlib.free;
      auto p2 = cast(void function(void*))&core.stdc.stdlib.free;
      auto p3 = cast(void function(void*)
pure)&core.stdc.stdlib.free;
      auto pp1 = core.stdc.stdlib.malloc(5);
      auto pp2 = core.stdc.stdlib.malloc(5);
      auto pp3 = core.stdc.stdlib.malloc(5);
      writeln(p1);
      p1(pp1);
      writeln(p2);
      p2(pp2); //This hangs
      writeln(p3); //Never reaches here
      p3(pp3);
}
//--------

Am I doing something wrong? Could somebody else test this? I'm on
win32.

I've also been getting some object violations trying to use this
cast...
Apr 29 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/29/2013 10:19 AM, monarch_dodra wrote:

 I'm getting strange behavior trying to cast to pure. This is my
 test program:


You're casting a C function to a D function. This will cause crashes.
Apr 29 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 29 April 2013 at 18:34:46 UTC, Walter Bright wrote:

 On 4/29/2013 10:19 AM, monarch_dodra wrote:

 I'm getting strange behavior trying to cast to pure. This is my
 test program:


 You're casting a C function to a D function. This will cause 
 crashes.


OK, so say I have a documented pure C function, how do I call it 
from a pure scope? You say, take the address and cast it, but not 
if it's C...
Apr 29 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/29/2013 1:03 PM, monarch_dodra wrote:

 On Monday, 29 April 2013 at 18:34:46 UTC, Walter Bright wrote:

 On 4/29/2013 10:19 AM, monarch_dodra wrote:

 I'm getting strange behavior trying to cast to pure. This is my
 test program:


 You're casting a C function to a D function. This will cause crashes.


 OK, so say I have a documented pure C function, how do I call it from a pure
 scope? You say, take the address and cast it, but not if it's C...


extern (C) {
     int foo(int);
}

extern (C) {
     pure int function(int) fp_pure_t;
}

...
cast(fp_pure_t)(&foo)
Apr 29 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 29 April 2013 at 20:51:50 UTC, Walter Bright wrote:

 On 4/29/2013 1:03 PM, monarch_dodra wrote:

 On Monday, 29 April 2013 at 18:34:46 UTC, Walter Bright wrote:

 On 4/29/2013 10:19 AM, monarch_dodra wrote:

 I'm getting strange behavior trying to cast to pure. This is 
 my
 test program:


 You're casting a C function to a D function. This will cause 
 crashes.


 OK, so say I have a documented pure C function, how do I call 
 it from a pure
 scope? You say, take the address and cast it, but not if it's 
 C...


 extern (C) {
     int foo(int);
 }

 extern (C) {
     pure int function(int) fp_pure_t;
 }

 ...
 cast(fp_pure_t)(&foo)


Thanks. That (kinda) worked. I just had to add an alias, because 
the compiler was complaining about: "Error: fp_pure_t is used as 
a type"

This works though:

//====
extern (C) {
     int foo(int);
}
extern (C) {
     alias pure int function(int) fp_pure_t;
}
...
cast(fp_pure_t)(&foo)
//====

That works.
Apr 29 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/29/2013 10:42 PM, monarch_dodra wrote:

 Thanks. That (kinda) worked. I just had to add an alias, because the
 compiler was complaining about: "Error: fp_pure_t is used as a type"


Yeah, I should have tested it before posting!
Apr 30 2013
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Tuesday, 30 April 2013 at 17:06:00 UTC, Walter Bright wrote:

 On 4/29/2013 10:42 PM, monarch_dodra wrote:

 Thanks. That (kinda) worked. I just had to add an alias, 
 because the
 compiler was complaining about: "Error: fp_pure_t is used as a 
 type"


 Yeah, I should have tested it before posting!


Oh, no problem, you gave me the right answer anyways, that's what
counts. Thanks.
Apr 30 2013
prev sibling parent Kenji Hara <k.hara.pg gmail.com> writes:
Hmm. Interesting approach. I tried to utilize the "trusted pure" concept .

template TrustedPure(alias func)
{
    import std.traits, std.algorithm;

    alias F1 = FunctionTypeOf!(func);
    static if (functionAttributes!F1 & FunctionAttribute.pure_)
    {
        alias TrustedPure = func;
    }
    else
    {
        alias F2 = SetFunctionAttributes!(
            F1,
            functionLinkage!F1,
            functionAttributes!F1 | FunctionAttribute.pure_);

        auto ref TrustedPure(A...)(auto ref A args)
        pure    // mark as expected
         system // represent 'unsafe' operation.
        {
            // forward!args does not work, because
            // std.algorithm.move is not pure...
            return (cast(F2*)&func)(/*forward!*/args);
        }
    }
}
void main() pure
// cannot add  safe, because TrustedPure functions are always  system
{
    import core.stdc.stdlib;
    alias pmalloc = TrustedPure!(core.stdc.stdlib.malloc);
    alias pfree = TrustedPure!(core.stdc.stdlib.free);

    auto p = cast(int*)pmalloc(int.sizeof);
    *p = 100;
    pfree(p);
}

Kenji Hara


2013/4/30 monarch_dodra <monarchdodra gmail.com>


 I'm getting strange behavior trying to cast to pure. This is my
 test program:

 //--------
 import std.stdio;
 import core.stdc.stdlib;

 void main()
 {
      auto p1 = &core.stdc.stdlib.free;
      auto p2 = cast(void function(void*))&core.stdc.**stdlib.free;
      auto p3 = cast(void function(void*)
 pure)&core.stdc.stdlib.free;
      auto pp1 = core.stdc.stdlib.malloc(5);
      auto pp2 = core.stdc.stdlib.malloc(5);
      auto pp3 = core.stdc.stdlib.malloc(5);
      writeln(p1);
      p1(pp1);
      writeln(p2);
      p2(pp2); //This hangs
      writeln(p3); //Never reaches here
      p3(pp3);
 }
 //--------

 Am I doing something wrong? Could somebody else test this? I'm on
 win32.

 I've also been getting some object violations trying to use this
 cast...
Apr 29 2013
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/29/2013 5:08 AM, monarch_dodra wrote:

 I've hit this issue before: In D, if the *managed* memory runs out, then it is
 an error (since then *everything* crumbles: arrays, GC. etc). The reason it is
 an error is that since the memory is managed by the language, there is nothing
 the user can do anyway, so throwing is pointless.

 for unmanaged memory, on the otherhand, the user *can* do something about it,
so
 throwing is better.


You cannot call a function pure if it sometimes throws a recoverable exception 
and sometimes does not, and this is not based on the supplied arguments.
Apr 29 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 29 April 2013 at 10:58:45 UTC, monarch_dodra wrote:

 Is there *any* way to make a call to a non-pure function in a 
 pure context, if you know you won't violate your own purity?

 This is something you can do with  safe ( trusted), but what 
 about pure?


This raise the case once again for trusted as a statement and not 
as a qualifier. Tis would solve the purity issue.


 Related question:
 Can a function that "sometimes throws" be considered as pure?


Yes as long at its behavior depends only on parameters.
Apr 29 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/29/2013 3:58 AM, monarch_dodra wrote:

 Is there *any* way to make a call to a non-pure function in a pure context, if
 you know you won't violate your own purity?


Vee haf veys:

1. put "debug" before the impure code (but you'll have to compile with -debug)

2. put the impure code in a separate function, take its address, and cast its 
address to being a pointer to a pure function. (Of course, such a cast should
be 
rejected by  safe code.)

3. Put the code in an extern(C) function, compiled separately as impure, but 
declared as pure in the client. C functions don't get name mangling, so the 
compiler won't know it's impure.


I feel it's a good thing that you'll need to jump through some hoops to do
this, 
otherwise 'pure' would not be very useful.




 Related question:
 Can a function that "sometimes throws" be considered as pure?


deadalnix's answer is correct.
Apr 29 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
I've been struggling with this, so here are my observations:

On Monday, 29 April 2013 at 18:31:15 UTC, Walter Bright wrote:

 On 4/29/2013 3:58 AM, monarch_dodra wrote:

 Is there *any* way to make a call to a non-pure function in a 
 pure context, if
 you know you won't violate your own purity?


 2. put the impure code in a separate function, take its 
 address, and cast its address to being a pointer to a pure 
 function. (Of course, such a cast should be rejected by  safe 
 code.)


This doesn't work with CTFE. I'm currently not seeing how I could 
make a function that needs to make a "trusted pure" call work at 
compile time: The function pointer cast will fail during CTFE, 
and if I add a "if (__ctfe)" block without it, then the function 
will be impure, due to the code inside the "if (__ctfe)" block.

I've yet to solve this problem.


 3. Put the code in an extern(C) function, compiled separately 
 as impure, but declared as pure in the client. C functions 
 don't get name mangling, so the compiler won't know it's impure.


Unfortunately, this doesn't work with templates. You have to 
force instantiation by inserting a straight up (dummy) call to 
the function, but that immediately makes the caller impure...


 I feel it's a good thing that you'll need to jump through some 
 hoops to do this, otherwise 'pure' would not be very useful.


I agree, but these aren't hoops, they're pole vaults.

FYI, the problem I'm trying to fix is this one:
* "uninitializedArray" returns an array with un-initialized 
elements. This, by definition, is not pure, since the value 
returned is garbage. I'm fixing the function so that it becomes 
*impure*.
* "array" is implemented in terms of "uninitializedArray": 
Allocate an array, and then fill it. "array" is pure, since its 
return is defined. array also works with ctfe.

I'm at a deadlock on this one.
Aug 19 2013
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
monarch_dodra:


 FYI, the problem I'm trying to fix is this one:
 * "uninitializedArray" returns an array with un-initialized 
 elements. This, by definition, is not pure, since the value 
 returned is garbage. I'm fixing the function so that it becomes 
 *impure*.
 * "array" is implemented in terms of "uninitializedArray": 
 Allocate an array, and then fill it. "array" is pure, since its 
 return is defined. array also works with ctfe.


Here are some examples of what I think you are talking about. 
foo1 is pure, foo2 is pure (but currently the type system doesn't 
complain), foo3 is pure again because all array items are 
deterministically initialized using only pure/immutable data:


// Pure:
int[3] foo1(in int x) pure {
     int[3] arr;
     return arr;
}

// Not pure:
int[3] foo2(in int x) pure {
     int[3] arr = void;
     return arr;
}

// Pure:
int[3] foo3(in int x) pure {
     int[3] arr = void;
     arr[] = x;
     return arr;
}

void main() {}


I presume foo2 should be refused as not pure. The array() 
function is like foo3, it creates data that is not pure, not 
deterministic, but then overwrites it all with referentially 
transparent information. So on the whole foo3 is pure and array() 
is often pure.

The problem is that while writing down the proof of the purity of 
foo3 is probably not too much hard, later the D compiler is not 
able to verify such proof. So some alternative solution is 
needed. The trusted pure you talk about is a solution, it means 
saying to the compiler, "trust me I have a proof of purity of 
this function". But programmers should be trusted as little as 
possible if you want a reliable language and reliable programs. 
So perhaps some mid-way solution is preferable.

Andrei used the idea of cooked and uncooked variables, it's 
probably used here:

class Bar1 {
     immutable int[2] x;
     this() {
     }
}

Bar1 gives the error:
Error: constructor test.Bar1.this missing initializer for 
immutable field x


While this gives no errors:

class Bar2 {
     immutable int[2] x;
     this() {
         x[0] = 1;
     }
}


Perhaps using a similar strategy you can accept a function like 
this:

int[3] foo3(in int x) pure {
     int[3] arr = void;
     arr[] = x;
     return arr;
}


Because now arr is not uncooked, it was overwritten by 
referentially transparent data...

For the function array() this is not enough, because instead of 
"= void" you have a function that returns some void-initialized 
data. To help the D type system a bit perhaps an annotations like 
 void_init is useful, to be attached to functions like 
uninitializedArray and minimallyInitializedArray.

Bye,
bearophile
Aug 19 2013
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 19 August 2013 at 12:55:54 UTC, bearophile wrote:

 The problem is that while writing down the proof of the purity 
 of foo3 is probably not too much hard, later the D compiler is 
 not able to verify such proof.

 Bye,
 bearophile


Right, that is pretty much it. EG:

//----
import core.stdc.stdlib;

int* myPureFun(int i) pure
{
     auto p = cast(int*) malloc(int.sizeof);
     if (!p) assert(0);
     *p = i;
     return p;
}
//----

I can solve this the same way (kind of) as with safety, by 
marking the function as I can mark a function as trusted, by 
casting the function as pure:

//----
int* myPureFun(int i) pure
{
     alias extern (C) void* function(size_t) pure PureF_t;
     auto p = cast(int*) (cast(PureF_t)&malloc)(int.sizeof);
     if (!p) assert(0);
     *p = i;
     return p;
}
//----

A basic "I know what I'm doing compiler" kind of assertion.

My only issue with doing this is I'm afraid it might be wrong: A 
"trusted" function means nothing to the compiler.

However, in the above example, I *marked* malloc as pure, and 
even though "myPureFun" is conceptually pure, *malloc* remains 
impure, and I don't know how the compiler deals with being told 
it is pure.

Is that code snippet wrong?
Aug 19 2013
prev sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 19 August 2013 at 12:55:54 UTC, bearophile wrote:

 For the function array() this is not enough, because instead of 
 "= void" you have a function that returns some void-initialized 
 data. To help the D type system a bit perhaps an annotations 
 like  void_init is useful, to be attached to functions like 
 uninitializedArray and minimallyInitializedArray.

 Bye,
 bearophile


After re-reading "pure" documentation, I think I can solve the 
problem by marking uninitialized/minimallyInitialized as "weakly 
pure". From there using it in a pure context is trivial.

This is in line with what GC.malloc does.
Aug 19 2013
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, April 29, 2013 12:58:44 monarch_dodra wrote:

 Is there *any* way to make a call to a non-pure function in a
 pure context, if you know you won't violate your own purity?
 
 This is something you can do with  safe ( trusted), but what
 about pure?
 
 For example, "free" is not pure, because you can't call it twice
 on the same pointer. But if you manage the pointer yourself
 inside a struct, you can guarantee the purity of your own
 functions. But the language won't allow you to do that.


You can cast a pointer to the function. std.datetime does that for the 
LocalTime and UTC singletons. Take a look at the semi-recently added 
std.traits.SetFunctionAttributes.


 Related question:
 Can a function that "sometimes throws" be considered as pure?


Throwing has nothing to do with purity. pure is purely a question of whether 
the function accesses module-level or static variables which can possibly be 
mutated after they're initialized. Strong purity (which is required for 
most/all optimizations) is then places additional requirements on the function 
parameters, but purity itself is simply a question of whether the function 
accesses module-level or static variables. So, throwing has nothing to do with 
it.

- Jonathan M Davis
Apr 29 2013