• Paul Backus (83/83) Nov 12 2021 In the recent [discussion about reference counting][1], a
• zjh (3/5) Nov 12 2021 This technique is worth utilizing.
• Timon Gehr (18/21) Nov 12 2021 It does not matter how you do it if the compiler assumes it does not
• tsbockman (19/35) Nov 13 2021 That's a compiler bug. The explicit return type of `ptr` is being
• Timon Gehr (3/6) Nov 13 2021 Nonsense. Results of strongly pure function calls implicitly convert to
• tsbockman (10/16) Nov 13 2021 I see. This does not appear to be documented anywhere in the spec
• Timon Gehr (6/24) Nov 13 2021 Yes, the documentation around qualifiers is pretty incomplete in
• Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= (3/9) Nov 13 2021 D needs to stop adding weirdness. If you want strong purity,
• Imperatorn (2/10) Nov 13 2021 Who can improve the documentation?
• rikki cattermole (3/4) Nov 13 2021 Anyone.
• Imperatorn (4/9) Nov 13 2021 Well, yeah, what I mean was who can "figure out if the compiler
• Paul Backus (3/15) Nov 13 2021 The compiler is correct; see
• Imperatorn (2/18) Nov 13 2021 Oh, didn't see that post
• rikki cattermole (1/1) Nov 13 2021 The bug report Thomas opened: https://issues.dlang.org/show_bug.cgi?id=2...
• Paul Backus (7/23) Nov 13 2021 The conversion itself is not documented, but the spec does
• Timon Gehr (4/7) Nov 13 2021 How else are you going to represent your values? I really don't get this...
• Paul Backus (18/26) Nov 13 2021 In a language where valid programs cannot distinguish between
• Imperatorn (4/11) Nov 13 2021 Many use the "compromise edition" to pure, I think it's ok. Pure
• Paul Backus (4/19) Nov 13 2021 [`pureMalloc`][1] does not return the same output for a given
• Imperatorn (2/22) Nov 13 2021 Yeah, these words are not precise enough.
• Paul Backus (5/24) Nov 13 2021 What does "equivalent" mean, here? The intent seems to be
• Andrei Alexandrescu (3/27) Nov 16 2021 The upside of these missing definitions is you have the opportunity to
• Dukc (17/37) Nov 13 2021 I think this means that while the language lets you to change
• Paul Backus (11/18) Nov 13 2021 What you do is have the compiler enforce the preconditions
• Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= (8/20) Nov 13 2021 That would undermine performance and render pure even more
• Dukc (14/30) Nov 13 2021 Not at all. He suggested having the compiler to catch the
• Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= (24/29) Nov 13 2021 He argued against using identity and relying fully on content,
• Dukc (8/22) Nov 14 2021 In `@safe` code with only the language constructs, that is. You
• Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= (20/23) Nov 14 2021 Yes, but this is where things are starting to get hazy. Can we
• Imperatorn (3/7) Nov 13 2021 It might be crazy 😅, but would be nice if some variant of this is
• Dukc (9/12) Nov 13 2021 Unless I miss something else, this is otherwise a good idea, BUT:
• Elronnd (7/7) Nov 13 2021 I think ultimately, the main problem is that d hasn't

Paul Backus <snarwin gmail.com> writes:

In the recent [discussion about reference counting][1], a 
question that's repeatedly come up is, how can we store a pointer 
to immutable data in immutable memory?

So far, suggested solutions have included:

- Add a `__mutable` qualifier that functions as a "back door" to 
`immutable` and `const`.
- Store the mutable data at some offset outside the object's 
declared layout, and access it with pointer arithmetic.
- Store the mutable data in a global associative array.

All of these solutions have unsatisfying tradeoffs: some are 
incompatible with `pure`; others require language changes to work 
without causing undefined behavior. It seems inevitable that we 
will have to accept some kind of compromise--our only choice is 
which one.

Except...what if we don't? What if the One Weird Trick that will 
give us everything we want has been sitting under our noses all 
along?

I present: `TailUnqual`!

```d
import tail_unqual;

void main()  safe pure
{
     // Qualifiers on a TailUnqual variable are not 
transitive--that is,
     // they apply only to the variable itself, not the data it 
points to.
     immutable TailUnqual!(int*) p = new int(123);
     assert(*p == 123);

     // We can't mutate `p` itself, because it's immutable...
     assert(!__traits(compiles,
         p = immutable(TailUnqual!(int*))(new int(789))
     ));

     // ...but we can mutate the data it points to!
     *p = 456;
     assert(*p == 456);
}
```

How does it work? How can I possibly break such a fundamental 
language rule and get away with it? As it turns out, the answer 
is almost embarrassingly simple:

* You're allowed to cast a pointer to an integer, and then cast 
that integer back to the same pointer.
* You're allowed to convert integer values freely between 
mutable, const, and immutable.

So, all we have to do is (a) store the pointer as an integer, and 
(b) convert it *back* to a pointer every time we access it. Add a 
little bit of `union` seasoning to satisfy the GC, and you get 
the embarrassingly simple implementation behind this magic trick:

```d
module tail_unqual;

import std.traits: isPointer;

private union HiddenPointer
{
     // Stores a pointer, cast to an integer
     size_t address;

     // Enables GC scanning (and prevents some other UB)
     void* unused;
}

struct TailUnqual(P)
     if (isPointer!P)
{
     private HiddenPointer data;

     this(P ptr) inout
     {
         data.address = cast(size_t) ptr;
     }

      trusted  property
     P ptr() inout
     {
         // This is safe because:
         //  - `address` is always the active field of `data`
         //  - `data.address` was originally cast from a `P`
         return cast(P) data.address;
     }

     alias ptr this;
}
```

[Try it yourself on `run.dlang.io`.][2]

What do you think? Is it just crazy enough to work, or just plain 
crazy? Is there some fatal safety violation I've overlooked? Let 
me know in your replies!

[1]: https://forum.dlang.org/thread/smc5jb$2u8t$1 digitalmars.com
[2]: https://run.dlang.io/is/OVTbD3

zjh <fqbqrr 163.com> writes:

On Saturday, 13 November 2021 at 06:50:48 UTC, Paul Backus wrote:
 - Add a `__mutable` qualifier that functions as a "back door" 
 to `immutable` and `const`.

This technique is worth utilizing.
The `pointer` remains immutable and the `pointee data` is mutable.

Timon Gehr <timon.gehr gmx.ch> writes:

On 13.11.21 07:50, Paul Backus wrote:
 
 What do you think? Is it just crazy enough to work, or just plain crazy?

Second option.

 Is there some fatal safety violation I've overlooked?

It does not matter how you do it if the compiler assumes it does not 
happen...

DMD 2.098.0:

```d
void main() safe{
     immutable TailUnqual!(int*) p = new int(123);
     auto mut = p.ptr;
     immutable immut = p.ptr;
     import std.stdio;
     writeln(*immut); // 123
     *mut=0;
     assert(mut is immut);
     assert(*mut == *immut);
     writeln(*mut," ",*immut); // 0 123
}
```

tsbockman <thomas.bockman gmail.com> writes:

On Saturday, 13 November 2021 at 07:20:37 UTC, Timon Gehr wrote:
 It does not matter how you do it if the compiler assumes it 
 does not happen...

 DMD 2.098.0:

 ```d
 void main() safe{
     immutable TailUnqual!(int*) p = new int(123);
     auto mut = p.ptr;
     immutable immut = p.ptr;
     import std.stdio;
     writeln(*immut); // 123
     *mut=0;
     assert(mut is immut);
     assert(*mut == *immut);
     writeln(*mut," ",*immut); // 0 123
 }
 ```

That's a compiler bug. The explicit return type of `ptr` is being 
ignored when assigning to `immutable immut`. Stupid fix:

```D

     private __gshared int dummy = int.min;

      trusted  property
     P ptr() inout
     {
         P ret = & dummy; // Persuade the frontend to actually use 
the explicit return type.
         ret = cast(P) data.address;
         return ret;
     }

```

Now the compiler will correctly identify your `main` as illegal:
```
onlineapp.d(43): Error: cannot implicitly convert expression 
`p.ptr()` of type `int*` to `immutable(int*)`
```

Timon Gehr <timon.gehr gmx.ch> writes:

On 13.11.21 09:30, tsbockman wrote:

 
 That's a compiler bug.

Nonsense. Results of strongly pure function calls implicitly convert to 
immutable. Your "fix" just avoided purity inference.

tsbockman <thomas.bockman gmail.com> writes:

On Saturday, 13 November 2021 at 08:32:42 UTC, Timon Gehr wrote:
 On 13.11.21 09:30, tsbockman wrote:

 
 That's a compiler bug.

 Nonsense. Results of strongly pure function calls implicitly 
 convert to immutable. Your "fix" just avoided purity inference.

I see. This does not appear to be documented anywhere in the spec 
at the moment.

I found definitions for "strongly pure" and "pure factory 
function", but nothing about exceptions to the normal implicit 
conversion rules. After being defined, the latter term is never 
used again in the spec.

There is also no mention of this in the "Creating Immutable Data" 
section:
     https://dlang.org/spec/const3.html#creating_immutable_data

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/13/21 10:06 AM, tsbockman wrote:
 On Saturday, 13 November 2021 at 08:32:42 UTC, Timon Gehr wrote:
 On 13.11.21 09:30, tsbockman wrote:

 That's a compiler bug.

 Nonsense. Results of strongly pure function calls implicitly convert 
 to immutable. Your "fix" just avoided purity inference.

 
 I see. This does not appear to be documented anywhere in the spec at the 
 moment.
 
 I found definitions for "strongly pure" and "pure factory function", but 
 nothing about exceptions to the normal implicit conversion rules. After 
 being defined, the latter term is never used again in the spec.
 
 There is also no mention of this in the "Creating Immutable Data" section:
      https://dlang.org/spec/const3.html#creating_immutable_data

Yes, the documentation around qualifiers is pretty incomplete in 
general, which is one of the problems plaguing this discussion. The 
specification does clearly state that Paul's trick is not allowed though.

NB: This is the relevant bit of DMD source code:
https://github.com/dlang/dmd/blob/master/src/dmd/dcast.d#L856-L872

Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:

On Saturday, 13 November 2021 at 09:37:41 UTC, Timon Gehr wrote:
 Yes, the documentation around qualifiers is pretty incomplete 
 in general, which is one of the problems plaguing this 
 discussion. The specification does clearly state that Paul's 
 trick is not allowed though.

 NB: This is the relevant bit of DMD source code:
 https://github.com/dlang/dmd/blob/master/src/dmd/dcast.d#L856-L872

D needs to stop adding weirdness. If you want strong purity, 
define it, give it an explicit signifier.

Imperatorn <johan_forsberg_86 hotmail.com> writes:

On Saturday, 13 November 2021 at 09:37:41 UTC, Timon Gehr wrote:
 On 11/13/21 10:06 AM, tsbockman wrote:
  [...]

 Yes, the documentation around qualifiers is pretty incomplete 
 in general, which is one of the problems plaguing this 
 discussion. The specification does clearly state that Paul's 
 trick is not allowed though.

 NB: This is the relevant bit of DMD source code:
 https://github.com/dlang/dmd/blob/master/src/dmd/dcast.d#L856-L872

Who can improve the documentation?

rikki cattermole <rikki cattermole.co.nz> writes:

On 13/11/2021 11:06 PM, Imperatorn wrote:
 Who can improve the documentation?

Anyone.

The only problem is figuring out if the compiler is correct or the docs.

Imperatorn <johan_forsberg_86 hotmail.com> writes:

On Saturday, 13 November 2021 at 10:08:27 UTC, rikki cattermole 
wrote:
 On 13/11/2021 11:06 PM, Imperatorn wrote:
 Who can improve the documentation?

 Anyone.

 The only problem is figuring out if the compiler is correct or 
 the docs.

Well, yeah, what I mean was who can "figure out if the compiler 
is correct or the docs" :)

Paul Backus <snarwin gmail.com> writes:

On Saturday, 13 November 2021 at 12:59:41 UTC, Imperatorn wrote:
 On Saturday, 13 November 2021 at 10:08:27 UTC, rikki cattermole 
 wrote:
 On 13/11/2021 11:06 PM, Imperatorn wrote:
 Who can improve the documentation?

 Anyone.

 The only problem is figuring out if the compiler is correct or 
 the docs.

 Well, yeah, what I mean was who can "figure out if the compiler 
 is correct or the docs" :)

The compiler is correct; see 
https://forum.dlang.org/post/rimxulakeebqfnuzeidw forum.dlang.org

Imperatorn <johan_forsberg_86 hotmail.com> writes:

On Saturday, 13 November 2021 at 13:01:08 UTC, Paul Backus wrote:
 On Saturday, 13 November 2021 at 12:59:41 UTC, Imperatorn wrote:
 On Saturday, 13 November 2021 at 10:08:27 UTC, rikki 
 cattermole wrote:
 On 13/11/2021 11:06 PM, Imperatorn wrote:
 Who can improve the documentation?

 Anyone.

 The only problem is figuring out if the compiler is correct 
 or the docs.

 Well, yeah, what I mean was who can "figure out if the 
 compiler is correct or the docs" :)

 The compiler is correct; see 
 https://forum.dlang.org/post/rimxulakeebqfnuzeidw forum.dlang.org

Oh, didn't see that post

rikki cattermole <rikki cattermole.co.nz> writes:

The bug report Thomas opened: https://issues.dlang.org/show_bug.cgi?id=22509

Paul Backus <snarwin gmail.com> writes:

On Saturday, 13 November 2021 at 09:06:23 UTC, tsbockman wrote:
 On Saturday, 13 November 2021 at 08:32:42 UTC, Timon Gehr wrote:
 On 13.11.21 09:30, tsbockman wrote:

 
 That's a compiler bug.

 Nonsense. Results of strongly pure function calls implicitly 
 convert to immutable. Your "fix" just avoided purity inference.

 I see. This does not appear to be documented anywhere in the 
 spec at the moment.

The conversion itself is not documented, but the spec does 
explain why the original code is not allowed:

 A pure factory function is a strongly pure function that 
 returns a result that has mutable indirections. All mutable 
 memory returned by the call may not be referenced by any other 
 part of the program, i.e. it is newly allocated by the 
 function. Nor may the mutable references of the result refer to 
 any object that existed before the function call.

So, having a strongly-pure function return a mutable pointer to 
an existing object is forbidden.

I guess this is what we get for trying to make memory allocation 
count as "pure."

Timon Gehr <timon.gehr gmx.ch> writes:

On 13.11.21 13:42, Paul Backus wrote:
 
 I guess this is what we get for trying to make memory allocation count 
 as "pure."

How else are you going to represent your values? I really don't get this 
trend of questioning whether memory allocation should be `pure`. I think 
it may be caused by thinking at the wrong level of abstraction.

Paul Backus <snarwin gmail.com> writes:

On Saturday, 13 November 2021 at 13:11:05 UTC, Timon Gehr wrote:
 On 13.11.21 13:42, Paul Backus wrote:
 
 I guess this is what we get for trying to make memory 
 allocation count as "pure."

 How else are you going to represent your values? I really don't 
 get this trend of questioning whether memory allocation should 
 be `pure`. I think it may be caused by thinking at the wrong 
 level of abstraction.

In a language where valid programs cannot distinguish between 
different pointers to the same value, or observe the side effects 
of memory allocation, it is natural to define `pure` such that it 
allows memory allocation.

D is not that kind of language. A valid D program can very easily 
distinguish between different pointers to the same value, and 
observe the side effects of memory allocation. In a language like 
D, defining `pure` such that it allows memory allocation means 
that (a) you can't, in general, assume a strongly-pure function 
has no observable side effects and is referentially transparent 
[1], which makes `pure` less useful for static analysis and 
optimization; and (b) some functions that do have no observable 
side effects and are referentially transparent nevertheless 
cannot be `pure`, which is weird and unintuitive.

[1] Actually, the spec says that the compiler is allowed to 
assume referential transparency anyway, which turns this from an 
unfortunate limitation into a loaded foot-gun.

Imperatorn <johan_forsberg_86 hotmail.com> writes:

On Saturday, 13 November 2021 at 14:32:15 UTC, Paul Backus wrote:
 On Saturday, 13 November 2021 at 13:11:05 UTC, Timon Gehr wrote:
 [...]

 In a language where valid programs cannot distinguish between 
 different pointers to the same value, or observe the side 
 effects of memory allocation, it is natural to define `pure` 
 such that it allows memory allocation.

 [...]

Many use the "compromise edition" to pure, I think it's ok. Pure 
in that sense is just same out for a given in, doesn't matter 
what the function does inside.

Paul Backus <snarwin gmail.com> writes:

On Saturday, 13 November 2021 at 14:35:36 UTC, Imperatorn wrote:
 On Saturday, 13 November 2021 at 14:32:15 UTC, Paul Backus 
 wrote:
 On Saturday, 13 November 2021 at 13:11:05 UTC, Timon Gehr 
 wrote:
 [...]

 In a language where valid programs cannot distinguish between 
 different pointers to the same value, or observe the side 
 effects of memory allocation, it is natural to define `pure` 
 such that it allows memory allocation.

 [...]

 Many use the "compromise edition" to pure, I think it's ok. 
 Pure in that sense is just same out for a given in, doesn't 
 matter what the function does inside.

[`pureMalloc`][1] does not return the same output for a given 
input.

[1]: https://druntime.dpldocs.info/core.memory.pureMalloc.html

Imperatorn <johan_forsberg_86 hotmail.com> writes:

On Saturday, 13 November 2021 at 14:43:00 UTC, Paul Backus wrote:
 On Saturday, 13 November 2021 at 14:35:36 UTC, Imperatorn wrote:
 On Saturday, 13 November 2021 at 14:32:15 UTC, Paul Backus 
 wrote:
 On Saturday, 13 November 2021 at 13:11:05 UTC, Timon Gehr 
 wrote:
 [...]

 In a language where valid programs cannot distinguish between 
 different pointers to the same value, or observe the side 
 effects of memory allocation, it is natural to define `pure` 
 such that it allows memory allocation.

 [...]

 Many use the "compromise edition" to pure, I think it's ok. 
 Pure in that sense is just same out for a given in, doesn't 
 matter what the function does inside.

 [`pureMalloc`][1] does not return the same output for a given 
 input.

 [1]: https://druntime.dpldocs.info/core.memory.pureMalloc.html

Yeah, these words are not precise enough.

Paul Backus <snarwin gmail.com> writes:

On Saturday, 13 November 2021 at 16:57:47 UTC, Imperatorn wrote:
 On Saturday, 13 November 2021 at 14:43:00 UTC, Paul Backus 
 wrote:
 On Saturday, 13 November 2021 at 14:35:36 UTC, Imperatorn 
 wrote:
 Many use the "compromise edition" to pure, I think it's ok. 
 Pure in that sense is just same out for a given in, doesn't 
 matter what the function does inside.

 [`pureMalloc`][1] does not return the same output for a given 
 input.

 [1]: https://druntime.dpldocs.info/core.memory.pureMalloc.html

 Yeah, these words are not precise enough.

To be fair, the language spec is also not very precise about this:

 An implementation may assume that a strongly pure function that 
 returns a result without mutable indirections will have the 
 same effect for all invocations with equivalent arguments. It 
 is allowed to memoize the result of the function under the 
 assumption that equivalent parameters always produce equivalent 
 results.

What does "equivalent" mean, here? The intent seems to be 
something like "equal, if you ignore differences in memory 
addresses," but the term is never actually defined.

Andrei Alexandrescu <SeeWebsiteForEmail erdani.com> writes:

On 2021-11-13 12:14, Paul Backus wrote:
 On Saturday, 13 November 2021 at 16:57:47 UTC, Imperatorn wrote:
 On Saturday, 13 November 2021 at 14:43:00 UTC, Paul Backus wrote:
 On Saturday, 13 November 2021 at 14:35:36 UTC, Imperatorn wrote:
 Many use the "compromise edition" to pure, I think it's ok. Pure in 
 that sense is just same out for a given in, doesn't matter what the 
 function does inside.

 [`pureMalloc`][1] does not return the same output for a given input.

 [1]: https://druntime.dpldocs.info/core.memory.pureMalloc.html

 Yeah, these words are not precise enough.

 
 To be fair, the language spec is also not very precise about this:
 
 An implementation may assume that a strongly pure function that 
 returns a result without mutable indirections will have the same 
 effect for all invocations with equivalent arguments. It is allowed to 
 memoize the result of the function under the assumption that 
 equivalent parameters always produce equivalent results.

 
 What does "equivalent" mean, here? The intent seems to be something like 
 "equal, if you ignore differences in memory addresses," but the term is 
 never actually defined.

The upside of these missing definitions is you have the opportunity to 
introduce them. I'm sure related pull requests would be received well.

Dukc <ajieskola gmail.com> writes:

On Saturday, 13 November 2021 at 14:32:15 UTC, Paul Backus wrote:
 In a language where valid programs cannot distinguish between 
 different pointers to the same value, or observe the side 
 effects of memory allocation, it is natural to define `pure` 
 such that it allows memory allocation.

 D is not that kind of language.

I think it's supposed to be. Quoting the language spec:

 Implementation Defined: An implementation may assume that a 
 strongly pure function that returns a result without mutable 
 indirections will have the same effect for all invocations with 
 equivalent arguments. It is allowed to memoize the result of 
 the function under the assumption that equivalent parameters 
 always produce equivalent results. A strongly pure function may 
 still have behavior inconsistent with memoization by e.g. using 
 casts or by changing behavior depending on the address of its 
 parameters. An implementation is currently not required to 
 enforce validity of memoization in all cases. If a strongly 
 pure function throws an Exception or an Error, the assumptions 
 related to memoization do not carry to the thrown exception.

I think this means that while the language lets you to change 
what `pure auto foo(immutable(int)[])` returns based on the 
address of the argument, the language is allowed to cache the 
result as if you could not do that. The spec only talks about 
immutable references, but still.

 [1] Actually, the spec says that the compiler is allowed to 
 assume referential transparency anyway, which turns this from 
 an unfortunate limitation into a loaded foot-gun.

Ah, you already realized what I just said above. Yeah, it's an 
unfortunate trap but what do you do? Compiler optimisation is 
based on assumptions and assumptions usually necessiate pitfalls 
like this.

I think it would be a good idea to provide a compiler switch to 
disable `pure` assumptions for programs that want to give 
optimisations away for more reliability, but it shouldn't be a 
language rule.

Fortunately it's "only" implementation-defined thing - not fully 
undefined behaviour.

Paul Backus <snarwin gmail.com> writes:

On Saturday, 13 November 2021 at 15:01:08 UTC, Dukc wrote:
 [1] Actually, the spec says that the compiler is allowed to 
 assume referential transparency anyway, which turns this from 
 an unfortunate limitation into a loaded foot-gun.

 Ah, you already realized what I just said above. Yeah, it's an 
 unfortunate trap but what do you do? Compiler optimisation is 
 based on assumptions and assumptions usually necessiate 
 pitfalls like this.

What you do is have the compiler enforce the preconditions 
necessary to make its assumptions hold. So, if the result of a 
`pure` function is not supposed to depend on the specific values 
of any pointers, `pure` functions should be forbidden from 
comparing pointers, casting them to integers, or performing any 
other operations that might introduce such a dependency.

Or, alternatively, if that's too much trouble, you recognize that 
the assumption underlying this optimization is invalid, and you 
remove it from the compiler and the language spec. (Which I 
expect is what will happen in practice, sooner or later.)

Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:

On Saturday, 13 November 2021 at 15:16:43 UTC, Paul Backus wrote:
 What you do is have the compiler enforce the preconditions 
 necessary to make its assumptions hold. So, if the result of a 
 `pure` function is not supposed to depend on the specific 
 values of any pointers, `pure` functions should be forbidden 
 from comparing pointers, casting them to integers, or 
 performing any other operations that might introduce such a 
 dependency.

That would undermine performance and render pure even more 
useless than it already is.

 Or, alternatively, if that's too much trouble, you recognize 
 that the assumption underlying this optimization is invalid, 
 and you remove it from the compiler and the language spec. 
 (Which I expect is what will happen in practice, sooner or 
 later.)

Just remove the notion of normative "strongly pure", it is a big 
mistake to split a concept based on the input typing. Confusing 
two concepts like this in one construct is making the language 
more complex than keeping the concepts separate. It is ugly. Keep 
it clean, keep semantics of each concept simple.

Dukc <ajieskola gmail.com> writes:

On Saturday, 13 November 2021 at 18:06:37 UTC, Ola Fosheim 
Grøstad wrote:
 On Saturday, 13 November 2021 at 15:16:43 UTC, Paul Backus 
 wrote:
 What you do is have the compiler enforce the preconditions 
 necessary to make its assumptions hold. So, if the result of a 
 `pure` function is not supposed to depend on the specific 
 values of any pointers, `pure` functions should be forbidden 
 from comparing pointers, casting them to integers, or 
 performing any other operations that might introduce such a 
 dependency.

 That would undermine performance and render pure even more 
 useless than it already is.

Not at all. He suggested having the compiler to catch the 
mistakes, not the runtime.

I'm all in for that, if we can find ways which don't break too 
much valid code. I'm not sure that is possible though.

 Just remove the notion of normative "strongly pure", it is a 
 big mistake to split a concept based on the input typing. 
 Confusing two concepts like this in one construct is making the 
 language more complex than keeping the concepts separate. It is 
 ugly. Keep it clean, keep semantics of each concept simple.

I'm assuming you mean that the language would allow caching 
values even with arguments with mutable indirection, if the 
compiler can prove they are similar to an earlier call. I agree 
but that will do nothing to solve the issue (repeated calls to 
functions being wrongly skipped because accidently misusing 
`pure`). Rather the problem will be magnified.

Well, I almost agree. The problem is that [`free` would 
break](https://issues.dlang.org/show_bug.cgi?id=22277).

Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:

On Saturday, 13 November 2021 at 18:55:11 UTC, Dukc wrote:
 Not at all. He suggested having the compiler to catch the 
 mistakes, not the runtime.

He argued against using identity and relying fully on content, 
that is crazy expensive in common situations/algorithms.

This is not practical in a language where you expect to use 
library abstractions. As such, you end up not being able to use 
"pure" for anything that is non-trivial.

 I'm assuming you mean that the language would allow caching 
 values even with arguments with mutable indirection, if the 
 compiler can prove they are similar to an earlier call.

If we want the programmer to use "proper purity" for caching 
values in libraries then it should be a separate construct that 
is enforcing norms on the program that are stronger than the 
regular "weakly pure". It should have a different syntax and very 
clear normative standards so people understand the difference. 
Meaning dealing rigorously with termination, nontrivial 
destructors, exceptions, allocations etc etc. If not able to deal 
with it rigorously: ban those constructs. Compiler type system 
should err on the side of correctness.

If you want the compiler to cache values, then it should just be 
based on static analysis of weakly pure. Compiler optimisation 
should err on the side of correctness.

But saying that "weakly pure" syntax with immutable input is 
providing a different set of semantics that somehow evolve over 
time on some sort of slippery slope is likely to lead up to a 
mess.

Just stick to the keyword ```pure``` meaning "no globals", and 
leave it at that. Want more? Introduce a new well defined concept.

Dukc <ajieskola gmail.com> writes:

On Saturday, 13 November 2021 at 21:26:46 UTC, Ola Fosheim 
Grøstad wrote:
 On Saturday, 13 November 2021 at 18:55:11 UTC, Dukc wrote:
 Not at all. He suggested having the compiler to catch the 
 mistakes, not the runtime.

 He argued against using identity and relying fully on content, 
 that is crazy expensive in common situations/algorithms.

 This is not practical in a language where you expect to use 
 library abstractions. As such, you end up not being able to use 
 "pure" for anything that is non-trivial.

In ` safe` code with only the language constructs, that is. You 
could still write sound ` trusted pure` library functions that 
can do those in efficient way.

 I'm assuming you mean that the language would allow caching 
 values even with arguments with mutable indirection, if the 
 compiler can prove they are similar to an earlier call.

 Just stick to the keyword ```pure``` meaning "no globals", and 
 leave it at that. Want more? Introduce a new well defined 
 concept.

My mistake, you were proposing the exact opposite of what I 
thought. That's the surest way to go but I hope it won't be 
necessary. It'd leave `pure` a bit lame.

Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:

On Sunday, 14 November 2021 at 10:32:14 UTC, Dukc wrote:
 In ` safe` code with only the language constructs, that is. You 
 could still write sound ` trusted pure` library functions that 
 can do those in efficient way.

Yes, but this is where things are starting to get hazy. Can we 
convince ourselves that various combinations of ``` trusted 
pure``` ensure that object identity does not become a dependency? 
Do we know whether it is easy to ensure this when writing custom 
library types? I don't know…

Maybe there is a methodology (design technique) that can be made 
explicit, but this  is where mere arguments don't convince. We 
have to set up the rules that has to be followed and analyse them.

I think explicit rules for programmers to be followed are needed, 
in general, for a system level language that both wants to have a 
higher level type system, and also want to provide "assumption 
mechanisms" like ``` trusted```.

Of course, system level programming should also give the 
programmer freedom to customize the runtime, like being able to 
restart threads on failure etc. So there might be good reasons to 
bend the rules of "pure". That is an extra layer of trickiness 
that I think often is forgotten in D discusssions. Often 
arguments are made as if D is a proper higher level language 
(more abstract).

Imperatorn <johan_forsberg_86 hotmail.com> writes:

On Saturday, 13 November 2021 at 06:50:48 UTC, Paul Backus wrote:
 In the recent [discussion about reference counting][1], a 
 question that's repeatedly come up is, how can we store a 
 pointer to immutable data in immutable memory?

 [...]

It might be crazy 😅, but would be nice if some variant of this is 
made to work, because it's a very simple solution

Dukc <ajieskola gmail.com> writes:

On Saturday, 13 November 2021 at 06:50:48 UTC, Paul Backus wrote:
 What do you think? Is it just crazy enough to work, or just 
 plain crazy? Is there some fatal safety violation I've 
 overlooked? Let me know in your replies!

Unless I miss something else, this is otherwise a good idea, BUT: 
The `ptr()` function _must not be pure_.

Why? If you're casting an integer to a pointer, you're 
essentially opening yourself access to global state. The user 
code can still dereference `TailUnqual` in impure code and fetch 
the address again for a normal pointer to use in `pure` code.

Smaller nitpic, I'd also prefer `opUnary(string op)()(if op == 
"*")` to `alias this`.

Elronnd <elronnd elronnd.net> writes:

I think ultimately, the main problem is that d hasn't 
strictly-defined pointer provenance semantics.  So we end up with 
these ad-hoc rules like 'a pure function's result may be 
immutable'.  The C memory model was a great accomplishment on the 
part of boehm, and it should probably not be emulated; 
nevertheless, deviating from it has annoying implications for 
code generation from gdc and ldc.