www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP1000: Scoped Pointers (Discussion)

reply Dicebot <public dicebot.lv> writes:
http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

The first DIP has just landed into the new queue. It is a 
proposal from language authors and thus it bypasses usual 
nitpicking process and proceeds straight to requesting community 
(your!) feedback.

Essentially, it is an attempt to solve reference lifetime problem 
by extending implementation of `scope` keyword.

Proposal text: 
https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md

Few notes:

- Please submit pull requests to adjust the markdown document if 
you want to propose any improvements (mentioning  WalterBright 
and  andralex for confirmation).
- The proposal refers to a number of other documents and it is 
recommended to become familiar at least briefly with all of them.
- At this point the question I'd personally suggest to be 
evaluated is "does this proposal enable enough useful designs?". 
A good check would be to try taking some of your projects and see 
if having DIP1000 approved and implemented could improve them.
Aug 10 2016
next sibling parent Enamex <enamex+d outlook.com> writes:
On Wednesday, 10 August 2016 at 20:36:38 UTC, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

 - Please submit pull requests to adjust the markdown document 
 if you want to propose any improvements (mentioning 
  WalterBright and  andralex for confirmation).


Personally I wouldn't recommend directly submitting pull requests 
for improvement suggestions. Keep those in the forum/git 
discussions (maybe open an RFC issue on GitHub?) with the pull 
requests only after enough people have commented on the 
suggestions.

(I just don't want to have to follow several PR threads for every 
single improvement suggestion to every DIP in its final comment 
period.)
Aug 10 2016
prev sibling next sibling parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Wednesday, 10 August 2016 at 20:36:38 UTC, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

 The first DIP has just landed into the new queue. It is a 
 proposal from language authors and thus it bypasses usual 
 nitpicking process and proceeds straight to requesting 
 community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime 
 problem by extending implementation of `scope` keyword.

 Proposal text: 
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md

 Few notes:

 - Please submit pull requests to adjust the markdown document 
 if you want to propose any improvements (mentioning 
  WalterBright and  andralex for confirmation).
 - The proposal refers to a number of other documents and it is 
 recommended to become familiar at least briefly with all of 
 them.
 - At this point the question I'd personally suggest to be 
 evaluated is "does this proposal enable enough useful 
 designs?". A good check would be to try taking some of your 
 projects and see if having DIP1000 approved and implemented 
 could improve them.


 Walter

What's current support in the master DMD branch for this DIP and 
what remains to be done? I know that some support was added in 
between the various  safe-ty fixes during July and August:
https://github.com/dlang/dmd/pulls?q=is%3Apr+author%3AWalterBright+created%3A"2016-06-07+..+2016-07-25
+sort%3Acreated-asc but I can't tell what's the overall progress.

BTW, great work fixing all those  safe-ty holes! The amount bugs 
you've managed to fix during this period is amazing!
Aug 11 2016
next sibling parent ZombineDev <petar.p.kirov gmail.com> writes:
On Thursday, 11 August 2016 at 07:13:34 UTC, ZombineDev wrote:

 On Wednesday, 10 August 2016 at 20:36:38 UTC, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

 The first DIP has just landed into the new queue. It is a 
 proposal from language authors and thus it bypasses usual 
 nitpicking process and proceeds straight to requesting 
 community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime 
 problem by extending implementation of `scope` keyword.

 Proposal text: 
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md

 Few notes:

 - Please submit pull requests to adjust the markdown document 
 if you want to propose any improvements (mentioning 
  WalterBright and  andralex for confirmation).
 - The proposal refers to a number of other documents and it is 
 recommended to become familiar at least briefly with all of 
 them.
 - At this point the question I'd personally suggest to be 
 evaluated is "does this proposal enable enough useful 
 designs?". A good check would be to try taking some of your 
 projects and see if having DIP1000 approved and implemented 
 could improve them.


  Walter

 What's current support in the master DMD branch for this DIP 
 and what remains to be done? I know that some support was added 
 in between the various  safe-ty fixes during July and August:
 https://github.com/dlang/dmd/pulls?q=is%3Apr+author%3AWalterBright+created%3A"2016-06-07+..+2016-07-25
+sort%3Acreated-asc but I can't tell what's the overall progress.

 BTW, great work fixing all those  safe-ty holes! The amount 
 bugs you've managed to fix during this period is amazing!


For some strange reason, the link doesn't work in the web 
interface. Here is it again: https://git.io/v64sj
Aug 11 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/11/2016 12:13 AM, ZombineDev wrote:

 What's current support in the master DMD branch for this DIP and what remains
to
 be done?


The current support has been stalled for 17 days now.

https://github.com/dlang/dmd/pull/5972



 I know that some support was added in between the various  safe-ty
 fixes during July and August:
 https://github.com/dlang/dmd/pulls?q=is%3Apr+author%3AWalterBright+created%3A"2016-06-07+..+2016-07-25"+sort%3Acreated-asc
 but I can't tell what's the overall progress.


Those are more reasonably considered bug fixes rather than 'return scope' which 
is the key feature of the DIP.



 BTW, great work fixing all those  safe-ty holes! The amount bugs you've managed
 to fix during this period is amazing!


Actually, it's been quite fun and satisfying. I want to finish it!
Aug 11 2016
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 10/08/16 22:36, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

 The first DIP has just landed into the new queue. It is a proposal from
 language authors and thus it bypasses usual nitpicking process and
 proceeds straight to requesting community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime problem by
 extending implementation of `scope` keyword.

 Proposal text: https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


DIP 1_000? Is this the one DIP to rule them all, making all the 900 DIPs 
or so you skipped over not necessary :)

-- 
/Jacob Carlborg
Aug 11 2016
next sibling parent ZombineDev <petar.p.kirov gmail.com> writes:
On Thursday, 11 August 2016 at 08:07:08 UTC, Jacob Carlborg wrote:

 On 10/08/16 22:36, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

 The first DIP has just landed into the new queue. It is a 
 proposal from
 language authors and thus it bypasses usual nitpicking process 
 and
 proceeds straight to requesting community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime 
 problem by
 extending implementation of `scope` keyword.

 Proposal text: 
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


 DIP 1_000? Is this the one DIP to rule them all, making all the 
 900 DIPs or so you skipped over not necessary :)


Probably :D, though it was the first available (See 
https://github.com/dlang/DIPs/blame/master/README.md#L27 for more 
info). Maybe one of the other two DIPs should have taken 1000, 
but I think it's nice coincidence, considering the implications 
of DIP1000 ;)
Aug 11 2016
prev sibling parent Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: DIP1000: Scoped Pointers (Discussion)
References: <rwxcfapvpfiqmfsuixlf forum.dlang.org>
 <nohbnd$tvc$1 digitalmars.com>
In-Reply-To: <nohbnd$tvc$1 digitalmars.com>

--U6llWv2gSsUPXuoVm8OBqrQxCNXj8Nrnv
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/11/2016 11:07 AM, Jacob Carlborg wrote:

 On 10/08/16 22:36, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

 The first DIP has just landed into the new queue. It is a proposal fro=




m

 language authors and thus it bypasses usual nitpicking process and
 proceeds straight to requesting community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime problem by
 extending implementation of `scope` keyword.

 Proposal text: https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.=




md

=20
 DIP 1_000? Is this the one DIP to rule them all, making all the 900 DIP=


s

 or so you skipped over not necessary :)


I have decided to start counting new DIPs from 1000 so that it is
immediately clear when someone casually mentions "DIP N" if the person
refers to old or new queue.


--U6llWv2gSsUPXuoVm8OBqrQxCNXj8Nrnv--
Aug 11 2016
prev sibling next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Wednesday, 10 August 2016 at 20:36:38 UTC, Dicebot wrote:

 Proposal text: 
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


Can someone talk me through the lifetime algebra for the 
following?

void foo()
{
     int a;
     int** c;
     void bar()
     {
         int* b = &a;
         c = &b;
     }
     bar();
     *c; //undefined behaviour
}

I think it's this:

lifetime(a) = all of foo
lifetime(c) = infinite (because null) //initially
lifetime(b) = lifetime(&a) = lifetime(a) = all of foo // ????
lifetime(c) = lifetime(&b) = lifetime(b) = all of foo //on 
assignment in bar

That would suggest that dereferencing c was OK, because the 
lifetime is all of foo. What am I missing?
Aug 11 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/11/2016 1:29 AM, John Colvin wrote:

 Can someone talk me through the lifetime algebra for the following?

 void foo()
 {
     int a;
     int** c;
     void bar()
     {
         int* b = &a;  <= ok, b has a smaller lifetime than a
         c = &b;    <= error, c has a larger lifetime than b
     }
     bar();
     *c; //undefined behaviour
 }
Aug 11 2016
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Thursday, 11 August 2016 at 08:45:38 UTC, Walter Bright wrote:

 On 8/11/2016 1:29 AM, John Colvin wrote:

 Can someone talk me through the lifetime algebra for the 
 following?

 void foo()
 {
     int a;
     int** c;
     void bar()
     {
         int* b = &a;  <= ok, b has a smaller lifetime than a
         c = &b;    <= error, c has a larger lifetime than b
     }
     bar();
     *c; //undefined behaviour
 }




but according to this rule:

"For an unrestricted pointer, visibility is dictated by the usual 
lexical scope rules. Lifetime, however is dictated by the 
lifetime of the data to which the pointer points to."

b should have the same lifetime as a, no?
Aug 11 2016
next sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Thursday, 11 August 2016 at 08:53:22 UTC, John Colvin wrote:

 On Thursday, 11 August 2016 at 08:45:38 UTC, Walter Bright 
 wrote:

 On 8/11/2016 1:29 AM, John Colvin wrote:

 Can someone talk me through the lifetime algebra for the 
 following?

 void foo()
 {
     int a;
     int** c;
     void bar()
     {
         int* b = &a;  <= ok, b has a smaller lifetime than a
         c = &b;    <= error, c has a larger lifetime than b
     }
     bar();
     *c; //undefined behaviour
 }




 but according to this rule:

 "For an unrestricted pointer, visibility is dictated by the 
 usual lexical scope rules. Lifetime, however is dictated by the 
 lifetime of the data to which the pointer points to."

 b should have the same lifetime as a, no?


ping on this? I want to understand what's going on here and be 
another set of eyes checking for mistakes, but I'm stuck on this 
bit of the spec.
Aug 12 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/11/2016 1:53 AM, John Colvin wrote:

 On Thursday, 11 August 2016 at 08:45:38 UTC, Walter Bright wrote:

 On 8/11/2016 1:29 AM, John Colvin wrote:

 Can someone talk me through the lifetime algebra for the following?

 void foo()
 {
     int a;
     int** c;
     void bar()
     {
         int* b = &a;  <= ok, b has a smaller lifetime than a
         c = &b;    <= error, c has a larger lifetime than b
     }
     bar();
     *c; //undefined behaviour
 }




 but according to this rule:

 "For an unrestricted pointer, visibility is dictated by the usual lexical scope
 rules. Lifetime, however is dictated by the lifetime of the data to which the
 pointer points to."

 b should have the same lifetime as a, no?


When b is assigned the address of a, then the contents of b, as accessible 
through b, have the lifetime of b.
Aug 12 2016
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 August 2016 at 08:29:37 UTC, John Colvin wrote:

 On Wednesday, 10 August 2016 at 20:36:38 UTC, Dicebot wrote:

 Proposal text: 
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


 Can someone talk me through the lifetime algebra for the 
 following?

 void foo()
 {
     int a;
     int** c;
     void bar()
     {
         int* b = &a;
         c = &b;
     }
     bar();
     *c; //undefined behaviour
 }

 I think it's this:

 lifetime(a) = all of foo
 lifetime(c) = infinite (because null) //initially
 lifetime(b) = lifetime(&a) = lifetime(a) = all of foo // ????
 lifetime(c) = lifetime(&b) = lifetime(b) = all of foo //on 
 assignment in bar

 That would suggest that dereferencing c was OK, because the 
 lifetime is all of foo. What am I missing?


You are not missing anything. This DIP is flawed because it only 
handle on indirection level and break down for anything more than 
this.
Aug 11 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/11/2016 3:00 AM, deadalnix wrote:

 This DIP is flawed because it only handle on
 indirection level and break down for anything more than this.


It actually does two levels - the address of the pointer variable ('ref') and 
the contents of the pointer variable ('scope'). But you are essentially
correct. 
This is not a general annotation system for any pointer in a non-trivial graph 
of pointers. There is no notion of 'borrowing'.

The question, however, is whether such a system is needed. The intent with this 
DIP is for people constructing such graphs to make them safe by using reference 
counted containers for the nodes rather than networks of annotated raw pointers.

This scheme is good enough to prevent the escape of addresses on the stack, and 
for containers to prevent the escape of pointers to their contents.
Aug 11 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11.08.2016 12:13, Walter Bright wrote:

 On 8/11/2016 3:00 AM, deadalnix wrote:

 This DIP is flawed because it only handle on
 indirection level and break down for anything more than this.


 It actually does two levels - the address of the pointer variable
 ('ref') and the contents of the pointer variable ('scope'). But you are
 essentially correct. This is not a general annotation system for any
 pointer in a non-trivial graph of pointers. There is no notion of
 'borrowing'.

 The question, however, is whether such a system is needed. The intent
 with this DIP is for people constructing such graphs to make them safe
 by using reference counted containers for the nodes rather than networks
 of annotated raw pointers.

 This scheme is good enough to prevent the escape of addresses on the
 stack, and for containers to prevent the escape of pointers to their
 contents.


Assuming all the soundness issues are fixed, I think the general scheme 
it is not expressive enough. E.g. it does not allow lazy range 
algorithms from std.algorithm and std.range to be used on a slice of 
stack-allocated data. The selling points of a language are not real 
selling points if they are mutually exclusive.
Aug 12 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/12/2016 12:45 PM, Timon Gehr wrote:

 Assuming all the soundness issues are fixed, I think the general scheme it is
 not expressive enough.


Possibly true, but general expressivity comes at a large cost in complexity and 
difficulty for users.



 E.g. it does not allow lazy range algorithms from
 std.algorithm and std.range to be used on a slice of stack-allocated data.


If that does turn out to be a problem, lazy algorithms can detect it (because 
they won't compile) and use an eager version for such cases.
Aug 12 2016
prev sibling parent Kagamin <spam here.lot> writes:
On Thursday, 11 August 2016 at 08:29:37 UTC, John Colvin wrote:

 lifetime(a) = all of foo
 lifetime(c) = infinite (because null) //initially
 lifetime(b) = lifetime(&a) = lifetime(a) = all of foo // ????
 lifetime(c) = lifetime(&b) = lifetime(b) = all of foo //on 
 assignment in bar

 That would suggest that dereferencing c was OK, because the 
 lifetime is all of foo. What am I missing?


I believe the rule should be 
lifetime(&e)=visibility(e)&lifetime(e).
Aug 11 2016
prev sibling next sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
```
void foo() {
     int c;

     ....
     int* e;
     e = &c; // Error, lifetime(e's view) is &infin; and is 
greater than lifetime(c)
}
```

The DIP should make clear that this is wanted for a container 
library.
Additionally, I miss how this DIP fits in the overall plan of 
getting rid of the GC. As long as there isn't a written 
masterplan how to combine those ideas I consider this DIP to be 
incomplete.

I think this change is not worth it. I believe there is an 
inherent trade-off for every programming language between 
usability and theoretically correctness. And IMO this DIP is 
pushing D way to far in the direction of theoretically 
correctness. There are far more important things for the adoption 
of D, like making the frontend a library, shipping with multiple 
backends.
Aug 11 2016
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 11/08/2016 9:06 PM, Robert burner Schadek wrote:

 ```
 void foo() {
     int c;

     ....
     int* e;
     e = &c; // Error, lifetime(e's view) is &infin; and is greater than
 lifetime(c)
 }
 ```

 The DIP should make clear that this is wanted for a container library.
 Additionally, I miss how this DIP fits in the overall plan of getting
 rid of the GC. As long as there isn't a written masterplan how to
 combine those ideas I consider this DIP to be incomplete.

 I think this change is not worth it. I believe there is an inherent
 trade-off for every programming language between usability and
 theoretically correctness. And IMO this DIP is pushing D way to far in
 the direction of theoretically correctness. There are far more important
 things for the adoption of D, like making the frontend a library,
 shipping with multiple backends.


I strongly disagree with this.
If you need proof of where it can be used checkout my managed[0] memory 
concept.

[0] 
https://github.com/rikkimax/alphaPhobos/blob/master/source/std/experimental/memory/managed.d
Aug 11 2016
prev sibling next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Thursday, 11 August 2016 at 09:06:50 UTC, Robert burner 
Schadek wrote:

 The DIP should make clear that this is wanted for a container 
 library.
 Additionally, I miss how this DIP fits in the overall plan of 
 getting rid of the GC. As long as there isn't a written 
 masterplan how to combine those ideas I consider this DIP to be 
 incomplete.


It's already clear that we need much better escape analysis and 
guards for safe reference counting.
There isn't any open question that this is the necessary enabler 
for more RC and less GC.
Aug 11 2016
parent Robert burner Schadek <rburners gmail.com> writes:
On Thursday, 11 August 2016 at 09:45:07 UTC, Martin Nowak wrote:

 It's already clear that we need much better escape analysis and 
 guards for safe reference counting.
 There isn't any open question that this is the necessary 
 enabler for more RC and less GC.


How do you know? Is there a sketch for the planed RC 
implementation (where can I read it?)
Aug 11 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/11/2016 2:06 AM, Robert burner Schadek wrote:

 The DIP should make clear that this is wanted for a container library.
 Additionally, I miss how this DIP fits in the overall plan of getting rid of
the
 GC. As long as there isn't a written masterplan how to combine those ideas I
 consider this DIP to be incomplete.

 I think this change is not worth it. I believe there is an inherent trade-off
 for every programming language between usability and theoretically correctness.
 And IMO this DIP is pushing D way to far in the direction of theoretically
 correctness. There are far more important things for the adoption of D, like
 making the frontend a library, shipping with multiple backends.


Without this, we cannot have reference counting that is memory safe.

10 years ago, you'd be right. But these days, with unending cases of expensive 
exploits using memory safety holes, the world has changed. Programmers are no 
longer going to accept non-safe languages. Businesses are no longer going to 
want to pay for security breaches due to pointer bugs. No responsible 
organization is going to rely on better training to not have security bugs.

D can either get ahead of the curve or be crushed by it.

I've looked for a long time for a scheme that required minimal annotations, not 
wanting to create a 'bondage and discipline' annotation language. This DIP
looks 
like the best we can do. The 'return ref' feature required very few annotations 
to be able to safely compile all of Phobos 'ref' code, and this just extends 
that idea to 'return scope'.

The lack of memory safety is likely what will finally push C into obsolescence. 
C++ will require such extensive retrofit to make it memory safe that that may 
seriously blunt its use in new projects.

D absolutely must have memory safety, and ASAP.
Aug 11 2016
parent reply Robert burner Schadek <rburners gmail.com> writes:
Ok, I disagree, but there is no way that I can proof to be right. 
You can also not proof that you're right, so I'll trust your 
judgement.
Aug 11 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/11/2016 4:40 AM, Robert burner Schadek wrote:

 Ok, I disagree, but there is no way that I can proof to be right. You can also
 not proof that you're right, so I'll trust your judgement.


Thanks for the vote of confidence! I hope to justify it.
Aug 11 2016
prev sibling next sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
Can I do this:

```
struct Foo { int a; }
auto rcs = RefCountedSlice!Foo(); // assume rcs.length > 0

scope Foo zero = rcs[0];
zero.a = 1337;
assert(rcs[0].a == 1337);
```

with the DIP. I could find it.
Aug 11 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/11/2016 4:46 AM, Robert burner Schadek wrote:

 Can I do this:

 ```
 struct Foo { int a; }
 auto rcs = RefCountedSlice!Foo(); // assume rcs.length > 0

 scope Foo zero = rcs[0];
 zero.a = 1337;
 assert(rcs[0].a == 1337);
 ```


No, because the lifetime of zero exceeds the lifetime of rcs[0].
Aug 11 2016
parent reply Robert burner Schadek <rburners gmail.com> writes:
On Thursday, 11 August 2016 at 22:00:06 UTC, Walter Bright wrote:

 On 8/11/2016 4:46 AM, Robert burner Schadek wrote:

 Can I do this:

 ```
 struct Foo { int a; }
 auto rcs = RefCountedSlice!Foo(); // assume rcs.length > 0

 scope Foo zero = rcs[0];
 zero.a = 1337;
 assert(rcs[0].a == 1337);
 ```


 No, because the lifetime of zero exceeds the lifetime of rcs[0].


Hm, that is really bad IMO. The shown use case is properly the 
most common use case. Is there no way to make it transitive so 
the lifetime of rcs[0] is the lifetime of rcs.

To be blunt, this is a real show-stopper for me.
Aug 12 2016
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 12 August 2016 at 09:42:35 UTC, Robert burner Schadek 
wrote:

 On Thursday, 11 August 2016 at 22:00:06 UTC, Walter Bright 
 wrote:

 On 8/11/2016 4:46 AM, Robert burner Schadek wrote:

 Can I do this:

 ```
 struct Foo { int a; }
 auto rcs = RefCountedSlice!Foo(); // assume rcs.length > 0

 scope Foo zero = rcs[0];
 zero.a = 1337;
 assert(rcs[0].a == 1337);
 ```


 No, because the lifetime of zero exceeds the lifetime of 
 rcs[0].


 Hm, that is really bad IMO. The shown use case is properly the 
 most common use case. Is there no way to make it transitive so 
 the lifetime of rcs[0] is the lifetime of rcs.

 To be blunt, this is a real show-stopper for me.


No, the DIP doesn't handle several levels of indirection.
Aug 12 2016
parent reply Robert burner Schadek <rburners gmail.com> writes:
 No, the DIP doesn't handle several levels of indirection.


What about:

struct Bar { int a; int b }
auto rcs = RefCountedTree!(string,Bar)();

fcs["bar"].a = 1337;  // log n
fcs["bar"].b = 1338;  // log n

? I need to pay log n twice to assign two members
Aug 12 2016
next sibling parent Lodovico Giaretta <lodovico giaretart.net> writes:
On Friday, 12 August 2016 at 10:24:22 UTC, Robert burner Schadek 
wrote:

 No, the DIP doesn't handle several levels of indirection.


 What about:

 struct Bar { int a; int b }
 auto rcs = RefCountedTree!(string,Bar)();

 fcs["bar"].a = 1337;  // log n
 fcs["bar"].b = 1338;  // log n

 ? I need to pay log n twice to assign two members


What you are proposing is having the returned value survive not 
till the end of the expression, but longer. The question is how 
long. If, instead of a tree, you use an hashmap, then the 
reference to rcs["bar"] may die long before rcs itself (because 
of a rehash).
So for this to work the compiler should consider the references 
returned by the functions to be valid until a mutable operation 
is applied to the container, i think.
Aug 12 2016
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/12/2016 3:24 AM, Robert burner Schadek wrote:

 ? I need to pay log n twice to assign two members



In order to return a pointer that lasts longer than an expression, make it a
ref 
counted entity as well.
Aug 12 2016
prev sibling next sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 12 Aug 2016 10:24:22 +0000, Robert burner Schadek wrote:


 No, the DIP doesn't handle several levels of indirection.


 
 What about:
 
 struct Bar { int a; int b }
 auto rcs = RefCountedTree!(string,Bar)();
 
 fcs["bar"].a = 1337;  // log n
 fcs["bar"].b = 1338;  // log n
 
 ? I need to pay log n twice to assign two members


In the worst case, you can change the API so you can write:

rcs.update("bar", (x) { x.a = 1337; x.b = 1338; });
Aug 12 2016
parent Robert burner Schadek <rburners gmail.com> writes:
On Friday, 12 August 2016 at 14:02:50 UTC, Chris Wright wrote:

 In the worst case, you can change the API so you can write:

 rcs.update("bar", (x) { x.a = 1337; x.b = 1338; });


yes, but that will not catch on. And I think (x) must be (scope 
ref x)
Aug 12 2016
prev sibling parent Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: DIP1000: Scoped Pointers (Discussion)
References: <rwxcfapvpfiqmfsuixlf forum.dlang.org>
 <urqpbofhcsirsnlktcve forum.dlang.org> <noish3$30cv$1 digitalmars.com>
 <nhchaxmkacxeghfvmjge forum.dlang.org> <psxlflqkpmhbculqppur forum.dlang.org>
 <lvwcqhrtlfxatmynfsfd forum.dlang.org>
In-Reply-To: <lvwcqhrtlfxatmynfsfd forum.dlang.org>

--b5PTQXKeeSjoL96gKvnLiJ0xEFgv75Hij
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/12/2016 01:24 PM, Robert burner Schadek wrote:

 No, the DIP doesn't handle several levels of indirection.


=20
 What about:
=20
 struct Bar { int a; int b }
 auto rcs =3D RefCountedTree!(string,Bar)();
=20
 fcs["bar"].a =3D 1337;  // log n
 fcs["bar"].b =3D 1338;  // log n
=20
 ? I need to pay log n twice to assign two members


Note though that while it won't work with example of `RefCountedSlice`
implementation in DIP (which marks return value as scope reducing
lifetime to expression), it should be possible to instead mark method
itself (=3D=3D hidden `this` argument) as `return scope` thus saying that=

return value has same lifetime as `this`.




--b5PTQXKeeSjoL96gKvnLiJ0xEFgv75Hij--
Aug 18 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/12/2016 2:42 AM, Robert burner Schadek wrote:

 Hm, that is really bad IMO. The shown use case is properly the most common use
 case. Is there no way to make it transitive so the lifetime of rcs[0] is the
 lifetime of rcs.


   rcs[0].a = 1337;
   assert(rcs[0].a == 1337);

It shouldn't be a big problem if the containers are shallow - the optimizer can 
remove the redundant computations.


 To be blunt, this is a real show-stopper for me.


Generally, it will require a more thorough use of encapsulation of containers 
and operations on them, if one is to use malloc/free or ref counting safely.
Aug 12 2016
prev sibling next sibling parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
My comments:

1) `scope` on local variables is unnecessary. Requiring it 
probably makes the implementation easier, but it doesn't need to 
be specified explicitly because it can always be inferred. It's a 
tradeoff: do we want to make it easier for compiler writers to 
implement, or do we want to ease the burden for the end users of 
the feature?

2) The lifetime algebra contains lots of rules saying "only 
applicable to pointer types". That's an indication that some of 
the definitions weren't well chosen. I believe it is a 
consequence of all values having a lifetime, even though it is 
defined to be identical to the visibility for non-reference 
types. Adjusting the definitions will probably simplify things. 
E.g. `lifetime(&e) = visibility(e)` instead of `lifetime(&e) = 
lifetime(e)`

3) LHS and RHS need to be treated differently, at least for the 
ternary operator (min for RHS, max for LHS); consider

int *a;
int b;
int *c;
int d;
c = cond ? &b : &d;    // ok, rejected because c outlives b
(cond ? a : c) = &b;   // not ok, accepted even though a outlives 
b

An equivalent situation can be crafted by using a function with 
return scope and multiple scope parameters. If the return value 
is used on the LHS, the max of all the involved lifetimes has to 
be used.

As a consequence, there must be two lifetimes associated with 
every reference.

4) The DIP doesn't address mutable aliasing at all. As a 
consequence, the example `RefCountedSlice` is unsafe:

auto arr = RefCountedSlice!int(10);
auto ptr = &arr[5];
arr = RefCountedSlice!int(42);
*ptr = 1;    // use after free
Aug 11 2016
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 August 2016 at 13:36:06 UTC, Marc Schütz wrote:

 2) The lifetime algebra contains lots of rules saying "only 
 applicable to pointer types". That's an indication that some of 
 the definitions weren't well chosen.


Yes, the spec should limit itself to types with and without 
indirections.


 3) LHS and RHS need to be treated differently, at least for the 
 ternary operator (min for RHS, max for LHS); consider


I proposed computing revalue and lvalue lifetime differently. 
When reading, you need to make the most concervative lifetime 
compuation, but when writing, you need to make the least 
concervative lifetime computation, in both cases to prevent use 
after free.

The DIP choses instead to limit the number of indirection 
supported, which I don't think is the right tradeof.



 As a consequence, there must be two lifetimes associated with 
 every reference.


Pretty much.


 4) The DIP doesn't address mutable aliasing at all. As a 
 consequence, the example `RefCountedSlice` is unsafe:

 auto arr = RefCountedSlice!int(10);
 auto ptr = &arr[5];
 arr = RefCountedSlice!int(42);
 *ptr = 1;    // use after free


There are essentially 2 aproaches I know of: creating a copy of 
the owner for the lifetime ptr, or disabing the owner (accessing 
arr while ptr is alive is an error).

Both require ownership to work.
Aug 11 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/11/2016 6:36 AM, Marc Schütz wrote:

 1) `scope` on local variables is unnecessary. Requiring it probably makes the
 implementation easier, but it doesn't need to be specified explicitly because
it
 can always be inferred.


Yes, it can be easily inferred from the initializer, and I plan to do it that 
way. The 'return' of 'return scope' can also be frequently inferred like it is 
for 'return ref'.



 2) The lifetime algebra contains lots of rules saying "only applicable to
 pointer types". That's an indication that some of the definitions weren't well
 chosen. I believe it is a consequence of all values having a lifetime, even
 though it is defined to be identical to the visibility for non-reference types.
 Adjusting the definitions will probably simplify things. E.g. `lifetime(&e) =
 visibility(e)` instead of `lifetime(&e) = lifetime(e)`

 3) LHS and RHS need to be treated differently, at least for the ternary
operator
 (min for RHS, max for LHS); consider

 int *a;
 int b;
 int *c;
 int d;
 c = cond ? &b : &d;    // ok, rejected because c outlives b


With conditionals, the scope of it is the narrowest scope of each of its leaves.


 (cond ? a : c) = &b;   // not ok, accepted even though a outlives b


I had overlooked the lvalue case, but you're right. The rules for it are
inevitable.



 An equivalent situation can be crafted by using a function with return scope
and
 multiple scope parameters. If the return value is used on the LHS, the max of
 all the involved lifetimes has to be used.


That's right.



 As a consequence, there must be two lifetimes associated with every reference.




 4) The DIP doesn't address mutable aliasing at all. As a consequence, the
 example `RefCountedSlice` is unsafe:

 auto arr = RefCountedSlice!int(10);
 auto ptr = &arr[5];
 arr = RefCountedSlice!int(42);
 *ptr = 1;    // use after free


The idea is to have containers return references by 'return ref' or 'return 
scope' so the internal references can't escape the expression they're used in.
Aug 11 2016
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 11 August 2016 at 21:57:06 UTC, Walter Bright wrote:

 With conditionals, the scope of it is the narrowest scope of 
 each of its leaves.


 (cond ? a : c) = &b;   // not ok, accepted even though a 
 outlives b


 I had overlooked the lvalue case, but you're right. The rules 
 for it are inevitable.


I raised that in November 2014 already.

To be honest I'm to a point were I'm starting withdrawing from 
these conversation because I have no idea how to reach to you 
guys. I could write on toilet paper before using it to the same 
effect.
Aug 12 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/12/2016 3:12 AM, deadalnix wrote:

 I raised that in November 2014 already.


I remember the discussion.



 To be honest I'm to a point were I'm starting withdrawing from these
 conversation because I have no idea how to reach to you guys. I could write on
 toilet paper before using it to the same effect.


I believe I understood your point. I just didn't agree that was the best way 
forward for D. I didn't expect you'd changed your mind, either :-)
Aug 12 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/12/16 6:12 AM, deadalnix wrote:

 To be honest I'm to a point were I'm starting withdrawing from these
 conversation because I have no idea how to reach to you guys.


You committed to author a proposal a few months back. Did you make 
progress on that? -- Andrei
Aug 12 2016
next sibling parent reply Seb <seb wilzba.ch> writes:
On Friday, 12 August 2016 at 12:29:21 UTC, Andrei Alexandrescu 
wrote:

 On 8/12/16 6:12 AM, deadalnix wrote:

 To be honest I'm to a point were I'm starting withdrawing from 
 these
 conversation because I have no idea how to reach to you guys.


 You committed to author a proposal a few months back. Did you 
 make progress on that? -- Andrei


Are you referring to this one?

https://github.com/dlang/DIPs/pull/16
Aug 12 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/12/16 8:28 AM, Seb wrote:

 On Friday, 12 August 2016 at 12:29:21 UTC, Andrei Alexandrescu wrote:

 On 8/12/16 6:12 AM, deadalnix wrote:

 To be honest I'm to a point were I'm starting withdrawing from these
 conversation because I have no idea how to reach to you guys.


 You committed to author a proposal a few months back. Did you make
 progress on that? -- Andrei


 Are you referring to this one?

 https://github.com/dlang/DIPs/pull/16


That was related to safety and lifetimes. I've created the study group 
specifically to encourage positive contribution from talented folks 
instead of negative reaction. Sadly it hasn't bore fruit yet. It is 
surprising deadalnix still frames the discussion as him proposing things 
and not being heard; my perception is he consistently refuses to 
contribute literally anything in spite of my desperate attempts to get 
him to work on something - anything -, and instead chooses to wait on 
the sidelines and criticize destructively any attempt to make progress. 
-- Andrei
Aug 12 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/12/16 8:28 AM, Seb wrote:

 On Friday, 12 August 2016 at 12:29:21 UTC, Andrei Alexandrescu wrote:

 On 8/12/16 6:12 AM, deadalnix wrote:

 To be honest I'm to a point were I'm starting withdrawing from these
 conversation because I have no idea how to reach to you guys.


 You committed to author a proposal a few months back. Did you make
 progress on that? -- Andrei


 Are you referring to this one?

 https://github.com/dlang/DIPs/pull/16


One of our clocks is skewed, which is it? -- Andrei
Aug 12 2016
parent Seb <seb wilzba.ch> writes:
On Friday, 12 August 2016 at 12:41:07 UTC, Andrei Alexandrescu 
wrote:

 On 8/12/16 8:28 AM, Seb wrote:

 On Friday, 12 August 2016 at 12:29:21 UTC, Andrei Alexandrescu 
 wrote:

 On 8/12/16 6:12 AM, deadalnix wrote:

 To be honest I'm to a point were I'm starting withdrawing 
 from these
 conversation because I have no idea how to reach to you guys.


 You committed to author a proposal a few months back. Did you 
 make
 progress on that? -- Andrei


 Are you referring to this one?

 https://github.com/dlang/DIPs/pull/16


 One of our clocks is skewed, which is it? -- Andrei


Wait - you are not "from the future"? :O

http://i.imgur.com/9pvCtXR.png

(I just checked my phone and it seems to be in sync)
Aug 12 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 12 August 2016 at 12:29:21 UTC, Andrei Alexandrescu 
wrote:

 On 8/12/16 6:12 AM, deadalnix wrote:

 To be honest I'm to a point were I'm starting withdrawing from 
 these
 conversation because I have no idea how to reach to you guys.


 You committed to author a proposal a few months back. Did you 
 make progress on that? -- Andrei


The comment I made in 2014 on this very DIP came with a proposal 
of what to change in it (namely, computing lifetime differently 
for r and l values, as to get the lowest possible lifetime when 
reading and the largest possible one when writing).

You have the detail in your inbox for close to 2 years now.
Aug 13 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 08/13/2016 03:25 AM, deadalnix wrote:

 On Friday, 12 August 2016 at 12:29:21 UTC, Andrei Alexandrescu wrote:

 On 8/12/16 6:12 AM, deadalnix wrote:

 To be honest I'm to a point were I'm starting withdrawing from these
 conversation because I have no idea how to reach to you guys.


 You committed to author a proposal a few months back. Did you make
 progress on that? -- Andrei


 The comment I made in 2014 on this very DIP came with a proposal of what
 to change in it (namely, computing lifetime differently for r and l
 values, as to get the lowest possible lifetime when reading and the
 largest possible one when writing).

 You have the detail in your inbox for close to 2 years now.


Thanks. Do you have a date or title of the thread? -- Andrei
Aug 13 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Saturday, 13 August 2016 at 14:35:09 UTC, Andrei Alexandrescu 
wrote:

 You have the detail in your inbox for close to 2 years now.


 Thanks. Do you have a date or title of the thread? -- Andrei


In the very thread that ended up with DIP1000 .

I suggested various changes int he DIP, which I'm copying bellow. 
You can read it in context digging around the 26/11/2014 .

Copying the proposed changes to DIP 1000:
=========================================

Ok, resurrecting the thread, I'd like to propose another way to 
specify this. The end result should be very close but should 
provide a simpler/clearer mental model.

Every expression has now has a lifetime associated with it, and 
can be marked as "scope". It is only possible to assign b to a if 
b has a lifetime equal or greater than a's.

An infinite lifetime is a lifetime greater or equal than any 
other lifetime. Expression of infinite lifetime are:
  - literals
  - GC heap allocated objects
  - statics and enums.
  - rvalues of type that do not contain indirections.
  - non scope rvalues.

Dereference share the lifetime of the dereferenced expression (ie 
infinite lifetime unless the expression is scope). Address of 
expression shared the lifetime of the base expression, and in 
addition gain the scope flag.

Comment: Using these rule, we basically define any indirection 
being of infinite lifetime by default, and we propagate the 
lifetime when scope. The addition of the scope flag for address 
of is necessary to disallow taking address->dereference to yield 
an infinite lifetime.

Variables delcarations (including parameters) have the lifetime 
of the block they are declared in (2 pitfalls here, I don't have 


these variables shared the lifetime of the variable, unless they 
qualify for infinite lifetime. Parameter's lifetime are 
unordered, meaning smaller than infinite, greater than the 
function's scope, but not equal to each other nor greater/smaller 
than each others.




const scope works for Rust, but isn't gonna fly for D. That is a 
good idea, but not worth the cost.

I'd like to discuss the pro and cons of such a definition 
(compared to rust for instance):
  - We can't specify explicit lifetime for parameters. I'm not 
sure how limiting it is. This is something we can extend toward 
anyway in the future, so I suggest we put that asside for now.
  - We can't define lifetime of indirections explicitly. This 
clearly limit the expressiveness compared to Rust, but I it allow 
for scope to be a storage class rather than a type qualifier, and 
I think this is a winner when looking at the complexity / 
expressiveness ratio.
  - In D, mutability and burrowing are decoupled, which i think is 
a plus compared to Rust. That prevent us to go for things like 
the proposed const scope, but simply the concept of both 
mutabilities and burrowing in D, while allowing some pattern that 
aren't easy to get in Rust (multiple writable reference to one 
object in a single thread for instance, is trivial in D).

Maybe I'm convincing myself, but I think we have a winner here.

Andrei, what do you have in mind when you say scope should be the 
default ? I do think this is a sensible default, but I fail to 
see how this is gonna translate with existing code in a nice 
manner.
Aug 13 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 08/13/2016 12:16 PM, deadalnix wrote:

 I suggested various changes int he DIP, which I'm copying bellow.


That's the spirit. Thanks. -- Andrei
Aug 13 2016
prev sibling next sibling parent Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On Thursday, 11 August 2016 at 21:57:06 UTC, Walter Bright wrote:

 On 8/11/2016 6:36 AM, Marc Schütz wrote:

 4) The DIP doesn't address mutable aliasing at all. As a 
 consequence, the
 example `RefCountedSlice` is unsafe:

 auto arr = RefCountedSlice!int(10);
 auto ptr = &arr[5];
 arr = RefCountedSlice!int(42);
 *ptr = 1;    // use after free


 The idea is to have containers return references by 'return 
 ref' or 'return scope' so the internal references can't escape 
 the expression they're used in.


So &arr[5] would be an error. The bug can still occur when 
passing arr and arr[5] both by reference to a function, but that 
case can (in future) be solved by making RefCountedSlice an  rc 
object*.

* https://wiki.dlang.org/DIP77
Aug 12 2016
prev sibling parent reply Kagamin <spam here.lot> writes:
On Thursday, 11 August 2016 at 21:57:06 UTC, Walter Bright wrote:

 auto arr = RefCountedSlice!int(10);
 auto ptr = &arr[5];
 arr = RefCountedSlice!int(42);
 *ptr = 1;    // use after free


 The idea is to have containers return references by 'return 
 ref' or 'return scope' so the internal references can't escape 
 the expression they're used in.


What expressions are allowed? Will this work?

auto arr = RefCountedSlice!int(10);
void f(scope ref int n)
{
   arr = RefCountedSlice!int(42);
   n = 1; // use after free
}
f(arr[5]);
Aug 12 2016
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 12, 2016 at 01:27:08PM +0000, Kagamin via Digitalmars-d wrote:

 On Thursday, 11 August 2016 at 21:57:06 UTC, Walter Bright wrote:

 auto arr = RefCountedSlice!int(10);
 auto ptr = &arr[5];
 arr = RefCountedSlice!int(42);
 *ptr = 1;    // use after free


 
 The idea is to have containers return references by 'return ref' or
 'return scope' so the internal references can't escape the
 expression they're used in.


 
 What expressions are allowed? Will this work?
 
 auto arr = RefCountedSlice!int(10);
 void f(scope ref int n)
 {
   arr = RefCountedSlice!int(42);
   n = 1; // use after free
 }
 f(arr[5]);


Nice one!

But I believe the current rules ought to catch this case as well,
because if RefCountedSlice!int(42) returns by scope, then assigning it
to arr would be illegal (lifetime of arr is longer than lifetime of the
new slice).


T

-- 
"No, John.  I want formats that are actually useful, rather than over-featured
megaliths that address all questions by piling on ridiculous internal links in
forms which are hideously over-complex." -- Simon St. Laurent on xml-dev
Aug 12 2016
prev sibling next sibling parent Guillaume Piolat <first.last gmail.com> writes:
On Wednesday, 10 August 2016 at 20:36:38 UTC, Dicebot wrote:

 Proposal text: 
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


I trust the leadership that this is an important issue (including 
for perception), but...

Does memory safety deserve such an extensive use of complexity?
 From a (very) naive POV, how come ref/scoped/ safe can't be 
"replaced" wholesale by a type constructor owned!T + lifetimes, 
that this DIP otherwise introduces.

This DIP ensures unsafe references do not escape, but I can't 
remember having such a problem even once, sorry.

Does this allow throwing exceptions in  nogc code? That would be 
great.
Aug 11 2016
prev sibling next sibling parent reply sclytrack <fake hotmail.com> writes:
On Wednesday, 10 August 2016 at 20:36:38 UTC, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

 The first DIP has just landed into the new queue. It is a 
 proposal from language authors and thus it bypasses usual 
 nitpicking process and proceeds straight to requesting 
 community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime 
 problem by extending implementation of `scope` keyword.

 Proposal text: 
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md

 Few notes:

 - Please submit pull requests to adjust the markdown document 
 if you want to propose any improvements (mentioning 
  WalterBright and  andralex for confirmation).
 - The proposal refers to a number of other documents and it is 
 recommended to become familiar at least briefly with all of 
 them.
 - At this point the question I'd personally suggest to be 
 evaluated is "does this proposal enable enough useful 
 designs?". A good check would be to try taking some of your 
 projects and see if having DIP1000 approved and implemented 
 could improve them.



There is confusion between what lifetime and visibility in the 
DIP.
(Well, I am confused)

1) infinite lifetime null

int * p;

The lifetime(p) is infinite while the visibility(p) starts here.
It seems to be the lifetime of p is determined by the value 
assigned to it,
which in this case is null and the lifetime of lifetime(null) is 
infinite.


2) Fundamentals of scope.

This declares 4 scope rules as of 11 August of which the first 
rule is:

"A scope variable can only be initialized and assigned from 
values that have lifetimes longer than the variable's lifetime"


int global_var;
int* global_ptr;

void bar(scope int* input);

void fun1() {
     scope int* a = &global_var; // OK per rule 1, 
lifetime(&global_var) > lifetime(a)
     int b;
     a = &b; // Disallowed per rule 1, lifetime(&b) < lifetime(a)

Shouldn't the rule be:

"A scope variable can only be initialized and assigned from 
values that have lifetimes longer than the variable's VISIBILITY"

As in the case above. What is the lifetime(a) "scope int *a" when 
the &global_var
has not yet been assigned to it? It doesn't have a lifetime yet,
it gets the lifetime from &global_var.


Peter
Aug 11 2016
next sibling parent reply sclytrack <fake hotmail.com> writes:
On Thursday, 11 August 2016 at 21:28:57 UTC, sclytrack wrote:

 On Wednesday, 10 August 2016 at 20:36:38 UTC, Dicebot wrote:

 [...]



 There is confusion between what lifetime and visibility in the 
 DIP.
 (Well, I am confused)

 [...]



What are those puppy dogs doing there? I don't have a gravatar 
account.
Aug 11 2016
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 8/11/16 5:42 PM, sclytrack wrote:

 On Thursday, 11 August 2016 at 21:28:57 UTC, sclytrack wrote:

 On Wednesday, 10 August 2016 at 20:36:38 UTC, Dicebot wrote:

 [...]



 There is confusion between what lifetime and visibility in the DIP.
 (Well, I am confused)

 [...]



 What are those puppy dogs doing there? I don't have a gravatar account.


Apparently fake hotmail.com does. Perhaps that's not your real email?

-Steve
Aug 11 2016
prev sibling parent Yuxuan Shui <yshuiv7 gmail.com> writes:
On Thursday, 11 August 2016 at 21:28:57 UTC, sclytrack wrote:

 On Wednesday, 10 August 2016 at 20:36:38 UTC, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

 The first DIP has just landed into the new queue. It is a 
 proposal from language authors and thus it bypasses usual 
 nitpicking process and proceeds straight to requesting 
 community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime 
 problem by extending implementation of `scope` keyword.

 Proposal text: 
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md

 Few notes:

 - Please submit pull requests to adjust the markdown document 
 if you want to propose any improvements (mentioning 
  WalterBright and  andralex for confirmation).
 - The proposal refers to a number of other documents and it is 
 recommended to become familiar at least briefly with all of 
 them.
 - At this point the question I'd personally suggest to be 
 evaluated is "does this proposal enable enough useful 
 designs?". A good check would be to try taking some of your 
 projects and see if having DIP1000 approved and implemented 
 could improve them.



 There is confusion between what lifetime and visibility in the 
 DIP.
 (Well, I am confused)


I'm a confused by the terminology as well.

lifetime(p) denotes the lifetime of the value which is referred 
to by 'p', is that correct? And visibility(p) denotes the 
lifetime of 'p' itself?

I found this very confusing...
Aug 11 2016
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10.08.2016 22:36, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

 The first DIP has just landed into the new queue. It is a proposal from
 language authors and thus it bypasses usual nitpicking process and
 proceeds straight to requesting community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime problem by
 extending implementation of `scope` keyword.

 Proposal text: https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


This proposes to add a kind of information the type system is supposed 
to track (lifetimes), which cannot be communicated losslessly across 
function and aggregate boundaries. Hence it won't work well in its 
current form.
Aug 12 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/12/2016 12:34 PM, Timon Gehr wrote:

 This proposes to add a kind of information the type system is supposed to track
 (lifetimes), which cannot be communicated losslessly across function and
 aggregate boundaries. Hence it won't work well in its current form.


I believe it does track across function boundaries, because function parameters 
can be annotated.

Aggregate boundaries, no, because annotating fields with 'scope' is not
allowed, 
like 'ref' fields are not allowed.
Aug 12 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12.08.2016 22:22, Walter Bright wrote:

 On 8/12/2016 12:34 PM, Timon Gehr wrote:

 This proposes to add a kind of information the type system is supposed
 to track
 (lifetimes), which cannot be communicated losslessly across function and
 aggregate boundaries. Hence it won't work well in its current form.


 I believe it does track across function boundaries, because function
 parameters can be annotated.
 ...


Yes, to some extent, but what the type system knows and makes use of at 
the call site is more informative than what can be tracked around 
function boundaries. The effect is that there are pieces of code that 
cannot be abstracted into their own functions, which creates friction.


 Aggregate boundaries, no, because annotating fields with 'scope' is not
 allowed, like 'ref' fields are not allowed.


This rules out structs that wrap data and adapt it in some interesting 
way, a very common idiom in (generic) D code.
Aug 12 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/12/2016 2:03 PM, Timon Gehr wrote:

 On 12.08.2016 22:22, Walter Bright wrote:

 I believe it does track across function boundaries, because function
 parameters can be annotated.


 Yes, to some extent, but what the type system knows and makes use of at the
call
 site is more informative than what can be tracked around function boundaries.
 The effect is that there are pieces of code that cannot be abstracted into
their
 own functions, which creates friction.


I don't understand your comment.



 Aggregate boundaries, no, because annotating fields with 'scope' is not
 allowed, like 'ref' fields are not allowed.


 This rules out structs that wrap data and adapt it in some interesting way, a
 very common idiom in (generic) D code.


Consider:

     struct S { int* a, b; }

     int x;
     S s;
     s.a = &x;

should be semantically equivalent, as far as scope rules go, to:

     int x;
     int* a;
     a = &x;

(Note that 'scope' is inferred here.)

What would not work is:

     int x;
     S* ps = new S;
     ps.a = &x;
Aug 12 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 08/12/2016 03:34 PM, Timon Gehr wrote:

 On 10.08.2016 22:36, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

 The first DIP has just landed into the new queue. It is a proposal from
 language authors and thus it bypasses usual nitpicking process and
 proceeds straight to requesting community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime problem by
 extending implementation of `scope` keyword.

 Proposal text: https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


 This proposes to add a kind of information the type system is supposed
 to track (lifetimes), which cannot be communicated losslessly across
 function and aggregate boundaries. Hence it won't work well in its
 current form.


Could it work in a restricted manner, yet permissive enough to be 
useful? -- Andrei
Aug 12 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12.08.2016 22:22, Andrei Alexandrescu wrote:

 On 08/12/2016 03:34 PM, Timon Gehr wrote:

 On 10.08.2016 22:36, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

 The first DIP has just landed into the new queue. It is a proposal from
 language authors and thus it bypasses usual nitpicking process and
 proceeds straight to requesting community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime problem by
 extending implementation of `scope` keyword.

 Proposal text: https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


 This proposes to add a kind of information the type system is supposed
 to track (lifetimes), which cannot be communicated losslessly across
 function and aggregate boundaries. Hence it won't work well in its
 current form.


 Could it work in a restricted manner, yet permissive enough to be
 useful? -- Andrei


The issue is that there is information that the type system tracks that 
cannot be passed around effectively at the type system level.

This generally leads to a painful "fighting-the-compiler" experience 
when trying to use the feature in a not-completely-trivial capacity 
(c.f. inout, it is also a case of this). I think users of the language 
generally expect features to compose. So even if it is useful for the 
handful of special cases it was designed for, there will be a lot of 
complaints.
Aug 12 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 08/12/2016 04:58 PM, Timon Gehr wrote:

 On 12.08.2016 22:22, Andrei Alexandrescu wrote:

 On 08/12/2016 03:34 PM, Timon Gehr wrote:

 On 10.08.2016 22:36, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org

 The first DIP has just landed into the new queue. It is a proposal from
 language authors and thus it bypasses usual nitpicking process and
 proceeds straight to requesting community (your!) feedback.

 Essentially, it is an attempt to solve reference lifetime problem by
 extending implementation of `scope` keyword.

 Proposal text:
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


 This proposes to add a kind of information the type system is supposed
 to track (lifetimes), which cannot be communicated losslessly across
 function and aggregate boundaries. Hence it won't work well in its
 current form.


 Could it work in a restricted manner, yet permissive enough to be
 useful? -- Andrei


 The issue is that there is information that the type system tracks that
 cannot be passed around effectively at the type system level.

 This generally leads to a painful "fighting-the-compiler" experience
 when trying to use the feature in a not-completely-trivial capacity
 (c.f. inout, it is also a case of this). I think users of the language
 generally expect features to compose. So even if it is useful for the
 handful of special cases it was designed for, there will be a lot of
 complaints.


Can you please give examples of cases that are (a) unlikely to be 
supported with reasonable effort, and (b) likely to cause problems in 
usability? Thanks! -- Andrei
Aug 12 2016
parent deadalnix <deadalnix gmail.com> writes:
On Friday, 12 August 2016 at 21:36:20 UTC, Andrei Alexandrescu 
wrote:

 Can you please give examples of cases that are (a) unlikely to 
 be supported with reasonable effort, and (b) likely to cause 
 problems in usability? Thanks! -- Andrei


Anythign that work with a linkedlist or a tree structure.
Aug 12 2016
prev sibling next sibling parent reply Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: DIP1000: Scoped Pointers (Discussion)
References: <rwxcfapvpfiqmfsuixlf forum.dlang.org>
In-Reply-To: <rwxcfapvpfiqmfsuixlf forum.dlang.org>

--UfULrDOCkHQFjCM8dO8ueBpVbJXmkedJW
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/10/2016 11:36 PM, Dicebot wrote:

 http://forum.dlang.org/post/pqsiqmkxenrwxoruzaml forum.dlang.org


I am still uncertain myself about the proposal, mostly because can't
completely grasp rules for returning as scope. My litmus test example is
very simple (for desired semantics):

```
struct Container
{
    int data;

    static struct Range
    {
        int* pdata;
        // range methods skipped for clarity ..
    }

    Range asRange ( )
    {
        return Range(&this.data);
    }
}

void main ( )
{
    Container container;

    import std.stdio;
    writeln(container.asRange()); // case 1, OK

    scope r =3D container.asRange(); // case 2, OK
    auto r =3D container.asRange(); // case 3, not OK
}
```

It looks like
(https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md#scope-function=
-returns)
I can annotate `scope Range asRange ( )` to force result into rvalue and
make case 1 and case 3 work as desired but there is no way to express
lifetime relation between returned struct and host container so that
case 2 will work.

If it is indeed correct, I am going to call it a complete showstopper.
Any scope proposal is of interest to me only if it can be used to
implement some form of borrowship semantics (even awkwardly looking) on
top - everything else is a nice addition to have but in no way justifies
added language weight.

Note that section
https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md#owning-containe=
rs
explain similar example (but with refcounting instead of stack
allocation) but it does not show semantics at call site and is either
subject to same limitation (can't store result of `opIndex`) or
contradicts previous spec.


--UfULrDOCkHQFjCM8dO8ueBpVbJXmkedJW--
Aug 14 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/14/2016 7:37 AM, Dicebot wrote:

 struct Container
 {
     int data;

     static struct Range
     {
         int* pdata;
         // range methods skipped for clarity ..
     }

     Range asRange ( )
     {
         return Range(&this.data);
     }
 }

 void main ( )
 {
     Container container;

     import std.stdio;
     writeln(container.asRange()); // case 1, OK

     scope r = container.asRange(); // case 2, OK
     auto r = container.asRange(); // case 3, not OK
 }



A great example. Let's analyze by peeling back the syntactic sugar:

----
'container' is a stack variable

'container.int' is a stack variable

'asRange()' returns an instance of 'Range' which is a stack variable, call it 
'range'

'range.pdata' is a stack variable

Range range.pdata = &container.data; // range is inferred as 'scope' because
                                      // container.data is a stack variable

auto r = range; // r is inferred as 'scope' because 'range' is 'scope'
----

It's a bit advanced, but it falls entirely in the DIP rules and is workable
with 
some implementation effort (not sure how much).

The idea is, if something is ultimately a stack variable, the scope rules apply.
Aug 14 2016
next sibling parent Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: DIP1000: Scoped Pointers (Discussion)
References: <rwxcfapvpfiqmfsuixlf forum.dlang.org>
 <nopvno$fih$1 digitalmars.com> <noqior$1c5h$1 digitalmars.com>
In-Reply-To: <noqior$1c5h$1 digitalmars.com>

--ja2Ib3uEnQ32D63uWEc46bh7qH3m2pu2s
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/14/2016 11:02 PM, Walter Bright wrote:

 It's a bit advanced, but it falls entirely in the DIP rules and is
 workable with some implementation effort (not sure how much).
=20
 The idea is, if something is ultimately a stack variable, the scope
 rules apply.


That is an unexpected application of described rules - I feel like I
need some time to fit it into my head before I can respond :)



--ja2Ib3uEnQ32D63uWEc46bh7qH3m2pu2s--
Aug 14 2016
prev sibling parent reply Kagamin <spam here.lot> writes:
On Sunday, 14 August 2016 at 20:02:35 UTC, Walter Bright wrote:

 auto r = range;


You said that if a value (range in this case) is returned by 
scope, its lifetime is restricted to its expression, hence it 
can't be assigned to a variable, because the variable outlives 
the expression.
Aug 15 2016
next sibling parent Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: DIP1000: Scoped Pointers (Discussion)
References: <rwxcfapvpfiqmfsuixlf forum.dlang.org>
 <nopvno$fih$1 digitalmars.com> <noqior$1c5h$1 digitalmars.com>
 <aplhgrgjxhigmkapjjdj forum.dlang.org>
In-Reply-To: <aplhgrgjxhigmkapjjdj forum.dlang.org>

--kWroaTELCReNBrTHOgpo360odff1n0eU3
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/15/2016 12:32 PM, Kagamin wrote:

 On Sunday, 14 August 2016 at 20:02:35 UTC, Walter Bright wrote:

 auto r =3D range;


=20
 You said that if a value (range in this case) is returned by scope, its=



 lifetime is restricted to its expression, hence it can't be assigned to=



 a variable, because the variable outlives the expression.


As far as I understood Walter, his explained behaviour should happen if
function is NOT annottated with scope and will be a default out-of-box
one when trying to "borrow" stack-allocated data. Not sure yet how it
should work with more complex cases (i.e. container stores a slice /
pointer internally and not just int).


--kWroaTELCReNBrTHOgpo360odff1n0eU3--
Aug 15 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/15/2016 2:32 AM, Kagamin wrote:

 On Sunday, 14 August 2016 at 20:02:35 UTC, Walter Bright wrote:

 auto r = range;


 You said that if a value (range in this case) is returned by scope, its
lifetime
 is restricted to its expression, hence it can't be assigned to a variable,
 because the variable outlives the expression.


A very good question. If the functions parameters are 'return scope', that
means 
the returned value has the (smallest) scope if its 'return scope' parameters.
If 
there are no 'return scope' parameters, the scope of the return value is
limited 
to the expression.

This is why things like this will work:

   scope int* foo(return scope int* p) { return p; }

   int i;
   int* q = foo(&i); // ok

because the compiler knows that the address of i is what is being returned, and 
q has a shorter lifetime than i.

Also:

   scope int* foo(scope int* p) { return p; }

would be an error, and:

   scope int* foo(int* p) { return p; }
   int* q = foo();  // error
Aug 15 2016
parent reply Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: DIP1000: Scoped Pointers (Discussion)
References: <rwxcfapvpfiqmfsuixlf forum.dlang.org>
 <nopvno$fih$1 digitalmars.com> <noqior$1c5h$1 digitalmars.com>
 <aplhgrgjxhigmkapjjdj forum.dlang.org> <nosad0$vsi$1 digitalmars.com>
In-Reply-To: <nosad0$vsi$1 digitalmars.com>

--i4uVCpXsGtRQh5qo3JxCW3CgRxj3BGuA2
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/15/2016 02:52 PM, Walter Bright wrote:

 On 8/15/2016 2:32 AM, Kagamin wrote:

 On Sunday, 14 August 2016 at 20:02:35 UTC, Walter Bright wrote:

 auto r =3D range;


 You said that if a value (range in this case) is returned by scope,
 its lifetime
 is restricted to its expression, hence it can't be assigned to a
 variable,
 because the variable outlives the expression.


=20
 A very good question. If the functions parameters are 'return scope',
 that means the returned value has the (smallest) scope if its 'return
 scope' parameters. If there are no 'return scope' parameters, the scope=



 of the return value is limited to the expression.


Does that mean that a scope return in aggregate method will bind to
aggregate lifetime if method itself is annotated with scope too? For
example:

struct Container
{
    scope Range asRange ( ) scope
    {
        return Range(&this.data);
    }
}



--i4uVCpXsGtRQh5qo3JxCW3CgRxj3BGuA2--
Aug 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/15/2016 5:17 AM, Dicebot wrote:

 On 08/15/2016 02:52 PM, Walter Bright wrote:

 On 8/15/2016 2:32 AM, Kagamin wrote:

 On Sunday, 14 August 2016 at 20:02:35 UTC, Walter Bright wrote:

 auto r = range;


 You said that if a value (range in this case) is returned by scope,
 its lifetime
 is restricted to its expression, hence it can't be assigned to a
 variable,
 because the variable outlives the expression.


 A very good question. If the functions parameters are 'return scope',
 that means the returned value has the (smallest) scope if its 'return
 scope' parameters. If there are no 'return scope' parameters, the scope
 of the return value is limited to the expression.


 Does that mean that a scope return in aggregate method will bind to
 aggregate lifetime if method itself is annotated with scope too? For
 example:

 struct Container
 {
     scope Range asRange ( ) scope
     {
         return Range(&this.data);
     }
 }


If the method is annotated with scope, the scope applies to the 'this' pointer.
Aug 15 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Mon, 15 Aug 2016 06:36:00 -0700, Walter Bright wrote:

 If the method is annotated with scope, the scope applies to the 'this'
 pointer.


Oh god, the attribute explosion is getting worse.

I think at this point the proper way to use attribute-based D features is 
to write your code without them, then write a tool that will add all the 
attributes to your source code that it can while not breaking anything. 
Like now it's sensible to write a method:

public final void foo() scope inout  nogc nothrow  safe pure {}

I think the solution is to turn every function into a no-args template, 
but then you can't use virtual methods.
Aug 15 2016
next sibling parent Robert burner Schadek <rburners gmail.com> writes:
 public final void foo() scope inout  nogc nothrow  safe pure {}

 I think the solution is to turn every function into a no-args 
 template, but then you can't use virtual methods.


** sarcasm on **

Don't say that, you give them ideas ;-)

** sarcasm off **
Aug 15 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/15/2016 7:48 AM, Chris Wright wrote:

 Oh god, the attribute explosion is getting worse.

 I think at this point the proper way to use attribute-based D features is
 to write your code without them, then write a tool that will add all the
 attributes to your source code that it can while not breaking anything.
 Like now it's sensible to write a method:

 public final void foo() scope inout  nogc nothrow  safe pure {}

 I think the solution is to turn every function into a no-args template,
 but then you can't use virtual methods.



We've made a lot of progress with inferring attributes, and I have some ideas
to 
do that even more.
Aug 15 2016
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Mon, Aug 15, 2016 at 02:43:01PM -0700, Walter Bright via Digitalmars-d wrote:

 On 8/15/2016 7:48 AM, Chris Wright wrote:

 Oh god, the attribute explosion is getting worse.
 
 I think at this point the proper way to use attribute-based D
 features is to write your code without them, then write a tool that
 will add all the attributes to your source code that it can while
 not breaking anything.  Like now it's sensible to write a method:
 
 public final void foo() scope inout  nogc nothrow  safe pure {}
 
 I think the solution is to turn every function into a no-args
 template, but then you can't use virtual methods.


 
 
 We've made a lot of progress with inferring attributes, and I have
 some ideas to do that even more.


I'm looking forward to the day we have pervasive attribute inferrence
throughout the language.  It's the only sane way to deal with
attributes, because they inevitably grow unmanageably numerous. In an
ideal world I'd say *all* attributes should be inferred (and only
specified where the user wishes to ensure the attribute is actually
inferred). But in our non-ideal world it would be interesting to see how
far we can get.


T

-- 
A program should be written to model the concepts of the task it performs
rather than the physical world or a process because this maximizes the
potential for it to be applied to tasks that are conceptually similar and, more
important, to tasks that have not yet been conceived. -- Michael B. Allen
Aug 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/15/2016 4:30 PM, H. S. Teoh via Digitalmars-d wrote:

 I'm looking forward to the day we have pervasive attribute inferrence
 throughout the language.  It's the only sane way to deal with
 attributes, because they inevitably grow unmanageably numerous. In an
 ideal world I'd say *all* attributes should be inferred (and only
 specified where the user wishes to ensure the attribute is actually
 inferred). But in our non-ideal world it would be interesting to see how
 far we can get.


I think we can pretty much get everywhere except for:

1. virtual functions
2. explicit API interfaces
3. separate compilation
4. function pointers
5. recursion

Adding an attribute then becomes a guarantee.

Note how successful this has been for templates.
Aug 15 2016
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Mon, Aug 15, 2016 at 04:46:15PM -0700, Walter Bright via Digitalmars-d wrote:

 On 8/15/2016 4:30 PM, H. S. Teoh via Digitalmars-d wrote:

 I'm looking forward to the day we have pervasive attribute
 inferrence throughout the language.  It's the only sane way to deal
 with attributes, because they inevitably grow unmanageably numerous.
 In an ideal world I'd say *all* attributes should be inferred (and
 only specified where the user wishes to ensure the attribute is
 actually inferred). But in our non-ideal world it would be
 interesting to see how far we can get.


 
 I think we can pretty much get everywhere except for:
 
 1. virtual functions
 2. explicit API interfaces


True.



 3. separate compilation


Under the current state of separate compilation, yes.  But in theory it
*should* be possible, at least in some of the common use cases.



 4. function pointers


But this shouldn't be a problem as long as the function itself is
covariant with the pointer?



 5. recursion


[...]

I think it should be possible even with recursion, at least if there is
a non-recursive branch in the function(s). Of course, it may not
necessarily be *feasible* to implement the required analysis in the
compiler. But maybe for the simplest cases, e.g.:

	auto func(...) {
		if (condition)
			return finalResult;
		else
			return func(...);
	}


T

-- 
If Java had true garbage collection, most programs would delete themselves upon
execution. -- Robert Sewell
Aug 22 2016
next sibling parent reply Meta <jared771 gmail.com> writes:
On Monday, 22 August 2016 at 17:22:28 UTC, H. S. Teoh wrote:

 I think it should be possible even with recursion, at least if 
 there is a non-recursive branch in the function(s). Of course, 
 it may not necessarily be *feasible* to implement the required 
 analysis in the compiler. But maybe for the simplest cases, 
 e.g.:

 	auto func(...) {
 		if (condition)
 			return finalResult;
 		else
 			return func(...);
 	}


 T


I must be missing something here, but isn't it trivial to infer 
attributes for a recursive function? By definition you either 
have a call to the current function, which is idempotent in 
regards to attributes, or some non-recursive work that is done, 
in which case you follow the normal inference algorithm. I know 
this is exactly the basic case that you have shown, but even for 
more complex cases such as mutually-recursive functions I can't 
see this being unmanageable.
Aug 22 2016
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Mon, Aug 22, 2016 at 06:02:20PM +0000, Meta via Digitalmars-d wrote:

 On Monday, 22 August 2016 at 17:22:28 UTC, H. S. Teoh wrote:

 I think it should be possible even with recursion, at least if there
 is a non-recursive branch in the function(s). Of course, it may not
 necessarily be *feasible* to implement the required analysis in the
 compiler. But maybe for the simplest cases, e.g.:
 
 	auto func(...) {
 		if (condition)
 			return finalResult;
 		else
 			return func(...);
 	}
 
 
 T


 
 I must be missing something here, but isn't it trivial to infer
 attributes for a recursive function? By definition you either have a
 call to the current function, which is idempotent in regards to
 attributes, or some non-recursive work that is done, in which case you
 follow the normal inference algorithm. I know this is exactly the
 basic case that you have shown, but even for more complex cases such
 as mutually-recursive functions I can't see this being unmanageable.


Yeah, this can probably be extended to more complex cases. But you may
start running into limitations with the current compiler design once the
recursion involves multiple functions, since AFAIK the compiler
separately compiles each function, so the inference may not be able to
resolve two mutually-recursive functions at the same time.  But maybe
there's a way around it that I haven't thought of.

In any case, making inference work with recursive functions is an
important step in making inroads to maximizing attribute inference
throughout the language.


T

-- 
Lottery: tax on the stupid. -- Slashdotter
Aug 22 2016
prev sibling parent Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: DIP1000: Scoped Pointers (Discussion)
References: <nopvno$fih$1 digitalmars.com> <noqior$1c5h$1 digitalmars.com>
 <aplhgrgjxhigmkapjjdj forum.dlang.org> <nosad0$vsi$1 digitalmars.com>
 <nosbss$124v$1 digitalmars.com> <nosgfv$19kj$1 digitalmars.com>
 <noskni$gg6$5 digitalmars.com> <notd15$13fn$1 digitalmars.com>
 <mailman.1007.1471304253.3131.digitalmars-d puremagic.com>
 <notk86$1den$1 digitalmars.com>
 <mailman.94.1471887004.3111.digitalmars-d puremagic.com>
In-Reply-To: <mailman.94.1471887004.3111.digitalmars-d puremagic.com>

--Is9i65kD2ajDcSJbmqdIID0OhSrk3aSQU
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

On 08/22/2016 08:22 PM, H. S. Teoh via Digitalmars-d wrote:

 3. separate compilation


=20
 Under the current state of separate compilation, yes.  But in theory it=



 *should* be possible, at least in some of the common use cases.


Separate compilation is perfectly fixable if we agree for it to work on
package / static library level and not on module / object file level.
But that may need .di generation improvements and the `export` proposed
changes from Benjamin.


--Is9i65kD2ajDcSJbmqdIID0OhSrk3aSQU--
Aug 23 2016
prev sibling parent reply Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: DIP1000: Scoped Pointers (Discussion)
References: <rwxcfapvpfiqmfsuixlf forum.dlang.org>
In-Reply-To: <rwxcfapvpfiqmfsuixlf forum.dlang.org>

--jB1ULrA2WnQ7tAG2UNIffboBbDl9xLEGf
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Two another more nitpicky comments:

1) Deprecation process proposal right now says this:

- remove -scope, issue warning when errors are detected
- replace warnings with deprecation messages

PLEASE FOR THE SAKE OF ALL DEAR NEVER EVER DO THIS

Deprecations must always come first. Warnings may or may not follow
later but any deprecation process must mandatory start with deprecation
message, not exceptions and no excuses.

2) Ignoring `scope` for variables with no indirections.

The fact that DMD is extremely accepting when it comes to nonsense
declarations/attributes has been is notable contribution to harming
learning curve. It often tricks new developers into thinking that their
code is checked for something when in fact compiler simply silently
skips extra annotations.

Instead, nonsense application of `scope` to data with no indirections
should be deprecated completely as part of overall deprecation process
for implementing the proposal.


--jB1ULrA2WnQ7tAG2UNIffboBbDl9xLEGf--
Aug 14 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/14/2016 7:42 AM, Dicebot wrote:

 2) Ignoring `scope` for variables with no indirections.

 The fact that DMD is extremely accepting when it comes to nonsense
 declarations/attributes has been is notable contribution to harming
 learning curve. It often tricks new developers into thinking that their
 code is checked for something when in fact compiler simply silently
 skips extra annotations.

 Instead, nonsense application of `scope` to data with no indirections
 should be deprecated completely as part of overall deprecation process
 for implementing the proposal.


The difficulty with that is dealing with generic types.
Aug 14 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/14/2016 12:39 PM, Walter Bright wrote:

 On 8/14/2016 7:42 AM, Dicebot wrote:

 2) Ignoring `scope` for variables with no indirections.

 The fact that DMD is extremely accepting when it comes to nonsense
 declarations/attributes has been is notable contribution to harming
 learning curve. It often tricks new developers into thinking that their
 code is checked for something when in fact compiler simply silently
 skips extra annotations.

 Instead, nonsense application of `scope` to data with no indirections
 should be deprecated completely as part of overall deprecation process
 for implementing the proposal.


 The difficulty with that is dealing with generic types.


Consider another case:

   struct S {
     int i;
     int* p;
   }

   scope S s;

   return s.i; // ok
   return s.p; // error!

The 'scope' only applies to the indirection parts of a type. Therefore,

   scope int i;

should not be an error.
Aug 14 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/14/2016 7:42 AM, Dicebot wrote:

 Two another more nitpicky comments:

 1) Deprecation process proposal right now says this:

 - remove -scope, issue warning when errors are detected
 - replace warnings with deprecation messages

 PLEASE FOR THE SAKE OF ALL DEAR NEVER EVER DO THIS

 Deprecations must always come first. Warnings may or may not follow
 later but any deprecation process must mandatory start with deprecation
 message, not exceptions and no excuses.


Hmm, but we've always done warning => deprecation => error
Aug 14 2016
parent reply Dicebot <public dicebot.lv> writes:
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: DIP1000: Scoped Pointers (Discussion)
References: <rwxcfapvpfiqmfsuixlf forum.dlang.org>
 <noq010$frc$1 digitalmars.com> <noqi2g$1au4$1 digitalmars.com>
In-Reply-To: <noqi2g$1au4$1 digitalmars.com>

--VOenWLqoAOK5W94j1UXm1obw55rx1L3kE
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 08/14/2016 10:50 PM, Walter Bright wrote:

 On 8/14/2016 7:42 AM, Dicebot wrote:

 Two another more nitpicky comments:

 1) Deprecation process proposal right now says this:

 - remove -scope, issue warning when errors are detected
 - replace warnings with deprecation messages

 PLEASE FOR THE SAKE OF ALL DEAR NEVER EVER DO THIS

 Deprecations must always come first. Warnings may or may not follow
 later but any deprecation process must mandatory start with deprecatio=




n

 message, not exceptions and no excuses.


=20
 Hmm, but we've always done warning =3D> deprecation =3D> error


Yes, and it was a major disaster because most sensible projects compile
with warnings as errors enabled (it is also the default in dub) and thus
putting a warning breaks a many more times more code than putting a
deprecation. We have already talked that through before with Martin and
it seemed acknowledged that pattern has to be changed to "deprecation ->
(optional warning) -> error".


--VOenWLqoAOK5W94j1UXm1obw55rx1L3kE--
Aug 14 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/14/2016 2:12 PM, Dicebot wrote:

 On 08/14/2016 10:50 PM, Walter Bright wrote:

 Hmm, but we've always done warning => deprecation => error


 Yes, and it was a major disaster because most sensible projects compile
 with warnings as errors enabled (it is also the default in dub) and thus
 putting a warning breaks a many more times more code than putting a
 deprecation. We have already talked that through before with Martin and
 it seemed acknowledged that pattern has to be changed to "deprecation ->
 (optional warning) -> error".


Ok, that makes sense. Thanks for the explanation!
Aug 14 2016