www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What is the current state of scope and member functions?

reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
It is my understanding that with the changes for DIP 1000, scope now applies
to member functions, in which case, it marks the implicit this parameter as
scope. And I get deprecation messages in some of my code complaining about
stuff like popFront on a range type not being scope, which would seem to
indicate that scope does indeed now apply to member functions. However, I
don't seem to be getting any inference for it. For instance, this very
simple example does not compile:

void main()
{
    import std.traits;
    static assert(functionAttributes!(Range!int.popFront) &
                  FunctionAttribute.scope_);
}

struct Range(T)
{
    void popFront()
    {
    }
}

The member function is templated (because the type is templated), and it
does literally nothing, so I would expect scope to be inferred, but it
clearly isn't. To make matters even weirder, if I explicitly put scope on
popFront, it _still_ fails to compile. And printing out the function's type
gives

pure nothrow  nogc  safe void()

in both cases, so this doesn't seem to be a problem with std.traits or my
usage of it. Rather, it seems to indicate that not only is scope not being
inferred for popFront, but it's being outright ignored when it's used
explicitly. This would imply that scope is _not_ supposed to be used on
member functions.

And whether I use -preview=dip1000 seems to have no effect on this
particular example. scope is not inferred if it isn't explicitly there, and
it's ignored if it is explicitly there.

So, am I misunderstanding something about how scope is supposed to work, and
it's not supposed be used on member functions? And if that's the case, why
am I getting deprecation messages about member functions not being scope
within an  safe function? Or is this a compiler bug that needs fixing?

- Jonathan M Davis
Sep 10 2023
parent reply Dennis <dkorpel gmail.com> writes:
On Sunday, 10 September 2023 at 21:23:06 UTC, Jonathan M Davis 
wrote:

 in both cases, so this doesn't seem to be a problem with 
 std.traits or my usage of it. Rather, it seems to indicate that 
 not only is scope not being inferred for popFront, but it's 
 being outright ignored when it's used explicitly.


`scope` on a variable is implicitly removed when the variable's 
type has no pointers. This includes the `this` parameter, and 
your example struct has 0 members, so 0 pointers.

I removed this behavior a few months ago for regular parameters 
because checking if the type had pointers caused forward 
reference errors: https://github.com/dlang/dmd/pull/14561

The `this` parameter doesn't cause forward reference errors, but 
perhaps `scope` should also stay there for consistency and 
simplicity.
Sep 10 2023
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, September 10, 2023 4:15:48 PM MDT Dennis via Digitalmars-d wrote:

 On Sunday, 10 September 2023 at 21:23:06 UTC, Jonathan M Davis

 wrote:

 in both cases, so this doesn't seem to be a problem with
 std.traits or my usage of it. Rather, it seems to indicate that
 not only is scope not being inferred for popFront, but it's
 being outright ignored when it's used explicitly.


 `scope` on a variable is implicitly removed when the variable's
 type has no pointers. This includes the `this` parameter, and
 your example struct has 0 members, so 0 pointers.

 I removed this behavior a few months ago for regular parameters
 because checking if the type had pointers caused forward
 reference errors: https://github.com/dlang/dmd/pull/14561

 The `this` parameter doesn't cause forward reference errors, but
 perhaps `scope` should also stay there for consistency and
 simplicity.


Well, I don't know what the overall pros and cons of the current behavior
are, but I found it to be very confusing. AFAIK, no other attributes
magically disappear like that. And having the behavior between the this
parameter and explicit parameters differ is likely to add to the confusion
for folks as well. DIP 1000 is already arguably overly complicated for what
it buys us, and I'd think that removing unnecessary complexity with regards
to what's going on with scope would be a good thing.

- Jonathan M Davis
Sep 10 2023
parent reply Dennis <dkorpel gmail.com> writes:
On Monday, 11 September 2023 at 00:12:29 UTC, Jonathan M Davis 
wrote:

 Well, I don't know what the overall pros and cons of the 
 current behavior are, but I found it to be very confusing.


In a templated struct, you can annotate member functions scope 
without it cluttering up the function type when the struct is 
instantiated without pointers.


 AFAIK, no other attributes magically disappear like that.


__gshared and shared disappear on immutable variables.


 DIP 1000 is already arguably overly complicated for what it 
 buys us, and I'd think that removing unnecessary complexity 
 with regards to what's going on with scope would be a good 
 thing.


We could try remove it.
Sep 12 2023
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, September 12, 2023 3:26:02 AM MDT Dennis via Digitalmars-d wrote:

 On Monday, 11 September 2023 at 00:12:29 UTC, Jonathan M Davis

 AFAIK, no other attributes magically disappear like that.


 __gshared and shared disappear on immutable variables.


That's only because they're redundant given that immutable is implicitly
shared. It would be like how const disappears on a template parameter when
the type itself is already const - e.g. const(T) being const(int) when T is
const(int). So, they don't really disappear, and what's happening is very
obvious based on the types. This is in stark contrast to scope just outright
disappearing even though it's explicitly used just because the compiler
decided that there were no indirections. There is logic to it disappearing
when there are no indirections, since it doesn't really do anything, but
it's not at all intuitively obvious that that's how things would work,
whereas with double consts or shared immutable, the redundant attribute
basically has to go away, so it's far less surprising or confusing.


 DIP 1000 is already arguably overly complicated for what it
 buys us, and I'd think that removing unnecessary complexity
 with regards to what's going on with scope would be a good
 thing.


 We could try remove it.


Given the incredible complexity that DIP 1000 has, I really think that
inconsistencies like this should be removed. Honestly, I have a hard time
believing that DIP 1000 is going to go over well in the long run if it stays
anywhere near as complex as it is. IMHO, it's currently well beyond what the
average programmer is going to reasonably be able to understand, and my gut
reaction is that it would be better to just slap  trusted in a bunch of
places to shut the compiler up about scope than it would be to try to
actually make it work - though unfortunately, that's not going to work very
well with templated code (particularly in libraries), which is precisely
where I'm seeing the compiler complain about scope even though I'm not doing
stuff like taking the address of local variables.

So, anything that can be done to reduce the complexity (particularly
unnecessary complexity) and reduce the number of special cases is likely to
be a good thing. And having parameters and this parameters behave
differently with regards to scope seems unnecessarily confusing.

- Jonathan M Davis
Sep 12 2023
parent reply Dennis <dkorpel gmail.com> writes:
On Tuesday, 12 September 2023 at 09:52:58 UTC, Jonathan M Davis 
wrote:

 and my gut reaction is that it would be better to just slap 
  trusted in a bunch of places to shut the compiler up about 
 scope than it would be to try to actually make it work - though 
 unfortunately, that's not going to work very well with 
 templated code (particularly in libraries), which is precisely 
 where I'm seeing the compiler complain about scope even though 
 I'm not doing stuff like taking the address of local variables.


Please file a bug when you get lifetime errors while not taking 
the address of a local. Note that slicing a local static array is 
taking the address, and it being allowed in  safe code without 
dip1000 is a long standing accepts-invalid bug. dip1000 is 
strictly allowing more code to compile, and all the breakage 
comes from the accepts-invalid bug. Robert's DConf '23 proposal 
to simplify by disallowing  safe code slicing local static arrays 
altogether  strictly breaks more code.

As for removing the scope-stripping of member functions, I 
remember I tried that, but failed because it would break code in 
an unexpected way (forward references again):

https://github.com/dlang/dmd/pull/14232#issuecomment-1162906573
Sep 12 2023
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, September 12, 2023 4:48:13 AM MDT Dennis via Digitalmars-d wrote:

 On Tuesday, 12 September 2023 at 09:52:58 UTC, Jonathan M Davis

 wrote:

 and my gut reaction is that it would be better to just slap
  trusted in a bunch of places to shut the compiler up about
 scope than it would be to try to actually make it work - though
 unfortunately, that's not going to work very well with
 templated code (particularly in libraries), which is precisely
 where I'm seeing the compiler complain about scope even though
 I'm not doing stuff like taking the address of local variables.


 Please file a bug when you get lifetime errors while not taking
 the address of a local. Note that slicing a local static array is
 taking the address, and it being allowed in  safe code without
 dip1000 is a long standing accepts-invalid bug. dip1000 is
 strictly allowing more code to compile, and all the breakage
 comes from the accepts-invalid bug. Robert's DConf '23 proposal
 to simplify by disallowing  safe code slicing local static arrays
 altogether  strictly breaks more code.


IMHO, the big mistake was making static arrays implicitly convert to dynamic
arrays. Being able to slice them to get dynamic arrays is fine, but
automatically slicing them is a lot like automatically taking the address of
a local variable to get an implicit conversion to a pointer. The need for
scope would go down considerably if such conversions weren't happening in an
essentially invisible manner, and the programmer was required to explicitly
slice a static array to get a dynamic array just like they're required to
explicitly take the address of a local variable to get a pointer.
Unfortunately, Walter didn't agree with that idea (probably in large part
because it would have required deprecating existing behavior), so instead of
fixing it and reducing the need for something like scope, we got DIP 1000
instead.

In general, I'm not in favor of being as extreme as Robert was proposing
with regards to removing stuff from  safe, but DIP 1000 seems like a pretty
extreme set of changes to solve a fairly small problem that most D
programmers who aren't constantly interacting with C code should only rarely
need to worry about. Simply getting rid of the implicit conversion from
static arrays to dynamic arrays would go a _long_ way IMHO.

As for reporting stuff, I will if I can, but even figuring out what's
supposed to be happening - particularly in highly templated code - can be
difficult if you're not really well-versed in exactly how scope is supposed
to work right now. scope seems to have become very complicated in the search
to make more and more code work with it, thus reducing the need for
 trusted, instead of just requiring that the programmer use  trusted when
taking the address of a local (or slicing a static array) and then letting
them make sure that they're doing it right. I can appreciate the sentiment
and goal behind scope, but the more I deal with it, the more it seems like
it's a nuke trying to take out an ant. At the moment, I'm inclined to
believe that it adds _way_ too much complication to the language for minimal
benefit.

Quite possibly the biggest design smell with regards to scope is return
scope and how the order and adjacency of the attributes changes the
semantics. Almost no one is going to remember that kind of thing. But scope
has become ridiculously complicated in general. And honestly, as things
stand, if DIP 1000 were enabled by default, I would probably stop using
 safe entirely just to save myself the headaches that scope introduces - or
at least, I'd stop using it whenever the compiler complained about scope.


 As for removing the scope-stripping of member functions, I
 remember I tried that, but failed because it would break code in
 an unexpected way (forward references again):

 https://github.com/dlang/dmd/pull/14232#issuecomment-1162906573


Well, I certainly don't know what is going to be possible on the
implementation side of things, but every inconsistency that's added to
scope's design is going to make it harder to understand. And if it's too
hard to understand, most people will just avoid it rather than benefiting
from the feature. So, if inconsistencise can reasonably be removed, they
should be.

- Jonathan M Davis
Sep 12 2023