www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Scope of variables

reply bearophile <bearophileHUGS lycos.com> writes:
This post is a follow-up of something I've written almost one year ago, that I
have summarized here:
http://d.puremagic.com/issues/show_bug.cgi?id=5007

Feel free to ignore this post if you are busy, or if you didn't appreciate the
precedent post.


In my C/D programs some common troubles (or even bugs) are caused by the
scoping of variable/constant names: (1) wrongly thinking I am using a local
name while I am using a name from an outer scope, or (2) using an outer scope
name while I wrongly think I'm using a local name.

Outer scopes are struct/class attributes, global variables, or
variabiles/constants defined in the outer function when you are working in an
inner function.

Given the frequency of troubles such mistakes cause me, I don't understand why
newer languages don't try to improve this situation with stricter visibility
rules. (In D the visibility situation is more complex than C because there are
structs/classes and there are inner functions too, so I think just copying C
rules isn't enough. There are even inner classes, despite I have used them only
once in D, to port some Java code to D. I think they are useful mostly to port
Java code to D).

In Python3 the default is the opposite one, and it seems safer (this
explanation about Python is not the whole story, but it's a good enough
approximation for this discussion): in a function if you want to use a variable
name that't not defined locally you have to use the "nonlocal" (from Python2
there is also "global" keyword for similar usage, but it's not good enough if
you want to use inner functions):

# program#1
def foo():
    x = 10
    def bar():
        nonlocal x
        x += 1


Some examples of hiding outer names in D this (is shows just the first half of
the problem):

// program#2
int i;
void foo(int i) {} // hiding global variable
struct Bar {
    int j, w;
    static int k;
    void spam(int j) { // hiding instance variable
        int w; // hiding instance variable
    }
    void baz(int k) { // hiding static variable
        void inner(int k) {} // hiding outer variable
    }
    static void red() {
        int k; // hiding static variable
    }
}
void main() {}


You can't adopt the Python solution in D. But doing the opposite is possible,
the idea is disallowing hiding present in outer scopes (so they generate an
error) unless you mark the variable with something, like a "local":


// program#3
int i;
void foo(local int i) {}
struct Bar {
    int j, w;
    static int k;
    void spam(int local j) {
        local int w;
    }
    void baz(local int k) {
        void inner(local int k) {}
    }
    static void red() {
        local int k;
    }
}
void main() {}


Time ago I have even thought about an  outer attribute, that is optional and
it's more similar to the Python3 "nonlocal". If you add  outer at a function
signature, you have to list all the names from outer scopes you will use inside
the current function (and if you don't use them all it's an error again). In
SPARK there is something similar, but more verbose and it's not optional (I
don't remember if the Splint lint has something similar):


// program#4
int i = 1;
int j = 2;
 outer(out i, in j) void foo(int x) {
  i = x + j;
}
struct Foo {
  int x;
   outer(in x, inout j) void bar() {
    j += x;
  }
}
void main() {}


"local" is more light to use, in syntax too, while  outer() is more heavy but
it's more precise (but to make it work I think "local" can't be optional). Of
the two ideas I like  outer() better.  outer is meant to avoid bugs but I think
it's usable in debugging too, when you find a bug that I think is scope-related
I add some  outer() to make the code more strict and make the bug stand out.

In theory with some more static introspection (like a __traits that given a
function/method name, returns an array of variables used inside it (or used by
functions called by the function), and how they are used, if read, written or
both) it's possible to add an user-defined attribute like  outer with user code.


In bug 5007 Nick Sabalausky has shown a simpler idea:

// program#5
int globalVar;
class Foo() {
    int instanceVar;
    static int classVar;

     explicitLookup // Name subject to change
    void bar() {

        int globalVar;   // Error
        int instanceVar; // Error
        int classVar;    // Error

        globalVar   = 1; // Error
        instanceVar = 1; // Error
        classVar    = 1; // Error

        .globalVar       = 1; // Ok
        this.instanceVar = 1; // Ok
        Foo.classVar     = 1; // Ok
    }
}

Bye,
bearophile
Jun 24 2011
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
    void baz(local int k) {
        void inner(local int k) {}
    }

This is unneeded. By declaring a local variable, it's *obviously* local - there's no point in saying local again! Now, I've forgotten that I had a variable declared before, but adding more stuff to the argument list wouldn't change anything, because if I was looking at the argument list, I would have realized the local variable was there anyway! If D was a bad language that allowed implicit variable definitions, it might make sense to do this, but we already declare all vars so it adds nothing.
 void foo(int i) {} // hiding global variable

This is a good thing because you can reason about it locally. If you declare a variable locally, you know it is going to work. What I've taken to doing is if I definitely want to access a class var, I'll just write this and use the dot to get to a global. That way, it's always clear what's going on without looking back at the function definition. I don't want that to be required though.
Jun 24 2011
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Adam D. Ruppe:

    void baz(local int k) {
        void inner(local int k) {}
    }

This is unneeded. By declaring a local variable, it's *obviously* local - there's no point in saying local again!

 If D was a bad language that allowed implicit variable definitions,
 it might make sense to do this, but we already declare all vars so
 it adds nothing.

You are right, but the purpose of that "local" annotation is a bit different: if you don't use that annotation, you define a local k and you already have a variable named k in an outer name space, the compiler is supposed to generate an error. So "local" is a way to say the compiler that you know there is an outer name "k" and you want to hide it. Maybe "local" is not the best possible name for this annotation, "hider" seems more clear :-)
 What I've taken to doing is if I definitely want to access a
 class var, I'll just write this and use the dot to get to a global.
 
 That way, it's always clear what's going on without looking
 back at the function definition.
 
 I don't want that to be required though.

The purpose of the _optional_ explicitLookup annotation by Nick Sabalausky is that one, to ask the compiler to enforce what you do. Bye, bearophile
Jun 24 2011
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-06-24 17:39, Adam D. Ruppe wrote:
    void baz(local int k) {
    
        void inner(local int k) {}
    
    }

This is unneeded. By declaring a local variable, it's *obviously* local - there's no point in saying local again! Now, I've forgotten that I had a variable declared before, but adding more stuff to the argument list wouldn't change anything, because if I was looking at the argument list, I would have realized the local variable was there anyway! If D was a bad language that allowed implicit variable definitions, it might make sense to do this, but we already declare all vars so it adds nothing.
 void foo(int i) {} // hiding global variable

This is a good thing because you can reason about it locally. If you declare a variable locally, you know it is going to work. What I've taken to doing is if I definitely want to access a class var, I'll just write this and use the dot to get to a global. That way, it's always clear what's going on without looking back at the function definition.

That's why a lot of people typically use a different naming scheme for member variables (e.g. prepending them with _ or m_). As long as you're smart about naming, it's really not a problem.
 I don't want that to be required though.

Yeah. That would not be fun. As it is, there are a variety of ways that you can go about differentiating between globals, locals, and member variables. There's no need for the compiler or language to get into it any further than they already do. Being too strict about stuff like that would get very annoying. - Jonathan M Davis
Jun 24 2011
next sibling parent reply Peter Alexander <peter.alexander.au gmail.com> writes:
On 25/06/11 2:42 AM, Jonathan M Davis wrote:
 That's why a lot of people typically use a different naming scheme for member
 variables (e.g. prepending them with _ or m_). As long as you're smart about
 naming, it's really not a problem.

 I don't want that to be required though.

Yeah. That would not be fun. As it is, there are a variety of ways that you can go about differentiating between globals, locals, and member variables. There's no need for the compiler or language to get into it any further than they already do. Being too strict about stuff like that would get very annoying. - Jonathan M Davis

Agree with this 100%. This is a solved problem. No need for throwing more and more language features at it.
Jun 25 2011
parent bearophile <bearophileHUGS lycos.com> writes:
Peter Alexander:

 This is a solved problem. No need for throwing more and more language 
 features at it.

I agree, from what I am seeing in Haskell it's a mostly solved problem. Bye, bearophile
Jun 25 2011
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Jonathan M Davis:

That's why a lot of people typically use a different naming scheme for member
variables (e.g. prepending them with _ or m_). As long as you're smart about
naming, it's really not a problem.<

I too use the leading underscore. The problem with using name tags like the leading underscore is lack of enforcement from the compiler (there is only a partial enforcement: you have to use the same name and new instance attributes don't pop out of existance if you assign to a wrong attribute name by mistake, as in Python), and lack of standards. In other code you often find other wans to denote member variabiles. In Java you mostly have classes and nested classes, in C you have global variables and local variables (and function arguments). But in D the situation is more complex, you have non-lambda inner functions too, so the normal safeties used in C maybe are not enough. Do you use non-lambda inner functions often in your code? I do. D is more complex and more powerful than both C and Java, so maybe it also needs more safeguards that are not needed in C and Java. You have to keep in mind that D code has new kinds of bugs (or higher frequency of some kinds of old bugs) because D has more features than C/Java. Just trying to avoid bugs typical of C code is not enough in D! You have also to try to avoid D-specific bugs, coming from its differnces and new features.
 I don't want that to be required though.


Both the outer() and the idea by Sabalausky are fully optional. You are allowed to not use them, and removing them from code doesn't change what the code does. If library code that you are using uses them, you are not forced to use them in your code too.
There's no need for the compiler or language to get into it any further than
they already do.<

I am finding problems about the hiding of variable names, since years (I have some bug reports in bugzilla about sub-problems of this), so I presume for me there is a vague need for something better, to avoid the troubles I am finding. Such problems waste some of my time.
Being too strict about stuff like that would get very annoying.<

I am not sure that's true. The outer() is optional, this means you probably don't want to use if if you are writing 50 lines long programs, or if you are not using global variables and non-lambda inner functions a lot, and if even most of your functions are pure too. outer() is meant to help debugging too, when you find a bug, you add those annotations, and I think this helps uncover some problems. There are different kinds of programs, and different kinds of programmers, in some cases more strictness is desired. My experience tells me that sometimes more strictness avoids bugs that otherwise later make me waste far more time than being strict in the first place. But of course, this requires a bit of self-discipline and experience. I program in Python too, but it has nonlocal/global and its situation is very different, so you can't compare it will with coding in D. Bye, bearophile
Jun 25 2011
parent bearophile <bearophileHUGS lycos.com> writes:
Jonathan M Davis:

 The situation in D with regards to variable names isn't really any more 
 complex in D than it is in C++,

I don't agree. In D I use many inner functions, so I have to be careful about where a variable is defined, if global, if local, or if local in the outer function. This burns some of my time.
 and in my experience it's virtually never a 
 problem in C++. Smart variable naming makes things clear. This is a complete 
 non-issue IMHO. It rarely creates bugs.

Seeing my recurring mild troubles, and how SPARK has chosen a more strict attitude, I think you are not fully right.
 Non offense, but honestly, if you're 
 seeing much in the way of bugs from using the wrong variable, I have to wonder 
 if you're doing something wrong.

None taken, you're one of the most polite and gentle persons around here, I appreciate this. Surely different programmers have different cognitive capabilities. Sometimes in CLisp I get lost in parentheses, while I usually know where the North is while I program in Haskell. There's lot of free space to explore while you design a programming language, and I think current languages usually have parts that can be improved. Designing good computer languages is one of the most difficult activities I see around, they are interfaces between quirky and very complex mammals and refined and increasingly complex tech agglomerates able to approximate universal computation machines. Bye, bearophile
Jun 25 2011
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-06-25 05:52, bearophile wrote:
 Jonathan M Davis:
That's why a lot of people typically use a different naming scheme for
member variables (e.g. prepending them with _ or m_). As long as you're
smart about naming, it's really not a problem.<

I too use the leading underscore. The problem with using name tags like the leading underscore is lack of enforcement from the compiler (there is only a partial enforcement: you have to use the same name and new instance attributes don't pop out of existance if you assign to a wrong attribute name by mistake, as in Python), and lack of standards. In other code you often find other wans to denote member variabiles. In Java you mostly have classes and nested classes, in C you have global variables and local variables (and function arguments). But in D the situation is more complex, you have non-lambda inner functions too, so the normal safeties used in C maybe are not enough. Do you use non-lambda inner functions often in your code? I do. D is more complex and more powerful than both C and Java, so maybe it also needs more safeguards that are not needed in C and Java. You have to keep in mind that D code has new kinds of bugs (or higher frequency of some kinds of old bugs) because D has more features than C/Java. Just trying to avoid bugs typical of C code is not enough in D! You have also to try to avoid D-specific bugs, coming from its differnces and new features.

The situation in D with regards to variable names isn't really any more complex in D than it is in C++, and in my experience it's virtually never a problem in C++. Smart variable naming makes things clear. This is a complete non-issue IMHO. It rarely creates bugs. Non offense, but honestly, if you're seeing much in the way of bugs from using the wrong variable, I have to wonder if you're doing something wrong. - Jonathan M Davis
Jun 25 2011