www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Need review: explicit package protection

reply "Dicebot" <public dicebot.lv> writes:
Finally got to cleanup and submit this PR:
https://github.com/D-Programming-Language/dmd/pull/3651

While proposed change is very small (and backwards-compatible)
and not worth separate DIP, it is still a language change and
needs community approval.

Copy of description:

========================================

Currently there is no way to use package protection attribute
with deeply nested package hierarchy, forcing to either use flat
one or public protection. This is one of blocking issues for
further usage of package.d in Phobos and one of reasons why
namespace hacks are so popular.

For example, if helpers in std.internal will be marked as
package, only std.internal will be able to access those, but not
rest of std. This PR fixes it by allowing package(<pkgname>)
syntax to explicitly define owning package.

This new syntax will work:

---
module std.internal.mod1;
package(std) void foo() {}
module std.mod2;
---
import std.internal.mod2;
void bar() { foo(); }
----

Exact semantics can are described by added "protection" tests to
test/compilable (last commit in this PR).

Plain package behavior is unchanged and thus no breaking changes
introduced.
Jun 08 2014
next sibling parent Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 08/06/14 17:37, Dicebot via Digitalmars-d wrote:
While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

Looks cool to me. :-)
 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

Is it possible to permit multiple packages access in this way? For example, package(std.math, std.random) void foo() { ... }
Jun 08 2014
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2014-06-08 17:37, Dicebot wrote:
 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.

Is the idea that anything nested in the specified package has access to the symbol? -- /Jacob Carlborg
Jun 08 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 8 June 2014 at 16:27:12 UTC, Joseph Rushton Wakeling 
via Digitalmars-d wrote:
 Is it possible to permit multiple packages access in this way?  
 For example,

     package(std.math, std.random) void foo() { ... }

Of course it is possible but I don't think it is a good project structure to encourage. If these modules are so closely related they should be moved to single math-n-stuff package.
Jun 08 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 8 June 2014 at 19:02:58 UTC, Jacob Carlborg wrote:
 Is the idea that anything nested in the specified package has 
 access to the symbol?

Yes, this is how it is implemented right now.
Jun 08 2014
prev sibling next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
08-Jun-2014 19:37, Dicebot пишет:
 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

Even during my first experiments with turning parts of Phobos into packages I've been bitten by this. So basically +1 from me. -- Dmitry Olshansky
Jun 08 2014
prev sibling next sibling parent Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 08/06/14 23:47, Dicebot via Digitalmars-d wrote:
 Of course it is possible but I don't think it is a good project structure to
 encourage. If these modules are so closely related they should be moved to
 single math-n-stuff package.

Yea, I was concerned about the opposite, that you'd wind up with lots of package(std) access when in fact it's just a couple of modules that actually need access to the functionality in question.
Jun 08 2014
prev sibling next sibling parent Mike Parker <aldacron gmail.com> writes:
On 6/9/2014 12:37 AM, Dicebot wrote:

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.

Big +1 from me. I can see myself making good use of this.
Jun 09 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 08 Jun 2014 11:37:04 -0400, Dicebot <public dicebot.lv> wrote:

 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.

Yes, this becomes more crucial with the idea of splitting up a file seamlessly with the package.d idiom. A file that already has package-accessible functions CANNOT be split up without an improvement like this. Given that nothing can utilize unauthorized functions, you can only give more access to your own functions, I think this is a worthwhile improvement. -Steve
Jun 09 2014
prev sibling next sibling parent "Dejan Lekic" <dejan.lekic gmail.com> writes:
On Sunday, 8 June 2014 at 15:37:06 UTC, Dicebot wrote:
 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.

+1 Definitely worth merging! However, package module still have few issues. Issue #1) Few times I asked myself "what am i importing, package or a module?" when I used package module, so whenever I import a package, I add a short comment, something like: // assuming I have import foo.bar.baz; // package import Issue #2) Package module is not possible in projects with flat structure (projects whose authors did not reserve directories for packages. Example: Imagine developer has *all* his D sources in /home/dejan/src/d/myawesomeproject because he does not like big directory structures. Say his project has two packages foo.bar and foo.baz . And has following files in his project directory: tools.d // module foo.bar.tools; control.d // module foo.bar.control; screen.d // module foo.baz.screen; window.d // module foo.baz.window; package.d // can be only one 'package.d' within a single directory! main.d So, we have to rename the file into something else. Most likely developer would try with foo_bar_package.d and foo_baz_package.d, but it is not possible to do module foo.bar; // conflicts with previous modules or module foo.bar.package; // package is a reserved word So in this case package module is not possible, and developer has to use classic approach if he/she wants to import all modules within foo.bar or foo.baz packages.
Jun 09 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Monday, 9 June 2014 at 13:51:30 UTC, Dejan Lekic wrote:
 However, package module still have few issues.

 Issue #1)

 Few times I asked myself "what am i importing, package or a
 module?" when I used package module, so whenever I import a
 package, I add a short comment, something like:

 // assuming I have
 import foo.bar.baz; // package import

Why is this important? `package.d` is intentionally designed so that package imports are indistinguishable from module imports. That way library author can adjust module structure without breaking user code.
 Issue #2)
 Package module is not possible in projects with flat structure
 (projects whose authors did not reserve directories for 
 packages.

 Example:
 Imagine developer has *all* his D sources in
 /home/dejan/src/d/myawesomeproject because he does not like big
 directory structures. Say his project has two packages foo.bar
 and foo.baz . And has following files in his project directory:

 tools.d // module foo.bar.tools;
 control.d // module foo.bar.control;
 screen.d // module foo.baz.screen;
 window.d // module foo.baz.window;
 package.d // can be only one 'package.d' within a single
 directory!
 main.d

 ...

I'd call this "INVALID WON"T FIX" :) D module system is defined to have strong 1-to-1 matching with file system. Any attempt to circumvent that in favor of personal preferences is asking for trouble and is not worth supporting.
Jun 10 2014
prev sibling next sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 10 Jun 2014 11:48:17 +0000
Dicebot via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On Monday, 9 June 2014 at 13:51:30 UTC, Dejan Lekic wrote:
 Issue #2)
 Package module is not possible in projects with flat structure
 (projects whose authors did not reserve directories for
 packages.

 Example:
 Imagine developer has *all* his D sources in
 /home/dejan/src/d/myawesomeproject because he does not like big
 directory structures. Say his project has two packages foo.bar
 and foo.baz . And has following files in his project directory:

 tools.d // module foo.bar.tools;
 control.d // module foo.bar.control;
 screen.d // module foo.baz.screen;
 window.d // module foo.baz.window;
 package.d // can be only one 'package.d' within a single
 directory!
 main.d

 ...

I'd call this "INVALID WON"T FIX" :) D module system is defined to have strong 1-to-1 matching with file system. Any attempt to circumvent that in favor of personal preferences is asking for trouble and is not worth supporting.

+1 - Jonathan M Davis
Jun 10 2014
prev sibling next sibling parent "Boyd" <gaboonviper gmx.net> writes:
On Sunday, 8 June 2014 at 15:37:06 UTC, Dicebot wrote:
 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.

Perfect! With this, the package protection actually becomes useful:) Cheers, Boyd
Jun 11 2014
prev sibling next sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 08 Jun 2014 15:37:04 +0000
Dicebot via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.

I would be inclined to argue that this should be done in a way that restricts its use to subpackages of the package that it's being added to, otherwise this kind of breaks the whole idea of package and becomes something closer to C++'s friend. So, std.internal.foo could have symbols marked with package(std), but mystuff.bar.foo couldn't, because it's not a subpackage of std. - Jonathan M Davis
Jun 11 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 11 Jun 2014 10:09:18 -0400, Jonathan M Davis via Digitalmars-d  
<digitalmars-d puremagic.com> wrote:

 On Sun, 08 Jun 2014 15:37:04 +0000
 Dicebot via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.

I would be inclined to argue that this should be done in a way that restricts its use to subpackages of the package that it's being added to, otherwise this kind of breaks the whole idea of package and becomes something closer to C++'s friend. So, std.internal.foo could have symbols marked with package(std), but mystuff.bar.foo couldn't, because it's not a subpackage of std.

I think this is a given. But note that you can write code that "lives" in the same package via a module statement today. i.e., inside mystuff/bar/foo.d: module std.internal.foo; -Steve
Jun 11 2014
prev sibling next sibling parent "Dejan Lekic" <dejan.lekic gmail.com> writes:
 I'd call this "INVALID WON"T FIX" :) D module system is defined 
 to have strong 1-to-1 matching with file system. Any attempt to 
 circumvent that in favor of personal preferences is asking for 
 trouble and is not worth supporting.

I can only agree with this, but fact is that D allows flattened project structure (all modules in all packages in a single directory)... It should be noted that people who prefer such structure are on their own. I myself prefer to have a directory per-package, as it is IMHO more clear and organised.
Jun 11 2014
prev sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 11 Jun 2014 11:27:35 -0400
Steven Schveighoffer via Digitalmars-d <digitalmars-d puremagic.com>
wrote:

 On Wed, 11 Jun 2014 10:09:18 -0400, Jonathan M Davis via
 Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I would be inclined to argue that this should be done in a way
 that restricts
 its use to subpackages of the package that it's being added to,
 otherwise this
 kind of breaks the whole idea of package and becomes something
 closer to C++'s
 friend. So, std.internal.foo could have symbols marked with
 package(std), but
 mystuff.bar.foo couldn't, because it's not a subpackage of std.

I think this is a given. But note that you can write code that "lives" in the same package via a module statement today. i.e., inside mystuff/bar/foo.d: module std.internal.foo;

True, but then it's treated as part of that package rather than in a package of its own, so all the code that uses it is affected and not just its internals. It's still unfortunate that that can be done, but I don't think that we realistically prevent it. However, allowing for _any_ package to be added to with package(foo) would not be restrictive in the same way and would likely be more prone to abuse, and we _can_ prevent that easily enough by restricting its used to subpackages. - Jonathan M Davis
Jun 11 2014