## digitalmars.D - DIP16: Transparently substitute module with package

• Andrei Alexandrescu (7/7) Mar 30 2012 Starting a new thread from one in announce:
• deadalnix (13/20) Mar 30 2012 I was waiting for it :D
• Timon Gehr (18/25) Mar 30 2012 Those are two proposals.
• Timon Gehr (1/1) Mar 30 2012 On second thought, issue 2 is probably not that much of a problem.
• Nick Sabalausky (8/14) Mar 30 2012 That occurred to me, and I thought about proposing the same thing you're...
• Robert Clipsham (12/19) Mar 30 2012 The proposal doesn't say what happens when package.d is not found but
• Nick Sabalausky (6/27) Mar 30 2012 That would effectively be the same as Java's "import foo.*" and a lot of...
• Steven Schveighoffer (12/17) Mar 30 2012 I think package.d should be allowed to specify module. Otherwise,
• Andrej Mitrovic (21/22) Mar 30 2012 "That means a program that imports std.algorithm may use "std.sort"
• Nick Sabalausky (3/13) Mar 30 2012 Or maybe just require the module name ends with ".package"
• Brad Anderson (4/26) Mar 30 2012 I was actually kind of surprised when I found out this doesn't work. It
• F i L (11/34) Mar 30 2012 Ya that was the behavior I expected as well. Would be great if it
• Andrej Mitrovic (7/8) Mar 30 2012 Btw, I bet with the help of hackers like e.g. Kenji Hara we'll have
• Jonathan M Davis (42/52) Mar 30 2012 The first part with package.d seems like a good idea and certainly affec...
• Rene Zwanenburg (6/13) Mar 30 2012 I'm not sure if that's a good idea. I'd prefer a new kind of
• Andrei Alexandrescu (5/20) Mar 30 2012 Why should you?
• Jonathan M Davis (66/92) Mar 30 2012 std.sort works because there's only one sort. If there are two, you get ...
• Timon Gehr (2/4) Mar 31 2012 +1. That is even better than treating the package module specially.
• Martin Nowak (2/8) Mar 31 2012 That already works, doesn't it?
• Timon Gehr (2/11) Mar 31 2012 It already works indeed. My bad.
• Jonathan M Davis (12/25) Mar 31 2012 Does it? I thought that std.range.replace wouldn't work (even though std...
• deadalnix (3/29) Mar 31 2012 I did propose that in another thread, so this is a +1 too.
• H. S. Teoh (11/23) Mar 30 2012 [...]
• deadalnix (3/32) Mar 31 2012 You are reversing the logic. It have to be shown that it is not a
• Martin Nowak (4/12) Mar 31 2012 You can turn that argument around, std.sort(arr) instead of sort(arr)
• Jonathan M Davis (6/18) Mar 30 2012 No, I don't think so. If I understand the proposal correctly, it would e...
• Jonathan M Davis (14/34) Mar 30 2012 It would certainly be desirable in some cases, but I believe that the re...
• Nick Sabalausky (26/26) Mar 30 2012 My comments:
• Michel Fortin (42/43) Mar 30 2012 Since you're askingâ€¦
• Steven Schveighoffer (11/32) Apr 02 2012 s =
• Michel Fortin (23/37) Apr 02 2012 If you have ambiguous templates in the same module, it'll always be
• Steven Schveighoffer (23/51) Apr 04 2012 How does this happen? The FQN cannot be ambiguous.
• Michel Fortin (51/61) Apr 04 2012 Sure it can if I follow DIP16, because module names can become ambiguous...
• Steven Schveighoffer (21/54) Apr 04 2012 OK, but when is it ever valid to refer to a module when the semantic
• Timon Gehr (5/49) Apr 04 2012 No symbol is resolved until semantic, but I don't think hiding the
• Steven Schveighoffer (16/32) Apr 04 2012 hm... maybe we'd have to have new __traits that would disambiguate, like...
• Michel Fortin (31/54) Apr 04 2012 I don't think option 2 is realistic (see my other post).
• Michel Fortin (30/47) Apr 04 2012 Forgot to follow through with what "import std.algorithm" would do.
• Steven Schveighoffer (58/103) Apr 05 2012 =
• deadalnix (9/26) Apr 05 2012 The behavior you described has been proposed for public import, and have...
• Steven Schveighoffer (44/77) Apr 05 2012 But
• Timon Gehr (2/4) Apr 05 2012 Have you tried it?
• Steven Schveighoffer (17/22) Apr 05 2012 I just did. OK, what the hell are we arguing about then?!
• Jonathan M Davis (17/45) Apr 05 2012 What doesn't work is being able to turn a module into a package with the...
• Steven Schveighoffer (12/26) Apr 05 2012 But so what? nobody has any code like:
• Jonathan M Davis (34/65) Apr 05 2012 The issue is code organization. If you want to split up std.algorithm (o...
• Steven Schveighoffer (12/27) Apr 05 2012 My view is that people will not import the smaller modules, they will on...
• Andrei Alexandrescu (5/18) Apr 05 2012 I think we should be looking for a solution that not only allows
• Jonathan M Davis (15/35) Apr 05 2012 Yeah. If all we want to do is continue to always import std.algorithm, t...
• Michel Fortin (53/56) Apr 05 2012 I think it'd be valuable too. But how do you do that without creating
• deadalnix (6/13) Apr 06 2012 It isn't possible. But as already mentioned, all name doesn't make sense...
• Andrei Alexandrescu (6/22) Apr 06 2012 One other desirable feature is library distribution. Library writers
• deadalnix (2/26) Apr 06 2012 Good point. That is an argument for package.d, all.d or _.d .
• David Gileadi (3/4) Apr 06 2012 Or [packagename].d, where [packagename] is the name of its sibling
• deadalnix (7/11) Apr 06 2012 No, because in this case, this isn't in the same folder. Still I prefer
• deadalnix (42/65) Apr 06 2012 Why not limit name collision to name which make sense ?
• Michel Fortin (26/27) Apr 06 2012 You're proposing that fully qualified names sometime work and sometime
• Steven Schveighoffer (21/43) Apr 06 2012 Of course you *can* do this. I think you mean "and allows one to refer ...
• Jonathan M Davis (25/37) Apr 06 2012 Which is actually considered bad practice in Java, though a lot of peopl...
• Jacob Carlborg (5/15) Apr 07 2012 import foo.bar.*
• Jonathan M Davis (10/27) Apr 07 2012 Like I said, some people do like to do it, but Eclipse doesn't like you ...
• Kapps (8/24) Apr 07 2012 http://stackoverflow.com/questions/147454/why-is-using-a-wild-card-with-...
• Steven Schveighoffer (13/23) Apr 09 2012 I think either you or I am missing something.
• Jonathan M Davis (20/46) Apr 09 2012 Okay. I reread DIP15 again. I guess that I scanned over it too quickly b...
• deadalnix (8/72) Apr 05 2012 I know. I think you answered too fast.
• Steven Schveighoffer (22/25) Apr 05 2012 Another way to look at it:
• Michel Fortin (12/31) Apr 05 2012 I have the same reserve about it's lack of flexibility too. It would be
• Michel Fortin (25/34) Apr 04 2012 You'd need a whole lot of context.
• Andrei Alexandrescu (7/19) Apr 05 2012 Interesting. But isn't there an ambiguity when the symbol is not the
• Steven Schveighoffer (11/32) Apr 05 2012 Stop reading all my posts except for the ones I just posted. I was
• Andrei Alexandrescu (10/13) Apr 05 2012 Not entirely (I was aware of the way public import works). An issue does...
• Steven Schveighoffer (15/27) Apr 05 2012 Right, but if one only ever imports std.algorithm, who cares what the
• Jonathan M Davis (11/19) Apr 05 2012 That's assuming that you insist on keeping all of the documentation in o...
• Steven Schveighoffer (10/33) Apr 05 2012 I thought the whole point was code maintenance? Not documentation
• Jonathan M Davis (14/35) Apr 05 2012 If the module isn't large enough to be split for documentation, I find i...
• Steven Schveighoffer (19/57) Apr 05 2012 Why do we ever need to split modules for documentation? Just fix the do...
• Jonathan M Davis (17/39) Apr 05 2012 That may or may not be desirable (certainly in the case of smaller types...
• Steven Schveighoffer (9/39) Apr 06 2012 ddoc's output leaves a lot to be desired. The unorganized links at the ...
• Jonathan M Davis (22/69) Apr 06 2012 There's no question that the links at the top suck, and std.datetime is ...
• deadalnix (4/17) Apr 05 2012 The first one isn't a problem. It isn't too many names, it is 2 names,
• deadalnix (6/28) Apr 05 2012 The whole point of this thread is to import symbols from submodules into...
• Jacob Carlborg (4/10) Apr 04 2012 I do that all the time in my libraries.
• Ary Manzana (32/37) Mar 30 2012 Great. Large modules are my main complaint about D. :-)
• Chris NS (10/10) Mar 30 2012 I'm pretty impressed with the idea, and look forward to its
• Nick Sabalausky (3/10) Mar 30 2012 There's always the "package" access specifier.
• Chris NS (6/19) Mar 30 2012 True, though it bears revisiting in its own right. I've never
• Steven Schveighoffer (5/13) Apr 02 2012 They would lose their friend status. Classes that you need to be
• Derek (41/44) Mar 31 2012 We solved this issue in the Euphoria Programming language by introducing...
• foobar (24/31) Mar 31 2012 Hooray! I was loudly complaining about this issue for years. I'm
• deadalnix (35/42) Mar 31 2012 OK, here is an alternative proposal, based on what has been said here,
• Martin Nowak (7/14) Apr 03 2012 What about supporting package initalization?
• deadalnix (15/34) Apr 04 2012 That is pretty much what was proposed by people in this thread.
• Martin Nowak (6/9) Apr 04 2012 pkg.d would always be a module, hence result in a module/package conflic...
• deadalnix (10/19) Apr 05 2012 This is true, but why would it be a conflict ?
• Steven Schveighoffer (26/32) Apr 05 2012 BTW, this case makes the part of DIP16 which wants to shortcut fully
• Ary Manzana (18/51) Apr 05 2012 It's a shortcut to the module function foo().
• Steven Schveighoffer (24/86) Apr 06 2012 That's slightly different, because you must *always* qualify struct b's ...
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means
that he would approve a pull request implementing DIP16 (subject to
regular correctness checks).

Destroy!

Andrei

Mar 30 2012
Le 30/03/2012 16:46, Andrei Alexandrescu a écrit :
Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means
that he would approve a pull request implementing DIP16 (subject to
regular correctness checks).

Destroy!

Andrei

I was waiting for it :D

First, as all.d is already a convention in many D projects, why choose
package.d ? I understand this is a keyword, but this will also be
painfull for many developer.

Second, what the rule of a .d file and a folder existing, but in
different path (think -I switch of the compiler).

In the first place, is the module declaration really usefull ?

About the lookup rule, I understand that if I import a.b and use the
function a.b.c.foo , then i must be able to refers to it as a.b.foo, but
why a.foo ? It seems to me like going too far in the modification for no
benefit (it can only increase the number of collision, and has no
benefit I can think of).

Mar 30 2012
Timon Gehr <timon.gehr gmx.ch> writes:
On 03/30/2012 04:46 PM, Andrei Alexandrescu wrote:
Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means
that he would approve a pull request implementing DIP16 (subject to
regular correctness checks).

Destroy!

Andrei

Those are two proposals.

I think the first one should be implemented.

I don't really like the second one.

1. It is an over-general solution, because it does not solve a general
problem.

2. Library writers lose control. All symbol names in the package are
reserved when the package is deployed. It will be a breaking change to
introduce a sub-package that happens to have the same name as some
symbol defined in any module in the same package hierarchy.

3. symbol lookup is already hard enough to get right, because compile
time reflection and conditional code generation can introduce
ambiguities and contradictions. DMD does not get it right. It is likely
that this change would make fixing this in a general but
not-too-conservative way considerably harder.

Maybe it would be better to just interpret foo.bar.baz as
foo.bar.package.baz if foo.bar is a package that has been imported via
the foo.bar.package rewrite? Of course, issue 2 probably would remain.

Mar 30 2012
Timon Gehr <timon.gehr gmx.ch> writes:
On second thought, issue 2 is probably not that much of a problem.

Mar 30 2012
"Nick Sabalausky" <a a.a> writes:
"Timon Gehr" <timon.gehr gmx.ch> wrote in message
news:jl4jmg$2j1r$1 digitalmars.com...
I don't really like the second one.

1. It is an over-general solution, because it does not solve a general
problem.

Maybe it would be better to just interpret foo.bar.baz as
foo.bar.package.baz if foo.bar is a package that has been imported via the
foo.bar.package rewrite?

That occurred to me, and I thought about proposing the same thing you're
suggesting, but on second thought I wasn't so sure:

If I need to disambiguate between "std.algorithm.find" and
"foo.bar.baz.find", it might be nice to be able to just say "Meh,
just...that one in Phobos, ie 'std'". Or "Just go with that 'foo' one".

I could go either way, really.

Mar 30 2012
Robert Clipsham <robert octarineparrot.com> writes:
On 30/03/2012 15:46, Andrei Alexandrescu wrote:
Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means
that he would approve a pull request implementing DIP16 (subject to
regular correctness checks).

Destroy!

Andrei

The proposal doesn't say what happens when package.d is not found but
foo/bar/ exists.

Given that a lot of people will just use public import foo.bar.*; in
that file, would it make sense for package.d missing to imply import
foo.bar.*? That would save typing out every single file in there. Of
course it would also be annoying if you wanted to import everything
except one file, as you'd then have to type out every single import anyway.

The other option is to error, which is probably a more sane option.

--
Robert
http://octarineparrot.com/

Mar 30 2012
"Nick Sabalausky" <a a.a> writes:
"Robert Clipsham" <robert octarineparrot.com> wrote in message
news:jl4l5t$2m62$1 digitalmars.com...
On 30/03/2012 15:46, Andrei Alexandrescu wrote:
Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means
that he would approve a pull request implementing DIP16 (subject to
regular correctness checks).

Destroy!

Andrei

The proposal doesn't say what happens when package.d is not found but
foo/bar/ exists.

Given that a lot of people will just use public import foo.bar.*; in that
file, would it make sense for package.d missing to imply import foo.bar.*?
That would save typing out every single file in there. Of course it would
also be annoying if you wanted to import everything except one file, as
you'd then have to type out every single import anyway.

That would effectively be the same as Java's "import foo.*" and a lot of
people have issues with that.

The other option is to error, which is probably a more sane option.

That's what I'd suggest doing. Just treat it like importing any other
missing package.

Mar 30 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means
that he would approve a pull request implementing DIP16 (subject to
regular correctness checks).

I think package.d should be allowed to specify module.  Otherwise,
something like /usr/include/d/std/datetime/package.d, what is the assumed
package?  If module is not useful here, it is not useful anywhere.

I think the omission should be strictly anything after the package
directory.  For example, if you have std/algorithm/package.d and
std/algorithm/sorting.d, where package.d imports sorting.d, you can refer
to std.algorithm.sorting.sort by omitting sorting, but not by omitting
algorithm or std.

Other than that, this is a good change.

-Steve

Mar 30 2012
Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
Destroy!

"That means a program that imports std.algorithm may use "std.sort"
for the symbol "std.algorithm.sort"."

That's quite interesting. Would that also mean that you could do:
import std.algorithm;  // has indexOf
import std.string;  // has indexOf
void main() {
string.indexOf("foo", "foo"); -> std.string.indexOf
}

That would ease usage of Phobos a little bit. OTOH 'string' already is
a keyword and things might get hairy..

Still this is one of the few proposals I like. My only caveat is the
comment: "except the file is not allowed to use the "module"
declaration.". Wouldn't it be better if we explicitly declared a
module as a package instead? In foo\bar\package.d:
package foo.bar;

Since the "module" declaration must be on the first line (or second
line after shebang), you could special-case DMD to allow the package
keyword to be used here. I know D likes to abuse a keyword for
multiple things (hello Mr. Static!), but I think we could live with
it.

Mar 30 2012
"Nick Sabalausky" <a a.a> writes:
"Andrej Mitrovic" <andrej.mitrovich gmail.com> wrote in message
news:mailman.1240.1333130858.4860.digitalmars-d puremagic.com...
Still this is one of the few proposals I like. My only caveat is the
comment: "except the file is not allowed to use the "module"
declaration.". Wouldn't it be better if we explicitly declared a
module as a package instead? In foo\bar\package.d:
package foo.bar;

Since the "module" declaration must be on the first line (or second
line after shebang), you could special-case DMD to allow the package
keyword to be used here. I know D likes to abuse a keyword for
multiple things (hello Mr. Static!), but I think we could live with
it.

Or maybe just require the module name ends with ".package"

Mar 30 2012
Brad Anderson <eco gnuk.net> writes:
On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic <
andrej.mitrovich gmail.com> wrote:

On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
Destroy!

"That means a program that imports std.algorithm may use "std.sort"
for the symbol "std.algorithm.sort"."

That's quite interesting. Would that also mean that you could do:
import std.algorithm;  // has indexOf
import std.string;  // has indexOf
void main() {
string.indexOf("foo", "foo"); -> std.string.indexOf
}

I was actually kind of surprised when I found out this doesn't work.  It
seems so natural to resolve ambiguity using as little context as necessary.

That would ease usage of Phobos a little bit. OTOH 'string' already is
a keyword and things might get hairy..

Still this is one of the few proposals I like. My only caveat is the
comment: "except the file is not allowed to use the "module"
declaration.". Wouldn't it be better if we explicitly declared a
module as a package instead? In foo\bar\package.d:
package foo.bar;

Since the "module" declaration must be on the first line (or second
line after shebang), you could special-case DMD to allow the package
keyword to be used here. I know D likes to abuse a keyword for
multiple things (hello Mr. Static!), but I think we could live with
it.


Mar 30 2012
"F i L" <witte2008 gmail.com> writes:
On Friday, 30 March 2012 at 18:15:57 UTC, Brad Anderson wrote:
On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic <
andrej.mitrovich gmail.com> wrote:

On 3/30/12, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:
Destroy!

"That means a program that imports std.algorithm may use
"std.sort"
for the symbol "std.algorithm.sort"."

That's quite interesting. Would that also mean that you could
do:
import std.algorithm;  // has indexOf
import std.string;  // has indexOf
void main() {
string.indexOf("foo", "foo"); -> std.string.indexOf
}

I was actually kind of surprised when I found out this doesn't
work.  It
seems so natural to resolve ambiguity using as little context
as necessary.

Ya that was the behavior I expected as well. Would be great if it
worked like that. Just back trace the reference until the
ambiguity is resolved.

// -----

Also, I'm probably missing something here, but I never understood
why importing a package doesn't work like it does in
Actionscript/Java/others...

import foo.bar.*; // everything
import foo.bar.all; // custom

That makes a lot of sense to me.

Mar 30 2012
Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Btw, I bet with the help of hackers like e.g. Kenji Hara we'll have
this implemented in a matter of days (if it gets accepted). Compare
that to having a C++ committee that would spend the next 5 years
writing a 100-page spec on what is or isn't a package, heheh. Didn't
Bjarne say on Going Native that he doesn't really know what a module
is or should be? He should try D sometime. ;)

Mar 30 2012
"Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 30, 2012 09:46:19 Andrei Alexandrescu wrote:
Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means
that he would approve a pull request implementing DIP16 (subject to
regular correctness checks).

Destroy!

The first part with package.d seems like a good idea and certainly affects some
of what I've been doing with std.datetime. In order to split it right now, you
need a new package name, leaving the original for importing everything, and
this provides a better way of dealing with that.

However, I'm very nervous about the second part. e.g. std.sort instead of
std.algorithm.sort seems like a bad idea to me. It increases the odds of name
conflicts for little benefit. Not to mention, it'll make it a lot more
confusing
to find what modules stuff is actually in if people start doing stuff like

std.sort(arr);

In the case of sort, you may know where it's from - particularly since it's so
common - but the less well-known the function is, the less likely that is at
all obvious where it comes from, and if you're dealing with 3rd party
software, then it wouldn't be at all obvious. For instance, how would you know
that party.foo is really party.bar.foo? You wouldn't. Being so lax about
importing could really harm code readibility (and maintainibility, since it
increases the odds of name clashes). So, I'm inclined to say that that is a

I'd propose that we make it so that if a module publicly imports another
module, then you could treat it as if it were in that module. So, because
std.datetime.package publicly imports std.datetime.systime, you could use
std.datetime.SysTime instead of std.datetime.systime.SysTime. The compiler
would need to realize that they're exactly the same symbol (we've had bugs
relating to importing with : and the like which ended up creating new symbols,
and we don't want that here), but that shouldn't be all that hard. That gives
you control over which symbols are able to be treated as if they were in a
given package rather than affecting everything indiscriminitely (and if you use
: with public imports, it should then be possible, to restrict it to the
_exact_ set of symbols that you want if you don't want all of the symbols to
be treated as if they were in that module).

Another question is how this affects the documentation. Does package.d generate
a page just like the other modules do? The lack of a module declaration could
make that difficult (not impossible, but it would probably require changes to
ddoc in addition to the module stuff). Also, does that page get treated in any
special manner in how the documentation is laid out, because it's for the
package as a whole (probably more of a site question than a ddoc one though)?
I'd like to be able to have a page describing the package as a whole for
std.datetime in addition to having the individual pages rather than just
splitting it up, and leaving the programmer to read each of the individual
pages with no overview. And I think that however package.d works, it needs to
enable that.

- Jonathan M Davis

Mar 30 2012
"Rene Zwanenburg" <renezwanenburg gmail.com> writes:
On Friday, 30 March 2012 at 18:39:44 UTC, Jonathan M Davis wrote:
I'd propose that we make it so that if a module publicly
imports another
module, then you could treat it as if it were in that module.
So, because
std.datetime.package publicly imports std.datetime.systime, you
could use
std.datetime.SysTime instead of std.datetime.systime.SysTime.

I'm not sure if that's a good idea. I'd prefer a new kind of
import statement, perhaps something like:

// module std.datetime.package
alias import std.datetime.systime;

which is similar to a public alias of everything in that module?

Mar 30 2012
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/30/12 1:39 PM, Jonathan M Davis wrote:
However, I'm very nervous about the second part. e.g. std.sort instead of
std.algorithm.sort seems like a bad idea to me. It increases the odds of name
conflicts for little benefit.

Example?

Not to mention, it'll make it a lot more confusing
to find what modules stuff is actually in if people start doing stuff like

std.sort(arr);

In the case of sort, you may know where it's from - particularly since it's so
common - but the less well-known the function is, the less likely that is at
all obvious where it comes from, and if you're dealing with 3rd party
software, then it wouldn't be at all obvious. For instance, how would you know
that party.foo is really party.bar.foo? You wouldn't.

Why should you?

Being so lax about
importing could really harm code readibility (and maintainibility, since it
increases the odds of name clashes). So, I'm inclined to say that that is a

Maybe if you produce a solid example, I'd be convinced.

Andrei

Mar 30 2012
"Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 30, 2012 14:33:58 Andrei Alexandrescu wrote:
On 3/30/12 1:39 PM, Jonathan M Davis wrote:
However, I'm very nervous about the second part. e.g. std.sort instead of
std.algorithm.sort seems like a bad idea to me. It increases the odds of
name conflicts for little benefit.

Example?

std.sort works because there's only one sort. If there are two, you get a
conflict (e.g. if you had std.path.sort which sorted paths in some
path-specific
manner). If std.path.sort existed now, then std.sort wouldn't work, and you'd
be forced to specify std.algorithm.sort or std.path.sort, and that's fine. It
would be similar to having to specify std.algorithm.indexOf when you've
imported both std.string and std.algorithm. But the problem is when
std.path.sort is added _later_.

All of a sudden, code which used std.sort and worked is now broken. The
problem does currently exist in that if we added indexOf to another module -
say std.array - then code which imported either std.string or std.algorithm as
well as std.array would break with the addition of std.array.indexOf, but your
proposal makes it worse. Not only does it provide another way in which adding
a function could result in conflicts when existing code is recompiled, but it
makes it so that if you add any function anywhere in the _entire standard
library_ which has the name as an existing one, you get a conflict (if anyone
uses std.x rather than x or the full import path).

D does a good job of providing ways to fix name conflicts, but it doesn't do a
good job of preventing them when adding new symbols to a library (primarily
because it doesn't use static imports by default), and your proposal makes
that part of the problem worse. If std.x were to become common practice, then
any time that you added a symbol to a library when that symbol was already
used by another module, you'd create conflicts (combined with the fact that
private doesn't hide symbol names but merely makes them inaccessible, this
could result in a lot of symbol name conflicts).

Not to mention, it'll make it a lot more confusing
to find what modules stuff is actually in if people start doing stuff like

std.sort(arr);

In the case of sort, you may know where it's from - particularly since
it's so common - but the less well-known the function is, the less likely
that is at all obvious where it comes from, and if you're dealing with
3rd party software, then it wouldn't be at all obvious. For instance, how
would you know that party.foo is really party.bar.foo? You wouldn't.

Why should you?

Do you know what the foo function does? If you don't, you're going to have to
look it up. And if you don't know what module it comes from, you can't do
that. You also have to know where foo is from if a foo function is added to
another module and causes a conflict, because you're going to have to give the
full import path to actually use it. That's currently true with just bare foo
as well, but party.foo gives the illusion of specifying where foo is from
without actually specifying where it's from. At least right now, if foo is
used with its import path, you know that that's actually its import path.

Also, what happens if we want to add a module named sort later? The fact that
people are using std.sort means that adding std.sort as a module will break
code. Granted, it's not very likely that we're going to add a module named
sort, but there are plenty of other symbol names that it could happen with.
But then again, if we decided to provide a module with all of the major sort
algorithms, then maybe we _would_ create a module named std.sort. Just because
we don't see a need now doesn't mean that we won't later. In either case, by
allowing std.x where x is a symbol in any sub-module of std, you're going to
create conflicts any time that you add a module which has the same name as an
existing symbol anywhere in the library.

Being so lax about
importing could really harm code readibility (and maintainibility, since
it
increases the odds of name clashes). So, I'm inclined to say that that is
a

Maybe if you produce a solid example, I'd be convinced.

Well, as I've pointed with a few examples here, your proposal will increase
the chances of adding symbol conflicts any time that a symbol is added to a
library all just so that you can do std.algorithm.sort instead of
std.algorithm.submodule1.sort once sort has been moved to
std.algorithm.sumodule1. And we could make it possible to do
std.algorithm.sort without adding all of those possible conflicts.

The simplest solution would simply be to make it so that if std.algorithm.sort
is used, and std.algorithm is a package with a std.algorithm.package module,
then the compiler looks in all of the sub-modules of std.algorithm to find
sort. That solves the problem right there without increasing the odds of
symbol conflicts across the entire library like your proposal does.

But personally, I like the idea of making it so that publicly imported symbols
can be accessed as if they were in the module that publicly imported them
(with package.d being treated as if it had the same name as the package that
it's in). That's essentially how it already works except when specifying the
full import path for a symbol. And that way, you can specify in
std.algorithm.package.d exactly what you want to be imported when
std.algorithm is imported (including using : to restrict it to specific symbols
in a module), and only those symbols will be treated as if they were part of
std.algorithm - both for importing purposes and when specifying the import
path when using a symbol. The library maintainer then has control over which
symbols get used with which import paths.

- Jonathan M Davis

Mar 30 2012
Timon Gehr <timon.gehr gmx.ch> writes:
On 03/30/2012 11:35 PM, Jonathan M Davis wrote:
But personally, I like the idea of making it so that publicly imported symbols
can be accessed as if they were in the module that publicly imported them

+1. That is even better than treating the package module specially.

Mar 31 2012
"Martin Nowak" <dawg dawgfoto.de> writes:
On Sat, 31 Mar 2012 13:06:36 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

On 03/30/2012 11:35 PM, Jonathan M Davis wrote:
But personally, I like the idea of making it so that publicly imported
symbols
can be accessed as if they were in the module that publicly imported
them

+1. That is even better than treating the package module specially.

That already works, doesn't it?

Mar 31 2012
Timon Gehr <timon.gehr gmx.ch> writes:
On 03/31/2012 03:49 PM, Martin Nowak wrote:
On Sat, 31 Mar 2012 13:06:36 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

On 03/30/2012 11:35 PM, Jonathan M Davis wrote:
But personally, I like the idea of making it so that publicly
imported symbols
can be accessed as if they were in the module that publicly imported
them

+1. That is even better than treating the package module specially.

That already works, doesn't it?


Mar 31 2012
Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, March 31, 2012 17:30:36 Timon Gehr wrote:
On 03/31/2012 03:49 PM, Martin Nowak wrote:
On Sat, 31 Mar 2012 13:06:36 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:
On 03/30/2012 11:35 PM, Jonathan M Davis wrote:
But personally, I like the idea of making it so that publicly
imported symbols
can be accessed as if they were in the module that publicly imported
them

+1. That is even better than treating the package module specially.

That already works, doesn't it?

Does it? I thought that std.range.replace wouldn't work (even though std.range
publicly imports std.array), because replace isn't part of std.range. You
don't need to import std.array if you import std.range, because std.range
does, but you can't specificy it's path as if it were in std.range. I'll have
to check...

Okay. You're right. std.range.replace _does_ work. So then the only issue is
making it so that public imports in std.datetime.package are treated as if
they were in std.datetime. That seems like the cleanest solution to me. It
goes along quite nicely with making it so that anything in
std.datetime.package is imported when importing std.datetime.

- Jonathan M Davis

Mar 31 2012
Le 30/03/2012 23:35, Jonathan M Davis a Ã©crit :
On Friday, March 30, 2012 14:33:58 Andrei Alexandrescu wrote:
On 3/30/12 1:39 PM, Jonathan M Davis wrote:
However, I'm very nervous about the second part. e.g. std.sort instead of
std.algorithm.sort seems like a bad idea to me. It increases the odds of
name conflicts for little benefit.

Example?

std.sort works because there's only one sort. If there are two, you get a
conflict (e.g. if you had std.path.sort which sorted paths in some
path-specific
manner). If std.path.sort existed now, then std.sort wouldn't work, and you'd
be forced to specify std.algorithm.sort or std.path.sort, and that's fine. It
would be similar to having to specify std.algorithm.indexOf when you've
imported both std.string and std.algorithm. But the problem is when
std.path.sort is added _later_.

I mentionned that as being a ptotential issue, so this is a +1 .

But personally, I like the idea of making it so that publicly imported symbols
can be accessed as if they were in the module that publicly imported them
(with package.d being treated as if it had the same name as the package that
it's in). That's essentially how it already works except when specifying the
full import path for a symbol. And that way, you can specify in
std.algorithm.package.d exactly what you want to be imported when
std.algorithm is imported (including using : to restrict it to specific symbols
in a module), and only those symbols will be treated as if they were part of
std.algorithm - both for importing purposes and when specifying the import
path when using a symbol. The library maintainer then has control over which
symbols get used with which import paths.

- Jonathan M Davis

I did propose that in another thread, so this is a +1 too.

Mar 31 2012
"H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Mar 30, 2012 at 05:35:25PM -0400, Jonathan M Davis wrote:
[...]
But personally, I like the idea of making it so that publicly imported
symbols can be accessed as if they were in the module that publicly
imported them (with package.d being treated as if it had the same name
as the package that it's in). That's essentially how it already works
except when specifying the full import path for a symbol. And that
way, you can specify in std.algorithm.package.d exactly what you want
to be imported when std.algorithm is imported (including using : to
restrict it to specific symbols in a module), and only those symbols
will be treated as if they were part of std.algorithm - both for
importing purposes and when specifying the import path when using a
symbol. The library maintainer then has control over which symbols get
used with which import paths.

[...]

+1.

I also have my doubts about the wisdom of std.sort implicitly binding to
std.algorithm.sort. It's nice syntactic sugar in the short term, but as
Jonathan points out, it can cause headaches in the long term. I say we
should hold off on that part of the proposal.

T

--
Microsoft is to operating systems & security ... what McDonalds is to gourmet
cooking.

Mar 30 2012
Le 30/03/2012 21:33, Andrei Alexandrescu a Ã©crit :
On 3/30/12 1:39 PM, Jonathan M Davis wrote:
However, I'm very nervous about the second part. e.g. std.sort instead of
std.algorithm.sort seems like a bad idea to me. It increases the odds
of name
conflicts for little benefit.

Example?

Not to mention, it'll make it a lot more confusing
to find what modules stuff is actually in if people start doing stuff
like

std.sort(arr);

In the case of sort, you may know where it's from - particularly since
it's so
common - but the less well-known the function is, the less likely that
is at
all obvious where it comes from, and if you're dealing with 3rd party
software, then it wouldn't be at all obvious. For instance, how would
you know
that party.foo is really party.bar.foo? You wouldn't.

Why should you?

Being so lax about
importing could really harm code readibility (and maintainibility,
since it
increases the odds of name clashes). So, I'm inclined to say that that
is a

Maybe if you produce a solid example, I'd be convinced.

Andrei

You are reversing the logic. It have to be shown that it is not a
problem to do the change. The other way around is flawed logic.

Mar 31 2012
"Martin Nowak" <dawg dawgfoto.de> writes:
 However, I'm very nervous about the second part. e.g. std.sort instead of
std.algorithm.sort seems like a bad idea to me. It increases the odds of
name
conflicts for little benefit. Not to mention, it'll make it a lot more
confusing
to find what modules stuff is actually in if people start doing stuff
like

std.sort(arr);

You can turn that argument around, std.sort(arr) instead of sort(arr)
provides some extra context for disambiguation.
You'd be able to do that for public imports in the package.d module anyhow,
so it makes sense to support it for implicit packages as well.

Mar 31 2012
"Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 30, 2012 20:06:57 Andrej Mitrovic wrote:
On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
Destroy!

"That means a program that imports std.algorithm may use "std.sort"
for the symbol "std.algorithm.sort"."

That's quite interesting. Would that also mean that you could do:
import std.algorithm; // has indexOf
import std.string; // has indexOf
void main() {
string.indexOf("foo", "foo"); -> std.string.indexOf
}

No, I don't think so. If I understand the proposal correctly, it would enable
std.indexOf (which doesn't help you at all in this case), not string.indexOf.
It's trying to make it so that you can treat a symbol in a sub-module as it
were in a higher module, and string.indexOf doesn't help with that at all.

- Jonathan M Davis

Mar 30 2012
"Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 30, 2012 12:15:44 Brad Anderson wrote:
On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic <

andrej.mitrovich gmail.com> wrote:
On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
Destroy!

"That means a program that imports std.algorithm may use "std.sort"
for the symbol "std.algorithm.sort"."

That's quite interesting. Would that also mean that you could do:
import std.algorithm; // has indexOf
import std.string; // has indexOf
void main() {

string.indexOf("foo", "foo"); -> std.string.indexOf

}

I was actually kind of surprised when I found out this doesn't work. It
seems so natural to resolve ambiguity using as little context as necessary.

It would certainly be desirable in some cases, but I believe that the reason
that it doesn't work is due to the ambiguities that it would create. I'd have
to go dig up old discussions on it though to remember all of the details.

alias is supposed to solve the problem, but it doesn't really work all that
well for it, since private doesn't hide symbols, it only makes them
inaccessible (just like with C++). So, creating aliases in a module causes
problems in other modules that import that module, even if the aliases are
private. There are definitely some folks pushing for private to actually start
hiding symbols (IIRC, there's even a pull request for it), but I don't know
what the odds of convincing Walter are. If/Once that happens, alias will
actually become usable for this sort of situation, and the inability to do
string.indexOf won't be as big a deal.

- Jonathan M Davis

Mar 30 2012
"Nick Sabalausky" <a a.a> writes:
My comments:

1. My first impression was that using "foo/bar/package.d" instead of
"foo/bar.d" seemed a bit odd and messy. But I realize now that cleverly
solves the issue where "foo/bar.d" would be considered to be inside a
different package from "foo/bar/*.d". So I like that. Personally, I think I
would have gone with "foo/bar/_.d" as that sorts much, much better, but
naming debates can go on forever, and I can live with "package.d"

2. I don't understand any of this:

-------------------------------------------
When looking up the symbol "foo.bar.baz", currently an exact match is
needed. However. when looking up ".baz" or simply "baz", a flexible lookup
is used that has many advantages (less verbose, hijacking detection etc).
Therefore we think similar flexibility should be imparted to "foo.bar.baz",
as follows:

If a qualified symbol "foo.bar.baz" appears in code, the compiler considers
"foo.bar" a prefix that sets the starting point of the lookup, and then
proceeds with looking up "baz" from that starting point. That means a
program that imports std.algorithm may use "std.sort" for the symbol
"std.algorithm.sort".
-------------------------------------------

I *do* understand "a program that imports std.algorithm may use "std.sort"
for the symbol 'std.algorithm.sort'", and I think that's a good idea. It
solves a problem I hadn't even thought of. But I don't understand that stuff
I quoted above. Perhaps you could reword/clarify?

3. Other than that stuff, I'm very much in favor of this. I'll have some of
that!

Mar 30 2012
Michel Fortin <michel.fortin michelf.com> writes:
On 2012-03-30 14:46:19 +0000, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> said:

Destroy!

One thing the current system avoids is unresolvable symbols. If two
symbol name clashes, you just qualify them fully and it'll always be
unambiguous. For instance:

.std.algorithm.sort(â€¦)

Now, if std.algorithm becomes both a module and a package, you could
have both a sort function and a sort submodule with no way to
distinguish between the two, even when fully qualified. I think this is
why D does not allow modules to have the same name as packages.

I understand that you try to work around this problem by inventing a
.std.algorithm.package scope. Then you make it's content imported
automatically inside the .std.algorithm scope for backward
compatibility (and convenience). The problem is that if
.std.algorithm.package contains a sort function and there is also a
module called std.algorithm.sort, the fully-qualified name of that
'sort' module will become ambiguous. Moreover, whether the
fully-qualified name .std.algorithm.sort is ambiguous or not depends on
what modules were imported, which is not a very desirable behaviour.

So to make sure there is no unresolvable fully-qualified names, when
importing std.algorithm.sort the compiler should make sure that no
symbol called 'sort' already exist in the .std.algorithm scope (which
includes the symbols in .std.algorithm.package and all other packages
inside std.algorithm). This is clearly untenable.

- - -

I recognize the need. If I may, I'll propose something simpler:

1. allow both std/algorithm.d and std/algorithm/sort.d to exist
2. importing std.algorithm.sort will also implicitly import
std.algorithm (if it exists) and std (if it exists)
3. if any symbol of std.algorithm clash with the std.algorithm.sort
module name, you get an error when importing std.algorithm.sort.

Effectively, importing std.algorithm.sort becomes synonymous to
importing std, std.algorithm, and std.algorithm.sort. This is what's
needed to detect clashes in fully-qualified names.

The only issue now (beside a few more imports) is that if std.algorithm
imports any of its submodule it becomes a circular import. That's
usually fine in D, but not when you have module constructors.

--
Michel Fortin
michel.fortin michelf.com
http://michelf.com/

Mar 30 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Mar 2012 17:45:36 -0400, Michel Fortin
<michel.fortin michelf.com> wrote:

On 2012-03-30 14:46:19 +0000, Andrei Alexandrescu  =

<SeeWebsiteForEmail erdani.org> said:

Destroy!

One thing the current system avoids is unresolvable symbols. If two  =

symbol name clashes, you just qualify them fully and it'll always be  =

unambiguous. For instance:

.std.algorithm.sort(=E2=80=A6)

Now, if std.algorithm becomes both a module and a package, you could  =

have both a sort function and a sort submodule with no way to  =

distinguish between the two, even when fully qualified. I think this i=

s  =

why D does not allow modules to have the same name as packages.

I understand that you try to work around this problem by inventing a  =

.std.algorithm.package scope. Then you make it's content imported  =

automatically inside the .std.algorithm scope for backward compatibili=

ty  =

(and convenience). The problem is that if .std.algorithm.package  =

contains a sort function and there is also a module called  =

std.algorithm.sort, the fully-qualified name of that 'sort' module wil=

l  =

become ambiguous. Moreover, whether the fully-qualified name  =

.std.algorithm.sort is ambiguous or not depends on what modules were  =

imported, which is not a very desirable behaviour.

So this becomes an error.  I don't see this as a major problem.  Just
don't name a module sort inside std/algorithm.

This is no different than ambiguous templates, which are allowed until y=
ou
want to instantiate one.

-Steve

Apr 02 2012
Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-02 13:04:31 +0000, "Steven Schveighoffer"
<schveiguy yahoo.com> said:

On Fri, 30 Mar 2012 17:45:36 -0400, Michel Fortin

The problem is that if .std.algorithm.package
contains a sort function and there is also a module called
std.algorithm.sort, the fully-qualified name of that 'sort' module wil l
become ambiguous. Moreover, whether the fully-qualified name
.std.algorithm.sort is ambiguous or not depends on what modules were
imported, which is not a very desirable behaviour.

So this becomes an error.  I don't see this as a major problem.  Just
don't name a module sort inside std/algorithm.

This is no different than ambiguous templates, which are allowed until y ou
want to instantiate one.

If you have ambiguous templates in the same module, it'll always be
ambiguous irrespective of what you import (and you can blame the
module's designer for it). If you have ambiguous templates residing in
different modules the symbol will be unambiguous until you've imported
the second module (same as overloaded functions). At that point you can
disambiguate using the fully-qualified name of the template (or
function).

Whereas if the fully-qualified name of a module becomes ambiguous
because of a symbol in another module, there is no way to disambiguate.
All you can do is avoid importing the two conflicting modules together,
just like when you encounter two headers trying to define the same
symbol in C/C++.

With the way D modules were designed, this cannot happen because you
can't have a module with the same name as a package. I always thought
this was done on purpose, but I might be wrong. Whatever we do, I think
it'd be a nice property to preserve that fully-qualified names should
always work.

--
Michel Fortin
michel.fortin michelf.com
http://michelf.com/

Apr 02 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 02 Apr 2012 20:44:09 -0400, Michel Fortin
<michel.fortin michelf.com> wrote:

On 2012-04-02 13:04:31 +0000, "Steven Schveighoffer"
<schveiguy yahoo.com> said:

On Fri, 30 Mar 2012 17:45:36 -0400, Michel Fortin

The problem is that if .std.algorithm.package
contains a sort function and there is also a module called
std.algorithm.sort, the fully-qualified name of that 'sort' module wil
l
become ambiguous. Moreover, whether the fully-qualified name
.std.algorithm.sort is ambiguous or not depends on what modules were
imported, which is not a very desirable behaviour.

So this becomes an error.  I don't see this as a major problem.  Just
don't name a module sort inside std/algorithm.
This is no different than ambiguous templates, which are allowed until
y ou
want to instantiate one.

If you have ambiguous templates in the same module, it'll always be
ambiguous irrespective of what you import (and you can blame the
module's designer for it). If you have ambiguous templates residing in
different modules the symbol will be unambiguous until you've imported
the second module (same as overloaded functions). At that point you can
disambiguate using the fully-qualified name of the template (or
function).

Whereas if the fully-qualified name of a module becomes ambiguous
because of a symbol in another module, there is no way to disambiguate.
All you can do is avoid importing the two conflicting modules together,
just like when you encounter two headers trying to define the same
symbol in C/C++.

How does this happen?  The FQN cannot be ambiguous.

For example, if we change std/algorithm.d into:

std/algorithm/sorting.d which defines sort
std/algorithm/finding.d which defines find
std/algorithm/package.d which imports sorting.d and finding.d

Then how can importing some other module make the FQN ambiguous?  The FQN
for sort is std.algorithm.sorting.sort.  Now, if you define the symbol
sort inside std/algorithm/package.d, you have an ambiguity, but so what?
If you are responsible for the std.algorithm module, aren't you
responsible for all the files in that pseudo-package?  I don't see how it
changes things from today.  Basically, my point is the only ambiguity FQN
in the package.d file can create is with other symbols within that same
directory, therefore within the same module.

All we have to do is adopt the practice that phobos' package.d will only
publicly import other modules.  It won't define any symbols.  Then there
will always be an unambiguous FQN.

Hm... maybe you mean if you can have std/algorithm/package.d do something
like import std.file, which contains another sort function, is that now
packaged under std.algorithm.sort?  I don't think that's possible in the
given DIP.

-Steve

Apr 04 2012
Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-04 14:08:34 +0000, "Steven Schveighoffer"
<schveiguy yahoo.com> said:

On Mon, 02 Apr 2012 20:44:09 -0400, Michel Fortin
<michel.fortin michelf.com> wrote:

Whereas if the fully-qualified name of a module becomes ambiguous
because of a symbol in another module, there is no way to disambiguate.
All you can do is avoid importing the two conflicting modules
together,  just like when you encounter two headers trying to define
the same  symbol in C/C++.

How does this happen?  The FQN cannot be ambiguous.

Sure it can if I follow DIP16, because module names can become ambiguous.

Let's try this with an example. First, let's define a pretty standard module:

std/algorithm/sort.d:

module std.algorithm.sort;

void sort(T)(T[] array);

Here the fully-qualified name of the sort function is
.std.algorithm.sort.sort. But according to DIP16's lookup rules, the
sort function is also available (if not ambiguous) at:

std.sort
std.algorithm.sort

Question 1: since there is already a module at .std.algorithm.sort,
doesn't the module name become ambiguous with the sort function it
itself contains?

Let's assume the module's name take priority and does not conflict so
we can continue. Now we create the package.d file:

std/algorithm/package.d:

import std.algorithm.sort;

And now I write this somewhere in my code:

std.algorithm.sort

Question 2: does std.algorithm.sort refer to the std.algorithm.sort
*module* or to std.algorithm.package.sort *function* publicly imported
from the std.algorithm.sort module?

Again, we could decide that the module takes priority. But having
symbols take priority over one another is not how D has resolved
ambiguities up to now; what D does usually is make it a hard error. If
we make it an error the fully-qualified name of anything in the
std.algorithm.sort module becomes inaccessible. If we do not make it an
error, the module name shadows the function imported in the package.
And the problem with shadowing is that it can silently change what code
you're calling depending on what you've imported (if you need an

You might think I'm trying to split hair in four to find flaws, that no
one is going to do things that dumb, but I unfortunately think the
problematic pattern is already quite common. How many times have we
seen modules containing a class, variable, or function having the same
name as the module's name? What should happen when you publicly import
those modules in the package.d file?

The practice might not be too prevalent in Phobos because modules tend
to do a lot of things and are therefore named more generically, but it
still happens. For instance:

std.array.array
std.getopt.getopt
std.regex.regex

Say you wanted to create a package.d file directly for the whole
package std, what should be done for those?

--
Michel Fortin
michel.fortin michelf.com
http://michelf.com/

Apr 04 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
<michel.fortin michelf.com> wrote:

On 2012-04-04 14:08:34 +0000, "Steven Schveighoffer"
<schveiguy yahoo.com> said:

The FQN cannot be ambiguous.

Sure it can if I follow DIP16, because module names can become ambiguous.

Let's try this with an example. First, let's define a pretty standard
module:

std/algorithm/sort.d:

module std.algorithm.sort;

void sort(T)(T[] array);

Here the fully-qualified name of the sort function is
.std.algorithm.sort.sort. But according to DIP16's lookup rules, the
sort function is also available (if not ambiguous) at:

std.sort
std.algorithm.sort

Question 1: since there is already a module at .std.algorithm.sort,
doesn't the module name become ambiguous with the sort function it
itself contains?

OK, but when is it ever valid to refer to a module when the semantic
expectations are for something other than a module?  I can only think of
two places where module names are used, inside an import statement and
inside a module statement (three if you count the prefix of a FQN).  Maybe
I'm missing some case...

Let's assume the module's name take priority and does not conflict so we
can continue. Now we create the package.d file:

std/algorithm/package.d:

import std.algorithm.sort;

And now I write this somewhere in my code:

std.algorithm.sort

Question 2: does std.algorithm.sort refer to the std.algorithm.sort
*module* or to std.algorithm.package.sort *function* publicly imported
from the std.algorithm.sort module?

The function.  A symbol that is not specified as the module name in import
statement or in a module statement is always *not* a module.  I think our
one saving grace here is that when you want to import a specific symbol
from a module, this is not the syntax:

import std.stdio.writefln;

So there is never any ambiguity as to whether you mean a module identifier
or other symbol.

You might think I'm trying to split hair in four to find flaws, that no
one is going to do things that dumb, but I unfortunately think the
problematic pattern is already quite common. How many times have we seen
modules containing a class, variable, or function having the same name
as the module's name?

Tango anyone? :)  But yes, I think the issue really becomes, we need to
look at context when deciding the semantic meaning of a symbol.  I don't
think this violates the context-free grammar, because wouldn't this only
come into play at the semantic level?  Not a compiler writer/hacker, so I
don't know.

Say you wanted to create a package.d file directly for the whole package
std, what should be done for those?

No, let's not do that.  Ever. :)

-Steve

Apr 04 2012
Timon Gehr <timon.gehr gmx.ch> writes:
On 04/04/2012 07:53 PM, Steven Schveighoffer wrote:
On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
<michel.fortin michelf.com> wrote:

On 2012-04-04 14:08:34 +0000, "Steven Schveighoffer"
<schveiguy yahoo.com> said:

The FQN cannot be ambiguous.

Sure it can if I follow DIP16, because module names can become ambiguous.

Let's try this with an example. First, let's define a pretty standard
module:

std/algorithm/sort.d:

module std.algorithm.sort;

void sort(T)(T[] array);

Here the fully-qualified name of the sort function is
.std.algorithm.sort.sort. But according to DIP16's lookup rules, the
sort function is also available (if not ambiguous) at:

std.sort
std.algorithm.sort

Question 1: since there is already a module at .std.algorithm.sort,
doesn't the module name become ambiguous with the sort function it
itself contains?

OK, but when is it ever valid to refer to a module when the semantic
expectations are for something other than a module? I can only think of
two places where module names are used, inside an import statement and
inside a module statement (three if you count the prefix of a FQN).
Maybe I'm missing some case...

__traits(allMembers, pack.age.mod.ule);

...
You might think I'm trying to split hair in four to find flaws, that
no one is going to do things that dumb, but I unfortunately think the
problematic pattern is already quite common. How many times have we
seen modules containing a class, variable, or function having the same
name as the module's name?

Tango anyone? :) But yes, I think the issue really becomes, we need to
look at context when deciding the semantic meaning of a symbol. I don't
think this violates the context-free grammar, because wouldn't this only
come into play at the semantic level? Not a compiler writer/hacker, so I
don't know.

No symbol is resolved until semantic, but I don't think hiding the
module/package symbol if any clashing symbol in the module/any
subpackage exists is a satisfactory solution.

Apr 04 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:

On 04/04/2012 07:53 PM, Steven Schveighoffer wrote:
OK, but when is it ever valid to refer to a module when the semantic
expectations are for something other than a module? I can only think of
two places where module names are used, inside an import statement and
inside a module statement (three if you count the prefix of a FQN).
Maybe I'm missing some case...

__traits(allMembers, pack.age.mod.ule);

hm... maybe we'd have to have new __traits that would disambiguate, like
__traits(allModuleMembers...)

This doesn't seem like a huge barrier.

Tango anyone? :) But yes, I think the issue really becomes, we need to
look at context when deciding the semantic meaning of a symbol. I don't
think this violates the context-free grammar, because wouldn't this only
come into play at the semantic level? Not a compiler writer/hacker, so I
don't know.

No symbol is resolved until semantic, but I don't think hiding the
module/package symbol if any clashing symbol in the module/any
subpackage exists is a satisfactory solution.

Then we must come up with a way to hide the submodules of a virtual module.

I tried this, which fails on the current compiler:

import x;

int x;

So it seems we have two choices here:

1. DIP16 needs to get more complex to make package submodules not
accesible as individual modules.
2. Start contextually interpreting identifiers at least in the case of
modules vs. non-modules.

I'd suggest option 2 allows for better backwards compatibility and more
flexibility.

-Steve

Apr 04 2012
Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-04 19:48:32 +0000, "Steven Schveighoffer"
<schveiguy yahoo.com> said:

On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:

No symbol is resolved until semantic, but I don't think hiding the
module/package symbol if any clashing symbol in the module/any
subpackage exists is a satisfactory solution.

Then we must come up with a way to hide the submodules of a virtual module.

I tried this, which fails on the current compiler:

import x;

int x;

So it seems we have two choices here:

1. DIP16 needs to get more complex to make package submodules not
accesible as individual modules.
2. Start contextually interpreting identifiers at least in the case of
modules vs. non-modules.

I'd suggest option 2 allows for better backwards compatibility and more
flexibility.

I don't think option 2 is realistic (see my other post).

I don't think option 1 is an improvement over what we have. I mean, if
you're going to hide the submodules, what is the benefit compared to
just using a different package name for the implementation modules? You
can already refactor std.algorithm this way with no change in the
compiler:

module std.algorithm;

public import std.algorithm_impl.sort;
public import std.algorithm_impl.map;
public import std.algorithm_impl.blah_blah_blah;
â€¦

If we add a language feature, it should be an noticeable improvement
over this situation.

I think we need a third option.

Here's an idea: we could allow modules having a single symbol with the
same name as the module to behave as if they were the symbol itself,
just like templates behaves. For instance:

module std.algorithm.sort;

void sort(int[] t);

Now you can import std.algorithm.sort and then use the
std.algorithm.sort fully qualified name as if it was a function, even
though it's the module name (std.algorithm.sort.sort would be the
function's name).

Or maybe we could just allow "alias sort this" at module level? Or is

--
Michel Fortin
michel.fortin michelf.com
http://michelf.com/

Apr 04 2012
Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-05 00:50:49 +0000, Michel Fortin <michel.fortin michelf.com> said:

I think we need a third option.

Here's an idea: we could allow modules having a single symbol with the
same name as the module to behave as if they were the symbol itself,
just like templates behaves. For instance:

module std.algorithm.sort;

void sort(int[] t);

Now you can import std.algorithm.sort and then use the
std.algorithm.sort fully qualified name as if it was a function, even
though it's the module name (std.algorithm.sort.sort would be the
function's name).

Or maybe we could just allow "alias sort this" at module level? Or is

Forgot to follow through with what "import std.algorithm" would do.

If std.algorithm is a package, then it should only contain modules
(which can each pose for a function/class/struct/variable thanks to the
feature described above). Importing the package std.algorithm would
open the std/algorithm.d file, read a list of package from the file,
and proceed to import each of them (just as if you'd have imported each
of them separately).

Here is an idea for declaring the list of modules to import when you
import a package in std/algorithm.d:

package std.algorithm;

module std.algorithm.sort;
module std.algorithm.map;
...

(I'm _not_ using a package.d file because that'd incur two lookups: one
for std/algorithm.d in case it's a module, and one for
std/algorithm/package.d in case it's a package. It also removes the
possibility of both files existing at the same time which would be
confusing.)

This way, modules and packages stay exactly the same in the language as
they are now, except that you can now import a package directly instead
of importing each module separately. And modules with a single symbol
with the same name as the module are treated as if they were that
symbol. This allows migrating a module to a package without breaking
the existing code base, but the drawback is that each symbol in the
module becoming a package need to become its own submodule.

--
Michel Fortin
michel.fortin michelf.com
http://michelf.com/

Apr 04 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 04 Apr 2012 20:50:49 -0400, Michel Fortin  =

<michel.fortin michelf.com> wrote:

On 2012-04-04 19:48:32 +0000, "Steven Schveighoffer"  =

<schveiguy yahoo.com> said:

On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr <timon.gehr gmx.ch>  =

wrote:

No symbol is resolved until semantic, but I don't think hiding the  =

=

module/package symbol if any clashing symbol in the module/any   =

subpackage exists is a satisfactory solution.

Then we must come up with a way to hide the submodules of a virtual =

=

module.
I tried this, which fails on the current compiler:
import x;
int x;
So it seems we have two choices here:
1. DIP16 needs to get more complex to make package submodules not   =

accesible as individual modules.
2. Start contextually interpreting identifiers at least in the case o=

f   =

modules vs. non-modules.
I'd suggest option 2 allows for better backwards compatibility and  =

more  flexibility.

I don't think option 2 is realistic (see my other post).

I saw your other post.  I see that a major issue with the way DIP16  =

intends to shortcut fully qualified names is demonstrated by this simple=
=

example:

a/b/c.d:

module a.b.c;

void foo() {}
struct c
{
static void foo() {}
}

main.d:
import a.b.c;

void main()
{
a.b.c.foo(); // cannot determine what this is.  Is it the global  =

function, or a shortcut FQN to a.b.c.c.foo?
}

This should be counter-case enough to disqualify that idea -- it will  =

break existing code without ever adding any package.d files.  Thanks for=
=

explaining.

I don't think option 1 is an improvement over what we have. I mean, if=

=

you're going to hide the submodules, what is the benefit compared to  =

just using a different package name for the implementation modules?

The advantage is you can split your logical module into submodules for  =

maintenance purposes.

You can already refactor std.algorithm this way with no change in the =

=

compiler:

module std.algorithm;

public import std.algorithm_impl.sort;
public import std.algorithm_impl.map;
public import std.algorithm_impl.blah_blah_blah;
=E2=80=A6

If we add a language feature, it should be an noticeable improvement  =

over this situation.

I agree, option 1 is not very convincing.

I think we need a third option.

Here's an idea: we could allow modules having a single symbol with the=

=

same name as the module to behave as if they were the symbol itself,  =

just like templates behaves. For instance:

module std.algorithm.sort;

void sort(int[] t);

Now you can import std.algorithm.sort and then use the  =

std.algorithm.sort fully qualified name as if it was a function, even =

=

though it's the module name (std.algorithm.sort.sort would be the  =

function's name).

Or maybe we could just allow "alias sort this" at module level? Or is =

it  =

I don't like this proposal, simply because this means one function/symbo=
l  =

per submodule.  It should be more flexible than that.  But I like the li=
ne  =

of thinking.

Let's re-examine the issue.  We need to allow splitting of a module X.d =
=

into pieces for maintenance (or possibly accessibility -- disallowing  =

friends).  But we don't want to break code which currently uses FQN to  =

access X's symbols.

I think the following might work:

algorithm.d:

import this =3D std.algorithm_impl.sort;

Which then imports std.algorithm_impl.sort, and effectively aliases all =
=

its symbols into algorithm.d.  If std.algorithm_impl.sort defines a  =

function called sort, then it's also aliased to std.algorithm.sort.  In =
=

essence, sort has *two* FQN, but there are no FQN that refer to more tha=
n  =

one symbol.

I purposely left out the package.d idea because it's orthogonal to this.=

-Steve

Apr 05 2012
Le 05/04/2012 13:46, Steven Schveighoffer a Ã©crit :
I don't like this proposal, simply because this means one
function/symbol per submodule. It should be more flexible than that. But
I like the line of thinking.

Let's re-examine the issue. We need to allow splitting of a module X.d
into pieces for maintenance (or possibly accessibility -- disallowing
friends). But we don't want to break code which currently uses FQN to
access X's symbols.

I think the following might work:

algorithm.d:

import this = std.algorithm_impl.sort;

Which then imports std.algorithm_impl.sort, and effectively aliases all
its symbols into algorithm.d. If std.algorithm_impl.sort defines a
function called sort, then it's also aliased to std.algorithm.sort. In
essence, sort has *two* FQN, but there are no FQN that refer to more
than one symbol.

I purposely left out the package.d idea because it's orthogonal to this.

-Steve

The behavior you described has been proposed for public import, and have
been discussed. This is interesting.

You propose an alternative syntax with is fine and have the advantage to
not disturb already existing public imports, but have the drawback to
create a new syntax, again.

If such a syntax is adopted, what would be the point of public imports ?
If it is still useful, then your syntax is a better choice, otherwise,
we'd better modify public import.

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix gmail.com> wrot=
e:

Le 05/04/2012 13:46, Steven Schveighoffer a =C3=A9crit :
I don't like this proposal, simply because this means one
function/symbol per submodule. It should be more flexible than that. =

But
I like the line of thinking.

Let's re-examine the issue. We need to allow splitting of a module X.=

d
into pieces for maintenance (or possibly accessibility -- disallowing=

friends). But we don't want to break code which currently uses FQN to=

access X's symbols.

I think the following might work:

algorithm.d:

import this =3D std.algorithm_impl.sort;

Which then imports std.algorithm_impl.sort, and effectively aliases a=

ll
its symbols into algorithm.d. If std.algorithm_impl.sort defines a
function called sort, then it's also aliased to std.algorithm.sort. I=

n
essence, sort has *two* FQN, but there are no FQN that refer to more
than one symbol.

I purposely left out the package.d idea because it's orthogonal to th=

is.
-Steve

The behavior you described has been proposed for public import, and ha=

ve  =

been discussed. This is interesting.

You propose an alternative syntax with is fine and have the advantage =

to  =

not disturb already existing public imports, but have the drawback to =

=

create a new syntax, again.

If such a syntax is adopted, what would be the point of public imports=

?  =

If it is still useful, then your syntax is a better choice, otherwise,=

=

we'd better modify public import.

No, public imports simply mean that you can view the publicly imported  =

module.  It does *not* add aliases to the importing module.

for example:

foo.d:

module foo;

int x;

bar.d:
module bar;
public import foo;

int y;

main.d:
import bar; // publicly imports foo.

void main()
{
foo.x =3D 5; // ok, publicly imported via bar (non public, this woul=
d be  =

an error)
bar.y =3D 5; // ok
bar.x =3D 5; // error, public import doesn't create new fully qualif=
ied  =

name for foo.x
}

With the system I propose, using the specific technique would make it so=
=

bar.x would also be valid, and would refer to foo.x

I think we need new syntax, or a new language feature, because you don't=
=

want to alter the operation of existing code.  Simply changing public  =

imports would cause lots of existing code to be reinterpreted, possibly =
in  =

a way not intended, or that would be ambiguous.

-Steve

Apr 05 2012
Timon Gehr <timon.gehr gmx.ch> writes:
On 04/05/2012 02:58 PM, Steven Schveighoffer wrote:
No, public imports simply mean that you can view the publicly imported
module. It does *not* add aliases to the importing module.

Have you tried it?

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:

On 04/05/2012 02:58 PM, Steven Schveighoffer wrote:
No, public imports simply mean that you can view the publicly imported
module. It does *not* add aliases to the importing module.

Have you tried it?

I just did.  OK, what the hell are we arguing about then?!

DIP16 is worksforme :)

See this part of the spec:

http://dlang.org/module.html

Read the part on public modules.  You may understand why I didn't know
about that "feature" (which seems to work on all the installed compilers I
have, back to 2.033).  I just read the public import part of TDPL, and
there it is, all spelled out quite nicely.  I'm going to file a bug
against the spec...  grrr...

I'm now firmly in the "we don't need to change anything, just refactor the
modules and use public import" camp.  We don't even need to change
ANYTHING in the compiler!  Forget everything I said before in this thread
;)

I suppose the only thing we don't get is being able to have a module and a
package with the same FQN.  I don't see that being a major issue.

-Steve

Apr 05 2012
"Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:
On 04/05/2012 02:58 PM, Steven Schveighoffer wrote:
No, public imports simply mean that you can view the publicly imported
module. It does *not* add aliases to the importing module.

Have you tried it?

I just did. OK, what the hell are we arguing about then?!

DIP16 is worksforme :)

See this part of the spec:

http://dlang.org/module.html

Read the part on public modules. You may understand why I didn't know
about that "feature" (which seems to work on all the installed compilers I
have, back to 2.033). I just read the public import part of TDPL, and
there it is, all spelled out quite nicely. I'm going to file a bug
against the spec... grrr...

I'm now firmly in the "we don't need to change anything, just refactor the
modules and use public import" camp. We don't even need to change
ANYTHING in the compiler! Forget everything I said before in this thread
;)

I suppose the only thing we don't get is being able to have a module and a
package with the same FQN. I don't see that being a major issue.

What doesn't work is being able to turn a module into a package with the same
name. Right now, we could create a std.alg package with sub-modules containing
all of std.algorithm's functionality and change std.algorithm to pubicly
import them all, but you can't turn std.algorithm itself into a package
without breaking code. The package.d portion of the proposal makes it so that
you can.

Now, public import's current behavior is _almost_ enough to make the second
part completely unnecessary. The only change that would be required would be
to make it so that std.algorithm.x looks in std/algorithm/package.d if there's
no std.algorithm.x module, because otherwise the public imports in package.d
would be putting all of the symbols in std.algorithm.package, not
std.algorithm (that, and package.d isn't legal at present).

So, the whole point of this proposal - to seemlessly allow the transition of a
module to a package in place - _does_ require a language/compiler change. But
moving stuff from a module to a new package does work already.

- Jonathan M Davis

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 14:24:10 -0400, Jonathan M Davis <jmdavisProg gmx.com>
wrote:

On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr gmx.ch> I
suppose the only thing we don't get is being able to have a module and a
package with the same FQN. I don't see that being a major issue.

What doesn't work is being able to turn a module into a package with the
same
name. Right now, we could create a std.alg package with sub-modules
containing
all of std.algorithm's functionality and change std.algorithm to pubicly
import them all, but you can't turn std.algorithm itself into a package
without breaking code.

But so what?  nobody has any code like:

import std.algorithm.sort;

So who cares where that module goes?  I agree it would be ideal to put it
there, but I don't think it's strictly necessary.  And there is no need
for the shortcut for fully qualified names.

So, the whole point of this proposal - to seemlessly allow the
transition of a
module to a package in place - _does_ require a language/compiler change.

I don't see how.  Just move the code into another module and publicly
import that module from std/algorithm.d.  Problem pretty much solved.

BTW, importing a directory was already proposed in DIP15.
http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

-Steve

Apr 05 2012
"Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 14:24:10 -0400, Jonathan M Davis <jmdavisProg gmx.com>

wrote:
On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr gmx.ch> I
suppose the only thing we don't get is being able to have a module and a
package with the same FQN. I don't see that being a major issue.

What doesn't work is being able to turn a module into a package with the
same
name. Right now, we could create a std.alg package with sub-modules
containing
all of std.algorithm's functionality and change std.algorithm to pubicly
import them all, but you can't turn std.algorithm itself into a package
without breaking code.

But so what? nobody has any code like:

import std.algorithm.sort;

So who cares where that module goes? I agree it would be ideal to put it
there, but I don't think it's strictly necessary. And there is no need
for the shortcut for fully qualified names.

So, the whole point of this proposal - to seemlessly allow the
transition of a
module to a package in place - _does_ require a language/compiler change.

I don't see how. Just move the code into another module and publicly
import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm (or
std.datetime or whatever) into multiple modules, you have to create a new
package with a completely different name with no connection to the original
save for the fact that the original publicly imports it. For instance, with
the work that I've done thus far on splitting std.datetime, I've had to create
a std.dtime package to hold the modules and have std.datetime pubicly import
them. This automatically creates the issue of what the difference between them
is (for anyone new to Phobos) and does not indicate their relation at all in
the hierarchy. It would be much cleaner to be able to turn std/datetime.d into
std/datetime/ with a package.d in it along with the new modules.

No, we don't _have_ to do something to make it so that std.algorithm can be
turned into std/algorithm/ while still not breaking code. But it would be very
nice. Certainly, I don't understand why this DIP would ever have been proposed
if Andrei didn't find it valuable.

BTW, importing a directory was already proposed in DIP15.
http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

Yes, but having a package.d with the public imports gives you much finer-
grained control over what gets imported, and DIP15 doesn't solve the problem
of fully qualified uses of std.alorgithm.sort not breaking when
std.algorithm.sort gets moved to something like std.algorithm.sorting.d. So,
DIP15 doesn't work as a means of seemlessly breaking up a module. It just
_mostly_ works as one (since people usually don't fully qualify symbols).
Also, package.d would give us a means for documenting a package, which I would
very much like to be able to do. Having std.datetime give an overview of the
std.dtime package is definitely worse than having a means of having the package
document itself - which the package.d file should be able to give us.

The package.d portion of DIP16 allows a means of controlling what importing a
package would mean, it provides a means of turning a module into package in
place, and it potentially provides a way of documenting a package - all of
which are valuable. Whether they're valuable enough to merit a language change
is obviously up for debate, but certainly, as the designer and primary
maintainer of one of the main targets for being split up, I very much like the
idea of being able to split up a module in place rather than having to create
a new package with a new name with no obvious relation to the original.

- Jonathan M Davis

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis <jmdavisProg gmx.com>
wrote:

On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:

I don't see how. Just move the code into another module and publicly
import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm (or
std.datetime or whatever) into multiple modules, you have to create a new
package with a completely different name with no connection to the
original
save for the fact that the original publicly imports it.

My view is that people will not import the smaller modules, they will only
ever import std.algorithm.

Look at std.bigint, which imports modules from std.internal.math.  Nobody
ever imports std.internal.math.??? because they just import std.bigint.

BTW, importing a directory was already proposed in DIP15.
http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

Yes, but having a package.d with the public imports gives you much finer-
grained control over what gets imported, and DIP15 doesn't solve the
problem
of fully qualified uses of std.alorgithm.sort not breaking when
std.algorithm.sort gets moved to something like std.algorithm.sorting.d.

I think you are forgetting that all current code imports std.algorithm,
which will register the symbol std.algorithm.sort via public import.  No
code will break, even if it uses FQN.

The only way to "break" code is to write new import statements.  I don't
think anyone will do that unknowingly.

-Steve

Apr 05 2012
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/5/12 4:26 PM, Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis
<jmdavisProg gmx.com> wrote:

On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:

I don't see how. Just move the code into another module and publicly
import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm (or
std.datetime or whatever) into multiple modules, you have to create a new
package with a completely different name with no connection to the
original
save for the fact that the original publicly imports it.

My view is that people will not import the smaller modules, they will
only ever import std.algorithm.

I think we should be looking for a solution that not only allows
replacing module -> package transparently, but also allows people to
import the newly introduced fine-grained modules.

Andrei

Apr 05 2012
"Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 16:43:24 Andrei Alexandrescu wrote:
On 4/5/12 4:26 PM, Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis

<jmdavisProg gmx.com> wrote:
On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:
I don't see how. Just move the code into another module and publicly
import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm (or
std.datetime or whatever) into multiple modules, you have to create a new
package with a completely different name with no connection to the
original
save for the fact that the original publicly imports it.

My view is that people will not import the smaller modules, they will
only ever import std.algorithm.

I think we should be looking for a solution that not only allows
replacing module -> package transparently, but also allows people to
import the newly introduced fine-grained modules.

Yeah. If all we want to do is continue to always import std.algorithm, then
the DIP is more or less pointless. It's the splitting of the API among
multiple packages while allowing the programmer to either call/import it like
he has been or to call/import it from the new module explicitly that the DIP
is trying to enable.

If we make it possible to split std.algorithm into multiple modules in place,
then we avoid breaking code, keep the code organized in the same hierarchy -
only more detailed - and allow the programmer to import on either a roughly or
finely grained level, depending on which they prefer. And I really like how
this could enable us to have package-specific documentation, whereas all
documentation is currently module-specific and doesn't enable us to provide a
page which gaves an overview of a package. That's not always necessary, but
there are times when it would be quite nice (e.g. std.datetime).

- Jonathan M Davis

Apr 05 2012
Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-05 21:43:24 +0000, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> said:

I think we should be looking for a solution that not only allows
replacing module -> package transparently, but also allows people to
import the newly introduced fine-grained modules.

I think it'd be valuable too. But how do you do that without creating
ambiguous fully qualified names?

One way would be that by importing a module inside a package, the
compiler would check first that the package file contains no symbol
with that module name. In other words, if you import
std.algorithm.sorting, it'd check that "algorithm" is not a symbol
defined in the "std" package file (if it exists for std) and that no
"sorting" symbol exists in the "std.algorithm" package file. That would
work. But the problem is that if the package file publicly imports all
of its submodules, then you'll need to parse all the imported files to
determine if the module you're trying to import conflicts with a
package-level symbol.

A more practical option would be limit the package files to only two things:

1. a list of modules to import when you import the package (importing
the package would be akin importing all modules in this list)

2. a list of symbols aliased in the package's namespace (aliased
symbols can be accessed using the package name as a prefix and through
selective imports)

Since the list of aliases is stored directly in the package file,
reading the package file to get the names of each alias is enough to
tell there is no conflict with the module name when you're trying to
import it. No need to open the other modules the package refers to, or
to resolve the symbols the aliases refer to: you only need the name of
each alias to verify there is no conflict.

For instance, a package file could look like this:

package std.algorithm;

// modules to import when a file is importing the package
// (those are not imported inside this package's namespace)
module std.algorithm.sorting;
module std.algorithm.mapping;
...

// symbols to alias to this package namespace
// (importing a module with one of these names is an error)
alias std.algorithm.sorting.completeSort  completeSort;
alias std.algorithm.sorting.isSorted      isSorted;
alias std.algorithm.sorting.partialSort   partialSort;
alias std.algorithm.sorting.schwartzSort  schwartzSort;
alias std.algorithm.sorting.sort          sort;
...

The drawback is that it's hard to maintain the list of fully-qualified
names up to date if you change it in the various modules. But if the
goal is only to preserve backward compatibility, creating that list of
aliases could be a one-time thing. New symbols would continue to be
imported when you import std.algorithm, but they'd not be available
directly under std.algorithm: you'd need to use
std.algorithm.sorting.newSort if you find yourself in a situation that
requires a fully-qualified name.

--
Michel Fortin
michel.fortin michelf.com
http://michelf.com/

Apr 05 2012
Le 06/04/2012 01:32, Michel Fortin a écrit :
On 2012-04-05 21:43:24 +0000, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> said:

I think we should be looking for a solution that not only allows
replacing module -> package transparently, but also allows people to
import the newly introduced fine-grained modules.

I think it'd be valuable too. But how do you do that without creating
ambiguous fully qualified names?

It isn't possible. But as already mentioned, all name doesn't make sense
in all situation, so most of the time, disambiguation can be done.

Plus, we want to be able to split module when they grow, and in such a
situation, collisions will never happen, because all symbols comes from
the same module in a first place.

Apr 06 2012
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/6/12 4:43 AM, deadalnix wrote:
Le 06/04/2012 01:32, Michel Fortin a écrit :
On 2012-04-05 21:43:24 +0000, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> said:

I think we should be looking for a solution that not only allows
replacing module -> package transparently, but also allows people to
import the newly introduced fine-grained modules.

I think it'd be valuable too. But how do you do that without creating
ambiguous fully qualified names?

It isn't possible. But as already mentioned, all name doesn't make sense
in all situation, so most of the time, disambiguation can be done.

Plus, we want to be able to split module when they grow, and in such a
situation, collisions will never happen, because all symbols comes from
the same module in a first place.

One other desirable feature is library distribution. Library writers
should be able to offer library xyz as a sheer directory. Then library
users should be able to import path.to.xyz and simply get going, or even
rename xyz to xyz1 and import path.to.xyz1 without a problem.

Andrei

Apr 06 2012
Le 06/04/2012 16:49, Andrei Alexandrescu a écrit :
On 4/6/12 4:43 AM, deadalnix wrote:
Le 06/04/2012 01:32, Michel Fortin a écrit :
On 2012-04-05 21:43:24 +0000, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> said:

I think we should be looking for a solution that not only allows
replacing module -> package transparently, but also allows people to
import the newly introduced fine-grained modules.

I think it'd be valuable too. But how do you do that without creating
ambiguous fully qualified names?

It isn't possible. But as already mentioned, all name doesn't make sense
in all situation, so most of the time, disambiguation can be done.

Plus, we want to be able to split module when they grow, and in such a
situation, collisions will never happen, because all symbols comes from
the same module in a first place.

One other desirable feature is library distribution. Library writers
should be able to offer library xyz as a sheer directory. Then library
users should be able to import path.to.xyz and simply get going, or even
rename xyz to xyz1 and import path.to.xyz1 without a problem.

Andrei

Good point. That is an argument for package.d, all.d or _.d .

Apr 06 2012
On 4/6/12 10:52 AM, deadalnix wrote:
Good point. That is an argument for package.d, all.d or _.d .

Or [packagename].d, where [packagename] is the name of its sibling
folder (std/algorithm.d in the case of std/algorithm/sort.d).

Apr 06 2012
Le 06/04/2012 20:03, David Gileadi a écrit :
On 4/6/12 10:52 AM, deadalnix wrote:
Good point. That is an argument for package.d, all.d or _.d .

Or [packagename].d, where [packagename] is the name of its sibling
folder (std/algorithm.d in the case of std/algorithm/sort.d).

No, because in this case, this isn't in the same folder. Still I prefer
that opetion because :

1/ You mostly distribute lib as archives, or in a managed way (package
manager, build system, etc . . .)
2/ It is simpler as import a.b.c always does the same thing, and other
solution introduce complexity in that process, and possible cases of error.

Apr 06 2012
Le 05/04/2012 23:43, Andrei Alexandrescu a Ã©crit :
On 4/5/12 4:26 PM, Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis
<jmdavisProg gmx.com> wrote:

On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:

I don't see how. Just move the code into another module and publicly
import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm
(or
std.datetime or whatever) into multiple modules, you have to create a
new
package with a completely different name with no connection to the
original
save for the fact that the original publicly imports it.

My view is that people will not import the smaller modules, they will
only ever import std.algorithm.

I think we should be looking for a solution that not only allows
replacing module -> package transparently, but also allows people to
import the newly introduced fine-grained modules.

Andrei

Why not limit name collision to name which make sense ?

For instance, import std.a.b.c is a module. if it refers also to a
function, this import doesn't make any sense, so, even if we have a name
collision, this isn't a big deal (except maybe for reflection ?).

Same goes for std.a.b.c(); which is a function call, and obviously not
the module. Here what I propose to resolve names :

1/ import does always find the .d corresponding file. No exception.
2/ Module a.b.c is in package a, a.b and a.b.c . Any package declaration
in a.b.c match package a.b (one level is removed).
3/ When a name is used in the code and have to be resolved, the
following process occurs :
- The compiler find all stuff that have this name.
- The compiler discard all stuffs that have this name and doesn't make
sense.
- If all remaining items are overload of the same item, then standard
best match rule apply.
- If all remaining items aren't in the same module, or overload or
different items, an error occurs. This is never a problem in the case of
big modules splitted in submodules.

Some examples :

a.d

public import a.b;  // import a/b.d

class b {
static void foo() {}
}

****************

a/b.d

public import a.b.c;  // import a/b/c.d

void foo(int i) {}

****************

a/b/c.d

void foo() {}

****************

main.d

import a;

foo();  // foo from a/b/c.d
foo(2);  // foo from a/b.d
a.foo();  // foo from a/b/c.d
a.b.foo();  // Error, match both a/b.d and a/b/c.d
a.b.foo(2);  // foo from a/b.d
a.b.c.foo();  // foo from a/b/c.d

Apr 06 2012
Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-06 09:41:27 +0000, deadalnix <deadalnix gmail.com> said:

Why not limit name collision to name which make sense ?

You're proposing that fully qualified names sometime work and sometime
do not work based on what you've imported and the context of usage.
This makes any FQN conflict hard to detect, because the error will be
dependent on the context in which you use the name.

You say it'll never happen for modules spitted in submodules, but I
disagree. Someday, someone will add a symbol to a submodule without
checking every package/module file underneath. Unit tests in the module
won't catch the problem because for them to catch the problem the
parent module/package has to be imported. Only on certain usages, with
the right imports and in the right context will the error occur.

In fact, if you add disambiguation based on what "makes sense" you're
making the error more obscure and harder to detect in the first place
because sometime it'll occur and sometime not depending on hard to
predict conditions.

If we allow arbitrary symbols to live inside a package, any conflict in
FQN should be caught as soon as possible, ideally as soon as you
compile after having added the symbol, and even if you don't use the
symbol anywhere. Just like how you get an error today when you try to
import a module matching a known package name, you shouldn't be allowed
to import a module matching a known symbol name, or create a symbol
matching a module fully qualified name.

--
Michel Fortin
michel.fortin michelf.com
http://michelf.com/

Apr 06 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 17:43:24 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

On 4/5/12 4:26 PM, Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis
<jmdavisProg gmx.com> wrote:

On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:

I don't see how. Just move the code into another module and publicly
import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm
(or
std.datetime or whatever) into multiple modules, you have to create a
new
package with a completely different name with no connection to the
original
save for the fact that the original publicly imports it.

My view is that people will not import the smaller modules, they will
only ever import std.algorithm.

I think we should be looking for a solution that not only allows
replacing module -> package transparently, but also allows people to
import the newly introduced fine-grained modules.

Of course you *can* do this.  I think you mean "and allows one to refer to
the fine grained module as if it were imported from the top-level module".

I don't think that benefit is very great.  Why shouldn't we expect people
to use the module name they actually imported to refer to a module?  If
you want selective import, we have:

import std.algorithm : sort;

I think the real benefit to splitting up the module comes from being able
to draw clear lines between which pieces of a large module are related.

I feel like most people will still import the main package module, and not
the submodules.  I don't think I ever wrote a piece of java code that
didn't have:

import java.io.*;

I keep coming back to std.container.  Already it's large, and full of
unrelated types.  It's only going to get worse.

Now, I fully agree that having some way to import a package by itself
without having to import all its modules would be ideal (as well as
splitting a large module into submodules that live in the same
namespace).  I think DIP15 has that covered.

-Steve

Apr 06 2012
Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote:
I feel like most people will still import the main package module, and not
the submodules.  I don't think I ever wrote a piece of java code that
didn't have:

import java.io.*;

Which is actually considered bad practice in Java, though a lot of people do
like to do it. What's generally considered good practice is to explicitly
import every module/class, and Eclipse likes to underline imports in red if
you don't.

I keep coming back to std.container.  Already it's large, and full of
unrelated types.  It's only going to get worse.

Now, I fully agree that having some way to import a package by itself
without having to import all its modules would be ideal (as well as
splitting a large module into submodules that live in the same
namespace).  I think DIP15 has that covered.

DIP15 doesn't fix the explicit path problem though. You can't change
std/algorithm.d into std/algorithm/ (with sorting.d, search.d, etc.) without
breaking code. You could make std/algorithm.d publicly import std/alg/* and
then DIP15 would allow you to import std.alg to get all of its sub-modules,
but you're still forced to use a module to publicly import symbols as part of
a migration path, and you can't split a module in place.

DIP15 also less powerful than having a package.d, since it'll automatically
import ever public symbol when it may be desirable to have it only import a
subset of them or to publicly import something from another module (e.g. as
part of a migration path - like moving the benchmarking code in std.datetime
to std.benchmark rather than lumping it with the std.datetime stuff when it's
split into separate modules). That may or may not be a big deal, but it does
mean that DIP15 isn't as powerful as DIP16's package.d solution in that
regard.

DIP15 also doesn't give a means of documentating a package. By having a
package.d file, we have a really good place to provide an overarching
description for the package in ddoc.

So, while I don't like the second part of DIP16 at all (i.e. the std.sort
stuff), I think that the package.d part is a much better way to go.

- Jonathan M Davis

Apr 06 2012
Jacob Carlborg <doob me.com> writes:
On 2012-04-07 02:25, Jonathan M Davis wrote:
On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote:
I feel like most people will still import the main package module, and not
the submodules.  I don't think I ever wrote a piece of java code that
didn't have:

import java.io.*;

Which is actually considered bad practice in Java, though a lot of people do
like to do it. What's generally considered good practice is to explicitly
import every module/class, and Eclipse likes to underline imports in red if
you don't.

import foo.bar.*

Is used all over the place in SWT.

--
/Jacob Carlborg

Apr 07 2012
Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, April 07, 2012 18:45:15 Jacob Carlborg wrote:
On 2012-04-07 02:25, Jonathan M Davis wrote:
On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote:
I feel like most people will still import the main package module, and
not
the submodules.  I don't think I ever wrote a piece of java code that
didn't have:

import java.io.*;

Which is actually considered bad practice in Java, though a lot of people
do like to do it. What's generally considered good practice is to
explicitly import every module/class, and Eclipse likes to underline
imports in red if you don't.

import foo.bar.*

Is used all over the place in SWT.

Like I said, some people do like to do it, but Eclipse doesn't like you to,
and there are quite a few Java folks who argue that it's bad practice. I
forget what the reasons were (maybe increased buld times due to pulling in
more symbols or more issues with symbol conflicts - I don't recall), but I
wasn't particularly convinced when I heard them. Regardless though, there's a
definite contingent against importing with * in the Java community, and it was
my understanding that that contigent was the majority of that community, but I
don't know.

- Jonathan M Davis

Apr 07 2012
"Kapps" <opantm2+spam gmail.com> writes:
On Saturday, 7 April 2012 at 17:44:20 UTC, Jonathan M Davis wrote
Like I said, some people do like to do it, but Eclipse doesn't
like you to,
and there are quite a few Java folks who argue that it's bad
practice. I
forget what the reasons were (maybe increased buld times due to
pulling in
more symbols or more issues with symbol conflicts - I don't
recall), but I
wasn't particularly convinced when I heard them. Regardless
though, there's a
definite contingent against importing with * in the Java
community, and it was
my understanding that that contigent was the majority of that
community, but I
don't know.

- Jonathan M Davis

Their reasoning sounds more due to various packages reinventing
things or being poorly split than actual flaws with package
imports themselves. Besides, D already addresses the issues of
indicating which one you want, without the horribly long package
names that Java has. Seems to me their examples are more like
'import std.*' than 'import std.datetime.*'.

Apr 07 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 06 Apr 2012 20:25:23 -0400, Jonathan M Davis <jmdavisProg gmx.com>
wrote:

DIP15 doesn't fix the explicit path problem though. You can't change
std/algorithm.d into std/algorithm/ (with sorting.d, search.d, etc.)
without
breaking code. You could make std/algorithm.d publicly import std/alg/*
and
then DIP15 would allow you to import std.alg to get all of its
sub-modules,
but you're still forced to use a module to publicly import symbols as
part of
a migration path, and you can't split a module in place.

I think either you or I am missing something.

In DIP15, if you define std/algorithm/_.d, and then import std.algorithm,
it imports std/algorithm/_.d, which then 1. publicly imports other
modules, and 2. aliases symbols to the name std.algorithm.symbol.  At
least, this is how I understand the intent.  It seems equivalent to me to
the package.d proposal, it's just using _.d instead of package.d.

If you import std.algorithm.sorting, and try and use std.algorithm.sort,
yes it will not work.  But this does not break existing code (which does
not import std.algorithm.sorting), and I find it odd that we want to make
std.algorithm.sort work if you don't import std.algorithm.

-Steve

Apr 09 2012
"Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, April 09, 2012 08:55:27 Steven Schveighoffer wrote:
On Fri, 06 Apr 2012 20:25:23 -0400, Jonathan M Davis <jmdavisProg gmx.com>

wrote:
DIP15 doesn't fix the explicit path problem though. You can't change
std/algorithm.d into std/algorithm/ (with sorting.d, search.d, etc.)
without
breaking code. You could make std/algorithm.d publicly import std/alg/*
and
then DIP15 would allow you to import std.alg to get all of its
sub-modules,
but you're still forced to use a module to publicly import symbols as
part of
a migration path, and you can't split a module in place.

I think either you or I am missing something.

In DIP15, if you define std/algorithm/_.d, and then import std.algorithm,
it imports std/algorithm/_.d, which then 1. publicly imports other
modules, and 2. aliases symbols to the name std.algorithm.symbol. At
least, this is how I understand the intent. It seems equivalent to me to
the package.d proposal, it's just using _.d instead of package.d.

If you import std.algorithm.sorting, and try and use std.algorithm.sort,
yes it will not work. But this does not break existing code (which does
not import std.algorithm.sorting), and I find it odd that we want to make
std.algorithm.sort work if you don't import std.algorithm.

Okay. I reread DIP15 again. I guess that I scanned over it too quickly before
and/or misremembered it. I had understood that it was proposing that
importing std.algorithm where std.algorithm was a package would be the
equivalent of importing std.algorithm.* in Java and that there were no extra
files involved. So clearly, I've been misunderstanding things here.

So, yeah. DIP15 is basically the same as DIP16 except without the std.sort
nonsense and the fact that it uses _.d instead of package.d. Using package.d
has the advantage of package being a keyword, making it so that no one is
going to accidentally create a module that will be treated specially, but it
has the downside of likely requiring more special handling by the compiler. I
don't really care which we pick though.

My main point though, misunderstandings aside, is that it would be _really_
nice to be able to split up Â a package in place and that without an
enhancement of some kind, we can't do Â that without breaking code. DIP15
appears to fit the bill quite nicely in that regard though. The part of
DIP16 which is really bad is the std.sort stuff which. Public importing
combined with either the first part of DIP16 or with DIP15 seems to take
care of the problem quite nicely.

- Jonathan M Davis

Apr 09 2012
Le 05/04/2012 14:58, Steven Schveighoffer a Ã©crit :
On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix gmail.com> wrote:

Le 05/04/2012 13:46, Steven Schveighoffer a Ã©crit :
I don't like this proposal, simply because this means one
function/symbol per submodule. It should be more flexible than that. But
I like the line of thinking.

Let's re-examine the issue. We need to allow splitting of a module X.d
into pieces for maintenance (or possibly accessibility -- disallowing
friends). But we don't want to break code which currently uses FQN to
access X's symbols.

I think the following might work:

algorithm.d:

import this = std.algorithm_impl.sort;

Which then imports std.algorithm_impl.sort, and effectively aliases all
its symbols into algorithm.d. If std.algorithm_impl.sort defines a
function called sort, then it's also aliased to std.algorithm.sort. In
essence, sort has *two* FQN, but there are no FQN that refer to more
than one symbol.

I purposely left out the package.d idea because it's orthogonal to this.

-Steve

The behavior you described has been proposed for public import, and
have been discussed. This is interesting.

You propose an alternative syntax with is fine and have the advantage
to not disturb already existing public imports, but have the drawback
to create a new syntax, again.

If such a syntax is adopted, what would be the point of public imports
? If it is still useful, then your syntax is a better choice,
otherwise, we'd better modify public import.

No, public imports simply mean that you can view the publicly imported
module. It does *not* add aliases to the importing module.

for example:

foo.d:

module foo;

int x;

bar.d:
module bar;
public import foo;

int y;

main.d:
import bar; // publicly imports foo.

void main()
{
foo.x = 5; // ok, publicly imported via bar (non public, this would be
an error)
bar.y = 5; // ok
bar.x = 5; // error, public import doesn't create new fully qualified
name for foo.x
}

With the system I propose, using the specific technique would make it so
bar.x would also be valid, and would refer to foo.x

I think we need new syntax, or a new language feature, because you don't
want to alter the operation of existing code. Simply changing public
imports would cause lots of existing code to be reinterpreted, possibly
in a way not intended, or that would be ambiguous.

-Steve

I know. I think you answered too fast.

I wasn't stating that public import are the same as what you propose. I
was discussing the fact that both probably answer the same need and that
2 different syntax is something to avoid.

Code wouldn't be broken in unexpected way, because thing will cause
either a compile error (collision of names, but the problem exists with
other solutions too) or is illegal in the current shape of the language.

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:26:16 -0400, deadalnix <deadalnix gmail.com> wrot=
e:

Le 05/04/2012 14:58, Steven Schveighoffer a =C3=A9crit :
On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix gmail.com>  =

wrote:

Le 05/04/2012 13:46, Steven Schveighoffer a =C3=A9crit :

No, public imports simply mean that you can view the publicly importe=

d
module. It does *not* add aliases to the importing module.

[snip]

I know. I think you answered too fast.

I wasn't stating that public import are the same as what you propose. =

I  =

was discussing the fact that both probably answer the same need and th=

at  =

2 different syntax is something to avoid.

Code wouldn't be broken in unexpected way, because thing will cause  =

either a compile error (collision of names, but the problem exists wit=

h  =

other solutions too) or is illegal in the current shape of the languag=

e.

My assertion above is 100% *WRONG*.  See this bug I just filed.   =

http://d.puremagic.com/issues/show_bug.cgi?id=3D7830

In fact public imports today are *exactly* the same as what I proposed. =
I  =

I think we are (well, at least I am) on a wild goose chase here.  All th=
at  =

is left to discuss is if/how we want to allow importing of packages.

But I think DIP15 already covers that  =

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

I think DIP16 is worksforme/invalid, and should be closed.

-Steve

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix gmail.com> wrote:

If such a syntax is adopted, what would be the point of public imports ?
If it is still useful, then your syntax is a better choice, otherwise,
we'd better modify public import.

Another way to look at it:

A public import saves you from having to import dependent modules that you
should be importing anyway.
A 'this' import treats the other modules as if they were actually part of
the imported module in terms of namespace.

To give an example, std.range imports std.array.  std.array is a module
all on its own, and has a specific set of functionality.  std.range has a
different set of functionality, but you would never want to have std.range
imported without also importing std.array.  Another example would be a
derived class module publicly importing the base class module.

On the other hand, something like std.container could have 15 different
container types in it.  Each of these container types should really live
in their own module, in terms of maintenance and separation of private
data.  For example, std.container.RedBlackTree has no business accessing
the private members of std.container.Array.  But because they live in the
same module, it can.  However, we don't want to define
std.container.RedBlackTree.RedBlackTree in terms of namespace.  So we have
one of these combining modules, and everything still lives in the
std.container namespace, but we get all the benefits of separating the
code into individual modules.

-Steve

Apr 05 2012
Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-05 11:46:38 +0000, "Steven Schveighoffer"
<schveiguy yahoo.com> said:

I don't like this proposal, simply because this means one function/symbo l
per submodule.  It should be more flexible than that.  But I like the li ne
of thinking.

I have the same reserve about it's lack of flexibility too. It would be
a nice thing to have for all those libraries having one module per
class as it'd reduce the length of the fully qualified name you have to
use when you need to disambiguate. But for moving a module like
std.algorithm to a package, it's cumbersome.

Let's re-examine the issue.  We need to allow splitting of a module X.d
into pieces for maintenance (or possibly accessibility -- disallowing
friends).  But we don't want to break code which currently uses FQN to
access X's symbols.

I think the following might work:

algorithm.d:

import this = std.algorithm_impl.sort;

Which then imports std.algorithm_impl.sort, and effectively aliases all
its symbols into algorithm.d.  If std.algorithm_impl.sort defines a
function called sort, then it's also aliased to std.algorithm.sort.  In
essence, sort has *two* FQN, but there are no FQN that refer to more tha n
one symbol.

This is what a public import should do.

--
Michel Fortin
michel.fortin michelf.com
http://michelf.com/

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:33:14 -0400, Michel Fortin
<michel.fortin michelf.com> wrote:

On 2012-04-05 11:46:38 +0000, "Steven Schveighoffer"
<schveiguy yahoo.com> said:

I don't like this proposal, simply because this means one
function/symbo l
per submodule.  It should be more flexible than that.  But I like the
li ne
of thinking.

I have the same reserve about it's lack of flexibility too. It would be
a nice thing to have for all those libraries having one module per class
as it'd reduce the length of the fully qualified name you have to use
when you need to disambiguate.

I see the point you are going for.  I don't really like the
one-class-per-module style as much as how phobos has one module per
category.  But that's not an excuse not to examine this possibility.

I think it's a different issue than what DIP16/DIP15 is trying to solve.

But for moving a module like std.algorithm to a package, it's cumbersome.

Not really.  Just move the files to another directory (i.e.
std/internal/algorithm) and publicly import them.

Let's re-examine the issue.  We need to allow splitting of a module X.d
into pieces for maintenance (or possibly accessibility -- disallowing
friends).  But we don't want to break code which currently uses FQN to
access X's symbols.
I think the following might work:
algorithm.d:
import this = std.algorithm_impl.sort;
Which then imports std.algorithm_impl.sort, and effectively aliases all
its symbols into algorithm.d.  If std.algorithm_impl.sort defines a
function called sort, then it's also aliased to std.algorithm.sort.  In
essence, sort has *two* FQN, but there are no FQN that refer to more
tha n
one symbol.

This is what a public import should do.

I wasn't aware of this.  Mostly because the spec omits that feature.  I
have to retract almost everything I've argued, because we already have the
means to fix what I think DIP16 is trying to solve without any compiler
changes.

-Steve

Apr 05 2012
Le 05/04/2012 16:47, Steven Schveighoffer a Ã©crit :
But for moving a module like std.algorithm to a package, it's cumbersome.

Not really. Just move the files to another directory (i.e.
std/internal/algorithm) and publicly import them.

std/internal isn't good. If I just want to import sort, I would have to
do std.internal.algorithm.sort, which isn't good.

Plus, the package accessibility modifier would be broken with such a
pattern.

The all.d, package.d, _.d, or, as I propose foldername.d (and submodules
into foldername) do not break package accessibility modifier, which is
better.

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:53:34 -0400, deadalnix <deadalnix gmail.com> wrot=
e:

Le 05/04/2012 16:47, Steven Schveighoffer a =C3=A9crit :
But for moving a module like std.algorithm to a package, it's  =

cumbersome.

Not really. Just move the files to another directory (i.e.
std/internal/algorithm) and publicly import them.

std/internal isn't good. If I just want to import sort, I would have t=

o  =

do std.internal.algorithm.sort, which isn't good.

import std.algorithm : sort;

BTW, this doesn't work today, I'll file a bug.

Plus, the package accessibility modifier would be broken with such a  =

pattern.

The all.d, package.d, _.d, or, as I propose foldername.d (and submodul=

es  =

into foldername) do not break package accessibility modifier, which is=

=

better.

I don't really know what you mean, I didn't read that post.  Would you  =

mind posting a link or repeating that argument?

-Steve

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:59:31 -0400, Steven Schveighoffer  =

<schveiguy yahoo.com> wrote:

On Thu, 05 Apr 2012 10:53:34 -0400, deadalnix <deadalnix gmail.com>  =

wrote:

Le 05/04/2012 16:47, Steven Schveighoffer a =C3=A9crit :
Not really. Just move the files to another directory (i.e.
std/internal/algorithm) and publicly import them.

std/internal isn't good. If I just want to import sort, I would have =

to  =

do std.internal.algorithm.sort, which isn't good.

import std.algorithm : sort;

BTW, this doesn't work today, I'll file a bug.

Nevermind, it works the same as if sort is defined in a std.algorithm  =

module.  I'm kind of surprised the FQN doesn't work though...

-Steve

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:59:31 -0400, Steven Schveighoffer
<schveiguy yahoo.com> wrote:

On Thu, 05 Apr 2012 10:53:34 -0400, deadalnix <deadalnix gmail.com>
wrote:
Plus, the package accessibility modifier would be broken with such a
pattern.

The all.d, package.d, _.d, or, as I propose foldername.d (and
submodules into foldername) do not break package accessibility
modifier, which is better.

I don't really know what you mean, I didn't read that post.  Would you
mind posting a link or repeating that argument?

Also, nevermind :)  I realize what you are talking about now.

This is a good point.  Of course, there are other mechanisms to do this,
and DIP15 already proposes an improvement for this.

However, at least in the case of phobos, I don't think there's a lot of
instances of package accessibility modifier.

-Steve

Apr 05 2012
Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-04 17:53:24 +0000, "Steven Schveighoffer"
<schveiguy yahoo.com> said:

But yes, I think the issue really becomes, we need to  look at context
when deciding the semantic meaning of a symbol.  I don't  think this
violates the context-free grammar, because wouldn't this only  come
into play at the semantic level?  Not a compiler writer/hacker, so I
don't know.

You'd need a whole lot of context.

Let's say I write:

.a.b.c.d = .e.f.g.h;

Can you tell me which letters are the name of a package/module, and
which are the name of something else? It could be this:

module a.b;
struct c { static int d; }

module e.f.g;
int h;

or it could be this:

module a;
struct b { struct c { struct d { static void opAssign(int); } } }

module e;
struct f { enum g { h = 1 } }

or a multitude of other variants. If you can't know by looking at the
assignment above, how will the parser know?

Fully qualified names really need to refer to a single thing.

Say you wanted to create a package.d file directly for the whole
package  std, what should be done for those?

No, let's not do that.  Ever. :)

Phobos is just an example. People will want to use "package.d" files
for their own libraries too, and will complain if it doesn't work.

--
Michel Fortin
michel.fortin michelf.com
http://michelf.com/

Apr 04 2012
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/4/12 10:53 AM, Steven Schveighoffer wrote:
On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
<michel.fortin michelf.com> wrote:
Question 2: does std.algorithm.sort refer to the std.algorithm.sort
*module* or to std.algorithm.package.sort *function* publicly imported
from the std.algorithm.sort module?

The function. A symbol that is not specified as the module name in
import statement or in a module statement is always *not* a module. I
think our one saving grace here is that when you want to import a
specific symbol from a module, this is not the syntax:

import std.stdio.writefln;

So there is never any ambiguity as to whether you mean a module
identifier or other symbol.

Interesting. But isn't there an ambiguity when the symbol is not the
last one in the chain? Consider:

a.b.c.x

Could be static member d of class c in module a.b, or module member d in
module a.b.c.

Andrei

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:35:52 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

On 4/4/12 10:53 AM, Steven Schveighoffer wrote:
On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
<michel.fortin michelf.com> wrote:
Question 2: does std.algorithm.sort refer to the std.algorithm.sort
*module* or to std.algorithm.package.sort *function* publicly imported
from the std.algorithm.sort module?

The function. A symbol that is not specified as the module name in
import statement or in a module statement is always *not* a module. I
think our one saving grace here is that when you want to import a
specific symbol from a module, this is not the syntax:

import std.stdio.writefln;

So there is never any ambiguity as to whether you mean a module
identifier or other symbol.

Interesting. But isn't there an ambiguity when the symbol is not the
last one in the chain? Consider:

a.b.c.x

Could be static member d of class c in module a.b, or module member d in
module a.b.c.

Stop reading all my posts except for the ones I just posted.  I was
completely wrong in what I thought the compiler couldn't do, because the
spec is lacking.

I currently think DIP16 is invalid/worksforme (public imports allows
splitting a module into a package).  All that is left is how we could
specifically import a package with one import statement.

DIP15 covers that.  But I don't even think we need that, we can fix Phobos
without any compiler changes.

-Steve

Apr 05 2012
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/5/12 7:43 AM, Steven Schveighoffer wrote:
I currently think DIP16 is invalid/worksforme (public imports allows
splitting a module into a package). All that is left is how we could
specifically import a package with one import statement.

Not entirely (I was aware of the way public import works). An issue does
exist - there are "too many names", i.e. the alias pulled in the
importing module and also the name being imported. This makes for odd
synonyms such as std.algorithm_package.sort.sort being the same as
std.algorithm.sort. The other issue is that obviously algorithm_package
and algorithm must have distinct names, which makes the scheme a bit
awkward at least until we define a compelling convention. I guess we can
live with all that.

Andrei

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 11:23:12 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

On 4/5/12 7:43 AM, Steven Schveighoffer wrote:
I currently think DIP16 is invalid/worksforme (public imports allows
splitting a module into a package). All that is left is how we could
specifically import a package with one import statement.

Not entirely (I was aware of the way public import works). An issue does
exist - there are "too many names", i.e. the alias pulled in the
importing module and also the name being imported. This makes for odd
synonyms such as std.algorithm_package.sort.sort being the same as
std.algorithm.sort.

Right, but if one only ever imports std.algorithm, who cares what the
submodule FQNs are?

AIUI, DIP16 also doesn't really reduce the number of names either.

The other issue is that obviously algorithm_package and algorithm must
have distinct names, which makes the scheme a bit awkward at least until
we define a compelling convention. I guess we can live with all that.

Agreed.  I think Don mentioned std.math already does something, we may
want to look at that model.

A couple issues that still need consideration:

1. If std.algorithm the module becomes std.algorithm the package, what
happens with ddoc?  We probably *do* need a compiler solution to this.
2. deadalnix pointed out that if we come up with a scheme where the
package module and its submodules are in the same directory, the package
accessibility qualifier can be used (hey look, a use for the package
keyword!).

-Steve

Apr 05 2012
"Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:
A couple issues that still need consideration:

1. If std.algorithm the module becomes std.algorithm the package, what
happens with ddoc? We probably *do* need a compiler solution to this.

That's assuming that you insist on keeping all of the documentation in one
file. That arguably defeats the purpose of splitting up the modules. If there
isn't enough in the module to split the documentation, then why do you need to
split the module?

What _would_ be valuable and the package.d could provide is an overview of the
package. The ddoc comment for the package.d module can become the
documentation for the package as a whole.

2. deadalnix pointed out that if we come up with a scheme where the
package module and its submodules are in the same directory, the package
accessibility qualifier can be used (hey look, a use for the package
keyword!).

Yes. std.datetime will need that once it's split. Without that, much of it
can't be split and/or code would have to be needlessly duplicated.

- Jonathan M Davis

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis <jmdavisProg gmx.com>
wrote:

On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:
A couple issues that still need consideration:

1. If std.algorithm the module becomes std.algorithm the package, what
happens with ddoc? We probably *do* need a compiler solution to this.

That's assuming that you insist on keeping all of the documentation in
one
file. That arguably defeats the purpose of splitting up the modules. If
there
isn't enough in the module to split the documentation, then why do you
need to
split the module?

I thought the whole point was code maintenance?  Not documentation
splitting... I would have expected people to continue to treat
std.algorithm like it was one module, even though it imports several
sub-modules for its implementation.

What _would_ be valuable and the package.d could provide is an overview
of the
package. The ddoc comment for the package.d module can become the
documentation for the package as a whole.

2. deadalnix pointed out that if we come up with a scheme where the
package module and its submodules are in the same directory, the package
accessibility qualifier can be used (hey look, a use for the package
keyword!).

Yes. std.datetime will need that once it's split. Without that, much of
it
can't be split and/or code would have to be needlessly duplicated.

Hm.. I just thought of something, as long as the main "package" module
imports everything from the same directory, and doesn't define anything,
this isn't an issue.

-Steve

Apr 05 2012
"Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 15:26:14 Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis <jmdavisProg gmx.com>

wrote:
On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:
A couple issues that still need consideration:

1. If std.algorithm the module becomes std.algorithm the package, what
happens with ddoc? We probably *do* need a compiler solution to this.

That's assuming that you insist on keeping all of the documentation in
one
file. That arguably defeats the purpose of splitting up the modules. If
there
isn't enough in the module to split the documentation, then why do you
need to
split the module?

I thought the whole point was code maintenance? Not documentation
splitting... I would have expected people to continue to treat
std.algorithm like it was one module, even though it imports several
sub-modules for its implementation.

If the module isn't large enough to be split for documentation, I find it hard
to believe that it needs to be split for maintenance. And if all you care
about is sub-modules for implementation and want all of the functions in the
same module still, then this DIP is pointless. All you have to do is declare
undocumented sub-modules which hold the various implementations and have the
actual module call them. We already do this sort of thing in Phobos to get
around static destructors screaming about circular dependencies.

I only see the need to split of a module as the DIP suggests when it's too
big, and if the documentation isn't large enough to be an issue, then I don't
see how the module itself is going to be too large unless it has a ton of
helper stuff, and that stuff can be seemlessy put in another, undocumented
module without needing to create a package.

- Jonathan M Davis

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 17:02:13 -0400, Jonathan M Davis <jmdavisProg gmx.com>
wrote:

On Thursday, April 05, 2012 15:26:14 Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis
<jmdavisProg gmx.com>

wrote:
On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:
A couple issues that still need consideration:

1. If std.algorithm the module becomes std.algorithm the package,

what
happens with ddoc? We probably *do* need a compiler solution to this.

That's assuming that you insist on keeping all of the documentation in
one
file. That arguably defeats the purpose of splitting up the modules.

If
there
isn't enough in the module to split the documentation, then why do you
need to
split the module?

I thought the whole point was code maintenance? Not documentation
splitting... I would have expected people to continue to treat
std.algorithm like it was one module, even though it imports several
sub-modules for its implementation.

If the module isn't large enough to be split for documentation, I find
it hard
to believe that it needs to be split for maintenance.

Why do we ever need to split modules for documentation?  Just fix the doc
generator so it's not as monolithic.  For instance, have one page per
class or struct.

And if all you care
about is sub-modules for implementation and want all of the functions in
the
same module still, then this DIP is pointless. All you have to do is
declare
undocumented sub-modules which hold the various implementations and have
the
actual module call them. We already do this sort of thing in Phobos to
get
around static destructors screaming about circular dependencies.

You are starting to see my point :)  But I think the issue is not so much
that you are splitting the implementation, but splitting up the API into
related modules.

For example, std.container.  Imagine we have a robust set of 15
containers.  Why should those all be in one file?  They have nothing to do
with eachother except they are in the same namespace.  Why does
RedBlackTree need to be able to access the internals of Array?

As the writer of RedBlackTree, I want to be able to test and develop my
container without having to worry about the rest of std.container.  But I
also would like to have the FQN of it to be std.container.RedBlackTree.
Public imports allow this *today* without any changes.

Note that size isn't so much an issue for me as being able to
compartmentalize and develop individual pieces of a module.

-Steve

Apr 05 2012
"Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 17:33:50 Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 17:02:13 -0400, Jonathan M Davis <jmdavisProg gmx.com>
If the module isn't large enough to be split for documentation, I find
it hard
to believe that it needs to be split for maintenance.

Why do we ever need to split modules for documentation? Just fix the doc
generator so it's not as monolithic. For instance, have one page per
class or struct.

That may or may not be desirable (certainly in the case of smaller types, I'd
argue that it isn't). By doing it on a module basis, you have far more
control. But regardless, that would be a major change to ddoc.

In either case, the size of the documentation page for a module is currently
closely tied to the number of public symbols in the module, so if you have a
large API, it can become desirable to split it up simply because the
documentation page for it is too large. And by splitting up the API, you fix
that problem. Not to mention, if the module is mostly free functions, putting
the documentation for each class or struct on its own page doesn't help
anyway. So, while that may be a good change to ddoc in at least some cases, it
doesn't solve the problem in general. For instance, it would help
std.datetime, but it wouldn't help std.algorithm at all.

And if all you care
about is sub-modules for implementation and want all of the functions in
the
same module still, then this DIP is pointless. All you have to do is
declare
undocumented sub-modules which hold the various implementations and have
the
actual module call them. We already do this sort of thing in Phobos to
get
around static destructors screaming about circular dependencies.

You are starting to see my point :) But I think the issue is not so much
that you are splitting the implementation, but splitting up the API into
related modules.

As I understand it, the entire point of this DIP is to enable splitting up the
API cleanly without breaking code. The implementation can already be split up
seemlessly.

- Jonathan M Davis

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 19:14:42 -0400, Jonathan M Davis <jmdavisProg gmx.com>
wrote:

On Thursday, April 05, 2012 17:33:50 Steven Schveighoffer wrote:
Why do we ever need to split modules for documentation? Just fix the doc
generator so it's not as monolithic. For instance, have one page per
class or struct.

That may or may not be desirable (certainly in the case of smaller
types, I'd
argue that it isn't). By doing it on a module basis, you have far more
control. But regardless, that would be a major change to ddoc.

ddoc's output leaves a lot to be desired.  The unorganized links at the
top suck.  Using module order to show symbols instead of category of
symbols.

How is a doc page ever "too big"?  Even std.datetime loads in a second.
It's more that it's "too disorganized".

And if all you care
about is sub-modules for implementation and want all of the functions

in
the
same module still, then this DIP is pointless. All you have to do is
declare
undocumented sub-modules which hold the various implementations and

have
the
actual module call them. We already do this sort of thing in Phobos to
get
around static destructors screaming about circular dependencies.

You are starting to see my point :) But I think the issue is not so much
that you are splitting the implementation, but splitting up the API into
related modules.

As I understand it, the entire point of this DIP is to enable splitting
up the
API cleanly without breaking code. The implementation can already be
split up
seemlessly.

As can the API via public imports.

-Steve

Apr 06 2012
Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, April 06, 2012 07:56:41 Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 19:14:42 -0400, Jonathan M Davis <jmdavisProg gmx.com>

wrote:
On Thursday, April 05, 2012 17:33:50 Steven Schveighoffer wrote:
Why do we ever need to split modules for documentation? Just fix the doc
generator so it's not as monolithic. For instance, have one page per
class or struct.

That may or may not be desirable (certainly in the case of smaller
types, I'd
argue that it isn't). By doing it on a module basis, you have far more
control. But regardless, that would be a major change to ddoc.

ddoc's output leaves a lot to be desired.  The unorganized links at the
top suck.  Using module order to show symbols instead of category of
symbols.

How is a doc page ever "too big"?  Even std.datetime loads in a second.
It's more that it's "too disorganized".

There's no question that the links at the top suck, and std.datetime is the
poster child for why they suck. The page is too large for how its organized,
and it _does_ take significantly longer to load than other Phobos pages, even
if it's still usable. There's no question that better links would help
significantly though.

Regardless, there's enough content there that it would arguably be better
served if it were multiple pages.

And if all you care
about is sub-modules for implementation and want all of the functions

in

the
same module still, then this DIP is pointless. All you have to do is
declare
undocumented sub-modules which hold the various implementations and

have

the
actual module call them. We already do this sort of thing in Phobos to
get
around static destructors screaming about circular dependencies.

You are starting to see my point :) But I think the issue is not so much
that you are splitting the implementation, but splitting up the API into
related modules.

As I understand it, the entire point of this DIP is to enable splitting
up the
API cleanly without breaking code. The implementation can already be
split up
seemlessly.

As can the API via public imports.

Not and keep stuff in the same place in the hierarchy. You have to create a
completely separate package. The point of the DIP was to make it so that you
could do it _in place_ by turning a module like std/algorithm.d into the
package std/algorithm/ with modules inside it.

Right now, I can go and split up std.datetime into a new package (e.g.
std.dtime) and then publicly import it all from std.datetime, but then the new
package is completely separate from the module which imports everything. It
also makes it look like std.datetime should go away in favor of just keeping
std.dtime (the same with std.algorithm if it gets split), which may or may not
actually be desirable. Organizationally, it would be much nicer to be able to
turn std/datetime.d into std/datetime/, and _that_ is what the DIP is trying
to enable. The rest can be done now but not that. You can't split up the API
in place.

- Jonathan M Davis

Apr 06 2012
Le 05/04/2012 17:23, Andrei Alexandrescu a Ã©crit :
On 4/5/12 7:43 AM, Steven Schveighoffer wrote:
I currently think DIP16 is invalid/worksforme (public imports allows
splitting a module into a package). All that is left is how we could
specifically import a package with one import statement.

Not entirely (I was aware of the way public import works). An issue does
exist - there are "too many names", i.e. the alias pulled in the
importing module and also the name being imported. This makes for odd
synonyms such as std.algorithm_package.sort.sort being the same as
std.algorithm.sort. The other issue is that obviously algorithm_package
and algorithm must have distinct names, which makes the scheme a bit
awkward at least until we define a compelling convention. I guess we can
live with all that.

Andrei

The first one isn't a problem. It isn't too many names, it is 2 names,
and it happen when explicitly told to do so.

The second is.

Apr 05 2012
Le 05/04/2012 16:35, Andrei Alexandrescu a Ã©crit :
On 4/4/12 10:53 AM, Steven Schveighoffer wrote:
On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
<michel.fortin michelf.com> wrote:
Question 2: does std.algorithm.sort refer to the std.algorithm.sort
*module* or to std.algorithm.package.sort *function* publicly imported
from the std.algorithm.sort module?

The function. A symbol that is not specified as the module name in
import statement or in a module statement is always *not* a module. I
think our one saving grace here is that when you want to import a
specific symbol from a module, this is not the syntax:

import std.stdio.writefln;

So there is never any ambiguity as to whether you mean a module
identifier or other symbol.

Interesting. But isn't there an ambiguity when the symbol is not the
last one in the chain? Consider:

a.b.c.x

Could be static member d of class c in module a.b, or module member d in
module a.b.c.

Andrei

The whole point of this thread is to import symbols from submodules into
a module. Obviously, this tends to create collisions. This one is hard
to solve and should probably be an error.

However, such a collision will also appear with DIP16, and DIP16 cause
also other type of collisions. So that one is superior - even if not perfect

Apr 05 2012
Jacob Carlborg <doob me.com> writes:
On 2012-04-04 18:33, Michel Fortin wrote:

You might think I'm trying to split hair in four to find flaws, that no
one is going to do things that dumb, but I unfortunately think the
problematic pattern is already quite common. How many times have we seen
modules containing a class, variable, or function having the same name
as the module's name? What should happen when you publicly import those
modules in the package.d file?

I do that all the time in my libraries.

--
/Jacob Carlborg

Apr 04 2012
Ary Manzana <ary esperanto.org.ar> writes:
On 3/30/12 10:46 PM, Andrei Alexandrescu wrote:
Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means
that he would approve a pull request implementing DIP16 (subject to
regular correctness checks).

Great. Large modules are my main complaint about D. :-)

If I correctly understand the second part (because I couldn't understand
the text in the proposal until I read some comments here), then it makes
sense. Is it like this?

sort(...) -> search sort in every module out there
std.sort(...) -> search sort in every module that's in the std package

If both std.algorithm.sort and std.path.sort exist, or something like
that, then you would anyway get a clash so you'd have to fully qualify it.

But if std.algorithm.sort and foo.bar.sort and you'd import both:

import std.algorithm.package;
import foo.bar.package;

and you'd wanted to use both, then it could be convenient:

std.sort(...)
foo.sort(...)

Though I wonder if this indeed happens a lot. That's why I would wait
until there's a real need for it. The main complaint people have is not
having a way to import all files in a directory, which is the first
point, but I never heard a complaint about the second point.

Also, I think it would make sense to change the first part to this:

* If the compiler sees a request for importing "foo.bar" and "foo/bar"
is a directory, then automatically look for the file
"foo/bar/package.d". *If it doesn't exist, automatically expand the
import to import all files under that directory.* If both "foo/bar.d"
and "foo/bar/" exists, compilation halts with an error.

That way you have convenience and safety. Most of the time people just
put in package.d a list of all the files in that directory. Maybe
sometimes (not sure) people restrict that list to some modules. And in
those cases you can just restrict the list in package.d

Please, it's the year 2012. Compilers need to be smarter. Save people
some typing time. You save them typing all the imports. But then you
make them typing them in that pacakge.d file. Hmm...

Mar 30 2012
"Chris NS" <ibisbasenji gmail.com> writes:
I'm pretty impressed with the idea, and look forward to its
implementation, but I do have one question.  How does this affect
(if at all) the implicit "friend" relationship of declarations?
Since package "foo.bar" is treated as a single module, do the
classes in "foo/bar/alpha.d" and "foo/bar/beta.d" have access to
each other's private members?

I'm not sure whether I favor them losing or keeping their
"friend" status.

----------
Chris NS

Mar 30 2012
"Nick Sabalausky" <a a.a> writes:
"Chris NS" <ibisbasenji gmail.com> wrote in message
news:ugopmohijjcnnrchuwbe forum.dlang.org...
I'm pretty impressed with the idea, and look forward to its
implementation, but I do have one question.  How does this affect (if at
all) the implicit "friend" relationship of declarations?  Since package
"foo.bar" is treated as a single module, do the classes in
"foo/bar/alpha.d" and "foo/bar/beta.d" have access to each other's private
members?

I'm not sure whether I favor them losing or keeping their "friend" status.

There's always the "package" access specifier.

Mar 30 2012
"Chris NS" <ibisbasenji gmail.com> writes:
On Saturday, 31 March 2012 at 06:39:07 UTC, Nick Sabalausky wrote:
"Chris NS" <ibisbasenji gmail.com> wrote in message
news:ugopmohijjcnnrchuwbe forum.dlang.org...
I'm pretty impressed with the idea, and look forward to its
implementation, but I do have one question.  How does this
affect (if at all) the implicit "friend" relationship of
declarations?  Since package "foo.bar" is treated as a single
module, do the classes in "foo/bar/alpha.d" and
"foo/bar/beta.d" have access to each other's private members?

I'm not sure whether I favor them losing or keeping their
"friend" status.

There's always the "package" access specifier.

True, though it bears revisiting in its own right.  I've never
been completely satisfied with the horizontal visibility, and
would have preferred 'package' be defined as "visible within
current package, *and subpackages* of the current."  But now I'm
getting a bit off-topic.

Mar 30 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 31 Mar 2012 00:23:32 -0400, Chris NS <ibisbasenji gmail.com> wrote:

I'm pretty impressed with the idea, and look forward to its
implementation, but I do have one question.  How does this affect (if at
all) the implicit "friend" relationship of declarations?  Since package
"foo.bar" is treated as a single module, do the classes in
"foo/bar/alpha.d" and "foo/bar/beta.d" have access to each other's
private members?

I'm not sure whether I favor them losing or keeping their "friend"
status.

They would lose their friend status.  Classes that you need to be
"friends" would have to go into the same submodule, which makes perfect
sense to me.

-Steve

Apr 02 2012
Derek <ddparnell bigpond.com> writes:
On Sat, 31 Mar 2012 01:46:19 +1100, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

We solved this issue in the Euphoria Programming language by introducing
the concept of 'public include'.

The 'include' directive is similar to DPL's 'import' directive. Normally
when a file (module) is included, its public identifiers are *only*
visible to the including module and not further module up the include
tree. However, if a module is 'public include'd then its public
identifiers are set up *as if* they were actually declared in the
including module rather than the included module.

Thus if Euphoria's module called datetime.e gets too large, we split it
into, say two or three new modules and replace the code in datetime.e with
corresponding 'public include' statements, thus making it a type of
package definition. This means that existing application code does not
have to be modified and future code can choose to include either the
entire package called datetime.e or individual modules that go into making
that package.

From the POV of a developer, creating a package out of what was a module
is transparent. They do not even know or have to care in order to use it.
There are no complicated access rules that can one day trip people up.

We made one tweak to the public identifier declaration though to cater for
situations in which an identifier which was currently local to a module,
but when moved to a new module when the original was transformed to a
package, still had to be visible to the code in the package file but
invisible to code including the package. We created the 'export' qualifier
on an identifier declaration to signal such an item.

Original module code ...

function to_seconds(datetime x)  /* A local function to the module */

public function local_to_gmt(datetime x) /* A publicly exposed function
*/

New module code ...

export function to_seconds(datetime x)  /* A function visible to the
package file only */

public function local_to_gmt(datetime x) /* A publicly exposed function
*/

This means that application that include the package datetime.e will not
see to_seconds() but code in the datetime.e will see it, and
local_to_gmt() will be seen by the application code and the package code.

--
Derek Parnell
Melbourne, Australia

Mar 31 2012
"foobar" <foo bar.com> writes:
On Friday, 30 March 2012 at 14:46:16 UTC, Andrei Alexandrescu
wrote:
Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's
approval means that he would approve a pull request
implementing DIP16 (subject to regular correctness checks).

Destroy!

Andrei

Hooray! I was loudly complaining about this issue for years. I'm
glad this is finally going to be taken care of.

1. Regarding naming - as others mentioned it's inconvenient for
sorting. How about changing the file extension instead of the
name? e.g have a "algorithm.package" instead of "algorithm.d"?
The compiler could even ignore the name of the file as long as it
has the proper extension - check for existence of "*.package".
The compiler should enforce that there's at most one such package
file to prevent ambiguities - regardless of the chosen naming
scheme.

2. Regarding documentation - I'd encourage (via a D style
guideline) putting the overview of the package in this file. A
tool such as Ruby's codnar would nicely compliment the reference
generated by DDoc. see https://www.ruby-toolbox.com/gems/codnar
for details.

3. std.algorithm should be deprecated and eventually removed. All
code *is* essentially algorithms and as such importing this
module makes as much sense as doing:
import code; // generic much?

4. regarding the look-up rules - I'm unsure about this part of
the proposal. Others already voiced concerns about issues this
might bring.

Mar 31 2012
Le 30/03/2012 16:46, Andrei Alexandrescu a écrit :
Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means
that he would approve a pull request implementing DIP16 (subject to
regular correctness checks).

Destroy!

Andrei

OK, here is an alternative proposal, based on what has been said here,
and in other thread. I think it is a better alternative because it is

1/ you can defined both a.d (d source code) and a (folder) in the same
folder.
2/ import a; will always look for a.d . a folder. a existing isn't an error.
3/ import a.b will look for b.d in a folder. If a.d exists, it isn't an
error.
4/ The package a include what in a.d and everything defined in d files
in the folder a and its subfolders.
5/ What is publicly imported get automatically aliased in the importing
module. See example below if it is not clear :

a.d :

public import a.b;

b.d :

void foo() {}

automatic aliasing in a.d would look like :

public import a.b;

// Automaticaly added aliases :
alias b.foo foo;

And usage from third party code :

import a;

void main() {
foo();       // OK
a.foo();     // OK
a.b.foo();   // OK
}

And with static import :

static import a;

void main() {
foo();       // Error
a.foo();     // OK
a.b.foo();   // OK
}

Mar 31 2012
"Martin Nowak" <dawg dawgfoto.de> writes:
On Fri, 30 Mar 2012 16:46:19 +0200, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means
that he would approve a pull request implementing DIP16 (subject to
regular correctness checks).

Destroy!

Andrei

What about supporting package initalization?
I basically proposed that if a submodule of a package
was imported, a static import of the package is implicitly
http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

Apr 03 2012
Le 03/04/2012 19:44, Martin Nowak a écrit :
On Fri, 30 Mar 2012 16:46:19 +0200, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval
means that he would approve a pull request implementing DIP16 (subject
to regular correctness checks).

Destroy!

Andrei

What about supporting package initalization?
I basically proposed that if a submodule of a package
was imported, a static import of the package is implicitly
http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

That is pretty much what was proposed by people in this thread.

Basically, it boils down to adding automatically aliases with public
imports.

I would prefer use pkg.d file instead of pkg/_.d (I though of it a lot
recently, and this is what make more sense, even if not my initial
proposal).

With pkg/package.d or pkg/_.d you can ends up with unnecessary
complexity in choosing the file wich is imported, and create error
cases. For instance :

What happen if both pkg.d and pkg/_.d exists ? If it is not in the same
path (think -I compiler option). In one case, this is an issue, in the
other this isn't.

This file convention solution is superior to the _.d one or package.d
one. DIP15 is superior to D16 IMO.

Apr 04 2012
"Martin Nowak" <dawg dawgfoto.de> writes:
 What happen if both pkg.d and pkg/_.d exists ? If it is not in the same
path (think -I compiler option). In one case, this is an issue, in the
other this isn't.

pkg.d would always be a module, hence result in a module/package conflict.
We'd need to directly search for a subdirectory to decide whether it's a
package.

Maybe someone with experience of big Python projects has some valuable
insights, but I think one point of adding the file inside a folder is to
prevent from unintended pickup of folders.

Apr 04 2012
Le 05/04/2012 02:55, Martin Nowak a Ã©crit :
What happen if both pkg.d and pkg/_.d exists ? If it is not in the
same path (think -I compiler option). In one case, this is an issue,
in the other this isn't.

pkg.d would always be a module, hence result in a module/package conflict.
We'd need to directly search for a subdirectory to decide whether it's a
package.

This is true, but why would it be a conflict ?

Package already is a tree structure, so it shouldn't cause much trouble.

Maybe someone with experience of big Python projects has some valuable
insights, but I think one point of adding the file inside a folder is to
prevent from unintended pickup of folders.

Using eponymous trick, you'd always pick up a file, never a folder.
Hence, lookup rules get easier. Plus it is D-ish.

I have no doubt that this approach is working in python, and could work
in D. This is exactly why this was my first proposal. But this doesn't
means that we can't do better. I have tested quite a lot of way to
achieve that, in several languages, and this is usually a messy topic,
with no single solution.

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means
that he would approve a pull request implementing DIP16 (subject to
regular correctness checks).

Destroy!

BTW, this case makes the part of DIP16 which wants to shortcut fully
qualified names invalid, or at least costly (I posted this code in another
part of the thread, but I thought I'd bring it up higher).

The following is valid code today:

a/b.d:

module a.b;

void foo() {}
struct b
{
static void foo() {}
}

main.d:
import a.b;

void main()
{
a.b.foo();
}

If DIP16 were to be implemented, this becomes ambiguous.  Is a.b.foo() the
module function foo() from a.b, or is it a shortcut for a.b.b.foo()?

The main issue is, because you can shortcut the FQN, and a chain of
identifiers can have repeated identifiers in them, ambiguity is possible.

Michel Fortin pointed out at least three cases of this in phobos (although
I can't see how they could be ambiguous, he has a good point).

-Steve

Apr 05 2012
Ary Manzana <ary esperanto.org.ar> writes:
On 4/5/12 10:55 PM, Steven Schveighoffer wrote:
On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval
means that he would approve a pull request implementing DIP16 (subject
to regular correctness checks).

Destroy!

BTW, this case makes the part of DIP16 which wants to shortcut fully
qualified names invalid, or at least costly (I posted this code in
another part of the thread, but I thought I'd bring it up higher).

The following is valid code today:

a/b.d:

module a.b;

void foo() {}
struct b
{
static void foo() {}
}

main.d:
import a.b;

void main()
{
a.b.foo();
}

If DIP16 were to be implemented, this becomes ambiguous. Is a.b.foo()
the module function foo() from a.b, or is it a shortcut for a.b.b.foo()?

It's a shortcut to the module function foo().

If I replace main.d with:

void main() {
foo();
}

Is foo() the module function foo() from a.b, or is it a shortcut for
a.b.b.foo()? Here you have no doubts. What your mind does it: find a top
level symbol "foo" in all imported modules. If you find more than one,
error.

You must apply the same logic for "a.b.foo()". First you search "foo" in
the "a.b" symbol. Here you find it: it's the top level function "foo" in
"a.b". Then you stop searching.

However, if you can't find it in the module "a.b", you search a top
level symbol "foo" in all modules that are in package "a.b". That's it.
You don't search "foo" in every possible nesting: just module nesting.

The main issue is, because you can shortcut the FQN, and a chain of
identifiers can have repeated identifiers in them, ambiguity is possible.

As I said before, it's not a shortcut of the FQN: it's just a shortcut
for the module name.

Apr 05 2012
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 23:18:07 -0400, Ary Manzana <ary esperanto.org.ar>
wrote:

On 4/5/12 10:55 PM, Steven Schveighoffer wrote:
On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval
means that he would approve a pull request implementing DIP16 (subject
to regular correctness checks).

Destroy!

BTW, this case makes the part of DIP16 which wants to shortcut fully
qualified names invalid, or at least costly (I posted this code in
another part of the thread, but I thought I'd bring it up higher).

The following is valid code today:

a/b.d:

module a.b;

void foo() {}
struct b
{
static void foo() {}
}

main.d:
import a.b;

void main()
{
a.b.foo();
}

If DIP16 were to be implemented, this becomes ambiguous. Is a.b.foo()
the module function foo() from a.b, or is it a shortcut for a.b.b.foo()?

It's a shortcut to the module function foo().

If I replace main.d with:

void main() {
foo();
}

Is foo() the module function foo() from a.b, or is it a shortcut for
a.b.b.foo()? Here you have no doubts. What your mind does it: find a top
level symbol "foo" in all imported modules. If you find more than one,
error.

That's slightly different, because you must *always* qualify struct b's
foo with a preceeding b.

You must apply the same logic for "a.b.foo()". First you search "foo" in
the "a.b" symbol. Here you find it: it's the top level function "foo" in
"a.b". Then you stop searching.

However, if you can't find it in the module "a.b", you search a top
level symbol "foo" in all modules that are in package "a.b". That's it.
You don't search "foo" in every possible nesting: just module nesting.

What if a.b is a struct, and it's the only possible match?  We don't
search for it?  At some point the struct b has to come into play.  Or are
you saying we cannot shortcut struct FQN's?

I suppose if we prefer to match modules before types, then name lookup for
fully qualified names only becomes ambiguous with packages allowed to have
their own modules, so it shouldn't affect existing code.

The main issue is, because you can shortcut the FQN, and a chain of
identifiers can have repeated identifiers in them, ambiguity is
possible.

As I said before, it's not a shortcut of the FQN: it's just a shortcut
for the module name.

My example *was* a shortcut for the module name.  I did not imply that you
could shortcut the other parts of the FQN.

What about hijacking though?  For example:

module a.b

struct c
{
static void foo() {}
}

people now use a.c.foo() to avoid having to type the whole thing

But along comes someone who creates:

module a.c;
void foo() {}

Now, doesn't this usurp a.c.foo() without warning?

-Steve

Apr 06 2012